Import PDF File Scripting - AI Importing: Difference between revisions
Created page with "Import PDF File Scripting - AI Importing" |
No edit summary |
||
| Line 1: | Line 1: | ||
Import PDF File Scripting | {{ScriptingNav}} | ||
== Overview == | |||
The Invoice PDF Import System provides automated processing of supplier invoices using AI-powered document analysis. The system extracts invoice data from PDF files and creates AP (Accounts Payable) invoices in the accounting system with minimal manual intervention. | |||
== Key Features == | |||
* '''Multi-file processing''': Process multiple PDF invoices in a single operation | |||
* '''AI-powered extraction''': Uses vision AI models to read and interpret invoice documents | |||
* '''Invoice type detection''': Automatically distinguishes between Service and Parts invoices | |||
* '''Data validation''': Performs mathematical verification of totals and line items | |||
* '''Automated invoice creation''': Generates complete AP invoices with line items | |||
* '''Batch processing''': Optional batch creation for grouping related invoices | |||
== Invoice Types == | |||
=== Service Invoices === | |||
Service invoices contain work performed on specific vehicles or equipment. Key characteristics: | |||
* Include vehicle/equipment identifiers (Serial Number, Make, Model, Meter Reading) | |||
* Contain detailed service description text | |||
* Service description may span multiple pages | |||
* Serial Number is used as the Department identifier | |||
=== Parts Invoices === | |||
Parts invoices list individual components or products purchased. Key characteristics: | |||
* Focus on part numbers and quantities | |||
* No vehicle-specific information | |||
* Line items include Part Number + Description format | |||
== Data Extraction == | |||
The system extracts the following information from invoice PDFs: | |||
=== Header Information === | |||
* Invoice Type (Service/Parts) | |||
* Invoice Number | |||
* Invoice Date | |||
* Order Number (Sales Order for Parts, Purchase Order for Service) | |||
* Serial Number (Service invoices only) | |||
* Customer Reference Number | |||
* Service Description (Service invoices only) | |||
=== Line Items === | |||
For each line on the invoice: | |||
* Description (Part Number + Description for parts) | |||
* Quantity | |||
* Unit Price | |||
* Extended Total (line total) | |||
=== Financial Totals === | |||
* Net Subtotal (excluding tax) | |||
* Total GST/Tax | |||
* Total Amount (including tax) | |||
== AI Model Configuration == | |||
The system supports multiple AI vision providers: | |||
* '''Anthropic''' (default): Claude vision models | |||
* '''OpenAI''': GPT vision models | |||
Configuration parameters: | |||
* <code>AIModel</code>: Model identifier string | |||
* <code>AIModelId</code>: Specific model version | |||
* <code>AIVisionEnabled</code>: Enable/disable AI processing | |||
* <code>AIApiKey</code>: Encrypted API key for authentication | |||
== Data Validation == | |||
The extraction process includes built-in validation: | |||
=== OCR Accuracy === | |||
* Disambiguates common character confusions (Z/2, O/0, I/1/l, S/5, G/6) | |||
* Reads all pages of multi-page invoices | |||
=== Mathematical Verification === | |||
* Verifies sum of line items matches subtotal (within 0.02 tolerance) | |||
* Confirms GST calculation (≈ 15% of subtotal) | |||
* Validates total = subtotal + GST | |||
== Invoice Creation Process == | |||
=== Line Item Processing === | |||
'''For Service Invoices:''' | |||
* Creates a nominal line item with service description | |||
* Adds offsetting line item for calculation purposes | |||
* Uses extracted line items for parts/materials used | |||
'''For Parts Invoices:''' | |||
* Converts each extracted line item to invoice row | |||
* Calculates GST inclusive totals | |||
* Computes unit prices from extended totals | |||
=== Department Assignment === | |||
* Service invoices: Uses Serial Number as Department | |||
* Parts invoices: Department left blank or as specified | |||
=== Financial Calculations === | |||
All amounts are stored GST exclusive with separate tax tracking: | |||
<pre> | |||
GST Exc Total = Line Extended Total | |||
Total With GST = GST Exc Total × 1.15 | |||
GST Amount = Total With GST - GST Exc Total | |||
Unit Price = GST Exc Total ÷ Quantity | |||
</pre> | |||
=== Invoice Record Creation === | |||
Creates AP invoice with: | |||
* Transaction Type: APINV | |||
* Vendor/Supplier: OtherPartyId (default: "5521") | |||
* Location: Configurable (default: "Misc") | |||
* Payment Date: 20th of month following invoice date | |||
* Reference: Invoice Number from PDF | |||
* Line items with all pricing and tax details | |||
=== Batch Processing === | |||
Optional batch creation groups related invoices: | |||
* Batch ID defaults to supplier/vendor ID | |||
* Includes batch comments for tracking | |||
* Supports recurring batch patterns | |||
== Error Handling == | |||
The system includes comprehensive error handling: | |||
=== PDF Processing Errors === | |||
* Invalid or corrupted PDF files | |||
* Missing or unreadable pages | |||
* Unsupported PDF formats | |||
=== Data Extraction Errors === | |||
* No invoice detected in PDF | |||
* Invalid JSON response from AI model | |||
* Missing required fields | |||
* Multiple invoices in single PDF (selects largest by subtotal) | |||
=== Invoice Creation Errors === | |||
* Invalid supplier/vendor ID | |||
* Missing required master data (items, departments) | |||
* Transaction validation failures | |||
* Batch creation conflicts | |||
== Usage Guidelines == | |||
=== File Selection === | |||
# Use the file picker dialog to select one or more PDF files | |||
# Only PDF files are supported | |||
# Multiple files can be processed in sequence | |||
=== Best Practices === | |||
* Ensure invoice PDFs are clear and readable | |||
* Verify supplier master data exists before import | |||
* Check that department codes (serial numbers) are valid | |||
* Review created invoices for accuracy | |||
* Use batch processing for related invoices from same supplier | |||
=== Post-Import Actions === | |||
* Review generated invoices in the system | |||
* Verify line item details and totals | |||
* Confirm department assignments | |||
* Check payment dates | |||
* Approve batches when ready for posting | |||
== Technical Architecture == | |||
=== Components === | |||
* '''Import Scripter''': Main orchestration class | |||
* '''AI Vision Service''': PDF analysis and data extraction | |||
* '''Invoice Functions''': Business logic for invoice creation | |||
* '''Transaction Functions''': Database persistence layer | |||
=== Data Flow === | |||
<pre> | |||
PDF File → AI Vision Analysis → JSON Extraction → | |||
Data Validation → Invoice Row Creation → | |||
AP Invoice Generation → Database Persistence | |||
</pre> | |||
=== Performance Considerations === | |||
* Batch loading of items and departments | |||
* Dictionary-based lookups for master data | |||
* Minimal database queries during processing | |||
* Asynchronous AI processing support | |||
== Configuration Requirements == | |||
=== System Parameters === | |||
* AI API credentials (encrypted) | |||
* Default supplier/vendor ID | |||
* Default location ID | |||
* Invoice numbering scheme | |||
* Tax calculation rules (GST rate) | |||
* Payment terms (default 20 days after month-end) | |||
=== Master Data Prerequisites === | |||
* Supplier/vendor records | |||
* Item master data (for parts) | |||
* Department codes (for service invoices) | |||
* Location codes | |||
* Transaction type definitions | |||
== Integration Points == | |||
The Invoice PDF Import integrates with: | |||
* '''Item Management''': Item lookup and validation | |||
* '''Department Management''': Department code resolution | |||
* '''Transaction Processing''': Invoice creation and posting | |||
* '''Batch Management''': Batch header creation and tracking | |||
* '''AI Vision Services''': External API for document analysis | |||
== See Also == | |||
* [[Import Scripting]] | |||
* [[Transaction Processing]] | |||
* [[AI Vision Services]] | |||
[[Category:Scripting]] | |||
[[Category:Import Functions]] | |||
[[Category:AI Features]] | |||
Revision as of 03:23, 10 November 2025
Overview
The Invoice PDF Import System provides automated processing of supplier invoices using AI-powered document analysis. The system extracts invoice data from PDF files and creates AP (Accounts Payable) invoices in the accounting system with minimal manual intervention.
Key Features
- Multi-file processing: Process multiple PDF invoices in a single operation
- AI-powered extraction: Uses vision AI models to read and interpret invoice documents
- Invoice type detection: Automatically distinguishes between Service and Parts invoices
- Data validation: Performs mathematical verification of totals and line items
- Automated invoice creation: Generates complete AP invoices with line items
- Batch processing: Optional batch creation for grouping related invoices
Invoice Types
Service Invoices
Service invoices contain work performed on specific vehicles or equipment. Key characteristics:
- Include vehicle/equipment identifiers (Serial Number, Make, Model, Meter Reading)
- Contain detailed service description text
- Service description may span multiple pages
- Serial Number is used as the Department identifier
Parts Invoices
Parts invoices list individual components or products purchased. Key characteristics:
- Focus on part numbers and quantities
- No vehicle-specific information
- Line items include Part Number + Description format
Data Extraction
The system extracts the following information from invoice PDFs:
Header Information
- Invoice Type (Service/Parts)
- Invoice Number
- Invoice Date
- Order Number (Sales Order for Parts, Purchase Order for Service)
- Serial Number (Service invoices only)
- Customer Reference Number
- Service Description (Service invoices only)
Line Items
For each line on the invoice:
- Description (Part Number + Description for parts)
- Quantity
- Unit Price
- Extended Total (line total)
Financial Totals
- Net Subtotal (excluding tax)
- Total GST/Tax
- Total Amount (including tax)
AI Model Configuration
The system supports multiple AI vision providers:
- Anthropic (default): Claude vision models
- OpenAI: GPT vision models
Configuration parameters:
AIModel: Model identifier stringAIModelId: Specific model versionAIVisionEnabled: Enable/disable AI processingAIApiKey: Encrypted API key for authentication
Data Validation
The extraction process includes built-in validation:
OCR Accuracy
- Disambiguates common character confusions (Z/2, O/0, I/1/l, S/5, G/6)
- Reads all pages of multi-page invoices
Mathematical Verification
- Verifies sum of line items matches subtotal (within 0.02 tolerance)
- Confirms GST calculation (≈ 15% of subtotal)
- Validates total = subtotal + GST
Invoice Creation Process
Line Item Processing
For Service Invoices:
- Creates a nominal line item with service description
- Adds offsetting line item for calculation purposes
- Uses extracted line items for parts/materials used
For Parts Invoices:
- Converts each extracted line item to invoice row
- Calculates GST inclusive totals
- Computes unit prices from extended totals
Department Assignment
- Service invoices: Uses Serial Number as Department
- Parts invoices: Department left blank or as specified
Financial Calculations
All amounts are stored GST exclusive with separate tax tracking:
GST Exc Total = Line Extended Total Total With GST = GST Exc Total × 1.15 GST Amount = Total With GST - GST Exc Total Unit Price = GST Exc Total ÷ Quantity
Invoice Record Creation
Creates AP invoice with:
- Transaction Type: APINV
- Vendor/Supplier: OtherPartyId (default: "5521")
- Location: Configurable (default: "Misc")
- Payment Date: 20th of month following invoice date
- Reference: Invoice Number from PDF
- Line items with all pricing and tax details
Batch Processing
Optional batch creation groups related invoices:
- Batch ID defaults to supplier/vendor ID
- Includes batch comments for tracking
- Supports recurring batch patterns
Error Handling
The system includes comprehensive error handling:
PDF Processing Errors
- Invalid or corrupted PDF files
- Missing or unreadable pages
- Unsupported PDF formats
Data Extraction Errors
- No invoice detected in PDF
- Invalid JSON response from AI model
- Missing required fields
- Multiple invoices in single PDF (selects largest by subtotal)
Invoice Creation Errors
- Invalid supplier/vendor ID
- Missing required master data (items, departments)
- Transaction validation failures
- Batch creation conflicts
Usage Guidelines
File Selection
- Use the file picker dialog to select one or more PDF files
- Only PDF files are supported
- Multiple files can be processed in sequence
Best Practices
- Ensure invoice PDFs are clear and readable
- Verify supplier master data exists before import
- Check that department codes (serial numbers) are valid
- Review created invoices for accuracy
- Use batch processing for related invoices from same supplier
Post-Import Actions
- Review generated invoices in the system
- Verify line item details and totals
- Confirm department assignments
- Check payment dates
- Approve batches when ready for posting
Technical Architecture
Components
- Import Scripter: Main orchestration class
- AI Vision Service: PDF analysis and data extraction
- Invoice Functions: Business logic for invoice creation
- Transaction Functions: Database persistence layer
Data Flow
PDF File → AI Vision Analysis → JSON Extraction → Data Validation → Invoice Row Creation → AP Invoice Generation → Database Persistence
Performance Considerations
- Batch loading of items and departments
- Dictionary-based lookups for master data
- Minimal database queries during processing
- Asynchronous AI processing support
Configuration Requirements
System Parameters
- AI API credentials (encrypted)
- Default supplier/vendor ID
- Default location ID
- Invoice numbering scheme
- Tax calculation rules (GST rate)
- Payment terms (default 20 days after month-end)
Master Data Prerequisites
- Supplier/vendor records
- Item master data (for parts)
- Department codes (for service invoices)
- Location codes
- Transaction type definitions
Integration Points
The Invoice PDF Import integrates with:
- Item Management: Item lookup and validation
- Department Management: Department code resolution
- Transaction Processing: Invoice creation and posting
- Batch Management: Batch header creation and tracking
- AI Vision Services: External API for document analysis