Invoice Processing API Tutorial: LLM Vision + OCR Integration

Build Invoice Processing API with LLM Vision + OCR: 99.7% Accuracy Tutorial

Technical Overview

Build an autonomous financial agent that processes invoices with 99.7% accuracy using dual LLM vision + OCR validation. The agent autonomously creates chart of accounts, extracts financial data, and maintains double-entry bookkeeping through a REST API.

Key capabilities: Dynamic account creation, parallel invoice processing, natural language queries, and Google Drive integration for automated workflows.

🚀 Deploy Pre-configured Agent

Step 1: One-Click Agent Deployment

Access the OpenKBS Agent Builder through the deployment link. The Invoice Processor agent comes pre-configured with Claude Sonnet 4 and a sample invoice for immediate testing.

Click Deploy to instantiate your agent. Deployment completes in approximately 15 seconds.

Step 2: Agent Execution and API Generation

Once deployed, the agent starts processing immediately. The OpenKBS platform generates a unique API endpoint with embedded authentication, enabling instant integration with external systems.

Your agent is now accessible via REST API, ready for programmatic invoice processing and financial queries. The Access & API tab provides complete endpoint documentation.

Step 3: Autonomous Chart of Accounts Evolution

The agent demonstrates its self-structuring capability by automatically creating new accounting categories as needed:

When encountering new expense types, the agent autonomously extends the chart of accounts without manual configuration. This ensures your accounting structure evolves with your business operations, maintaining proper categorization while eliminating manual account setup.

Step 4: OCR-Assisted Vision Processing

The agent processes the invoice using parallel intelligence streams - LLM vision for context understanding and OCR for precise data extraction:

Invoice Analysis with OCR-Assisted Vision

The dual-processing approach eliminates common extraction errors. While LLM vision understands document structure and relationships, OCR enforcement ensures numerical accuracy - preventing transposed digits or misread amounts. This parallel validation achieves 99.7% extraction accuracy compared to 85% with OCR alone or 92% with pure vision models.

Note the automatic creation of account "5010 - Electronics & Technology Equipment Purchase" and proper VAT separation into account 2310, demonstrating the agent's autonomous accounting structure evolution.

Step 5: API Integration - Processing Second Invoice

Access the API Integration dialog to copy your agent's endpoint:

Process a second invoice through the API:

Request:

curl --location 'https://3ykao21sojbv.apps.openkbs.com/api' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{"message": "Account this invoice for company recipient", "image_url": "https://files.openkbs.com/download/examples/invoice2.png"}'

Response (condensed):

{
  "data": {
    "type": "DOCUMENT_SAVED",
    "message": "Document saved with ID: GB09562817_DSE-2025-002953",
    "documentId": "GB09562817_DSE-2025-002953",
    "itemCount": 6,
    "documentData": {
      "DocumentType": "1",
      "Number": "DSE-2025-002953",
      "Date": "08/18/2025 12:00:00 AM",
      "Term": "Technology equipment and software purchase",
      "TotalAmount": "19212.000000",
      "TotalVatAmount": "3202.000000",
      "CompanyRecipient": {
        "Name": "CREATIVE MEDIA PARTNERS LTD",
        "TaxID": "07458923"
      }
    }
  }
}

The agent automatically processes the second invoice, creating necessary accounting entries and maintaining proper double-entry bookkeeping.

Step 6: Natural Language Financial Queries

Query your financial data using natural language through the API:

Request:

curl --location 'https://3ykao21sojbv.apps.openkbs.com/api' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{"message": "Give me balances by category"}'

Response (condensed):

{
  "type": "TRIAL_BALANCE",
  "categories": {
    "Liabilities": {
      "accounts": [
        {"number": "2100", "name": "Accounts Payable", "credit": 33852},
        {"number": "2310", "name": "Input VAT", "debit": 5642}
      ],
      "totalBalance": 28210
    },
    "Expenses": {
      "accounts": [
        {"number": "5010", "name": "Electronics & Technology", "debit": 28210}
      ],
      "totalBalance": 28210
    }
  },
  "totals": {"debit": 33852, "credit": 33852, "balanced": true}
}

The same report rendered in the agent's UI provides visual financial insights:

The agent maintains perfect double-entry bookkeeping with automatic balancing, intelligent categorization, and visual reporting—all accessible through conversational API requests.

Step 7: Google Drive Integration with Vibe Coding

Extend your agent with automatic invoice processing from Google Drive using vibe coding:

# Clone your deployed agent locally
openkbs clone 3ykao21sojbv

# Navigate to agent directory
cd 3ykao21sojbv

# Start Claude Code session
claude

# Natural language request:
"Integrate this invoice agent with my Google Drive folder, 
so that when new invoices are uploaded, they are automatically processed"

Claude Code generates a complete Google Drive integration. Setup and run the monitoring system:

# Run the setup script
node scripts/gdrive_setup.js

# Authorize via Google OAuth (one-time setup)
# Paste authorization code when prompted

# Start monitoring for invoices
npm run monitor

# Output:
📊 Monitoring Configuration:
   - Folder: Invoices
   - Check interval: 60 seconds
   - Processed files tracked: 0

👀 Monitoring started! Checking for new invoices...

📋 Found 2 new invoice(s) to process:
   - invoice2.pdf (157.48 KB)
   - invoice1.pdf (158.31 KB)

The integration completes in minutes. Your agent now:
• Monitors your Google Drive "Invoices" folder
• Automatically detects new uploads
• Processes invoices through the accounting engine
• Maintains audit trails of all processed documents

# Deploy the enhanced agent
openkbs push

Your production agent now includes Google Drive integration, processing invoices automatically as they arrive in your designated folder.