Setting Up Local LLM for Lekhya AI

Run AI features on-premise with a local LLM. Your financial data never leaves your server.

Why local LLM?

Financial data is sensitive. Using a local model means invoice content, GSTIN, and ledger data never goes to an external API. Suitable for CA firms, government clients, and privacy-first businesses.

Recommended Models for ERP Use

Llama 3.2 (3B)

Recommended for small setups

Best for invoice extraction + coding on CPU/low RAM. Runs on 4GB RAM.

ollama pull llama3.2

Llama 3.1 (8B)

Recommended for medium setups

Better reasoning for complex queries. Needs 8GB+ RAM.

ollama pull llama3.1

Mistral 7B

Great for data extraction tasks

Excellent at structured output (JSON extraction). Good for invoice parsing.

ollama pull mistral

DeepSeek-R1 (7B)

Best for reconciliation logic

Strong math reasoning — useful for reconciliation tasks.

ollama pull deepseek-r1

Step 1 — Install Ollama

# Linux / Ubuntu (VPS / Hostinger Cloud)

curl -fsSL https://ollama.com/install.sh | sh

# Verify installation

ollama --version

# Pull recommended model for Lekhya

ollama pull llama3.2

# Start Ollama service (or it auto-starts)

ollama serve

# Default API: http://localhost:11434

Step 2 — Configure Lekhya .env

# In lekhya-app/.env

AI_DRIVER=ollama

AI_ENDPOINT=http://localhost:11434/api/generate

AI_MODEL=llama3.2

AI_MAX_TOKENS=2048

AI_TEMPERATURE=0.1

# For cloud providers (fallback)

# AI_DRIVER=anthropic

# ANTHROPIC_API_KEY=sk-ant-...

# AI_MODEL=claude-haiku-4-5-20251001

Step 3 — How Lekhya Calls the AI

Lekhya uses a single AiProvider interface. Switch models by changing .env — no code changes.

Invoice Extraction Prompt (what Lekhya sends)

You are an Indian GST accounting assistant. Extract structured invoice data from the following text. Return ONLY valid JSON with this structure: { "invoice_number": "INV-001", "invoice_date": "2024-04-15", "vendor_name": "Supplier Pvt Ltd", "vendor_gstin": "29ABCDE1234F1Z5", "line_items": [ {"description": "IT Services", "hsn_sac": "998314", "quantity": 1, "rate": 10000, "cgst_rate": 9, "sgst_rate": 9} ], "total_taxable": 10000, "total_gst": 1800, "total_amount": 11800 } Invoice text: {extracted_text_here}

Step 4 — OCR for PDF/Image Invoices

# Install Tesseract OCR (Ubuntu)

sudo apt-get install tesseract-ocr tesseract-ocr-hin

# Install PHP wrapper

composer require thiagoalessio/tesseract_ocr

# For PDF to image conversion

sudo apt-get install ghostscript imagemagick

composer require spatie/pdf-to-image

Tip: For vision-capable models (LLaVA, Llama 3.2 Vision), you can skip OCR and send the image directly to the model. Enable this in .env: AI_USE_VISION=true

Hardware Requirements

SetupRAMCPU/GPUModelBest For
Minimal4GBAny modern CPULlama 3.2 3BBasic extraction, small volumes
Recommended16GBCPU or GPULlama 3.1 8BProduction use, faster responses
High Volume32GB+GPU (NVIDIA)Llama 3.1 70BCA firms, bulk processing

AI Predictions & Activity Monitoring

Cash Flow Prediction

AI analyzes your AR/AP aging, seasonal patterns → predicts cash position 30/60/90 days out. View in Dashboard → AI Insights.

Spend Anomalies

AI monitors monthly vendor payments for outliers. "Transport costs 3x this month" → flagged for review.

GST Filing Reminders

AI predicts how much GST is due based on current-month postings. Alert before due date.

ITC Optimization

AI spots ITC that's about to expire (2-year limit) and flags invoices to claim before cutoff.