Automating LHDN e-Invoice compliance for 2,000+ documents a month

Malaysia's move to mandatory e-Invoicing changed the paperwork of every business that falls under it. Invoices, credit notes and debit notes now have to be submitted electronically to LHDN, the Inland Revenue Board of Malaysia, through the MyInvois system, in a structured format that passes validation. For a company that receives supplier documents as PDFs and spreadsheets, that is a lot of manual work: open the attachment, read the fields, type them into a system, check them, submit, and chase whatever comes back rejected.

I led the development of a multi-tenant platform that does this automatically. It processes more than 2,000 financial documents a month: it extracts the data, validates it, transforms it into the required format and submits it, for several separate business tenants at once. This post explains how a pipeline like this is put together and the design decisions that mattered most.

A note before starting: e-Invoicing rules are specific and they change. Anything in this post about the regulation is general background. For the actual requirements, always use LHDN's current official guidance.

The problem with doing it by hand

Manual e-Invoice handling has three costs that grow with volume.

Time. Every document means reading and re-typing the same fields: invoice number, dates, supplier and buyer tax identification numbers, line items, tax and totals. At a few documents a day that is an annoyance. At hundreds a week it is someone's whole job.

Errors. Re-typing produces typos, and typos produce rejections. A tax identification number with one wrong character, totals that do not add up to the line items, or a date in the wrong format each mean a document that bounces back and has to be fixed and resubmitted.

Visibility. When the process lives in inboxes and spreadsheets, nobody can easily answer basic questions: which documents are still pending, which were rejected and why, and what was submitted for which company last month.

Automation addresses all three, but only if it is built carefully. A pipeline that submits bad data quickly is worse than a person who submits good data slowly.

The shape of the pipeline

The platform follows a classic extract, transform and load (ETL) shape, with a validation stage that does most of the real work.

1. Ingest. Documents arrive as email attachments. The pipeline connects to the relevant mailboxes through the Microsoft Graph API, finds new messages, and pulls the PDF and Excel attachments.
2. Extract. Each document is parsed and its fields pulled out: identifiers, parties, dates, line items, tax and totals.
3. Validate. The extracted data is checked against the rules it has to satisfy before anything is submitted.
4. Transform. Valid data is turned into the structured JSON the submission step expects.
5. Submit. The document is submitted, and the result, accepted or rejected, is recorded.
6. Record. Everything is stored in PostgreSQL so every document has a traceable history.

On the home page of this site there is a simulated replay of steps two to five: a supplier invoice is scanned, its fields are highlighted as they are extracted, the JSON builds up line by line, the validation checks tick off, and the document is submitted. The real platform does the same thing, thousands of times a month, without anyone watching.

Ingestion: meeting documents where they already are

The first design decision was not to ask anyone to change how they work. Suppliers already email invoices. Staff already forward documents to shared mailboxes. So the pipeline starts in the mailbox.

Microsoft Graph makes this possible for Microsoft 365 mailboxes. With the right permissions, a service can list messages, read attachments and mark messages as processed. A few things matter in practice:

• Authentication uses OAuth 2.0, so the pipeline never holds anyone's password, and access can be scoped and revoked.
• Processing must be idempotent. If the pipeline sees the same email twice, for example after a restart, it must not submit the same invoice twice. Every document gets a stable identity so duplicates are recognised and skipped.
• Not every attachment is an invoice. Signatures, logos and unrelated files arrive too. The pipeline filters by type and content before doing anything expensive.

Extraction: PDFs and spreadsheets are different problems

Spreadsheets are structured but inconsistent. Every supplier lays out their Excel files differently, with different column names, header rows in different places and merged cells. Extraction here is mostly about mapping: working out which column holds which field for each layout, and handling the variations reliably.

PDFs are the opposite: visually consistent for a human but unstructured for a machine. The same field can sit in a different place on every supplier's template. Extraction means locating each field, reading it, and normalising it: dates into one format, amounts into numbers, and identifiers stripped of stray spaces.

The practical lesson from both is to separate finding a value from trusting it. Extraction produces candidate values. The next stage decides whether they are good enough to submit.

Validation: the stage that earns its keep

Validation is where the pipeline protects the business from its own data. Before any document is submitted, it passes a set of checks. Typical checks include:

• Required fields are present. No document goes forward without the identifiers, dates and parties it needs.
• Formats are correct. Tax identification numbers, dates and codes match the patterns they are supposed to follow.
• The numbers reconcile. Line items add up to the subtotal, tax is computed correctly for the rate that applies, and the subtotal plus tax equals the total.
• The structure is valid. The final JSON matches the schema the submission step requires.

When a check fails, the document does not disappear. It is flagged with a clear reason, such as "total does not match line items", so a person can look at exactly that document and fix exactly that problem. The goal is for people to spend their time only on the documents that genuinely need judgement, instead of re-typing the ones that do not.

This is the core idea of the whole platform: deterministic checks around every automated step. Extraction can be clever, but validation must be strict and predictable.

Multi-tenancy: many businesses, one platform

The platform serves several separate business tenants. Multi-tenancy adds its own requirements on top of the pipeline itself.

Isolation. One tenant's documents, credentials and records must never be visible to another. Every piece of data carries its tenant, every query is scoped to a tenant, and credentials are stored and used per tenant.

Per-tenant configuration. Different tenants have different mailboxes, different suppliers and different document layouts. Configuration lives with the tenant, not in the code, so adding a tenant is a setup task rather than a development project.

Independent failure. A problem with one tenant's mailbox or credentials must not stop the others. Each tenant's processing is independent, so an issue is contained to where it happened.

Clear reporting per tenant. Each business needs to see its own status: what was received, what was submitted, what was rejected and why.

Recording everything

Every document that passes through the platform leaves a record in PostgreSQL: where it came from, what was extracted, which checks it passed or failed, what was submitted and what came back. That record serves several purposes.

• Audit. Financial documents need a trail. When someone asks what happened to a particular invoice, the answer is a query, not an investigation.
• Retries. When a submission fails for a temporary reason, the stored state means it can be retried safely without re-extracting anything.
• Improvement. Patterns in validation failures show where extraction needs work, for example a supplier whose layout keeps causing a particular field to be misread.

Handling rejections and resubmissions

Even with strict validation, some submissions will come back rejected. The reasons vary: a rule the pipeline did not know about, a value that was valid in format but wrong in fact, or a temporary problem on the receiving side. How the platform handles these cases decides whether automation saves time or just moves the work around.

Three kinds of failure need different treatment:

• Temporary failures, such as timeouts or a service being briefly unavailable, should be retried automatically with a sensible delay. Nobody needs to see them unless they keep happening.
• Data problems need a person. The document goes to a review queue with the rejection reason attached, the extracted values shown next to the original file, and a way to correct and resubmit. The person fixes one field instead of re-entering the document.
• Rule problems, where the same kind of rejection keeps appearing across many documents, point to a gap in validation. The fix belongs in the pipeline, so the check catches it next time before submission.

Treating these differently keeps the review queue short and meaningful. If every hiccup landed in front of a person, they would quickly learn to ignore the queue.

A document's journey, end to end

Here is what happens to a single supplier invoice.

A supplier emails a PDF invoice to a shared accounts mailbox at 9:12 in the morning. Within minutes the pipeline, polling that tenant's mailbox through Microsoft Graph, sees the new message, confirms the attachment is an invoice it has not processed before, and downloads it.

Extraction reads the supplier's name and tax identification number, the invoice number and date, four line items, the tax amount and the total. Normalisation turns the date into a standard format and the amounts into numbers.

Validation runs. The identification number matches its expected pattern. The line items add up to the subtotal. The tax matches the rate applied to the subtotal. The total equals the subtotal plus tax. The transformed JSON passes the schema check.

The document is submitted. The response comes back accepted, and the record in PostgreSQL is updated with the submission result and identifiers. On the tenant's dashboard, the invoice moves from "received" to "submitted".

Nobody opened the PDF. Nobody typed a field. If any check had failed, the invoice would instead be sitting in the review queue with a one-line reason and the original file beside it.

What made the difference

Looking back, a handful of decisions mattered more than any clever extraction technique:

• Start from the mailbox. Automation that requires people to change their habits gets adopted slowly. Automation that sits behind habits they already have gets adopted immediately.
• Make every step idempotent. Restarts, retries and duplicate emails are normal. The pipeline must be safe to run twice.
• Separate extraction from validation. Let extraction be flexible and let validation be strict.
• Explain every rejection. A flagged document with a specific reason takes a minute to fix. A vague failure takes an hour.
• Isolate tenants completely. Data, credentials, configuration and failures all stay within the tenant they belong to.
• Keep an audit trail from day one. Financial automation without traceability will not be trusted, and it should not be.

Is this worth automating for your business?

If you receive a steady stream of invoices, credit notes or debit notes as PDFs or spreadsheets and someone is re-typing them, the answer is usually yes. The signals to look for are volume (dozens of documents a week or more), repetition (the same suppliers and layouts again and again) and pain (rejections, backlogs, or a person whose week disappears into data entry).

The first step is not software. It is a clear picture of your current flow: where documents arrive, who touches them, what gets checked and where things go wrong. From there, the pipeline design above adapts to your mailboxes, your suppliers and your systems.

Summary

A multi-tenant e-Invoice platform is an ETL pipeline with a strict validation stage at its centre. Documents are ingested from the mailboxes where they already arrive, extracted from PDFs and spreadsheets, checked against format and reconciliation rules, transformed into structured JSON and submitted, with every step recorded. Built this way, it handles more than 2,000 financial documents a month across several businesses, and people only touch the documents that actually need a human decision.

If you want to talk about automating document handling in your own business, ask Alfred for a quick answer or send me a message.