I'm building an invoice-reconciliation agent for a client this year: it reads incoming email, figures out which ones are invoice inquiries, pulls out the invoice numbers involved, and checks their status against SAP and Ariba. I can't name the client, but I can tell you exactly how the build went, because it went the way most of these go — by "turns out."
The first attempt was deliberately boring: no LLM anywhere in the extraction path. The principle was avoid the model where you don't need it, and invoice numbers looked like exactly the kind of thing a regex could own — a pattern, sitting in a wall of text. That held for about as long as it took to see a real inbox. The regex was happy to match phone numbers, PO numbers, anything shaped enough like a number sequence, because "shaped like a number sequence" was all it actually knew how to check. It had no idea what an invoice number was. It just knew what one looked like, badly.
Then the inputs stopped being plain text at all. Some invoice lists arrived as PDFs. Some arrived as spreadsheets — and some of those spreadsheets had multiple sheets, one with the real line items and another that was just a summary, so "read the spreadsheet" wasn't one operation, it was "figure out which sheet is the real one, first." And the column holding the invoice number wasn't even consistently named: some vendors called it Invoice Number, some called it Doc Number, and some called it SAP ID — which was real, and correct, and also not our invoice number at all. It was the vendor's own SAP identifier for the transaction on their side. Same concept everyone needed to talk about, three-plus names, one of them actively misleading. That's field mapping, just one hop earlier than I usually meet it — not between two systems, but between what each vendor calls a thing before it ever reaches a system at all.
And I'd built around two counts that were both wrong. I assumed one invoice number per email; most emails referenced several. I assumed a reply per invoice; what the client actually wanted was one reply per email, listing every invoice we found and its status.
Every one of those was tempting to patch in place — a special case for the summary sheet, a special case for the SAP ID column, a loop around the reply so it batches instead of firing once per invoice. And for a while, that's what the code did, because that's the fastest way to keep the client's inbox moving while you're still learning what you're building. But patched in place enough times, you can't find anything anymore. A new vendor shows up with a fourth name for the same column, and the honest answer to "where do I add that" is "wherever the last three got added," which isn't an answer, it's a shrug. That's the bill nobody budgets for showing up early, inside a single codebase, instead of showing up years later across a whole integration.
What was actually missing wasn't more branches. It was stages that had been implicit the whole time and never got written down as stages: pull candidate values out of whatever the source is — body text, PDF, spreadsheet, whichever sheet is the real one — normalize whatever name that value arrived under to one canonical field, check that the candidate is actually invoice-shaped before trusting it, resolve it to our invoice number if what came in was someone else's identifier for the same thing, look up status in each system, translate each system's own word for "paid" into one shared vocabulary, and only after all of that, gather everything that came out of one email and answer it once. Eight stages. The code had all eight, technically — just braided into three or four functions instead of named as eight separate, small, answerable things.
I keep reaching for "maintainable" and "discoverable" when I talk about wanting to fix this, and both words are doing the same job: standing in for simple, because simple is hard to justify on its own and maintainable sounds like a requirement. But pressed on it, what I actually mean is narrow and checkable: can I point at the one place responsible for a given decision, without reading the whole pipeline to find it. "Is this string actually an invoice number" should live in exactly one place. So should "what does SAP mean when it says paid." Right now some of those decisions live in one place and some are duplicated in two or three, and the difference between those two states is the entire gap between maintainable and not.
Here's the part I didn't expect going into this: the zero-token attempt wasn't a wrong turn I'd take back. It was the only way to actually meet the real emails — you don't discover that vendors have three names for one column, or that summary sheets exist, or that "paid" and "posted" are the same fact in two different mouths, by thinking harder about the design up front. You discover it by watching something too simple fail on real input, over and over, until the failures themselves outline the shape you should have built. I couldn't have designed those eight stages on day one. Nobody could have. The design pass I'm doing now isn't correcting a mistake — it's finally transcribing what forty edge cases already taught the code, back when it was still small enough to only handle one of them at a time.
Field Mappings