Kamali · AI
A promise to a funder is an account, not an extraction
Reading only the newest agreements found one obligation. Reading a funder's whole history, in one go, found twelve.
An NGO owes things to its funders: a report by June, a final report at the end of the project, a logo on a sign, a visit. They owe things back: the second tranche, after the first report. Missing one of these costs money and trust. So Kamali needs to know, for every funder, what was promised and what has happened since.
One field was the wrong shape
The first version kept one "next obligation" per funder. That's not how it works. One funder can have a dozen open promises across several agreements. So it's a table now: each promise with the document and the exact sentence it comes from, a direction (we owe them, or they owe us), and a lifecycle. A promise that gets replaced is marked superseded, never deleted. Whether it's been done is worked out from what happened afterwards, not stored as a guess.
Read the whole history
My first pass read only the newest eight agreements per funder, to keep it cheap. For one funder that found a single obligation. Reading its whole history found twelve. Several of the eight slots had also gone to duplicates, because the Word and PDF copies of a letter counted twice.
Batching was its own trap. Reading eight documents at a time, a covering note that said "please find our report attached" looked like a promise. It isn't, it's a delivery. Now the commitment clauses are quoted once, when each document is first classified, and then one call per funder reads that funder's whole history at once. That's also cheaper: 21,000 tokens instead of 199,000.
Rules that come from the data
- A promise is made before it comes due. If a document's date is after the due date it mentions, the document is the delivery, not the promise.
- Ids come from what the promise is, not where it sits in a list. Otherwise a re-run renumbers everything.
- The same input should give the same answer. Three runs gave 20, 24 and 23 promises until the model's randomness was turned off.
It's less an extraction than bookkeeping. The model reads sentences; the code keeps the account.