Retrieval gives you style, not facts

The voice work is going well, so today I try the real thing: a report email to a foundation, with nothing but the author, the funder and the purpose. Kamali finds similar emails the team has sent and writes in their style.

Three attempts

The first two fail a check I'd added: every number in the email must appear in the brief. The model is copying figures from the example emails. Other years, other funders, the wrong programme. Plausible and false.

The third passes, because it contains no numbers at all. It reads well and says nothing. Nobody would send it.

What the real one had

I put the generated email next to the one the team really sent. The real one is about 2,900 characters with about twelve concrete facts: how many people were reached, which milestone, what the money paid for, what comes next. The generated one is about 1,750 characters with none.

That's the whole problem in one comparison. Retrieval found good examples, and good examples give you tone, structure and length. They don't give you this year's facts for this funder. If the facts aren't in what the model is given, it either invents them or leaves them out.

So the product changes

As long as Kamali doesn't hold the facts, it can't write the part of the email that matters. So the next weeks are about facts, not prompts:

• who each funder is and who the people behind it are,
• every exchange with them, from the email folders themselves,
• what was promised in each direction,
• and the figures in years of reports, read once and kept with their source.

The model's job shrinks to something it's good at: putting known facts into the right words. Finding the facts is the job of everything around it.