Kamali · Search
In an NGO's archive, the newest document should usually win
Combining meaning and keyword search helped. What took "right document in the top five" from 33% to 90% was something else: preferring this year's edition.
Search over an organisation's archive sounds like a solved problem. Embed everything, find the nearest. On this archive it gets the right document into the top five for 33% of my test questions.
Two arms
Search now runs two ways at once inside Postgres. One by meaning, with an open embedding model running on the server. One by words, with Postgres's full text search, stemmed and unstemmed. The two rankings are merged with reciprocal rank fusion. Filters like funder, year or document type apply before both.
Each section of a document is embedded with a short context line in front: its type, funder, year, programme and heading path. A "results" heading means little alone; "annual report 2025, vocational training, results" means a lot.
I also had a stronger model review the design before building it out. A second reader is cheap at that stage and catches things early.
The trick that mattered
Organisations like this one write in yearly editions. The 2024 report and the 2025 report are nearly the same text with different figures. By meaning alone, last year's version often wins, and then everything downstream quotes old numbers.
So sections that are within a small distance of the best match are ordered newest document first. That one rule takes "right document in the top five" from 33% to 90% on my 40 tuning questions, and "an outdated document shown first" from 5 in 9 to 1 in 9. On 20 held-out questions it's 65%; about 87% when I check the misses by hand.
Rerankers
A reranker reorders the top results with a heavier model. One I tried has a non-commercial licence, so it's out. An Apache-licensed one on the top five helps: the right document first goes from 30% to 45%. Reranking the top thirty makes things worse and takes seven seconds.
Dates matter as much as words
Newest-first only works if the dates are right. A document is dated from its filename first, then its own "last saved" date, then the file's time, never by when a copy was made. And the free checks run before any paid evaluation.