Kamali · Search · Photos
A photo archive, searchable on a 4 GB server
Years of photos, read once, searched by meaning on a small box. Most of the work was in what to leave out.
An NGO's photo archive is its memory and its marketing. It's also usually a folder tree nobody can search. This one is well over a hundred thousand files. The goal: type "people planting trees, last spring" and get the right photos, on a server with 4 GB of memory, without paying per photo.
Read once
Each photo is read once, about a second each. It gets an identity from its content hash, its full metadata is kept, and two copies are made: 1280-pixel WebP for viewing and small thumbnails. Location data is stripped from the copies. A CLIP model, quantised to 8 bits, runs on the CPU and turns each photo into a vector. Nothing is paid for.
Two other models looked better and had licences that forbid commercial use, so they're out. Keeping one exiftool process alive per worker took metadata from 0.2 seconds of CPU per photo to 0.006.
Asking in words
A question goes to a cheap model first, which splits it into what to look for, dates, tags and orientation. Then the CLIP text encoder, on the server, searches. A result has to beat twenty generic descriptions, like "a photo" or "children", by a clear margin, or Kamali says it found nothing close and offers the nearest anyway. Each search costs under a tenth of a cent, and repeated questions are cached.
One detail matters to users: the same question in French and English should give the same results, so results follow what you typed, not the model's rewrite of it.
Speed came from the database, not the model. Postgres had 128 MB of shared memory for a 163 MB index, so a cold search took 2.2 seconds and a warm one 6 ms. Giving it 768 MB and warming it up brings every search to about 10 ms.
Bursts, sideways, broken
- Bursts: phones shoot ten near-identical frames. Photos fold into a series when they're very close by CLIP and by perceptual hash, to the previous and the first shot. About a third of the library folds.
- Sideways: photos without camera orientation data get checked by a model, and a review page lets a person confirm. The model is overruled more often than it's right, which is exactly why the review page exists.
- Broken: a few hundred originals are damaged on the drive itself. Most get rebuilt from intact copies or RAW previews found elsewhere on the share. The rest are marked defect and left alone.
Videos get the same treatment, cut into scenes and searchable next to the photos.