The phone's index

One index per phone, named after the phone, created only when it is certainly missing.

Every phone gets one Opensolr Index of its own, in your account. The app finds it, creates it or creates it again — and only ever creates one when it is certain it is missing. The phone also keeps its own copy of what that index holds, which is why so little traffic reaches it.

Reuse itRead addressand password;check theconfig version. Read it onceOnly when thephone has nocopy of it yet. Create itA vectorenvironment,then the app'sconfiguration. Nothing toreadIt is empty;the photos goup from thephone. Is it in the account'sindex list?photos_<ANDROID_ID>__dense The indexone perphone yes no Read down once into the phone's copy: at install, at a reinstall, or after picking an existingdevice. Kept in step by every write.A network error, a server error or a refused key never creates anything: the next sync tries again.
The indexone per phone Is it in the account'sindex list?photos_<ANDROID_ID>__dense yes Reuse itRead address andpassword; check theconfig version. Read it onceOnly when the phone hasno copy of it yet. no Create itA vector environment,then the app'sconfiguration. Nothing to readIt is empty; the photosgo up from the phone. Read down once into thephone's copy: at install, at areinstall, or after picking anexisting device. Kept in stepby every write.A network error, a servererror or a refused key nevercreates anything: the nextsync tries again.

Figure 1 — the life of the phone's index: found and reused, or created when it is certainly missing, and the one read that fills the phone's copy of it.

01 · The name

The index is named photos_<ANDROID_ID>__dense. ANDROID_ID is the identifier Android gives this app on this phone: it stays the same when the app is reinstalled, and it is different on every other phone.

Reinstall on the same phone

Sign in, and the app finds the index by its name. Nothing is indexed again, but this is the one moment the whole index is read down, once, to rebuild the phone's copy of it; after that a Re-Sync adds and deletes the differences.

A new phone, same account

A different ANDROID_ID, so a different name. Before creating anything, the app asks Re-use one of these indexes from your Opensolr account?, listing the phones that already have photos in the account: pick the one this phone used to be (after a reset, or a replacement with the same photos) and it carries on with those, every photo keeping its identity because its id is its path inside the phone's storage; or No, start a new index. Picking an existing one triggers the same single read of that index into this phone's copy.

The name is decided once and remembered on the phone (the one created here, or the one picked as this device); it is never derived again at every start. Every photo index carries the phone it was created for (device_name, device_id on the platform side, sent with create_index), which is what the question shows. Phones that stop syncing are covered in phones you no longer use.

02 · Reusing it

When the name is in the account's index list, the app reads the index's address and password from the account and compares the configuration version the index reports (/opensolr-photos-config) with the one the app ships. Equal: the sync goes on as a Re-Sync. Older: the owner is asked to reset it, see sync. Newer: the app asks to be updated and leaves the index alone. The app never deletes an index; a reset empties it and then hands every photo up again, five per call, carrying the tags, the people and your wording out of the phone's own copy, so nothing you wrote is lost in the round trip.

03 · The phone's copy of it

Since version 2.5 the phone mirrors its index. For every document in the index it keeps the id, the media id, the path, the folder, the file name, the type, the size, the md5 of the file, the dates, the camera make and model, the lens and the rest of the EXIF, the position and the place (city, region, province, community, country), the words the photo was read into, your own wording, the printed text read out of it, the people, your tags, and which model read the photo.

Two things it deliberately does not hold: the search vector and the duplicate keys. Both are large, both are only useful inside Solr, and both are worked out there.

The copy is read out of the index once — at install, at a reinstall, or after picking an existing device's index — in one pass, ten thousand documents per page. From then on it is never read again: every write the app makes updates the copy with the document the server says it wrote, so the two stay in step. A read that does not reach the end does not count, and is done again at the next start.

What it is for, in index terms:

  • A sync no longer walks the index at all. The comparison is your folders against the copy, both on the phone, so a sync with nothing to do reads nothing out of the index. For a library of 10,000 photos version 2.4 made about 21 requests and downloaded about 1.8 MB against the index on every sync, only to learn that nothing had changed.
  • Plain browsing — no words typed, no filters — asks the index nothing. The years, the months, the days, their counts and the photos in them all come from the copy.
  • Tag and name suggestions, and the “already on these photos” list in the tagging sheet, are answered from the copy.
  • The filter lists are asked for once and kept until a sync actually writes something.
Which side is the truth

The copy on the phone. The index is the only thing that searches, lexically and by vector, but the copy is what it is filled back from: a Reset or a new configuration empties the index and writes it again from the copy, 50 documents per request, with no photo sent again and no AI requests used, and a document the index lost is rewritten from it. The copy itself is read from the index only once, at install or reinstall.

One consequence for the index itself: it may sit one sync behind the phone. Tagging is local-first, so tags, names and wording are saved on the phone and finished there, and the sync that starts straight after carries them up through photos_words, 50 photos per call with no pictures attached. Query the index in that window and it still answers with the previous words.

04 · Creating it
  1. Pick the nearest region. Opensolr places the index in the region nearest to where the phone connects from, among the regions that run the app's configuration. Nothing is asked and nothing is shown; when the place cannot be told, the index goes to the default region.
  2. Create the index with that name in that environment.
  3. Upload the configuration that ships inside the app: schema.xml, solrconfig.xml and the analyzer files.
  4. Wait until it answers with the new schema. Right after a create the index needs a few seconds to reload and apply its password and firewall rule; the app waits that out instead of mistaking it for an error.

This is the same on every plan. A plan without vector search still gets an index named photos_<ANDROID_ID>__dense, on an environment that runs vector search, with the same schema; the plan decides only what is put in it. Without vector search nothing is sent to be read, so the vector field stays empty and the photos are indexed by their dates, camera, place, file name and your own words, and searched lexically.

05 · When it vanished

The account's index list is read at the start of every sync, so an index deleted from the control panel, removed as unused, or gone for any other reason is noticed at the next sync and created again the same way, empty, with a notification saying so. Getting the photos back into it is a separate step: the phone's copy still describes the index as it was, so the sync that recreated it has nothing to add. Re-read or Reset on the Sync screen hands every photo to Opensolr again, with the tags, the people and the wording the copy holds, and a picture Opensolr has already read is served from its cache.

The one rule that prevents stray indexes

An index is only created after the account's index list was read successfully and did not contain it. A network error, a server error or a refused key never lead to a create: they end the run, and the next sync — forced, scheduled, or started by pulling the photo grid down — simply tries again.

06 · Looking at it yourself

It is a regular Opensolr Index. Open it from the app's account screen or from the control panel to query it, back it up or empty it.

It is plain Apache Solr: query it or export every document with any Solr client, back it up, and run it anywhere. Changing it from outside the app is the one thing to be careful with: edits made to documents from the control panel or any other tool do not reach the phone's copy.

  • Emptied from anywhere, including the control panel: the next sync finds the index empty while the phone's copy holds the photos, and fills it back from the copy.
  • Reset on the Sync screen does the same on purpose: it empties the index and fills it again from the copy on the phone.
  • Re-read leaves the index as it is and writes every photo over what is there, which is enough when the index still holds its documents and only their contents are wrong.
  • Deleting the index from the control panel is noticed (see above): it is created again and filled from the phone's copy.

Opensolr Photos is open source and MIT licensed. Questions about your Opensolr account, index or plan go to opensolr.com/contact; questions about the app itself belong on GitHub.

Opensolr Photos Documentation