Create vector embeddings for every document in your Opensolr Index

Pages Opensolr crawls and documents you push with Data Ingestion get their meaning vectors on their own. embed_opensolr_index is for documents you put in an index another way, such as your own indexing code: it reads them back, makes a meaning vector for each one and writes it into the vector field.

Read this first. The vector is written with a Solr atomic update. Solr rebuilds each document from its stored fields and docValues, so a field that is neither stored="true" nor docValues="true" is lost from every document this call touches. Check your schema.xml before you run it, or send your documents again yourself with vectors made by batch_embed. The rules: Solr atomic updates.

01 · What the index needs

A vector field of 1,024 dimensions with cosine similarity. Indexes with the Opensolr schema already have it, named embeddings.

The update log in solrconfig.xml, which atomic updates need. The Opensolr configuration has it.

<field name="embeddings" type="knn_vector" indexed="true" stored="false"/>
<fieldType name="knn_vector" class="solr.DenseVectorField" vectorDimension="1024" similarityFunction="cosine"/>

<updateHandler class="solr.DirectUpdateHandler2">
  <updateLog>
    <int name="numVersionBuckets">65536</int>
    <int name="maxNumLogsToKeep">10</int>
    <int name="numRecordsToKeep">10</int>
  </updateLog>
</updateHandler>

You edit both files in the Configuration tab of your index: Configuration files.

02 · The request

https://api.opensolr.com/solr_manager/api/embed_opensolr_index

GET or POST.

Parameter	Default	What it does
`email`	required	The email of your Opensolr account.
`api_key`	required	Your API key, or a scoped key that allows `embed_opensolr_index`.
`index_name`	required	The index to add vectors to.
`emb_solr_fields`	`title,description,text`	The fields whose text makes the vector, comma separated. Their values are joined with spaces. A document with no text in them is skipped.
`emb_solr_embeddings_field_name`	`embeddings`	The vector field to write.
`emb_full_solr_grab`	off	`yes`: work through every document that has no vector in that field yet, until none is left or the AI allowance runs out.
`emb_solr_start`	`0`	Without `emb_full_solr_grab`: where to start in the index.
`emb_solr_rows`	`10`	Without `emb_full_solr_grab`: how many documents to take.

Without emb_full_solr_grab the call takes the documents at that position whether or not they already have a vector, and makes their vectors again. Vectors are written to the index as the call goes, in groups of up to 100 documents.

03 · The answer

A JSON object, when the call has finished:

{
  "status": "ok",
  "processed": 1000,
  "updated": 998,
  "gpu_count": 998,
  "capped": false,
  "model": "intfloat/multilingual-e5-large-instruct"
}

Field	What it holds
`processed`	Documents read from the index.
`updated`	Documents whose vector was written.
`gpu_count`	Vectors made. This is what counts against the AI allowance.
`capped`	`true` when the call stopped because the AI allowance ran out. Run it again when there are requests left.
`model`, `cache_tag`	The model that made the vectors.

Answer	Why
`VECTOR_NOT_ALLOWED`	AI is switched off on the plan of the account that owns the index.
`ERROR_AI_MONTHLY_QUOTA_EXCEEDED` (HTTP 429)	Nothing is left of this month's AI allowance.
`{"error":"Solr fetch failed: ..."}`	The index refused the read.
`{"error":"Solr update failed", ...}`	The index refused a write, for example because the vector field is missing or has another size. The call stops there.
`ERROR_EMBED_SERVICE_UNAVAILABLE`	The vectors could not be made. Send it again later.
`ERROR_NOT_CORE_OWNER` (HTTP 404)	The index is not one your account can use.

04 · What it uses from your plan

Each document embedded counts as one AI request of the account that owns the index. There is no cache here: every vector is made again.

The call never spends more than what is left of the allowance: it stops and answers "capped": true.

Every plan includes a monthly AI allowance of 500 AI requests; see pricing and API Quota.

05 · Examples

# every document that has no vector yet
curl -s https://api.opensolr.com/solr_manager/api/embed_opensolr_index \
  -d "email=YOUR_EMAIL" \
  -d "api_key=YOUR_API_KEY" \
  -d "index_name=YOUR_INDEX" \
  -d "emb_solr_fields=title,description,text" \
  -d "emb_full_solr_grab=yes"

# 50 documents, starting at the 101st
curl -s https://api.opensolr.com/solr_manager/api/embed_opensolr_index \
  -d "email=YOUR_EMAIL" \
  -d "api_key=YOUR_API_KEY" \
  -d "index_name=YOUR_INDEX" \
  -d "emb_solr_start=100" \
  -d "emb_solr_rows=50"

import requests

r = requests.post("https://api.opensolr.com/solr_manager/api/embed_opensolr_index", data={
    "email": "YOUR_EMAIL",
    "api_key": "YOUR_API_KEY",
    "index_name": "YOUR_INDEX",
    "emb_solr_fields": "title,description,text",
    "emb_full_solr_grab": "yes",
}, timeout=3600)
print(r.text)

06 · Related

Make the vectors yourself, before you index: batch_embed and embed.

How Opensolr uses the vectors: Meaning vectors, done for you and Hybrid search.

All the vector calls in one place: Meaning vectors by API.