Opensolr API Endpoint: embed_opensolr_index

⚠️ Important Disclaimer (Read Before Use!)

In order to not lose any fields from your existing index, make sure that inside your schema.xml, all fields are defined with docValues="true" AND/OR stored="true".

Otherwise, when we create your embeddings vector, you will lose any field that does not have docValues="true" AND/OR stored="true".

Because of this, it's highly recommended to understand the implications of Solr atomic updates clearly. For most users, the safer approach is to create embeddings at indexing time, using the /embed endpoint, especially if you rely on non-stored or non-docValues fields for downstream features.

Please review the official documentation on Solr Atomic Updates to fully understand these implications before using this endpoint.

⚠️ Important Pre-Req:

First make sure you have this in schema.xml

<!--VECTORS-->
<field name="embeddings" type="vector" indexed="true" stored="true" multiValued="false" required="false" />
<fieldType name="vector" class="solr.DenseVectorField" vectorDimension="384" similarityFunction="cosine"/>

Or run the following to create your field and type definition inside your schema.xml via the native Solr API:

$ curl -u <INDEX_USERNAME>:<INDEX_PASSWORD> https://<OPENSOLR_INDEX_HOST>solr/<OPENSOLR_INDEX_NAME>/schema/fieldtypes -H 'Content-type:application/json' -d '{
  "add-field-type": {
    "name": "vector",
    "class": "solr.DenseVectorField",
    "vectorDimension": 384,
    "similarityFunction": "cosine"
  }
}'

$ curl -u <INDEX_USERNAME>:<INDEX_PASSWORD> https://<OPENSOLR_INDEX_HOST>solr/<OPENSOLR_INDEX_NAME>/schema/fields -H 'Content-type:application/json' -d '{
  "add-field": {
    "name":"embeddings",
    "type":"vector",
    "indexed":true,
    "stored":false, // true if you want to see the vectors for debugging
    "multiValued":false,
    "required":false,
    "dimension":384,  // adjust to your embedder size
    "similarityFunction":"cosine"
  }
}'

Seocond make sure you have this in solrconfig.xml:

<!-- The default high-performance update handler -->
<updateHandler class="solr.DirectUpdateHandler2">
      
        <updateLog>
          <int name="numVersionBuckets">65536</int>
          <int name="maxNumLogsToKeep">10</int>
          <int name="numRecordsToKeep">10</int>
        </updateLog>

.....

</updateHandler>

Overview

The embed_opensolr_index endpoint allows Opensolr users to generate and store text embeddings for documents in their Opensolr indexes using a Large Language Model (LLM). These embeddings power advanced features such as semantic search, classification, and artificial intelligence capabilities on top of your Solr data.

Endpoint URL

https://api.opensolr.com/solr_manager/api/embed_opensolr_index

Supports both GET and POST methods.

Authentication Parameters

Embedding Parameters

Special Notes on Field Configuration

• The API grabs data from the fields listed in emb_solr_fields, which defaults to title,description,text, but you may specify any fields from your index for embedding.
• You can specify emb_solr_embeddings_field_name to match the embeddings field in your schema.
• IMPORTANT: Your Solr index must have an embeddings field defined in your schema.xml. Example configuration:

<field name="embeddings" type="vector" indexed="true" stored="false" multiValued="false"/>
<fieldType name="vector" class="solr.DenseVectorField" vectorDimension="384" required="false" similarityFunction="cosine"/>

• Replace embeddings and vector with your custom names if you use different field names.

Atomic Updates in Solr (Brief Explanation)

Solr atomic updates update only the fields you specify in the update request. Other fields—including those defined as non-stored (stored=false)—are not changed or removed by an atomic update. However, since non-stored fields cannot be retrieved from Solr, you cannot use them to generate embeddings after indexing time.

If you ever replace an entire document (full overwrite), non-stored fields will be lost unless you explicitly provide their values again.

Parameter Details

• email: Your Opensolr account email. Used for authentication.
• api_key: Secret API key from your Opensolr dashboard.
• index_name: Name of the Solr core/index (must belong to your account).
• emb_solr_fields: Fields to embed from your Solr documents. Comma-separated. Only text fields are supported.
• emb_solr_embeddings_field_name: The Solr field where the generated embedding vector will be stored.
• emb_full_solr_grab: Set to yes to embed all documents in the index; otherwise, the endpoint uses pagination.
• emb_solr_start: Offset for selecting documents (for pagination through large datasets).
• emb_solr_rows: Number of documents to embed per request (batch size).

Example Usage

Example: Minimal POST Request

POST https://api.opensolr.com/solr_manager/api/embed_opensolr_index
Content-Type: application/x-www-form-urlencoded

[email protected]&api_key=YOUR_API_KEY&index_name=your_index

Example: Full POST Request

POST https://api.opensolr.com/solr_manager/api/embed_opensolr_index
Content-Type: application/x-www-form-urlencoded

[email protected]&api_key=YOUR_API_KEY&index_name=your_index&emb_solr_fields=title,content&emb_solr_embeddings_field_name=embeddings&emb_full_solr_grab=yes

Example: GET Request

GET https://api.opensolr.com/solr_manager/api/[email protected]&api_key=YOUR_API_KEY&index_name=your_index

Endpoint Behavior

1. Authenticates the user with email and api_key.
2. Selects the index named index_name.
3. Fetches document data from the fields listed in emb_solr_fields.
4. Generates embeddings for each document and stores them in the specified emb_solr_embeddings_field_name.
5. If emb_full_solr_grab is yes, processes all documents; otherwise uses emb_solr_start and emb_solr_rows for batch processing.
6. Responds with plain text status/progress updates.

Use Cases

• Enable vector/semantic search on your Opensolr index
• Prepare your data for AI and LLM-powered analysis
• Periodically update embeddings after document changes

Support

For more information or help, visit Opensolr Support or use your Opensolr dashboard.