Opensolr API Endpoint: image_to_text
Overview
The image_to_text endpoint turns a picture into words. You upload an image, and you get back a short list of labels describing what is in it, ready to be used as an ordinary search query.
This is what powers search by image on Opensolr hosted search pages: the visitor uploads a photo, the words come back, and the normal search runs with those words. Nothing about your index changes — no new fields, no image vectors, no re-indexing.
The endpoint compares the picture against a fixed vocabulary of around 51,000 visually grounded concepts (Open Images, ImageNet-21k, iNaturalist species, Google's product taxonomy, dog and cat breeds, flowers, dishes) and returns the closest matches with their similarity scores. When those matches say the picture is a piece of text (a product label, a receipt, a document, a poster) it also reads the text with OCR and puts that in the query instead of, or next to, the labels. See mode below.
Every picture is also checked for a barcode or QR code. When one is found it is decoded and handed back as its own field, codes — it is never mixed into the search query. So there are three layers working on every image: what it looks like, what is printed on it, and any code on it.
Endpoint URL
https://api.opensolr.com/solr_manager/api/image_to_text
Supports only POST requests.
Authentication & Core Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
| string | Yes | Your Opensolr registration email address. | |
| api_key | string | Yes | Your API key from the Opensolr dashboard. |
| index_name | string | Yes | Name of your Opensolr index/core to use. |
Image Parameters
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| image | file | Yes* | – | The image itself, sent as a normal multipart/form-data file upload. |
| image | string | Yes* | – | Alternatively, the image as base64. A data: URI prefix is accepted. |
| top_k | integer | No | 12 | How many labels to return. Values are clamped to a maximum of 50. |
* Send one or the other. If both are present, the uploaded file wins.
Example: file upload
curl -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \ -F "email=YOUR_EMAIL" \ -F "api_key=YOUR_API_KEY" \ -F "index_name=YOUR_INDEX" \ -F "top_k=8" \ -F "image=@/path/to/photo.jpg"
Example: base64
curl -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \ --data-urlencode "email=YOUR_EMAIL" \ --data-urlencode "api_key=YOUR_API_KEY" \ --data-urlencode "index_name=YOUR_INDEX" \ --data-urlencode "image@/path/to/photo.b64"
Response
{ "text": "Lion, Puma (Animal), Cougar, Masai lion, Felidae", "mode": "clip", "ocr_text": "", "ocr_title": "", "ocr_codes": [], "codes": [], "labels": [ { "label": "Lion", "score": 0.3141 }, { "label": "Puma (Animal)", "score": 0.3043 }, { "label": "Cougar", "score": 0.3020 }, { "label": "Masai lion", "score": 0.3001 }, { "label": "Felidae", "score": 0.2962 } ], "thumb_b64": "…", "preview_b64": "…", "model": "openai/clip-vit-large-patch14", "vocabulary_size": 50991, "ocr_ms": 0, "codes_ms": 0, "took_ms": 82 }
| Field | Description |
|---|---|
| text | The finished query — feed this straight into your search. Depending on mode it is the labels joined with , or the text read off the picture. |
| mode | clip: the picture is a thing and text is the labels, in score order. ocr: the picture carries text (a label, a shelf tag, a receipt, a document); the labels are dropped and text is the largest print on it plus any article, model or EAN style codes. An ocr query is exact tokens, so run it lexical rather than hybrid — that is what the hosted search page does automatically. |
| ocr_text | Everything the OCR read, cleaned up. Empty when OCR did not run. |
| ocr_title | The largest print in the picture: the product name on a label, the headline on a poster. This is what text starts with in ocr mode. |
| ocr_codes | Article, model or EAN style tokens read off the picture (6048297, G1-88-E), up to three. Appended to text. |
| codes | Any barcode or QR code found on the picture: [{ "type": "...", "text": "..." }]. Checked on every image regardless of mode, empty when none is found. Never part of text — open the value if it is a URL, or search for it as an exact code, on your own. |
| labels | Each label with its cosine similarity, highest first. Always the full visual answer, whatever mode is. |
| thumb_b64 | A 320px JPEG preview of what was received, base64. Useful for showing the visitor which picture produced the results. |
| preview_b64 | A larger 1400px JPEG preview, base64. |
| model | The model that produced the labels. |
| vocabulary_size | How many concepts the image was compared against. |
| ocr_ms | Milliseconds spent reading text, 0 when OCR did not run. |
| codes_ms | Milliseconds spent scanning for a barcode or QR code. |
| took_ms | Server-side processing time in milliseconds. |
Using it for search
The whole point is that the output is plain text, so it goes through your existing search unchanged:
# 1. Describe the image WORDS=$(curl -s -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \ -F "email=YOUR_EMAIL" -F "api_key=YOUR_API_KEY" \ -F "index_name=YOUR_INDEX" -F "top_k=8" \ -F "image=@photo.jpg" | jq -r .text) # 2. Search with them, exactly like a typed query curl -s "https://api.opensolr.com/solr_manager/api/embed_and_search" \ --data-urlencode "email=YOUR_EMAIL" \ --data-urlencode "api_key=YOUR_API_KEY" \ --data-urlencode "index_name=YOUR_INDEX" \ --data-urlencode "query=$WORDS"
Because the query is text, everything you already tuned keeps working: hybrid search, facets, filters, freshness bias, query elevation and caching.
Accepted images
| Limit | Value |
|---|---|
| Maximum size | 20 MB |
| Formats | JPEG, PNG, WEBP, GIF, BMP, TIFF, JPEG 2000, HEIC/HEIF, AVIF |
| Maximum pixels | 50 megapixels |
The format is determined from the content of the file, never from its name or from the Content-Type header you send. A file named photo.png that is not really an image is rejected. Animated images are read as their first frame, and EXIF metadata (including any GPS coordinates) is discarded during processing.
Errors
| HTTP | Meaning |
|---|---|
| 400 | No image supplied, invalid base64, or the file could not be decoded. |
| 413 | The image is larger than 20 MB, or declares more pixels than allowed. |
| 415 | The file is not an image, or is in a format that is not accepted. |
| 429 | Rate limit or monthly AI request cap reached. |
Special Notes
- Each call counts as one AI request against your monthly AI quota, exactly like
embed. See the pricing page page. - The endpoint is rate limited per minute and per hour like every other Opensolr API endpoint.
- If you use a scoped API key, tick image_to_text in the key's scopes, otherwise the call returns
ERROR_SCOPED_KEY_ENDPOINT_NOT_ALLOWED. - Your image is not stored. It is processed in memory and discarded; the previews returned to you are freshly encoded copies.
- Labels are English. Vector search crosses languages on its own, so an English label set still retrieves documents written in other languages.
- This endpoint lives on
api.opensolr.com, like every other AI endpoint.