AI-API - Search by Image (image_to_text)

Opensolr API Endpoint: image_to_text

Overview

The image_to_text endpoint turns a picture into words. You upload an image, and you get back the names of the things in it and one sentence describing it, ready to be used as an ordinary search query.

This is what powers search by image on Opensolr hosted search pages: the visitor uploads a photo, the words come back, and the normal search runs with those words. Nothing about your index changes — no new fields, no image vectors, no re-indexing.

Two readers work on every picture at the same time. An image model (Florence-2) names the objects it finds in the picture and writes one sentence about it, in its own words, with no fixed list of concepts. At the same time the picture is read with OCR (tesseract, on the CPUs of api.opensolr.com). When the OCR reads real text in it (a product label, a receipt, a document, a poster: at least four confidently read words of three letters or more), that text becomes the query instead of the model's words. See mode below. When it reads nothing worth keeping, the query is the model's words. If you only need the printed text, call image_ocr directly, which takes ten pictures per call.

Every picture is also checked for a barcode or QR code. When one is found it is decoded and handed back as its own field, codes — it is never mixed into the search query. So there are three layers working on every image: what it looks like, what is printed on it, and any code on it.

Endpoint URL

https://api.opensolr.com/solr_manager/api/image_to_text

Supports only POST requests.

Authentication & Core Parameters

Image Parameters

* Send image (a file or base64) for one picture, or images for up to ten. The single-picture answer is unchanged: the object itself, no wrapper. If both image and an uploaded file are present, the uploaded file wins.

Example: file upload

curl -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \
  -F "email=YOUR_EMAIL" \
  -F "api_key=YOUR_API_KEY" \
  -F "index_name=YOUR_INDEX" \
  -F "image=@/path/to/photo.jpg"

Example: base64

curl -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \
  --data-urlencode "email=YOUR_EMAIL" \
  --data-urlencode "api_key=YOUR_API_KEY" \
  --data-urlencode "index_name=YOUR_INDEX" \
  --data-urlencode "image@/path/to/photo.b64"

Response

{
  "text": "lion, A lion lying in the grass next to a tree.",
  "mode": "clip",
  "caption": "A lion lying in the grass next to a tree.",
  "ocr_text": "",
  "ocr_title": "",
  "ocr_codes": [],
  "codes": [],
  "labels": [
    { "label": "lion", "score": 1.0 }
  ],
  "thumb_b64": "…",
  "preview_b64": "…",
  "model": "florence-community/Florence-2-base",
  "vocabulary_size": 0,
  "ocr_ms": 240,
  "codes_ms": 12,
  "took_ms": 310
}

Using it for search

The whole point is that the output is plain text, so it goes through your existing search unchanged:

# 1. Describe the image
WORDS=$(curl -s -X POST "https://api.opensolr.com/solr_manager/api/image_to_text" \
  -F "email=YOUR_EMAIL" -F "api_key=YOUR_API_KEY" \
  -F "index_name=YOUR_INDEX" \
  -F "image=@photo.jpg" | jq -r .text)

# 2. Search with them, exactly like a typed query
curl -s "https://api.opensolr.com/solr_manager/api/embed_and_search" \
  --data-urlencode "email=YOUR_EMAIL" \
  --data-urlencode "api_key=YOUR_API_KEY" \
  --data-urlencode "index_name=YOUR_INDEX" \
  --data-urlencode "fresh=no" \
  --data-urlencode "q=$WORDS"

Because the query is text, it runs like a typed search: on the hosted search page with everything set up there, or through embed_and_search with the Search Tuning of your index.

Accepted images

The format is determined from the content of the file, never from its name or from the Content-Type header you send. A file named photo.png that is not really an image is rejected. Animated images are read as their first frame, and EXIF metadata (including any GPS coordinates) is discarded during processing.

Errors

Special Notes

• Each picture counts as one AI request against your monthly AI quota: 0.5 for reading its text (OCR) and 0.5 for working out what it shows. A picture answered from cache counts nothing. With images, charged says how many the call counted.

Every plan includes a monthly AI allowance of 500 AI requests; see pricing and API Quota.

• The endpoint is rate limited per minute and per hour like every other Opensolr API endpoint.
• If you use a scoped API key, tick image_to_text in the key's scopes, otherwise the call returns ERROR_SCOPED_KEY_ENDPOINT_NOT_ALLOWED.
• Your image is not stored. It is processed in memory and discarded; the previews returned to you are freshly encoded copies.
• The object names and the sentence are English. Vector search crosses languages on its own, so English words still retrieve documents written in other languages. The OCR reads English and Romanian.
• This endpoint lives on api.opensolr.com, like every other AI endpoint.