Meaning Vectors, Done for You
Opensolr turns every page it crawls and every document you push into a meaning vector: a list of 1,024 numbers that captures what the text is about. Every search gets one too. Texts that mean the same thing get close vectors, even when they use different words, and that is how hybrid search finds results by meaning.
You do nothing for it. One multilingual model, multilingual-e5-large-instruct, makes every vector, for your documents and for the searches, on Opensolr's own servers.
What goes into the vector
- A web page: its title, its description and the structured data found on the page (JSON-LD, framework data). The rest of the page text goes in only when you turn on Include detected body in embedder for that site, in the cog of its row in Crawl URLs: Rules per site. Leave it off unless your pages carry details nothing else has, like product specifications or prices; on most sites the body text is mostly menus and footers.
- A PDF or another document: its text.
- A pushed document: the title, the description and the text you send.
What it uses from your plan
Meaning vectors use the AI of the plan of the account that owns the index. Every plan includes a monthly AI allowance of 500 AI requests; see pricing and API Quota.
- Each new vector counts as one AI request. A text that was turned into a vector before is free.
- When the allowance runs out during a crawl, the pages already in your index keep their vectors and new pages are indexed without one. During a push, the rest of the documents are indexed without vectors.
- Without AI in the plan, crawling and Data Ingestion still index everything, without vectors, and every search matches on the words.
A document without a vector is still found by its words.
Use them in your own code
Turn any text into a vector, many texts at once, or add vectors to every document already in an index: Meaning vectors by API.