Configuration

Import XML Data into Your Opensolr Index

Solr's DataImportHandler (DIH) was removed in Solr 9. The modern approach is simple: POST your XML data directly to Solr's /update endpoint via HTTP. No plugins needed, no data-config.xml, no special configuration — just send XML over HTTP.

This guide shows you how to parse XML files and index them into your Opensolr Index using curl, PHP, and Python.

The Solr XML Format

Solr accepts documents in a specific XML format. Every XML file you send must follow this structure:

<?xml version="1.0" encoding="UTF-8"?>
<add>
  <doc>
    <field name="id">product-001</field>
    <field name="title">Wireless Headphones</field>
    <field name="description">Premium noise-cancelling wireless headphones</field>
    <field name="price">149.99</field>
    <field name="category">Electronics</field>
    <field name="in_stock">true</field>
  </doc>
  <doc>
    <field name="id">product-002</field>
    <field name="title">Bluetooth Speaker</field>
    <field name="description">Portable waterproof Bluetooth speaker</field>
    <field name="price">79.99</field>
    <field name="category">Electronics</field>
    <field name="in_stock">true</field>
  </doc>
</add>

Key rules:

• The root element is <add>
• Each document is wrapped in <doc>
• Each field uses <field name="fieldname">value</field>
• Field names must match fields defined in your schema.xml
• The id field is required (unique key for each document)
• You can include as many <doc> elements as you want in a single <add> block

Example 1: curl — Quick Import from Command Line

The simplest way to import. Post the XML file directly to Solr:

#!/bin/bash
# =====================================================================
# CONFIGURATION — Replace these with YOUR Opensolr index details
# =====================================================================
SOLR_HOST="YOUR_HOST"
INDEX_NAME="YOUR_INDEX"
USERNAME="opensolr"
PASSWORD="YOUR_API_KEY"

XML_FILE="products.xml"

# Import the XML file into Solr
echo "Importing $XML_FILE into $INDEX_NAME..."

curl -u "$USERNAME:$PASSWORD" \
  -H "Content-Type: text/xml" \
  --data-binary @"$XML_FILE" \
  "https://$SOLR_HOST/solr/$INDEX_NAME/update?commit=true&wt=json"

echo ""
echo "Done!"

For large files, you can split and send in chunks — see the PHP and Python examples below.

Example 2: PHP — Parse & Index XML with Chunking

This script reads an XML file, parses each <doc> element, and sends them to Solr in configurable batches. Handles large files without running out of memory.

<?php
// =====================================================================
// CONFIGURATION — Replace these with YOUR Opensolr index details
// =====================================================================
$solr_host  = "YOUR_HOST";
$index_name = "YOUR_INDEX";
$username   = "opensolr";
$password   = "YOUR_API_KEY";
$xml_file   = "products.xml";
$batch_size = 500;  // Documents per batch (adjust based on doc size)

$solr_url = "https://$solr_host/solr/$index_name/update?commit=true&wt=json";

// Load and parse the XML file
$xml = simplexml_load_file($xml_file);
if ($xml === false) {
    die("Error: Could not parse $xml_file
");
}

$docs = $xml->doc;
$total = count($docs);
echo "Found $total documents in $xml_file
";

// Send documents in batches
$batch = [];
$sent = 0;

foreach ($docs as $doc) {
    $batch[] = $doc->asXML();

    if (count($batch) >= $batch_size) {
        $sent += send_batch($batch, $solr_url, $username, $password);
        $batch = [];
        echo "  Indexed $sent / $total
";
    }
}

// Send remaining documents
if (!empty($batch)) {
    $sent += send_batch($batch, $solr_url, $username, $password);
    echo "  Indexed $sent / $total
";
}

echo "Done! $sent documents indexed.
";

function send_batch($docs, $url, $user, $pass) {
    $xml_body = "<add>
" . implode("
", $docs) . "
</add>";

    $ch = curl_init($url);
    curl_setopt_array($ch, [
        CURLOPT_USERPWD        => "$user:$pass",
        CURLOPT_POST           => true,
        CURLOPT_POSTFIELDS     => $xml_body,
        CURLOPT_HTTPHEADER     => ["Content-Type: text/xml"],
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_TIMEOUT        => 120,
    ]);

    $response = curl_exec($ch);
    $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);

    if ($http_code !== 200) {
        echo "  ERROR (HTTP $http_code): $response
";
        return 0;
    }

    return count($docs);
}

Example 3: Python — Parse & Index XML with Chunking

Same approach in Python. Uses xml.etree.ElementTree for parsing and requests for HTTP.

# import_xml.py — Parse and index XML documents into Opensolr
import xml.etree.ElementTree as ET
import requests
import sys

# =====================================================================
# CONFIGURATION — Replace these with YOUR Opensolr index details
# =====================================================================
SOLR_HOST  = "YOUR_HOST"
INDEX_NAME = "YOUR_INDEX"
USERNAME   = "opensolr"
PASSWORD   = "YOUR_API_KEY"
XML_FILE   = "products.xml"
BATCH_SIZE = 500  # Documents per batch

SOLR_URL = f"https://{SOLR_HOST}/solr/{INDEX_NAME}/update?commit=true&wt=json"

def send_batch(docs, url, auth):
    """Send a batch of <doc> elements to Solr."""
    xml_body = "<add>
" + "
".join(docs) + "
</add>"
    resp = requests.post(
        url,
        data=xml_body.encode("utf-8"),
        headers={"Content-Type": "text/xml"},
        auth=auth,
        timeout=120,
    )
    if resp.status_code != 200:
        print(f"  ERROR (HTTP {resp.status_code}): {resp.text}")
        return 0
    return len(docs)

def main():
    xml_file = sys.argv[1] if len(sys.argv) > 1 else XML_FILE
    auth = (USERNAME, PASSWORD)

    # Parse the XML file
    tree = ET.parse(xml_file)
    root = tree.getroot()

    # Collect all <doc> elements
    docs = []
    for doc in root.findall("doc"):
        docs.append(ET.tostring(doc, encoding="unicode"))

    total = len(docs)
    print(f"Found {total} documents in {xml_file}")

    # Send in batches
    sent = 0
    for i in range(0, total, BATCH_SIZE):
        batch = docs[i : i + BATCH_SIZE]
        sent += send_batch(batch, SOLR_URL, auth)
        print(f"  Indexed {sent} / {total}")

    print(f"Done! {sent} documents indexed.")

if __name__ == "__main__":
    main()

Example 4: Python — Convert Non-Solr XML to Solr Format

If your XML is not in Solr's <add><doc> format (e.g., it's a product feed, RSS, or custom XML), you need to convert it first:

# convert_and_index.py — Convert arbitrary XML to Solr format and index
import xml.etree.ElementTree as ET
import requests

SOLR_HOST  = "YOUR_HOST"
INDEX_NAME = "YOUR_INDEX"
USERNAME   = "opensolr"
PASSWORD   = "YOUR_API_KEY"
SOLR_URL   = f"https://{SOLR_HOST}/solr/{INDEX_NAME}/update?commit=true&wt=json"

# Example: convert a product catalog XML like:
# <catalog>
#   <product sku="ABC123">
#     <name>Widget</name>
#     <price>19.99</price>
#   </product>
# </catalog>

tree = ET.parse("catalog.xml")
root = tree.getroot()

solr_docs = []
for product in root.findall("product"):
    sku = product.get("sku", "")
    name = product.findtext("name", "")
    price = product.findtext("price", "0")
    category = product.findtext("category", "")
    desc = product.findtext("description", "")

    doc = f"""<doc>
  <field name="id">{sku}</field>
  <field name="title">{name}</field>
  <field name="price">{price}</field>
  <field name="category">{category}</field>
  <field name="description">{desc}</field>
</doc>"""
    solr_docs.append(doc)

print(f"Converted {len(solr_docs)} products to Solr format")

# Send to Solr in one batch (or chunk for large datasets)
xml_body = "<add>
" + "
".join(solr_docs) + "
</add>"
resp = requests.post(
    SOLR_URL,
    data=xml_body.encode("utf-8"),
    headers={"Content-Type": "text/xml"},
    auth=(USERNAME, PASSWORD),
    timeout=120,
)

if resp.status_code == 200:
    print("Indexed successfully!")
else:
    print(f"Error: {resp.status_code} — {resp.text}")

Handling Large XML Files

For files with thousands or millions of documents, keep these in mind:

Quick Reference

The same /update endpoint also takes JSON (Content-Type: application/json), so you can skip XML entirely.

eDisMax is the query parser most production Solr deployments run on. It takes the words a user typed, spreads them across the fields you nominate, decides how many of those words a document has to contain, and rewards documents where the words appear together. This guide covers every parameter that does that work — qf, mm, pf, pf2, pf3, ps, ps2, ps3 — and shows the exact query Solr builds for each one.

Every parsed query printed on this page is a real capture from a live Opensolr index, taken with debugQuery=true. None of it is illustrative.

01 · What eDisMax actually does

DisMax stands for disjunction maximum: each term is run against several fields at once and the field that scores best is the one that counts. eDisMax is the extended version. It keeps that behaviour and adds full Lucene syntax, shingle-based phrase boosting, function boosting and a set of controls that make relevance tuning a configuration job rather than a string-building job.

The parameters, one page each

The same parameters, on Opensolr

Everything above is standard Solr and works the same on any installation. On Opensolr you edit schema.xml and solrconfig.xml in the control panel (edit a file), and saving reloads the index.

On an index with site search, the hosted search page builds qf, pf, pf2 and pf3 from one set of field weights, and passes Minimum Match through as literal mm syntax. You set both in the Search Tuning panel, read at query time, with no reload: search tuning, field weights, minimum match.

Solr edgy_text and edgeNgram Explanation

1. edgy_text Field Type:

The edgy_text field type typically uses Edge N-gram tokenization to create prefix-based tokens for indexing. This means the field is indexed in a way that allows matching partial terms or phrases, improving search results, especially for autocomplete or search-as-you-type functionality.

What is Edge N-gram Tokenization?

• Edge N-grams are tokens that represent progressively longer prefixes of a word. For example, if you tokenize the word "example", the Edge N-gram tokenizer could produce tokens like:
  • "e"
  • "ex"
  • "exa"
  • "exam"
  • "examp"
  • "exampl"
  • "example"

This is useful for autocompletion, as it allows users to search by just typing the beginning of a word (or a term) and get matches based on the indexed prefixes.

How edgy_text works in Solr:

The edgy_text field type in Solr is usually implemented using the Edge N-gram filter. This filter generates prefixes of the input text and indexes those prefixes. It works well for fields where you want to allow prefix-based matching or autocomplete-like behavior.

2. The edgeNgram Filter:

The edgeNgram filter, when used with a Solr field type, can be applied to a field to tokenize the input data (like product names, categories, or SKUs) into progressively longer prefixes of words.

For Example:

If a field like product_name contains "laptop", the edgeNgram filter might produce the following tokens:

• "l"
• "la"
• "lap"
• "lapt"
• "lapto"
• "laptop"

This allows a search query like "lap" to match "laptop", making it perfect for search-as-you-type or autocomplete features.

Example of Edge Ngram in Solr:

Here's how you can configure both edgy_text and edgeNgram in Solr:

1. edgy_text FieldType Definition:

This edgy_text field type uses the Edge N-gram filter to break words into progressively longer prefixes.

<fieldType name="edgy_text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <!-- Whitespace tokenizer to split the text into tokens based on spaces -->
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <!-- Edge N-gram filter to create prefixes from each word -->
    <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <!-- Similar tokenization for query analysis -->
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

• minGramSize="1": Specifies that the smallest prefix (starting point) will be of length 1 (e.g., "l" from "laptop").
• maxGramSize="25": Limits the length of the token to a maximum of 25 characters. You can adjust this based on your needs.

2. Using edgy_text for Fields:

Once you've defined this field type, you can apply it to any field where you want to enable prefix matching. For instance, for a product name or SKU:

<field name="product_name" type="edgy_text" indexed="true" stored="true"/>
<field name="sku" type="edgy_text" indexed="true" stored="true"/>

These fields now support partial matching for searches, such as a query for "lap" returning "laptop".

3. Benefits:

• Prefix matching: Users can search by typing partial words, and Solr will return all results that match the prefix.
• Autocomplete: This can be used to create search-as-you-type features in your application, where suggestions appear as users type.
• Efficient search: Since Solr indexes only prefixes, it can efficiently match terms, especially with large datasets.

4. Use Case:

Consider an e-commerce site where users are typing product names in a search box. As the user types the prefix of the product name (e.g., "lap" for "laptop"), Solr can quickly return relevant results because the field is indexed with all possible prefixes of the product name. This results in faster search suggestions and matches.

Conclusion:

• edgy_text with Edge N-gram: This combination helps create an efficient, powerful search feature for prefix-based matches, such as autocompletion.
• Use Cases: Ideal for search-as-you-type functionality, improving user experience by providing real-time search suggestions or partial matches.