Solr imports CSV natively through the /update/csv handler. It is the fastest way to bulk-load an index without writing any code.
Uploading CSV data to an Opensolr index, start to finish.
01 · Importing with curl
curl "https://YOUR_SOLR_HOST/solr/YOUR_INDEX/update/csv?commit=true" \ --data-binary @your_data.csv \ -H "Content-Type: application/csv"
02 · The parameters worth knowing
| Parameter | What it does |
|---|---|
commit=true | Commits immediately, so the rows are searchable when the request returns. |
separator=%09 | Tab separated instead of comma, for TSV files. |
skip=unwanted_column | Ignores a column that exists in the file but not in your schema. |
fieldnames=id,title,description | Overrides the header row when the column names do not match your field names. |
overwrite=true | Updates the existing document when the unique key already exists. |
03 · When your file has no id column
Solr can generate the unique key for you. Add this update processor chain to solrconfig.xml, just below one of the </searchComponent> closing tags:
<updateRequestProcessorChain> <processor class="solr.UUIDUpdateProcessorFactory"> <str name="fieldName">id</str> </processor> <processor class="solr.LogUpdateProcessorFactory" /> <processor class="solr.RunUpdateProcessorFactory" /> </updateRequestProcessorChain>
Every imported row then gets a UUID automatically.
04 · Preparing the file
The first row must be the header, and the column names must match the field names in schema.xml.
Save it as UTF-8, or non-ASCII characters will arrive damaged.
Quote any field that contains a comma, with double quotes.
Strip the byte order mark that some editors add at the start of the file.
05 · When it goes wrong
Unknown field
A column in your header is not defined in schema.xml. Add the field, or skip the column with skip=.
Document is missing mandatory uniqueKey field: id
The file has no id column. Add one, or configure the UUID processor above.
The rows imported but nothing is searchable
The commit is missing. Add commit=true, or send a commit of your own afterwards.