Search code examples
solr

solr creating densevector field via curl POST request

I would like to use the new feature in solr which is the DensevectorField, but I am not able to see any examples where I can post the schema via curl to create this field type.

Normally, I would do:

curl -s -X POST -H 'Content-type:application/json' --data-binary '{"add-field": {"name":"some_field", "type":"text_general", "multiValued":false, "stored":true, "indexed":true, "termVectors":true }}' "http://localhost:8984/solr/$core_name/schema"

I would like to know what is the equivalent type for DensevectorField instead of text_general.

I see examples where they have hand edited the schema - including the ones on the doc, but I would like to create this via the curl POST request.

I am not able to find this anywhere on the docs however..

Solution

  • When creating a Solr core, we likely start with the default schema and tweak it as needed. In this default schema, several field and fieldType definitions are already defined so that we can quickly get the most out of Solr without too much hassle (see default field types in Solr 9.0.0).

    Though, all use cases are not covered (it includes only the basics, the schema would get too big otherwise). In specific case like yours, implementing neural search, you will have to define not only the field that makes use of solr.DenseVectorField, but also the fieldType that provides it. For example :

    <fieldType name="knn_vector" class="solr.DenseVectorField" vectorDimension="4" similarityFunction="cosine" knnAlgorithm="hnsw" hnswMaxConnections="10" hnswBeamWidth="40"/>
<field name="vector" type="knn_vector" indexed="true" stored="true"/>

    Now, to do that on a managed schema using the Schema API, this amounts to doing the following request :

    curl -X POST -H 'Content-type:application/json' --data-binary '{
    "add-field-type":{
        "name":"knn_vector",
        "class":"solr.DenseVectorField",
        "vectorDimension":"4",
        "similarityFunction":"cosine",
        "knnAlgorithm":"hnsw",
        "hnswMaxConnections":"10",
        "hnswBeamWidth":"40"
    },
    "add-field":{
        "name":"vector",
        "type":"knn_vector",
        "indexed":true,
        "stored":true 
    }
}' "http://localhost:8984/solr/$core_name/schema"

    See Schema API :