Search code examples
solrelasticsearchpysolr

Moving documents from SOLR to elasticsearch

I'm working on moving all documents in SOLR to elasticsearch using Python code with Pysolr.

With pysolr i'm able to access only 499 documents, it there a way to access the entire set of documents.

Solution

  • The documents could be accessed in sets of 499 using the offset and size parameter in pysolr API, i found this after checking the Pysolr api more in detail. The sample code is attached below :

    limit = 499
offset = 0
while True:
    kwargs = {'sort':'id DESC'}
    response = source.search(q='projectid:774 AND source:FACEBOOK', start=offset, rows=limit, **kwargs  )
    print(len(response.docs))

    offset = offset + limit

    if len(response.docs) == 0:
        break

    # Do neccessary Operations
    pass