Search code examples
datesolrutc

Solr 'Invalid Date String' Exception


I'm new to Solr. I successfully indexed some data, but after passing a date attribute to an appropriate solr field I'm receiving an exception:

I'm passing the String "15.06.2011 13:51:41", which is a common date format for countries like Germany. However, after some googling I found out, that Solr expects a date to be in the UTC format.

My specific question is: Can I transform my format to UTC at some point at Solr? Would this be a case for the DateFormatTransformer or is it a MUST to pass it in the correct format initially?

EDIT: I am not using a DataImportHandler. I'm using the DirectUpdateHandler2 by passing the data directly to Solr. Since it's called 'direct update', are my chances pretty bad in transforming anything there?

For the records, the exception I got was:

ERROR - 2013-09-13 15:52:07.705; org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: Invalid Date String:'15.06.2011 13:51:41'
    at org.apache.solr.schema.DateField.parseMath(DateField.java:182)
    at org.apache.solr.schema.TrieField.createField(TrieField.java:616)
    at org.apache.solr.schema.TrieField.createFields(TrieField.java:655)
    at org.apache.solr.schema.TrieDateField.createFields(TrieDateField.java:157)
    at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
    at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
    at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73)
    at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:210)
    at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
    at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
    at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)

Solution

  • According to the Solr Date Formatting documentation, this format is required:

    The format used is a restricted form of the canonical representation of dateTime in the XML Schema specification – a restricted subset of ISO 8601.

    YYYY-MM-DDThh:mm:ssZ

    • YYYY is the year.
    • MM is the month.
    • DD is the day of the month.
    • hh is the hour of the day as on a 24-hour clock.
    • mm is minutes.
    • ss is seconds.
    • Z is a literal 'Z' character indicating that this string representation of the date is in UTC

    Note that no time zone can be specified; the String representations of dates is always expressed in Coordinated Universal Time (UTC). Here is an example value:

    1972-05-20T17:33:18Z

    You can optionally include fractional seconds if you wish, although any precision beyond milliseconds will be ignored.

    The only option is to transform the date prior to sending it into Solr. If you are using DataImportHandler, it would be possible within that context/process.