Search code examples
erlangcouchdb

What is causing CouchDB changes_reader_died


I'm trying to configure a filtered replication bidirectionnal between two databases. Each database has a document into the _replicator database that set the replication towards the database. Each database have the same _design document with the filter and they got the same filter parameters.

Although I have been looking over the web, I didn't find the cause of my problem. I hope you guys can help me.


Log output :

 [Fri, 26 Aug 2016 19:36:31 GMT] [error] [<0.22247.80>] ** Generic server <0.22247.80> terminating 
** Last message in was {'EXIT',<0.22219.80>,changes_reader_died}
** When Server state == {state,<0.22219.80>,<0.22249.80>,20,
                            {httpdb,
                                "REPLACEDFORSECURITYREASONS",
                                nil,
                                [{"Accept","application/json"},
                                 {"User-Agent","CouchDB/1.6.1"}],
                                30000,
                                [{socket_options,
                                     [{keepalive,true},{nodelay,false}]}],
                                10,250,<0.22065.80>,20},
                            {httpdb,
                                "http:REPLACEDFORSECURITYREASONS",
                                nil,
                                [{"Accept","application/json"},
                                 {"User-Agent","CouchDB/1.6.1"}],
                                30000,
                                [{socket_options,
                                     [{keepalive,true},{nodelay,false}]}],
                                10,250,<0.22223.80>,20},
                            [],nil,nil,nil,
                            {rep_stats,0,0,0,0,0},
                            nil,nil,
                            {batch,[],0}}
** Reason for termination == 
** changes_reader_died

[Fri, 26 Aug 2016 19:36:31 GMT] [error] [<0.22243.80>] {error_report,<0.34.0>,
                        {<0.22243.80>,crash_report,
                         [[{initial_call,
                            {couch_replicator_worker,init,['Argument__1']}},
                           {pid,<0.22243.80>},
                           {registered_name,[]},
                           {error_info,
                            {exit,changes_reader_died,
                             [{gen_server,terminate,6,
                               [{file,"gen_server.erl"},{line,744}]},
                              {proc_lib,init_p_do_apply,3,
                               [{file,"proc_lib.erl"},{line,239}]}]}},
                           {ancestors,
                            [<0.22219.80>,couch_replicator_job_sup,
                             couch_primary_services,couch_server_sup,
                             <0.35.0>]},
                           {messages,[]},
                           {links,[<0.22244.80>]},
                           {dictionary,
                            [{last_stats_report,{1472,240191,741233}}]},
                           {trap_exit,true},
                           {status,running},
                           {heap_size,233},
                           {stack_size,27},
                           {reductions,158}],
                          [{neighbour,
                            [{pid,<0.22244.80>},
                             {registered_name,[]},
                             {initial_call,{erlang,apply,2}},
                             {current_function,
                              {couch_replicator_worker,queue_fetch_loop,5}},
                             {ancestors,[]},
                             {messages,[]},
                             {links,[<0.22243.80>]},
                             {dictionary,[]},
                             {trap_exit,false},
                             {status,waiting},
                             {heap_size,610},
                             {stack_size,10},
                             {reductions,4}]}]]}}
[Fri, 26 Aug 2016 19:36:31 GMT] [error] [<0.22247.80>] {error_report,<0.34.0>,
                        {<0.22247.80>,crash_report,
                         [[{initial_call,
                            {couch_replicator_worker,init,['Argument__1']}},
                           {pid,<0.22247.80>},
                           {registered_name,[]},
                           {error_info,
                            {exit,changes_reader_died,
                             [{gen_server,terminate,6,
                               [{file,"gen_server.erl"},{line,744}]},
                              {proc_lib,init_p_do_apply,3,
                               [{file,"proc_lib.erl"},{line,239}]}]}},
                           {ancestors,
                            [<0.22219.80>,couch_replicator_job_sup,
                             couch_primary_services,couch_server_sup,
                             <0.35.0>]},
                           {messages,[]},
                           {links,[<0.22249.80>]},
                           {dictionary,
                            [{last_stats_report,{1472,240191,741235}}]},
                           {trap_exit,true},
                           {status,running},
                           {heap_size,233},
                           {stack_size,27},
                           {reductions,162}],
                          [{neighbour,
                            [{pid,<0.22249.80>},
                             {registered_name,[]},
                             {initial_call,{erlang,apply,2}},
                             {current_function,
                              {couch_replicator_worker,queue_fetch_loop,5}},
                             {ancestors,[]},
                             {messages,[]},
                             {links,[<0.22247.80>]},
                             {dictionary,[]},
                             {trap_exit,false},
                             {status,waiting},
                             {heap_size,610},
                             {stack_size,10},
                             {reductions,4}]}]]}}

Best regards.


Solution

  • While doing a deep analyze of the log, I found that there was a timeout error before the changes_read_died.

    The error was pretty explicit :

    Fri, 26 Aug 2016 20:01:06 GMT] [info] [<0.2238.0>] Replication `"c48f6c26aa44689de43ee5ffaa18c7ad+continuous"` is using:
        4 worker processes
        a worker batch size of 500
        20 HTTP connections
        a connection timeout of 30000 milliseconds
        10 retries per request
        socket options are: [{keepalive,true},{nodelay,false}]
    [Fri, 26 Aug 2016 20:01:06 GMT] [info] [<0.2204.0>] XXX.XXX.X.XX- - GET /akpaper/_changes?filter=global%2FbySite&IDSITE=MILLS2&feed=continuous&style=all_docs&since=0&heartbeat=10000 200
    [Fri, 26 Aug 2016 20:01:06 GMT] [error] [<0.2204.0>] OS Process Error <0.204.0> :: {<<"compilation_error">>,
                                                        <<"Expression does not eval to a function. (ffunction(doc, req) {    return doc._deleted || doc.IDSITE == req.query.IDSITE;})">>}
    [Fri, 26 Aug 2016 20:01:06 GMT] [info] [<0.2204.0>] XXX.XXX.X.XX - - GET /akpaper/_changes?filter=global%2FbySite&IDSITE=MILLS2&feed=continuous&style=all_docs&since=0&heartbeat=10000 500
    [Fri, 26 Aug 2016 20:01:06 GMT] [error] [<0.2204.0>] httpd 500 error response:
     {"error":"compilation_error","reason":"Expression does not eval to a function. (ffunction(doc, req) {    return doc._deleted || doc.IDSITE == req.query.IDSITE;})"}
    

    All I did to correct the problem was just to correct my filter function wich could not be eval.