Search code examples
loggingnginxmatomo

Setup Nginx logs to directly imports in Piwik


I have Piwik installed and working fine, but I want to configure Nginx to directly import logs as they are created, thus preventing to parse logs files.

I know this can be done in Apache using the following:

# Log to piwik as well
CustomLog "|/path/to/import_logs.py --option1 --option2 ... -" myLogFormat

but I couldn't find anything like this on Nginx.

Has anyone knows how to do this?


Solution

  • This isn't possible in Nginx, see this post in the mailing list.

    No, nginx will not support logging to a pipe since this is simlpy waste of CPU. You may try tail -F /path/to/log | app This is almost the same waste of CPU as piped log.

    If you want to know why it it's slow:

    Logging to pipe is a CPU waste because it causes a lot of context switches 
    and memory copies for every log operation: 
    
    1) nginx writes to a pipe, 
    2) context switch to script, 
    2) script reads from the pipe, 
    3) script processes line, 
    4) script writes to a database, 
    5) context switch to nginx. 
    
    instead of single memory copy operation to a log file. 
    

    You'll have to create another script that's running and tailing the log files, then piping the data into Piwik.