I'm starting to send log file data to a remote server with rsyslog.
I have a development (server1) and production (server2). Both systems use the same version of CentOS and rsyslog. The rsyslog.conf and httpd config files are identical (save the server names above).
The problem is that one of the systems is adding it's own IP to rsyslog stream, the other is not. In the log data below, my local IP is 1.2.3.4, and the server IP's are in the 99.99.99.X subnet.
server1 - CentOS 6.5 (dev, 99.99.99.77)
Name : rsyslog
Arch : x86_64
Version : 5.8.10
Release : 10.el6_6
httpd conf:
ErrorLog "| tee -a /var/log/httpd/error_log | logger -thttpd_server1_ssl_error -plocal6.err"
CustomLog "|tee -a /var/log/httpd/access_log | logger -thttpd_server1_ssl_access -plocal5.notice" combined
rsyslog output: Note that only the remote IP is included...
Msg: Jul 6 11:57:24 server1 httpd_access: 1.2.3.4 - - [06/Jul/2015:11:57:23 -0400] "GET /somepage.html HTTP/1.1" 200 - "https://server1.top.level.domain/" "text/html" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0"
server2 - CentOS 6.5 (prod, 99.99.99.99)
Name : rsyslog
Arch : x86_64
Version : 5.8.10
Release : 10.el6_6
httpd conf:
ErrorLog "| tee -a /var/log/httpd/error_log | logger -thttpd_server2_ssl_error -plocal6.err"
CustomLog "|tee -a /var/log/httpd/access_log | logger -thttpd_server2_ssl_access -plocal5.notice" combined
rsyslog output: Note that both remote and server IP are included...
Msg: Jul 6 11:58:49 server2 httpd_access: 1.2.3.4 99.99.99.99 - - [06/Jul/2015:11:58:49 -0400] "GET https://server1.top.level.domain/ HTTP/1.1" 200 443 20
The problem was a difference in LogFormat
in the httpd.conf
files between the two servers.
On server2, I changed
LogFormat "%h %A %l %u %t \"%r\" %>s %p %b" combined
to
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{Content-Type}o\" \"%{User-Agent}i\"" combined
which matched what was on server1.
I then restarted apache, and now have identical rsyslog data streaming from both servers (meaning I am just seeing the Remote user IP not the remote and server IP in the outbound log data).