I am facing a very strange issue with Kamailio, our Azure VM is D2 configuration with 2 cores and 7 GB of RAM, Ubuntu 14.04 LTS, through-out the day we see high CPU usage from Kamailio, and intermittently we face issue that Kamailio stops accepting new registrations. When I check the location table, or via kamctl ul show, it shows only around 900+ users. But still Kamailio could not take new registrations with that small number of user.I believe Kamailio can handle 350000 users per core. Also tcp_connections are at tcp_max_connection = 60000, I receive following error:
Feb 4 01:02:21 urcmcs /usr/local/sbin/kamailio[1956]: ERROR: [tcp_read.c:299]: tcp_read_data(): error reading: Connection timed out (110) Feb 4 01:02:21 urcmcs /usr/local/sbin/kamailio[1956]: ERROR: [tcp_read.c:1326]: tcp_read_req(): ERROR: tcp_read_req: error reading Feb 4 01:02:42 urcmcs /usr/local/sbin/kamailio[1969]: ERROR: [tcp_read.c:299]: tcp_read_data(): error reading: Connection timed out (110) Feb 4 01:02:42 urcmcs /usr/local/sbin/kamailio[1969]: ERROR: [tcp_read.c:1326]: tcp_read_req(): ERROR: tcp_read_req: error reading Feb 4 01:02:54 urcmcs /usr/local/sbin/kamailio[1959]: ERROR: [tcp_read.c:299]: tcp_read_data(): error reading: Connection timed out (110) Feb 4 01:02:54 urcmcs /usr/local/sbin/kamailio[1959]: ERROR: [tcp_read.c:1326]: tcp_read_req(): ERROR: tcp_read_req: error reading
Searched everywhere but cannot find a solution...
I finally found the issue, Ubuntu kernel 3.19.0-28-generic, is known to have some problems with network performance. This was causing very high usage and connection problems. Updating the kernel to the newest solved the problem.