I have a very strange problem.
We implemented a Soap API to talk to a third party. The API sort of works. There is an IsAlive method which checks if the third party service is alive.
Our application is hosted on an Ubuntu 14.04 server under Apache with Mod-Mono (Mono).
We call this API when we load a particular page. It works up to a point then we get an error. I.E. You can make a request to the IsAlive xxx (It's random) amount of times and it works, then next time you try and do it the following exception gets thrown:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.Net.WebException: Error writing headers ---> System.IO.IOException: Error while sending TLS Alert (Fatal:InternalError): System.IO.IOException: The authentication or decryption has failed. ---> Mono.Security.Protocol.Tls.TlsException: buffer underrun
at Mono.Security.Protocol.Tls.RecordProtocol.ReadRecordBuffer (Int32 contentType, System.IO.Stream record) [0x00000] in <filename unknown>:0
at Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (IAsyncResult result) [0x00000] in <filename unknown>:0
at Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0 ---> System.IO.IOException: The authentication or decryption has failed. ---> Mono.Security.Protocol.Tls.TlsException: buffer underrun
at Mono.Security.Protocol.Tls.RecordProtocol.ReadRecordBuffer (Int32 contentType, System.IO.Stream record) [0x00000] in <filename unknown>:0
at Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (IAsyncResult result) [0x00000] in <filename unknown>:0
at Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Net.WebConnection.EndWrite (System.Net.HttpWebRequest request, Boolean throwOnError, IAsyncResult result) [0x00000] in <filename unknown>:0
at System.Net.WebConnectionStream+<SetHeadersAsync>c__AnonStorey1.<>m__0 (IAsyncResult r) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
at System.Net.HttpWebRequest.GetResponse () [0x00000] in <filename unknown>:0
at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse (System.Net.WebRequest request) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Web.UI.Page.ProcessException (System.Exception e) [0x00000] in <filename unknown>:0
at System.Web.UI.Page.ProcessRequest (System.Web.HttpContext context) [0x00000] in <filename unknown>:0
at ASP.somepage_aspx.ProcessRequest (System.Web.HttpContext context) [0x00000] in <filename unknown>:0
at System.Web.HttpApplication+<Pipeline>c__Iterator1.MoveNext () [0x00000] in <filename unknown>:0
at System.Web.HttpApplication.Tick () [0x00000] in <filename unknown>:0
Mono.Security.Protocol.Tls.TlsException: buffer underrun
at Mono.Security.Protocol.Tls.RecordProtocol.ReadRecordBuffer (Int32 contentType, System.IO.Stream record) [0x00000] in <filename unknown>:0
at Mono.Security.Protocol.Tls.RecordProtocol.InternalReceiveRecordCallback (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
The API at this point does not work anymore. Even if we refresh the page and make another call. We try to re-request 2-3 times then we get another error.
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.Net.WebException: The request timed out
at System.Net.HttpWebRequest.GetRequestStream () [0x00000] in <filename unknown>:0
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke (System.String method_name, System.Object[] parameters) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Web.UI.Page.ProcessException (System.Exception e) [0x00000] in <filename unknown>:0
at System.Web.UI.Page.ProcessRequest (System.Web.HttpContext context) [0x00000] in <filename unknown>:0
at ASP.somepage_aspx.ProcessRequest (System.Web.HttpContext context) [0x00000] in <filename unknown>:0
at System.Web.HttpApplication+<Pipeline>c__Iterator1.MoveNext () [0x00000] in <filename unknown>:0
at System.Web.HttpApplication.Tick () [0x00000] in <filename unknown>:0
System.Net.WebException: The request timed out
at System.Net.HttpWebRequest.GetRequestStream () [0x00000] in <filename unknown>:0
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke (System.String method_name, System.Object[] parameters) [0x00000] in <filename unknown>:0
The only way to make the API work again is if we restart the Ubuntu Server.
Here is a snippet of how we send the request.
public string TestSiteStatus()
{
getSiteStatus site = new getSiteStatus();
site.site = "XXX";
string res = "";
using (SiteServiceImplService service = new SiteServiceImplService())
{
res = service.getSiteStatus(site);
}
return res;
}
Does anyone know what is wrong here? Is there a problem with my implementation or is this a Mono framework issue?
I have tried the instructions on this page: http://www.mono-project.com/docs/faq/security/ but no success. When I run thetest, it passes the first and sometimes the second time but fails after that.
Turns out there was a certificate issue with our third party. They updated their certificates, we added them to our stores, and it worked.
Our the API SSL cert was vulnerable to a LogJam attack. The ciphering of the certificate caused problems with our server.
This link helped us test and find the problem.