Search code examples
qtdebuggingopenssl

qt openssl issues, how to further debug?


I am getting weird SSL error message on my deployed application, not on every message but definitely too often. The message in Qt reads:

Error while reading: error:1C80006E:Provider routines::invalid tag

I located the responsible lines as here and here

My question would be: Is there a way to enable advanced debug options in qt and openSSL, so that I can see the messages exchanged and/or SSL internal errors?

Some additional information on what I already tried:

  • Communication without SSL, that works. But us unsafe obviously
  • Different Qt Versions, from 5.11 up to the latest 6.x.
  • Different Server software, responsible for the encryption, from AWS Classic Load Balancer to Nginx
  • Different physical connections, over the network or loopback
  • Different ciphers, even some that should not even have the tag that fails.
  • Different Operating systems in the client, from Ubuntu 16 to Debian 12. It worked in Ubuntu 16, on Debian 12 it does does not. OpenSSL changed from UB16 to DEB12 (1.0.x vs 3.x). I think that's narrows down the issue, but I cannot simply downgrade forever, I have to find the exact problem.

Update: Turned out that inside ossl_gcm_get_ctx_params (code) the AEAD tag length is 0, and therefore it fails. It failed with the same error for four different cases, so it was hard to understand... Found that by using LD_PRELOAD and recompiling openSSL... Do not yet understand what that means but feels like a progress :D


Solution

  • Solved: After way too long days of debugging... another use of OpenSSL within the application was influencing Qt's OpenSSL usage. The mentioned error came from that usage, not Qt's.