Search code examples
perlsslmojoliciouslwp-useragent

Mojo::UserAgent fails to verify certificate, where LWP::UserAgent succeeds

I have a simple perl script that uses LWP::UserAgent to connect to a secure site. It works fine. When I use Mojo::UserAgent, it fails to validate the certificate. This is reliable and repeatable. The basic Perl code is:

use strict;
use warnings;
use IO::Socket::SSL 1.980;
use LWP::UserAgent;
use Mojo::UserAgent;

$IO::Socket::SSL::DEBUG=3;

my $dst = "<DOMAIN>";
my $url = "<URL-AT-DOMAIN>";

my $A_OR_B = 1;

my $ua;

if ($A_OR_B) {
    $ua = Mojo::UserAgent->new();
    $ua->connect_timeout(20);
} else {
    $ua = LWP::UserAgent->new();
}

my $resp = $ua->get($url);

if ($A_OR_B) {
    print $resp->result->message;
    print $resp;
} else {
    print $resp->status_line."\n";
}

The output from the IO::Socket debugging is:

For the Mojo (failure):

DEBUG: .../IO/Socket/SSL.pm:3010: new ctx 48892560
DEBUG: .../IO/Socket/SSL.pm:1638: don't start handshake: IO::Socket::SSL=GLOB(0x2e957d8)
DEBUG: .../IO/Socket/SSL.pm:787: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:829: using SNI with hostname <DOMAIN>
DEBUG: .../IO/Socket/SSL.pm:864: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2911: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2864: ok=0 [3] /O=Digital Signature Trust Co./CN=DST Root CA X3/O=Digital Signature Trust Co./CN=DST Root CA X3
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:900: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:900: local error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:903: fatal SSL error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:3059: free ctx 48892560 open=
DEBUG: .../IO/Socket/SSL.pm:3063: free ctx 48892560 callback
DEBUG: .../IO/Socket/SSL.pm:3070: OK free ctx 48892560
SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
 at /home/briefly/bad.pl line 26.

and the output for the LWP version (success), is:

DEBUG: .../IO/Socket/SSL.pm:3010: new ctx 41136976
DEBUG: .../IO/Socket/SSL.pm:762: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:764: socket connected
DEBUG: .../IO/Socket/SSL.pm:787: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:829: using SNI with hostname <DOMAIN>
DEBUG: .../IO/Socket/SSL.pm:864: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:880: set socket to non-blocking to enforce timeout=180
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:917: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:937: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2911: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:917: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:937: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2864: ok=1 [2] /C=US/O=Internet Security Research Group/CN=ISRG Root X1/C=US/O=Internet Security Research Group/CN=ISRG Root X1
DEBUG: .../IO/Socket/SSL.pm:2864: ok=1 [1] /C=US/O=Internet Security Research Group/CN=ISRG Root X1/C=US/O=Let's Encrypt/CN=R3
DEBUG: .../IO/Socket/SSL.pm:2864: ok=1 [0] /C=US/O=Let's Encrypt/CN=R3/CN=tls.automattic.com
DEBUG: .../IO/Socket/SSL.pm:1840: scheme=www cert=41975232
DEBUG: .../IO/Socket/SSL.pm:1850: identity=< **VERY LONG LIST OF DOMAINS** >
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:907: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:917: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:937: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:894: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:897: done Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:952: ssl handshake done
DEBUG: .../IO/Socket/SSL.pm:3059: free ctx 41136976 open=
DEBUG: .../IO/Socket/SSL.pm:3063: free ctx 41136976 callback
DEBUG: .../IO/Socket/SSL.pm:3070: OK free ctx 41136976
200 OK

Does anyone have any insights?

Solution

  • I would suggest that LWP:UserAgent and Mojo::UserAgent use different trust stores. LWP::UserAgent will default to using Mozilla::CA while Mojo::UserAgent not. Try to enforce the use of Mozilla::CA with Mojo::UserAgent with

    $ua->ca(Mozilla::CA::SSL_ca_file());