find out how an LWP request was sent "over the wire"

If I send a request with LWP, there is a convenience function as_string which tells me what request I just sent. Very handy, and in truth I have no problems with it. Except that I just noticed that it is surely lying to me. For instance, this code:

use v5.14.2;
use LWP;
my $response = LWP::UserAgent->new->get('http://user:[email protected]/');
say $response->request->as_string;

Gives this output

GET http://user:[email protected]/
User-Agent: libwww-perl/6.13

But surely the URL was not sent like that! The library must have parsed out the username and password, and added the appropriate headers, and added a host header, and so on. Is there an easy way to find out what was actually sent?

Solution

There's LWP::ConsoleLogger::Everywhere¹, which you can just load to get all the details of both the request as well as the response. The request will be taken right before it's sent over the wire, and the response from when it comes back.

All you need to do is use LWP::ConsoleLogger::Everywhere anywhere in your code. If you want more control, the main module LWP::ConsoleLogger in that distribution will let you tweak settings easily.

However, this is not the real data that goes over the wire. If you want to know what it receives, you need to either monitor the connection with something like tcpdump and then take a look at it (which is quite advanced networking stuff), or maybe change the endpoint to your own IP address, or simply 127.0.0.1, and then use netcat to listen on a specific port.

$ nc -l 8080

If you send your request to that port, you'll see it in netcat.

^{1) Disclaimer: I'm a contributor for that module}