I am able to successfully log into a website using Firefox but having trouble replicating that log in using node-fetch. There are three stages to the log-in process:
in Firefox Dev Tools, I can see both the working headers and working request body. When I click on "Request" in Dev Tools, I see a 'Form data' heading with the proper values of 'email' and 'password'. When I click on "Headers" in Dev Tools, here is the successful Firefox request:
POST /login/action HTTP/2
Host: www.website.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded
Content-Length: 46
Origin: https://www.website.com
Connection: keep-alive
Referer: https://www.website.com/login
Cookie: _sessiontoken=sessionTokenHere
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
TE: trailers
On Firefox, that returns a 302 (redirect) response with these headers:
HTTP/2 302 Found
date: Sun, 31 Oct 2021 17:25:03 GMT
content-type: text/html; charset=utf-8
cache-control: no-cache
location: https://www.website.com/portal
x-runtime: 9
set-cookie: auth_token=authtokenHere; path=/
set-cookie:
set-cookie: _sessiontoken=sessionTokenHere; path=/; HttpOnly
cf-cache-status: DYNAMIC
[omitted for brevity: expect-ct, report-to, nel, server, cf-ray, alt-svc, X-Firefox-Spdy]
Now, here are the POST options I am trying to use with node-fetch in node.js:
{
method: 'POST', headers: {
Host: 'www.website.com'
,'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0'
,Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
'Accept-Encoding': 'gzip, deflate, br',
'Content-Length': 46,
Origin: 'https://www.website.com',
Connection: 'keep-alive',
Referer: 'https://www.website.com/login',
Cookie: '_sessiontoken=sessionTokenHere'
'Upgrade-Insecure-Requests': '1',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-User': '?1',
TE: 'trailers',
body: URLSearchParams { 'email' => 'myEmail','password' => 'myPassword'}
However, when using node-fetch with the above options it returns a 200, not a 302. Here are the headers of that response (printed via console.log() within the fetch() function):
[Object: null prototype] {
date: [ 'Sun, 31 Oct 2021 18:12:01 GMT' ],
'content-type': [ 'text/html; charset=utf-8' ],
'transfer-encoding': [ 'chunked' ],
connection: [ 'keep-alive' ],
vary: [ 'Accept-Encoding' ],
'x-runtime': [ '5' ],
'cache-control': [ 'private, max-age=0, must-revalidate' ],
'set-cookie': [
'_sessionToken=sessionTokenHere; path=/; HttpOnly'
],
'cf-cache-status': [ 'DYNAMIC' ],
'content-encoding': [ 'br' ],
[omitted for brevity: expect-ct, report-to, nel, server, cf-ray, alt-svc]
}
I am puzzled why it works in Firefox but not with node-fetch. A few notes:
node-fetch automatically follows redirects. It will automatically go to the next redirected url (Location
http header value).
If you want to catch the 301 or 302 call, you can set redirect
property to manual
in the options like redirect: "manual"
, this way you manage the redirection yourself
For example the following code will catch the 301
call (it'll do the same for your 302
in theory, implemented here):
const fetch = require("node-fetch");
(async () => {
const response = await fetch("https://stackoverflow.com/users/2614364", {
redirect: "manual",
});
console.log(response.status);
console.log(response.headers.get("set-cookie"));
})();
I suppose you want to get the cookies using response.headers.get("set-cookie")