Twitter endpoint request gets HTML response in Node app deployed to Heroku

I made a small script querying a Twitter public endpoint to ask for a username's availability.

A cURL request to this endpoint throws a JSON response formatted like this:

$ curl 'https://twitter.com/users/username_available?username=jack'

{
  "valid":false,
  "reason":"",
  "msg":"Nombre de usuario no disponible",
  "desc":"Nombre de usuario no disponible"
}

Based on this, I wrote a small NodeJS app which makes this same request (using Request package).

Both CURL and my app work fine when run locally.

But in Heroku both CURL and my app get a full HTML response (with Twitter's mobile version) instead of JSON, so my app can't parse and evaluate.

Anyone had similar issues, and know how to get it working?

Thanks!

Edit (extra info): Script is running in a web dyno in a free plan. Can't figure out how to switch to a worker dyno (if that's even still a thing)

Edit 2 (update): Now it's running on a worker Dyno, but the issue persists. Still getting HTML responses.

Solution

The endpoint you are calling is not a documented or supported Twitter API endpoint, so there should be no expectation of support, or guarantee that it will work. If you try to hit such an endpoint regularly from an unknown IP address then you should expect the IP to be blocked by Twitter's infrastructure as a protective measure.