Search code examples
proxyhttp-proxysocksscopuspybliometrics

Using a proxy that requires authentication with pybliometrics

I am using pybliometrics, a Python interface to the Scopus API, to download the abstracts of some papers.

Unfortunately Scopus only works inside the network of the university that subscribed to it. I am currently at home and whenever I try to download something using pybliometrics it gives me the following error:

pybliometrics.scopus.exception.Scopus401Error: The requestor is not authorized to access the requested view or fields of the resource

I need to use my university's proxy in order to enter the internet with the IP address of my university. The proxy has a WPAD configuration file available, but I fail to realize how to use it with pybliometrics. The pybliometrics documentation says to add a block in the configuration file like this:

[Proxy]
ftp = socks5://127.0.0.1:1234
http = socks5://127.0.0.1:1234
https = socks5://127.0.0.1:1234

But this proxy requires authentication. How can I specify the proxy username and password?

EDIT: I have tried setting up the block in config.ini like:

[Proxy]
ftp = http://username:[email protected]:8080
http = http://username:[email protected]:8080
https = http://username:[email protected]:8080

but it still fails with the following error message:

requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.elsevier.com', port=443): Max retries exceeded with url: /content/abstract/scopus_id/84983158344?view=META_ABS (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))
Solution

  • The problem was that my proxy requires DigestAuth rather than BasicAuth.