Search code examples
pythonpython-3.xpippypi

Is there a way to access search results from PyPI without scraping?


I'm working on a GUI for managing Python virtual environments. So far I was able to implement the most features I wanted to provide to the user. But I'm stuck with one thing:

While creating a virtual environment the users can install packages into it if they want. For this I would like to let them perform a search like pip search <package> from the command line. The results will be displayed in a table view. The problem I have is that I'm not sure what is the best way to get the search results.

I tried using the built-in module subprocess and did a pip search to populate the table with the results. This is possible, however it's quit tricky, because I have to format the output first (package name, version, description) to fit the table.

As because for this is a lot of nested loops and string manipulation needed, I looked for a way to access the data directly, ideally without having to scrape Python Package Index.


EDIT:

I considered using PyPI's XML-RPC API, but there's a note that it's going to be deprecated in the future and is not recommended for use, so I'm unsure if I should use it in my project.

The XML-RPC API will be deprecated in the future. Use of this API is not recommended, and existing consumers of the API should migrate to the RSS and/or JSON APIs instead.

Users of this API are strongly encouraged to subscribe to the pypi-announce mailing list for notices as we begin the process of removing XML-RPC from PyPI.

Is there another way to get the search results from PyPI or is the XML-RPC API the only one at the moment?


Solution

  • The XML-RPC Search endpoint was temporarily disabled in mid-December 2020, because of ever increasing request load to the search endpoint. As of now, it is not currently possible to search packages on pypi.org with an API at all.