I have a proxy pypi repository which can only be updated by a specific user but accessed by all, I have a private pypi repo and a virtual repo to access them.
My problem is that after a while, any pip install on the virtual repo will fail, finding no versions of the module. I tried to reach to it directly via https//myartifactory/proxy_name-cache/.pypi/module_name and I get a 404.
https://jfrog.com/knowledge-base/how-to-fix-404-error-resource-has-expired/
I learned from here about the expiracy policy, and indeed, if I remove the -cache from the previous URL or if I use my virtual repository instead (without being logged in artifactory), the module is updated and the pip install starts to work.
Why doesn't the pip install command triggers the same mechanism that updates the module and make it downloadable and how can I fix this ?
@EDIT
I'll try to explain better I've populated the cache from pypi yesterday. Today the artifacts are marked as expired (I can see that by doing a request at https//myartifactory/proxy_name-cache/.pypi/module_name.html which returns a 404)
Therefore, doing a pip install module_name doesn't work (Could not find a version that satisfies the requirement ...).
That's my problem
The fact is that if I do the same request that returned me the 404 error, but on proxy_name or virt_name instead of proxy_name-cache, then artifactory updates the artifact with its upstream version and then, it starts to work (until it's marked as expired again)
This behaviour makes me wonder why doing a pip install instead of a simple request (like a curl or with a browser) doesn't update the artifact with its upstream version too
To illustrate my problem :
pip install -i https://myartifactory.net/api/pypi/virt/simple requests # Could not find a version...
curl https://myartifactory.net/pypi-cache/.pypi/requests.html # 404, resource has expired
curl https://myartifactory.net/virt/.pypi/requests.html # 200, returns the html page of versions, meaning it has updated the module with the upstream version
pip install -i https://myartifactory.net/api/pypi/virt/simple requests # Is now working
Thanks for the additional information :)
So the issue here is that Artifactory is marking the metadata/index files (on remote repositories) as expirable, that in order to make sure that the metadata/index files are up to date.
In your case, you don't want to give users the ability to change the data, fetch from the remote by them selfs) and for that reason you want them to direct to the '-cache' repository.
To achieve what you want you can move/copy the content from the remote repository to a local repository. This will make sure that Artifactory will calculate the metadata and it won't be expirable as in the remote repository.
In case of copy, if you are worried about duplication of space, don't be as Artifactory has checksum based storage which means that it doesn't matter how many copies of the artifact you have as it will only create an additional pointer in the database meaning a link from the original file to that pointer :)