Search code examples
elixirelixir-mixhex-pm

building the hex registry file offline


Adapting Elixir and all the tools in its ecosystem to work with a different build system.

In this system, the packages and their dependencies are managed separately and Hex is made to work in offline mode. (grab the tarballs)

It's working with one caveat: every time I import a new package I need to also import the latest registry file from hexpm and I cannot use packages that are not published through hex unless they are at the top level in the deps chain.

Given a bunch of tarballs (and assuming that the dependencies between them are satisfied, how would one go about building a hex registry file that works with them.

What I have so far:

  • looked at the registry file format and seen it's an ets file. Can load and inspect it; now I need to generated
  • looked at how the website builds the registry file, but it's super complicated for my needs
  • I struggle a bit to understand why there is a need for a registry file (and if there is, why can't each package contain the needed info in the metadata, making the need for a central registry obsolete)

Anyway, if anyone played with Hex and can provide some guidance on how to do this I would appreciate it.


Solution

  • It's a bit hard to give good information and advice without more information on your use case. Could you elaborate a bit more on what you are doing and why you are doing it? I will try my best to answer the question though.

    Here is the specification for the registry format: https://github.com/hexpm/specifications/blob/master/registry.md.

    The format is fairly simple and it would not require too much code to build the ETS file yourself.

    I struggle a bit to understand why there is a need for a registry file (and if there is, why can't each package contain the needed info in the metadata, making the need for a central registry obsolete)

    The registry is needed for the dependency resolution in the Hex client. It is possible for the resolver to try many different versions of packages, if the client had to fetch each package version to see if it resolved a lot of useless HTTP requests would have to be made. The registry is there as an optimization so we only have to fetch a single file to do the full resolution.

    I think what you may want is to depend on local package tarballs directly since you imply you do the dependency resolution yourself. Is that correct? I have opened an issue on the client to support this: https://github.com/hexpm/hex/issues/261