I use Conda and have a private Conda repo.
The max amount of layers a function can use is five. So I can't just upload every one of my private Conda packages as a separate layer- because functions might need to use more than 5 of them
I'm unclear on how layers are supposed to solve this problem or if they are not designed for this sort of thing.
Would I have just one lambda layer for the base Conda environment and then have the lambda use that layer to import my private Conda packages? Or should I be uploading my entire function as a new layer itself, where I would build my Conda Python app locally with all my private Conda deps, upload it as a new layer and then have a different function call it?
You could try building all you private packages into a separate - single - layer, and then using that (single) layer of your own with your functions. ...would that work for you? (there is no restriction in layers that they only have to contain single packages/modules)
If your base conda env also has public packages, you could also have a separate layer you build for that environment - so then in your functions you'd include two layers - one for your base conda env, and one for all your private packages.
With my code, I use conda for dev, but not for deployment. At present I've built and use a private layer containing skimage, but I build all my private packages in with the lambda's themselves, rather than bundling them as a layer. But I've thought about moving that way in the future.
I also use AWS's SAM for building and packaging, which makes things a lot easier. But I'm not sure how easy or not it is to make it work with conda instead of pip.
Here's a post I did on using SAM to make layers - might be some help if you go that route (...you can use SAM to build/install layers right now, but it takes a bit of arm twisting)