Packaging a Python project and its associated IPython magic extension

Question

I am in the process of deploying to Pypi a Python project, let's call it foobar. I would like to distribute it with a shell command and an IPython magic command. I use Poetry, and the relevant part of my .toml configuration file is:

  [tool.poetry.scripts]
  foobar = 'foobar.cli:main'
  foobar_magic = 'foobar.magic:load_ipython_extension'

After uploading this to TestPypi and installing it with pip, the shell command (foobar) works as expected. However, executing %load_ext foobar_magic in a Jupyter Notebook fails with:

  ModuleNotFoundError: No module named 'foobar_magic'

According to the documentation:

You can put your extension modules anywhere you want, as long as they can be imported by Python’s standard import mechanism.

Under the same notebook, I have verified that !foobar and import foobar both work. How can I make foobar_magic be found too?

Moreover, although I'm not there yet, I guess the suffix of the entry point is wrong too. Indeed, the function I specify after the : will be called with no arguments, but the function load_ipython_extension() expects an IPython instance.

So I feel completely lost, and can't find any relevant documentation for deploying IPython Notebook extensions.

---

Edit 1. %load_ext foobar.magic unexpectedly works, and the magic %foobar does not complain about the arguments. I don't understand why, and why it is %foobar and not %foobar_magic as declared.

Edit 2. the foobar_magic = ... stuff is ignored or useless. Suppressing it has no consequence on %load_ext foobar.magic. I think the latter invocation might be ok. But it's a little annoying not to understand what's going on.

Answer 1

I finally found a workaround:

1. Delete the line foobar_magic = ... of my .toml.

2. Move the contents of foobar/magic.py to foobar/__init__.py (originally empty), guarded with the following two lines:

   import sys
   if "ipykernel" in sys.modules:
      # magic stuff
   

3. This file being executed each time the module is imported, it is now enough to do (under a notebook):

   %load_ext foobar
   

   The guard ensures the magic stuff is executed if and only if foobar is imported from IPython.

This does not answer my original question, and I still do not fully understand how these entry points are supposed to work, but I am happy with the actual result.