Search code examples
pythonjupyter-notebooknbconvert

Jupyter nbconvert function with multiple metadata masks

Problem

I'm trying to clear all output and almost all metadata from a jupyter notebook. When I run the following command,

jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
--ClearMetadataPreprocessor.enabled=True \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="{('language_info', 'name'), 'kernelspec'}" \
--to=notebook --log-level=ERROR my_notebook.ipynb

I get the output I want,

 "metadata": {
  "kernelspec": {
   "display_name": "my-kernel",
   "language": "python",
   "name": "my-kernel"
  },
  "language_info": {
   "name": "python"
  }
 }

but also get this warning:

/usr/local/miniconda/lib/python3.7/site-packages/traitlets/traitlets.py:2935: FutureWarning: --ClearMetadataPreprocessor.preserve_nb_metadata_mask={('language_info', 'name'), 'kernelspec'} for containers is deprecated in traitlets 5.0. You can pass `--ClearMetadataPreprocessor.preserve_nb_metadata_mask item` ... multiple times to add items to a list.
  FutureWarning,

Given the suggestion from the FutureWarning and the docs, I ran

jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
--ClearMetadataPreprocessor.enabled=True \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="{('language_info', 'name')}" \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="{'kernelspec'}" \
--to=notebook --log-level=ERROR my_notebook.ipynb

However, instead of getting my expected output, the notebook has no metadata:

"metadata": {}

Question

  1. What's going on here? Is it a dependency version issue?
  2. How can I get the metadata mask to work without a FutureWarning?
Solution

  • I think it is simpler than you trying.
    I just tried this and I think it does close to what you want and *importantly, no warning. (I see the warning when trying your first block in the environment, and so I expect it would be there.):

    jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
--ClearMetadataPreprocessor.enabled=True \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="language_info" \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="name" \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="kernelspec" \
--to=notebook --log-level=ERROR index.ipynb

    I don't think you actually need `"name" there. I think this works, too:

    jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
--ClearMetadataPreprocessor.enabled=True \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="language_info" \
--ClearMetadataPreprocessor.preserve_nb_metadata_mask="kernelspec" \
--to=notebook --log-level=ERROR index.ipynb

    I will admit I'm not seeing how to submit the tuple so you can leave just the name field of the 'language_info' section.

    In reply to the comment about the tuple

    Attempts and results:

    jovyan@jupyter-binder-2dexamples-2drequirements-2dph7y6uqp:~$ jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
> --ClearMetadataPreprocessor.enabled=True \
> --ClearMetadataPreprocessor.preserve_nb_metadata_mask= 'language_info', 'name'  \
> --ClearMetadataPreprocessor.preserve_nb_metadata_mask="kernelspec" \
> --to=notebook --log-level=ERROR index.ipynb
usage: jupyter-nbconvert [-h] [--debug] [--show-config] [--show-config-json] [--generate-config] [-y] [--execute] [--allow-errors] [--stdin]
                         [--stdout] [--inplace] [--clear-output] [--no-prompt] [--no-input] [--allow-chromium-download]
                         [--disable-chromium-sandbox] [--show-input] [--embed-images] [--sanitize-html] [--log-level NbConvertApp.log_level]
                         [--config NbConvertApp.config_file] [--to NbConvertApp.export_format] [--template TemplateExporter.template_name]
                         [--template-file TemplateExporter.template_file] [--theme HTMLExporter.theme]
                         [--sanitize_html HTMLExporter.sanitize_html] [--writer NbConvertApp.writer_class]
                         [--post NbConvertApp.postprocessor_class] [--output NbConvertApp.output_base]
                         [--output-dir FilesWriter.build_directory] [--reveal-prefix SlidesExporter.reveal_url_prefix]
                         [--nbformat NotebookExporter.nbformat_version]
                         [extra_args ...]
jupyter-nbconvert: error: unrecognized arguments: index.ipynb
jovyan@jupyter-binder-2dexamples-2drequirements-2dph7y6uqp:~$ 
jovyan@jupyter-binder-2dexamples-2drequirements-2dph7y6uqp:~$ jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
> --ClearMetadataPreprocessor.enabled=True \
> --ClearMetadataPreprocessor.preserve_nb_metadata_mask= ('language_info', 'name')  \
bash: syntax error near unexpected token `('
jovyan@jupyter-binder-2dexamples-2drequirements-2dph7y6uqp:~$ --ClearMetadataPreprocessor.preserve_nb_metadata_mask="kernelspec" \
> --to=notebook --log-level=ERROR index.ipynb
bash: --ClearMetadataPreprocessor.preserve_nb_metadata_mask=kernelspec: command not found
jovyan@jupyter-binder-2dexamples-2drequirements-2dph7y6uqp:~$

    Do you know the syntax that works for leaving just the name entry for language_info?