Application #1: nbconvert

Distinction between use cases for nbformat and nbconvert

nbformat: creating and validating notebooks

nbconvert: manipulate existing notebooks.

Nbconvert use cases

  • converting notebooks into other formats
    • web-display: html, slides (with reveal.js)
    • publishable documents: LaTeX/PDF, ASCIIDoc
    • plain-text: rst, markdown
    • executable scripts: e.g., *.py

Nbconvert use cases (cont.)

  • manipulating notebook content
    • cell magic (%%R) code highlighting
    • removing content
    • extracting images references for plain-text formats (LaTeX, markdown)
  • executing notebooks from the command-line

Command Line Interface


In [ ]:
!jupyter nbconvert my_demo_notebook.ipynb --to markdown
!cat my_demo_notebook.ipynb

NbConvertApp

  • manages the CLI.
  • manages the configuration as established both on the command line and via traitlet config files.
  • wraps base class functionality.

Nbconvert configuration: traitlets

Configuration is specified using traitlets

  • as on-disk config file: jupyter_nbconvert_config.(py|json)
  • command line arguments: jupyter nbconvert --template=basic
  • passed to instance: Exporter(config=Config())

Exporters

Orchestration layer.

Keyed to the --to <exporter_name> command line argument.

Exporters specify many aspects of conversion pipeline. E.g.:

  • which preprocessors
  • whether & which template
  • output format

The resources dictionary

In addition the NotebookNode instance, exporters create a resources dictionary.

This is useful for passing information that not be in the notebook itself.

  • Notebook styling (for html export)
  • metadata for populating a jekyll header

Preprocessors

Notebook to notebook transformations.

ExecutePreprocessor: This enables CLI execution.
TagRemovePreprocessor: Removes cells tagged with particular tags specified as traitlets.

Templates (and filters)

Nbconvert uses Jinja2 templates.

Templates inherit from one another.

Templates can access filters that can transform the content passed through them.

One of the most common filters passes the plaintext representation of a cell's source to pandoc for conversion.

pandoc & pandocfilters

pandoc converts between formats

pandocfilters manipulate pandoc's intermediate JSON representation.

Writers & Postprocessors

Writers handles the Exporter's final output.

FilesWriter writes to disk.

Postprocessors manipulate the file after the Writer is finished.

ServePostProcessor serves html file.

Entrypoints

In addition to being highly configurable, we have a mechanism for 3rd party libraries to register Exporters using entrypoints.

  1. Define JekyllExporter
  2. package it in my_jekyll_exporter
setup(,
      entry_points = {
        'nbconvert.exporters': 
            ['jekyll = my_jekyll_exporter:JekyllExporter']
      }
)

Entrypoints (contd.)

And with pip install my_jekyll_exporter

Anyone can use your exporter with
jupyter nbconvert --to jekyll

Multi-notebook workflows

Nbconvert works on single notebooks.

Multiple notebook conversion is still in early days.

Examples:

  • bookbook (html or pdf books from multiple notebooks)
  • multi_rise (one slideshow from many notebooks)