Embedding Plotly in a Jekyll Post

Yes, you can include Plotly’s interactive graphics inside your posts, and here is how!

In this post, we will see how you can:

• generate Jekyll posts from Jupyter notebooks;
• include Plotly figures inside the posts;
• and render transparent images.

import pandas as pd
from plotting import express as px
from plotting import graph_objects as go

# Example from: https://plotly.com/python/sliders/

df = px.data.gapminder()
fig = px.scatter(
    df,
    x="gdpPercap",
    y="lifeExp",
    animation_frame="year",
    animation_group="country",
    size="pop",
    color="continent",
    hover_name="country",
    log_x=True,
    size_max=55,
    range_x=[100, 100000],
    range_y=[25, 90],
)

fig.show()

Notebook setup

When executing notebooks, Plotly figures are directly included the notebook. Because those graphics are HTML images with Javascript, including this HTML code in the markdown output can cause some issues with the Liquid templating system used by Jekyll. This is because the generated Javascript contains curly braces, which Liquid tries to interpret, but should not.

All the steps mentioned below, and more, are performed when importing _notebooks/plotting.py.

Changing the default renderer

Plotly offers a variety of renderers, one of which renders the Plotly figures in separated HTML files, that are included using iframe elements.

Among the IFrame renderers, we will use the iframe_connected renderer, that loads the necessary Plotly files from a CDN. This is useful to avoid including the whole Plotly library in your HTML files.

You can change the default plotly renderer with:

import plotly.io as pio

pio.renderers.default = "iframe_connected"

Patching the HTML directory

One issue with IFrame renderers is that they will generate all external files inside the same directory.

You can patch this by creating a custom renderer and make it the default:

import os
from uuid import uuid4  # To generated a new random directory for each notebook

import plotly.io as pio
from plotly.io._base_renderers import IFrameRenderer

pio.renderers["custom"] = IFrameRenderer(
    html_directory=os.path.join("../assets/notebooks", "html_" + str(uuid4())),
    include_plotlyjs="cdn",
)

pio.renderers.default = "custom"

Transparent background

If you support multiple themes on your website (e.g., light and dark), you might want your figures to render nicely with both themes.

Unfortunately, Plotly does not support dynamically changing the theme of a given figure, at least not easily.

A solution to this problem can be to specify a common Plotly theme that looks nice with all your website themes. In most cases, you’d like this theme to have a transparent background.

For this, you can also define a custom theme and make it the default:

from plotly.graph_objs import Figure

fig = Figure()
fig.update_layout(
    template="plotly_white",  # Change this to any Plotly built-in theme
    paper_bgcolor="rgba(0,0,0,0)",  # Transparent
    plot_bgcolor="rgba(0,0,0,0)",  # Transparent
    font_color="#d37e34",  # A nice orange font
    scene=dict(
        xaxis=dict(
            backgroundcolor="rgba(0,0,0,0)",  # Transparent
        ),
        yaxis=dict(
            backgroundcolor="rgba(0,0,0,0)",  # Transparent
        ),
        zaxis=dict(
            backgroundcolor="rgba(0,0,0,0)",  # Transparent
        ),
    ),
)

templated_fig = pio.to_templated(fig)
pio.templates["custom"] = templated_fig.layout.template

pio.templates.default = "custom"

Converting notebooks to posts

Jekyll generates posts from markdown files, and nbconvert can can convert notebooks into various formats, including markdown.

Therefore, it is just a matter of combining the two tools rightfully to generate Jekyll posts from Jupyter notebooks.

If not already, please install nbconvert:

pip install nbconvert

The following contents is inspired from notebooks_to_posts.sh.

Executing notebooks

When a notebook is run, it can generate some asset files, like plot images, that are included in the notebook using relative paths.

The important point is that, if you move a post to another directory, you have to move its assets accordingly. Additionally, Jekyll requires posts’ assets to be placed in the assets directory.

To satisfy both constraints, the best is to set the output directory of jupyter notebook to be inside the assets directory, e.g., with the following command:

echo "Executing notebooks..." && \
    [ -d _notebooks ] && \
    [ "$(find _notebooks -type f -iname '*.ipynb')" ] && \
    python3 -m jupyter nbconvert _notebooks/*.ipynb \
        --ExecutePreprocessor.kernel_name=python3 \
        --execute \
        --to markdown \
        --output-dir assets/notebooks || \
    echo "No notebook found"

The first three lines are here to check if: (1) the _notebooks directory exists and (2) if there is at least one notebook inside.

Let us examine the various arguments:

• _notebooks/*.ipynb let us call the command on each notebook in he directory;
• --ExecutePreprocessor.kernel_name=python3 makes sure we execute the notebooks with the given kernel. This is important if you plan to use this in GitHub workflows because the kernel installed inside a workflow might not have the same name as the one you used to create the notebook;
• --execute actually executes the notebook (recommended);
• --to markdown specifies the output file type;
• and --output-dir assets/notebooks places the output files in the given directory, as mentioned above.

Fixing relative paths

Prior to moving the notebooks (converted to markdown) to the _posts directory, we must fix the path to the assets to match how Jekyll works.

For this, we will use sed, a command-line tool that can search and replace text in files, using regexes.

echo "Changing link to html image files" && \
    find assets/notebooks -type f -iname '*.md' | \
    xargs -n 1 sed -i -E 's/src="\.\.\//src="\//'

This command fixes paths starting with ../ to /, e.g., ../assets/some_path.html becomes /assets/some_path.html.

Keeping track of source path

Optionally, we may want to store the path to the original notebook file that generated each post. To do so, we add a source: <path> line to the YAML front matter of each post generated from a notebook.

This is especially useful if you want to provide some “suggest an edit” button on your posts, to link to the original file on GitHub.

This is what this very long Python line does:

echo "Changing relative path for posts generated from notebooks" && \
    find assets/notebooks -type f -iname '*.md' | \
    xargs -n 1 python3 -c "import sys,os;file=sys.argv[1];lines=open(file).readlines();index=lines.index('---\n');basename=os.path.basename(file);path=os.path.join('_notebooks', basename[:-2] + 'ipynb\n');lines.insert(index + 1, f'source: {path}');open(file, 'w').writelines(lines);"

Moving generated posts to correct directory

Finally, we can move the newly created posts inside the _posts directory:

echo "Moving new posts to _posts directory" && \
    mv assets/notebooks/*.md _posts/

More examples

Here are a few more examples from Plotly’s documentation.

Range slider

# Example from: https://plotly.com/python/time-series/

df = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/"
    "datasets/master/finance-charts-apple.csv"
)

fig = go.Figure(go.Scatter(x=df["Date"], y=df["mavg"]))

fig.update_xaxes(
    rangeslider_visible=True,
    tickformatstops=[
        dict(dtickrange=[None, 1000], value="%H:%M:%S.%L ms"),
        dict(dtickrange=[1000, 60000], value="%H:%M:%S s"),
        dict(dtickrange=[60000, 3600000], value="%H:%M m"),
        dict(dtickrange=[3600000, 86400000], value="%H:%M h"),
        dict(dtickrange=[86400000, 604800000], value="%e. %b d"),
        dict(dtickrange=[604800000, "M1"], value="%e. %b w"),
        dict(dtickrange=["M1", "M12"], value="%b '%y M"),
        dict(dtickrange=["M12", None], value="%Y Y"),
    ],
)

fig.show()

3D Surface

# Example from: https://plotly.com/python/3d-surface-plots/

# Read data from a csv
z_data = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/"
    "datasets/master/api_docs/mt_bruno_elevation.csv"
)

fig = go.Figure(data=[go.Surface(z=z_data.values)])
fig.update_traces(
    contours_z=dict(
        show=True, usecolormap=True, highlightcolor="limegreen", project_z=True
    )
)
fig.update_layout(
    title="Mt Bruno Elevation",
    autosize=False,
    scene_camera_eye=dict(x=1.87, y=0.88, z=-0.64),
    width=500,
    height=500,
    margin=dict(l=65, r=50, b=65, t=90),
)

fig.show()

Treemap

# Example from: https://plotly.com/python/plotly-express/

df = px.data.gapminder().query("year == 2007")
fig = px.treemap(
    df,
    path=[px.Constant("world"), "continent", "country"],
    values="pop",
    color="lifeExp",
    hover_data=["iso_alpha"],
)
fig.show()