Can we plot image data in Altair?

Question

I am trying to plot image data in altair, specifically trying to replicate face recognition example in this link from Jake VDP's book - https://jakevdp.github.io/PythonDataScienceHandbook/05.07-support-vector-machines.html.

Any one had luck plotting image data in altair?

Answer 1

Altair features an image mark that can be used if you want to plot images that are available at a URL; for example:

import altair as alt
import pandas as pd

source = pd.DataFrame.from_records([
      {"x": 0.5, "y": 0.5, "img": "https://vega.github.io/vega-datasets/data/ffox.png"},
      {"x": 1.5, "y": 1.5, "img": "https://vega.github.io/vega-datasets/data/gimp.png"},
      {"x": 2.5, "y": 2.5, "img": "https://vega.github.io/vega-datasets/data/7zip.png"}
])

alt.Chart(source).mark_image(
    width=50,
    height=50
).encode(
    x='x',
    y='y',
    url='img'
)

Altair is not as well suited to displaying 2-dimensional data arrays as images, because the grammar is primarily designed to work with structured tabular data. However, it is possible to do using a combination of flatten transforms and window transforms.

Here is an example using the data from the page you linked to:

import altair as alt
import pandas as pd
from sklearn.datasets import fetch_lfw_people
faces = fetch_lfw_people(min_faces_per_person=60)

data = pd.DataFrame({
    'image': list(faces.images[:12])  # list of 2D arrays
})

alt.Chart(data).transform_window(
    index='count()'           # number each of the images
).transform_flatten(
    ['image']                 # extract rows from each image
).transform_window(
    row='count()',            # number the rows...
    groupby=['index']         # ...within each image
).transform_flatten(
    ['image']                 # extract the values from each row
).transform_window(
    column='count()',         # number the columns...
    groupby=['index', 'row']  # ...within each row & image
).mark_rect().encode(
    alt.X('column:O', axis=None),
    alt.Y('row:O', axis=None),
    alt.Color('image:Q',
        scale=alt.Scale(scheme=alt.SchemeParams('greys', extent=[1, 0])),
        legend=None
    ),
    alt.Facet('index:N', columns=4)
).properties(
    width=100,
    height=120
)