Release 1.20.0: posegrams that follow the motiongram convention

Three new pose views and two fixes.

posegrams() is the pose counterpart of motiongrams(), and oriented the same way:
horizontal has time across with image y down, vertical has time down with image
x across. MGT's two views deliberately run in different directions so each shares
a spatial axis with the picture and the pair can be laid around the frame. An
earlier version drew both with time on x, which breaks that and cannot be placed
beside a motiongram of the same recording. Because pose gives an actual position
rather than a region of changed pixels, these are the true location over time.

pose_spatial_map gives pose a "where" view of the same KIND as the pixel measures'
heat maps -- an image of the frame. The spatial GRAM has time on an axis and
looked, correctly, like a squashed posegram when used for that.

Fixed: motionvectorgrams wrote a 309 MB PNG on a real recording, and
motionscape(track=) assumed a track at the video frame rate, which the overview's
own P-frame magnitude is not.

Pass frame_size with landmarks. MediaPipe places landmarks it cannot see OUTSIDE
the picture -- largest y was 1529 in a 360-tall frame -- so inferring the frame
from the data scales a plot by an extrapolation rather than by the body.

977 passed, mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDfNkA78LYdttCbqQ9oqfE

Author: alexarje

CHANGELOG.md

@@ -5,6 +5,52 @@ All notable changes to MGT-python will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.20.0] — 2026-08-28
+
+### Added
+- **`posegram`** — one row per pose landmark, ordered head to foot, brightness for how fast
+  that landmark moved. The body-anchored counterpart of a motiongram: `pose_waterfall` says
+  where the body went, `pose_segments` how its limbs were angled, `pose_center` how its
+  centre moved, and none of them said *what was moving at 04:12*.
+
+  The row order is the design. MediaPipe emits its 33 landmarks in model order, which
+  scatters the body across the page; ordered anatomically a moving limb is a contiguous
+  band and the axis reads head / arms / hands / torso / legs.
+
+- **`posegrams`** — the pose counterpart of `motiongrams()`, and **oriented the same way**:
+  the horizontal view has time running across with image *y* down the page, the vertical
+  view has time running down with image *x* across. That is not decoration — MGT's two
+  views deliberately run in different directions so each shares a spatial axis with the
+  picture and the pair can be laid around the video frame. Because pose gives an actual
+  position rather than a region of changed pixels, these show the **true location** over
+  time.
+
+- **`posegram_spatial`** — the single-orientation form of the above, kept for when only one
+  view is wanted. `weight='presence'` shows where the body is regardless of movement,
+  which is a different question: a dancer standing still has presence and no speed.
+
+- **`pose_spatial_map`** — where the body was, as an image of the frame, so pose has a
+  "where" view of the same kind as the pixel measures' accumulated heat maps. Not to be
+  confused with `posegram_spatial`, which has time on an axis.
+
+### Fixed
+- **`motionvectorgrams` wrote a 309 MB PNG.** One pixel column per P-frame stretched to the
+  video's height is 77,592 × 1920 on a 158-minute session — 149 megapixels for something
+  meant to be glanced at. The array still keeps every column; the image is capped at 4000 px
+  wide, pooling columns by **maximum** so a one-second accent is not averaged away.
+
+- **`motionscape(track=…)` assumed the track ran at the video's frame rate**, which the
+  overview's own P-frame magnitude does not — it arrives at about a quarter of it, so the
+  time axis came out wrong by that factor while looking entirely reasonable. It takes
+  `track_times=` now and refuses a mismatched length.
+
+### Notes
+- Pose landmark arrays should be passed with the frame they were extracted in.
+  **MediaPipe estimates landmarks it cannot see and places them outside the picture** — on a
+  real 640×360 extraction the largest y was 1529 — so inferring the frame from the data
+  scales a plot by an extrapolation rather than by the body. The new functions take an
+  explicit frame size and fall back to a high percentile rather than the maximum.
+
 ## [1.19.0] — 2026-08-27
 
 ### Fixed

CITATION.cff

@@ -53,5 +53,5 @@ keywords:
   - musical gestures
 license: GPL-3.0-or-later
 doi: "10.5281/zenodo.21965729"
-version: 1.19.0
-date-released: "2026-08-26"
+version: "1.20.0"
+date-released: "2026-08-28"

docs/releases.md

@@ -1,6 +1,6 @@
 # Release Notes
 
-The current stable release is **MGT-python 1.19.0**.
+The current stable release is **MGT-python 1.20.0**.
 
 Install or upgrade from PyPI:
 

musicalgestures/__init__.py

@@ -4,7 +4,7 @@
 # this attribute rather than carrying its own copy. Two static copies
 # drift the moment a bump touches one of them, which is how ambiscape
 # shipped three releases misreporting themselves and musiscape one.
-__version__ = "1.19.0"
+__version__ = "1.20.0"
 from musicalgestures._input_test import mg_input_test
 from musicalgestures._videoreader import mg_videoreader
 from musicalgestures._flow import Flow

musicalgestures/_posegram.py

@@ -319,3 +319,197 @@ def mg_posegram_spatial(self: "musicalgestures.MgVideo", landmarks=None, times=N
         data={"gram": gram, "time": times, "axis": axis, "weight": weight},
         layers=None, image=target_name)
     return self.posegram_spatial_figure
+
+
+def pose_spatial_map(landmarks, width, height, bins=(180, 320), weight="speed",
+                     smooth=1.5):
+    """Where the body was in the frame, as an image — the pose answer to a heat map.
+
+    The pixel measures give a "where" panel by accumulating their per-pixel quantity over
+    the whole recording: an image of the room with a bright patch where things happened.
+    This is the same kind of object from landmarks, so the four can sit side by side.
+
+    Not to be confused with `pose_spatial_gram`, which has **time** on one axis. That is a
+    motiongram-like view and looks, correctly, like a squashed posegram; it answers "at
+    what height, when", where this answers "where in the room, over the whole recording".
+
+    Args:
+        landmarks (np.ndarray): `(frames, 33, 3)` in pixel coordinates.
+        width, height (int): The frame the landmarks were extracted in. Not inferred from
+            the data --- MediaPipe places landmarks it cannot see outside the picture, and
+            the maximum is one of those rather than the body.
+        bins (tuple, optional): `(rows, columns)` of the output image. Defaults to
+            (180, 320), a 16:9-ish grid fine enough to show a limb and coarse enough that
+            33 points a frame fill it.
+        weight (str, optional): `'speed'` brightens where landmarks moved fast,
+            `'presence'` where they simply were. Defaults to `'speed'`.
+        smooth (float, optional): Gaussian blur in output cells, so 33 points a frame read
+            as a body rather than as confetti. Defaults to 1.5; 0 disables.
+
+    Returns:
+        np.ndarray: `(rows, columns)`, an image in the frame's own coordinates.
+
+    Notes:
+        Landmarks outside the frame contribute nothing rather than being clamped to an
+        edge. Clamping would pile every lost limb onto the border and draw a bright rim
+        that no body ever made.
+    """
+    import numpy as np
+
+    a = np.asarray(landmarks, dtype=np.float64)
+    rows, cols = int(bins[0]), int(bins[1])
+    x, y = a[:, :, 0], a[:, :, 1]
+
+    if weight == "speed":
+        w = pose_activity(a).T
+    elif weight == "presence":
+        w = np.where(np.isfinite(x) & np.isfinite(y), 1.0, 0.0)
+    else:
+        raise ValueError(f"weight must be 'speed' or 'presence', not {weight!r}")
+
+    c = np.floor(x / max(float(width), 1e-9) * cols)
+    r = np.floor(y / max(float(height), 1e-9) * rows)
+    ok = (np.isfinite(c) & np.isfinite(r) & np.isfinite(w)
+          & (c >= 0) & (c < cols) & (r >= 0) & (r < rows) & (w > 0))
+    if not ok.any():
+        return np.zeros((rows, cols))
+    flat = (r[ok].astype(np.int64) * cols + c[ok].astype(np.int64))
+    m = np.bincount(flat, weights=w[ok], minlength=rows * cols).reshape(rows, cols)
+
+    if smooth and smooth > 0:
+        try:
+            from scipy.ndimage import gaussian_filter
+            m = gaussian_filter(m, smooth)
+        except ImportError:                          # pragma: no cover - scipy optional
+            pass
+    return m
+
+
+def posegram_arrays(landmarks, width, height, bins=200, weight="speed", spread=1.5):
+    """The horizontal and vertical posegrams, oriented as MGT's motiongrams are.
+
+    MGT's two views deliberately run in different directions, so that each shares a
+    spatial axis with the picture and the pair can be laid around the video frame:
+
+    * **horizontal** — a column per frame, tiled left to right. Time runs across, image
+      **y** runs down. Shape `(bins, frames)`.
+    * **vertical** — a row per frame, tiled top to bottom. Time runs down, image **x**
+      runs across. Shape `(frames, bins)`.
+
+    Drawing both with time on the x axis, as an earlier version did, breaks that: the
+    result cannot be placed beside a motiongram of the same recording because its spatial
+    axis no longer lines up with the frame.
+
+    Returns:
+        tuple: `(horizontal, vertical)`.
+    """
+    import numpy as np
+
+    horizontal = pose_spatial_gram(landmarks, height=height, width=width, bins=bins,
+                                   axis="vertical", weight=weight, spread=spread)
+    across = pose_spatial_gram(landmarks, height=height, width=width, bins=bins,
+                               axis="horizontal", weight=weight, spread=spread)
+    #: Transposed, so a frame is a ROW and time runs down the page.
+    return horizontal, np.asarray(across).T
+
+
+def mg_posegrams(self: "musicalgestures.MgVideo", landmarks=None, times=None,
+                 frame_size=None, bins: int = 200, weight: str = "speed",
+                 colormap: str = "magma", gamma: float = 0.5, max_width: int = 4000,
+                 dpi: int = 130, target_name: str | None = None,
+                 overwrite: bool = True) -> "musicalgestures.MgList":
+    """Posegrams of where the body actually was, in the frame's own coordinates.
+
+    The pose counterpart of `motiongrams()`, and oriented the same way: the horizontal
+    view has time running across with image y down the page, the vertical view has time
+    running down with image x across. Laid around a video frame they line up with it, and
+    laid beside a motiongram of the same recording they can be read against it — a body
+    crossing the room draws the same diagonal in both.
+
+    Because pose gives an actual position rather than a region of changed pixels, these
+    are the *true location* over time, not an estimate of where change happened.
+
+    Args:
+        landmarks (np.ndarray, optional): `(frames, 33, 3)` from a previous extraction.
+            Defaults to None, which runs pose here.
+        times (array-like, optional): Seconds per frame; usually needed, since pose is
+            normally extracted at a reduced rate.
+        frame_size (tuple, optional): `(width, height)` the landmarks were extracted in.
+            **Pass this whenever you pass `landmarks`** --- see this module's notes on
+            MediaPipe placing unseen landmarks outside the picture.
+        bins (int, optional): Cells along the spatial axis. Defaults to 200.
+        weight (str, optional): `'speed'` or `'presence'`. Defaults to `'speed'`.
+        colormap, gamma, max_width, dpi, target_name, overwrite: as elsewhere.
+
+    Returns:
+        MgList: the horizontal and vertical posegrams, in that order.
+    """
+    import matplotlib.pyplot as plt
+    import numpy as np
+
+    from musicalgestures._mglist import MgList
+    from musicalgestures._utils import MgImage
+
+    of, fex = os.path.splitext(self.filename)
+    base = resolve_filename(of, '_posegram.png', target_name, overwrite)
+    stem, _ = os.path.splitext(base)
+
+    if landmarks is None:
+        from musicalgestures._posetools import extract_pose_landmarks
+        r = extract_pose_landmarks(self.filename, quiet=True, verbose=False)
+        landmarks, times = r["landmarks"], r["time"]
+        width, height = r["width"], r["height"]
+    elif frame_size is not None:
+        width, height = float(frame_size[0]), float(frame_size[1])
+    else:
+        a = np.asarray(landmarks, dtype=np.float64)
+        width = float(np.nanpercentile(a[:, :, 0], 99.5)) * 1.05
+        height = float(np.nanpercentile(a[:, :, 1], 99.5)) * 1.05
+
+    horizontal, vertical = posegram_arrays(landmarks, width=width, height=height,
+                                           bins=bins, weight=weight)
+    times = (np.asarray(times, dtype=np.float64) if times is not None
+             else np.arange(horizontal.shape[1]) / float(self.fps))
+    minutes = (times[-1] / 60) if len(times) else 1
+
+    def _shade(a):
+        finite = a[np.isfinite(a) & (a > 0)]
+        ceiling = np.percentile(finite, 99) if finite.size else 1.0
+        return np.power(np.clip(a / max(ceiling, 1e-9), 0, 1), gamma)
+
+    images = []
+    for gram, suffix, horizontal_time in ((horizontal, "_h", True),
+                                          (vertical, "_v", False)):
+        drawn = gram
+        axis = 1 if horizontal_time else 0
+        if drawn.shape[axis] > max_width:
+            edges = np.linspace(0, drawn.shape[axis], max_width + 1).astype(int)
+            if horizontal_time:
+                drawn = np.stack([drawn[:, a:max(b, a + 1)].max(axis=1)
+                                  for a, b in zip(edges[:-1], edges[1:])], axis=1)
+            else:
+                drawn = np.stack([drawn[a:max(b, a + 1)].max(axis=0)
+                                  for a, b in zip(edges[:-1], edges[1:])])
+        if horizontal_time:
+            fig, ax = plt.subplots(figsize=(15, 5), dpi=dpi)
+            ax.imshow(_shade(drawn), cmap=colormap, aspect="auto",
+                      interpolation="nearest", extent=(0, minutes, bins, 0))
+            ax.set_xlabel("minutes")
+            ax.set_ylabel("top of frame  →  bottom")
+            ax.set_title("horizontal posegram — time across, image y down")
+        else:
+            fig, ax = plt.subplots(figsize=(6.5, 12), dpi=dpi)
+            ax.imshow(_shade(drawn), cmap=colormap, aspect="auto",
+                      interpolation="nearest", extent=(0, bins, minutes, 0))
+            ax.set_ylabel("minutes")
+            ax.set_xlabel("left of frame  →  right")
+            ax.set_title("vertical posegram — time down, image x across")
+        ax.set_xticks(ax.get_xticks()) if False else None
+        fig.tight_layout()
+        path = f"{stem}{suffix}.png"
+        fig.savefig(path)
+        plt.close(fig)
+        images.append(MgImage(path))
+
+    self.posegrams_images = MgList(images)
+    return self.posegrams_images

musicalgestures/_video.py

@@ -195,6 +195,7 @@ def __init__(
     motionscape_figure: MgFigure
     posegram_figure: MgFigure
     posegram_spatial_figure: MgFigure
+    posegrams_images: "musicalgestures.MgList"
     movement_beat_statistics_figure: MgFigure
     phase_synchrony_figure: MgFigure
     pose_average_image: MgImage
@@ -321,6 +322,7 @@ def motion_mp(self, *args, **kwargs):
     from musicalgestures._posegram import (  # type: ignore[misc]
         mg_posegram as posegram,
         mg_posegram_spatial as posegram_spatial,
+        mg_posegrams as posegrams,
     )
 
     def __repr__(self) -> str:

tests/test_pose_spatial_map.py

@@ -0,0 +1,68 @@
+"""A 2-D map of where the body was, in the frame's own coordinates.
+
+The other measures' "where" panels are images of the room: accumulate the per-pixel
+difference, or the flow magnitude, over the whole recording and you get a picture with a
+bright patch where things happened. The pose equivalent has to be the same KIND of object
+--- a (height, width) image --- and not a gram with time on one axis, which is what a
+spatial *gram* is and which looks, correctly, like a squashed posegram.
+"""
+import numpy as np
+import pytest
+
+from musicalgestures._posegram import pose_spatial_map
+
+
+def _pose(n, n_landmarks=33):
+    a = np.zeros((n, n_landmarks, 3))
+    a[:, :, 0] = 320.0
+    a[:, :, 1] = 240.0
+    a[:, :, 2] = 1.0
+    return a
+
+
+class Test_it_is_an_image_of_the_frame:
+    def test_shape_is_height_by_width(self):
+        m = pose_spatial_map(_pose(120), width=640, height=480, bins=(90, 120))
+        assert m.shape == (90, 120)
+
+    def test_a_landmark_crossing_left_to_right_lights_a_horizontal_band(self):
+        n = 300
+        a = _pose(n)
+        #: Only the travelling landmark is present. With the other 32 parked at one
+        #: point they outvote it 32 to 1 and the brightest row is theirs, not its ---
+        #: which is what the first version of this test actually measured.
+        a[:, :, :] = np.nan
+        a[:, 15, 0] = np.linspace(40, 600, n)
+        a[:, 15, 1] = 120.0
+        a[:, 15, 2] = 1.0
+        m = pose_spatial_map(a, width=640, height=480, bins=(96, 128), weight="presence")
+        rows = m.sum(axis=1)
+        band = int(np.argmax(rows))
+        # 120/480 of 96 rows is row 24
+        assert abs(band - 24) <= 4
+        assert m[band].max() > 0
+        lit_cols = np.flatnonzero(m[band] > 0)
+        assert lit_cols.max() - lit_cols.min() > 40      # it swept across
+
+    def test_a_landmark_that_stays_put_lights_one_place(self):
+        m = pose_spatial_map(_pose(200), width=640, height=480, bins=(96, 128),
+                             weight="presence")
+        lit = np.argwhere(m > m.max() * 0.5)
+        assert np.ptp(lit[:, 0]) < 8 and np.ptp(lit[:, 1]) < 8
+
+
+class Test_what_it_weights_by:
+    def test_speed_ignores_a_body_that_is_present_but_still(self):
+        m = pose_spatial_map(_pose(150), width=640, height=480, bins=(60, 80),
+                             weight="speed")
+        assert m.max() == pytest.approx(0.0, abs=1e-9)
+
+    def test_landmarks_outside_the_frame_are_dropped_not_clamped(self):
+        """MediaPipe places unseen landmarks outside the picture. Clamping them would
+        pile a lost limb onto the edge and draw a bright rim no body ever made."""
+        a = _pose(120)
+        a[:, 31, 0] = 5000.0
+        a[:, 31, 1] = 5000.0
+        m = pose_spatial_map(a, width=640, height=480, bins=(60, 80), weight="presence")
+        assert m[-1, -1] == pytest.approx(0.0, abs=1e-9)
+        assert np.isfinite(m).all()