A Musical Timeline for SpaceMusic

Time we can't see

A SpaceMusic performance is a thing that happens over time — colours drift, plugins swap, a slider rides up into a drop. The engine can already capture those moves and replay them. What it cannot do is show them to you as anything you can look at, grab, or trim.

The recorder we shipped in plan 062 writes every parameter change to a flat text file — one event per line, a timestamp, a value. Play it back and the moves happen again. That is genuinely useful, and it is also a black box: there is no picture of what was recorded, no way to loop four bars of it, no way to nudge a keyframe that landed a beat early, and no relationship at all to the musical grid the rest of the system runs on. Time exists in SpaceMusic, but nothing holds it as an object you can edit.

This document is the design for the thing that does — a timeline editor that is a first-class citizen of the UI, the way the file browser is: a web surface backed by a companion in the engine, talking over a channel protocol, portable to any future front-end. The interesting part is not the drawing. It is that building it forces two halves of the system that have never spoken — a musical clock and a wall-clock recorder — to finally agree on what "now" means.

Two clocks that never met

SpaceMusic already has a real musical transport. It is a vvvv patch called MainTimer, and it is more capable than its obscurity suggests: it produces a running beat position, the current bar, a BPM, a play/stop state, a time signature, and phase — and it can be driven by an external MIDI clock or by Ableton Link, so it locks to whatever else is on stage. This is the clock the visuals already dance to.

The recorder from plan 062 has its own clock too — a plain stopwatch counting milliseconds from the moment you press record. It knows nothing about beats, bars, or tempo. So we have two clocks in one system: one musical and shared but trapped inside a patch, one wall-clock and disconnected. A timeline has to pick a relationship between them, and that choice colours everything downstream. There were three honest options.

1. Unify on beats. Make the musical clock the one true transport; automation records and plays in beats, follows tempo, loops on bar lines. Coherent and sync-ready — but it means rewriting the recorder and player onto a beat clock, and a clip can never be "exactly 2.5 seconds" again.
2. Two clocks, one a picture. Keep automation on wall-clock milliseconds and draw the bars as a non-binding ruler. Cheap, no rewrite — but the automation ignores tempo, can't loop musically, and the two clocks drift forever. It fights the whole "musical, Link-synced" goal.
3. Hybrid warp. Store both a millisecond and a beat position for every event, and let each clip choose: time-locked (plays at real seconds) or tempo-locked (follows the beat), exactly like Ableton's warp switch. The most flexible, and the one we chose.

We took the hybrid. It is the heaviest to build, but it is the only option that lets a lighting cue stay pinned to real seconds while a filter sweep next to it stretches with the tempo — which is the actual thing performers want. The cost is bookkeeping: every recorded event now needs its beat position at capture, and one component has to own the beats-to-seconds conversion so nothing drifts. That component is a C# port of the clock.

Where we are today

Three pieces exist; none of them touch. The recorder and player (plan 062) capture and replay a flat .smrec.jsonl log through a shared write path (ChannelWriter) and a small command layer (CommandRegistry) that already knows how to re-issue a scene load or fire an instance-add. MainTimer computes the musical transport, but publishes almost none of it — only BPM is exposed as a public channel; beat, bar, and play-state live as patch operations no other code can read. And there is no UI at all: you record blind and play back blind.

So the work is not "add a widget." It is to bridge MainTimer into channels the web can read, promote the flat log into a structured document you can edit, and give both a face. That bridging is the load-bearing idea.

The engine is the timeline; the UI is a lens

The file browser already taught us the pattern worth copying. The web browser is only a picker — it never loads a file; it emits a request on a channel and the engine does the real work. The contract between them is not code, it is a set of named channels carrying JSON. Any future front-end reimplements the view and reuses the same channels, and every existing patch keeps working. We apply that split wholesale: the engine (C#) is authoritative — it owns the clock, the document, and playback — and every UI is a view and editor over channels.

Concretely, that means four legs. We port MainTimer's clock into a C# service, MainClock, so beat/bar/phase/play-state finally become public channels under a Runtime.Time.* namespace. The recorder and player evolve into document-aware versions that read and write a new .smtimeline artifact stored in the library — above scenes, because a timeline can itself trigger scene loads. And a bespoke web page reads the transport channels to draw a playhead, and sends edits and transport commands back over a Runtime.Timeline.Command channel. Everything crosses the same public channel hub; nothing in the engine knows the web exists.

Read it left to right. Hardware clock feeds MainClock; MainClock publishes the transport onto the hub; the web page reads it to move a playhead and sends edits and transport commands back on the timeline command channel; the player writes recorded values back onto the parameters through the same ChannelWriter a live web edit uses, and the recorder captures the reverse. The document lives in the library and is loaded through the existing file browser. The dashed line down the middle is the only wall that matters: everything to its left is the engine, everything to its right is a replaceable view.

Tracks, clips, and two time bases

The document is four levels deep. A timeline holds tracks; a track holds clips; a clip holds keyframes (or MIDI notes, or command events). A track binds to one thing — a parameter channel, a MIDI device, or the command layer — and recording auto-creates a track the first time that thing changes, dropping one clip per record pass onto its lane. Record the same slider twice and you get two clips on one track, which is exactly the "it fills up as I play, and I see the same move again later" feel we were after.

Inside a clip, we do not keep the raw sixty-samples-a-second we captured. We thin them into sparse keyframes with a per-segment interpolation mode — linear, smooth, step, or hold — so a slider ramp becomes a curve you can grab, discrete controls step cleanly, and files stay small. The player interpolates between keyframes each frame. And every keyframe carries both a millisecond and a beat stamp, because a clip declares its warp: a tempo clip is played against the beat clock and stretches when the BPM changes; a time clip is played against real seconds and never moves. That is the hybrid from section two, made concrete.

Two rules keep this sane. First, only MainClock ever converts between beats and seconds — nothing else does the arithmetic, so the two time bases can't drift apart. Second, when clips on one track overlap, the latest wins within its span, and outside every clip the parameter is simply live — the timeline is not holding it hostage. The recorder, player, and their write path are the plan-062 nodes grown up, not rewritten: the recorder gains a beat stamp and a thinning pass, the player gains warp and a loop region, and both still lean on ChannelWriter and CommandRegistry so a scene-load event on the timeline replays through the exact code a live scene load uses.

Building it in slices

The scope is real — a C# clock port, a document model, two capture paths, warp, keyframe thinning, a bespoke editor, and library save/load. The way not to drown is to run a thin slice through the whole stack first and prove the scariest seam — clock to channels to screen — before pouring in the backend. Everything after that thickens a spine that already works.

Phase one ports the clock and renders today's recordings against the beat grid, read-only — if the playhead tracks the patch transport and an old take lands on the right lanes, the whole spine is proven. Phase two brings the document and beat-aware capture and playback; the acceptance test is a tempo change mid-playback that a tempo-locked clip actually follows. Phase three adds the moderate edit set — move and trim clips, set an in/out loop, tweak a keyframe in an inspector — and save/load through the file browser. Phase four proves the track-type seam with a second real payload, MIDI. The honest note on the diagram is the dashed line: phase four is the biggest single lift and the least de-risked by the phases before it, so P1–P3 stand on their own as a shippable v1 and MIDI can slip to v1.1 without any rework — the seam is designed for exactly that cut.

Why this matters

"A timeline is not a feature we bolt onto the side. It is the moment SpaceMusic stops being only a thing you play live and becomes a thing you can compose."

Right now every performance is improvised in the sense that nothing survives the moment unless a person is holding the knob. Give the system an editable, musical timeline and three things change at once. Moves become repeatable — record a build, loop it, trust it. They become musical — locked to the same beat clock as everything else on stage, warping with the tempo, syncing to Ableton Link or an incoming MIDI clock. And because we build it as a portable channel contract rather than a one-off screen, the timeline becomes a reusable primitive: the same surface that shows parameter automation today hosts MIDI tomorrow and any recorded stream after that, in this UI or the next one.

The risk we are accepting is the hybrid warp and the two time bases — real bookkeeping, real chances to drift. We are containing it by making one component own the conversion and by proving the spine before we lean on it. If we skip the timeline, we keep a black-box recorder that only its author can trust, and the musical clock stays trapped in a patch nobody else can read. The plan is to let them meet.

Glossary

Terms and proper nouns used above, in plain language.

MainTimer

The existing vvvv patch that computes SpaceMusic's musical transport — beat, bar, BPM, play-state, time signature — and can lock to an external MIDI clock or Ableton Link. We plan to port it to a C# service, MainClock.

channel / hub

vvvv's named, observable value slots. The hub is the app-wide registry; any code reads or writes a channel by its string path. Our whole UI-to-engine contract is JSON on named channels.

.smrec.jsonl

The flat, append-only recording format from plan 062 — one JSON event per line, timestamped in milliseconds. The raw capture the new document supersedes.

.smtimeline

The proposed structured, editable document: tracks holding clips holding keyframes, stored as a library artifact and loaded like a scene.

ChannelWriter

The shared C# write path that applies a value to a parameter channel — used identically by live web edits and by timeline playback, so a recorded move replays exactly as it was applied.

CommandRegistry

The plan-062 table of semantic commands (scene load, instance add/remove, menu actions) with a replay policy each, so playback can re-issue a scene load rather than fight it.

warp

A per-clip switch: tempo-locked clips follow the beat clock and stretch with BPM; time-locked clips play at fixed real seconds. Borrowed from Ableton Live.

keyframe

A single point of automation — a value at a position — with an interpolation mode to the next point. Captured dense, then thinned to a sparse, editable set.

clip

A bounded region on a track produced by one record pass, with its own warp mode and in/out. Clips can overlap; the latest wins in its span.

track

One lane bound to one thing — a parameter, a MIDI device, or the command layer — holding that thing's clips over time.

WireTag

The small type code (float, bool, string, bang, enum key, …) carried with each value so the write path knows how to apply it.

Ableton Link

A protocol for keeping tempo and beat phase in sync across apps and machines on a network. MainTimer can consume it; the node is currently absent and needs a C# binding.

MIDI clock

A stream of timing pulses from an external device or DAW that MainTimer can follow to set tempo and beat position.

Runtime.Time.* / Runtime.Timeline.*

The proposed channel namespaces: transport state flowing engine-to-UI, and edit/transport commands flowing UI-to-engine — mirroring the file browser's Runtime.Browser.* contract.