api/the public surface — $(), the verbs, the operators, batch, runtime. The one entry you import.the original sin of reactive UI
Almost every reactive system re-runs the whole computation when a single
value changes. data is a reactive data engine that moves only the
change: $() wraps a value, operators derive incrementally-maintained
views, sinks deliver consolidated deltas, render binds to the DOM. Work
scales with the change, not the dataset.
Like crossfilter's incremental aggregation meets Solid-style fine-grained DOM — for UIs over large, fast-changing data: dashboards, analytics, grids, live feeds.
0 dependencies · type-stripped, not bundled · SCHEDULE_VERSION 4 · 330+ tests · MIT
one write, one path
Almost every reactive system re-runs the whole computation when a single
value changes. data is a reactive data engine that moves only the
change: $() wraps a value, operators derive incrementally-maintained
views, sinks deliver consolidated deltas, render binds to the DOM. Work
scales with the change, not the dataset.
Like crossfilter's incremental aggregation meets Solid-style fine-grained DOM — for UIs over large, fast-changing data: dashboards, analytics, grids, live feeds.
trades.t3.qty.update()
| key | sym | side | qty | px |
|---|
— dependencies · type-stripped, not bundled · SCHEDULE_VERSION — · — tests · MIT
change & derivation
A write is a delta — three row verbs, add · update ·
remove, in one batch envelope — and every operator consumes and
emits exactly that. So a filter, a count, a grouping stays live for the price
of the change, not the table. data keeps the table and its stream as
one thing: wrap a value in $(), chain operators, bind to the DOM.
Like crossfilter's incremental aggregation meets Solid-style fine-grained DOM — for UIs over large, fast-changing data: dashboards, analytics, grids, live feeds.
0 dependencies · type-stripped, not bundled · SCHEDULE_VERSION 4 · 330+ tests · MIT
The same engine you pick above, doing two jobs. Top: one order-book tick
stream through the same depth-chart-and-ladder viz — every tick is a deep-path
write, and data moves a couple of bucket counters per delta
(length(bucket) is aggregate-decomposable: O(1) per change); every
peer re-walks all 150,000 orders once per frame (O(N)), and the green baseline
is data against the 16 ms budget. Bottom: the same library filtering
231,083 flight rows as you brush — a brush is one
filters.get(name).update([lo, hi]), and between walks
only the rows entering or leaving the window. Every implementation is the real
library doing the idiomatic thing; every number is this machine's.
Two workloads, one engine, nothing to hide behind. Left: a 150,000-order book
ticking at the rate you set — each tick is one deep-path write, all of a frame's
ticks land in one consolidated commit, and length(bucket)×2,
filter→length and avg move a counter apiece, so the frame
cost is the flat line under the 16 ms budget. Right: 231,083 flight rows under four
brushes — a brush is one filters.get(name).update([lo, hi]);
between walks only the rows crossing the boundary, and
intersect and length(fn) fold the deltas. The figures
are this machine's, measured as you watch.
render(list(trades))drag the number — every move is trades.t3.qty.update(v)
const trades = $({ t3: { sym: 'NVDA', qty: 500, … }, … })
batch(() => samples.forEach(v => trades.t3.qty.update(v)))
trades.sink({ init, apply })—
trades.sink({
init: snapshot => show(snapshot),
apply: b => show(b), // { seq, origin, rows, order, scalar }
})
runtime().onCommit(c => …)—
const buys = trades.filter(t => t.side === 'buy')
const largest = buys.za('qty', 2), typical = buys.median('px')
const volume = buys.sum('qty'); runtime().onCommit(trace)
two sinks · one batchrender(list(largest))WireBatch → ingest(replica)render(ol, list(largest, t => li(b(t.sym), t.qty))) wireSink(largest, wb => ingest(replica, wb.records))
You dragged one number and watched it travel. The write became one delta carrying its
path and its previous value; it flowed through exactly the views that read it —
typical never lit, because a median over px learns nothing from a
qty — and two sinks received the same consolidated batch: one rendered a
list with a single DOM instruction, one replayed it into a replica that stayed identical.
That is the whole engine — three row verbs, three order verbs, one envelope, and work
proportional to the change.
The same engine you pick above, doing two jobs. Left: a -order book
ticking at the rate you set — every tick is one deep-path write, a frame's ticks land
in one commit, and data moves a bucket counter apiece
(length(bucket) is aggregate-decomposable: O(1) per delta) while every
peer re-walks the whole book once per frame (O(N)); green is data against the
16 ms budget. Right: the flight rows under four brushes — a brush is
one filters.get(name).update([lo, hi]); between walks only
the rows crossing the boundary, intersect and length(fn)
fold the deltas, and the ms/brush is that write, settle included, timed on every
pointer move as you drag — under a peer, the same moves through that library's
filters and histograms, in the same window. Every implementation is the real library
doing the idiomatic thing; crossfilter's index is incremental too, and its walk is a
tighter loop over the same crossing rows.
A change is small. You set one field on one row. Yet most reactive stacks respond by re-running everything downstream — re-filtering the whole list, re-summing every number, diffing an entire tree — and then trusting a memoiser to paper over the waste.
data takes the change literally. A write carries its exact path, and each operator in the chain does work scoped to that path alone:
trades.t3.qty.update(380) // one delta · path ['qty'] · prev 500
No diff pass. No list re-render. No selector re-run over 231,083 rows. The cost of an update is the length of the path it travels — which is why the race above holds at any rate, and why it scales to datasets that make recompute-everything systems stall.
A change is small. You set one field on one row. Yet most reactive stacks respond by re-running everything downstream — re-filtering the whole list, re-summing every number, diffing an entire tree — and then trusting a memoiser to paper over the waste.
data takes the change literally. A write carries its exact path, and each operator in the chain does work scoped to that path alone:
trades.t3.qty.update()
sourcetrades · rows
row t3 · qty
one call · one deep path
batchseq · rows
one row verb · carries prev
No diff pass. No list re-render. No selector re-run over the whole table. The cost of an update is the length of the path it travels — which is why the race above holds at any rate, and why it scales to datasets that make recompute-everything systems stall.
One source, many derived views. Wrap a collection in $() and
chain operators; each returns a view maintained incrementally from the deltas
its source emits — the same three row verbs and three order verbs, in and out.
Every row below runs live off one shared $(trades) feed — pick a
side, drag a bound, watch the top-5 reshuffle, see the groups recount as
qty and px tick.
$(trades) · 12 rows · seq 0 · —
trades.filter(t => t.side === 'buy')O(Δ)trades.map(t => t.qty * t.px)O(Δ)trades.gt('qty', 300)O(Δ)trades.between('px', [100, 200])O(Δ)const bounds = $([100, 200]) · trades.between('px', bounds) · a drag is one write — bounds.set(0, 100) — and the view re-bounds
buys.intersect(large)O(Δ)buys.except(large)O(Δ)trades.avg('px')O(Δ)—
trades.length(t => t.sym)O(Δ)trades.group(t => t.sym)O(Δ)trades.lengthBuckets(t => band(t.qty))O(Δ)trades.some(t => t.qty > 1000)O(Δ)buys.za('qty', 5)O(Δ log n)trades.az('px').top(3)O(Δ log n)trades.reverse()O(n)trades.max('px')O(Δ log n)trades.reduce(add, remove, 0)O(Δ)trades.distinct(t => t.sym)O(Δ)trades.to(v => Object.keys(v).length)O(n)trades.median('px')O(Δ log n)—
trades.tap(c => log(c))O(1)trades.keys()O(1)Every row is the real operator on a live feed; the cost column is the class of work a change costs, not a benchmark. 33 operators in 4 categories — exactly what exportContract() reports.
One source, many derived views. Wrap a collection in $() and
chain operators; each returns a view maintained incrementally from the deltas
its source emits — the same three row verbs and three order verbs, in and out.
All below run live off one shared $(trades) feed — pick any
operator and its result ticks in the panel as qty and px move.
$(trades) · rows · seq · —
one row in, one row out — the delta passes straight through
maintained from the delta alone — prev out, next in
needs an order or the whole set — an index does the work
side effects and projections — no view state of their own
the multiple is data's cost as the unit; peers re-walk the source on every change — nothing prints here until it has been timed in this tab
Every chip is the real operator on a live feed. The cost word is the class of work a change costs; the multiple is that cost against eight peers doing the idiomatic thing — re-walking the source per change — measured in this tab on demand, and the figure on the panel is the peak, the widest gap. operators in categories — exactly what exportContract() reports.
one package, nothing bundled — dist/ is the source type-stripped file-for-file, one module per file. In the repo it stays source-served: Node runs the .ts with --experimental-strip-types; this page runs the same files, type-stripped the same way.
api/the public surface — $(), the verbs, the operators, batch, runtime. The one entry you import.seam/the outside-world boundary — ingest, lane, wireSink, mountrender/ + jsx/el / list / text, the HTML / SVG builders, classic + automatic JSXdevtools/the inspector panel + DOM overlay — mounts on demand, never autoimport { $ } from 'data'
const trades = $({
t1: { sym: 'AAPL', side: 'buy', qty: 200, px: 187.5 },
t2: { sym: 'MSFT', side: 'sell', qty: 120, px: 412.1 },
t3: { sym: 'NVDA', side: 'buy', qty: 500, px: 118.4 },
})
const buys = trades.filter(t => t.side === 'buy')
const largest = buys.za('qty', 2)
const volume = buys.sum('qty')
trades.t3.qty.update(380) // → update t3 .qty 500 → 380
import { render, el, list, batch } from 'data'
const ol = document.querySelector('ol')
render(ol, list(largest, t =>
el('li', null, el('b', null, t.sym), t.qty)))
batch(() => {
trades.t3.qty.update(380)
trades.t1.qty.update(260)
}) // one commit, one paint
JSX works too (jsxImportSource: "data" — classic h or the automatic runtime).
render() does per-key surgical updates — element identity and focus survive across changes.
one package, nothing bundled — dist/ is the source type-stripped file-for-file, one module per file. In the repo it stays source-served: Node runs the .ts with --experimental-strip-types; this page runs the same files, type-stripped the same way.
devtools/ ships an inspector panel and a DOM overlay. The
panel is a right-edge dock: the live graph as a Tree or a
DAG, every commit's deltas as they settle, and the settle time
of each node. It mounts on demand — never on its own — and the console
API alone ($.inspect, $.graph,
$.trace, $.profile, $.cascades)
doesn't pay the panel's bytes. The overlay walks the other way:
$.fromDOM($0) picks any bound element back to the view
that rendered it; $.highlight(view) lights up its rows.
Everything the panel shows is a projection of two runtime primitives —
runtime().graph() and runtime().onCommit(c => …)
— so the numbers are the kernel's own, measured only while a hook is live.
devtools/ ships an inspector panel and a DOM overlay.
The panel is a right-edge dock — this is it, drawn inline over a
five-node graph this section runs on the page's engine, with a
scripted writer feeding it real writes. devtools/index.ts
alone — the graph, trace and profile helpers this page imports —
never loads the panel; the data/devtools entry also
docks it on import unless the URL says ?nopanel, and
devtools/panel/ mounts only through mountPanel().
runtime().graph()onCommit — measured only while a hook is live$.highlight(view) lights its rows
Console-only: $.inspect(h) · $.graph() ·
$.trace(fn) · $.profile(fn) · $.cascades(fn).
$.trace subscribes for the duration of fn and disposes after — tracing has zero steady-state cost.
$.devtools.panel.open() / .close() drive it from the console in any page that imports data/devtools — $.inspect(h) alone stays in the console.
Everything runs under an executable timing contract — SCHEDULE.md,
SCHEDULE_VERSION 4: eleven numbered clauses, each pinned by a test, and
the same test file runs in the consumer's CI against data HEAD.
AggregateError)1 test✓init(snapshot) then only deltas (a mid-batch attach defers to the commit)2 tests✓add on an existing key is illegal; consolidation rules2 tests✓coalesce('microtask' | 'frame') — sugar, not semantics1 test✓pathCopy / pathDelete (SCHEDULE_VERSION 2, fero W3)4 tests✓conform(node) attaches a legality checker and a replay sink to any view: every commit is re-folded and the fold must equal the view.
Everything runs under an executable timing contract — SCHEDULE.md,
SCHEDULE_VERSION —: eleven numbered clauses, each pinned by a test, and
the same test file runs in the consumer's CI against data HEAD.
const trades = $({ t3: { sym: 'AAPL', qty: 500 } }), got = []
trades.sink({ apply: b => got.push(b) })
trades.t3.qty.update(380) // a bare write = a synchronous batch of one
print(`seq ${got[0].seq}: ${show(got[0].rows[0])} — delivered before update() returned`)
// →
trades.t3.qty.update(380) // Object.is-equal at the leaf
print(`nothing emitted: still ${got.length} batch, seq still ${runtime().seq}`)
// →
·
const rows = $([{ sym: 'AAPL', qty: 200 }, { sym: 'NVDA', qty: 90 }]); let fired = 0
rows.sink({ apply: b => print(`at batch close: the sink fires once (seq ${b.seq}) · fired ${++fired}`) })
batch(() => {
rows.insert({ sym: 'MSFT', qty: 40 })
print(`mid-batch: rowCount() → ${rows.rowCount()}, the read sees the write · sinks fired: ${fired}`)
})
// →
// →
·
const rows = $([{ qty: 50 }, { qty: 120 }, { qty: 300 }])
const big = rows.filter(r => r.qty > 100), n = big.length() // height 1, height 2
const name = { [rows[node].id]: 'source', [big[node].id]: 'filter', [n[node].id]: 'length' }
const hook = runtime().onCommit(c => print(`seq ${c.seq} settled ${c.nodes.map(x => name[x.id]).join(' → ')}, each once`))
big.sink({ apply: () => print(`the sink fires after settle: n already reads ${n[value]}`) })
rows.get(0).set('qty', 500); hook.dispose() // 50 → 500 joins big; n 2 → 3
// →
// →
·
const rows = $([{ qty: 1 }, { qty: 2 }]), was = rows.rowCount(); let sibling = 0
rows.sink({ apply() { throw new Error('boom') } })
rows.sink({ apply: () => sibling++ })
try { rows.insert({ qty: 3 }) } catch (e) { print(`${e.constructor.name} after the drain: ${e.errors.length} sink failed (${e.errors[0].message})`) }
print(`state intact: rowCount ${was} → ${rows.rowCount()}, the write committed · the sibling sink still fired ${sibling}×`)
// →
// →
·
const a = $({ n: 1 }), b = $({ n: 0 }), log = []
a.sink({ apply: c => { log.push(`seq ${c.seq}: a.n = ${a.n[value]}`); b.n.update(a.n[value] * 2) } })
b.sink({ apply: c => log.push(`seq ${c.seq}: b.n = ${b.n[value]}`) })
a.n.update(21) // an effect that writes
print(`${log.join(' · ')} — the effect's write became the NEXT commit`)
// →
·
const mine = Symbol('peer-A'), replica = $({}), calls = { mine: 0, other: 0 }
replica.sink({ origin: mine, apply: () => calls.mine++ }) // declares what it must not hear
replica.sink({ apply: b => { calls.other++; calls.echo = b.origin === mine } })
ingest(replica, [{ t: 'add', k: 'a', v: { n: 1 } }], { origin: mine })
print(`peer-A's own sink: ${calls.mine} applies, skipped · the other sink: ${calls.other} apply, batch.origin === mine: ${calls.echo}`)
// →
·
const rows = $([{ qty: 1 }, { qty: 2 }]), got = {}
const sink = { init: s => (got.init = s.size), apply: b => (got.first ??= b.seq) }
batch(() => { rows.insert({ qty: 3 }); rows.sink(sink); rows.insert({ qty: 4 }) }) // attached mid-batch
print(`init saw ${got.init} rows — the settled batch (seq ${runtime().seq}), both inserts · applies so far: ${got.first ?? 'none'}`)
rows.get(0).set('qty', 9)
print(`apply first fired at seq ${got.first} — the NEXT commit`)
// →
// →
·
const rows = $([{ qty: 1 }]), t = $({ t: { qty: 500 } }), got = []
rows.sink({ apply: b => got.push(b) }); t.sink({ apply: b => got.push(b) })
batch(() => { const k = rows.insert({ qty: 2 }); rows.get(k).remove() }) // add + remove, one batch
print(`add + remove annihilated: ${got.length} batches delivered, no delta`)
batch(() => { t.t.qty.update(1); t.t.qty.update(2); t.t.qty.update(3) }) // three writes, one key
print(`${got.length} batch, ${got[0].rows.length} delta: ${show(got[0].rows[0])} — first prev, last value`)
// →
// →
·
const q = () => [[{ n: 1 }], [{ n: 2 }]], commits = { sync: 0, co: 0 } // two chunks that settle in one tick
const feed = (c = q()) => ({ [Symbol.asyncIterator]: () => ({ next: () => Promise.resolve({ done: !c.length, value: c.shift() }) }) })
const sync = fromAsync(runtime(), feed()), co = fromAsync(runtime(), feed(), { coalesce: 'microtask' })
handleFor(sync.source).sink({ apply: () => commits.sync++ }); handleFor(co.source).sink({ apply: () => commits.co++ })
while (sync.status() === 'pending' || co.status() === 'pending') await new Promise(r => setTimeout(r))
print(`default: ${q().length} chunks → ${commits.sync} commits · coalesce('microtask'): ${commits.co} commit · same ${co.source.rowCount()} rows each`)
// →
·
const t = $({ t3: { meta: { tag: 'old', book: null } } }), got = []; t.sink({ apply: b => got.push(...b.rows) })
t.t3.meta.tag.remove() // nested remove: first-class (pathDelete)
t.t9.meta.tag.remove() // absent target: idempotent no-op
t.t3.meta.book.depth.update(3) // null intermediate: vivified (pathCopy)
print(`${show(got[0])} · deleted: ${got[0].deleted} · row.meta = ${JSON.stringify(got[0].row.meta)}`)
print(`${got.length} deltas for three writes — the absent-target remove emitted nothing · ${show(got[1])}, vivified from null`)
// →
// →
·
const src = $({})
ingest(src, [{ t: 'add', k: '\x00key', v: { px: NaN, at: undefined } }])
const row = src.get('\x00key')[value]
print(`${src.rowCount()} row · key ${JSON.stringify('\x00key')} live: ${row !== undefined} — a NUL byte is just a key`)
print(`px is NaN: ${Number.isNaN(row.px)} · row owns 'at': ${'at' in row}, at === undefined: ${row.at === undefined} — both first-class`)
// →
// →
·
Every proof above is a real program run in this tab against the page's engine — the code is literal, print() writes its // → line, show(delta) formats a row delta as op key .path prev → next. conform(node) attaches a legality checker and a replay sink to any view: every commit is re-folded and the fold must equal the view.
Every commit can leave the runtime as a WireBatch — JSON-safe, replayable in
application order — and enter another one through ingest() or the hot
lane(). The two runtimes below are joined by nothing but that wire; write on
either side and the other side receives exactly the change.
wireSink(here, cross, { origin: fromWire })// nothing on the wire yet
wireSink(there, cross, { origin: fromWire, initial: false })origin tokens: each side ignores batches that carry its own origin — clause 6.
One consumer today, and the essay that explains the idea. fero runs on the engine in production; the essay is why the engine has the shape it has; the proof and the tooling are the same source, running in this tab.
Its substrate imports one entry — ../../data/api/index.ts — and
uses seven names from it. Its npm test runs data's conformance
suite against data HEAD, so a clause edit that forgets its
SCHEDULE_VERSION bump fails in both repos before it ships. The
lane record shape is a wire-stability surface: fero's kernel records enter
the engine verbatim, and that shape does not move.
../../data/api/index.tsconformance/schedule.test.ts · fero's CI · against data HEAD{ type: 0|1|2, key: [rowKey, …fieldPath], value, at? }An eight-section essay on the duality at the core of the engine: a table and its stream of changes are the same thing in two forms, so a view — a filter, a count, a grouping — stays live for the price of the change, not the table — O(Δ), not O(N) — flowing only through the views that read it, down to a single minimal DOM instruction. The readability of recompute, the cost of hand-tuned deltas, and none of the bookkeeping.
The executable contract — one version assert and seventeen clause tests — running here through node:test / node:assert shims over the same source the page serves. The same file rides in fero's CI.
The devtools panel: the live graph as Tree / DAG, per-commit deltas, per-node settle ms, and a DOM overlay for the render layer. devtools/index.ts and devtools/dom.ts, the same source the engine ships. Mounts on demand — never on import.
examples/ returns with the v4 gallery — crossfilter, chat, to-do.
One consumer in production today: fero, a replicated, durable data kernel — deterministic core, real-TCP and real-WAL shells — whose every write and replica is a data source under this contract. Its CI runs data's conformance suite against data HEAD, so a clause that changes without its version bump fails there before it fails here.
A deterministic kernel proven both ways from the same bytes — a seeded simulation drives a thousand chaos trajectories a run, and the same kernel runs over real TCP and a real write-ahead log. Per-cell last-writer-wins replication heals tail loss from partitions and dial races, membership is a committed view, and durability carries a restore-from-WAL proof: kill a whole cluster at any seed and no write it confirmed to disk is lost.
fero →data v4conformance/schedule.test.tsin fero's CI · against data HEADSCHEDULE_VERSION —examples/ returns with the v4 gallery — crossfilter, chat, to-do.