Further explorations into figure and ground.
AI: Summary
This session of the Future Text Lab continued the month’s theme of figure and ground — foreground and background — by asking how, in XR and spatial computing, a person decides what to bring forward to work on and what to leave in the periphery when potentially every digital thing they own is available at once. The conversation ranged across the art-historical roots of the term, the cognitive science of reading and decision-making, the internal structure of arguments and citations, and the very practical problem of how a richly formatted written piece might be re-presented in space. A participant’s published essays and a spatial writing tool, Author on macOS served as live test case at some points.
The strongest reframe of the session was treating “everything available at once” as the core failure mode rather than the promise of XR — the “Veruca Salt problem.” The real design challenge is selective, on-demand access to aspects of a thing, not total simultaneous display, which connects directly to the question of how a place or a history could surface multiple viewpoints without bombarding the viewer with all of them at once.
A genuine tension surfaced around the provocation “is writing thinking, or is writing the product of thinking?” — with the counter-position that the hardening, flattening and linearisation of writing crystallises something a more fluid medium cannot. This carried a useful analogy: you cannot fully know a software experience, a painted room, or even a thought until you externalise and build it, which is itself the argument for constructing XR experiences rather than only theorising about them.
The cost of choosing emerged as a unifying lens. Decisions consume real energy; “flow” decisions inside a game are cheap because one is already engaged, whereas intellectual decisions demand oxygen, water and caffeine. The same branching choice is an opportunity when curiosity is high and a barrier when it is not — captured in the line “nobody likes to be chased down the rabbit hole.” This reframes interface design as the management of decision cost and explains the durable appeal of authored linear narrative and of shortcuts like CliffsNotes and AI summaries.
AI chat logs were recognised as a newly available artifact of process, making Vannevar Bush’s associative trails visible in a way previously possible only through diaries and notes — the gap between Madison, who recorded everything, and Jefferson, who recorded little. Requiring students to submit their chat logs exposes the structure of their struggle, and layering this record (the conversation, the record of it, and a teacher’s reading of that record) is itself a new species of figure and ground.
A shift from prediction to affordance was articulated: rather than an all-knowing recommendation system synthesising round-the-clock tracking, which would likely fail, the better move is to make the medium itself more conducive to cross-pollination among heterogeneous materials — text, video, podcasts, art — so that “the medium of interaction can become the medium of cognition,” offloading some of what the brain’s unseen “dark matter” used to carry.
Linear text was reconceived as decomposable in space: a document is a set of arguments an author connected in a chosen order, and in 3D it might be deconstructed and its links made dynamic and reader-relative — what a given concept means for this reader — versus today’s opaque static hyperlinks that drop you into a disconnected window, a contrast drawn explicitly against the newer context-aware Siri.
Spatial stability was identified as the actual source of value, the “mind palace” or “cathedral”: one returns to a chapter the way one reaches by feel for a book on a familiar shelf, so layout position must persist — and AI’s habit of “exploding the nodes” with every fresh summary destroys exactly that stability. The working term “context” surfaced for this persistent, backgrounded knowledge sculpture, with general agreement that the name is too bland.
The physiology-versus-culture question about reading format proved unresolved and productive: sharp reading is confined to a tiny foveal region, roughly an A4 sheet at about sixty centimetres, so everything peripheral is necessarily “ground,” with motion rather than detail being the periphery’s real strength. Whether this format is dictated by physiology or merely learned from paper drew open disagreement — “the paper has learned us.”
Friction was treated as a design variable rather than a nuisance. Tiny unanticipated frictions — headset weight, the iris-scan login spinner, lingering social awkwardness — measurably reduce willingness to enter XR, and there is good friction worth preserving alongside bad friction to remove. Relatedly, the spatial “wow” of moving information by hand thrills and then fades to “that was cool, but why,” so the real target is “the wow and the work” — a realisation that has narrowed one tool’s strategy from solving XR wholesale toward doing just enough to earn Apple‘s promotional interest.
The under-examined direction of data drew attention: discussion usually centres on document data coming in, but the harder question is the data about us going out — gaze, mood, attention — potentially valuable for long-term self-knowledge yet a clear privacy minefield. Apple withholding eye-tracking from developers was seen as at once protective and limiting.
Navigation and overview were distinguished as two separate reading functions — returning to a half-remembered passage versus grasping the spine of an argument and what supports it — alongside the recurring tradeoff, likened to game design’s accuracy-versus-playability, that a concept map can be comprehensive or readable but rarely both. A navigation controller that moves through content rather than merely through space was proposed, prompted by Substack‘s section anchors.
A concrete interaction model was offered: mirroring the Forth programming language in VR with an implicit, gaze-driven data stack, where the temporal sequence of what one looks at supplies the operands and eliminates explicit drag-and-drop, to be sketched for a future session.
Comedy was framed as connection-making — the comedian stripping away the facade to reveal what is underneath is doing the same act as the writer — which the group connected to its own AI-assisted songs as simply another way of looking at the same material.
A sober read of the field also appeared: virtual worlds are dying as revenue-poor point-tools in an “XR winter” even as lightweight glasses and capable headsets quietly advance, with the long-unrealised killer app imagined as VR-to-real-life commerce such as ordering a virtual outfit to wear for real.