Long Documents in XR
AI Summary: This session centred on the evolving spatial interface of Author for visionOS, using Douglas Engelbart’s 1962 paper as a large-scale test document. The conversation moved from concrete design questions — how section cards should look, how selection states should be rendered, how visual differentiation by category or search could work in an immersive space — into a deeper conceptual inquiry about the nature of supplementary knowledge in digital documents. The group explored what endnotes, footnotes, and annotations become when freed from the constraints of paper, seeking both a functional model and appropriate terminology for “the extra stuff” that surrounds a primary text when it inhabits three-dimensional space.
Preview video above. Session below:
AI: Highlights & Insights
The toolbar interaction in visionOS, though built from simple sequential menu taps rather than the radial or gestural menus the group once envisioned, turns out to feel gestural in practice. Repeated use trains the hand into fluid spatial motions — tap-left for layout, tap-right for selection — so that what appears to be a mundane bar becomes a learned kinesthetic pattern. This was described as an “annoying surprise” because it is prosaic in design yet effective in use, undercutting the assumption that spatial interaction must look spectacular to work well.
A recurring tension surfaced between designing for freedom and designing with opinion. The observation was made that the best email advice — keep it under 300 words, ask only one question — is extremely opinionated, yet effective precisely because it constrains behaviour. This was extended to spatial document editing: imposing a maximum section length is not a limitation but an encouragement to create more semantic handles, echoing Robert Horn’s structured-text philosophy of titling every paragraph.
The distinction between reading and authoring was challenged. The argument was made that reading and writing are fundamentally the same cognitive activity — assembling a web of ideas from heterogeneous sources (personal experience, film, philosophy) into a coherent narrative. This reframing suggests that a tool designed only for one mode is designing against the grain of how knowledge actually works.
The concept of “extra stuff” — endnotes, annotations, rationale, mood boards, side-notes — emerged as a central design problem for spatial documents. The group recognised that the entire vocabulary of footnotes and endnotes is an artifact of paginated print, and that digital environments collapse the distinction between them. What matters is the interaction model: how does a user summon, dismiss, and spatially arrange supplementary knowledge relative to a primary text object? Several metaphors were proposed — knapsack inventory, collapsible mood board, shelf, call-out box — but no single term was adopted.
A connection was drawn to Keynote as an underappreciated spatial authoring tool. The observation that Keynote slides lack adequate space for rationale — presenter notes serve one function, sticky notes another, and neither is satisfactory — opened the idea that slides arrayed in spatial XR could have endnote-like satellite nodes that appear on interaction, creating a citation-density view across a presentation.
The experience of serendipitous proximity in physical libraries — finding a revelatory book shelved next to the one you sought — was held up as a design aspiration. The forced adjacency of a physical dictionary, where searching for “ubiquity” leads you to encounter “UBI,” was cited as evidence that older information architectures still offer benefits that purely search-driven digital systems sacrifice.
The rendering constraints of visionOS — where only pure black-on-white and white-on-black text remain legible, and any tinted background degrades readability — were noted as a significant material constraint that shapes all visual differentiation decisions for the Author map view.
A direct question was posed to Claude during the session: what might Ted Nelson or Doug Engelbart call a digital endnote? The response noted that Nelson would likely reject the premise, and surfaced several candidate terms — “tuck” (from origami, a fold hidden inside the form), “gloss” (commentary on a passage), and “scholion” (the ancient Greek term for a bound commentary by a later hand). The group also revisited the existing term “furl and unfurl” and considered “fold-out” as an inversion of the newspaper fold — expanding rather than concealing.
AI: Important
During the session, Claude was directly consulted for terminology. The query asked what Ted Nelson or Doug Engelbart might call a hidden explanatory link — in effect, a digital endnote that is concealed until summoned. The response drew on Nelson’s Xanadu philosophy (he would reject the premise of endnotes), Engelbart’s view-control and statement-level visibility concepts, and suggested “tuck,” “gloss,” and “scholion” as candidates. The term “stretch text” from Nelson’s work was also surfaced.