AI: Summary
Two newcomers reshaped this session. Kathryn Jones, who describes herself as a non-academic with a passion project, has spent four years building what began as two hundred quote slides from Annie Murphy Paul’s The Extended Mind into an app she calls Insatiable Mind — a way of turning books, images and lived experiences into artifacts she can move through, so that a Caravaggio at the Met suddenly connects to a church she visited on the Appian Way. Art Domi, a Houdini technical artist from Kosovo, arrived by way of an AI agent that found the community for him, carrying the observation that software developers are the last creative industry still working in files and folders rather than on a spatial canvas; his Tesser prototype was demoed in the second half. Brandel Zachernuk gave the fullest account yet of his W3C work on spatial web standards and confirmed that the model element now ships natively across Apple’s OS 27 platforms, while Frode demonstrated the same element being pulled out of an ordinary EPUB and into the room in Origami Text.
The centre of the session was a question Frode put to the group after Kathryn’s introduction: what is the small, shareable unit of thought that is not a document — an annotation, a note, a concept, something we could arbitrarily call “amber” — and where does it live? That produced the richest exchange of the day, including a real and only partly resolved disagreement between Frode and Mark Anderson about whether such a thing needs a “there there,” and a closing convergence, through Art’s demo and Brandel’s response to it, on level of detail as the organising principle for spatial work.
AI: Main Topic
The thing without a name. Kathryn’s account of standing in front of a painting and wanting to attach a memory to it prompted Frode to ask what kind of object that is. It is not a document — a document frames a point of view at a point in time — and “annotation” already carries too many meanings, from underlining in a margin to a separate paper written about another paper. Everyone agreed it is a tiny piece of knowledge that may or may not refer to something else and needs to be shareable. Nobody could name it, and the group treated that gap as both absurd and an opportunity: as Frode put it, without an instantiated form we are only fantasising, and dreaming is more useful than fantasising. The discussion moved from what it is, to where it lives, to how much structure it should carry, and ended by revealing that the participants imagine such objects in genuinely different ways.
AI: Highlights
Kathryn Jones’s introduction was the emotional anchor of the day. She described the interoceptive stress of finishing a book she has underlined and annotated, knowing the forgetting curve will take most of it, and how Annie Murphy Paul’s argument for turning what you learn into artifacts became a four-year project: quote slides on biophilic backgrounds rotating on a big office screen, then, after teaching herself Python and cloud architecture, a full app that now holds books, images and experiences. She is taking it to Greece next week. The line that drew the strongest response was that using AI heavily on the back end has left her, at fifty-seven, doing more of the thinking herself, not less. She was speaking to Annie Murphy Paul later the same day; Frode noted Paul is in one of the Future of Text books and that Barbara Tversky will be at the November symposium.
Art Domi’s introduction traced a path from architecture and spatial planning, through six years of procedural work in Houdini, to a conviction that “the future of work is spatial, without any doubt.” Every other complexity-handling industry — film, visual effects, games — has moved to canvases; developers have not. He found the community’s work, and Barbara Tversky’s lectures, through an agent he built, and described the experience as finding his own people.
Brandel Zachernuk’s introduction filled in what many regulars had only partly known: a career from failed video games through advertising computer graphics to Apple, then being pulled into the team that builds WebGL and WebGPU, and now proposing spatial web standards through the W3C. His stated aim is that spatial content be documents rather than hand-coded applications, adaptive from a single iota of information on a watch up to an entire room. He does not like headsets and does not use AI much; he sees VR as the least-worst way of envisaging a ubiquitous computing substrate, and text as the foundational component because “the more you have to do to present something, the more brittle it becomes.” He recounted electrical engineers at Apple finding his internal talk on embodied cognition and building a five- or six-dimensional pinchable parameter surface for wireless-charging circuit design — exactly what he had meant. He speaks only for himself, not his employer, and goes to Dublin later this month for the W3C conference to present spatial hypertext and spatial CSS.
Frode’s demo: a single-article EPUB in Origami Text on Vision Pro, laid flat on the table, then a 3D model pulled out of the page and into the room. It is a model element in a normal EPUB that also opens in Apple Books. He then showed this year’s Hypertext conference papers as EPUBs with citations arranged behind the open paper, controlled by buttons on the non-dominant hand. The conversion pipeline from ACM’s LaTeX was, in his words, the worst detail work he has done in years — but it works, and any of those proceedings downloaded as EPUB are now Origami Text documents capable of holding 3D models, even though no author has yet added one.
Tom Haymes asked why the buttons could not sit on the arm itself. Frode explained that arms are always foregrounded in the render so anything on them is occluded, and that in practice the non-dominant hand can hang by your side while you pinch without looking. Peter Dimitrios connected this to the light-pen failure: anything that makes you hold your arm up does not survive. Brandel added that “it’s going to be decades for us to discover what the heck to do with it — it’s so alien.”
That opened an extended exchange on eyes and bodies. Brandel reported that Meta’s newly announced glasses will give developers full eye tracking, for better and worse. Tom raised his amblyopia — one eye that wanders and that his brain does not use — and asked how eye-tracking calibration would cope; Brandel offered to check with people who have tried it and suggested the calibration step could detect it. Brandel then laid out why near-field work is the most uncomfortable in current headsets: every commercial device except the two-plane Magic Leap has a single focus distance, tuned for roughly seventy-five centimetres and beyond, which forces the vergence–accommodation conflict that makes many people, disproportionately women, sick. Meta has shown a varifocal surface, but nobody has a line of sight to shipping one. Tom recalled feeling unsteady for twenty-four hours after an early Oculus session. Frode’s counterpoint to Tom’s holodeck wish was that in the walk-around experiences the Lab now builds, “you are the 3D” — a globe you can move too easily becomes a toy, one you have to walk around becomes something else.
On EPUB adoption, Frode put the question directly to Brandel: what would it take for Safari or other browsers to render EPUBs natively? Brandel’s answer was that since EPUB is HTML in a zip, it is the kind of thing you would have to do work to stop happening on WebKit, with DRM the obvious complication, and that a Chrome-side reader should be trivial for Claude to build; his advice was to file official feedback, which does reach him and his colleagues. Frode noted his own experiments confirm there are many routes but none is drag-and-drop. Brandel’s own spatial demos, once on Glitch, are gone — “link rot is real.”
Mark Anderson then made the case that the academic use of EPUB may need a fork of the standard rather than subversion from inside — a reversal of the advice he gave Frode when he first saw Origami Text. EPUB is a toy of the book trade, excellent for getting a self-published novel out of the door, and its security posture and single-file conventions do not fit a document that needs multiple files, space-efficient bitmap images and stricter rules. What it does give, and what keeps it closest to what the Lab needs, is the separation of text from structure: a text stream with no typographic garbage, none of PDF’s “page 64” landing in the middle of a word, which is what machine learning actually wants. He noted there are entire genres of study devoted to removing typographic artifacts from OCR text; the better method is to put no garbage in.
The annotation discussion ran through several positions. Art proposed annotation as a lens or a layer, like the transparent sheet laid over a drawing in architecture school, potentially perspective-aligned to the viewer’s position and separable from the project layout beneath it. Mark recalled his conversation with Gabo Beaumont about federated corpora and observed that annotations are just objects with IDs: as long as they are labelled properly they can live wherever they need to be found, and the group had been unintentionally guided by the imagined physicality of where they sit. Tom argued that traditional annotation was segregated into private marginalia and public commentary only because paper made doing both hard, and that footnotes are formalised annotations that have lost the thinking behind them; his own interest, which he offered as a future talk, is “text as thought” — mapping thinking into text rather than building text around thinking, prompted by seeing his students’ trails of thought in their chat logs, the thing Vannevar Bush described in 1945. Peter distinguished an annotation that refers to a span from a comment on a document as a whole, and Frode confirmed the latter is already implemented in Seed and Origami Text precisely because it needs no internal addressability. Mark’s warning was against premature formalisation: we create false categories for comfort, and the less structure we impose on annotations the more we will get out of them; record the connection and its granularity and the categories will fall out. Frode accepted the warning but held that there must still be a there there — the thing has to be stored in some format, and if you send someone a blue dot you must send “blue dot” and a location. Tom’s reply was that the object does not matter; what matters is the connection to you — why this painting and not the one behind Mark. Frode’s practical instance of the nameless object is the JSON letters he and Eric Rangell are exchanging: each letter a file, a lightweight book that is not an email.
The disagreement between Frode and Mark sharpened when Mark introduced aphantasia. He can happily imagine a space containing a hundred things without any sense of what they are, and suspects the distribution of mental imagery runs more toward the abstract end than we assume; Frode, he suggested, sits at the realism end, for whom a thing without a “there” is hard to conceive. Frode challenged him to build an experience with nothings in space. Brandel resolved it with a ladder of abstraction: you never need to transmit the blue dot itself, only the information that lets a viewer construct a view containing it, and that information can sit anywhere from “baked onto a JPEG” to “this user hasn’t seen this picture, so it should have a dot.” Mark agreed that was exactly it: the only difference between them is how early in the process different people need specificity.
Art’s demo of Tesser showed a spatial workspace where a project’s documents render at different levels of detail depending on state, agents (he mentioned dropping in a small model such as Claude Haiku) can generate an initial project structure, and the layout itself becomes a kind of handwriting — outputs bottom right, inputs top left. His argument was that now software can be generated in minutes, structure matters more than code, and structure must be navigable from many perspectives, macro and micro. He also proposed that the annotation layer could itself be layered in depth: the sketch you made while explaining, behind it an agent’s live notes, behind that an agentic summary of why you drew that arrow. Brandel’s response introduced level of detail as the frame — the fluid transitions between scales that maps do so well because “space is fractal,” and that LLMs may be able to provide for purely digital material by summarising at each scale. Mark added that what the Lab actually does, in XR or on screen, is closer to two-and-a-half D, and that the half D is where level of detail has been happening all along without being named; full 3D risks showing more detail than anyone wants. He described a non-technical German philosophy professor who used Claude to implement several Tinderbox-style view specs at once and then made them mergeable, so you pull on an axis of visualisation when you need it — the maker refuses to share it and insists people build their own. Brandel recalled Bill Buxton’s sketch on the now-retired Channel 9 about the creative process of drawing on paper, and children’s marks whose meaning is only visible if you were there for the making; he noted you can learn 3D by watching someone work in a way you cannot learn programming by watching someone type.
Closing threads: Frode credited absent members — Alan’s insistence on big pieces of paper, Adam Wern’s principle that you rarely touch your own fingers by accident, Fabien’s level-of-detail work, Uli’s Soft Space fist-to-move-the-room gesture now adopted in both Origami Text and Author, and Brandel’s pinch-to-move-the-mural for Bob Horn’s murals that cured the joystick nausea. His takeaway from the Hypertext demos: the room makes it easy to fill the space with junk. Brandel announced that OS 27 has shipped with the model element supported natively on macOS, iOS, iPadOS and visionOS, with blog posts coming on webkit.org and Meta having implemented stereo inline WebXR that renders it accurately; Frode proposed a session on how it fits inside EPUB. Frode’s final provocation was that the iPhone duo — held cracked open in the hand like a book — will change digital reading more than the iPad did; Brandel mentioned the Device Posture API that would let a page know the fold state.
AI: Insights
The Lab has grown a vocabulary for documents and has none for the thing beneath them. The Origami Text work has settled what a document is: a strict EPUB carrying a framed point of view at a point in time. But the object Kathryn described — a memory attached to a painting, a thought that may or may not point at something, shareable but unbound — falls between annotation, note, concept and to-do item, and the session showed that the missing word is not a labelling problem but a design problem. The digital letters experiment with Eric is the first attempt to instantiate it.
The Frode–Mark disagreement is about when, not whether. It looked like a dispute over whether a shared object must have coordinates and a description. Brandel’s ladder of abstraction and Mark’s concession reframed it: everyone agrees the receiving system needs enough to construct a view; what differs is how early in the thinking process a given person needs that specificity to exist. That is a real design constraint for spatial tools — some users need a there before they can think, others are hampered by one — and it is also a warning that externalised views assume everyone sees the same thing.
Level of detail is the concept the Lab has been using without naming. Art arrived at it from Houdini and canvases, Brandel from maps and the fractal quality of space, Mark from the observation that everything the Lab draws is really two-and-a-half D. Frode’s concern that VR makes it easier to make a mess, and the Hypertext feedback that the room quickly holds too much, are the same problem seen from the user’s side. The suggestion that annotation itself could be layered in depth — sketch, live notes, agentic summary — is a concrete brief.
Bodies are not standard. Tom’s amblyopia, the vergence–accommodation conflict that sickens some people and not others, Fabien’s lack of stereopsis, Mark’s aphantasia and Brandel’s “we are not really standing still most of the time” all point the same way: the assumption that a spatial interface presents one experience to everyone is wrong at the level of eyes, inner ears and imagination. Tom’s request for a physical touch target is an accessibility argument, not just a preference.
The standards question has inverted. Mark, who once told Frode to subvert EPUB from within, now suspects academic documents need a fork; Brandel, inside the browser world, says native EPUB rendering is something you would have to work to prevent. The pragmatic path suggested by the session is to file feedback, prototype a browser reader with Claude, and let the model element — now shipping — be the demonstration of what EPUB can carry that PDF cannot.
Newcomers are arriving through AI. Art was brought here by an agent; Kathryn built her way here with self-taught Python and heavy back-end use of models. Both describe the outcome as thinking more, not less, and both came at spatial cognition through Barbara Tversky. That is a different kind of arrival from the academic and industry routes the community has relied on, and it is worth noticing.
AI: Resources Mentioned
The Extended Mind: The Power of Thinking Outside the Brain by Annie Murphy Paul — the book that started Kathryn Jones’s project; Paul appears in the Future of Text book series and Kathryn was speaking with her later the same day.
Insatiable Mind — Kathryn Jones’s personal app for turning books, images and experiences into interactive, spatially navigable artifacts; built for herself, not yet in XR.
Supersizing the Mind by Andy Clark — the book on embodied cognition and the extended-mind thesis that shaped Brandel Zachernuk’s career.
Barbara Tversky — cited independently by Kathryn and Art as an influence; coming to the Future of Text Symposium in London on 12 November, which will be run properly online this year.
Doug Engelbart interview, circa 2000 — recorded by Frode, audio recovered from a rotted web source, cleaned up and re-uploaded; link shared in chat, transcription to follow.
Origami Text — strict EPUB 3 plus metadata in a JSON wrapper and the HTML/XML; reader and converter on macOS, iOS and visionOS; open source with prompts for rebuilding it; several built-in ACM styles, user-overridable. Demonstrated with a model element pulled out of an EPUB in the headset.
ACM Hypertext 2026 proceedings — converted from LaTeX to EPUB via Origami Text, with Mark Anderson’s help and Claude in Xcode; papers now downloadable as Origami-capable EPUBs. Mark posted three open-access papers of interest, including recent spatial hypertext work, in the chat.
model element — Brandel’s W3C proposal, now natively supported in OS 27 across macOS, iOS, iPadOS and visionOS; a polyfill exists, Meta has implemented stereo inline WebXR support, and blog posts on webkit.org are coming.
W3C conference, Dublin, later this month — Brandel presenting spatial hypertext and spatial CSS proposals; Frode may attend.
Meta glasses with full eye tracking — announced the previous week; discussed for what developers and advertisers will do with gaze data.
Magic Leap — noted as the only commercial headset with two focal planes; Meta’s varifocal display research mentioned as unshipped.
Light-field camera — recalled by Tom for capturing all focal depths so depth of field can be chosen after the fact; neither he nor Frode could name it with confidence in the session.
Aphantasia — introduced by Mark as a lens on why participants imagine spatial objects differently.
Seed — federated hypermedia system by Gabo Beaumont; whole-document comments already interoperate with Origami Text.
Digital letters — the JSON-file letters Frode and Eric Rangell are exchanging as a lightweight, non-email document object.
Tesser — Art Domi’s spatial project-structure prototype with level-of-detail rendering, agent-generated structure and an annotation layer; the transcript renders the name inconsistently, so spelling should be confirmed with Art.
Miro — Art’s current canvas tool, which he expects to outgrow.
Tinderbox — praised by Brandel as an “almost disappointingly feature rich” system for arbitrary views over semi-structured data; roughly ten view specs in Engelbart’s sense. Mark will try to obtain video of the German philosophy professor’s Claude-built mergeable-views implementation.
Bill Buxton’s sketch on drawing as process — formerly on channel9.msdn.com, now retired; recalled by Brandel.
Soft Space by Uli — early spatial concept tool whose fist-to-move-the-room gesture Frode now uses in Origami Text and Author.
Bob Horn’s murals — the case where Brandel’s pinch-to-move-the-mural replaced a nausea-inducing joystick.
iPhone duo — Frode’s prediction that a foldable held like a book will change digital reading more than the iPad; Device Posture API mentioned by Brandel.
“Text as thought” — Tom Haymes’s proposed future talk on mapping thinking into text, drawing on student chat logs and Vannevar Bush.
Community question circulated by email this morning: if there is to be a new document format, what should it do for you that you cannot do today? Peter Dimitrios’s first answer is the cross-linking EPUBs currently cannot do.
Charlie — colleague whose email to Frode this morning was a reminder that academics have no time for new things, a reality any new format must respect.