We will look at Conference Dialog in XR and how we can think of Annotations as Knowledge Objects along the lines of Citations, Quotes and other explicit data in an academic knowledge space.
Imagine not only underlining when you read, but actively building a structure of the knowledge. Something which you can usefully access and analyze later.
This won’t be to everyone’s taste so it will be important to have a wide discussion of what this might be.
Proposal for Unified Annotation Infrastructure
Here are the meeting notes for 9 March 2026:
AI: Summary
This session centered on annotation as a practice and a design challenge, weaving together empirical research findings, philosophical reflections on meaning-making, and speculative design ideas for how annotation infrastructure might work in spatial and digital environments. The group explored what people actually do when they annotate, why they do it, and how to build systems that honor those purposes — from e-ink tablets and PDF markup to XR-based spatial knowledge objects.
AI: Main Topic
Jamie Blustein opened with a presentation drawing on two of his published research papers. The first was a field study of annotation practices using physical paper, conducted across multiple universities over six years with students in computer science, electronic commerce, library and information studies, and literature. Students were given scholarly texts printed on large ledger pages to prepare for seminar discussions. The study identified a taxonomy of annotation types — arrows, asterisks, boxes, brackets, circles, highlighting, text, underlining, and compounds — and found that only six distinct purposes drove annotation for this user group: interpretive engagement, working through problems, tracing progress and drawing connections, barely engaging (marking long blocks), procedural self-direction, and place-marking as an aid to memory. The study found strong individual variation in how people annotated, but consistency in why. The second paper discussed glossaries as a form of annotation, introducing a four-dimensional classification: whether a glossary is tied to one document or floating, whether it is private or collaborative, whether it is published or unpublished, and whether it is static or updateable.
AI: Highlights
Tom Haymes explicitly highlighted his personal copy of Geeks Bearing Gifts by Ted Nelson as an example of annotation going so far as to render a text practically unreadable — describing it as “marked to hell” — illustrating the tension between annotation as engagement and annotation as obstruction.
Tom Haymes in the chat wrote: “Reading is about making a better version of yourself, not making a better version of the author (or the annotator).” This was treated as a significant provocation by several participants.
Jamie Blustein in the chat wrote: “The annotator is making the text their own. The annotation and annotated text is a new text. The notion of palimpsest as opposed to written text as fetish.” This was reacted to enthusiastically by Frode Hegland.
Frode Hegland shared in the chat: “Annotations: Point to, and leave anchors, to meaning in the world.”
Tom Haymes coined the phrase “We need a sextant for knowledge” in the chat, which Frode Hegland immediately asked him to speak it, calling it “very evocative.”
Brandel Zachernuk noted in the chat that he has long been puzzled as to why Hypothes.is has not had more impact as a system for social annotation.
Peter Wasilko closed the session by noting that Grok had impressed him by drilling directly to dissertation PDFs rather than surfacing top-level fluff, a practical tip for academic research.
AI: Insights
Annotation as first-principles design problem. Jamie Blustein argued that most annotation tool development has focused on what can be recorded technically — RDF schemas, data models — rather than starting from why people annotate. Returning to the six human purposes he identified could reground digital annotation design in genuine user needs rather than technical affordance.
The palimpsest vs. the fetishized text. A recurring conceptual tension emerged between treating source documents as sacred objects (reluctance to mark up borrowed books, fear of annotating incorrectly in front of professors) and treating them as substrates for personal meaning-making. Jamie Blustein noted that even digital paper resolves this tension partially: you always have the clean base layer. The psychological barrier, however, is cultural and social, not technical.
Annotation shapes the reader. Tom Haymes made the point that pre-existing annotations — even anonymous crowd-sourced ones as in Kindle highlights — exert a power relation over readers, nudging them toward someone else’s interpretation. Jamie Blustein confirmed research showing students are influenced by a professor’s pre-annotated copy even when told to ignore the marks. The implication: annotation is not neutral; it is always a power move over future readers.
Annotations as knowledge objects in their own right. Frode Hegland proposed reframing annotations not as attached decorations on documents but as independent knowledge objects that can be lifted from their original substrate, aggregated across sources — papers, books, places, people, conference sessions — and spatially arranged. This reframing collapses the distinction between a note and an annotation: the only remaining difference is the presence or absence of a pointer to an origin.
The annotation-as-glossary insight. Jamie Blustein‘s second paper proposed that glossaries are a distinguished type of annotation. This connects directly to Frode Hegland‘s observation that reading Behave (by Robert Sapolsky) produces a need to extract domain-specific definitions as a personal working glossary — a form of annotation that is neither marginal nor incidental, but structurally productive.
Context as annotation management layer. Frode Hegland introduced a distinction between annotation objects as the active working material and annotation objects as background context (the “canon on the shelf”). The same annotation object can function as foreground or background depending on what you are currently working on. This has implications for interface design: systems need a way to manage not just the content of annotations but their current epistemic role.
Spatial navigation of knowledge as instrument design. Peter Wasilko proposed a combinator model for navigating annotation spaces in XR — noun-class categories on one side, verb-type combinators on the other — analogous to the Stargate dial-home device: a sequence of selections through dimensional categories that narrows a vast data space to a specific target. Tom Haymes extended this with the sextant metaphor: you need to know where you are, not just where you are going, and different users (expert vs. novice) need different instruments. Frode Hegland connected this to direct spatial manipulation — twisting an object to reveal a different facet — as preferable to tapping a menu bar.
Model element as knowledge unit. Frode Hegland asked Brandel Zachernuk whether a model element in spatial CSScould represent a knowledge unit — a term and a definition — with no visual rendering. Brandel confirmed that non-rendering model elements are possible and already used as invisible scaffolding for spatial constellations, meaning that the knowledge graph and the spatial graph can be the same structure.
Zoom chat as annotation. Tom Haymes noted in the chat that Zoom chat logs are themselves a form of annotation — running alongside the spoken discussion, responding to specific moments, extending the document of the meeting itself.
Annotation standardization as political choice. Frode Hegland flagged that the W3C annotation format assumes internet connectivity, whereas document-centric annotation work does not. Choosing a standard is therefore a philosophical stance about what annotation is for, not merely a technical decision.
AI: Resources Mentioned
Websites and URLs
https://futuretextlab.info/2026/03/09/annotations-format-w3c-shaped/ — Frode Hegland‘s current technical proposal for an annotation format shaped by the W3C standard, shared in the chat by Frode Hegland
https://thefutureoftext.org — the community’s main website, shared by Frode Hegland
https://web.hypothes.is — Hypothes.is, a social annotation platform, mentioned by Brandel Zachernuk with puzzlement at its limited adoption
https://archive.org/details/howtoliewithstat00huff — How to Lie with Statistics by Darrell Huff, shared by Tom Haymes as his favorite statistics book
https://www.youtube.com/watch?v=EjUElpLZZtc — a Bill Buxton video on the inherent temporality of annotation, approximately 15 years old at time of meeting, shared by Brandel Zachernuk
https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system — the Origin Private File System web API, suggested by Brandel Zachernuk as potentially analogous to NSFileCoordinator for file coordination on the web
https://wasabi-jpn.com/magazine/japanese-grammar/combined-particles/ — a resource on Japanese combined particles as grammatical combinators, shared by Peter Wasilko in relation to his combinator model
Books
Geeks Bearing Gifts — Ted Nelson, mentioned by Tom Haymes (his used copy was heavily annotated by a previous owner)
Behave — mentioned by Frode Hegland as a book he is currently reading at the recommendation of a partner named Andrea; used as an example of reading that produces glossary-type annotations
A Humument — artist’s book by Tom Phillips, mentioned by Jamie Blustein in which the artist draws over the pages of a Victorian novel, incorporating the letters into artwork; updated in multiple editions over the years
Mona Lisa Overdrive — William Gibson, mentioned by Jamie Blustein for a scene in which a character uses a hand photocopier to scan pages and glue them into a notebook
Information: A Short History — a compendium edited by Ann Blair and others, mentioned by Brandel Zachernuk in relation to copying, cutting, and pasting practices across history
Meaning (1975) — Michael Polanyi, quoted by Frode Hegland in the chat: “We live in the meanings we are able to discern”
People
Ted Nelson — hypertext pioneer, referenced for the phrase “text under glass” and for his time as a paste boy at the New York Times; also author of Geeks Bearing Gifts
Cathy Marshall — researcher at Microsoft, mentioned by Jamie Blustein for studies of graduate students who were reluctant to share annotations for fear of being judged
Vannevar Bush and Brush (research by Marshall and Brush) — mentioned by Jamie Blustein regarding reluctance among graduate students to share their annotations
Douglas Adams — Frode Hegland recalled one meeting with him in which Adams described the concept of “virtual graffiti” — writing left in a physical space readable only if you are present in that space
William Burroughs — mentioned by Brandel Zachernuk as the most prominent practitioner of the cut-up technique, a form of radical document manipulation
Ann Blair — editor of the information history compendium mentioned by Brandel Zachernuk
Darwin and his grandfather Erasmus Darwin — mentioned by Jamie Blustein as notable historical annotators, especially Erasmus Darwin who drew long lines to mark entire passages
Bloom — mentioned by Jamie Blustein in reference to Bloom’s taxonomy of educational objectives, against which he structured his annotation classification
Elian Gonzalez — used by Jamie Blustein as a case study in rhetorical codewords and spin in media coverage
Technologies and Products
Supernote — e-ink tablet used by Jamie Blustein, discussed for its pen-tip-free stylus, keyword tagging, star gesture recognition, and landscape mode limitations
Remarkable — e-ink tablet considered but rejected by Jamie Blustein due to its software-as-a-service business model
Sony tablet — previously used by Jamie Blustein, noted for needing frequent pen-tip replacement
Apple Vision Pro / visionOS — spatial computing headset used by Frode Hegland for the Author application development, discussed extensively regarding its control bar and interaction design
Author (app) — Frode Hegland‘s application for visionOS, demonstrated live with its annotation and highlighting capabilities
Kindle — mentioned by Tom Haymes and Jamie Blustein; Jamie expressed disappointment with most annotated passages surfaced in Kindle
Meta home environment — mentioned by Peter Wasilko as a spatial context for anchoring knowledge tables on wall surfaces
Grok — AI tool praised by Peter Wasilko for its ability to locate dissertation PDFs directly
Gemini — mentioned by Tom Haymes as useful for academic paper discovery, and referenced for its claim that World of Warcraft has the most flexible and customizable game UI
W3C Web Annotation standard — mentioned by Frode Hegland as partly relevant to his annotation format work but too internet-dependent for document-centric workflows
Acrobat — mentioned by Jamie Blustein as a tool he uses to annotate documents before distributing them to students
Better Touch Tool — mentioned by Peter Wasilko for its hierarchical Touch Bar button menus, proposed as an analogy for combinator-based interaction in visionOS
BibTeX — mentioned by Frode Hegland as a model of simplicity for document-centric annotation format design
Stargate (franchise) — Peter Wasilko referenced the dial-home device (DHD) from Stargate as a model for combinatorial spatial query construction
Foundation (Apple TV series) — mentioned by Frode Hegland referencing the “Prime Radiant” as an evocative spatial knowledge instrument metaphor
Oulipo — mentioned by Jamie Blustein as relevant to discussions of hypertext and cut-up literary practice
Blue Sky, X (Twitter), LinkedIn, Slack — mentioned by several participants in a closing discussion about fragmented social media ecosystems and difficulty maintaining a shared community communication channel
RSS — mentioned by Frode Hegland and Tom Haymes as potentially useful for community information sharing, with Mark Anderson (not present) cited as an advocate of RSS and blogging
Pre-Session Songs
Co-Authored with Claude after extensive research on the different aspects of annotations and re-implementation in Author and Reader:
Pre-Session Poem
Co-Authored with Claude after extensive research on the different aspects of annotations and re-implementation in Author and Reader:
The author draws a map of what they meant. The reader draws a map of what they found. Neither map is the territory. But laid together, edge to edge, they show the rivers that run under both — the questions that the text was built upon and the questions it left for the first reader brave enough to write I‘m not so sure in the margin of the margin of the world.
What changes is not the act. The monk and the scholar and the student with a yellow highlighter are performing the same gesture: stopping the eye, naming the catch, externalising the moment when the mind says here.
What changes is the aftermath. For the first time in the history of reading, many ‘here’ can gather. A thousand moments of attention, each one small, each one honest, can rise from their separate pages and find their shape —not the shape you planned but the shape that was always there, waiting in the negative space between everything you’ve read, visible only now that the margins have learned to speak to one another.