AI: Summary

In this development meeting on May 7, the FT Lab team discussed how to approach the design and functionality of a manuscript authoring tool in XR. Key discussions centered around defining the elements that can be interacted with in the spatial environment, how to organize and manipulate text in 3D space, and what the initial default view should look like. The team debated the shape and structure of frames that would hold content (such as headings, body text, references, glossaries), how these could be extracted and placed in space, and methods of organizing workspace elements like throwing items onto magnetic panels. There was also discussion about various text editing features, including linking frames, parallel text streams, and a “note to self” functionality that would be excluded from exports.

Frode Hegland, Fabien Bénétou, Mark Anderson, Peter Wasilko

AI: Decisions

The team agreed on a list of interactive elements to be included in their XR manuscript authoring tool: headings (for outline or sections), body text, special text (places, people, dates), glossary/defined concepts, references with annotations (possibly with extractable metadata), media (images, video, 3D audio, interactive elements), user notes, charts with interactive values and code, saved spatial configurations, and cutting/alternative edits/auditions.

The team decided to use “volumetric frames” as the initial representation for content elements that could be stacked, arranged, and viewed from different angles.

They agreed to rename the “faded text” feature in Author to “comments” to better reflect its function as text that would typically not be exported.

They decided that a good starting point for the prototype would be the ability to type in a rectangle, pull out text segments into space, and throw text elements onto magnetic panels.

AI: Speaker Summary

Frode Hegland

Frode focused on establishing concrete elements to build from for their XR manuscript tool. He proposed a list of interactable components including headings, body text, special text, glossary, references, media, and notes. He advocated for starting with a basic representation of stacked “volumetric frames” that could later be interacted with in various ways. Frode was concerned with making progress by locking down the basic building blocks while acknowledging they could change later. He demonstrated features from his Author software including a document checking function and discussed how abstracts could help readers quickly understand papers. He was excited about the potential for magnetic panels that could be used to organize content spatially. Frode also discussed renaming the “faded text” feature to better reflect its purpose as comments.

Fabien Bénétou

Fabien focused on technical implementation concerns and cautioned against flattening the 3D nature of their project by thinking too much in 2D. He explained how his existing framework already supports text typing in space and suggested that text elements could be made to snap to panels when released nearby. He clarified technical aspects about the “scene” concept in WebXR being the root of the object tree. Fabien was concerned with creating useful functionality for academics while not getting bogged down in perfect data structures. He explained how the system could support either persistent objects across different scenes or completely new content depending on what behavior they wanted to implement.

Mark Anderson

Mark cautioned against “premature formalization” and emphasized that different writers work in different ways. He highlighted how academic writing often involves contingent information that surrounds the main text, such as references and glossaries. Mark suggested focusing on addressability of content and understanding what strands of information users might want to extract from texts. He was concerned about designing for scale, noting that visualization approaches that work for small amounts of text often break down with larger documents. He advocated for considering the connections between elements rather than just the elements themselves, and warned against locking things down too definitively early in the design process.

Peter Wasilko

Peter contributed specific feature ideas including linked frames where text would flow between them when one frame overflowed, parallel text streams for different types of annotations or translations, and ways to visualize connections between different parts of a document. He proposed a system for tracking “open questions” across documents that would persist in the library, accessible from both Author and Reader applications. Peter described how this would help researchers track down answers to questions raised in one document that might be answered in another document months later. He also suggested the ability to create forward-looking references to content not yet written, and recommended calling the faded text feature “considerations.”

AI: Topics Discussed

WebXR

The team discussed using WebXR as the foundation for their manuscript authoring tool. Fabien explained that in the framework he’s using, “scene” is the root of the object tree in WebXR, similar to how the body element functions in HTML. The scene concept is fundamental to their implementation as it represents the environment where all other objects exist. They discussed how to handle scene transitions and object persistence across different scenes.

Gestures

The team discussed several gestures for the XR environment, including:

Pulling text out from a document and placing it freely in space
Throwing text elements towards magnetic panels for organization
Using a pinch gesture for selecting text (noted as problematic due to distance issues)
The potential for an “explode” gesture where users put their hand in and spread content apart
A gesture system where taking free-floating text through a ring would change its behavior to snap to panels when released

Sloan

The Sloan Foundation was not specifically discussed in this transcript.

Symposium

The Symposium was not discussed in this transcript.

Book

The Book was not specifically discussed in this transcript.

Visual-Meta

Visual-Meta was not specifically discussed in this transcript.

Timelines

Timelines were briefly mentioned by Frode when discussing how references could be viewed – he noted that users should be able to view references on a publication timeline, which they had “talked about a billion times.”

visionOS or Vision Pro development

Vision Pro or visionOS development was not specifically discussed in this transcript.

AI: Interesting anecdotes

Edgar, Frode’s son, made a brief appearance during the meeting. Peter commented that Edgar looked like Luke Skywalker, and Frode mentioned that Edgar was turning eight on his birthday the next day (May 8), which is VE Day in Europe. Frode noted that in America, VE Day has been changed to “Victory Day.”

Frode mentioned that Chris Gutteridge once told them that if you want to know what a paper is about, look at the last sentence of the abstract, describing it as “a bit of a heuristic, it’s a bit of a cheat.”

Peter shared an intriguing research question about a mysterious “Ashley HUD report” mentioned in a 1967 document about Epcot, which he couldn’t identify despite searching. He wondered if it might be related to the US Department of Housing and Urban Development and possibly connected to Disney’s Utilidor system.

AI: Concepts Introduced

Volumetric Frames – Introduced by Frode Hegland as frames that contain content but don’t have a fixed size independent of what’s on them. Unlike traditional pages, they expand or contract based on their content. They can contain either 2D content or extruded 3D elements.

Scene – Explained by Fabien Bénétou as the root of the object tree in their WebXR framework. He described it as analogous to the body element in HTML DOM, with nothing above it hierarchically.

Magnetic Panels – Discussed by Fabien and Frode as specialized areas in the XR space that content could be thrown against and would “stick to,” helping users organize different types of content.

Parallel Text – Introduced by Peter Wasilko as the concept of having multiple streams of text (like footnotes, technical commentaries, or translations) associated with positions in the primary text.

Note to Self – Discussed as a replacement term for “faded text” in Author, representing text that would typically be excluded from exports but maintained in the source document for the author’s reference.

Open Questions – Proposed by Peter Wasilko as a system for tracking unresolved research questions across documents in both Author and Reader applications.

Premature Formalization – Introduced by Mark Anderson as the problem of restricting choices too early in the design process without sufficient benefit, potentially limiting flexibility later.

AI: Notes for Claude

There were no direct references to Claude in this transcript.

AI: People Mentioned

Ted Nelson – Mentioned by Frode Hegland in a test document example that had an incorrect birth year (1868 instead of 1937), which was caught by the document checking feature.

Chris Gutteridge – Mentioned by Frode Hegland as having said that if you want to know what a paper is about, look at the last sentence of the abstract.

Luke Skywalker – Mentioned by Peter Wasilko when he told Edgar (Frode’s son) that he looked like Luke Skywalker.

Emily – Mentioned by Frode Hegland when he said “I’m not married to anybody but Emily,” in the context of not being committed to particular design decisions.

Ashley – Mentioned by Peter Wasilko as a mysterious person associated with a HUD report from 1967 referenced in a document about Epcot.

Bob Stein – Mentioned by Mark Anderson regarding “tapestry,” likely referring to a project or concept related to text visualization or organization.

Dave – Mentioned by Mark Anderson as someone he worked with who would “write a story and then circle back” or write notes to himself about adding something later.

Edgar – Frode Hegland’s son who briefly appeared during the meeting and was turning eight the next day.

Mark Bernstein – Mentioned by Peter Wasilko in a hypothetical example about wanting to link to something not yet written (“when I talk about Mark Bernstein’s hypertext paper”).

Charlie – Briefly mentioned by an unidentified speaker who entered the meeting saying “there’s a new XR judo. And he’s Charlie.”

AI: Product or Company Names Mentioned

Author – Mentioned repeatedly by Frode Hegland, referring to his text editor software with special features for academic writing.

Reader – Software companion to Author that allows for reading and annotating PDFs, mentioned several times by Peter Wasilko and Frode.

Adobe Photoshop – Mentioned by Frode Hegland when referencing a previous mockup he had created.

WebXR – The technology framework underlying their spatial computing project, particularly discussed by Fabien Bénétou.

Blender – 3D modeling software mentioned by Frode Hegland when noting he used to be “quite an expert in earlier versions.”

Final Cut Pro – Video editing software mentioned by Frode Hegland when describing the “auditions” feature for comparing different edits.

Scrivener – Writing software mentioned by Mark Anderson as an example of a tool that allows reordering content.

Overleaf – Collaborative online LaTeX editor mentioned by Mark Anderson as another tool that allows for reordering content.

HyperCard – Mentioned by Mark Anderson as an outdated technology metaphor that younger users might not recognize.

LaTeX – Mentioned by Mark Anderson showing a sample of his current writing, and by Peter Wasilko when referring to the knowledge package.

PageMaker – Desktop publishing software mentioned by Peter Wasilko as having had the feature to link text frames.

Framemaker – Desktop publishing software mentioned by Peter Wasilko as having had linked frames functionality that “fell by the wayside.”

ChatGPT – Mentioned by Frode Hegland when noting it could sometimes get facts wrong in the document checking feature.

ACM – Briefly mentioned by Frode Hegland in reference to abstract formatting conventions.

Foundation series – Mentioned by Frode Hegland as a visual reference point for how information might be visualized in their system.

Star Wars – Implicitly referenced through mention of Luke Skywalker character.

HUD (Department of Housing and Urban Development) – Discussed by Peter Wasilko and Mark Anderson regarding a mysterious report from 1967.

Disney/Walt Disney World/Epcot – Mentioned by Peter Wasilko in relation to the Utilidor system and its possible connection to a 1967 HUD report.

AI: Agreements & Disagreements

Agreements:

The team agreed on a list of interactive elements to build from, including headings, body text, special text, glossary, references, media, notes, and charts.
They agreed that the initial view should use volumetric frames to represent content elements.
There was consensus that users need different ways to visualize and interact with the same content depending on what they’re working on.
They agreed to rename the “faded text” feature to “comments” based on Mark’s suggestion.
They agreed that the ability to type, extract text, and organize it spatially would be a valuable starting point.
There was agreement that different writers work in different ways and the system should accommodate various writing styles.

Disagreements:

Mark Anderson expressed concern about the card metaphor, arguing it might not be familiar to younger users and that hard-edged representations weren’t appropriate for the fluid nature of writing.
There was some tension around how much to “lock down” the design elements – Frode advocated for establishing concrete building blocks while Mark cautioned against premature formalization.
There was disagreement about how much to focus on importing/exporting vs. creating an idealized environment – Frode wanted to focus on the “virginal thoughts” experience first while others emphasized the need for practical import/export functionality.
There was some disagreement about whether to prioritize 2D or 3D thinking in their design approach – Fabien was concerned about “flattening” the 3D nature of the project.

AI: Other

The meeting revealed an interesting tension between the need to make concrete progress on implementation and the desire to keep options open for different writing styles and workflows. Frode appeared to be pushing for definitive decisions to move forward with building something, while Mark consistently advocated for keeping possibilities open to accommodate diverse writing practices.

The team seems to be at a critical juncture where they’re transitioning from conceptual discussions to practical implementation, necessitating more concrete decisions about what to build first. However, they’re still actively exploring how spatial computing can transform academic writing and trying to balance innovation with usability.

The discussion about “note to self” functionality and Peter’s proposed “open questions” system highlighted the importance of metadata and supporting the messy, non-linear aspects of academic research and writing that often happen alongside the production of formal texts.

Chat Log URLs

No URLs were shared directly in the chat log visible in this transcript.

Chat Log Summary

There was no separate chat log included in this transcript, though there are mentions of items shared in Slack and in the Zoom chat during the meeting. This includes Frode sharing sketches on Slack, participants posting links in Slack, and Mark adding something to the sidebar at one point. Peter shared a link to what he believed might be the HUD report he was investigating. The full contents of these shared items are not included in the transcript.

Is there anything I’ve missed that might be important? The transcript covers a complex design discussion for an experimental XR authoring environment, with the team working to establish concrete elements while keeping flexibility for different writing approaches. The meeting represents an important transition from concept to implementation, with decisions about core interactive elements being made that will shape their prototype development. The intersection of spatial computing with academic writing practices appears to be relatively unexplored territory, making this work potentially groundbreaking.

7 May 2025