First Quarter

First quarter presentation by Fabien Benetou on the work on Authoring for the Alfred P. Sloan Foundation. Introduction and moderation by Frode Hegland. Brief summary of last week’s presentation by Fabien in Paris to Christian Simon. Presentation and dialog by Fabien Bénétou based on companion.benetou.fr/demos_example.html

Fabien Bénétou, Frode Hegland, Mark Anderson, Jim Strahorn, Peter Wasilko, Alan Laidlaw, Karl Hebenstreit, Karl Arthur Smink, Brandel Zachernuk, Tom Haymes, Leon Van Kammen, Rob, Edgar.

AI: Participant Summary

The participants were researchers, technologists, and academics who form part of a community exploring the future of text, specifically in relation to Extended Reality (XR). Most are regular participants in this ongoing research project partially funded by the Sloan Foundation.

Frode Hegland, Fabien Benetou, Mark Anderson, Jim Strahorn, Peter Wasilko, Alan Laidlaw, Karl Hebenstreit, Karl Arthur Smink, Brandel Zachernuk, Tom Haymes, Leon Van Kammen, Rob Swigart, Edgar.

AI: Summary

This meeting featured Fabien Benetou presenting his first quarter work on XR experiments with Frode Hegland demonstrating the capabilities in real-time using a Vision Pro headset. The demonstration showcased multiple interactions in XR including: moving text elements in 3D space, binding shortcuts to physical gestures, interacting with documents via highlighting and annotation, organizing reference cards, manipulating unfolding cubes, and capturing screenshots and audio while in XR. The presentation aimed to reduce pressure for the year-end symposium by breaking deliverables into quarterly reviews. The demo was designed to explore the unique affordances of spatial computing for academic work and knowledge organization. Participants discussed interaction design challenges, the importance of freedom vs. constraints in spatial organization, and ways to make XR experiences more accessible to non-technical users. The group also planned a potential 12-hour design session to further explore these concepts.

AI: Speaker Summary

Fabien Benetou explained his recent presentation in Paris at the Sorbonne University’s Fab Lab, emphasizing how breaking the work into quarterly reviews would help reduce pressure for the year-end symposium. He highlighted how the demonstrations were designed to scaffold users gradually into XR interactions while maintaining openness for experimentation. He described various technical approaches including snap-to panels, hand-attached controls, and ways to extract work from XR (screenshots, annotations, audio notes) for use in traditional workflows.

Frode Hegland demonstrated the XR capabilities by wearing a Vision Pro headset, moving through different demonstrations including text manipulation, PDF reading, highlighting, organizing reference cards, and manipulating 3D cubes. He emphasized the natural interactions and the freedom of placing elements anywhere in 3D space. He stressed the importance of exploring different interaction paradigms during this unique historical moment when spatial computing is just emerging.

Mark Anderson raised questions about command positioning and whether controls should be fixed in space or attached to the user in a HUD-like manner. He discussed the tension between messiness and structure in spatial arrangements, noting that appropriate levels of organization depend on audience and purpose. He highlighted his work creating JSON datasets for reference materials with metadata flags.

Jim Strahorn emphasized the importance of having a “base of operations” that remains stable amidst flexible spatial arrangements. He referenced Bob Horn’s presentations with large information murals as an example of organizing complex visual information effectively.

Alan Laidlaw discussed the satisfying heuristic of the “snap back” function when grabbing text, suggesting similar feedback mechanisms could provide baseline organization in VR. He appreciated demonstrations that showed how work done in VR could be extracted and used outside the environment.

Brandel Zachernuk provided technical insights about demonstrating VR content to external audiences, suggesting strategies to communicate depth in 2D video presentations. He recommended “defanging” programming code and providing clear user-focused framing to make demos more accessible to non-technical audiences.

Peter Wasilko shared information about a Harvard architecture professor teaching a course on designing “Fortress of Solitude” spaces in VR, suggesting the group invite him to present. He also recommended making VR interactions recordable as text transcripts for later review.

Leon Van Kammen observed that the current demos require verbal explanation of gestures unlike traditional screencasts where mouse movements are obvious, suggesting the need for visual indicators of interaction in demonstrations.

AI: Topics Discussed

WebXR was discussed as the underlying technology for Fabien’s demonstrations. Fabien mentioned that because they’re using WebXR rather than native VR, the experience is accessible through URLs that can be shared, and modifications to the environment (like changing colors) can be made through URL parameters. This allowed for persistent experiences that could be accessed across different devices. The Vision Pro headset had some implementation issues with WebXR that caused occasional freezing, which Fabien noted worked more smoothly on Quest devices without requiring users to exit and re-enter VR between demo sections.

Gestures

The demonstration featured several gestures that were essential to the XR interaction model:

• Pinching to grab and move objects in space
• Using the right hand primarily for movement and the left hand for execution
• Tapping on a sphere on the left wrist to reveal command snippets
• Using transparent hands to avoid obscuring items when interacting with them
• Snapping objects to panels by releasing them near a panel
• Dragging commands onto wrists to create shortcuts
• Drawing with pen tools using specific hand positions
• Taking screenshots by framing a view and triggering a capture command

Fabien noted that users in Paris were able to learn these gestures quickly, with one user naturally performing actions after just three demos without needing to look at instructions. There was discussion about using modifier keys on physical keyboards (like Shift) to change hand behavior, making hands green to indicate modified states.

AI: Symposium

The first quarter review presentation was explicitly mentioned as a strategy to reduce pressure for the end-of-year symposium. Fabien explained that by breaking their work into quarterly presentations, they could distribute the stress throughout the year rather than having all the pressure at the final symposium. The theme for this year’s symposium was mentioned as “a myriad of perspectives,” suggesting the focus on multiple viewpoints and approaches to spatial computing.

AI: Book

Frode briefly mentioned that this year’s book would share the theme with the symposium: “a myriad of perspectives.” This suggests the book will compile various viewpoints on spatial computing and XR text interactions. The discussions around organizing academic papers and references in space (with different columns for text, references, and headings) seemed particularly relevant to the book project as it related to academic writing in spatial environments.

AI: Other Topics

Several other notable topics were discussed:

1. Spatial organization of information – The challenge of organizing information in 3D space was discussed extensively, with tension between freedom of placement and the need for structure. Jim emphasized the need for a “base of operations” amid flexibility, and Mark noted that messiness is subjective and relates to degree of sharing.
2. Extracting work from XR – Several demonstrations showed how work done in XR (highlights, annotations, screenshots, audio recordings) could be saved and accessed outside VR, emphasizing that users wouldn’t spend their entire workday in XR.
3. Design session proposal – Frode proposed a 12-hour design session where participants could sketch ideas, present them, and iterate throughout the day, potentially with ongoing connection similar to keeping Skype running during workdays.
4. The unique historical moment – Frode emphasized that this is the only time in history when we’re first entering spatial computing, making it crucial to experiment with “virgin thinking” and “naive thinking” before conventions are established.
5. User experience and accessibility – Discussion about making XR experiences accessible to non-technical users, including reducing jargon, providing visual cues, and creating better onboarding experiences.
6. Backgrounds and environments – Various background options were demonstrated, from cartoon landscapes to minimal grids, with the observation that user preferences varied widely without clear patterns.

AI: Concepts Introduced

Snapping – Introduced by Fabien as a “super power affordance” where objects automatically align to nearby panels without requiring the user to perfectly position them, creating a satisfying interaction where the system completes what the user started.

Cube as primitive – Defined by Fabien as something “in between kind of directly the body, for example, the wrist or the environment. It’s something we can carry or move around at that things too and play with or work with.”

Room scale cube – Demonstrated by Frode as an unfolded cube expanded to fill much of the user’s environment, creating a room-sized information space.

Spatial hypertext – Referenced by Mark Anderson and Frode Hegland as organizing information spatially, with Mark working on a paper reviewing its history and application.

AI: People Mentioned

Simon – mentioned by Fabien Benetou as someone whose name should be linked to the Fab Lab, Don Norman – mentioned by Frode Hegland regarding observations about how the purpose of travel has changed, Kubrick – mentioned by Fabien Benetou as a reference point for taking screenshots in VR, Alexander (Christopher Alexander) – mentioned by Mark Anderson in reference to pattern books as a bridge to architecture, Bob Horn – mentioned by Jim Strahorn regarding presentations with large information murals, Dene – mentioned by Frode Hegland as someone who wanted to attend but had university obligations, Ismail – mentioned by Frode Hegland as someone who was supposed to be physically present but couldn’t due to a fire at a Heathrow substation, Leibniz and Newton – mentioned by Brandel Zachernuk regarding their historical disagreement, Ted Nelson – mentioned by Tom Haymes and Brandel Zachernuk regarding Xanadu demos and ideas from the 1970s through 2000s, Eric de Broche des Combes – mentioned by Peter Wasilko as a Harvard architecture professor teaching a course on designing in VR

AI: Product or Company Names Mentioned

Vision Pro – mentioned by Frode Hegland and others as the headset being used for the demonstration, Quest – mentioned by Fabien Benetou and Karl Arthur Smink as alternative VR headsets with different WebXR implementation, Heathrow Airport – mentioned by Frode Hegland regarding a fire at a substation that prevented Ismael from attending in person, Photoshop – mentioned by Frode Hegland as the tool used to create concept images showing different column layouts, Skype – mentioned by Frode Hegland referencing past work practices where the connection would remain open throughout workdays, Google Docs – mentioned by Frode Hegland as a potential tool for collaborative design sessions, Pixar – mentioned by Fabien Benetou as a metaphor for the immersive XR environment users would “take things out of”, Author – mentioned by Frode Hegland as his own software tool with mapping capabilities, Sloan Foundation – mentioned in the initial instructions as funding part of the work, Sorbonne University – mentioned by Fabien Benetou as the location of the Fab Lab where he presented, Apple – mentioned by Frode Hegland regarding fixing WebXR implementation issues on Vision Pro, Harvard – mentioned by Peter Wasilko regarding the professor teaching VR design, Blockly/Scratch – mentioned by Fabien Benetou in chat as visual programming examples, Recolude – mentioned by Karl Arthur Smink in chat as a company making tools to record VR experiences, JobSimulator – mentioned by Karl Arthur Smink in chat as a VR application with good interaction paradigms

AI: Other

The meeting had an interesting dynamic where theoretical discussion about spatial organization was immediately grounded in practical demonstrations. There was a tension between exploring the novel possibilities of XR while maintaining connections to established academic workflows. The group seemed to balance excitement about new interaction paradigms with pragmatic concerns about usability and integration with existing practices.

The demonstration with Frode using the Vision Pro while Fabien guided him created an effective way to both show the technology and discuss it simultaneously. Despite some technical glitches (which the group took in stride), this approach allowed participants to see how interactions actually worked while considering their implications.

Chat Log URLs

https://futuretextlab.info/2025/03/04/first-quarter-24-mar-2025/ https://www.puf.com/renoncer-aux-voyages https://www.loc.gov/resource/hhh.ca0212.sheet/?sp=2 https://www.loc.gov/resource/hhh.ca0212.sheet/?sp=7&st=image https://www.sorbonne-universite.fr/en/news/fablab-place-creation-and-innovation https://companion.benetou.fr/highlights_example.html https://www.gsd.harvard.edu/person/eric-de-broche-des-combes/ https://www.gsd.harvard.edu/course/fortress-of-solitude-spring-2025/ https://companion.benetou.fr/q1_step_screenshot_screenshot_1742833645139.jpg https://companion.benetou.fr/audio_notes_example.html https://companion.benetou.fr/demos_editor_example.html https://futuretextlab.info/2025-schedule/
https://meta-quest.github.io/immersive-web-emulation-runtime/ https://fabien.benetou.fr/Contact/Contact https://video.benetou.fr https://aignerlwilson.com/2022/06/11/spreadsheeting-a-novel/ https://mitpress.mit.edu/9780262524117/algorithm-animation/
http://mitp-content-server.mit.edu:18180/books/content/sectbyfn?collid=books_pres_0&id=1191&fn=9780262524117_sch_0001.pdf

Chat Log Summary

The chat log showed active engagement from participants alongside the main discussion. Fabien shared a link to a French book about travel that sparked conversation about physical vs. virtual experiences. Peter shared resources about architecture designs that might inspire VR spaces, including a Harvard professor teaching a VR design course.

Several technical discussions occurred in the chat, with participants sharing tools for recording VR experiences, visual programming approaches, and examples of successful VR interaction paradigms like JobSimulator. Mark added context about the JSON datasets used for reference cards, explaining they were hand-crafted to explore edge cases.

The chat included reflections on design principles, with Tom noting that “good thinking spaces move between mess and order dynamically” and Karl pointing out that many people have only ever used GUIs rather than command lines. Fabien also used the chat to share links to demos and examples that couldn’t be fully shown during the presentation.

Some side conversations developed, with Alan asking Fabien about privacy and security tools, and Karl inquiring about pitch deck feedback for XR ventures. There was also discussion about the challenges of presenting VR experiences to audiences who can’t directly experience them.

Why is the topic of the future of spaces in XR important?

The topic of the future of spaces in XR is important for several reasons articulated in this meeting:

1. Unique historical moment – As Frode emphasized, this is the only time in history when we’re first entering spatial computing, making it crucial to document and experiment with “virgin thinking” before conventions become established.
2. Fundamental cognitive challenges – The organization of information and thoughts in space represents one of the hardest cognitive challenges humans face, and spatial computing offers new approaches to this persistent problem.
3. Academic knowledge work – The group is specifically exploring how XR can transform academic workflows like reading, annotating, organizing references, and connecting ideas across documents.
4. Bridging disciplines – As demonstrated in Fabien’s presentation at the Sorbonne Fab Lab, these tools have potential across disciplines beyond computer science, including aeronautics, chemistry, physics, and other fields.
5. New interaction paradigms – The work explores fundamental questions about how humans will interact with information when freed from physical constraints, establishing design patterns that could influence the future of computing.
6. Augmentation rather than replacement – The emphasis on extracting work from XR environments reflects an approach focused on augmenting existing practices rather than replacing them, making the research more immediately applicable.

This work represents an opportunity to influence the fundamental grammar of spatial computing before commercial interests fully define the space, potentially leading to more thoughtful, human-centered approaches to information organization and knowledge work.