Debrief of last week’s ACM Hypertext Conference testing.
AI: Summary
This meeting was primarily a debrief session following Mark Anderson and Dene Grigar’s successful user testing of their XR knowledge environment at the ACM Hypertext Conference in Chicago. They tested 14 participants over several days, gathering valuable feedback about user interactions with their spatial hypertext system. The session revealed important insights about gesture design, feedback mechanisms, and the distinction between XR as a creative/brainstorming space versus traditional writing environments. Frode Hegland then presented detailed future development plans, including enhanced spatial layouts, ring menus for text interaction, knowledge volumes (3D containers for documents), and improved import/export workflows. The discussion emphasized moving beyond traditional paper metaphors while maintaining usability, with particular focus on developing new spatial interaction paradigms for text manipulation in XR environments.
Tess Rafferty, Frode Hegland, Dene Grigar, Mark Anderson, Peter Wasilko, Tom Haymes, Peter Dimitrios, Fabien Benetou, Karl Arthur Smink, Jimmy Sixdof, Ayaskant Panigrahi
AI: Speaker Summary
Tess Rafferty expressed enthusiasm about moving to London and joining new opportunities, and later showed strong appreciation for the museum-like quality of the proposed XR environments. She raised practical questions about card identification when text labels are removed and discussed generational differences in XR adoption, suggesting that people growing up with the technology might be more comfortable with tasks like writing in virtual spaces. She found the spatial concepts “delightful” and proposed adding whimsical elements like books appearing when writing is completed.
Frode Hegland moderated the session and emphasized the importance of academic critique in their “plussing” approach borrowed from Pixar. He presented an extensive vision for future development including room-based environments, volume manipulation, ring menus, and hybrid workflows between XR and traditional computing. He stressed the importance of supporting both passive and active thinking, drawing distinctions between different types of cognitive work. He advocated for maintaining some familiar elements while pushing spatial boundaries.
Dene Grigar provided detailed feedback from the Chicago testing, identifying key usability issues including lack of haptic feedback, confusion with gesture design, and problems with the cube metaphor. She emphasized that XR environments feel fundamentally different – more playful than traditional work spaces – making them ideal for brainstorming and creativity rather than long-form writing. She advocated for using the full dimensional capabilities of XR rather than treating objects as flat papers, and stressed the importance of volumetric thinking in spatial design.
Mark Anderson took extensive notes during the Chicago testing and provided technical insights about user behavior patterns, such as people naturally looking right after keyboard input. He strongly advocated for moving away from traditional paper metaphors and emphasized that people quickly adapted to abstract representations over textual labels. He argued for focusing on the underlying knowledge objects rather than their visual presentation, suggesting that less visual complexity often leads to better user understanding. He highlighted the positive response to extracting text fragments for manipulation.
Peter Wasilko shared his transformation from XR skeptic to enthusiast after being encouraged to try headsets. He described his strong 3D visualization abilities but acknowledged that actual XR experience offered something his imagination couldn’t replicate. He outlined a specific use case for augmented note-taking with physical books, envisioning pristine book annotation through XR overlays. His comments reflected both technical understanding and practical application concerns.
Tom Haymes contributed observations about human behavioral patterns, noting natural tendencies like looking ahead rather than up/down and drawing parallels to aircraft carrier design influenced by pilot behavior. He questioned whether keyboards represent outdated paradigms and emphasized the need to understand historical technological transitions. He advocated for moving beyond first-level technological adoption (copying existing paradigms) to second-level innovation (fundamental reshaping).
Peter Dimitrios focused on practical interface concerns, particularly peripheral vision limitations in current VR hardware and the challenges of seeing physical objects like keyboards and coffee cups. He suggested head-leaning gestures for hover interactions and discussed the evolution of input methods across different devices. His contributions centered on bridging current limitations with future possibilities.
Fabien Benetou provided technical insights and suggestions, including precise keyboard alignment with physical desks and advocating for dimensional objects (cuboids rather than rectangles). He emphasized the “unlearning” aspect of moving beyond traditional paradigms and shared resources for accountless VR experiences. His focus was on the technical implementation of spatial concepts.
Karl Arthur Smink brought practical design perspective, acknowledging the challenges of completely abandoning familiar paradigms while noting the value of affordances for user learning. He discussed the tension between versatility and cognitive load in dimensional interfaces, and referenced Job Simulator as an example of effective tool hiding in VR. His comments balanced innovation with usability concerns.
Ayaskant Panigrahi asked about the relationship between 3D spatial work and 2D representation needs, highlighting the importance of workflow integration. His questions focused on practical implementation details and the need for multiple representation modes.
AI: Topics Discussed
What was discussed regarding WebXR? WebXR was mentioned indirectly through Fabien’s reference to browser-based VR experiences and his sharing of Spaceframe.xyz. The discussion touched on the need for platform compatibility beyond the Apple Vision Pro, though specific WebXR protocols weren’t detailed.
What was discussed regarding gestures? Extensive discussion covered gesture design challenges from the Chicago testing. Key issues included the blue dot placement for right vs left-handed users, the confusion caused by translucent cube interactions, problems with pinch gestures lacking haptic feedback, and the need for gesture modifiers to prevent accidental activation. The ring menu concept was presented as a multi-level gesture system for text interaction, with discussions about developing new spatial grammar for XR interactions.
Were other topics discussed? The conversation covered spatial hypertext theory, the evolution from paper metaphors to dimensional thinking, academic publishing workflows, generational differences in technology adoption, the relationship between XR and AI development timelines, voice-to-text integration, accessibility considerations for different physical abilities, keyboard alternatives and typing evolution, and the philosophical implications of moving beyond traditional information paradigms.
Were there any interesting anecdotes? Peter Wasilko shared his transformation from XR skepticism based on strong 3D mental visualization abilities to enthusiasm after actual headset experience. Dene described users repeatedly trying to grab text titles instead of cards despite instructions. Mark noted the consistent pattern of users looking right after keyboard input, leading to speculation about psychological or ergonomic factors. Tom referenced aircraft carrier island placement being influenced by pilot behavioral tendencies under stress.
Did anyone seem to change their position during the call? No significant position changes were evident during this call, though Peter Wasilko’s earlier conversion from XR skeptic to enthusiast was referenced as an example of how direct experience can change perspectives.
What were the major outcomes of this session? The session established clear priorities for the next development phase: improving haptic feedback, simplifying card interactions, developing the ring menu system, implementing knowledge volumes, and creating better import/export workflows. The testing results validated XR as a brainstorming/creativity space while identifying specific usability improvements needed.
AI: Concepts Introduced
“Plussing” – Frode attributed this concept to Pixar, describing their process of bringing people into rooms to ask “what should we do more of, what can we do better” with a positive attitude toward iteration and improvement.
“Knowledge Volume” – Frode introduced this as a 3D container concept for organizing documents in spatial environments, allowing users to scale and enter volumes to interact with contained materials.
“Ring Menu” – Frode presented this as a multi-level gesture interface where users select text and perform specific hand movements to access layered options for finding, connecting, or manipulating content.
“2.5D” – Dene used this term to describe the hybrid use of 2D interfaces within 3D spaces, such as watching films in VR or using traditional desktop metaphors in spatial environments.
“Spatial Grammar” – Frode discussed developing entirely new gestural languages for XR interaction, moving beyond traditional input paradigms.
AI: People Mentioned
Vint Cerf (used as example in ring menu demonstration by Frode), Ken (unable to attend, mentioned by Frode), Peter (double Peters reference by Dene), Jamie Bluestein (test participant who had difficulties, mentioned by Dene and Mark), Mark Bernstein (test participant with digital media experience, mentioned by Dene), Rob Swigart (VR portal project collaborator, mentioned by Dene), Jimmy Bluestein (test participant who suggested speech input, mentioned by Mark), Brandon (referenced for early advice about digital text, mentioned by Mark), Walt Disney (referenced for plussing concept origin, mentioned by Peter Wasilko), Ted Nelson (referenced for “God the author and God the reader” concept, mentioned by Frode), Doug Engelbart (referenced for mouse invention and keyboard limitations, mentioned by Tom Haymes), Julie (Soft Space creator, mentioned by Frode), Jonathan Finn (inspired big gesture concept, mentioned by Frode), Adam Warren (early spline writing example, mentioned by Frode), Edgar (Frode’s son, mentioned for typing lessons, mentioned by Frode)
AI: Product or Company Names Mentioned
Apple Vision Pro (target platform for development, mentioned by Frode), Meta (headset manufacturer and account system, mentioned by Fabien), Pixar (plussing methodology source, mentioned by Frode), Author (Frode’s visionOS application, mentioned by Frode), Sloan Foundation (project funding source, mentioned by Dene), Starbucks (meeting location where Frode met his wife, mentioned by Frode), Google (search functionality example, mentioned by Frode), Netflix (passive thinking example, mentioned by Frode), TikTok (passive thinking example, mentioned by Frode), Photoshop (expert tool example, mentioned by Frode), Bluetooth (keyboard connectivity, mentioned by Mark), Tinderbox (Mark’s daily tool since 2003, mentioned by Mark), Story Space (1987 note-making reference, mentioned by Mark), PDF (problematic format for XR, mentioned by Mark), HTML/XML (preferred formats, mentioned by Mark), BibTeX (citation format, mentioned by Mark), ACM (conference and paper source, mentioned by Mark), Oculus (early VR experience, mentioned by Mark), iPhone (speech-to-text example, mentioned by Frode), Apple Watch (potential input device, mentioned by Frode), Siri (comparison point for speech input, mentioned by Frode), Star Trek (holodeck reference and Peter’s mental models, mentioned by Frode and Peter Wasilko), ErgoDox/Moonlander (alternative keyboard designs, mentioned by Fabien), Palm Pilot (historical input method, mentioned by Peter Dimitrios), Mozilla Hub/Croquet/Second Life (flat-screen VR alternatives, mentioned by Peter Dimitrios), Job Simulator (VR interface example, mentioned by Karl), Spaceframe.xyz (subscription software example, mentioned by Fabien and Peter Wasilko)
AI: Other
The meeting revealed an interesting tension between innovation and usability, with participants advocating for both radical departure from paper metaphors and careful attention to user onboarding. The academic context provided a valuable framework for constructive criticism, as noted by Dene’s emphasis on academic culture allowing for kind but honest critique. The discussion also highlighted the importance of real-world testing in revealing unexpected user behaviors and the need for iterative design based on actual usage patterns rather than assumptions.
Chat Log URLs
https://www.youtube.com/watch?v=NMS2VnDveP8 https://bsky.app/profile/benetou.fr/post/3lz6bpyvofk2q https://www.spaceframe.xyz billbuxton.com/MMUserLearn.html https://adjacentpossible.substack.com/p/the-blank-page-revolution
Chat Log Summary
The chat log reveals additional technical discussions about input methods, with participants sharing experiences about typing evolution from QWERTY legacy to modern alternatives like split keyboards and swipe typing. There was significant discussion about the historical context of technology adoption, including typewriter design influencing modern keyboards and the gradual shift away from traditional input methods. Participants shared technical solutions and workarounds, with Fabien offering Meta-account-free VR options for Peter Wasilko. The chat also revealed ongoing debates about the necessity of maintaining familiar paradigms versus pushing for radical innovation, with Karl emphasizing the learning curve challenges while others pushed for breaking away from legacy constraints. Tom shared resources about the historical impact of paper on thinking, and there were discussions about progressive disclosure and the cognitive load of dimensional interfaces.