Themed discussion moderated by Dene Grigar after brief introduction by Frode Hegland look at design of a hand-gesture menu system.
Introduction Overview
We are working on developing an interaction system for XR beyond the traditional keyboard and mouse/trackpad paradigm, where the user has richer controls over their information environments through richer gesture controls. This will be in adjunct to, and not in exclusion to, bluetooth keyboards, pens voice and so on. The design criteria is that a new user should be comfortable using the system after a brief introduction, and as the user becomes more familiar with the system a richer degree of opportunities should be become apparent to the user. Whether working on an Annotated Bibliography, a Knowledge Map or doing Information Triage, the user will need a set of commands to be able to execute and the interfaces through which to execute them. That is what this is about.
Commands
We will need to list commands to see if we can develop meaningful groupings. Initial basic mapping show:
• Some commands are relevant for both selected objects and when not selecting objects; Find & Retrieve, Clipboard, View
• Some commands are relevant for selected objects; More Info, Format, Connect
• And some do not relate to either; Settings
Operations
We now need to work to decide and design what should be available for both interaction modes and how the specific interactions should take place. Concurrently we will need to work on the issues of:
• How to bring information into the space
• How to specify View Layouts
• How to specify what is to be Hidden/Shown
• How to revert to earlier Views (previous and saved)
• How to how to connect to other spaces
• How to connect beyond XR
• How to reveal more information about an object
• How to reduce information about objects.
Interactions
There are two different categories of interactions we are looking at:
• Interactions with specific objects ‘at hand’ Ring Menu
• Interactions with the full space, not a specific object. Hand Menu
At this stage we have chosen to use a Hand Menu system for interactions which are not with a specific object and a Ring Menu for specific objects which the user is holding. This distinction is because while the Hand Menu is fast and intuitive, it is awkward to apply while also holding an object.
AI: Summary
This weekly meeting focused on defining interaction design for XR academic environments, particularly around citation management and information triage. The team discussed the maturation of their Sloan Foundation funded project and the need to transition from abstract concepts to concrete interface implementations. Key discussions centered on distinguishing between context menus (ring menus) for selected objects versus general menus (hand menus) for unselected actions, the concept of level-of-detail (LoD) interfaces that change based on distance and context, and the practical challenges of designing gesture-based interactions for academic productivity tasks like managing large datasets of references and citations.
Frode Hegland, Dene Grigar, Peter Wasilko, Fabien Bénétou, Brandel Zachernuk, Mark Anderson, Tom Haymes
AI: Speaker Summary
Frode Hegland served as the primary facilitator, pushing the group toward concrete interaction design decisions after years of abstract discussions. He presented structured lists of operations and interaction modes, advocating for separating ring menus (context-sensitive) from hand menus (general commands). He emphasized the urgency of moving from theoretical to practical implementation, expressing some frustration with continued abstract discussions when concrete design decisions were needed.
Dene Grigar focused on project management and intellectual property protection, emphasizing the need to document discoveries for copyright protection against potential corporate appropriation. She stressed the importance of writing and publishing their work, even in small increments, and discussed organizing remaining grant time into wish lists, documentation topics, and potential future grant opportunities.
Peter Wasilko introduced Research Rabbit as a compelling use case for XR citation management, describing how screen space limitations made him wish for headset-based interaction. He contributed specific interaction design ideas including multi-selection gestures, set operations for citations, and accessibility-based triage (local library vs. interlibrary loan). His examples provided concrete scenarios for the abstract interface discussions.
Fabien Bénétou demonstrated technical progress with syntax highlighting for code and browser extension development. He contributed important insights about level-of-detail interfaces, distinguishing between technical performance optimization and semantic representation changes based on distance or context. He emphasized the value of composable interaction modes rather than forcing binary choices between interface types.
Brandel Zachernuk provided theoretical framework through the concept of “transitive vs. intransitive actions” and introduced level-of-detail (LoD) as the proper technical term. He contributed insights about chirality in gesture design and drew parallels between spatial information representation and textual information needs. He advocated for concrete task-based design approaches rather than abstract interface planning.
Mark Anderson consistently pushed for starting with realistic, large-scale data scenarios to avoid over-optimizing for small datasets. He emphasized information triage as a core academic workflow, describing the iterative process of categorization and organization that researchers actually perform. He advocated against premature interface formalization without understanding the underlying task structure and data variability.
Tom Haymes emphasized the need for speed and automation in information input and iteration, viewing logistical hurdles as primary barriers to productivity in current information environments. He advocated for automated population of data spaces and questioned the appropriateness of symbolic representations in interface design.
AI: Topics Discussed
Citation management and visualization in XR environments
Information triage workflows for academic research
Level-of-detail (LoD) interfaces that change representation based on distance and context
Distinction between context menus (ring menus) and general command menus (hand menus)
Gesture design for XR interactions, including considerations for handedness and ergonomics
Multi-selection and set operations for managing collections of academic references
Integration of voice commands with hand tracking for XR productivity
Screen space limitations in traditional interfaces versus XR spatial advantages
Intellectual property protection and documentation strategies for open source research
Project timeline management and future grant opportunities
Browser extension development for XR integration
Syntax highlighting and code editing in XR environments
Academic productivity challenges and workflow optimization
Collaborative XR environments and multi-user considerations
AI: Concepts Introduced
Level-of-detail (LoD) – defined by Brandel Zachernuk as the technical term from computer graphics for changing representation based on distance, applied to information visualization rather than just performance optimization
Transitive vs. intransitive actions – introduced by Brandel Zachernuk to distinguish between actions that require an object (transitive) versus standalone actions (intransitive), relevant for menu design
Information triage – described by Mark Anderson as the core academic workflow of iteratively categorizing and organizing research materials
Ring menu vs. hand menu distinction – Frode Hegland proposed separating context-sensitive menus (ring) for selected objects from general command menus (hand) for unselected actions
Incremental formalization – Mark Anderson’s concept of gradually developing categories and organization patterns during the research process rather than pre-defining them
AI: People Mentioned
Bob Horne mentioned by Frode Hegland as someone who emphasizes writing things down and structuring discussions, Amanda Askell mentioned by Frode Hegland as an example of a well-known person, Martin Luther King referenced by Dene Grigar in context of his “Letter from Birmingham Jail” regarding moral vs. legal distinctions, Ludwig Wittgenstein mentioned by Dene Grigar as author of Tractatus for reference to small thought exercises format
AI: Product or Company Names Mentioned
Research Rabbit introduced by Peter Wasilko as a reference management and visualization tool similar to Papers Cube, Sloan Foundation mentioned multiple times as the project funder, Microsoft mentioned by Fabien Bénétou as example of company that appropriated open source code without attribution, ACM mentioned by Dene Grigar as publication venue for copyright protection, Creative Commons mentioned by Dene Grigar for book licensing, WorldCat mentioned by Peter Wasilko for interlibrary loan searches, VisionOS mentioned by Brandel Zachernuk and others as Apple’s spatial computing platform, Apple mentioned in context of VisionOS and pen support, Logitechmentioned by Brandel Zachernuk for VisionOS pen, Meta Quest mentioned by Brandel Zachernuk for hand tracking, PSVR2 mentioned by Brandel Zachernuk for controller comparison, Photoshop and Illustrator mentioned by Brandel Zachernuk as examples of complex tool interfaces, Tapestry mentioned by Tom Haymes regarding population speed issues, Shiki mentioned by Fabien Bénétou for syntax highlighting implementation, Elden Ring mentioned by Fabien Bénétou as example of level-of-detail in gaming
AI: Agreements & Disagreements
Agreement on the importance of information triage and managing large datasets rather than small test cases – supported by Mark Anderson, Frode Hegland, and Peter Wasilko
Agreement on the need for better documentation and intellectual property protection – Dene Grigar and Frode Heglandaligned on urgency of writing and publishing work
Disagreement on timing of concrete interface design – Frode Hegland pushed for immediate gesture and menu design decisions while Mark Anderson argued for more foundational work on task understanding
Agreement on the value of Research Rabbit as a compelling XR use case – Peter Wasilko’s introduction was well-received by the group
Agreement on the distinction between ring menus and hand menus – general acceptance of Frode Hegland’s proposed separation
Disagreement on level of abstraction – tension between Frode Hegland’s desire for concrete implementation and Mark Anderson’s preference for understanding underlying tasks first
Agreement on the need for composable interaction modes – Fabien Bénétou’s suggestion that multiple interaction types could work together was accepted
Chat Log URLs
Chat Log Summary
The chat log reveals active parallel discussion supporting the main conversation. Participants shared relevant links including Research Rabbit, Hoffman’s interface design book, and academic productivity articles. Technical discussions included IoT controlled fans (showing the informal atmosphere), syntax highlighting implementation details, and VisionOS demonstrations. Mark Anderson and Tom Haymes provided additional context about their work challenges, with Mark describing automated data processing struggles and Tom noting upcoming schedule conflicts. The chat shows strong engagement with shared resources and real-time reactions to presentations, particularly excitement about Fabien’stechnical demonstrations and Brandel’s VisionOS work.
Important Context That Might Be Missed
The meeting occurs during extreme heat conditions affecting multiple participants (104°F mentioned by Peter Wasilko, fans running for others), which may have influenced energy levels and focus. The project appears to be approaching a critical deadline with the Sloan Foundation grant, creating urgency around deliverables and documentation. Dene Grigar’s repeated emphasis on writing and intellectual property protection suggests potential commercial interest in their work that could threaten the open-source mission. The technical maturity level varies significantly among participants, with some contributing code demonstrations while others focus on theoretical frameworks. Tom Haymes will miss July meetings due to teaching conflicts, potentially affecting group dynamics. The ergonomics discussion around gesture design reveals Frode Hegland’s background in ergonomics, adding credibility to his interaction design proposals.