Reading

In our first year we focused on reading in XR. We therefore built a seated experience with 360 interaction, based on a knowledge worker being seated on a ‘swivel chair’.

Current Readership in XR Experience

The available information in this system is the ACM Hypertext Proceedings from 2022 and 2024, as well as The Future of Text Volume 5. Other documents can also be viewed in this environment.

When you click on the link to ‘Enter’, you will be presented by a Table of Contents and you will see your hands which are how you will interact with the text in this environment, there are no controllers.

Table of Contents. Immediately on entering you will see a table of contents. Tap on any category to view the individual papers, then drag any paper out into space to view a summary, then tap ‘-’ to close the summary or ‘+’ to open the full paper.

Point and Grab. You can interact with elements by pointing to them (making a ‘gun’ gesture) and an orange cursor box will appear, allowing you to point to items in the space which you can then pinch to grab onto or activate.

Manna Menu. When you look at your non-dominant hand a series of options will appear on your fingers. These were expected to be possible to tap but current headset technology is not up to the task so you will need to point to the items and pinch with your dominant hand.

Access Other Documents. You can read other documents in this environment and we will detail how this can be done soon.

To experience the system please click on any of the links below when you are in your headset. To save you time, you may want to bookmark this site: futuretextlab.info in the Web Browser in your headset.

The system was built by Andrew Thompson and was presented at ACM Hypertext ’24.

Quick Visual Walkthrough

After Hypertext ’24

Before Hypertext ’24

Sliver Sphere interactions

Question: Would it be elegant and useful to use a sphere to enter XR and also as access to controls? Mockup online.

Priority is text in the entry screen and tapping on ‘watch’ to reveal control panel

Swipe document open horizontally wide

Question: Would it be useful to swipe through a document open not just on one page, or two pages, but all pages? Mockup online.

PDF in XR and initial gestures

Questions: What should the initial environment look like and what should the initial gestures for interaction be?

Focus

Our key interaction is with a document’s References, in context with the document itself. We are working with the notion of frames, which are rectangles with a document or part of a document. The user will need to be able to interact with such frames smoothly and elegantly, this is core to our project.

Andrew is working on interactions for:

  • Scroll the contents of a frame up down/right left (currently flat palm)
  • Select an item in a frame, and action it (such as a link or button)
  • Place the frame in another location
  • Allow for items in one frame to be able to reference, visually, with a line, an item in an another frame

This is to allow the user to see all references in a list and how they relate to the document in question; single references to document locations, as well as volume and by criteria.

Audio: We are looking into speech to text for notes. We are also looking at the system speaking messages/alerts where appropriate.