We are living in the first age of AI and XR. Whilst AI is getting a lot of attention–for positive and negative reasons–we believe that to truly augment us, and not to have our thinking outsourced to AI, XR will be crucial to scaffold thought. The experience we have gained over the last two years puts us in good stead to continue working on this.
The Full Environment
Over the last two years there were many technical issues to address, and there were many interactions to experiment with. However what we found was the feeling of working in XR that was the most subtly surprising and interesting.
When we put on a headset we enter the XR experience with our eyes and hands–not as a detached ‘brain in a vat’. This is obvious but hard to explain. As far as our eyes and brain are concerned we are in a space as real as physical space. We can enjoy information virtually ‘hanging’ in space but walking into a piece of text or having our view cluttered with connective lines are still bothersome. It is space, but different.
The ability to change what we call your sculpture of information can be imagined happing in hologram form in the real world, but changing your entire workspace instantly provides something which cannot be replicated in the physical world.
The work we have done has primarily been inwards focused, with annotated bibliographies and maps of knowledge, in a freeform space. This summer we transitioned to also consider the outwards space of passive and active environments for framing and organizing knowledge–and to instantly produce a completely different environment to suit another aspect of the task at hand.
For this reason we urge you to try the XR experiences if you have access to a headset. If you copy the Web Address to this site and paste it in the Web Browser in your headset you can scroll down a bit on the front page to access experiences of reading and authoring on XR.
We are incredibly asking what can happen if we are able to change our information and environment as easily as turning a page–what can happen to the shape of our thoughts?
The full environment visually and spatially holds real potential. In order to make this useful for work it is also crucial for the user to be able to conveniently access their knowledge, which is currently not feasible on VR headsets. Our work has from the start taken files–documents–seriously, making them possible to import. We have further implemented the means through which the user can access their own computer’s files while in XR. What we are building is built on real user files and real user workflows.
Focus : Artificial Intelligence & Super-Human Intelligence
Artificial Intelligence. If we make the assumption at AI will be correct most of the time, but not all, then we can be concerned that our future selves will not be fully and correctly informed.
If we take the very positive position that AI will, in the near future be almost always right, we still have reason to be deeply concerned because if we–and our children–can assume to always have a correct answer readily at hand (not only to factual answers which could previously be search for on the Web, but also to complicated questions which require logic and could not be assumed to have been carried out before), we need to ask what that will that do to curiosity and the mental work of thought in order to deeply understand something. This has deep implications for the future of the human mind and human society.
If we can get all ‘the answers’, what is there left to think about? If thinking is not involved, how can we hope to understand?
It is also important to reflect on how using AI reduces the user’s interactions with primary sources.
XR for Super-Human Intelligence. The the Future Text Lab we take the position that eXtended Reality (in the form of headsets for the moment) can contribute to a different future, a much more human-positive future where AI supports human reasoning, understanding and communication, rather takes over thinking.
We believe that an environment where all of our knowledge is available to us in an accessible manner, can help us develop deeper and quicker ways to see and create connections, relationships and–not least–lacuna.
How this will not be done
This will not happen through market-driven corporate willpower, since that is limited to the ‘values’ of cashflows and is not inherently aligned with the values of human engagement and growth.
This will also not happen through the force of education since education will teach only what is shown to be effective and relevant. Governments will be, as we have seen, slow to understand and implement technological progress since governments are governed by people who are top level politicians, not thinkers and tinkers of tools and thought.
We will have to do this ourselves.
How this can be done
Many efforts will be required to realize the potential of thinking in XR.
For our part, we are continuing the work with the mission to provide a compelling, exciting, interesting and user focused knowledge sculpture environment where everything the user is interested in and everything they have experienced will be available without being overloading the experience.
This will be as least as compelling as the computer gamer’s environment of tools and experiences, as least as social as tagging and sharing images on social media and as fulfilling as self discovery combined with ever deeper development of understanding of the world and our place(s) within it.
We are fully aware that this is an ambitious goal, or ‘dream’ even. The dream will need to be commensurate with the challenge however.
Implementation
There are two main areas of exploration for the experiences we will build:
Data. In order to support building such an experience we will continue with working to make user data clearly and cleanly available, whether this involves metadata in various forms or clear data in the form of the information itself, including:
- Academic papers in PDF and HTML form
- Information structures in JSON
- Images, including with image map information
- Video and audio
- Calendar events
- Sharing of all of the above, in rich formats and in ‘Plain Text Spatial’ form for embedding in traditional media in robust form
- & more
Without data relevant to the user there is nothing. With only limited data the potential is unnecessarily limited.
Interactions. With the ambition to have (pretty much) everything of importance to the user available without cluttering the environment, interactions will need to be experimented with and tested for multiple approaches for how to:
- Add information, both from external sources and created in the space
- Find information in the space and external
- Hide and subsequently Reveal information,
- Connect information and when/how to reveal connections
- Configuring layouts for individual items and collections
- Multi-user experiences with scalable degrees of immersion and interaction, as we have been experimenting with
- Facilities to allow for AI augmented data import
- Facilities to allow for AI augmented data analysis
- AR/VR transitions
- & more
Interaction is where the ‘rubber hits the road’, where our eyes see the knowledge and our hands touch it. This is where we must apply our collective imagination so that we will not only end up with whatever interactions the commercial market finds most convenient, like we did with the graphical user interface on desktop computers and the app environments on smartphones. We must not be stuck to just one paradigm, we must dream today to allow others to dream richer tomorrow.
Specifically, we plan to experiment further with:
- Maps of knowledge
- Timelines
- Memory Palaces/Libraries
- Volumes of knowledge (spatial documents)
- Passive/Aesthetic environments
- Active environments which contribute to information configurations/layouts
- Small and large environments and their impact of different workflows
- Control systems including small (Mana/Hand menu) and large (Ring) interactions
- & much more
Onward
Credibility. Please have a look around this website to see what we have been doing over the last two years with reading, authoring and dialog to further this mission, as well as what we have done over the last decade and a half with the Future of Text community more generally. With XR we have built a myriad of experiences in WebXR as well as native software for visionOS.
This is only the beginning. Join us!