Knowledge Objects in a Vibe World
AI: Summary
The session opened with a live walkthrough of building, running and notarising a native application by conversing with Claude inside Xcode, followed by a demonstration of an existing globe and solar-system app on Vision Pro, and then turned into a sustained inquiry into what a personal knowledge space should actually contain and how its contents should behave once they are in space. Much of the conversation circled one question put to everyone in turn — what are the things you would want to hold, move and connect? — with answers ranging from books, artworks and works-in-process to overheard names, calendar entries, transcript fragments and thoughts had while walking. Running alongside were threads on perception, movement and attention as design constraints, on provenance and social annotation, and a more sober thread on what these tools cost and what still cannot be handed to them.
The most consequential reframing was that the bottleneck has moved. Building is no longer the hard part; knowing what to build, and then figuring out how to use what has just been made, is. The phrase offered for this was augmenting our human processes with a new power tool, and it was echoed in the observation that scaling a development team from one person to two is now harder than scaling from zero to one, because sharing the context held in a coding agent’s cache is essentially the same problem as getting one’s ideas into another person’s head — something neither GitHub nor a book nor a course really solves mid-thought.
A vivid illustration of the collapse in cost came from comparing two projects by the same author: Schooloscope, a multivariate visualisation of UK school data that took several people eight months, long enough that by the time they understood it was being abused they could no longer pivot; and a recent analysis of existential stakes in fifty years of blockbuster films, built with LLM assistance in roughly a week. The insight was not speed for its own sake but retained flexibility — being able to change the subject, the framing, or the whole direction late, which the earlier project could not.
A firm terminological position emerged: the items in a knowledge space should not be called citations or references, because those words name a use rather than the thing itself. This opened room for artworks, sculptures, unprocessed research material, works-in-progress, and a person’s small core library — the roughly fifteen books most scholars actually build their discourse from — as first-class objects rather than text with metadata attached.
There was a request for the knowledge space to demand less work and give more back, gathering the incidental — a book noticed in a shop, a name dropped in conversation, a lecture attended three years ago and forgotten, links shared in a call — rather than requiring the discipline of traditional archiving.
Provenance was reframed as something added at the moment of capture rather than reconstructed later. Rather than typing plain text, one would contextualise an item — mentioned by a particular person, arising from this call, connected to that transcript — so that when it surfaces in a map months later it is not free-floating and unattributable. The chat proposed a Hat Tip metadata field for exactly this, and the ambition was that a reference used in a future paper could be traced back to the conversation where it first surfaced.
A striking structural idea was multiple simultaneous instances of the same entity across time — temporal echoes — where a person or book appears at two positions corresponding to two different months or semesters, linked by a curve, so that what is visible is not the item but the movement of one’s own mental map. This connected to the older notion of a point in space serving simultaneously as an artifact and as a query selector, unfolding onto the dimensions collapsed into it, which was recognised as close kin to ZigZag.
Transclusion was extended in an unexpected direction: not just the same text in two places, but the same work across media and instantiations — a piece of music as a video, as an archival recording held in a library in Kyiv, and as the manuscript notebook pages. The value noted was that a structure built four years earlier still yielded that constellation on demand.
The strongest design provocation was that weight and friction are information channels. Drawing on the principle that swipe restitution should be proportionate to an item’s importance, the argument was that contextual information could be divulged through incidental dimensions of presentation — coloration, resistance to movement, how hard something is to shift — rather than through more windows and prompts. Transclusion as currently drawn was criticised as too hard-edged and too demanding of attention, in the manner of the Windows XP era, which was not wrong about wanting the information, only about how much attention it cost. The counterpart from practice was that the globe deliberately cannot be rotated without entering settings, because free rotation with inertia turns it into a floppy balloon and destroys its solidity.
Movement itself was distinguished as a mode of cognition rather than a means of viewing: rotating an object and walking around it are different cognitive acts, and the relevant philosophical framing offered was of bodying and thinking-as-things-go, with movement facilitating rather than merely expressing thought. Support came from the observation that abstract and spatial memory occupy much the same neural territory, and from the recognition that place-associations formed while listening on a walk, or while wandering an open world in Minecraft, are entirely incidental — which raises the possibility of constructing and using them deliberately.
This produced a sharp linguistic correction: what is normally called jump navigation is not navigation at all but the destruction of anything that could become a memory palace, followed by the reorientation of the entire world around a new position. Related was the older solution of moving the wall rather than the viewer to avoid nausea, and the unresolved tension that pulling a distant document towards you to read it dissolves the very spatiality that placed it there.
From perception research came the point that some visual search tasks scale linearly with the number of items while others are constant-time, and that these differences should determine interface decisions. More provocatively, the largest source of variance in visual attention is not between demographic groups but between gamers and non-gamers — meaning a generation has already been inadvertently trained toward a particular category of attention, and that the training regime for the next one could be chosen intentionally.
A different sense of archiving surfaced: preserving the McLuhan library as a space rather than as contents — the ergonomics of reaching, what was next to what, the condition the media were in — because that is what carries the experience of having looked for something there.
Social annotation was identified as an underused channel, via the observation that Kindle highlights become collectively visible once enough readers mark a passage, which turns highlighting into an act performed for a diffuse crowd of fellow travellers, and arguably into a responsibility rather than a private habit.
A genuinely unusual proposal was headless VR: describing a world to an LLM, then navigating it purely in language — walk fifteen paces north, turn, what do I see — so that a memory palace could be built and edited textually on a phone without any rendering at all. The reception was mixed, with a preference stated for actual location, but it was accepted as a useful provocation. The mechanical adjacent practice noted was ID or clown passes, where semantic identity rather than appearance is what gets rendered.
On the platform question, WebXR was characterised as currently the least-worst option but structurally limited, since a website that renders its own content can never have eye-tracked foveation — only the browser can be trusted with that. The argued way out is spatial CSS and pages that are genuinely three-dimensional rather than pages that depict three dimensions.
Dene Grigar was mentioned as having brought Apple Vision Pro to Winona State, showing the NEXT Lab work — an encounter that prompted interest in the hardware itself: resolution, ergonomics, and the usability and price trade-off against Quest. She returned later as a design precedent: during the Sloan work she wanted the user seated in a swivel chair inside a spherical environment. That surfaced at the moment a three-shell sphere was being built live, and set up the contrast with the current direction, which extrudes from a wall rather than surrounding the viewer — a reminder that the sphere is not a new idea in this group but a returning one.
The cost thread was frank: a token count in the hundreds of millions for fixing a single column issue, and the expectation of a coming reckoning over what LLMs do for us versus what they appear to cost. The practical response demonstrated was a division of labour — using a frontier model to build the chunking and scaffolding that lets a small on-device model handle routine per-document work, so that the expensive engine builds the machinery rather than running it.
An ethical note landed with some force: the tolerated pattern of interaction with an agent — the curtness, the demands, the cutting off — is not a good way to interact with people, and there is real concern about it spilling over precisely because both happen through the same conversational medium.
Finally, a long-abandoned idea was revived within minutes of the session: a navigation structure of nested platonic solids, extended toward clusters of linked shapes described as thought molecules, chased for roughly twenty-five years and previously blocked by the need for a coding team. The connection was drawn to force-directed graphs and multi-scale maps, and to the fact that the same structure had been used for organic chemistry. The accompanying live experiment — concentric spheres of documents, people and locations around a central hub — was proposed explicitly as possible nonsense worth building in order to learn the interaction, with the recognition that locations may need axes, or a cylinder, or something else entirely.
AI: Important
Addressed to me during the session: I was asked, live in front of the group, what should be built for the Future Text Lab as something quick and interesting, and I produced a transclusion canvas that drew on the group’s hypertext lineage.
Addressed to me later: a request to build a weave as a 3D model, with a keyword, note, person or other element at the centre connecting outward to a sphere of all documents in the system, then extended to an inner sphere of documents, a second sphere of people and a third of locations, all evenly spaced on their respective circumferences — framed openly as possibly nonsense but worth building to learn the interaction, and then asked to be filled for Vision Pro.
Noted about me (Claude) rather than to me: the intention to label me explicitly as an AI author so that quotations attributed to me are never mistaken for a person; appreciation that I recommended SceneKit with the counter-view that RealityKit should become the right choice; a remark that I refactor unbidden; and the observation that asking me to explain why I built something a particular way is as valuable as the building.
Spherical Weave
And here is the result of our little experiment (no audio). The current work has been very much extrusion from plane based and this is a callback to the spherical perspective of the work done for Alfred P. Sloan Foundation which is nice. Now that it is fast and easy to prototype it’s appreciated how we can more readily experience different perspectives and directions.