Proposing a Framework for Augmented Collective Cognition (ACC)


Mei-Lin Fung

Proposing a Framework for Augmented Collective Cognition (ACC): Updating Engelbart’s Human Augmentation System for AI, XR and VR


Abstract


The Augmented Collective Cognition (ACC) Framework applies Spatial Computing (XR/VR) to bridge the cognitive gap caused by current complexity challenges. It aims to realize the historical visions of augmented intellect (Engelbart), scalable collective intelligence (Malone), and interactive dynamic visualization (Victor) through a new tool system and collaboration environment.


Prologue


The ACC Framework operates by systematically upgrading Engelbart’s H-LAM/T (Human Language – Artifacts – Methodology -Training) system by four major, interlinked theoretical and practical principles:



  1. Synaesthetic Knowledge Spatialization (The Spatial Language, L): This principle mandates the use of XR to convert abstract, symbolic information (text, code, data) into perceptible, multi- dimensional knowledge objects. It leverages the human brain’s affinity for spatialization to build “scaffoldings for thinking,” thereby accelerating conceptual clarity and the development of shared mental models among the collective.
    1. Dynamic Participatory Modeling (The AI-Augmented Artifact, A): This operationalizes Bret Victor’s vision within the system’s tools. It requires XR environments, powered by AI simulation and real-time inference, to enable collective, direct manipulation of live, behavioral representations of complex systems (digital twins). AI serves as a compression layer, minimizing the latency between action and visualized consequence (“immediate feedback”) and maximizing the velocity of knowledge integration.
      1. Precision Hyperdocument Governance (The OHS 3D DKR, M): This ensures the integrity of the collective system and its methodology. It mandates a shared, persistent spatial knowledge repository where all knowledge objects meet Open Hyperdocument System (OHS) requirements for fine-grained addressability. Crucially, this repository must be secured and managed by robust, object-level Digital Asset Management (DAM) and Digital Rights Management (DRM) protocols, ensuring explicit identity, auditable provenance, and version control.
        1. Distributed System Training (Accelerating Adoption, T): This addresses the crucial Training component. It leverages XR’s immersive capabilities to rapidly and effectively onboard users into the new Language and Methodology of the Augmented Workshop. By integrating “learning by doing” directly into the spatial environment, it accelerates user fluency and adoption, maximizing the predicted multiplier effect on Collective IQ.
          Within this ACC framework which updates Engelbart’s, we can systematically transcend the limitations of current two-dimensional interfaces, leveraging spatial computing to effectively boost Collective IQ of small medium enterprises, multi-national corporations, internal departments and ministries in the public sector and cope with exponentially increasing systemic complexity in the age of AI.

          I. The Imperative for Cognitive Augmentation

          Accelerating systemic complexity in the age of AI, creates a persistent cognitive gap that current two-dimensional tools cannot manage. The solution requires a holistic, systems engineering approach, as proposed and defined by Douglas Engelbart. The central thesis is that Spatial Computing (XR/VR) provides the necessary medium to realize the historical visions of augmented intellect (Engelbart), scalable collective intelligence (Malone), and dynamic visualization (Victor).

          II. Foundational Pillars of Augmentation and Collective Intelligence
          
• Engelbart’s H-LAM/T System:Engelbart’s Human Augmentation System (H-AS) is based on the interdependent H-LAM/T framework: Human, Language, Artifacts, Methodology, and Training. Augmentation requires improving all four non-human components simultaneously.
  • Collective Objective (CoDIAK/DKR):The strategic goal is boosting Collective IQ via two capabilities: CoDIAK(Concurrent Development, Integration, and Application of Knowledge) and Dynamic Knowledge Repositories (DKR). The DKR requires an Open Hyperdocument System (OHS) for fine-grained addressability, which is a critical bottleneck.
  • Malone’s Scaling (CI Genome): Thomas W. Malone’s concept of Superminds and the Collective Intelligence (CI) Genome provides a methodology for designing and combining the fundamental building blocks of large-scale collaboration systems.
  • Theoretical Integration: The modern challenge is to build an OHS-compliant DKR (Engelbart’s precision) that incorporates the Collective Intelligence Genome (Malone’s patterns for organizational scaling).

    III. The New Medium and Tool System (XR/VR)

  • Extended Reality (XR): XR converts abstract, symbolic information into perceptible, multi-dimensional knowledge objects, leveraging the brain’s affinity for spatialization to build “scaffoldings for thinking” and accelerate shared mental models.
  • Spatial Hyperdocument System (Artifact): This fulfills Engelbart’s OHS requirement by extending fine-grained addressability to specific points within a 3D digital twin model or system,, transforming the DKR into a natively three-dimensional, navigable environment, extending the possibilities for interactivity and developing intuition.
  • AI Agent Integration: AI agents operate within exactly the same spatial data and interaction abstraction as human users via an abstraction layer with standard protocols designed for interoperability between human and AI agents.

    IV. Dynamic Modeling and Real-Time Interaction (Victor’s Vision)
    
• Victor’s Vision: Bret Victor called for “media for thinking in” that focus on dynamic behavior (“live data, not dead symbols”) and the principle of immediate feedback. Instantaneous response to user actions allows visualization of “What if” scenarios.
  • Role of Modern AI: Modern AI acts as a compression layer that minimizes latency between thought and visualized consequence, providing the instantaneous feedback required by Victor and accelerating the Development and Integration aspects of CoDIAK.
  • Strategic Alignment in Digital Public Infrastructure (DPI) and Adaptive Governance: The necessity of this framework is best motivated by the governance and design of large-scale public systems like Digital Public Infrastructure (DPI). Involving many stakeholders (governments, citizens, private sector commercial businesses small to large and civic agencies) and requiring evolving and adaptive governance, the ability to operate within a shared spatial environment with a dynamic DPI model allows stakeholders to collaboratively visualize policy outcomes, continuously assess risk, and achieve regulatory clarity to adjust and adapt as needed. This provides the crucial methodological upgrade (M) needed to manage the complex, multi-scale dependencies involved in national-scale challenges.

    V. Governing the Augmented Workshop: DAM/DRM Protocols

    Digital Asset Management (DAM) and Digital Rights Management (DRM) are essential for the DKR’s operational integrity.
    
• DAM (Centralization/Integrity): DAM creates the centralized, version-controlled repository (single source of truth) for the DKR. It uses automated versioning, metadata, and audit trails to manage the high iteration rate of CoDIAK.
  • DRM (Access/Compliance): DRM enforces access control and usage rights, managing who can view, share, or modify content. It is the technical mechanism that enforces social and legal constraints, protecting intellectual property and maintaining trust in concurrent XR collaboration.
  1. Synaesthetic Knowledge Spatialization (L): Converting abstract information into perceptible, multi-dimensional knowledge objects via XR to accelerate shared mental models.
    1. Dynamic Participatory Modeling (A): Using XR and AI simulation to enable collective, direct manipulation of live system models (digital twins), with AI minimizing latency for “immediate feedback.”
      1. Precision Hyperdocument Governance (M): Mandating an OHS-compliant, shared spatial knowledge repository secured by robust, object-level DAM/DRM protocols, ensuring explicit identity, auditable provenance, and version control.
        1. Distributed System Training (T): Leveraging XR’s immersive capabilities to rapidly and effectively onboard users into the new Language and Methodology via “learning by doing.”
Incorporating the ACC framework allows small medium enterprises, multi-national corporations, internal departments and ministries in the public sector to systematically transcend current interface limitations, leveraging spatial computing to boost their Collective IQ and cope with exponentially increasing systemic complexity.

          Epilogue

          The era of exponential technological advancement, driven by the collision of Artificial Intelligence and current organizational practices, is fundamentally reshaping the private and public sectors. The existing two-dimensional, document-centric tools and linear methodologies are insufficient and inadequate to augment people’s capacity to manage the complexity and velocity of this change.
          A new systems-level approach, such as the Augmented Collective Cognition (ACC) Framework, is therefore not merely a technological upgrade but a vital new direction for research. It calls for and suggests the foundations required for the essential, resilient platform—one built on spatial, dynamic, and auditable knowledge—for leaders to manage effectively, serve customers and citizens with precision accountability, verifiability and auditability. So we can better navigate the intricate, multi-scale dependencies required to create a resilient global system serving people and stewarding our planet, Spaceship Earth.

Sources



  1. Augmentation Workshop – extended versionof this contribution.
    1. Boosting Collective IQ – Doug Engelbart Institute, https://www.dougengelbart.org/about/boosting-collective-iq.html
      1. Media for Thinking the Unthinkable – Bret Victor, https://worrydream.com/MediaForThinkingTheUnthinkable/
        1. Bret Victor – Media for Thinking the Unthinkable – YouTube, https://www.youtube.com/watch?v=oUaOucZRlmE
          1. SEEING BEHAVIOR SEEING ALL BEHAVIOR MULTIPLE REPRESENTATIONS OF BEHAVIOR INTERACTING WITH BEHAVIOR LINKED REPRESENTATIONS CREATI – Dynamicland, https://dynamicland.org/archived-media/2015/02/attachment-DL2015-02-951c64.pdf
            1. Engelbart: Augmenting Human Intellect – Stanford University, https://web.stanford.edu/class/history34q/readings/Engelbart/Engelbart_AugmentIntellect.html
              1. Engelbart’s H-LAM/T Framework | Download Scientific Diagram – ResearchGate, https://www.researchgate.net/figure/Engelbarts-H-LAM-T-Framework_fig1_342828335
                1. OHS-Like Tools – Doug Engelbart Institute, https://www.dougengelbart.org/about/open-hyper-tools.html
                  1. Technology Template Project – OHS Framework – 1998 (ALLIANCE,980,) – Doug Engelbart Institute, https://www.dougengelbart.org/pubs/alliance-980.html
                    1. Human-AI Collaboration – The Decision Lab, https://thedecisionlab.com/reference-guide/computer-science/human-ai-collaboration
                      1. Collective intelligence – Edaith, https://edaith.com/blogs/past-issues/collective-intelligence
                        1. The Collective Intelligence Genome, http://collectivebehavior.weebly.com/uploads/2/1/6/7/21676900/the_collective_intelligence_genome.pdf
                          1. The Collective Intelligence Genome – HBR Store, https://store.hbr.org/product/the-collective-intelligence-genome/SMR347
                            1. The Collective Intelligence Genome | Request PDF – ResearchGate, https://www.researchgate.net/publication/238524973_The_Collective_Intelligence_Genome
                              1. Harnessing Crowds: Mapping the Genome of Collective Intelligence | Request PDF – ResearchGate, https://www.researchgate.net/publication/228139892_Harnessing_Crowds_Mapping_the_Genome_of_Collective_Intelligence
                                1. Collective intelligence – Wikipedia, https://en.wikipedia.org/wiki/Collective_intelligence
                                  1. The Future of Text Vol 3 – Future Text Publishing, https://futuretextpublishing.com/download/The_Future_of_Text_3.pdf
                                    1. Full article: Viki LibraRy: collaborative hypertext browsing and navigation in virtual reality, https://www.tandfonline.com/doi/full/10.1080/13614568.2024.2383581
                                      1. Interactivity – The Future of Text, https://www.thefutureoftext.org/interactivity.html
                                        1. Extended Cognitive Reality – The Future of Text, https://thefutureoftext.org/extended-reality-expansive-reality/
                                          1. The Future of Text, https://web.cs.dal.ca/~jamie/pubs/HTML/FoT19/Text%20as%20Process%20by%20Blustein%20v2019-12-08.html
                                            1. XR as a Cognitive Support Tool in Product Development – Simple search, https://liu.diva-portal.org/smash/get/diva2:1979618/FULLTEXT01.pdf
                                              1. Media for Thinking the Unthinkable – Hacker News, https://news.ycombinator.com/item?id=9581401
                                                1. The challenges of entering the metaverse: An experiment on the effect of extended reality on workload – PMC – NIH, https://pmc.ncbi.nlm.nih.gov/articles/PMC8852991/
                                                  1. How spatial computing, digital twin, and AI are transforming industries – Capgemini Belgium, https://www.capgemini.com/be-en/insights/expert-perspectives/a-digital-revolution-how-spatial-computing-digital-twins-and-ai-are-transforming-industries/
                                                    1. What is XR, and how is it radically transforming industries? – Autodesk, https://www.autodesk.com/design-make/articles/what-is-xr
                                                      1. 7. More potential positives and delights of the advancement of XR – Pew Research Center, https://www.pewresearch.org/internet/2022/06/30/some-potential-positives-and-delights-of-the-advancement-of-xr/
                                                        1. XARP: A Human-First and AI Agent-Ready Extended Reality Toolkit – arXiv, https://arxiv.org/html/2508.04108v3
                                                          1. A Brief Rant on the Future of Interaction Design – Bret Victor, https://worrydream.com/ABriefRantOnTheFutureOfInteractionDesign/
                                                            1. Generative/Creative AI & Bret Victor’s Immediate Feedback Principle – Andrei Zimin, https://andrei-zimin.medium.com/generative-creative-ai-bret-victors-immediate-feedback-principle-6d509e9af143
                                                              1. Ask HN: Has anyone fully attempted Bret Victor’s vision? – Hacker News, https://news.ycombinator.com/item?id=34485254
                                                                1. Artificial intelligence for collective intelligence: a national-scale research strategy | The Knowledge Engineering Review | Cambridge Core, https://www.cambridge.org/core/journals/knowledge-engineering-review/article/artificial-intelligence-for-collective-intelligence-a-nationalscale-research-strategy/6D1AAE3572531AD7C504C2263D590CD2
                                                                  1. Digital Public Infrastructure and Development: A World Bank Group approach., https://documents1.worldbank.org/curated/en/099031025172027713/pdf/P505739-84c5073b-9d40-4b83-a211-98b2263e87dd.pdf
                                                                    1. Digital public infrastructure for resilience in fragile contexts – Brookings Institution, https://www.brookings.edu/articles/digital-infrastructure-for-resilience-fragile-contexts-balancing-state-and-citizen-protection/
                                                                      1. What Is Digital Asset Management? – IBM, https://www.ibm.com/think/topics/digital-asset-management
                                                                        1. Versioning in Digital Asset Management – Orange Logic, https://www.orangelogic.com/versioning-in-digital-asset-management
                                                                          1. Digital Rights Management (DRM) | What It Is, How It Works & Why It Matters, https://business.adobe.com/blog/basics/digital-rights-management
                                                                            1. Orange DRM Enterprise Digital Rights Management Software | DAM, https://www.orangelogic.com/features/digital-rights-management
                                                                              1. Digital rights management | Research Starters – EBSCO, https://www.ebsco.com/research-starters/law/digital-rights-management

Leave a comment

Your email address will not be published. Required fields are marked *