11 Feb 2022

Video: https://youtu.be/Oh8yDKtPXD8

Chat Log: https://futuretextlab.info/2022/02/11/chat-11feb-2022/

Frode Hegland: So you’ve been on here a long time.

Fabien Benetou: You know? How are you doing?

Frode Hegland: I’m OK. Yeah, sorry about the the timing issue.

Fabien Benetou: No, I got confused. It’s Brandel the colonel who mentioned the meeting and I understood it was that in Brussels it was at 4pm. But it it’s now it’s so good.

Frode Hegland: Yeah, it’s 4pm UK time. You said you’re in Brussels?

Fabien Benetou: Yes. Yes. Next door?

Frode Hegland: Yeah, not very far away indeed. Yeah, we’ll be going through a few things. I have to finish a an article for ACM Hypertext on some other stuff. And so I guess we’ll be doing a little bit of arguing about HTML today. Of course, there will be also very much a big focus on the Virar side. Hi, Bob.

Bob Horn: Hi.

Frode Hegland: We have oh, please pronounce your name so that I may copy you.

Fabien Benetou: Sure. My name is Benita.

Frode Hegland: We have Fabian here today. A who is just joined us for the first time and we’ll say, who else comes in today, I guess. Brandel will be here at least.

Bob Horn: I T.

Frode Hegland: It’s going through the messages from the people. Mr Anderson.

Fabien Benetou: So I just made a cup of coffee.

Frode Hegland: Oh, Alan hasn’t been here for a while and I have a very important question for him. He said that the new Matrix movie was great and then he later said it wasn’t great and we need to find out what what’s going on. That’s kind of important.

Mark Anderson: I’d say there’s a balanced review, you know, good with some standard plot

Frode Hegland: As anyone seen it

Bob Horn: Here. I think it was ironically great.

Frode Hegland: I want to make right. Ok. Hang on. Does that mean you’re actually French, are you?

Fabien Benetou: Yes, I am. Ironically, for me,

Frode Hegland: The quintessential French

Fabien Benetou: Phrase

Frode Hegland: I Brandel. I hope Adam Warren will be here because we’ve been arguing furiously on Twitter,

Fabien Benetou: But it’s

Frode Hegland: Become useful effect, I think so that’s a good thing.

Bob Horn: While we’re doing introductory kinds of things, it would be helpful to me if we if we followed in an agenda a little bit, that is if we’re going to talk about HTML, I’m going to leave because that’s not an interest of mine and I have no, I know nothing about it and don’t intend to do anything about it. If we’re going to talk about AR and VR. I will stay or I will come in when the agenda says we’re going to do it. And if there are other topics on the agenda, I would like to know what they would be and then we can arrange it. And because my time is very, very squished these days and maybe other peoples are too, I don’t know,

Frode Hegland: You give us such a great compliment by assuming that there is any kind of organization whatsoever. So thank you for the compliment, Bob. That’s appreciated. Well, well. But let me let me answer your agenda question. There’s absolutely no agenda, but when it comes to different kinds of interests and different kinds of angles, we’re not going to be talking about HTML in any great detail. But we do need to agree on an approach because I have to hand in the paper for ACM Hypertext. And there are a few questions we can wait until later on today to go into that. But it is one of the things I would like is to have Tim Berners-Lee work on this as well. Obviously, that would be great. And to do that, it has to be a nice hook is that we also do this with the web. And I think we all agree that it would be useful to do it with the web. But we can’t just say, Oh, HTML is great. Let’s just add a few things to it. It has to be something new and useful. So that’s why Adam and I have been fighting furiously about useful perspectives earlier on Twitter. But he’s not here yet, so why don’t we start with Fabian? Would you like to introduce yourself? That could probably be a good and useful introduction for everyone.

Fabien Benetou: Sure. Sure. Well, actually, I like VR and HTML, and I think one of the most interesting is to bring in HTML to VR. The in practice, I’m a prototype artist and I work on my own. And at the European Parliament, the European Parliament has an innovation lab and basically I build WebEx prototypes there. And I’ve been managing my wikis and a lot of prototypes around it for 10 plus years, more or less, and including a link I showed a bit earlier with you on bringing in documents to VR in order to manage and organize my notes because to be honest, that’s what gets me going in VR at the moment or since I started yours, how to not be have all my information stuck behind a very thin wall? I find just that maddening. So, yeah, but that would be me.

Frode Hegland: So follow up questions then immediately. I am Norwegian. Remarkable is Norwegian company. I have no relationship with them. I love their products. I love the promise. But to me, it seems very much like an information ghetto. So I would really like if you could tell us a little bit more about how you manage to get the information in and out of that environment, because that’s important and fascinating.

Mark Anderson: So, yeah, it’s

Fabien Benetou: So the device itself. Honestly, it’s the slickest and nice and news, but remarkable has kind of choosing Linux, so you can tinker quite a bit with it. But the interface itself is closed source. So if you hack it like a proper hack, you can change it, but otherwise it’s a bit of a mess. But you can also entirely replace the interface and use another reader, another writer, and then do pretty much whatever you want with it then. But even then, the the documents that the remarkable manages are just in file metadata and then PDF and impose any buttons or just HTML, basically. So in the end, as long as you don’t want to tinker with the interface that remarkable itself provides, you can do quite a bit with it. It’s and it’s pretty. It’s not document, so you need to figure out everything yourself. But there is a quite active community of of hackers that do document make tools for all works well with it. Well, you have to go it.

Frode Hegland: I can hear a little bit of background sound. I’m not sure where it’s coming from, but thanks for that. And Brendan, we are introducing Fabian here today. And one of the things,

Fabien Benetou: Yeah,

Frode Hegland: One of the things he showed on Twitter earlier today was taking documents from remarkable the paper like tablets in VR, which is really fascinating. So he’s just talking about that now. And Fabian Peter just asked in chat if you have a link for the reader software or

Fabien Benetou: Any of that in the chat.

Frode Hegland: Do you have a relationship with the remarkable people, or are you just very clever at hacking?

Fabien Benetou: I’m happy clients. I don’t think remarkable has a relationship with anyone, especially since they got this money. They stole a flight. No more communication with the community, unfortunately, which is also why I’m going to do a little bit of Norwegian product, which is the by note, by buying. Sixty four of those guys just give the hardware and the goal is to bring Linux proper without any closed source interface to any hardware, mobile, etc.. So I think to be honest, if people are too locked in, the remarkable, the buying equivalent is, let’s say, a hardware at least part of a competitor equivalent. But then you can do you have to do it, though, because they don’t provide any software. So it’s up to you to implement. But then in very active community. So that’s the. Differently.

Frode Hegland: Are you able to now show us a little bit? Not everybody saw the work on Twitter. Do you want to do a share screen or share a link or anything?

Fabien Benetou: We look at that expression.

Frode Hegland: That’s an answer. All right.

Fabien Benetou: I’ll try. Yeah, I did not really prepare for this. Can you see my screen?

Frode Hegland: Yes. Slowly coming up now. Yep.

Fabien Benetou: Yep, yep. Ok. So I post most of my prototypes on Twitter. And basically that was this where I have my with the social VR experience. And then the desktop from my remarkable. So I get the metadata, which include the cover with the preview and then displayed there. The goal was that I have another prototype for the remarkable were basically based on the position of the cover of a document. Then I do an action, for example, send the PDF to another device or annotated as next research paper to read or discord or whatever. And why I did that in pubs is because I can invite other people to collaborate with me on that.

Frode Hegland: So before I get off my pedestal of asking lots of questions, I have one more before everyone else can dive in, and that is, are you a little bit familiar? Actually, that doesn’t matter. Here’s the real question. In that environment, the way that you use it, can you read the text and the documents through OCR or other means?

Fabien Benetou: Um. So you can read the actual text.

Frode Hegland: I mean computationally, not just visually.

Fabien Benetou: Well, yes, because you could the source of the document you are going to send. So in the end, you would use the source itself to to read it.

Frode Hegland: Have you been introduced to our visual meta thing?

Fabien Benetou: Hmm, I don’t think so, no.

Frode Hegland: It’s a really simple thing, oh, here is Adam. Let me just wait for him to connect. Hey, Adam, just pause there to wait for you to connect. Actually, could you because Adam has been working on something related all week? Could you please do a quick screen share again on what you’ve been doing? Yeah, yeah, sure. Adam, this is from the remarkable tablet. And this is what Bob Bjrn has been doing, which is why him and Brandel talk and why he’s here today.

Fabien Benetou: So that’s the eating device on the left, remarkable with the desktop of it. And then I bring the different documents to a social your experience in order to that’s implemented in another prototype. To be honest, the pain in my life say that I have features per prototype, but honestly, a unifying experiment. But anyway, that’s another problem. But in the other experience, then the action applied to a document is based on the position in space, and that’s done socially.

Frode Hegland: I needed to thank you. I needed to show this again to Adam, because Adam has brought documents into that same space. Were there the other day? I think he was a fox. It’s a funny place. So my question, just as you were entering Adam, was we have something we call visual meta, and it’s based on the metadata being in an appendix at the back of the documents. Are you familiar with Deb Tech or in Late Tech?

Fabien Benetou: No. Ok. I heard about it, but we went back,

Frode Hegland: So I don’t want to take too much time on this, but the very simple part of it is imagine you have a document where an appendix at the end it says title equals and then the name of the title a new line says author equals the name of the author and so on. So the document writes out in text what it is, all the metadata. So that’s what we are using as part of what we’re doing here. And one of the ways we’re looking at is how to put that into a VR environment. So of course, it can be programmatically, but I find it a bit strange, you can hold up almost like a JPEG of a page. But, you know, actually getting that so that it can be used, that metadata is another issue. So this is both something Brandel and Adam have been looking at. But I’m wondering, as you as the new guy, how you might want to extract that.

Fabien Benetou: Um, so as the new guy, anything I say is just completely naive, but I’m happy to do that. Is the internet, after all. So I would I would add that, no, I would keep the source actually and pull back from the source. So I would have the document itself, its metadata and and manipulate on that. So except if they were like you, M.Ed, I would try to keep one source of truth and I delete the original source. Ok.

Frode Hegland: That I am I am satisfied. I’m sure there are

Fabien Benetou: Many other chat for

Mark Anderson: You on visual matter. Yes, sir.

Frode Hegland: Ok, thank you, Adam. Are you ready to be showing what you’ve been showing screenshots of today? Because it’s so related, I don’t know where should we go next?

Adam Wern: And well, not really, it’s not much more than what you’re seeing on the screen shots, I’ve just been putting a lot of texts on screen. It’s hardly interactive. Um, so it’s not so much to see, really or. And. I found that I find that the idea of in a way, I’ve reinvented this mobile user interface of 2D, it feels like that actually going from the overview where where you see a full all pages at once and then zooming in to every single character. And I like that interaction having having instead of having thumbnails or other representation for for the text, letting the text be its own representation is nice. I think, of course you want supporting. Kind of supporting. Interface elements for for that, so you can show titles and and navigational metadata that comes goes when you show me in some world.

Frode Hegland: Not everyone has seen what you posted in a Twitter chat. I say Mark and Brendan have questions, but while they’re asking questions, could you share some of it with us, please? Are we really good for context, because it’s actually quite impressive despite you being Swedish and I’m Norwegian and I’m a bit jealous.

Adam Wern: The problem is that I have a one secure computer that is fenced off from the world with the wind, so I don’t have my working material on that computer.

Frode Hegland: That’s why you need universal control. That’s what you need.

Adam Wern: That so maybe it may be in a moment. I think you have one thing that I. I’m a bit it’s based on my wife is on call right now, so I’m juggling three kids here while she is doing the strange 40 hour on call things. So I kind of. So I will come back to you, continue with what you were doing. Yeah, I’d like to continue the track with Bob Bob’s track that we’ve been discussing on the side because I think that it’s really, really crucial whether we go two or three days as well.

Frode Hegland: Yeah. Yeah. Let’s. I look forward to that. Mark, do you want to go first with a question or.

Mark Anderson: Sure. No, there’s no such question. I popped in a link. People may be well ahead of me on this. I just saw it. I newsfeeds. But it’s a post by Robert Scoble. Not so much because it’s him, but I thought it was quite interesting. The the gist, the bit that I thought was pertinent was saying that actually, if you’re doing sort of more high intensity computing, which I think is, you know, most textual work may well fall in that part from the most casual of consumption of sort of say, reading a book is something you do sitting down. And saying that, so it sort of points far more towards doing air type stuff than VR. Um, and he make some other points in, they TED it’s easily readable blog, but anyway, I’d put a link in the sidebar, but I just thought that was interesting. He also happens to make to mention a certain fruit based company may or may not be having a VR product sometime soon. That’s basically it.

Frode Hegland: Yeah, it’s it’s a good piece, Brendan.

Brendan Langen: Yeah, I just wanted to ask Adam, I type my question into the chat, but what what do you imagine using the zoomable user interface for like what? What type of activities?

Adam Wern: And I think visualizing your documents. In a way, we have that in the folder structure of the operating system, but but the problem is that it mixes everything in there in a way, I want the documents and some of the documents pulled out and visualized while hiding lots of other documents that are completely unrelated. So I think a visualization of your your local library, we can call it library in a wide sense, including maybe videos and images and the support material that is thinking material. So one thing is visualizing everything and having it unified in a unified. And. In a unified system where you could look at things from afar and also going directly into them and instead of having textual names for things, actually seeing the if if it is something visual, you can just show it as a document. If the interface is supports quick zooming in and out, then you can have it letting objects be their own representations or to say. Of course, you can have search and all the other things that we have. But I really like having a visual representation. I like I like thumbnails, I like book covers. I look like everything that represents icons as well, of course. But but I like having the the the the visual representations. I like seeing the pages. I like seeing my comments from afar. I want to have that unify that it would be very interesting to see see an overview of all pages, including your comments and including any graffiti you put on it when you read from a fourth. That’s.

Brendan Langen: Yeah, that’s cool, I guess that question’s kind of for anyone to like I I also love a zoomable user interface, so I would I’m always curious, like how people

Bob Horn: Like using it. I would like to be able to zoom in and zoom through to the next levels of detail. A lot hypertext. Could I share my screen just to show people what I’m talking about? I’m going to I’m going to show detail of that information mural on the wall there, but it’s I can’t I can’t lift my 20 seven inch screen to go over there right now. Sorry, folks, we know that isn’t part of the technology yet, but I’m popping up this on my desktop and if if I can share my screen, you’ll let me share my screen.

Frode Hegland: Yeah, that’s all enabled.

Bob Horn: All right then. Can people see it now? Hmm. I’m moving it. But please let me know if I call

Fabien Benetou: Her in it, but I can see it.

Bob Horn: All right, but but we can, we can. I think I think we can zoom in on it. If I got the PDF there, if I got the JPEG, I’m not so sure.

Brandel Zachernuk : I put a link. I found it online, so you. There’s a link to PDF online.

Bob Horn: Yes, it’s also online, but here’s here’s zooming in, right? How do you like that? And moving around, of course, is kind of. Uh, difficult right now, because I can’t, you know, I can’t find the. Little thing, oh, here we go over here at the at the corner is where I can make it larger or smaller in the screen. And backing out and so forth. And what this you know what this enables and this has been done, this kind of visual display is done for groups, not for not, especially although it could be, you know, I use it for my individual, for organizing my individual information as well. But most of the work in the world is done with teams of people who have to get on the same page and this is a same page thing. And the what it shows is on on the left hand side, the far left hand side, the history of the relevant history of nuclear waste accumulation and the middle part. Oh golly, my next person is calling. Jim, can I call you back? Yes, thank you. Bye. And the middle part shows the decision making area, which I call the near present. It goes into a little bit into the past and a little bit into the future. That’s the way that’s where decision makers sit. I’ve been one. I worked with huge numbers of them, and that’s what they do. And then the far right side of this shows the forty thousand year plan.

Bob Horn: Or nuclear waste disposal in the United Kingdom. And the tangle of arrows shows some of the anticipated problems, which the people in the near the decision making team in the near present has to address. I can zoom in on some of those in case you want to see what some of them look like. Oops! Well, the Zoom, I guess it’s I could use a little bit better zoomable tool in some sort of way. All right. Maybe that’s a problem one of you folks can work on it. But here’s here’s some of the tangle of of problems that are that are anticipated. For the forty thousand year plan for nuclear waste disposal. And of course, we can zoom in a little farther. All decisions are short term criteria for site selection. Need to keep options open up closer to up in here? I don’t know if you can see my cursor. It’s right next to Her Majesty’s decision government’s decision decision on how to handle radioactive waste. They haven’t adopted the nuclear waste plan at the time I did the mural. There’s a whole set of of of of questions which could be zoom through what I think of as Zoom throughball or hypertext behind any one of those that the team. That is looking using the mural as as their reference point. Are due to address the problem. So I’ll stop there, and if there are any questions or anything like that, go ahead.

Frode Hegland: No, no. I was just going to say we’re getting into quite a lot of specifics on nuclear waste disposal as opposed to the visualization, but I think Brandel has a question.

Brandel Zachernuk : You know, I watched your presentation of this in Tony. I’m. Hi. Sorry, I’ve just. So I watched your presentation there, breaking down, breaking the walls of organizational ignorance on this, and I was really taken by it and wanted to know a little bit more about it. So how do you know sort of roughly how much information is in this? How many if this was a book, how many pages this book would be?

Bob Horn: Uh. There should be see, I think of these, these are these little chunks of information. There are probably about two hundred or, you know, give or take 50 chunks of of, you know what an ordinary text we call paragraphs or something like that or labeled paragraphs here. I call them information blocks or chunks. Some people call them snippets in our previous conversations in this room. And there are also about the same number of visual elements as I would call it. So something in the order of four hundred four hundred to six hundred combinations of visual and and and textual.

Brandel Zachernuk : Yeah, that makes sense. So, so probably roughly a hundred pages, if you’re imagine like sort of five paragraphs, a five, four or five paragraphs of page kind of thing.

Bob Horn: That’s Brandel. And I would say, Brendan, I would say behind each one of the small chunks here there would, you know, one would expect to have a page of information or even a whole document because we created we created these chunks from those documents.

Brandel Zachernuk : Oh, I see. So these are sort of just reference handles to to deeper information that Meyerriecks has. That’s awesome. Yes. Yeah. Because my next question was going to be before you are pulling this out at all was have you seen structured information, structured writing sort of represented within a domain user interface? But of course, the directamente mural is sort of natively zoomable by virtue of the fact that people can, can, can walk in and out of it. Because, yeah, that the the size of the type sort of provides its own sort of zoom basis. Have you have are you have you experienced or have you been involved with anything that uses sort of substantially even more different type size for this? One of the things that has occurred to me when I’ve been in museums and art galleries is that there’s often sort of a 10 times or or more difference in the type size between the things that you see as you walk into a door, walk into a gallery versus the things that you see is sort of the little name cards next to art spaces. So I’m curious, there are very different types, sizes in this, but have you seen people sort of really leverage the the notion of a freely zooming thing because even this still has an implicit Zoom level, implicit Zoom range in terms of the size of the cafeteria it was deployed in and the maximum distance that you can sort of expect to read out and things like that.

Bob Horn: Oh, well. Uh, one of the things that I mostly made these for, uh, not not for online use, but for, um, non-COVID conditions where people can walk up to them. And one of the things that human beings are very good is that we have legs and and eyes and we can move closer or farther back from a mural like that very easily. And in fact, a group of six or eight or 10 people can get up out of their chairs and gather around one of these. Just as one can, as you’re suggesting in a museum. And so. Yes, one, you know, I think there’s a lot of experimentation to be done. I mainly when I worked on this, worked on what’s the problem today? I mean, the mural that you see here was first made for the team of scientists and administrators in this agency who were who were planning for a blue ribbon government review of their whole program. And the managing director says, We’ve got to get aligned. We’ve got to get our whole team of of 60 or 70 people, mostly scientists and administrators, all on the same page. How do we do that? That’s your job, Mr. Horn. Um, and so, you know, that took a considerable process, but it’s one which they can continue to use, and they then after we had this, we can print these out, of course, on roughly four foot by 12 foot segments on on these large printers.

Bob Horn: They have printed one out, laminated it and put it in their cafeteria so that as you go through the line in the cafeteria, on the on, when you’re not looking at the food, you’re looking at the mural. And so it’s used there for briefing purposes, for example. Other ones of mine are used in boardrooms. A major companies, for example, I’ve saw one in in the in the boardroom of warehouses there, which is a big American forestry company. A mural that I made for the next 40 years of of sustainability. This is a way for me, this is a way of being able to think about larger subjects and yet drill down and keep the it’s it’s one of those things of how do we keep the context, how do we manage context? And I think we do that differently individually, which which was this this, you know, the this morning and other mornings like this in the future of text discussions are very important how we do it for ourselves. But also then how we do it for teams of people who are decision makers who are going to be making decisions about the next forty thousand years.

Brandel Zachernuk : And I’m curious. Sorry, Mark. Stop asking questions one day.

Mark Anderson: Don’t forget you want to ask Bob about this as well.

Brandel Zachernuk : I did. I did. Oh, here, he replied. And private to it. So yeah, I’m looking forward to sort of playing with it and saying what, what, what possibilities there might be to get it into a print on demand. In your experience building these things, I mean, a mural obviously has a canonical view. You don’t get to move the the pieces of information around based on the user and things like that when it’s printed out and laminated. Have you? What are your thoughts on having a canonical view or a singular view of the spatial arrangement of information like this versus having something that can kind of be flexibly rearranged? Because obviously, one significant and important benefit of this is that everybody will always agree where all of the pieces of information are. So if you want to find something, you just need to go to the same spot and it has the ability to trigger, you know, basically a proprioceptive sort of awareness of space. But, you know, obviously then the downside is that it only has one modality. It only has the ability to present one one view of that thing. Have you seen or experienced alternatives where things move around and what? What’s your take on the sort of relative benefits and things?

Bob Horn: Well, it’s certainly possible. You know, I could put up the illustrator and Adobe Illustrator version of this. And you guys, if you, you know, if you or you could say, move do that or if if if one could share screens and actually use my my Adobe illustrator, I know if that’s possible, you can move things around. What I have experienced is at the University of Illinois, Chicago, five years ago, six years ago, maybe I gave they have a large screen that is a computer driven screen that is five feet high and twenty six feet long. I put, uh, you know, fifteen or twenty or twenty five of my murals over in little postage stamp size and just clicking on them, they would pop out and then the students or the that. In this case it was students, but anybody in the room who had laptops and wanted to change to to at least add things to it or add comments to it, could do it from where they were sitting. So they had the beginnings of an interaction with a standard one without being able to move things around. But I, you know, in principle, we could move things around and in principle we’d want the the link to the documents, the zoom through able documents behind them to be still attached so we don’t lose them and so forth. But that’s about as far as we are, at least as far as I am, I don’t know if other other people have moved. Move further on this, I don’t know.

Brandel Zachernuk : Mm-hmm. I sent a link to to the people in the Twitter group, but not to that that whole email of some work that Jim Holland has been doing on document agnostic information spaces where he is disinterested or has a hostility to the idea of an application or of a document, but that you have information that you collect in a space such that you can sort of go into and go through and past. I think that this has very interesting parallels to what you’re describing of actually being able to to get to the detail that’s actually beyond the level of detail represented in this poster. So I think I think there are some really interesting things there.

Bob Horn: I don’t I don’t want to monopolize this conversation. It’s just I just wanted to present one one extra possibility. Yeah. And so I’ll I’ll if if it’s agreeable, I’ll close down this one. I’m very happy you have my email and horn Bob and EarthLink dot net and I’ll be happy to. Anybody who wants to drill into this sort of thing, I’m very happy to assume with you sometime over the next month or so.

Brandel Zachernuk : Thank you. Yeah. I think something that will be pretty, pretty low hanging fruit actually is to put this in a virtual reality at that 40 foot size that you describe. And and just just just walk along it, you know, just have teleportation ability. And I think that would be a really neat as a as a representation because obviously everybody at Nerax has had in the before times the ability to experience information at the scale. But it’s it’s something that you know, anybody could have with a headset, so it’ll be fun to play with. So I definitely intend to do that.

Bob Horn: Thank you. I have this shared copyright to all of two of all of 40 or 50 of these kinds of murals, different at different, you know, dimensions and so we can play with any of them and use them as experimental material.

Brandel Zachernuk : I think that would be really, really interesting to to be able to to kind of yield up in an environment like that because when I offer

Bob Horn: This group,

Brandel Zachernuk : Thank you.

Frode Hegland: It would be interesting. I saw some of Adam’s tweets, which is keeping secret because of a separate machine. You know, how many characters were you able to put into the environment? It was quite a large number, right?

Adam Wern: Yeah, half a million characters flying through it. So let’s see. I moved it to to my Zoom computer now. It’s not that secret.

Frode Hegland: Just, I know I’m just I’m just taking you on.

Adam Wern: But as long as I wouldn’t like to do something secret, but no, no, no. Let me share

Frode Hegland: This. Hang on, Adam. Just just just briefly before that, just really briefly. An additional thing that would be very interesting is to take Bob’s poster as it is. And put it into a space like that just to see, because it’s how many megabytes is the actual PDF both, you know, or anyone else.

Mark Anderson: One second.

Frode Hegland: Because even that without any further

Mark Anderson: 2.7 megabytes, sorry, two point seven megabytes.

Frode Hegland: So that should render. While right, there will be an interesting experiment, wouldn’t it, guys without any hyper things happening? A lot of thumbs, for the record. Ok. Yeah, sorry. Over to you, Adam. Just wanted to run that past, you guys.

Adam Wern: So let’s see. Can you see my screen? So this is the prototype I did before with the flat, where you drag things out to the margin and keep them the active reader thing. And then I moved it into 3D. So now we render a text in in 3D. It’s this visible to. Um, yep, looks great. So the idea is to it’s the same thing, it’s it’s nothing magic about 3-D because text is pretty flat in its in its usual form. But I wanted to just try to see what happens if you if you drag things out in 3D and place it in 3D. How does it feel? Basically, so this is on the screen you drag out or clip out interesting parts of a document to the to the side and build new kind of small either summaries. I imagine that you could do summaries of the document, for example, to the to the left, you do summaries and to the right there is no real margin, but it’s floating in space now, but that you can continue. Reading actively, I think it is it’s a thing I’m doing here because it’s so easy to scan documents, I really want the more active view of it taking objects or taking text out, rearranging it and writing more, writing your own notes here.

Adam Wern: So I’m just playing with the idea. How? How will this feel in 3-D taking flat documents, pulling things out, but in a way that it’s actually in in more dimensions? So I imagine that you could curve the text or put it in other directions. I haven’t done that yet, and I want to combine this with the hand hand gestures. You actually pull the pull the text out with your hands and and put it in whatever direction you want to put down. So you can rotate, rotate it by your hands. Because that is one thing that I find so fascinating with with hand interaction is that rotating rotating objects, which is quite messy in in a regular interface, it interfaces and flat. The interface is so, so easy in 3D or with the hands to put things on the side and in strange angles. So this combined with let’s see.

Frode Hegland: And when you say in 3-D, you meant in virtual reality, 3-D rather than clicking mouse on a screen 3-D, right? Yes.

Adam Wern: Yeah, yep. It would be nice to have your hands available in front of a flat screen as well, but it presents other problems because then you need to project the hands a bit into the screen, I guess, to to see where it is in the screen. Brandel may know more about that. Did it? Didn’t you do? Yes. How was that to predict it on the screen?

Brandel Zachernuk : Yes. So there’s a device. So I mean, you can do it optically if you have a system that is able to do to the image based segmentation and Google may have out of the box solutions for that, but otherwise a vision system that makes use of the implied foreshortening of finger joints and things, we’ll be able to kind of recover joint pose. But what I used was something called a lip motion, but $880 a little thing the shape of a USB stick or a thick USB stick. And it’s got it’s got a couple of infrared cameras and a couple of infrared floodlights with pulse width modulation lights so that it has does that shape recovery but from multiple vantage points? And then, yeah, you end up you have a position in space that it sort of corresponds to. And then you yeah, like you can make it so that it’s always relative to the camera position or you can have a locked off camera or you can tape it to the front of a head mounted display, which is what most people have done with them in practice. And they. And so, yeah, I think it’s it’s actually sort of stock on on things like the various XB one. That’s the way that they facilitated this by making use of an ultra leap device. But yeah, it’s it’s very, very straightforward if you don’t want to do it in VR. But if you do want to do it in VR, then you can just use an Oculus Quest with hand-tracking enabled and you get the same thing. So yeah.

Frode Hegland: Adam, have you been in VR with this or have you not built it for

Fabien Benetou: Vr 3D yet?

Adam Wern: No, I haven’t been because you need to. You need to sell your soul to be a developer, assigning all NDAs to get things, to get debugging. It’s it’s quite a. Yeah.

Frode Hegland: But can’t you use WebEx are as bad as asking and some of them?

Adam Wern: But then I need to debug the things, especially when I work with hands. I need to.

Fabien Benetou: Sorry to interrupt, but you can indeed. I don’t like Metta to be very direct, but they do good hardware. Sadly, very sadly. You don’t need to register a thing for anything, especially if you can have a second hand quest one. And more importantly, you can debug with, for example, Firefox to Verbeke, an experimental open-source with browser with a remote debugging. So you still do have quite a bit of

Adam Wern: Ok, yeah, I’ll start to be walking down that track, but it was quite old versions of Firefox, and I gave up a lot along the way. I will continue that and will try it.

Fabien Benetou: But TED this last week because Ejaria started to take over from Firefox Reality. So it’s alive again.

Adam Wern: Ok, nice. So the idea is to try to answer Brendan’s question again here. What I was imagining it was combining the kind of here is a future textbook, half a million characters on screen right now. The idea is to see the articles to the length of the articles and to get the sense of it and color coded in some way. Now it’s just random colors for different articles. And of course, it’s not very readable to read text like this with this contrast, but to have it in a distance. I want to see the labels with the names of the articles, but also color coding for things I like or different. If it’s about typography or or far future of text or something. So a way of color coding it and immediately stepping into an article with this other view combined so you can actually see your own, your own comments and the things you pulled out of that view in place here. So that’s one way of of doing it. And there is a kind of this fundamental fundamental. A conflict between having a fixed fixed layout. Coming back to this view, every time you visit the future of a textbook which is good for remembering things.

Adam Wern: But another idea is to to actually just bring in new article articles every time, shape your view, make it be very dynamic, dynamic, but then you get yeah, then you lose the sense of location that I find very valuable myself. So this is what I’ve been playing with, but I really like having having this kind of fluid. There is no you don’t jump into the document, really. You have you see many articles, but you still. And you can see an overview with some articles, you can see an overview with all articles, you can browse and look at your comments from from a distance and then you can go in and actually read. I really like that interaction. It removes so much of it of the interface. So I think it’s really worth exploring. So. So and this is 3-D now I just locked the view. So you are very so it looks very flat, but of course, you can spread it out over walls or cylinders or anything else that you like. That’s that’s what I’ve been playing with this week, just showing lots of text. Anything else?

Frode Hegland: I think we’re all very impressed. It was really cool in terms of the numbers, I don’t know if you count every character and space as a character, you have one million characters right

Adam Wern: There, a half a million, I think, right now, but it doesn’t feel like that is

Frode Hegland: Now. But in the book, if you count every character, including spaces, it is actually a million years.

Adam Wern: And I have half of the articles here right now, so it adds up. Yeah.

Frode Hegland: Now obvious question, obviously, you know, as a community, we need to help you get it into VR, ASAP, ASAP. That should be a new acronym. So we can all play it with our Oculus PCs, but also things like TED Nelson’s visible lines for citations. We need to start playing with that because that means I can send TED an email and he can say What blah blah blah. And then he’ll come, and then he’ll be very loud in a presentation and very useful asking all kinds of weird questions. What do you think of trying to implement that at some point?

Adam Wern: Uh, yeah, I’m not too big on lines here, I did do the opposite where I hoover over and highlight the corresponding passage.

Frode Hegland: I’m not saying the line should be there all the time, Adam. I think that just like you said, if you change the view, you need a way to snap back to a default view. The way that you can hide stuff absolutely becomes completely important. That’s going to I think there’s going to be a major as major research effort to hide things as it is to how to show things. And I’m going to shush a bit while Mark and then Peter says something.

Mark Anderson: Yeah, I’ll come back to the thing of derision raised, because that was just really interesting watching Adam stuff and I’m thinking, Yeah, you know, yes, I was. And when you said, Well, OK, but you keep having to come back to the start place, well, I thought, you urinate. You’re talking about trails. So if you’ll start places sort of a bit beyond the front door, as long as there’s an ability, in a sense, I mean for you, you might know where you left off. But if you wanted to allow somebody else to enter the space and get to the same starting point, essentially you’re just describing a sort of trail, you know, don’t start here, you know, walk into the room next door, turn right as it were, you know, if we’re in a physical space. So that’s quite interesting. Yeah, I too saw all those things and thought I thought of one of sort of Ted’s early demos, sort of Xanadu and things. The interesting point, though, is, of course, his thing was all about same text, effectively factory turning up in different places, and that actually happens very infrequently sometimes. I mean, the thing is, it’s really hard to get a probably a dataset that’s just sitting floating around there. There’s got an awful lot of stuff. But what we do have is we’ve got the ACM in reference citation. So you can if you can live with the fact that the references are not tied to specific text blocks either end.

Mark Anderson: We already have a corpus of just over a thousand nodes with links between them, and we could play around with that. And I did. I had a look, Nodar. Actually, I’ll have to sort of I’ll have to put some money in their pocket, but I’m going to wait for a week or so soon. But when I come back that I have a try at that because they allow a CSV import, because when I look at some of these, I mean, I think one or two percent ready, they look a bit rinky dink because you get in there and you make a couple of blobs in space and pull a line. I don’t want to start there. I want to start with, you know, several hundreds of items and things. Just making the making the blobs isn’t really that sexy. It’s actually seeing, you know, a developed network hypertext in a 3D space and saying, Well, you know, is it completely impossible to understand or can I start to do things with it? So that’s another angle I’d agree this thing about. Yes, one of I think part of the journey here is figuring out how we hide things. And I do think from your experiment from Adam experiments last year with the again with the ACM data, this idea of being able to just effectively dim stuff down so you reduce its visual salience.

Mark Anderson: Is that that feels a very right way to go because you don’t want to lose the data, you just want to push some things. So they are less prominent. Well, there may be hidden. I mean, there may be visually hidden completely, but they’re actually sort of there as it were. They’re they’re present in the data set you’re playing with. They’re just not visualized this second, because the whole idea is to reduce noise and concentrate there. I just flew back to the earlier thing. It won’t surprise some here that when I saw Bob Singer, I thought, Oh, that looks like a hypertext to me. It’s his big thing. And my immediate thought is, well, actually, what I’d really like to see is at this point, I’m less interested in the picture. I’d like to see all the bits that went into it and all the links. And then basically mapped that back onto the picture because one interesting thing is he rightly said, OK, I can put up this thing or, you know, thinking of some work Fabian is doing, you know, sort of European government or something that if you, you know, you got some big picture you want to show, but we’re not supposed to be getting on an aircraft these days and flying around and burning carbon. So if I can sit in my town or in my house and I can look at this and I can interact with it.

Mark Anderson: And then a sort of enhanced VR sort of an AR VR is I can look at this thing. Oh, right, I need to know more about this. I want to know more about this, and I can just pull that information forward from it. And that seems that none of that to me strikes me as innately is work to do, but it’s not innately complex. I mean, we’ve got we’ve got the sort of bits to do that, and it seems to me a really interesting project to do here is something that exists. Can we make can we take something that arguably already works because it has been working already? And can we make it better? And there’s and then and that to me, constitutes a valid test, and the answer may be no. But we won’t know if we don’t try. But the thing is, there is a complete I think that I think the lesson I’ve learned on the way is take something that’s already complete and do something with it because the amount of time it takes to make up data only means you end up with something that still isn’t nearly as random as you wanted it to be or difficult. So I’m quite interested in in us, in, you know, from our various social skills doing something more potentially with that.

Brandel Zachernuk : Yeah, so Bob just rejoined in the middle of that. The thing that that Mark was talking about, Bob was with something like that in direct mural having access to the bibliography. And he said that looks like a that looks like a hypertext to me. And so what possibilities there might be to be able to navigate the information beyond that page as well? If you have, you know, you have

Mark Anderson: A hypertext and feed it through the rollers, that’s what you end up with. And that’s not being rude to, but in the sense that, you know, so it’s been flat for a very deliberate purpose and very carefully structured into a 2D surface. But the fun would be to explode that back out into its sort of end dimensional bits and then reattach it to the 2D surface inside VR. So the point being that we can we can step from the actual mural on the wall into the same artifact in a virtual space. But but then have effectively have an augmented reality experience in the virtual space because I’m suddenly interested in ethical policy or I’m interested in just, you know, dystopian future or whatever, you know. But I want to know more about this point. I was going to say it’s unfortunate all the hard work done by and our government is still turning our little island into a massive dumpster fire. So. Anyway, Peter, interesting.

Frode Hegland: Peter, do you have something on that or something different?

Bob Horn: A couple of different things. First of all, I’m pasting in a link to a YouTube video of a system for doing 3D zoomable interfaces and that actually, if you look at the video actually like turn and twist to look at the perspective from different angles as it goes through, it looks like there’s some sort of a paid web app that has some of the functionality of blunders Grease Pencil, but with a

Frode Hegland: Simplified interface for creating multilayered sketches that can

Bob Horn: Also turn and move off at a different angle in 3D space from the simple flat on view that you normally see. It also occurs to me that it would be nice to have some sort of a system of spatially constrained 3D dragging to ensure that instead of being able to arbitrarily twist something in any direction

Fabien Benetou: Where you might actually wind up

Frode Hegland: Having the text that you’re trying to comment on.

Bob Horn: Show up edge on and not even realize where it is anymore. Basically, I have like a pre done 3D glide path that you could move things on to preset nodes, sort of like a snap to grid only in 3D for moving things around the three space. I think that’d be a really handy afford to try to experiment with. I wanted to ask Adam what libraries he is using to build that wonderful visualization he showed us. Also, I wanted to mention that if anyone on your local television can get a viewing of J Arena, which is an NHK program from Japan, they had the episode this week on Super Human Sport, and that was basically focusing on new AR and VR based sports

Fabien Benetou: That were being developed in Japan.

Bob Horn: Absolutely fascinating. There is a game called Huddle. Which is an air game where you’re actually generating representations of 3-D shields in front of you and firing energy balls at opponents while running around on an actual court. Which of course, has all the advantage of your movement being under your control, as opposed to you being moved in three spaces. So presumably those players, the way they’re running around, they’re obviously not having any sim sickness issues whatsoever. Or they’d be keeling over on the court, which they’re clearly not doing. So that was very intriguing. There is also a superhuman sport academy. I’ll try to find the link and paste that into the chat and. That’s basically everything that I had piled up in the queue.

Frode Hegland: Thanks for that. The mental canvas is pretty interesting, I have seen it before, but I think it was very relevant that you brought it up here. Any specific comments on that or onto? I’m going to I’ve

Mark Anderson: Got a quick one. What that made me think of is, I don’t know. I’m sure it’s probably a sort of a word for it. But you know, there’s people who sort of you have a conference or something and someone’s basically drawing, you know, busy. It’s marvelous, you know, drawing as it were in real time and doing that. And I thought when I saw the video that Peter posted, my mind went to the idea of, Well, you know, normally they do that on a piece of paper at the side of the room. Well, if that effectively is being done and effectively put into a virtual space, then you can begin to do again. Sort of then thinking back to, you know, Bob’s long infographics sort of thing. Well, you could begin to be augmented that so effectively. Another person, for instance, could be augmenting that with information so that you you not only have this nice, stylized, you know, deliberately stylized but engaging presentation of the subject matter at hand. But you actually have in pretty close time actual factual, actionable links because I think this thing is quite interesting. I get back to the I recall the thing, you know, Bob mentioned about the fact about decision makers, and I know some that this is a term of derision for many. But but you know, there are poor, benighted people who have to sit there and make difficult decisions based on frankly far less information than they like or have time to assemble.

Mark Anderson: So. Yeah, having making things more accessible is is a good, a good and valuable societal tool, I think so. I find all this really quite exciting because, you know, I mean, it may be well, it may be a long way away from entertainment and that kind of thing, but it is a really useful building block, you know, because I’ve been I’ve sort of been I dusted this off me back to it, and I’m very aware that, you know, I’m very aware of the point of where 30 years on from the time frame of reference, but I don’t think that invalidates a lot of the ideas in it at all. You know, it’s just that it might look a bit different. We did it now, but you know, because print has moved on things, but that’s, you know, I see through all that and I still think this idea of it goes back to this early conversation we had about having more useful table of contents in the loosest sense of the word. So not in necessarily in the print sense, but just having a a richer indexing to objects, you know, that decomposes down as far as you need to. Again, lends itself to this sort of presentation, I think.

Bob Horn: If we had a tool like that, boy, I sure could have used it over the last 20 years. By the way, all the information mural work that I’ve just been showing the nuclear waste and other murals was all conceived after writing. Mapping Hypertext. I had no idea of of what I would be. I would be called upon to make these large murals when I was writing mapping hypertext. Hmm. But I used the notion of the chunking of information, in particular the the sections on information mapping and argumentation mapping and so forth are our parts of that, by the way we did. We also did a major argumentation mapping project for the for the same UK agency in charge of nuclear waste disposal. Because you have to be able to when you’ve got very complex information, you have to be able to structure it in different ways. Yeah. And and they, you know, they needed to be able to show why they were why the plans that they had made the forty thousand year plans, why certain aspects of them had to be the way they are. Yeah, and not the way, some other way. Other possibilities were and and behind those, I mean, they have documents, you know, they all had documents and my colleague who was who was there in the UK was reading many of those documents. Fortunately, I didn’t have to. But that’s a team effort. Always again, it was on our side as well as on their side.

Mark Anderson: Hmm. I mean, another interesting part of the potential deconstruction is obviously, if I look at the sort of this big mural, the textual passages self-evidently sort of suggest themselves as being some kind of type textual object because it must come from, you know, some text was taken from somewhere. So that object exists. So find it. The more interesting part is then said, Well, what about the what you might call a purely visual element? So there’s some deliberate streaming in there. You know, we got the horizontal strands in different topics, the timeline. But there are, you know, there are effectively visual prompts from the areas and things. And there’s an interesting I mean, I don’t know the answers, but I could see that that’s another interesting sort of thing to be done, which is to work out again. What of the the non textual part also lends itself to some form of enhancement or because the whole idea is you’ve got to this point, I want I need to know more about this bit as you sort of prod your finger into it and hopefully you’ll get something back. And obviously, if it’s a little box of text on, you probably get the the source of that of that text and and, you know, in a well structured metadata set, you’d not only have that text, but you’d have the on links from it. And you know, that’s where we still see things like visual matter being in the round trying to help with the glue of of all of that. But yes, you know, it just comes to me, as I say, this is, you know, how how you do the same with some of the more purely visual. Yeah.

Bob Horn: Can I respond? You know, if you’re going to go on to another topic and you want to, that’s fine. But you like me to respond to the one you just, you know, and

Mark Anderson: I’m happy, afraid you’re in the chair and I see people have their hands

Frode Hegland: Up. Yeah, yeah, sure. Yeah, I mean, specifically, especially if it goes into a kind of the digital realm, I think that’s very worthwhile. But please, please continue.

Bob Horn: Well, yes, he asked about the the visual elements that tie together, in some cases structure for us perceptually in other cases. I have another book called Visual Language. It’s also out of print, which which is downloadable from my website, although again, a badly one, which the Internet Archive did where they. It’s intended to be read as a double page spread, but they they digitally scanned it as single pages. So you, you know, you have a little extra problem to putting those together. But anyway, from a linguistic standpoint, it is an analysis of the visual and verbal elements and how they at least as far as as as we could determine twenty years ago that these are integrated in many different ways also since then. For example, one of the things that I found was that arrows have a great, different, great, many different meanings I’ve I’ve collected and got somewhere on my hard drive a visualization of two hundred and thirteen meanings of arrows, which I have illustrated that is, these are not theoretical. These are found in actual documents, which I. Not do not have the references for anymore, because I wasn’t doing that. And and it wasn’t possible either at the time. It still isn’t possible. So there’s there’s work to be done to incrementally have the tools for thinking. And for then for manipulating the things that we want to do.

Frode Hegland: Absolutely, Bob. I completely agree, and that brings me up to the reason I have my hand up, which is, you know, I can do a lot in Photoshop, I can video edit. We all have different technical skills, but the kind of thing that Adam showed today that I know that Fabian and of course, Brandel also do. I don’t currently have the skills to do that, and it is a huge source of frustration, obviously. You know, for me to say, Oh Adam, could you possibly try so-and-so? And I think it’s getting more and more important for us to do more and more kinds of experiments. So I’m wondering, could we even have a tiny budget for this? But can we even pay someone to make an environment in whatever Fabian, Adam and Brandel thinks would be the best technical environments where all they do is allow for the import of text through whatever means we decide on and then a huge amount of parameters we can mess around with. Because, you know, the creative tension here, we have to decide what to do. I mean, it is an amazing time to be alive. I have made a word processor in a PDF viewer that most of you are aware, of course, cost me a huge amount of money on a personal level. Someone else did the actual work, but the fact that it’s even possible is absolutely amazing. That I am not Microsoft, and I could do that, and it’s as powerful as some of the competing platforms. How can we get to a point of doing that here? Does anyone have some technical suggestions on this?

Brandel Zachernuk : And that’s one of the rationales that I have for being able to make use of age HTML to turn it into virtual realities is dropping the general skill level required in order to make use of it. I’m very interested in pursuing a conversation with the people who create and maintain troika to see if they have. They would be interested in making use of the approach that I’ve taken for being able to render an entire document with its structure and hierarchy, rich text and markup into into their system because they’re they’re scientists, since field text rendering is obviously superior. So I mean, but but it’s like facilitating that makes it easier and makes it more reachable by people who have basic HTML editing as a skill. If people have have no development capacity whatsoever, it doesn’t help them too much, but in terms of how many people can do can do what, I think that’s that’s a very important sort of. First target and then sort of getting those examples out there to so that people are actually aware of and interested in it. I appreciate your your notion of of of of paying people for it.

Brandel Zachernuk : But I also think that if you can get a ready audience of undergraduates who are aware of the possibilities for information, manipulation and things like that, then then that that that is very exciting to me because I think that as I’ve said in the past, most people believe virtual reality at this point is is mostly for entertainment. And so having the opportunity and ability to play around it with it for expressive and productive purposes is something that can both justify the expenditure of their headsets for their computer science department and personally, but also be something that that can kind of provide a rationale and a basis for further study and things. So I think that relationships with with schools and and just popularizing what kind of capabilities and opportunities exist and what little skill is required, particularly on the basis of as a result of what, what, what happens with my my web daily is your stuff and troika and all of these other systems making those as approachable as possible as it does about as much as as expecting to pay anybody to do it as well.

Frode Hegland: Yeah, that’s that’s really worthwhile, and thanks for that. Brandel Fabiano, see you have your hand up with just a little bit more on this topic. So Adam and I have been sparring today a little bit over EdgeHTML visual matter and all of that stuff. And I think it’s becoming really, really important. And I think we need to decide on a few things, not in every great detail. But, you know, I have to submit soon the ACM hypertext paper on visual matter, where I introduce the fact that we are going into HTML and VR. So it may be at a different forum. We can go into that because what I wrote earlier today in the paper that I’m writing, we should probably do two things in the header of the HTML page dump the entire normal visual matter as we make it today. Just put it in there, but then also have one version that follows the syntax of HTML more clearly. And then, of course, we get into the issue of how do we do the addressing when we say a certain heading a special, how do we make sure it’s the right heading? Also, the glossary There is a way, of course, in HTML to deal with the glossary. Now how do we put those things in? Because I could relatively easily have author exports to this special HTML is not a problem. It’s very close to what we already do. And then we can start having an experiment where when Adam has the view that he just has instead of a dead link and, you know, citation and brackets that it has nothing behind it.

Frode Hegland: If we together as a community, back and forth fight for the best idea of the system, knowing that when something in brackets that has a certain meaning. But remember, I only have 10 days to do that, which means extremely little time in terms of this community because we don’t meet very often only twice a week, of course. So does anyone have any ideas on that and does anyone have any suggestion how to deal with that offline? Is that something that may be those who have a perspective on it can maybe put in a blog post or in an email or something, and then we just fight it out that way because we all have the same goal. So let’s just go at it, right? Concerned we can have real documents, I mean, my thesis is quite long as it should be as a huge amount of references, all of it’s properly done. If I export into this new thing, then instantly we have a large document with a huge amount of citations and all that stuff that we can start working with. We, you know, and a lot of other stuff that can put in anyway. Yeah, please think about that. I’m going to give the baton over to IBM and then Bob and Adam,

Fabien Benetou: So I’ll I’ll share my screen briefly again. Two little thing that I hope might help this. Of course, you’re still at some point development because it’s defining new DNA structure and new ways to interact. But this was a little example of a muted but where my friend and colleague is in VR playing with the game, and then I’m programming and updating the game live as they are. And I think has enough audience in order to it has to be prepared. Of course, you can’t build a game from scratch or the one the person spends heavily in the headset. I mean, if they’re really patient, why not? But otherwise, you can toggle blocks of code and experiments. I think to to explore, that’s a pretty good way to do it. And the other thing is a little homage I did four papers were basically, you do your block based programming in VR that’s also in home. So it’s also social so you can guide someone through and then you have a list of instruction steps. You move them around and see on the panel. It runs basically the script you want to do, and I think it means the person who is using it can find a way to script in video to the content they want to manipulate. So it doesn’t solve it. The core of the issue where you still have to have someone that’s going to code those either blocks or to be able to collaborate live. But I think there are better ways now to just read the same specification to someone and say, implement this, especially at this stage. If I understood correctly again a very naive about the whole process, but that there are still maybe more questions than answering, just like let’s build this.

Frode Hegland: Yeah, super cool, thank you. Was that in response to where we’re just talking about or did you also have something else since you have your hand up? Fabian?

Fabien Benetou: Yeah, I had something before. Thank you that that in my opinion, I forgot who mentioned it, but about grids, I think positioning on a grid. To me, that’s been the biggest challenge that on on one hand, you have the ore that provides you arguably natural affordances like for free. You got a world in front of you, which is pretty amazing. And then you can move and all that. I mean, you know, all this. But what what’s been very hard for me is what’s not natural anymore, but in a good way, it’s so snapping on a grid, for example, at some point it’s super natural. It’s something you couldn’t do to in your life. But that’s the kind of reference that makes the tool even more interesting in addition of what you get for free in the properties of your. But I have no idea except of what I’ve done by experimentation. And that’s also why I think one of the the two ways of making the time to the fastest time to iteration when you have this idea is, in my opinion, so fundamental.

Frode Hegland: Thanks very much. Bob, are you here? I can’t see you, but I can see your hand.

Bob Horn: Yeah, I well, I’ve decided in Zoom when I’m not interested in the topic of being discussed that to turn off my visual.

Frode Hegland: Got you. Take your hand down. That was a mixed message.

Bob Horn: I wanted to respond to the to the previous question, which I think it was you who proposed it. And that is, I have to ask, are you really serious about wanting some money to do some serious work? On the kind of problems, at least, that I was showing today and people were discussing. Uh, if you are, then there are or there are ways that groups get together and do that. And one of the things they do is they accumulate. You have a way of organizing and accumulating the problems that we see are addressed and then we have a way of discussing which ones of those are priority first. And then we write a proposal and there are proposals that are fundable in Europe by the European Commission and maybe other places. And you have to go through that kind of process. And if you’re not willing, if you’re not serious about this, then you know, then we’re just in a discussion.

Frode Hegland: Well, that was very pointed, Bob. The thing is this the community is fundamentally about the future of Texas. So there are many different aspects we talk about. We’ve been actually, I think we’re very close to having our 100 conversation. And as you know, we’ve had 10 annual symposium and two books and now we have a journal and a newsletter. So we’re very serious. But it’s also a matter of, you know, we don’t want to just settle down and become a company, either. So it’s a matter of finding the best ways to experiment and be inspired and do something. One of the key things that we’re working on is this whole visual media thing. And yes, I know it’s my PhD thesis, and I’m not saying everybody needs to use it, but I am saying that it is very, very important to combine rich metadata. Things need to know what they are. The origin of that thought for me was the game development. The game called Crysis, which was one of the first large world games. I wrote an article in Edge magazine, but one of the designers have been able to find it to my detriment, where the designer said making the A.I., which is what the article was about, wasn’t that hard. What the key was to make sure that everything in the game knows what it is. So a piece of wood knows to needs to know how much force to break it. Can it be seen through? Can it burn? All of these things need to be known that way. You can have a richly interactive environment. Are you shaking your head?

Bob Horn: I’m I’m saying not all the things, you know, as soon as you put in all, I think you’re wrong. I think selectively important things need to be connected. Yes and no about each other, but not everything. But anyway, go, please go on. Because again, you’re not addressing the at least you’re saying to me for the last. What you’ve just said is that this group at least is not, not the group that’s going to be serious about the kind of of of questions that I could use I could have used last last year on these projects and the last 20 years. And if they’re not interested in that and actually making progress on it, I’m I’m saying absolutely making progress on it. And that is done by identifying what the gaps are now, but also getting it funded and doing it.

Frode Hegland: Yes, but Bob, this is not a community. For one thing, that’s kind of important to keep in mind. So let’s be careful with the use of the word serious because we are all very serious but a different aspects and we have different priorities. You know, we don’t have a CEO, we don’t have funding, and that may not even be the best thing to go about. We have a tiny amount of funding. We can use a little bit, but we’re not just going to make interactive augmented text murals, but that will be part of it. We’re also not just going to use my thing visual matter. There will be parts of it. We’re also not just going to be using data sets that Mark Anderson has arranged for us. Will that will be part of it, et cetera. What we’re trying to do is what a commercial organization cannot do, and that is have the freedom to be a little frustrated and trying to implement, but also the freedom to think in different directions. And that is quite important as well, because otherwise we would, you know, I mean, talking of which one of the notes I have here, I got an email from the company immersed, who makes virtual screens and VR where you can immerse yourself in work. They would like to present to the group, and I emailed some of you about that. I’m wondering if that’s an appropriate monthly meeting we should have or will that be to commercial? So as a side question, what do you guys think of that having a presentation from Amherst?

Brandel Zachernuk : Um, I mean, from what I’ve seen there, their heart is in the right place in terms of producing sort of via productivity sort of innovations, but I would be curious. I mean, it would be really nice to make sure that we can sort of temper the conversation beforehand so that it’s not a general sales conversation because I think that there’s a there’s a distinct possibility that they’re thinking in that context would end up being pretty superficial and being like, you’ve got screens like, Yeah, we we know we’ve been talking about this. So. So if if it’s possible to kind of make sure that they have have forewarning that what is most interesting is the conceptual basis on which you decided to put things on screens and have them in places and what sort of consequences come from having arbitrary space for that, that the big, the big murals and things, then? Absolutely. But in the event that they think that it’s just another organization to sell stuff to, then that’s going to be pretty boring until we can pull them up and actually talk about and talk about what they’ve learned and what what they understand as the the sort of the the nature of information in these new spaces.

Fabien Benetou: Yeah, I would strongly

Bob Horn: I want to I want to respond at least to say I was not suggesting a commercial application. That’s something that proto added a frame to my comments, which I reject. I was talking about getting research money from the European Union, and there’s a project that we worked on of, you know, five or six years ago that would be an on on on resources for the European community where there are documents behind. I made three murals and there are documents behind them. And so the material is there and the the union proposal could be made before the European Union to keep this up to date, so to speak, because information like a research project that we had, which was several million euros was, you know, it was over and then that was done. And and so things are frozen in time. And one of the things about this is that we need tools that keep things up to date. So I’m talking about non profit. Research, probably universal, certainly university and certainly multiple country involvement, if one were to do what I’m what I’m talking about as a serious project,

Frode Hegland: Sorry for using the example of a commercial project, but doing full proper research like that is similar kind of commitment. I mean, if you have someone in Europe who would be able to initiate something like that, it would be interesting to look at. But we all have day jobs, so to speak. So I think we couldn’t do a full work effort for one single project. And as you know, the UK where I am, we left the EU to our great detriment. So my university and other organisations don’t have much say in Europe anymore, but. But yeah, no, I hear what you’re saying and it’s and it’s worth thinking about. But Fabian, I’ve kept you muted for too long. I’m sorry.

Fabien Benetou: Very quickly from what I understood. So I don’t know Imus, but I would say any non open source solution would probably be interesting in the moment to learn a few things. But as a building block again, based on what I naively and superficially at least understood will never be good enough because it seems you really have to go back to, let’s say, the foundation of the display and how to put text correctly and how they are going to provide interesting things to learn from. But as we’re building it up, I at least I don’t see it.

Frode Hegland: Yeah, fair point, Brendan.

Brendan Langen: Uh, yeah, I kind of just wanted to add a note, a thought around what Bob is saying and kind of the conversation around focusing on a project for funding. I am pretty strongly in favor of finding an avenue that we can hone our focus in on and whether or not that would be something that would be aligned with acquiring funding to do a research project. I’m not entirely sure, but I do love the idea of using the existing body of work that we have, and this has been a concept we’ve touched on a lot. A frame like Bob’s murals. And if we didn’t feel particularly excited about nuclear waste, we can choose one or the other. There’s plenty of work that he’s done. Those are the types of projects that I think we can take to a new medium using, say, VR as an example of what this group can do. So that’s that’s really just the high level thoughts on this, but definitely interested in something like that. Now what it takes, that would be another discussion. What is the scope of the project? What would we want to aim for? But I am hugely in favor of it. Take me an existing body of work that we appreciate and an artifact.

Frode Hegland: Thank you for that, Brendan. So my frustration on that point is the flip side of the coin of the diversity of our larger community. So that’s the hard and good part, so I wrote a piece a couple of weeks ago on, oh, entering the new virtual reality thing, isn’t it amazing? And some of the feedback, including from my good friend Adam, what? What the hell are you doing talking on all of our behalf like this? I don’t mind coaching you, Adam, because your points were fair enough. Absolutely. But the thing is, you know, we now have a domain, a WordPress site. And so far, there hasn’t been anybody posting anything other than me. Right. So if if we’re going to do something together, we need to find a place that we can communicate what we want, right? And I hope that my early writings like in the journal, you know, I don’t own the journal, I’m the editor. But if you guys want something in it, it’s going to go in there. You know, like my editorial control over the future of textbooks has been very simple. If someone’s passionate about text that goes in, it’s not for me to judge whether it’s good or bad because I’m not smart enough to know that someone in a hundred years could maybe tell us so. It’s the same thing here. But you know, like Adam and Brandel are very good at making. Rather than feeling comfortable writing an academic thing that maybe Mark Anderson would be great at, I certainly am not. So therefore, my question to the community in starting with you, Adam, since you have your hand up is what is the best forum to go about deciding what to do? Because in these groups, sometimes we discuss it a bit and then we go into demo mode and we go in different directions. So Adam, please? Yes, go ahead.

Adam Wern: Yeah. When I joined this community, I followed it for a year over a year, and then I joined last year or last year. I posted things. I wrote things on the circle community. I got no responses, basically. Then. A few of the articles were deleted the whole thread because you reorganised things late this December. We decided to or there was seemed to be a consensus of consensus of using the future text dot org and do a site there. Suddenly, you had a very passionate lab idea that you implemented and registered a completely new domain and started a blog without really checking in with us. So it felt like it was your your passion project that that was kind of a bit impulsive, that it wasn’t what we discussed before and almost agreed on before. So we agree on many things and more things than we do, and there is a bit hard for me, at least to follow with everything. And if I post at this dot in for site, will it be there in a few months or will there be a new site? We will rebrand things. I am a bit hesitant to put my effort into that, especially when it takes so much time for me to write as a non-native English speaker. And I’m also a bit I want to I want my word to be not. So we have different styles. I want them to be so precise that I can’t even formulate anything. Maybe. And you are a bit more youth Brandel word. So we have different styles there as well. And well. Yeah. So that’s what I have to say, why I’m not really writing. And also it takes time. So every second I do that, this second I can’t do. Coding and a rather prototype things and talk about them, because I feel that that is more worthwhile at the moment. So every hour I write this and our I don’t code basically, and I don’t have much time writing that, yeah, OK.

Frode Hegland: So for me to respond to that, the you’re basically made a very eloquent argument in favor of visual matter. There. And the reason I say that is, you know, as Circle costs me $80 a month to maintain. Right. It’s insane. There was some basic people, but it’s all locked away, it’s all their own communities. I’m trying to basically get rid of it. And I’ve tried to have, you know, community around things. It’s not money that I can really afford to throw away. I can afford to invest it in our community, absolutely. But I can’t afford to just throw it away. So that’s why, you know, we tried it. It didn’t work for our type of thing. So the reason I say that what you said was a passion plea for visual matter is that I really, really feel super strongly that if something is worth saying or expressing, put it in a format that is robust and that doesn’t require a server so that despite everybody hating PDF, including me. If we put things in there, it doesn’t matter what format we use, right, we can cite from it, we can refer to it. We can have it on our hard drive. You can print it out. We can do whatever. That’s why I’m so passionate for document based discourse.

Frode Hegland: You know, every single one of these calls we have, I have automatically paid for machine translation and then I have to assign who is speaking one, it takes quite a bit of time. And it’s worth it because at the end of the day, I put all of this and a monthly journal that is just our transcripts and the special hosted one. We have a wonderful human Danilo right in will be maybe listening to this. We put that in so that we can have that because yes, doing circle or even our current one, it is not ideal. And I’m very glad that Bob is in here screaming and shouting and the very good sense bob. But you know, let’s be a little serious about our discourse. And Adam, you said a couple of weeks ago when there was something came up, we have to be serious. I agree. Fabian is the new person here, so he’s come into a family argument in the best sense. So if we have ways to discuss this or what to do, rather than Adam being fantastically clever coming and showing us a half a million words and us being speechless and not going to do that would be useful anyway, probably in some sense, please.

Fabien Benetou: Well, first, I mean, it’s really the topic itself is really interesting and how passionate you all are. I think it really shows how much you care. So first of all, that’s it is beautiful. It’s tension, but again, it’s because you care. I think that’s great. And on a more less a technical note, I think there is always tension for again, naively, let’s say somebody like Adam who just made a prototype who finds it the building itself of the prototype is a process and that I interpret that. So you tell me you stop me if I’m wrong, but I don’t put it to collapse it to another medium that doesn’t give it justice. But still, as I think Freud you’re expressing is that medium that, let’s say a PDF won’t do justice is going to be a way to achieve it, preserve it over time and eventually make the conversation progress, get feedback, et cetera. So it’s it’s it’s from what I can tell at least and from my own experience. Also, it is a normal tension and I I cannot want to even begin thinking of a new prototype if I know I’m going to have to do a point about it at the end. But then at the same time, in order for it to progress, for other people to give feedback on it, I know it will have to collapse to another medium that’s going to be this flat screen or this document.

Frode Hegland: Ok, Fabian, thank you for that, that was hugely valuable, and so here’s the thing. For the benefit of you and a reminder for everyone else, especially Bob, what visual matter is because I really think it can help because it doesn’t collapse. That’s the whole point, right? So as I said in the beginning, you have a normal in a normal book, one of the first pages, you have copyright information. Imagine putting that at the end of the documents. Plus, title equals date equals all that stuff. But you also had. Here are the headings what levels they are and who authored them. Here are glossary terms. And here is every single citation in the document in an orderly, human and machine readable way, right? So easily. What we can do is also take Bob’s mural behind him on the wall. Mark it up in such a way and maybe even use an equivalent of image map or whatever it might be. So the whole point is that and this is why I talked about the game crisis. And Bob, you’re right, not every single thing has to be every single detail. That’s why it’s a game or a model. But it means that if you have pieces that are self-aware and can communicate what they are to other pieces, amazing things can happen. And for instance, Adam made a very, very useful citation map based on hypertext proceedings, which was not using visual matter. But the idea was that visual media could be slaughtered in and it would work right. So when I talked to Dean Gregor, who founded the project right now, it doesn’t matter. She pointed out that visual meta is the first analog digital hybrid medium, because if you print out a document that has visual mirror and you scan it and you run OCR, you’ve lost no metadata.

Bob Horn: So I’m all for. I’m all for this. I like it. I’ve heard, I’ve heard you discuss it over and over again. I agree it’s passed. We got it done. We have some need. It may have some next steps.

Frode Hegland: Yes, IBM is new. Fabian is new today, so that’s why I had to explain that. Hmm. Okay. Hang on, Marc. Just really,

Bob Horn: Really. So one of the problems in the group is how to have a cumulative up agreements on what is next. And and I don’t see us trying to have an agreement having agreements.

Frode Hegland: Bob, can I please finish this little thread because it’s kind of one that so the point is, so next week, I’m presenting visual matter to the American National Institute for Standards Organization, and I so with Vint Cerf. Right. So these are the things that we’re doing and the thing that I had to spend a month discussing with them to persuade them is that VR is amazingly important for work. He also said it’s for games and social. He now agrees it’s important for work. So that is why I asked you, Fabian, in the beginning of this conversation. That is, imagine if there is a visual matter stuff, couldn’t that be read in VR? They are so that the thing that bob that is brought up, which is so important. So instead of just having a flat document, you can take the table of contents out for the side. You can choose to turn on and off lines to anything that it cites so you can build this environment because the semantic value of this rectangular document is available to you. Right, so this needs to be presented and it’s going to be presented very soon. And also, Adam has been hammering me about using the same logic with HTML because of course, HTML is better than PDF other than for archival reasons. Of course, it is no question from my mind. So where do we put that stuff in? Because one thing that I’m really scared of is that humanity goes into VR and AR as though it’s a ghetto. It cannot be a ghetto. It has to be the same knowledge that we have in other media. You have to be able to take just like you did with remarkable In-N-Out’s. But that’s the final bit of my little speech here. When you go into an augmented environment and you put things everywhere, it should be storable in a flat form. So when you go out to that and deal with it in flat, flat land, let’s call it that. And then you go back someday and augmented everything can go back to the same space. And that’s why an appendix can do that.

Bob Horn: And this is, I agree, and this is a beautiful way of of outlining a proposal. What you just did and I and I have been telling quite a number of people write proposals, not articles for journals and so forth. Right. Right. It, you know, it was a beautiful outline for such a thing and you know, it just needs one one step further. The next thing that we’re proposing to do because it’s essential for the next step is this and please fund that as a research project.

Frode Hegland: Well, said, Bob.

Fabien Benetou: Mark, what do you got? I’m just going

Mark Anderson: Round the circle, this in the sense that I don’t think there’s a lack of seriousness. Part of the thing, I think, is that a number of people here are drawn from so disparate perspectives. And what they’re actually interested in doing is is is moving the ball and sometimes it’s easy to do it. The thing is doing it because no one told you you could is often easier than sitting around being allowed to tell when you were allowed to start. And I think some really interesting things. I look at the last year. I mean, I just think, for instance, the visualizations that Adam did last year are really fascinating. It’s been really interesting showing those to some people who who have got really, really excited in a way they wouldn’t get excited about the subject matter. What they’re actually responding to is the visualization. When you talk to them, they they suddenly, you know, it’s this thing. Most people in the world do not understand what metadata is. It’s just the exhaust product of their organization, except they know they have a lot of it. And somehow it’s expensive and and worth something. But they don’t know how it sees our underpants gnomes thing, but actually beginning to put things together. This is where visual metaphor is a good glue, which is where the sort of visualizations we’ve been exploring in non VR and AR have been interesting. And in fact, they they map across because one of the other things is, OK, so what do we need to build? It’s actually quite difficult to do that within a formal structure, so it doesn’t.

Mark Anderson: It in no way argues against what I’ve been suggesting. It’s it. It just is. It’s a somewhat Faustian bargain. I mean, the point is that some there will come a point where, you know, all or some of this may end up pushing into a project, whether it’s necessary to commit to it. At this point, I don’t know. Mostly because most of the people making traction here, you know, are doing something already. So to actually commit to a project. It’s kind of difficult because who who are the people going to be in the project because it might not be the people doing it. And I think I think so. That’s that’s why there might appear to be a reticence. It’s not. So it’s not a lack of serious seriousness of intent. It’s just that there’s there’s a dynamic and just this fact that sometimes it’s easier to make progress if you’re not having to sit in front of people in the peanut gallery, you know, kicking the tires and saying, Oh, can I have it in 15 points? Or, you know, why is it not green who have no understanding of what’s going on but just feel they need to stick there or in? Whereas actually here we’ve had really interesting discussions that I find always illuminating because I learn stuff you know about what might be possible, and it raises both interesting challenges, but also really interesting insights.

Frode Hegland: Yeah. Mark, thanks. That’s a good point. I say Brendan has to go in a second, but at one point and then one question. The point is we’ve also started a dialogue with the NIH, the National Institute of Health in America Land, and it has given me a whole new sense of energy for what this is all about visual matter. By the way, this is not VR, even though I’m passionate that these two things should support each other. And the reason I’m so excited by it, imagine every single medical or scientific document knowing what it is.

Fabien Benetou: I’m going to call it right now. I think I found the single biggest productivity hack of 2021. It’s the same

Frode Hegland: Somebody might need to mute.

Brandel Zachernuk : Yeah. Thanks, Peter.

Frode Hegland: Anyway, so because one of the things that visual matter can support is if you have a graph, it can. It can know what the x y axis is, so you can do a query like show me everything that happened in September or October 1994 and this one million documents and then an instant later, you have it because the structured data we all agree is good. This is just a super cheap and easy way to do it. So that is a hugely important and valuable thing. And I really think and this is all Brandel fault that we are now living in a very real world. We’re living in an augmented world. Sometimes we’re going to need flat land screens, right? It is backwards. It is. We’ve stepped in and we have to do it. So finally, from me right now is asking you, Adam, as a case study. How do you want to write? Do you want to do it like Twitter style, where I take what you’ve written and put it into the journal because I completely understand your reticence of not knowing if the platform is going to be there. That’s why I’m glad we have the journal, which is a PDF we all keep at the end of the month. Please think about that, and then I have a question for you and everyone. Can everyone here who wants to ask Fabian, you’re very welcome to be a part of this, of course, right? By Monday. What you actually want to do to help address Bob’s points is that is that a fair thing, whether it’s an email, blog or whatever? Can we all? Do that by Monday. Something, even if it is just talk, that’s absolutely fine if someone says, if you know, whatever it might be, can we start doing it and writing? Ok. There’s enough nodding, and I see you have your hand, Peter, but if Adam could just if he’s ready to reply, if he could go now, that’ll be fantastic.

Adam Wern: But the thing is that I get this question every second meeting or so by you further. What do you want to build? What do you actually want to do? And it seems like my answers are not enough for you because I don’t say visual tech. Maybe in my answer. Well, I want to build text prototypes of different sort to understand the medium better. I want to show them. I I think the videos are better documentations for interactive things where you talk through, show a demo, talk about it, maybe discuss it in the group and ask questions. I think that is a better documentation for understanding interactive, interactive things that it can’t really. And that is why we have so few interaction, good interaction design books because it’s so hard to capture it and and only look dated on printed on a page. So it really needs to be in video if the software can’t be around anymore. So that is, but you ask me the same question every second meeting, what do I want to? What do I want to build? And I’ve said it before. I prefer to do things where you do high quality stuff, high quality thinking, active reading, where you’re very engaged with the text. I don’t want to throw a million things. Now I throw half a million characters on screen, and I was a bit reluctant to do that because it’s too visual. I want to do high quality thinking support.

Frode Hegland: Ok, but in that case, Adam, do you want to start a YouTube video series?

Adam Wern: No, it takes too much time to do that, but I can have discussions with you and I can show my prototypes now and then. When you feel that it’s appropriate, you take a few minutes.

Frode Hegland: The reason I ask you every other meeting is because you’ve been busy lately, so you haven’t been able to be at every meeting, which is absolutely fair enough. But you know, you did say it’s time to get serious, and I think that prototype explorations are unquestionably important. But I also feel that the data that goes in and out is important, too, and I have offered to completely change everything that I do to support you. You know, from author, I don’t with you and Brandel, I wouldn’t mind doing a whole new way of posting into VR, but I think that I mean, both Doug and TED close friends. Doug changed the world, of course, but then the world changed and he didn’t change with it. Ted and I will say this on the record. He was never interested in import export. So all he ever had was demos. So I don’t think we’re being serious if all we do is what you do, really cool shit. And I say that as a compliment. Sorry for using the word shit. I didn’t mean it in any way to be derogatory. I can do really cool things on Photoshop. But so what’s

Adam Wern: Right? And I think you should do that if you want to help us build things, you should do more in Photoshop instead of writing the manifesto style articles. I don’t think it will change the world a bit to do doing that, even though it feels good for you. Maybe.

Frode Hegland: Ok, Adam. I know you’re Scandinavian and you have a specific way of working, but it can be quite hurtful when you say things like that and things that you’ve said about what I’ve written before, because yes, I can do things in Photoshop, but it will be flat and it won’t be interactive, right? And it isn’t my function or marks function. You know, both of us being non coders to support you, we’re supposed to support each other, right? And today I honestly had to nudge you to show. But, you know, I don’t exactly know where I can give feedback. I said, I want to see this in VR. You want to do it in a browser. It’s a very difficult conversation because we don’t have a working relationship and as a community, this is why I think everybody should clarify what they need so we can have a really good workflow and dialogue with each other. But because when you’re talking about me not changing the world because I don’t do things in a prototype, you know, I am the one presenting to The Tonight Show next week. I am the one who is present to this to the BBC. I am the one who has had this academically published. I am the one working with the inventor of the internet on this. So please don’t demean writing manifestos because manifestos are a way of looking at the infrastructure as well. I don’t think it’s more or less so if we can find ways to really support each other, I think that would be wonderful.

Bob Horn: And they are a step toward writing proposals I might say. I agree. Absolutely.

Frode Hegland: Anyway. Oh, Peter, you have your hand up from earlier or is it new? Sorry, from

Peter Wasilko : Earlier, I posted a link to an upcoming webinar from Library of Trinity College talking about some of their digital collaboration initiatives around some format that I never heard of before called International Image Interoperability Framework, and that sounds like something that should have visual media attached to it, so we might want to have a look at that. It looks like it overlaps our time, though, depending on what the time frame wants. That is for Dublin relative to you. So we might want to. Have some of us go to that webinar and report back to the group, or have all of us registered for the webinar to see it and then meet as a group after it ends? Just possibly, but

Frode Hegland: Have a look at the little write up they have on the

Peter Wasilko : Registration page. See if it’s something that you think will be valuable to us. Also, I think both your approach and Adam’s approach dovetail together nicely, and it’s just a matter of figuring out how to combine the two world views to get something that’s an even stronger gestalt than we would have either way.

Frode Hegland: Yeah. I don’t think Adam and I disagree on that, right? I mean, we’re not here because we hate each other. You know, we are here as a very disparate group because we’re trying to do something along the same lines.

Fabien Benetou: And yeah,

Frode Hegland: So that’s so. So that’s all good. But OK, who will help me make visual metaphor? Html. Hands up, please. Somebody has to. Oh God. Mark, you’re the you’re the easy catch here. You live right down the road. Brandel Will you help me, Adam? Will you help me? Fabian, will you help me?

Brandel Zachernuk : I still don’t really understand the function of visual media in this context. And so like I said, and you know, I’ve said this a number of times, and unfortunately, maybe it hasn’t come across, right? I say that I am not academic, not to be self denigrating. I obviously have a pretty high view of myself. It’s that I’m not familiar with a literary academic tradition of writing things for the communication of people to to people. What I do online is I post short clips on Twitter and YouTube of my explainers. And so it’s actually pretty foreign to me. So my my reluctance is that I it’s not something that I personally seem to get a lot of value out of when I see it happen to have long form articles and things. And so it’s not something that I that that I lean into in order to do those things. I I work in a in a in a commercial world where demos are the main sort of form currency of exchange. I work at. So this has nothing to do with what I do in my day job, but I work at Apple and that the way that we do stuff there is demos and we, we we demonstrate, we demonstrate these these little proofs of concept and we and then we those spark conversations and then we talk about appropriateness of it in the same way that Bob was sort of describing proposals. But that is the sort of the currency of exchange is those those demo snippets that show those opportunities, such that you can then have conversations that facilitate bigger demos and write writing very rarely comes into it.

Frode Hegland: I’m not asking anybody to write long pieces, but you know, it is some kind of thing to hook it onto, like the article that I put up that started with Adam screenshots and all your comments are in there. They’re now in our journal. But but on the specific question of visual matter and HTML, what I mean is when you have an HTML page generally, yes, you can have the author. It’s generally not there. The publication date is generally not there. I know those tags can be in the meta a list of citations so that you can actually analyze them or not. Their definitions are not there. The level of headings, yes, there can be there, but it doesn’t actually say if someone else wrote it. There is so much meta that can go in there. What I passionately want, really, really passionately want is for Bjrn, Brandel and Adam, who code in this space. I want the rest of us to give you a document that when you go into your world, has rich stuff that you can play with and we can say, No, no, no, I don’t want to see it like this. I want to be able to do it like that because right now you guys are doing magic and we’re basically just setting cheering without being able to give you proper feedback. The information is really important. Yes, Mark.

Mark Anderson: I just think, I think the alphabet soup of this co-invest, that code, actually, we’re we’re getting ahead of ourselves. So I think I mean, in a sense, that’s self-evident when the point you get to it. What I’m going back to sort of Brandel point about academic communication. Part of it is dealing, frankly, with all the Typekit kids who just come around said, Oh, it’s not very interesting because they don’t. They lack the intellectual capacity to look ahead. So part of it actually is giving them just the steps to move ahead. So, I mean, that’s kind of pejorative and I really mean it like that. But the point is, not everyone, you know, there are so many things I don’t know and will never understand that I’m quite happy to try and put my shoulder to the wheel of things where I think I do have an understanding. And one of the things you know again, one of the things I’ve really learned from just, you know, experience with jolts along the way is that, you know, thankfully, not many people really need to look at what all the data means. But somebody does are not just engineers, but people who actually understand the purpose of it because most people just want to consume it or make money off it. And that’s fine. But so part of what what visual matter has been doing has been just nudging this along, and eventually everyone will catch up.

Mark Anderson: And the whole point of visual matter is it doesn’t have to be written in anything in a sense, if you want, if it makes one more comfortable, it’s a concept which is sort of how I think of it. But it sounds it sounds more concrete if you say it’s a code, because the people who want it to be a code can be happy, it’s a code. That’s really it’s a it’s a conduit, it’s a conduit and it’s part of making meaning sustainable over time and things being self explaining. And therefore it can. It can. It can link and it can build on all sorts of things. And I think that’s that’s the way I see it. And so, you know, my my mind, apart from this really is to try and sort of chip away at that. And it’s really thinking through what’s the next, what’s the next fatuous objection that’s going to come up? Okay, what do we need to put in place to just keep the keep the debate on track so that the this little seedling, this germ we have that I think will grow into something quite powerful regardless of what it looks like in code terms, in due course has a chance. Has the chance to live and thrive.

Frode Hegland: Thank you, Mark Fabian.

Fabien Benetou: Yeah, I’m a bit of a naive idiot and I learn by doing so. For the next meeting, I’ll make a dive on the visual matter, even though I don’t know now what it is.

Frode Hegland: Strong words, strong words. Adam, you had your hand up and then it came down. I’m sure you have a comment on all of this.

Adam Wern: You asked directly if I can help you with a. I I think I prefer working from the other way around, finding out what we want to do first and see what data we need. We need to do that instead of instead. I still agree with you that it would be nice if EdgeHTML had or something else. All documents had rich data of all forms about the waters and every day, everything in order. But for the purpose of what we’re doing, I prefer it working the other way around. What do I need to to do to chop up a document that have it useful in three dimensions? What data do I want to show here? It may be its author name. I want here beside the text, then then I need water in the metadata and work backwards a bit more. So start with the job. And then find the way of getting the metadata there all the way down to the technical formats that we saved things or databases or whatever we store things in, so. So I don’t want to start in the other way to throw out lots of. That were complete because it has been done so many times before, there are so many metadata formats XRP, Dublin, Dublin Core or the RDF format scammer dot org with hundreds of different formats. The problems have never been that they’re OK formats. It’s that people don’t feel forms.

Frode Hegland: This is really important. Adam, thank you for that. Two things. First of all, I think we can consider the NIH people as an audience, basically people who are trying to do important health stuff and they’re reading documents that exist. How can we augment their reading? That could be one of our cases. But also, I have to tell you a difference between what I mean by visual matter and what I think you said there. Because yes, typing in all kinds of stuff about a document that is absolutely not what visual matter is. The only document that an author has to write is their own name and the title of the document. Everything else is structural. It’s there for free without any work whatsoever. And the manuscript, that’s what’s so important. So when you make a citation, everything is done. When you make a heading, everything is done. But when you go to a horrible PDF, it’s lost or when you do it to WordPress or whatever, it’s lost. So yes, all the Dublin core, all that stuff. As Mark was pointing out, I don’t give a flying monkey what visual meta is encoded, and if someone has Dublin Core and they could be making an effort to do that because yes, that is work. But it in their. Right. If you have some metadata you want to say about the document, you stick it in there and say what it is, we don’t care. So thank you for highlighting that, because adding metadata is expensive when it is actually added rather than an inherent part of the structure of the document. But before I give the mike over to Peter thinking just generally about NIH documents, as in semi academic scientific documents that people need to read? Is that something that can give us a bit of a focus for that kind of reading for now imagining taking that into a virtual environment? Goodness gracious me.

Bob Horn: Or can I? That could be very interesting, given given given that it, I’m now thinking about the what the proposal that I put in volume two of the future of text, which has to do with the million dollar project, which which is not just any old diagram, although I didn’t say it in the article. I mean, really good diagrams. Million, really good diagrams, which would each have visual metadata attached to them, would be a whole different way of representing a subject matter. And we’re, you know, and I give an illustration of it. So yes, and and the illustration is from the medical community.

Frode Hegland: Yeah, absolutely. That was a good example.

Bob Horn: And so, you know, yeah, boy, you know, NIH has is already ahead of us by using structured abstracts in their major journals, which they which they didn’t just have a discussion group about it, by the way. They ended up having a decision making group that the the major, the editors of the major journals said, we’re all going to decide to do this. And then they said, if you want to write an article for our journal, then you write a structured abstract. And everybody went along because they wanted to have their articles in the in the journals. And I’m planning, by the way, for volume three to write about structured abstracts for my next piece.

Frode Hegland: Thank you, Bob. That’s perfect. I think you’re about to say something as well. About the NHS kind of user group.

Adam Wern: Yeah, I have a. My partner is doing her PhD in the medical field and a clinical cancer research at my dinner table, and I’m it support doing fancy diagrams with the code and on as I have a certain insight to that process. And the thing is that I’m not really sure that reference management and that is under medical articles takes. They have lots of references, but it’s not really that kind of data that it’s a hard work or it’s, yeah, menial work. You have to do it. But it’s not that it’s reading articles, understanding mythology of other articles. So it’s reading support and understanding support for that is is much more important to understand other people’s research and to judge it or understand the flow of some of the benefits. So there are many other things that are maybe more important in the process than having the kind of the structured information in the back of the article. So that’s why I’m much more into active reading parts for the whole text. I think it’s more creates more value for. More people to have that kind of support, and that’s what I not.

Frode Hegland: Don’t you agree that it’s easier to have structure and to have active reading if the data is at least somehow structured? I mean, isn’t it a useful thing to know that a heading is heading?

Adam Wern: Yeah, it is, and

Frode Hegland: It isn’t the stuff I’m talking about, isn’t it, just to support the kind of visualizations you’re working on? You know, I don’t I’m not a data base guy. You know, I’m a visual guy, Chelsea School of Art, the whole package and all of that. The stuff that you’re doing is what’s exciting and important. You know, the interactions we can have no question. But there’s got to be something behind it. And I’m just saying when someone is writing stuff, they have all this amazing structure of their documents that’s thrown away. So let’s just retain it. That’s all I’m saying.

Bob Horn: Ok, I’ll let you.

Frode Hegland: Anyway, Peter, and we have to wrap up soon. We’re way over time today. Yeah.

Bob Horn: I would suggest the idea of using an epic markup to introduce visual matter and epic markups, an idea I’ve been kicking around for a while. And the idea is it would be to have something that doesn’t look at all like markup. That would just be statements in ordinary natural language that would be extremely restricted and be readily passable, from which you could generate the visual meta. For instance, the first statement would be this is a markup. So the system would know that there was epic markup in the document, but it would look just like an ordinary sentence. The next thing would be, you know, my name is and your name. And then we know that fraud. Hagelin is the person who wrote this piece of markup and to introduce headings that could be. Now we and then whatever the phrase is, it’s going to be the actual heading. So it would look and read like ordinary English, and there would only be, maybe at most say, an upper bound of eight or so canonical statements that an author could make to generate this, and it would be able to pass through email systems. It could pass through database systems. It wouldn’t be Jason. It wouldn’t be Yamal. It wouldn’t be any kind of a standard known existing format. Therefore, pastors looking for those formats wouldn’t see it. It would be invisible. It would look just like ordinary text. And then from that we would generate and that like epigenetics controlling the expression. In this case, the markup would control the expression of generating the content that would be extracted into the visual matter. And that could be a way of getting people to write it, too, because I found at least dealing with lawyers. If you show them anything that looks like a formal grammar. I mean, even if it’s just like a heading followed by a colon or the simplest markdown they take a look at and their brain just automatically classifies it is. This is code followed by I’m a lawyer, followed by I can’t deal with code followed by. I won’t even think about this no matter how. Peter, Peter,

Frode Hegland: Peter, I know you’re trying to simplify, but it’s kind of adding to the layers of kind of mark-up extraction. Most of this isn’t actually metadata in that sense. Sorry, Mark is going to say

Mark Anderson: No, because I just think I’m listening to to what I heard. Peter describing was was one way of putting visual matter into something that doesn’t basically have built in concept of visual matter. So Fred, your tools have this built into it. The author has a deliberate intent that it will. It will capture a structure. It will capture citation information, it will capture glossaries in this kind of thing. If you had another, if you had another. A context where that wasn’t there then. So in essence, the concept that Peter is putting across strikes you as perfectly communist. I mean, you know, it’s it’s just another way of getting to the same endpoint. Yeah, and I’m totally on board with his point that, you know, most people, you know, unlikable in this glaze over.

Frode Hegland: Let’s park that for a minute because Brandel has to go. Ok. But Brandel in terms of a little bit of the I don’t want to keep bugging you and Adam about, you know, what kind of things you want to do and how do you want to contribute to the conversations because we keep repeating each other. You have very much contributed in the Oh Fabian, are you running right now?

Fabien Benetou: In two minutes. Ok.

Frode Hegland: I look forward to seeing you all on Monday. But Brenda, like that the stuff you guys write in Twitter. That’s what I’m talking about. If we could have that, then the journal. Fantastic. I’m not asking anybody here to write long academic pieces. I’m really bad at that myself, and I don’t think that’s the value, but it allows us to cite each other and to build a discourse. And hopefully that data can go into VR for us to read it there soon. I love you all.

Brandel Zachernuk : Ok, so so what’s one thing I am working on? What an actual presentation of my general thesis is. The actual starting from like what Descartes did a study on through to what it means the sort of the Wikipedia of the future should look like. So. So there is there is. I have the state of a of a grand communication to that and that should be able to do a lot of those jobs. So, so rest assured, I’m thinking about that and I’ve made some progress. But yeah, no, I I I can put the stuff that I put in Twitter in the journal. I don’t object to that. I just mostly work with with things that are primarily demos. So yeah, I will be happy to furnish those things. But in terms of doing doing more, I’m not sure where I would. And in terms of where am I, the locus of interest sort of lies. It’s it’s in the mechanics it’s in. It’s understanding what it is that we do. And so that’s that’s what I’m doing. I got recording voices and putting them in spaces, and we are recently got a web socket thing. So I should be able to get the speech to text so that I can actually put this text in the space at those places within Quest VI web socket connection. So I can do that, visualize the words appearing as well as the waveform so that we can kind of scrub through them. So those those are the kinds of things that I’m drawn to sort of reflexively and and the place where, you know, I can be persuaded, but comparatively gently away from those things, because that’s what I’m sort of fastest and consequently best at it’s being

Frode Hegland: You and Brandel don’t understand is that we are at the beginning of this and what you say matters. The demos are crucial, otherwise it’s just chitchat. No question. But this is the beginning of this revolution. So to have down these arguments, these thoughts, these different perspectives is so important, and it’s also so important that it’s real data that goes in and out of stuff. Look forward to Monday. It was a good chat. It’s tough, but got to do it right.

Fabien Benetou: I’m sorry, I need to run, but I want to say one thing it is that capturing is indeed super important that there is nothing more frustrating than an amazing experience or something in a new medium that somehow you can’t read transcripts and can’t build on so. And thanks a lot for having me.

Brandel Zachernuk : Thanks for coming, Fabian. It was really awesome to have you part of it.

Frode Hegland: Have a great week, guys, and thanks for Brandel. Thanks for inviting Fabian. You mentioned that. So Adam, let’s keep fighting, but let’s keep building as well. I’m going to have to keep writing. Oh yeah, Brandel. It’s a to go through the weekend. Yeah, that’s it. But you have another minute. What I’m asking you about the visual matter, HTML stuff. I want to serve you in this. I want to be able to give you stuff for your demos and it can be just our basic journal chat. It can be whatever. But for you to have to scrape an old book that we have made is a bit pathetic, right?

Adam Wern: If not, it’s not so much scraping because you exported it as an EPUB and I use the EPUB book because the EPUB have paragraphs and it has scraped it. The conversion software has scraped a PDF, so it’s a bit circular. It’s a very circular way of doing it. So it would be fantastic to have that fine information directly from author to clean HTML with with the kind of metadata that we’re talking about, that would be very helpful. But at the moment, I don’t know exactly what’s missing and what we really need, so I can’t really answer that question as precisely as you may want at this moment. So you can

Frode Hegland: Look, you don’t have it, you don’t have to do the whole thing. But if you can tell me that the one the way that you want to see whatever is missing, let’s say even author name in an organized way. Just tell me, and I’ll put it in the gosh darn academic article that I’m writing. If it turns out that you were wrong and you want something else later, there’s absolutely nothing wrong with that, right? But just like for you, implementation is making it real for me, writing it is making it real. So we got to do that together, and then we’ve got to disagree. And then we can cite the fact that we disagreed so that we’re building an intellectual shape of what what we’re doing here. So I think as long as stuff comes out, it doesn’t matter if we agree.

Mark Anderson: You know what I see, what I see is just is basically just just just gently bubbling to the surface things that are difficult to find. So they may they’re either in, they’re not accessible or, you know, or they’re not. They are not described at point of receipt in a manner that it is conducive to the next step. So it’s and it’s difficult. I can see at the far end of that, you don’t know, you don’t know what you don’t know until you find that you don’t know it. So it’s difficult to write down and said, this is what I need, I think is an emergent process. And this is also one of the reasons I’m not saying, well, I’m happy to try and build up stuff because, you know, it’s it’s it’s a sort of iterative thing. You get someone say, Oh, right, and you fix one thing, and that just reveals the next layer of the problem. I think I think at this stage is way too difficult to guess. You know, soup to nuts and have the whole thing there. But what we can do is we can certainly nibble away at at things. So in other words, you know, because PDFs are never going to get fixed. But if if documents that want to produce data for the long term, which may have to produce PDFs may also in a different way in a different sort of side, channel may produce something to just right around that.

Frode Hegland: We have guest. I got to go turn the oven on. It’s my job. Ok? Take a look. Have a good weekend. Look forward to Monday. I hope you’re all there. Bye.

1 comment

Leave a comment

Your email address will not be published. Required fields are marked *