3 April 2024
Frode Hegland: Hello again.
Dene Grigar: I dropped it into your slack channel, so you should see it.
Frode Hegland: So let me say Nope. Oh, the main slack channel.
Dene Grigar: Future text.
Frode Hegland: Oh, here it is. Okay. Open.
Dene Grigar: I show this back in March. We talked about it in the meeting. Hi, Andrew. Or he was here. Come back. Morning, Andrew.
Frode Hegland: I remember the document very well. I just couldn’t find it. So. Was going to open it now.
Dene Grigar: I found it also in slack. I had put it in slack too, so right. We have a lot of stuff in slack.
Frode Hegland: Hello? Under.
Dene Grigar: More. Good to see you, darling. Happy Easter.
Mark Anderson: Hello, Malaysia. Sorry. Yes. I managed to have several meetings parked on me at short notice on a Wednesday, which I couldn’t escape. So I’m sorry I missed the last two.
Dene Grigar: That’s all right. Good. You’re here. Good morning. Andrew. You’re muted or something. Yes. Andrew, did you see my link about signing up for the DSI? I’m guessing that might be. Yes. Can’t hear you.
Frode Hegland: Mark, you said something. I saw there was a slack message from you saying, is there a non paywall? But for the life of me, I can’t figure out what you responded to. Could you tell me that was sorry?
Mark Anderson: That was to the New York Times thing. I was just wondering if there was a sort of, you know, an an overview.
Frode Hegland: Oh, was that by Peter or who posted that?
Mark Anderson: Sorry. I just thought I’d post it. Yeah. I just read the long Cory Doctorow thing about I have issues with social networks.
Dene Grigar: Cory Doctorow.
Mark Anderson: It’s the thing I took away from the thesis is there are trust issues.
Frode Hegland: Yeah, that’s a good a naturally intelligent summary you have there, Mark. That’s cool. Oh, Rob is here as well. Oh, fantastic. It’s always nice to see you, Rob, but particularly today. If if you have updated your headset, that is. Let me just get a link for us for our Agenda. We do have an agenda. People.
Dene Grigar: And we love.
Frode Hegland: Having an agenda.
Dene Grigar: Make an agenda we don’t like to follow. Good morning. Rob, how are you?
Frode Hegland: I don’t think he’s there yet. I think he’s half there yet. This. Yeah. Andrew, I saw your demo, obviously. And. Very nice. You are a consummate manager, managing the desires of many and the desires of me. We have colors and Jason all in one beautiful package. Just as it should be. Thought that. Okay.
Speaker4: Hi, everyone.
Dene Grigar: Hi, Rob.
Speaker4: Sorry I missed. Sorry I missed Monday. I’ve had computer issues.
Dene Grigar: What happened?
Speaker4: Oh, she don’t want to know. It was a nightmare. Okay, I changed my Google password, and it just wrecked everything.
Frode Hegland: Oh, yeah. That can be a bit much. Okay.
Speaker4: What I did update the headset to version 1.1, I guess 1.2. Maybe there’s another one.
Frode Hegland: But was it last night you did it or earlier.
Speaker4: Earlier.
Frode Hegland: Okay. Because the whole personas thing, if we have time today while everyone’s chatting, you and I could try it if we weren’t supposed to be radically changed. Right? Okay. I fixed.
Speaker4: The persona, I did that, yeah.
Frode Hegland: But these are its new personas where when we talk, we’re actually sharing space rather than it’s okay.
Dene Grigar: So glad that’s that’s going to be.
Speaker4: That’s I’m not home. So
Frode Hegland: You have it with you.
Speaker4: I don’t have it with me.
Frode Hegland: Oh my God, how can you live without it?
Speaker4: Well was taking my constitutional walk. On the other side of the campus.
Frode Hegland: Well, you’ve ended up somewhere nice. Yeah.
Speaker4: But it’s not my house.
Frode Hegland: Right. So I don’t know who’s coming today. So I think we are a quorum. Something I still don’t know what means, but they say it. Please click on the link in the chat for our agenda. And the first thing is what we just did. Just mention the whole vision update thing. It’s going to be really interesting to test that out. If you have chance whenever you’re home just just ping me. It would be lovely to test it with you as well.
Dene Grigar: I want to get out of class. I did not bring it today because I have so many meetings.
Frode Hegland: Yeah, I bring mine. Want to have coffee? You know. I showed it to one of my former students today, and she did the thing we’ve seen every time. Well, this is really cool. This is neat. The 3D or stereo video, everybody loves it. The environments, they go, wow. And there’s never enough time to go into the real stuff. Also, since we’re on vision Andrew I have actually got export. Now click the wrong thing. Export in reader for a JSON. So I’m going to try to share that right here. If you can have a look at it and see if it even remotely looks like what we’re talking about.
Andrew Thompson: I mean, it can kind of look whatever it wants because I’m gonna have to build a structure around it. Almost certainly it won’t look like the current export that the system makes. Just because it’s based on three.js and it has its own export. Can you guys hear me now? I don’t know, did my mic get fixed?
Frode Hegland: That’s fine.
Andrew Thompson: Okay. I didn’t change anything. I don’t know why it wasn’t working earlier. Oh.
Frode Hegland: It’s the internet. The world. Yeah. No. That’s fine. I just wanted you to have a look at it so you could tell me to tell them if there are any obvious changes to make cheap changes early on in this thing. It’s not a big thing.
Andrew Thompson: I do see some like weird non symbols, like, it’s it’s like the paragraph break symbol is part of. The text in here. Is that intentional? Are you using that for some reason?
Frode Hegland: We use that for breaking in. When we have a defined concept that comes up in a dialog, and we use that to do line breaks there, and we also use it in the visual meta to control the line breaks better there where they are not rendered.
Andrew Thompson: To like title has them on both sides.
Frode Hegland: Okay.
Andrew Thompson: Which is fine. I’m just curious what they were for, but.
Frode Hegland: Yeah. No, I saw a report to that too. To me. So these are the things we just need to report back to them. But there’s no urgency. I just wanted to have a thing. Then you work for your thing, and then we go back to that.
Andrew Thompson: I think I can just ignore those. I’m not sure it may register title differently, but as long as they’re always there, I can work around it.
Frode Hegland: It’s more important that you are happy. So if they need to change something, they should. So that’s that’s easy. So thank you for having a look. Dini, did you want to mention briefly our Sunday presentation?
Dene Grigar: Yeah, we had we went to this wonderful group of people. They’re not academics. It’s a straight, just webXR, you know, visionOS visionOS. And it was at noon on our time took place in this virtual environment. We’re all like little bubble heads bopping around this virtual space. That part was kind of goofy. But it was good. I mean, we get it was our first. Presentation, so I think it was a great way to start it. But I made some slides that were shown on this kind of virtual projecting system was well received. In fact, one of the people that were there came on Monday, which I thought was great. So I thought it was a good way to start our working together to do presentations. Informal group, very receptive. They had not had many academic kind of people in their group before. So it was a meeting of two different environments, two different kinds of people. Users. Good.
Speaker4: What were they mostly interested in?
Dene Grigar: Webxr is webXR group.
Speaker4: Creating environments.
Frode Hegland: Yeah.
Dene Grigar: Developing webXR systems, environments, experiences. So it was just basically production people. Coders, programmers.
Frode Hegland: Yeah. That was good. Was it was a good. So that was the Mozilla hubs, which used to be quite a cool place to make your own environments. But the tools just my brain just didn’t work for it. So that’s a real shame. It was cool.
Dene Grigar: At the time, right? I mean, that’s the thing when when we first were playing in those kind of spaces, well, actually, I shouldn’t say that it wasn’t cool. Second Life was cool, but when we came back to, I don’t know, I mean, this is something we can talk about later, but the history of VR and virtual spaces, I mean, we had some cool spaces back when we didn’t have the tech to do them right, but we still did them right. And then we started making these kind of, these kind of spaces that were like, it’s like a step backwards. So I need to take a little bio break and I’ll be back and talk about the Monday Book club. Be right back.
Speaker4: The Monday Book Club.
Frode Hegland: Hi, Brandel. I just put the agenda in there. Yeah. Denny has a nice idea for But what to do over the next few weeks? Yeah. So? Yeah, we have the agenda that we just we’re going through. Brandel. I have 1.2 beta visionOS. It means two things I’ve testflight again. I can test my own software now. Oh, that’s very exciting. And I got an email from Apple saying they want to know rights and things because they want to promote author, which is lovely. But at some point it’d be nice to do a a call with you and FaceTime because now the spatial FaceTime is on here. So that’s really cool. You have to tell us about it now that you can. It’s been like secret stuff.
Brandel Zachernuk: It’s not a thing I have much awareness of. Honestly, I work on the web and so I know that you can actually get the persona a picture as a texture within the web, which is pretty cool. Really?
Frode Hegland: Yeah.
Brandel Zachernuk: I don’t know that anybody’s noticed. And that’s that’s. And I have used it. But one of the things about being an internal developer and even actually being an authorized external spokesperson is that I’m authorized to talk about certain things and and other things. I just I don’t know what. Is known because as a developer, as an engineer, I have. Necessarily. Things that I can look at, ways that it works, that I’m that aren’t necessarily what the public has. So I don’t know. One, I’m not actually able to do any face terms.
Frode Hegland: No, no that’s fine. It’s just and.
Brandel Zachernuk: To I don’t know I don’t know what it actually is or isn’t in contrast to what, what people on the inside have. Sure. So I’m afraid there’s not much I can say about it, but it is really exciting. I was honestly confused when it wasn’t spatial. So for, for people in the public. So it’s a, it’s a relief to get it out there and.
Frode Hegland: Yeah, no. That’s cool. And I did not know that the textures were available on webXR, so that probably obvious to others, but not to me. That’s really cool.
Brandel Zachernuk: No, not not in WebEx. Sorry. They’re available in the web. So you can you can there’s no there’s no current agreement or standard for having a forward facing camera available within a WebEx session. Because there’s not been a device where that would even make sense to have a front facing camera on, on your face until and unless it’s a synthetic one. So, you know, it’s a very interesting thing to kind of pursue and look at, but it’s not something where the the details have actually been hammered out on what, what that means and where it ought to be supported and things like that. So in the same way that iPad removes the visibility of that stream when a window is not open, the same thing may happen in WebEx or today. That’s not necessarily long term, you know, the position that it’s going to land, but that’s that’s what it is, right?
Frode Hegland: Yeah. Well, that’s really good to know. That’s good to know both sides of that. Thank you. So yeah, the agenda Sunday book club. No. Monday book club. What is that, Danny?
Dene Grigar: Well, Frode is going to be kind of out for two weeks in Japan visiting family, which is great opportunity for him. But I was thinking that we’ve not really had time to do much of the more esoteric, philosophical, theoretical discussions about virtual reality. And, you know, we’re dipping our toe deeply into this world. You’re nice to get some of this under our belts. It’s been a while since I’ve read Benedict’s book, for example. There’s a lot of really good literature out there that starts from even the, you know, the 80s and 90s. And I thought it’d be fun to pull a couple of articles out. Scan them and drop them into our slack channel, and then we could talk about them for an hour for two Mondays in a row, from 9 to 10:00. So not eight in the morning, just the second hour. Spend an hour, you know, talking about these articles if you’re interested. If you’re not, fine, I get it. But it’d be I think it’d be kind of fun to to read these with some folks. It’s been a while since I’ve read these articles myself. Anybody interested? Any takers?
Frode Hegland: Yeah, I.
Brandel Zachernuk: I I’m not familiar with the literature, so that would be, that would be neat to be able to spend some time with it and do some discussion.
Dene Grigar: Great. How about you, Rob? Yeah.
Frode Hegland: You’re my guinea. If I have.
Mark Anderson: Time.
Frode Hegland: Mark.
Dene Grigar: Great. Okay, then I’ll pull something up tonight and we’ll start with just maybe something from the Benedict book came out, you know, very wonderful volume of collection of articles by some good thinkers. And it was the kind of a seminal text when it, when it came out. So I’ll pull that up and also look at a few other articles. So and we can I’ll drop them into the slack channel. We can vote to see which one we want to look at and go from there. Okay. Thanks. Be fun to read with academic and people that are smart. I read so much myself and John. My husband is not interested in this stuff at all. So it’s like I talked to him about this and he’s like, oh, okay. All right. So case studies, I.
Frode Hegland: Just added an announcement. If you could please reload, I apologize. Just two things in terms of timing. One, as I put in the chat here, I will be available just at a different time as you’ve gotten emails, because 5 a.m. in Japan is the earliest I can really do, which is 1 p.m. Pacific. But also there is the thought of having a future of text social this summer, most likely in June, where there will be no public book, no symposium, no talking. It’ll just be from Friday til Monday, or whatever time within that you might have in London. For those of you who would like to do that, if it’s convenient, and I just expect those of us who are around will talk about this stuff to some degree and just have a good hangout time. I know that some of the Europeans are up for this. I’m hoping Leon at least and Mark at least. And if the rest of you can, that would just be golden. So just something to consider. I don’t even know the dates yet. Once there’s a little bit of if you have some dates, you might be able to do it. Just please email me. And then, based on what few people say, I’ll try to make a jiggle. Jiggle and see what happens. Very, very loose at this point.
Dene Grigar: And further, can I mention that there’s been some discussion about GitHub and not getting access? So Greg’s here today and I’d like to get this taken care of. So if anybody wants to get into the GitHub repository and you haven’t been able to but you we have gotten your email address, please give us the email address that you are using for the GitHub repository. And then if you want me to, I can have Greg just open up the repository. Period.
Frode Hegland: The thing is the link that Peter was referring to on slack, I clicked on it too, and it also was not found. So I think it’s maybe just the link itself that’s incorrect. The one.
Andrew Thompson: No, the link is fine because I can click on it and get access as long as I’m logged in. If I’m not logged in to that account, it’s a 404.
Frode Hegland: Okay. Well. Okay, fine. Well, then. Yeah, that’s exactly what you said. All right. Good, good. Good to know.
Dene Grigar: All right. So does everybody have access to GitHub so they can log in.
Frode Hegland: And not Peter, but he’s not here today. So please, please add Peter again.
Dene Grigar: What is Peter’s address? The one that I used. He’s obviously not using. Which one does he want to use?
Frode Hegland: I don’t know, I’ll email him and copy you. That’d be great. Thanks.
Dene Grigar: Okay. And how about wasn’t it also Not Adam. Who else is trying to get into it? Bobby and.
Frode Hegland: Yeah, I’m not sure. Hang on. He’s busy, but he’s available to. I also added my email.
Leon van Kammen: To the chat. So yeah, then hopefully I will be able to access it. I’m taking.
Dene Grigar: Perfect. Thank you. I’ll get that added. Greg gets in at 9:00, so I’ll get him to start that here.
Frode Hegland: I think Danny, maybe just simply just to make it public. If Andre is okay with that, considering that’s where we’re headed anyway.
Dene Grigar: Andrew.
Andrew Thompson: Yeah, I’m I don’t mind. It’s not like this is something that’s going to be out there getting advertised. So it’s it’s not like anyone’s going to find it.
Frode Hegland: Just open up or something like that. Maybe not Sesame. Okay, so. Right. Okay. So we’ll open it up then. I mean, you will ask for it to be opened up, right? Any.
Dene Grigar: Yes, I will do that.
Frode Hegland: Okay. And then indeed we go to the use cases and case studies.
Dene Grigar: I had already posted about this before, but I’ll reiterate what I’m working on right now. So this document up. Most of you, I think, have seen it. I dropped it into slack, so you might have seen it there.
Frode Hegland: Yeah, yeah, we have seen it. But today we decide on exactly what we’re doing. Right. So that’s exciting. I’m just going to give them a fresh link so they can see what, because I put it on the web page. Okay, Danny, please go ahead. Now. The link is up. I can be quiet.
Dene Grigar: I’m going to do a screen share. As I mentioned, I usually start with the research question, and the question that I’m starting with is what is the effect of XR upon three common academic tasks? Now, this is not the only things that academics do, but there. In fact, I can come up with a list of maybe 20. But these are the three that I think are the most prominent. First, preparing to write an academic article. And that could include an article for a book. Right. So it’s writing. Secondly, editing review articles for an academic journal, which I do all the time. And thirdly, assessing a graduate seminar paper. Right now I’m doing a I’m an external reviewer on a dissertation on AI from Simon Fraser University. I’m having the I have a PDF that was sent to me and I can’t freaking do anything with it. I mean, I can’t I mean, I can’t mark it, I can’t do anything with it. I printed it out. Now I’m writing on it. Right? So it’s a real problem for academics to work with PDFs on these kind of things. But anyway, so specifically, does XR enhance or improve the quality, effectiveness, and efficiency of these three tasks and in what ways? So part. This is going to be divided into three parts. Part one is the ways in which these activities are undertaken, given the constraints and affordances of computer tools I’m currently using, as I mentioned, that PDF that I received 150 pages of PDFs for dissertation on AI that just it’s impossible to work with. So it’s important that I lay out the difficulty with dealing with that environment in a 2D space. The second part is the ways in which these activities can be undertaken. An excerpt currently and then and then you know what? What can I do with it now in a Vision Pro or three, you know, and not a lot right now, but.
Dene Grigar: Third. What is gained by expanding the tools? What can we do? What can I imagine it looking like? And already we’re heading those directions. This photo was showing on Monday. Having a place to be able to write in marginalia, being able to link to things. That’s what I’m imagining will be much more useful. I want to I actually want to manipulate that PDF. I want to work with those words. I want to I want to look back. So there’s a, you know, something like seven pages of references. You know that they’re separate from the text itself. You know, how can I check that reference in the midst of reading that text? So that I don’t have to go through the pages and pull up the references. There’s there’s got to be a better way of doing it. So, Mark, I know you’re working with Broder right now on his dissertation. It’s just impossible to work in, in this environment effectively. And it’s so much different with print. I mean, if I print it out, I can do stuff. I can lay things out on the table, I can mark things up. I can, you know, put little sticky tabs places. So when I go into that exam and I have to examine that candidate in April, you know, I have my sticky notes all over this thing and I can just pull. They’re color coded. Can we do something better than that in zer? So those are the kinds of things I’m thinking about for this case study. And I’ll stop there. A marquee can be very helpful with this too. Well.
Mark Anderson: I have a thought. Actually, one of the things just doing fun enough, some sort of rework at the moment, is that one things I realized is that that I think we miss when we talk about annotations is they have different purposes. So it’s the way you see it normally is just really low hanging. Let’s do corporate north, south, east and west and all have different. That’s I don’t know that’s fine in office but it’s not useful. For instance you may want different streams of annotation. You, you might be annotating something that’s already annotated. And you need to be absolutely certain that the, the annotations that you’re making are not admixed with the others. So they’re separately addressable. They can be displayed and hidden separately because you again, sometimes what you’re putting in there is obscuring what’s there. And silly things like not being able to use a spell checker inside. Inside inside annotation notes. Why? It’s just a text space. So what that tells me is the people who implemented these sort of things way back didn’t really think this was just this was a toy they gave you to make me go away. And why not use a sticky printed out and put a sticky on it? And that’s about where the sophistication is. Whereas if they were treated more like first class text objects within the document.
Mark Anderson: And I very much call what Denise said, you sort of in a sense, you want you want what you might think of as the paper document, almost as like the the sort of performative substrate in the middle. But it’s only one of the players in the whole process. All the real action is happening. You’re basically consuming on that. But you’re probably not writing back to it, because if you were, you’d be editing the live piece. And there are all sorts of reasons not to do that, not least which it might not be your own to edit. So I think that’s really interesting. And what’s more, I think all of this actually has use in a wider context. But I think the sort of the academic reading and writing thing is a place where it’s going to get it can get beaten up quickly, much more easily into into something more usable, and then shown to people who probably at first sight will understandably say, yeah, but so what? And then find that actually, no, they can do things because in a sense we all end up in some way annotating or editing stuff for all sorts of reasons and not necessarily even for work.
Frode Hegland: Thank you.
Dene Grigar: And as I mentioned before, when we had this discussion, I there’s indexes already, right? There’s already an index in a in a book, I re-index them, I make my own index because I don’t. I had my own way of thinking, my own language. I have my own needs. I know what I’m looking for specifically. And I don’t want to look at, you know, ten pages of index that mean nothing to me. So re-index things so that I can quickly pick up the book, turn to the cover, and look for that item in my own index photo.
Frode Hegland: But I think this is very important that we’re talking about many different things. So if you please click on the link I just put in, which is where I put what you wrote. Deenie if you all can click on that on our web page. On the top. I’ve added some some questions that are kind of important. The first one is what corpus should we work on? Because when we talk about books and we talk about a very long dissertation that’s very different from ACM articles. So today is when we decide which of those things we’re going to work on for now. Because they’ll be very different affordances for very different types of material, obviously.
Mark Anderson: Will they could you explain a bit more as to how you think they differ?
Frode Hegland: Yes. Acm articles tend to be not a lot of pages. A book has a lot of pages. So that means that just in terms of dealing with the volume, it’s different. And also if, for instance, a lot of these things have nothing to do with XR in itself, it’s they would also be problems when dealing with them on a normal computer screen, such as how to view the references while you’re reading. Of course, we have our own kind of proprietary system in in reader author. But even with the normal ACM article one thing that could be done is duplicate the article. You have the reading on the left, for instance, and the references on the right. So you could do that. So if you want to build affordances for a book and different kinds of literatures, I think the use cases will be different.
Dene Grigar: I think my article, I think my case study paper already defines the links. We’re looking at an article. So, for example, let me put my hand down. It drives me crazy when this sort of thing shows up. Okay. You look at my my question, you know writing an article that’s not a book that can be a three page, could be ten page, but I’m writing an article. Editing review. Editing review articles. So those are usually 750 words for for Leonardo. So it’s usually maybe two pages. Three pages. Write 250 250 words a page, three pages max. Now the graduate seminar. Graduate seminar paper that could be anywhere from seven pages to 30, depending on the topic. So I’ve already defined the length. It’s going to be not a book, right? Yes.
Frode Hegland: So we have. So we’re not focusing only on ACM then.
Dene Grigar: No. And in fact, I was I just as we were talking, I just had a brainstorm. Right brain flash. Since we’re going to have this Monday morning book club and we’re going to start with something that’s, you know, from the past that talks about theoretical issues with VR. And we’re going to look at how we think about VR today. Then let’s make that the case study. Let’s let’s put that in the case study. And it’s going to be something that I scan. But scan it. It’s going to be a PDF. So we start with that. The next week we can look at an HTML. We can do one of each.
Frode Hegland: I mean, if you want to do that for your book club, of course you can do what you like for that. But in terms of where we should put our focus for what kind of affordances to build and the coding that Ender’s going to, I think we have to be really clear with the with the use case. We have three important use cases here that you have written down editing review articles for academic journal. What is. Okay. No.
Dene Grigar: Those are always PDFs. Those everything is always PDFs. Well, I shouldn’t say. The review articles come to me as word documents, and then I have to send them back to Leonardo as an TF. But they’re, you know, they’re still not web pages. I love the idea. So let me just step back and say I love the idea of working with web pages. I do, and I hate PDFs. I say this with every fiber of my body. I fucking hate them and who knows how much I hate them. That said, everything I do academically is PDF. So if you’re talking about people like me, you’re talking about PDFs. Now, can we strip the PDF out of there and just have the words separate from that? Is there a step we can take to strip it out of PDF and move it in HTML? Okay, but that’s not what this project’s about.
Frode Hegland: I thought it was because ACM has started doing HTML as well. So I thought that we were doing reading PDF and where HTML is available. Then we can have special views for that.
Dene Grigar: Okay, then you just answered your own question.
Frode Hegland: No I’m not. No, I’m saying what I thought.
Dene Grigar: I’m just saying that that’s exactly I mean, that makes sense. Mark has his hand up. Mark, you know something? We should
Mark Anderson: Well, on the last point the the the factoid is that acm. Acm. Html became available from 20 2020. As far as I’m 2021, sorry, I think it is as far as I’m aware, so it certainly doesn’t go back beyond that. It might even start later, and only some of the existing corpus will do it, because it depends on whether any given journal or or conference chose to use the infrastructure that allowed it to happen. So but it’s where they’re going because ACM said, well, as of this year they won’t they won’t send you anything in print. They’ll send you a PDF and you print it out for yourself. But what’s a good thing actually with the planet? But I was I put my hand up really, so that I’m not sure. I really do see a massive distinction in terms of the formats. I think it’s terribly easy to get lost in the sort of the, the performative layer of the layout, actually. They’re all just like, I think the thing is, that’s one of the things we need to sort of take into account. It’s just a layout. And to a certain extent, if we treat publisher X and publisher Y as something, some totally different thing will not be helping ourselves.
Mark Anderson: They just have different weird and random ways of doing stuff. But it’s broadly the same. They don’t, you know, it’s a bit like anyone. We don’t all use every tool in the drawer, but we probably use most of them, and we probably use the ones we do use in a slightly different way. But as long as I don’t use a hammer to drive in screws, broadly, a hammer is a hammer. So I, I think it’s the fact that we’ve started using the ACM corpus and we have access to some documents to it is useful, but I don’t think I don’t think it we should sort of constrain ourselves artificially to, to, to that being the horizon, because I think that will weaken our insight. The truth is, what we have to hand to work with is ACM. So I suspect it will figure heavily on what we do. And in that way, it’s a sort of quasi constraint. But I’m, I’m sort of eager, not eager not to make it more of one than we need it to be. Thanks.
Dene Grigar: But you had your hand up.
Frode Hegland: No, no. Go ahead.
Dene Grigar: I want to say that last year. Let me put this down. Okay. Last year, when I was working with Rob Swigart on that article for ACM hypertext on VR, I did a literary search review for any article about VR published by ACM, and I did not find anything. There’s a couple of ones that mentioned VR, but they really weren’t about VR. So that was an interesting thing. So Mark, correct me if I’m wrong. But I found nothing that was that was about specifically about VR in a hypertextual environment, except except things I wrote outside of ACM. So I’m going to do a case study, and I’d like to do it on something about the Ark that I can’t use, the ACM library. It’s going to be something more about hypertext. And your article on seven types of hypertext would be a nice guinea pig for that. But I think it’d be kind of fun to do an article to do this case study on VR. That is a PDF. And then we can then also if we wanted to to strip it out and make an HTML and make that something we do also. Let’s start with the PDFs, because we do mention in the, in the Sloan Foundation that we’re looking at text and we’re, we do mention PDFs as kind of like the major thing. Not that I’m married to it, believe me. Thank you. Brandel.
Frode Hegland: Yeah.
Dene Grigar: So comments. Mark.
Mark Anderson: I just thought it’s a it’s annoying that I didn’t use the Tap system last year because to get an HTML version of my paper, I will have to create it, and I perhaps overprovisioned it with references. But there you go, Rob from my own back. But that aside, yes we we we could do that. I thought I don’t know if people have seen it. Fabian, I saw a post email from earlier sort of commenting on this subject of of what did he call it? But basically use cases and things. And I thought he made an interesting point that a key thing really to understand is the sort of chicken egg of what we want to do and what we need to have to be able to do it rather than get to too wound up in, in boundaries and implementation and sort of basically jurisdictional stuff. Which, which doesn’t particularly help us. And I think that that pays strongly to me insofar as the thing I find myself, the thing that makes me want to use XHR, in fact, the only thing that would really make me use XHR to do work as opposed to play at the moment, would be document deconstruction. So the real question, and I think this is where it has become necessary and pertinent to look at web pages, not so much because it’s HTML as opposed to something else, but it’s, it’s text in a more fluid, more a more tractable, more recyclable fashion.
Mark Anderson: It’s, you know, it’s not PDFs fault that it is what it is, but it also is sort of, sort of problematic. But I don’t want I don’t I wouldn’t want to look at it in that frame. I don’t think it’s useful to look at it in terms of winners and losers. The most useful thing is to sort of say that, okay, well, what more would we need to do? Say, for instance, to a PDF to make it as flexible as something we can do using HTML as a proxy? For sake of argument, I think we have to be mindful too, about just how much more we stick into PDFs. I mean, if a PDF is a document with its it with about, you know, seven different representations inside, at some point someone’s going to say, well, isn’t this unnecessary duplication? I think that’s something we should also be mindful of. Because PDF actually is a as I understand it, is essentially a portmanteau. So a bit like a tiff you can put in, you can put in basically whatever you like, as long as it abides by the rules of the format.
Mark Anderson: So you, you can put pretty much everything and indeed some early PDFs are just a photo in a, in a PDF wrapper. There’s no real text there at all. And I think there’s a, there’s a sort of lesson in that. So if we want to have additional metadata in there, we need to think Actually whether we create the degree to which we’re creating unnecessary duplication. I mean, the annoying part is if I was going to get one bit, I get rid of all the damn postscript. I don’t want the typesetting in that I really don’t. But you know, it’s there, so we’ll live with that. But I think this I was thinking, I mean, I made some visual metaphor for my, my, my thesis, and it was 105 pages of four point. And I don’t think that passes the sniff test with the general, with the general public. So I, I don’t I don’t say that because as somebody who gets visual media, this is more a wake up call to self saying, okay, well we need there’s a bit more optimization to do here somewhere. And I, I don’t know how to deal with that.
Dene Grigar: Mark. That’s great. And I want to I just wrote down as you were talking PDF is not the future. That’s not the future. That could get us. I don’t want to get us in a different direction of of this grant at all. But I think one of the things that might come out of our exploration is that fact. We can say, okay, we’ve done this with PDF. It doesn’t work well. What works better is HTML, and we’re recommending to the academic community that they begin to think about the format in which they produce things. And to be honest with you, we’re not going to print anything out anymore. Then we don’t need PDFs. Pdfs were meant to ratify to To sustain formatting across systems, right? I mean, it really was a way to kind of tamp things down and hold it in place because, as you know, if you make a word document and you make it on your computer, if you send it over the net to me, the formatting could get all whacked out. It happens to my students all the time. They send me their resume project. It’s supposed to be one page. It’s one page on their computer, and when I receive it, it’s a page in maybe two lines. So the formatting is all whacked out in words. So that’s why academics love PDFs, because once we say it’s this, no matter what happens, it comes to me as that. That said, we’re we we’re not going to be using email in the future like we’re using it. We’re not going to be doing these kinds of systems. So what is it that’s going to be the future? So the future of text. Next year, Frodo might be an interesting thing to think about is, okay, so we’ve done these this work with this future tool called XR, right? And we’ve discovered that PDF is not the future. What is the future of text in this VR environment, but the actual object of text. And I would argue it’s not PDF Mark Anderson.
Mark Anderson: Oh well, I’ll go first. I’ve spoken already though I’m pleased.
Dene Grigar: We aren’t pleased.
Leon van Kammen: Yeah, I have a small question. I’m a bit puzzled. Because PDF and HTML is both used by the academic world universities. Everybody uses it for some reason. Pdf seems to be more used for serious or let’s say, papers or so what I it almost seems like PDF PDFs have more somewhat like a stone tablet quality to them, perhaps. That makes it also annoyingly hard to edit. It’s more a read only thing. Yeah. Yeah, I’m just thinking if HTML was created by academics, then isn’t there? Hasn’t there been some attempt to turn it into a stone tablet as well? I mean, it’s completely fluid, I understand that, but yeah, the only thing I can think of is, for example, IPFs, where an HTML document has a sort of like unique address and you cannot overwrite it. I could imagine that as a stone tablet. But isn’t there something already years ago when hypertext was invented, the HTML version, that there was some some kind of way to to lock it or sort of to where the author could say, this is the final version? Well, wasn’t this invented? And why is it not used then?
Frode Hegland: That’s such it’s.
Leon van Kammen: Invented I.
Dene Grigar: Love that. Thank you for that. So the reason why HTML is not the go to formatting for academics is that most academics can’t work in HTML. I mean, only a handful of us, let’s say in America. It’s computer scientist. People in the digital humanities like I am media art. It’s that small, small group of people, right? Everybody else is working in PDFs. And so that’s part of it, is just being able to know how to use and work with HTML. But the second thing is. H. Html is fluid, it’s constantly changing. And at the time this I mean, now we’re getting into the philosophy stuff that turns me on. There’s this idea that when you write something down, it is a stone tablet. This is the truth, right? Seven types of hypertext. This is the truth, right? This is. I’m writing this down. It’s published. It’s been vetted. Everybody. In fact, it’s won an award. This is the truth. But the fact remains, every time I write something down, every time I think I’ve done something, I find new data, the notion of new knowledge. And it’s coming at us so fast. That Mark might have to do a second edition of that article. We call it auditioning. Right. So the audition articles all the time, the audition books all the time.
Dene Grigar: With HTML. You can you can track that right through the code. You can you can actually track the changes through versioning. You call it versioning in HTML editions and text. My lab has been working on the kind of like the stabilizing notion of versioning. Versioning HTML or versioning digital text. And but that’s such a rare thing. Not many people are thinking about that. And what helps us to be able to even make sure that the HTML has been changed or not been changed is to go back to the Wayback Machine, and you can check the veracity of a website by looking at that same site with that same URL on the Wayback Machine, and see the changes. So there is an apparatus already in place thanks to archive.org that tracks websites over time, and you can get a snapshot over years of that and months. But putting that together into a concept that is usable across academics, it’s not happening yet. Leon. And but it should. And this is something that I think would be a useful discussion to have as we play out this project. And I’ll now turn it over to Martin and Proto.
Mark Anderson: So to pick up a couple points because that Leon raised, I mean, I’d sort of put it like this. You know, I never underestimate human laziness or greed. Agreed. In the sense of money. So so the laziness part is, it’s it’s more work to do something in HTML. Now, if you’re just used to maybe writing a word document and bear in mind, a lot of people, even today, are still working in a system where although they were working at a PC or a mac, they’re sort of still in a secretary driven world. So you do your bit and then magic stuff happens behind the curtain that you don’t do. We’re we’re probably not in that group, but we’re quite atypical in that. So there is no incentive for someone who just gets stuff done in word to do something better. It’s not better for them. It’s worse for them because they’ve actually learned something. They’ve got to do extra work at this point to do it. So there’s that that pushes against it. The, the, the cost and greed thing is my my remembrance is the thing that really got PDF off the ground was the American court system went in absolutely feet first, quite early on in PDF and literally turned around and said to people, you can’t get through the doors with paper PDF only. And that that helped kick start a whole sort of the whole thing of enterprise tools. This is there’s this whole area of, of software that we don’t see that starts with a price of we can’t tell you, you, you know, if you if you need to ask, you can’t afford it.
Mark Anderson: That works at enterprise. But what a lot of what that’s doing was also driven by the fact that people had massive warehouses full of, you know, 50 years worth of invoices, and that’s what they were trying. So that’s one of the things they’re trying to get rid of. So there are a lot of there are a lot of factors playing in on this that aren’t, aren’t sort of strictly affecting what we’re doing. But explain why some of these things last. And I think it’s also the case that we mustn’t lose sight of the fact that there, there is value in having, in a sense, a copy of record. Now, whether it’s a, whether the copy of record is fixed typographically or any in the nature of what is written is perhaps up for grabs. I think the state I think most people’s expectation today, if you ask them, would be I mean, for instance, if you buy a house, you probably don’t want that someone to render it in a copy of CSS Zen Garden. I mean, it might look pretty funky, but you’d probably be worried about how how sort of how fixed your contract was. So we we’re in a transitional phase. And what we’re doing slightly is pulling against the, the drogue of people basically doing not doing what they don’t have to do because they don’t need to. And I don’t mean that in a sense of ill will.
Mark Anderson: It’s just that, why would you, if you didn’t have a need to change something that, as far as you’re concerned, works? I think what’s really interesting about what we’re doing here is we are looking at a place where it doesn’t work, and we we can begin to explain why. It doesn’t mean it’s all for naught. And there’s a vast, you know, 30, 20 years worth of PDFs that aren’t going to go anywhere fast because I doubt anyone’s going to pay to effectively remediate that body of work. Some of it, yes, but most of it no. So we had to be practical in those things. But at the same time, I don’t think that the difficult part is to accept that as a sort of status quo, but not feel bowed down in front of it such that you can’t change anything. Because we can’t we just can’t get the process otherwise. And I think the the potential affordances of that and the malleability of the exile space really offers something both for really creative work and for something that we might consider quite workaday. But it’s going to need a more flexible format. And I don’t think any of us yet know quite what it is. The things that we none of the things that we have quite work for it because none of the things we have were designed for it. I mean, HTML wasn’t designed for this either. Html is useful at the moment, because one of the things we discovered after the last sort of couple of years of, of the precursor exile stuff we did here in the future of text, is it’s probably the easiest way to move text in and out of the sort of the exile space other than in its sort of compartmentalized into a predefined document.
Mark Anderson: But, you know, it just wasn’t made for this. And, and before I go, sorry. One one last thing, and I’ll see the floor. I’ll go to a question Leon asked is another problem with things like HTML is they. They never went to. They never went the full sort of full distance of really bothering to to embrace issues that exist in the sort of the highest form of our, our writing in factual writing, as opposed to you know, literature in terms of things like endnotes and footnotes and references and bibliographies, which are fantastically complicated. They are deconstructed, all they could be done. But the people who wrote HTML, that wasn’t their problem. And it has been interesting to see the looking at the ACM, HTML now that they’ve obviously put some effort into making all the links go backwards and forwards because they realize that, for instance, you’re going to have the same problem HTML as you have in a PDF. You look at a reference, you go somewhere. Now where do you get back to where you were? That’s that’s a non-negotiable want from the reader. It’s a non-negotiable fail at the moment for most of the tech, but I know we’re addressing that. Deanie.
Dene Grigar: Yeah. Thanks. Great. I have students coming in in five minutes. I’m just going to say this very quickly. Leon, you made a good point about. Weren’t there ways to keep hypertext much more stratified? And the answer is yes. That was called a platform called Story Space HyperCard. I mean, when Mark published his story, he published afternoon a story and I couldn’t I can’t go in and change it. Right. It is what it is. That said, there’s 18 editions at afternoon of story because every time a new platform, a new, a new Operating system would be introduced or upgrades to the technology. The loss of a CD-ROM drive. He had to redo after the story, so I have 18 editions of it. So even that wasn’t as sturdy as we thought it would be. I do want to mention one other thing that came that I think is also important to note is to note. To think about is that when I mean academia is an institution, and the whole point of an of an institution is stability, is is intellectual stability. Right. And that’s our job is to be stable and, and to say this is truth, this is knowledge. The problem is we’re living in a time when truth is evolving.
Dene Grigar: It’s a series of evolving facts based on the knowledge at hand. And, you know, a hundred years ago, knowledge at hand would change at a very slow rate. So, you know, you find a rock and it’s 10,000 years old and maybe that will get published would take two years, four years to get that out into out into the academic world. And then another rock would be found that’s 20,000 years old. And that changed everything about the 10,000 year old rock. But that would take decades. Today, things are changing in the moment, and so the whole idea of truth being wobbly is very upsetting to the public. It’s caused us to lose respect. And faith because knowledge at hand is going to be changing. It isn’t the basic idea of intellectual pursuit. And so the PDF is kind of a false thing because it says this is truth. It’s published. It’s been peer reviewed. Everybody agrees with it. It’s an ACM, for God’s sakes. It’s such a prestigious organization. They’re not going to publish crap. But then, you know, Mark might find there’s eight, eight, you know, hypertext uses. And he’s going to have to redo that article. He’s just up, up, top sided, his whole premise.
Dene Grigar: And he’s turned upside down. The notion of ACM being a stable organization with stable ideas, there is no stability. And I look around the room and I’m looking at tables and chairs and things and and if we really had the ability to do it, we’d see that these things that are seem so stable that the desk that I have my hand on this is moving as I put my hand on it, because molecules are moving. I can’t discern that right now, but it’s a truth. Nothing is stable and we don’t. We can’t grasp that. And when we do, it makes us crazy. So what we’re suggesting will make people crazy. But doesn’t mean there’s no truth. It means that truth is only what we know in the moment. There’s some universal concepts that should not kill, but even that is wobbly. Because if someone comes to my house with a shotgun to kill John, you know that I’m going to cap that guy in his knees. Right. I mean, there are things that are always. Going to push against what we think of as truth. And that’s horrible. It’s horrible. Anyway, I’ll stop because I’ve got to open the doors for students.
Frode Hegland: But Frota yeah, I’m not so sure. I mean, it would have been really good if we stayed on topic and talked use cases, not the old HTML pdf discussion, but I don’t think academia is concerned with truth and I’m the least academic here. I haven’t even gotten my degree yet. I think academia is based on dialog, and you can only have real dialog when you are able to refer to something, and that’s what citations do. And that’s why we have to look at a timeline. Things will change. Absolutely. The problem with HTML is HTML. Promising. Html is just plain text for the external needs. There’s nothing there in an HTML document, right? Other than plain text. It has nothing there. It must have external resources. Pdf is also plain text, but it can also afford a layouts. And when you publish a PDF, the whole point I see of a PDF is it isn’t going to change. That means if someone sites it to say this is not how I see it or new evidence has come. You have that PDF to refer to. But if you have an ever changing like Google Docs, if you have an ever changing document or a word document being sent back and forth, you can’t cite it. So you don’t you lose your history. I think that’s a real problem. Like we should have flexible views, but we shouldn’t not have the ability to store things.
Dene Grigar: Well, I think this discussion has been very fruitful. And it’s this is the kind of stuff, I think just getting into the nuts and bolts of this is important, because the case study needs to be focused on one thing or another, and I suggest that we do PDFs. That said, I don’t think PDFs are the future. They’re not the future of text in XR especially. But we can’t say that until we prove it.
Frode Hegland: Leon I what’s the alternative then? Do you think to PDF for the future for academe?
Dene Grigar: I mean, you’re talking to an academic. For an academic, nothing. Pdf is what I’m using. That’s the problem.
Frode Hegland: No, no, no, this is a discussion that’s really, really important to stop having because we’ve had it so many times over the last few years in the community in different forms. And if there’s a viable alternative to PDF, mention it, bring it up. Let’s talk about it. If there isn’t, this really is a waste of time today. You know we have things to do. We all are very busy. So often it comes up and you know, when I had a meeting with the ACM Digital Library Board, there was also someone there saying PDF is horrible, it’s not the future. And I also then said, yeah, you know, there’s a lot of things to criticize, but what’s the alternative? And they said, we have no idea, right. Yeah.
Dene Grigar: But we have the opportunity of coming up with that. I’ve got to go. I’ve got my students here. But Leon, I hate missing you. What you’re going to say, can you say what you’re going to say?
Leon van Kammen: Yeah. Well. Oh How fast? Or I just wanted to say the reason why this comes up, I think fraud is that I think also we are mixing up use cases of the fast documents, let’s say editing linking, blah, blah, blah, like we are used to in HTML. And we are sort of conflating that with stone tablet like documents, for example PDFs, which they are no longer text literally, they are turned into zeros and ones and it’s a binary format. And so to have a, I think a more easy discussion about these things without falling into the file format trap is, I think maybe first identifying which type of editing you want to focus on, if it’s more the fast editing a word or a grammar mistake or a link here and there, like you can do with HTML, then that’s a certain direction you could go for in XR, but if it’s more like the stone tablet PDF kind of which is more read only then I suggest, I think it’s better to isolate certain use cases and focus on use cases like positioning. Something here or there, but not really. Not really editing, because then you kind of like, you’re bumping your head to the binary correct characteristics of the, the data you’re working with. So you have to serialize these, serialize. So in to, to put it short, PDF was not designed to be basically liquid. So long story short, I think this topic always comes up because we are sometimes talking about liquid use cases for non liquid documents. That’s what I think.
Dene Grigar: That’s perfect. That’s absolutely right on. Thank you. Frodo. Sorry we got on this track again, but I think it is an important issue and we’re going to keep hashing it out, I think, for the rest of eternity. But I won’t be seeing you. You’re going out to Japan tomorrow, so safe travels.
Frode Hegland: Thank you. I will be here for every meeting. It’ll just be a bit time shifted. But. But I think for this community, we didn’t get to finish the use cases. But I really do think we need to decide that as I think we have PDFs are what we deal with. However, if there is an HTML version available, we use it. Right? That is that is really important. Yeah. But also the discussions of the pros and cons, you know, that we have hours and hours of that. And one thing that I think is really wrong in the discussion, I’ll do it in one single minute. And that is they’re both texts. Right. But PDF can have images in it. Html cannot. Right. It is always referring to external stuff. It’s always somewhere else. Right. Or you can do some encoding, but you can’t put a video inside an HTML document. Then you’d have it. I see you’re shaking your head, Leon. You have to do funny things, but it’s not an HTML page anymore. So the thing is, representation wise, we’re going to be doing webXR. It’s a whole new world. But what we’re basing it on if if there is an alternative, we can talk about it. But I want to be able to cite papers. I don’t want them to have changed. When I come back to them, that’s really, really important.
Dene Grigar: All right. I love you all. Great morning. Got me excited. I’ve got my students in the room missing one. But I’m going on to our meeting. Have a wonderful rest of the hour, and I’ll see you soon.
Frode Hegland: Yep. Yeah. See you later. Thank you. Danny. So. Yeah.
Leon van Kammen: Sorry. Yeah. Sorry if I derailed the agenda a bit here, but I was just thinking I was thinking to really thinking use cases. Any use case. I bet Andrew has to code. Has to And I’m gonna. I’m not gonna mention the word PDF or HTML. From now on, imagine we have the document type Pu and this document type Andrew needs to sort of, like, implement some kind of use case with he has to deconstruct this file format. So this is what, what I also thought was very good about Mark Anderson’s mentioning of document deconstruction. So whatever the file format is, it has to be deconstructed in order to be projected in XR. So this deconstruction needs some kind of internal format. And we can say, now we don’t know. We don’t care about what that format is. We only care about use cases. But this this is probably not a good start to develop anything with. So there needs to be some kind of internal model of a document. And what I think is that for this internal module or model, this format, since we’re in the browser, I think the document deconstruction with HTML as a proxy, what Mark Anderson suggested could be a very interesting way to and useful and handy way to sort of like have a document in the browser memory and be able to work with it. In XR, its projection of, of that. So it doesn’t mean we have to render HTML.
Leon van Kammen: I’m just saying that the internal format of the XR environment, the invisible. Double internal data. Document data can be HTML, so we can basically use it as, as a as a database or to store extra annotations. I think that is a very interesting, interesting thought. And it doesn’t mean that the document has to be exported to an HTML file. This can all just happen in memory and whatever is being done to this XHR this document in XHR, that can be perhaps perhaps saved as this metadata can also be stored in this HTML document, maybe even in a, in a different format. Visual meta doesn’t matter. But the point is, is that if you have these two components, like the internal representation of a PDF document and some extra metadata attached to maybe a render, the annotations or references or links, if you have that, then I think that that would allow so many use cases compared to not having this intermediate format in the browser memory and trying to do everything in real time on the PDF object in memory because I think that would be very slow. So I think a good way to not end up in a PDF vs HTML discussion is to basically say, are we talking about the an internal memory format, or are we talking about a real physical file? So I was more thinking about the internal intermediate format.
Frode Hegland: Okay. And that’s how you take someone from Holland, puts them in Hungary and sends them across the globe with exactly what we need to do. Okay. I would go a little bit further than what you’re saying. I put the things down here because. But with all due respect, PDF or HTML. Who gives a shit, right? If they stuff that exists in the world, what we will use in the library, that’s just something that happens. I put in the chat here from our Sloan. Pitch, which was approved. And it does say different things for different uses. So I think we should do an under. You’ve been listening the whole day. But this particular bit. We need your full brain. I think Leon is 100% right. I think we should have an intermediary kind of an HTML dog would call it a Exe file or something. However that and the JSON and please shake your head Leon if you disagree should be virtually indistinguishable. I think that the PDF is two things in this world. One is the raw, plain look at me, I’m pretty. The second one is the appendix, where we have the initially made appendix, which is visual meta in some cases. And then we have the tradition of hello, I’m visual meta, you can throw me away. Right? So if we just take the best of HTML and JSON as the technical community here will dictate. Fantastic. You know, today my programmers implemented the ability that I think all of you were in the beginning to export from the library. You can export a JSON that really should be done with what Leon is saying. It should be fulfilling all those requirements. Because who gives a pardon? My honest language again? Who gives a shit what the original format is? You know, we should build the most optimal solution. So I’m trying to repeat what you’re saying, Leon, without being too boring, I just think we should also store it, not just have it as an intermediary. Under. What do you think of that craziness?
Andrew Thompson: So I. I might have missed earlier, but it seems to me like this was kind of the track we’re already on. I’m sort of missing what, like the revolutionary concept is from this discussion.
Frode Hegland: The revolutionary concept is, you know, Dean is relatively new to the community, so she hasn’t suffered our PDF HTML bytes over the years as much as some people, you know, Mark and I are very battle scarred in that regard. And the notion it’s really an emotional thing, forget whatever the original stuff is. And for the JSON, it may very well be a good idea to put some HTML in there where appropriate, because the whole idea is so. We mustn’t forget my personal software because it represents any other developer, right? So in my personal reader software, the library should ideally take. The citing to cite that document the headings for that document. You know, the normal visual metaphor, but also the entire thing. Because if you encode text in a PDF in a clean column, it’s completely extractable. It’s clean. Once you do any formatting, it goes haywire. We all know that, right? So all of that should be able to go by the JSON, which is just a sending mechanism really to your world and go back. Right. It is what we’ve been talking about. It’s just this kind of hang up about HTML versus PDF that got a bit off track. Mark, I’m sure you have a better perspective than that. Yeah.
Mark Anderson: Sorry. I mean, I think what I hear coming out from this is something that we’ve nibbled at here. Is that so? It’s shifting the perception from worrying about the name of where things came from and more about the elements. So it’s not just as simple as the JSON or something. It’s it is actually the structure in it as well. Because if we do that wrongly, that just gives that just gives Andrew a harder job trying to do what we want with it. So we can say that, you know, something like JSON into De 2024 is something that we broadly know and understand is a common way of putting of, of, of, of of moving stuff around. But I think, I think where we’ve been getting lost is that Unintentionally. We’ve been looking at how we get this out of the starting formats, and I actually think that is unhelpful at the moment. I really do think we should be thinking about the the parts. Of a document. So imagine the document. So in other words, you know, I mean, for instance, I’ve often used a word body copy. I don’t really like that because that comes from sort of type type typography and from and from and sort of writing sales copy itself. But but there is for instance, there may be a main linear narrative. There’s a main section of text, there are the structures within that.
Mark Anderson: How much of that structure, you know, where does that structure live? I don’t know, because it’s easy to say, oh, well, it just goes in some JSON. And it can, but that’s basically that is punting the difficulty down the line to somebody else. So I don’t care. It’s an implementation detail. And I think we need to be very careful about that because what we get at the end of it will be subtly constrained by the missed guesses we make. So I’m all for being really quite open and sort of failing early and often at this point in terms of, okay, so we did that. For instance, we got, for sake of argument, a load of JSON. Turns out that was completely the wrong structure. We had all the right things, just not right in the right order or the right granularity. And I think that will be very instructive to us. Because when we when we have these components, then we have the notion of these components and what they need to do and why they need to be in the form they’re in. Then, then we have a road map for both how we make, how we may make them from our sort of starting ingredients, these, these formats whose name I won’t now mention and how we may use them in, in the Excel space.
Frode Hegland: Yes. The other.
Mark Anderson: Thing I would say is we definitely need the VR. The, the VR definition definitely needs quite a lot of work because it’s very open ended at the moment. And I was finding myself trying to reference something and unintentionally at the moment, it rather says you can be whatever you like it to be. And I think we need to be a bit not, not constrictive, but just explain how things might go in. And so, you know, for instance, we talk about in big tech, it is not big tech. It is written using typography that looks like big tech, but it isn’t big tech, because if you put it through big tech, it won’t rent. It won’t work because it’s it’s subtly different. Big tech is big tech basically is LaTeX. It’s LaTeX compatible and we don’t need ours to be LaTeX compatible. And it doesn’t matter right up until the point where it is. So if we’re lazy, if we’re lazy in our terminology and things like that, this is exactly the things where it sort of bites us, if only because it gives the people who want to argue the narrow, the narrow cause an edge, an edge to the real thing. So in that sense, we would we we would do well, I think, to revisit what we’ve, we’ve written there and tighten that up. I don’t think it affects what we have at the moment. It just means that we can be more accurate going forward.
Frode Hegland: Right? Okay. A few things. So. What’s happening here? Something WordPress. Been acting up on some of the sites lately. The sleeve page. So just a little detail. It does say based on bib tech, it’s not bib tech. Absolutely. Agree. That’s important. Because of your text the other day. Mark, I have now added. Please note this is only a bit of it. There’s a lot more. So. So yeah, that’s.
Mark Anderson: That’s exactly what I’m saying. We need to we actually need to start writing that out because we’re we’re mentioning visual matter a lot. We’re mentioning it as being quite a core thing. And I think I had reason to go back and look at the site because I was trying to check something the other day and I thought, yeah, I don’t really understand this. I mean, I know because I know what it’s supposed to mean, but I wouldn’t know from reading this document is what I’m saying. So it’s just a matter of Polish, Polish, Polish. Yeah.
Frode Hegland: Yeah. So on what you just said I’ve updated our web page. We have a little bit of stuff. Now, if you go to XR experiments and then go to currents, you have Andrew stuff that we will have some time to go through. We’re not going to do that much on the other stuff. It is here. But if you go to interactions now, this is what I’ve done, user stories, which is I want to be able to this, that and the other. And it’s beginning to credit some of the people who want to put on things. You know, if you send it to me in a form I can put in very, very happy to. This is all here because it’s just trying to decide which specific interactions to build. Right. And they are based on the elements. And I think now because of this guy in Hungary we should change this sentence to not be from HTML unless noted. I think we should find a really clear way of saying Something like detailed metadata in an accessible way could be from HTML, but also extracted from PDF, right? What’s the more intelligent way to say that? Because it’s really.
Mark Anderson: I wouldn’t mention either the I thought the whole point is we’re trying not to mention the format. So I’d actually sort of remove that. I mean, it does raise the question. It does is visual meta an affordance only for PDF or is it if it’s wider? If it’s wider, then we want to rewrite it. Rewrite it up. Excuse me. Rewrite it more widely. Which doesn’t stop us.
Frode Hegland: You want. You’re right. There’s no need to write. All I can put it just a line here. But to look at it like that. Yeah. This is for the discussion. You’re right. We moved far beyond one one.
Mark Anderson: It’s one interesting challenge. So something we ought to take is, is to put put visual meta alongside something else and see how and if we can do that because that is us eating our own dog food. We and we actually ought to do it because we do say, we do glibly say, well, you can stick with anything. And I, I suspect that probably is true, but I’m not entirely sure we’ve actually done that except for PDF for all sorts of good reasons. So it’s not a that’s that’s just a realization that came to me as we discussed this now.
Frode Hegland: Okay. Visual meta is in the Sloan thing. Let’s not forget. Right. And visual meta is loose now, as loose as it possibly can be. All it is, is saying there’s stuff at the end of the document that can be sent separately via JSON. That’s all it is.
Mark Anderson: So that’s a PDF specific implementation. That’s really what I’m getting at.
Frode Hegland: So no it’s not. And in a word file or, or a HTML or whatever it is, it’s just a bit at the end that says what the document is. That’s all it is.
Mark Anderson: Doesn’t have to be at the end. I mean.
Frode Hegland: No, no, it doesn’t have to be at the end. But the reason is at the end is when people don’t have to read it. And and parsing is faster because the parsing will be from the end of the document, not from the front of the document.
Mark Anderson: So we need okay. So the point is we we need to express it in those terms because otherwise the wider audience don’t get don’t get the thought that’s gone into that.
Frode Hegland: Yeah. But that’s important for you and me. Not not not for anyone else.
Mark Anderson: No. That’s the whole point is no. It’s important for everybody else, not for you. We know what I’m saying is exactly the opposite. I couldn’t I misstated the whole point of saying it is to the to the wider audience can read that.
Frode Hegland: We’re also running out of our time, right? No, let’s not do this now. The person that needs to understand this is Andrew and Andrew, right? Basically, Andrew, for now, right. We got to nail down some use cases. We just have to do that. So we had.
Mark Anderson: We’ve got three of them. We know.
Frode Hegland: They’re not. Hang on hang on hang on. First of all you said to kind of disassemble the document, which is very useful. That’s what I highlighted here. So what are the the interactive elements we should at least have. Right. I’ve started listing them. This is probably the kind of stuff we need to agree on that should be extractable, right?
Mark Anderson: Yes. I mean, I have to sit down and think about that, but yes, essentially. I mean, one one thing that’s missing there is the actual text.
Frode Hegland: Once on WhatsApp.
Mark Anderson: The actual text, which doesn’t seem to be in there.
Frode Hegland: Hang on.
Mark Anderson: Which is the text as as as standing separate from the the wrapper, the the document that encoding that it comes in.
Frode Hegland: I have document reading of different sections. That’s what I mean by that. Very badly. Yeah.
Mark Anderson: No, but I think it needs. So one of the things we actually need to come to an understanding of is what the text is and what we mean by it.
Frode Hegland: But the full text of the document is the full text of the document. And then we also have to look at the things inside it. Right? Yes.
Mark Anderson: But we need to. Well funnily enough, but when you talk to people, it’s sort of not that if you copy everything out of some documents, you get all sorts of stuff that you didn’t think was the text. So it needs to explain what it means. So for instance, it’s probably you need to, for instance, be able to get the text shorn of things like sell copies, running headers and footers and something which are not that are part of the document but are not on the visual representation of the document, but they are not part of the central text or they they are only some of the text. Okay, we’re not yet drawing this distinction. So that’s what I was meaning at, by the way.
Frode Hegland: I don’t think we need to these are quote unquote common sense ones. If you do the.
Mark Anderson: Notes, these are really important things to get right. I couldn’t I’m sorry I they are important because otherwise you you it it’s a thing we tend we all do it because it’s nasty.
Frode Hegland: It’s pointless. Then these things are so are you know, when we’re talking about the text of a document, if you select text and a PDF and copy it and paste it and it’s a bit garbled, that is not the text of the document. That is a technical issue that we have to deal with. That has happened during the way, but no one is going to think that things moving around in a way that it wasn’t in the original. Is the text right? And if you it would be really, really nice if if anybody wants to add to this list so we can have a more of an agreement of the of which element should be interactable. Right? Andrew has started doing things like that. You know, with the reference section. We need to have an idea. I opened this section because you talked about kind of exploding the document and I completely agree with you. That is very important. So we need to know what things should be explodable. Right?
Mark Anderson: Yeah. I explodes probably a an inelegant term. What I, what I was meaning to all I was meaning was that that whereas if you started from, say, a book from 20 years ago we might want stuff that we wanted off that substrate. And what are the bits, what are the bits that we can get? What are the bits we want? What are the bits we can get? And are they, for instance, are they the same? Which is what I was driving at when I said, well, actually we do need to understand what the text is because we tend to we tend to ask for one thing, get another, and then lie the difference. So, so there is there is something in there because that gives us the cleanliness that then when we when we pass something, we make something. We give some data, for instance, to Andrew that he gets what he expects. Because one of the things I’ve really taken on board from a couple of things I’ve done making some JSON recently is people say, oh, that was interesting. That had basically the gist of it was had far fewer artifacts than I expected. And so I find that useful. Okay.
Frode Hegland: A very important thing is that in the Sloane system, my software is included. Okay. I am an exemplar outsider. So that means that for us to say that the visual data should be encoded as cleanly as author does. Or the JSON should be encoded cleanly. That is a given. We can say that this is the point of. This is to go to the ACM at the end of it and say, look, if you have clean systems doing this, that and the other, you can have these results. This is not going to work on old stuff. But Leon, please.
Leon van Kammen: How much minutes do we have? For the meeting? Because based on that, I will.
Frode Hegland: 33. 33.
Leon van Kammen: Okay, well, I have a question. Actually, I do like this. The focus on use cases and my concern is a bit is performance, and I perhaps Andrew can enlighten me a bit. Is it right now engineered in a way that the PDF is deconstructed to a document or a Dom object and then sort of based on what is in the the Dom object. Is then rendered via three.js. Or is it PDF? Some custom JavaScript which immediately maps to three.js in basically the XHR projection.
Andrew Thompson: Okay. I’m trying to follow what you’re saying there. We don’t have technically a PDF as the start. We’ve got the HTML export of the PDF because of the tagging. So the current JavaScript version that I have that I’ve written out doesn’t even touch a PDF. It just counts on it already being converted into HTML. So it takes an HTML import. And because of all the tagging, it knows what to grab where and then it maps it directly onto three.js. So it, most of it is just in three.js. The JavaScript itself, outside of three.js, is just used for searching through the HTML file, finding the the proper tags, and then being like, oh, we want the citation for this test and it grabs all that.
Leon van Kammen: This is really cool. This is great news because sorry to maybe I’m a bit late to the whole party, but then the whole HTML as an intermediate intermediate memory format is there, which is great news because it’s highly optimized for searching, querying headers, this or that. We this is really like on a low level like optimized instead of, you know, doing everything manually by JavaScript. This also gives us the benefit that if we have more documents, open Dom documents in memory, we can easier very easily and very fast query those documents at the same time. So this is really great news. So, yeah, that’s so you answered my question. And yes, maybe the conversation is not as revolutionary. And but you have really it has really informed me that I’m fully up to date with what is being done, and I’m very excited by it.
Andrew Thompson: Perfect. Yeah. Using the HTML is much nicer on my end, because I don’t have to make a whole interpretation algorithm to read through a PDF and try to find everything in the right spots, because it’s already tagged in the HTML. So if we keep going that route, it’ll be easy.
Leon van Kammen: One more thing. This is an idea. It’s maybe not useful, but if you have the So the PDF gets converted to the internal HTML, and then it might be mutated, it will be edited or some annotations. And afterwards you can also make you can sort of execute a diff between these two. And you can basically see only the changes. And this is really cool for a sort of Personal meta base almost of all your documents, let’s say all the meta data, which you can personally project on top of the documents. Instead of saving it to the document per se. So there’s two ways to to save this edited document. You can save it sort of in your personal library as annotations or edits, but you can also save it as a new version of the PDF. So that’s very interesting connection I just made.
Andrew Thompson: I actually really like that. Right now the HTML file is just kind of a read, only we get it, take all the data, and now it’s inside the JavaScript. And I do everything else there. It doesn’t write it back to the HTML. It has its own export into JSON. But saving it in the HTML would actually be. Perhaps a better route to go. It wouldn’t be able to save to the original location, because that’s not always writeable. But it could make a copy, perhaps, and then save all its information there and then the export would just be that file back out again with the additional data now saved inside of the HTML tags so it wouldn’t ever show up into anything other than our software that’s looking for it. That might be a much better route to go than what we currently have. Obviously that’s currently not the way we’ve built things, but I quite like that. It’s just adding an extra step in the middle, but giving us a much more consistent. Export because I had a big pain getting the save and export system working. With the way we currently have it. Which now is probably not the time for that, but I can talk about it a little bit more when we get to the the dev update. Thanks for getting the gears turning land.
Frode Hegland: Let’s go to the dev update. Mark, say what you have to and we go to the dev update.
Mark Anderson: Okay, I was I was only going to say that I’m. I was just thinking on the fact that how doing close linking works. If the text is stored within a form like HTML. I’m not saying this to be troublesome, but I’m just conscious that a lot of the linking used to work off character offsets within text back in the days when we didn’t have rich text. So text was just a string of characters. I’ll leave it. I’ll leave it there. It’s just something for us to bear in mind, because when we get to the point where we want to start linking things and doing close addressing, this may be something we need to address.
Frode Hegland: All right. Before. Okay, let’s just have a quick look at this then, because I’ve just started I’ve moved. Denise, use case into a research question. Page. So let’s because what you’re saying now is really important, Mark. Blah blah blah. The user, once they’re in the headset, these are some things they can do, right? One of them is And get rid of this nonsense is decide on how many pages and all of that one is going to be link. So let’s talk through that a little bit. It’s crucial to what Andrew’s going to do. Create. But okay, let’s. Since it’s reading, copy a citation from a specific. Section of text, right? What will that mean? By suggest that a document okay to me. And if you don’t agree strongly, tell me. If you don’t agree weakly, who cares? That’s okay. A published document is very different from a manuscript. The manuscript is what you do yourself. You can edit forever. Who cares when something is made public? That’s all I mean by published. Even just given to friends. It shouldn’t be changeable. There should be a new version. And of course we need to look at versioning and all those cool things in the future. Right? But I think that at least for what we’re doing now, the document when you make a new document should be a PDF with a very rich, extremely extractable visual matter. That visual matter should be exactly what Leon and Andrew are designing now. So that means that when you open up a document from all that metadata and you come across a section on chocolate cake, you should be able to select it, copy a citation. And what you’re actually citing is that PDF. So when someone then uses that link in the future, it will reference the original PDF and you know to that page if you want to look at it as a page, but it will have all the metadata to recreate the view you saw it in, which is one of the great things about what Andrew has been working on for today. Does that make sense? Sort of.
Andrew Thompson: Okay, we do have one thing we have to clarify. Since we’re back to just the angel, we’re we’re using actual PDFs, not HTML anymore, so nothing’s tagged. Are you going to have in your visual meta something that indicates where the different tags are supposed to go?
Frode Hegland: Don’t forget, we visual meta is hardly anything. We can decide that whenever we want can decide it. Right now, we can decide to use x, y coordinates. We can decide to use text string. We can. And pages.
Andrew Thompson: There’s a difference though, because visual meta goes at the end and HTML tagging goes in the middle. So HTML tagging is is hidden because it’s viewed differently. So if we’re talking just using visual meta we have to come up with a way of basically having the middle tags at the end that reference back, which is possible. I’m just wondering if that’s the route that we’ve decided to go now. Well
Frode Hegland: Let’s look at it this way. I’m a couple of stages more naive in this than you are, so I could imagine that one of the visual meta pages is the full HTML of rendering this. But it’s going to.
Andrew Thompson: Be huge though. You’re going to have massive file bloat.
Mark Anderson: Yeah, that doesn’t really strike me as practical. I mean, theoretically not practical.
Frode Hegland: Hang on. Not with CSS. Right. Html isn’t that massive. Right. But it’s all.
Andrew Thompson: The text again. You’re just doubling the file. Yeah.
Frode Hegland: You are absolutely. Mr. Anderson, how long is the average ACM article for reference?
Mark Anderson: A standard full article is normally 12 pages. 1012. Call it 12 to 14 A4 pages.
Frode Hegland: So one of the things that I’ve asked my breeder people to do today, because in the Vision Pro headset and I hope you have a chance to check it out soon, Andrew. And you Rob, very, very soon. You can have a PDF open to all the many pages. Not all, but many, many pages really wide. So I’ve asked them to not show the visual matter. Right. Visual matter is only really useful in readers that know what to do. If it’s a human going through, they’ll get to a page and see this stuff and they’ll stop reading. I mean, I deal with videos and pictures that are gigabytes to have 100 pages extra. You know, over time, if we do it, software will just ignore it. So I really don’t think that is a problem right now. It is not the most elegant. No question about it. But the whole thing of the benefit of of HTML for rendering is absolutely acknowledged. There’s no question about that. Yeah. Okay. More quickly. And then. Leon.
Mark Anderson: Sorry. No, no, Leon. Go ahead. I’ve got I’ve got an idea.
Leon van Kammen: Yeah, just just sorry. You know, I just came up with a very low hanging fruit. Easy use case. Pretty powerful to implement. Based on the fact that PDFs, especially imported ones, don’t really change that often. Let’s say especially very significant PDFs is the fact that the use case of selecting a piece of text in XHR could even be in 2D, whatever. Then it will basically copy a link to your clipboard, and this link basically has the the, the PDF file name plus the where, which page and where exactly that snippet is. So in other words, if you share that URL, somebody else can open that link in their VR headset and then immediately automatically have that PDF opened on the right page with the text highlighted, basically a quote. I don’t know if if that is interesting at all, but there is basically a standard a notation for this. And I will Posted in the chat. Right now, it’s basically called a text fragments and it’s interestingly enough, not used that much by HTML people. But I think since we’re having this PDF in HTML kind of thing. Pdf could definitely benefit. If it from this and has a higher, even a higher use case for this, right. So this is fairly, fairly simple to implement. It’s a real.
Mark Anderson: Shame. I something I was reminded of actually looking at some stuff from the other day is that, I mean, it’s frustrating that the PDF world never got behind using destinations because that that basically is an anchor hardwired into the postscript. Most, most bookmarks and things like tax and stuff are are depressingly fragile in PDFs. Because nobody felt it was worth their bother to implement them properly. The standard allows for it. The idea that I had listening to Andrew describing the problem, you know, going in and so. Okay and say, for instance, having to have two to effectively two complete renditions of the text is that it does one. I do wonder if you could actually essentially print your HTML in the PDF. But render it as if, in other words, you don’t show the HTML so so that in the the difficulty is that I mean, I don’t know how much people have looked into the internals of a PDF, but the weirdest thing is when you look in there, there’s no text, because what what you actually have is something called PostScript, which is a really bizarre language that basically says draw a glyph this shape, draw a vector shape this shape on this distance on the page, which is why it’s very y. It’s very tied to paper pagination. Because this was all to do with printing. We have to remember so there there isn’t a text you actually have to read out.
Mark Anderson: You have to get the postscript to do stuff. So when you select something in a, in a PDF reader, what it does is it goes and finds all the postscript works out the text it represents, puts it back together as best it can, which is why it sometimes makes mistakes. And here’s your text now so maybe what what what we’re arguing for is going forward, we, we need to sort of hold on less tightly to this whole postscript stuff that doesn’t really do us any favors. And. Employ. The fact that the PDF is is a portmanteau format. You can put all sorts of things into it and put text in, in a more, in a more usable sense. So in other words, that you know, if Andrew wants to get some text, he’s not having to go and look for lots of multiple implementations, which then cause you a further versioning problem, because these 2nd May get out of out of whack. So essentially you have one representation of the text that may, in its sort of traditional PDF form, look like a printed page, but in fact contains when it when you actually call the text comes back with some form of semantically relevant markup. That would seem a possible. We can’t do it today, but that would seem a very sensible way to move forward.
Frode Hegland: Right? Well, I started with the whole PhD in visual media and all of that stuff. I did try to do embeddings and I got all sorts of problems with security. I even tried embedding a word document so you could extract it yourself if you wanted to edit the original, and it just wouldn’t allow it to happen. But I agree with the approach. In general, there are different ways to show or not show, but we do have one really important issue though with this and that is if you. I mean, we could do a hack where we do everything that we’ve talked about. But if you cite a selection of texts and you give that citation to someone, the mechanism through which they find it is basically by finding that unique string of text on the document, because the PDF rendering and the HTML rendering will be different no matter what. So we, you know, we should basically allow them. How do you want to view this as, as an HTML bit or as the quote unquote original PDF?
Mark Anderson: Well, one problem I I’ve sort of found with PDFs is it’s the text is is basically not it’s actually less is less stable than you’d think because because of the PostScript layer, because it’s not really stored as text at all. You print it when you make a PDF, you actually print your text. And there is cleverness that allows people to magic text back out of the PDF. And this isn’t to not the format, it’s a it is a sort of genuine technical thing that you have to do so that there isn’t. That that text in a sense, you know, I don’t know what one’s looking for. In in the PDF.
Frode Hegland: When author exports to PDF and with the visual media, we do one column. And when you select it to extract, it’s fine. Any time you do anything sophisticated with the layout, you get issues. So you’re absolutely right. But with a modern PDF rendering engine, it is actually possible to get plain text out of it. And we are talking about going forward.
Mark Anderson: It’s possible, but not consistently possible is the problem. That’s what I mean because I still find with with PDFs produced today in seemingly, you know, current stuff, it’s difficult and it’s worth them. There was a really useful thing that Peter it was a it’s a, it’s a Reddit thing and it was talking about the problem. This like there is a whole industry area, things like trying to read tables out of PDF because it’s not impossible. You basically have to write a custom, a custom program literally for each, each production machine. So what that tells us is that there is there is a degree of complexity. The danger is saying, I did it and it works on my thing. I would say the, the, my experience is that that, that, that that case doesn’t work at scale. So we need to be a little more cautious in being in being too optimistic.
Andrew Thompson: I got one thing to maybe add, Mark. I like fundamentally, I’m, like, agreeing with what you’re saying. But also like. Because this is such a relatively small project in the scope of things like we have one developer on this. Yeah. Making everything the correct way is perhaps not something that we should hold ourselves to, or we may not get anything done. Agreed. So, like, I’m all on board for, like, let’s find a smart way to do it. But if the smart way is redesigning something from scratch, let’s be like well, that’s a future problem.
Mark Anderson: That’s too big. It’s it’s I totally agree. What I was, what I was sort of driving at in this is, is part of the useful stuff that I think we will render for the, for the, for the grant givers is, is an understanding of the, of the real problem at issue here? Because most people look at the surface layer of it and imagine picture a picture B in their mind. Okay, that’s sort of doable okay. But it’s actually, you know, we know it’s a little more complicated. And so we can usefully lay this out for them.
Frode Hegland: Okay. But Mark, it works with author and reader. Right. And that is one closed system, but it’s entirely open. It’s an exemplar. So the only issue we have, and that is this is a real issue of absolutely knowledge is glyphs, sometimes not glyphs. Excuse me. Ligatures. You know, when you have the F and the I. Obviously you know what it is Mark. Yeah yeah yeah yeah yeah yeah. Which can be programed around. But the point is we are trying to present an exemplar system where if all the pieces are done right, it’ll work. And I think that is absolutely fine, because that will make it clear that if ACM etc. don’t do all the pieces right, it won’t work.
Mark Anderson: Can you? So a useful thing to do is could you make could you make author actually create an ACM format document? I’m not thinking so much to column, which has a complexity of its own. Yeah, just.
Frode Hegland: Formats 100%, but I’m not going to pay for it. But I could. Yes.
Mark Anderson: Okay, but we need some way to to prove that point. Because at the moment and I’m and I’m not saying this as a critique, but the documents that author can currently produce are much simpler. So I would expect them to have fewer failures. Yeah, but it’s not.
Frode Hegland: It’s not. But that’s the key Mark. That is the entire key. The key is that the document itself is one thing. The visual media is something else. The visual media is super simple. That’s why it can be extracted. So we don’t have to worry about the entire ACM document or whatever it is. That’s just a photograph. That’s just just a picture. In this context, the visual meta is super, super.
Mark Anderson: We haven’t defined what the visual metaphor is. So this is why it’s I take it on trust, but we need to define what it.
Frode Hegland: Is in the sense, in the sense that it is an introduction and data. We have defined it fully. Right. Meaning? Yes. What? Mark. Mark, please. This is really important, right? What is inside the data is a very important, useful discussion that’s going to take a while. But in terms of extracting it, it is intro and data and it’s in a rectangle. Sure. I mean yes, that means that in terms of how to put it in for Andrew and take it out for Andrew, that’s done. The question of what should be in there is still very much on the table. Yeah, we know where it is. Right? Right. Okay. This is probably a good time, Andrew, to spend time on what you have got to show us. I will. Obviously the use case will need some more work, but I’m going to put in the link here to your latest and greatest. For those who have said, oh, Mark, are we going to see your headset for the first time? That’s exciting. So.
Mark Anderson: We’ll see. How do I tell quest three to stop thinking it’s using hand trackers and to use my hands. Is there a signal I can get? It’s.
Frode Hegland: That’s a good question. You know.
Andrew Thompson: So you’ve got the the hand tracking enabled or it’s not there yet.
Mark Anderson: Okay. I think probably the answer is to put these the other side of the room. Right.
Andrew Thompson: Yeah. If the controllers are not on, it won’t bother. I mean, it tries to turn them on by default if they’re close by, so I just leave them in the box and never worry about it.
Frode Hegland: No.
Mark Anderson: Because I tried your demo earlier and I got as far as a totally gray space, so I did something wrong.
Andrew Thompson: What you should be able to do as well is if the controllers are like on and it hits, that won’t ignore it as you tap them together twice and you put them down and that’s the signal to the headset to switch to hands.
Leon van Kammen: Which alpha number are we talking about in the URL.
Frode Hegland: On the URL, it’s the top one. It’s by date if you go to the page.
Andrew Thompson: 40224.
Frode Hegland: One thing would be nice is if you have a breaks in the. The references is really long, obviously. I’m not sure what.
Andrew Thompson: I did test that. And I ran into some problems with the current way things are set up. That’s not to say it’s impossible. It’s just I didn’t dedicate enough time to solve it because. Essentially, the three.js doesn’t know how large troika text is. So you have to manually tell it how far to leave spaces between them. And if you have word wrap on. So it’ll wrap the second lines. Three.js just has no idea where the text is. And you often get text overlapping itself because it doesn’t intelligently go to a new line. So that’s something I’ll have to figure out if we want to start doing line breaks like that.
Frode Hegland: Okay. So that’s an issue for the moment.
Andrew Thompson: Yeah, a trek a text has been having a lot of problems in general. Even just from testing. Like, I get frame rate drops when I have more than, like, maybe ten different citation things out. So it’s. I’m not really sure if troika is as robust as it’s been. Advertised as.
Frode Hegland: Try special thing.
Andrew Thompson: Yeah. The main thing in this update, you’ll notice, like it took me two weeks. I didn’t have one last week. And then it’s like there’s not much in my dev list. The main thing I was working on was the import export system. So you can now create the do your work inside the headset, and then at some point in the prism menu, you can export. It’ll save the whole scene as a JSON file. And you can import those again later. It’ll remember where things were. It’ll hopefully remember where everything’s connected. The how things interact with each other. Which was Something that I thought was going to be easier to do than it was, because I wasn’t sure how. Three.js interacts and saves information. And as it turns out, it’s very convoluted. So this may be more of a test rather than a final result. If we move forward and start saving information as visual meta will probably use this code as like a basis and rewrite a lot of it, which is fine. It’ll probably work better if we do that. But this is a a good test. No, because from as much as I’ve Done the initial testing. It does work, which is pretty cool to see. It’s kind of a save load system now. And you can send those JSON files to others and they should be able to load into your environment. Aside from that, though, if you aren’t working with the save and export, there really isn’t much new in this build except for there’s some text markup now so you can highlight stuff, which was a request by Dini.
Leon van Kammen: I just wanted to let you know I’m listening, but I’m also
Andrew Thompson: Oh, yeah.
Leon van Kammen: You’re fine.
Andrew Thompson: Yeah, that’s that’s usually how it goes when you get the headsets on. Everyone just kind of tinkers. No problem at all. I wouldn’t be surprised if there are some little bugs that I missed with the save and export. I tried going through every use case I could come up with, but That is something to keep in mind. Oh and I’ve, I don’t have like a a visual loading is done bar obviously. So if you do load in a JSON scene give it a minute to like properly build in the background. If you try to enter the XR scene before it’s built, you just won’t get anything. If you care, you can open the console log and it will tell you when it’s done loading there. But. No, that’s not feasible for most people. For our prototype, I figured it was fine, but. Obviously long term will want some kind of notification on the screen. We got a lot of UI stuff we got to figure out.
Frode Hegland: So right now we’re in the reference section. And we haven’t even decided in a sense how to get to it. But we’re learning about both technical and kind of usability. Yeah.
Andrew Thompson: My understanding is this this might be the last build for a while. That works just in the citations. Because we kind of had like a theme for a while. We were testing interactions, using the citations as a base. Since now we’re talking more about, like, file formatting and stuff like that, I might switch focus over there, which would be kind of like a new set of builds. No longer in the citation per se. I don’t know where I’ll put us. And I’ll probably pull back a bunch of the interaction and just focus on getting whatever new functionality we decide to work on. And then slowly kind of re-implement stuff that we like and cut the things we don’t.
Frode Hegland: Yeah. So I’d just like to ask everyone here, including you, Bob, despite pretending not to be academic. For the next stage, it seems to me that reading in webXR is just. Nobody’s going to want to read a book in there, except for maybe Randall, because he’s experimenting, right? The reason to go into webXR is to use the huge canvas for interactions. So for this next stage, should we continue to try to explode the documents as an exploded technical diagram? Yes, Mark, I agree we should get a better term. Or should we interact with the entire library? If we interact with the entire library, then we can do exactly the stuff you have. It’s just a slightly different what is in it. Let me just remind you briefly what I mean by that. Yeah.
Andrew Thompson: Wait, let’s try to switch over to the library stuff, because we haven’t done that yet. That’s definitely something that we, you know, we want to do, and it’s worth experimenting with. It’s going to take a lot of sort of collaboration between you and me, Frode, where like, we have to get exports from your software and imports on my end. So we’ll have to talk about that stuff quite a bit. Okay. But is.
Mark Anderson: There anything you can do, given your point? You’re at the point you made rightly earlier about, you know, well, we can’t necessarily build everything in great detail. I mean, are there any shortcuts you can make just to allow us or allow, allow, allow you to show us something in a sense, in terms of the library. In other words, is there a is is there a sort of quick thing we could semi fake just to start getting a sense of what it might be? Because I think the really good point you made was we haven’t looked at this yet, and until we do look at it, we won’t know what new things we’re going to discover which turn out to be problematic in a way we hadn’t envisaged.
Andrew Thompson: I think I’m seeing at like an absolute bare basics minimum. And this is pretty easy to start with, is a library is just a very small, probably a JSON export. That is just a list of URLs that all go to compatible documents. In this case it would be the ACM, HTML exports. But if we want to start looking into what we were talking about today as having an entire HTML appended onto. The end of the PDF document as visual meta like, let’s just go for it, let’s just try it. In which case it would be a list of URLs that all have that. And we’ll just assume that a library has already been checked and every file in the library is compatible, so I’m not going to bother with that stuff. Yeah. Right. So maybe we just do that, that that’s the starting point.
Frode Hegland: What I think we should do is very similar is have a JSON that explains the library doesn’t actually it’ll have links to the documents, but that’s in a sense irrelevant. This should be interactions in the library looking at the books, so to speak. Right. So the JSON.
Andrew Thompson: I don’t need you to generate that JSON road. The list, the list of the URL things. For now. I can do that just in the system because we’re only going to have one. Yeah.
Frode Hegland: That’s good. But there’s a couple of really important buts. They include if there are any extracted entity entities from these books or documents, rather, they need to be available as part of the JSON. I’ll show you why in a second. I think you’ve seen this under just a quick walk through this stuff we’re not going to work on. Now. This is just some little cutesy idea of working on a document very visionOS style. This is basically what you have now, except yours is interactive. So here’s the thing. Click to View and library. And now we have a big graph. But the key is at the bottom. Can you see the bottom. Sorry the text thing is over on my view. Let me just go back. So the key is really this. You have a list of you can have a list. Just normal lists in here. They’re very useful sometimes. List by author, date, title, obvious stuff right? Second one is timeline, which would need a little bit of designing, but all the documents can fit on a timeline. Right. And then we have this map view. And this is where it might to have an absolutely massive map view might be useful because of these two things. Oops. So on one side here on the left, you have, when you select a book, forget what it actually says. These would be titles of documents, right? If you select one, when should you see connection should be based on comments, highlights, entities or what’s you know that’s something the user should be able to choose. And on the right. Sorry, guys. You’re you’re actually blocking my view. There we can show what you want to see on this. Should should it be everything or should it just be people or titles or location? Institutions, citations, annotations, whatever. I’m not saying we need to do all of this immediately, but what I am saying is that. To start having all these elements and to really work on how they can move around, even if it’s static data, I think will put us in an interesting space. Does that make sense?
Andrew Thompson: I think for like obviously that all of that is way too far out. Like we can’t start with that. But it’s good to have that in mind. I think as a minimum, we’ll just have the library be just the names of all the documents in the library. Like that. Simple. And then if we want to go the next step, we would have maybe the lines drawer between documents if they cite each other. That seems like a simple way to start. We can of course add more functionality that later, but I think the big thing we need to do right now is getting one actually seeing a library of documents, because we’ve only ever worked with one document at a time. And then two, finding some way of saving that into your visual meta and sending that back and forth. That seems to be like to me right now, the big technical hurdle. And if we just want to.
Frode Hegland: Agree.
Andrew Thompson: With everything. If for now. Sorry. Yeah. If right now we want to. I don’t know how much your suggestion was a suggestion and how much of it was like a let’s do this of taking the document and saving it as HTML inside the visual meta. It technically doesn’t have to be HTML. It just needs to be tagged. But that’s one way of tagging it. If you are committing to that, that’s the direction we can go for now. Ultimately, all I care about is just having the different sections of the PDF one easy to access the text without a bunch of like confusing glyphs in there. Which is visual meta export. That’d be great. And then also second tagged into sections where, like, everything in it is labeled like they are in the HTML file, like this is a citation. This is a citation line. This is a link in the citation line. This is an author like all of that stuff. That’s really what I need to have access to. And we can decide if that’s something that development time should go in on my end, where I find a way of, like, translating text into that. Or if we want to just keep that as a separate thing. I don’t think this has to necessarily be something that you develop a workflow for. Frode, because we have the HTML exports already for all of the ACM papers. So if you just have a way of sticking that on as visual meta, that could work. What are you seeing with this?
Frode Hegland: I. If, first of all, I know Mark’s going to bite my knees, but the visual media can be current ACM stuff just stuck in the back, so to speak. But in the and this is really important that we’re all on the same page on virtual page PDF page. Haha. The JSON thingy should contain all the information about all the documents except the documents themselves, right? So it should have any useful metadata. So if Mark highlights the word chocolate, the JSON should have this document. Mark highlighted the word chocolate. That is really, really important because the annotations that Danny really wants, and I completely agree with her, but it should also be there if you write something in the margin. That should be an element that when you’re in your library view, you should be able to either search, navigate or view by.
Mark Anderson: Is that going to I mean, let’s say so a reasonable library is what say about a thousand documents. So every annotation in all the thousand documents that’s going to be in the live library JSON file. I’m just wondering if that’s going to get a bit bit big.
Frode Hegland: No, it’s not considering video streaming and pictures and stuff. It’s going to be fine.
Mark Anderson: Well, I know I was asking Andrew really because. Well, exactly. I mean, I’m not trying to I’m not trying to be difficult, but genuinely, I’ve made this mistake myself before. Just said, well, just stick it out on the wire, you know it, or I just don’t know how big it feels to me. I’m very conscious of this hand up, too, but it feels to me that we well, the thing we need to experiment with so we can see if that it’s going to cause a constraint. If it doesn’t, great. If it does, obviously we’ll have to be we’ll have to think of a different angle of attack.
Andrew Thompson: Yeah. I mean, if we’re just talking, which I think this was about the extra data in the library. Jason, I’m not worried about that. You’re not sticking the entire document in there. Luckily, this is just like a few extra lines of JSON, which is essentially negligible. That’s tiny. We start to get into issues when we start saving large amounts of content. This is just like some tags. We are going to get quite a bit of file bloat sticking the entire document in HTML at the end of the document. Right. Like that’s definitely going to be bloat. Is it enough to be an issue? I’m not sure. It seems a little bit counterintuitive to me right now, because if we’re using the HTML exports, why aren’t we just using the HTML exports? But I understand, like, oh, I want to have the visual meta in there as like a component.
Frode Hegland: No no no no no. This is why I’m saying that visual meta is basically introduction and data. And this does not have to go on the back of a document. Okay. Here’s a really simple specific flow. Mark Anderson is reading a paper that I wrote and has the word chocolate cake in it. He highlights it. And then he also writes in the document somewhere. This is an interesting recipe. End the visual matter at the back of the document, a new appendix will have. Will be called highlights, and any highlights from the document will be listed and another page will have. These are any stuff that he has written. These two things are currently not in the visual meta in reader, but reader has it in a database. So that means that when I search my library for the word chocolate, I can see that Mark has highlighted chocolate and I can open the document, which is useful to me. So when it comes to reader exporting, the JSON, it will export in one of the documents, will have all the citations stuff and blah blah blah. Then we’ll have a field highlighted text, a field annotated annotations just as text in that. And that is a pure normal JSON stream. Nothing to do with visual meta. Visual meta is just a notion of having it there when it comes to the JSON stream.
Andrew Thompson: Yeah, I think I think I’m on board with that. I temporarily switched back to talking about the entire export stuck on the end as visual meta. Which I’m just commenting on that part, which because that’s a lot we have to figure out still. Yeah.
Frode Hegland: I’m just going to. Share with you.
Andrew Thompson: I’m already ten minutes over. I have got to go. But I don’t want to leave mid discussion. What specific route do you want me to dive into right now?
Frode Hegland: Okay, on our webcam website we have interactions. And then there we have use case. That is what we’re building. Under here we have reading, which is where I’m working with the whole Dean things to make sure it’s correct. But now under library, this is where I think we all agreed we should work on now.
Andrew Thompson: Okay, so start with library. Yeah. And that’s all I need.
Frode Hegland: Yeah. And I’m going to talk to the guys here so you can look at it later. Feel free to go. Very good to see you. Thank you for staying late. And have a good Anyway, I hope to see you on Monday, just later on the day. Bye. Phew! Now that the adults are gone, we can talk freely.
Mark Anderson: I’ve, by the way, I got the thing working. I got these turned off in the demo, so.
Frode Hegland: Leon, please, please please please please please please, please.
Leon van Kammen: Yeah, I just I just wanted to say that about this demo and about the next steps. I really like this There is this button on the interface. If you press it, it will basically flip. You know what I mean? Basically the the. Yeah. Now you see a debug button or something. But in the I don’t know, like if this was already programed, then we can assume that it’s easy to repeat it. In the, you know, keeping in mind the repetition mantra of user interface design, maybe you could and now I’m referring to your prototype with the things sliding out on the side and your your use cases. Maybe it might be an interesting sort of separation to also introduce some kind of flip mechanism there where you basically would flip from a micro, let’s say the document is a micro view with perhaps some use cases or things sliding out. But if you would flip it, you could flip to the macro, the macro view of things, which means a library view or a perhaps this the meta view where, where you see the wires that, that screen you showed.
Leon van Kammen: Yeah, I was just thinking about that because
Frode Hegland: From.
Leon van Kammen: A practicality and pragmatist point of view, I think it’s very easy to implement. And in the second and the second reason is that if you want to if you want to move fast with the implementations, then maybe it’s first, it’s easy to focus on replacing the current document from if you click something in the library and then not not really caring yet about, you know, having all kinds of documents sort of instancing on the side at whatever if you can get that. Right. So basically the library view and opening a new document or replacing the current document and somehow getting this metadata saved. That would already be a huge step, I think if you would immediately go all in like, yeah, I want to have multiple documents that might be a bit too much.
Frode Hegland: To.
Leon van Kammen: Handle for a one person developer. That’s just some some thoughts I had. I don’t know how fast you want to move into multiple documents.
Brandel Zachernuk: But
Frode Hegland: Well, here’s the thing. In breeding and native on the Vision Pro was much higher resolution than WebEx, unfortunately. So that means that from my reader app, I’m trying to fulfill some of the requirements there so we don’t have to do them in webXR because reader does simpler stuff, but it does it cleanly. So that’s why it’s so important to be able to be in one environment, click a button and then you go to another. And that’s why I think that the whole library thing is really, really important. By the way, both of you are coming up for a future of tech social in June, right?
Leon van Kammen: I am discussing this with the boss here in the house.
Mark Anderson: Likewise. I don’t know my June schedule at the moment, but Possibly. Yeah.
Frode Hegland: Yeah, I hope I can And then. And then then you can play with this.
Leon van Kammen: Oh, that’s.
Frode Hegland: A good thing.
Mark Anderson: I sorry, I just lost the needle on the thing of the library. Do we actually have any. Recent sort of actual sort of data or observation work on people using their libraries in the loose sense of the word. Because the funny thing is, I find whenever I talk to a fellow academics about their library and one starts to look at their shoes because it’s like that thing you never really talk about because everyone’s worried that theirs is messier than you know. It’s that kind of thing. People don’t like to talk about referencing and stuff because it’s just something, something nobody wants to do. And I guess it’s like the teenage sex joke, you know, everyone else is doing it better. In fact, no one’s doing it at all. And so I, I sort of, I sort of wonder because I’m, you know, I was trying to think, okay, so what how do how do I actually interact with my library of documents and why do I do it? And to the extent to when I do it deliberately because I want that I want something that I wouldn’t get by just clicking a link in the document. I’m trying what I’m trying to do is get my my hands on the edge of what the real metadata needs to be in there, because it’s terribly easy to imagine in the mind’s eye a nice interaction that happens. That sounds really good, and then it turns out to actually be really good, but it doesn’t actually do anything useful. It does what it set out to do. But actually, that wasn’t really you realized that wasn’t actually what you wanted to do.
Frode Hegland: Leon, please. I’m trying to share something with you guys. Yeah, I’m.
Leon van Kammen: Thinking about that. You know what? If the cold, harsh truth is that most academics library is just a Dropbox or a G drive. Is that a cold truth?
Frode Hegland: Or
Leon van Kammen: Is that what most people.
Mark Anderson: Yeah, I I’m, I’m not going to say what you know. No, I think I think it is it’s one of the things I mean, I remember trying to get the doctoral college I was working for and said, well, you know, to actually ask all the people that they were funding to do, you know, degrees, what they actually did to do their referencing. And so, of course, you have 1 or 2 people you knew did really good stuff because you saw them using it all the time, and everybody else was sort of looking out the window and. And you know, I don’t know, probably just typing it straight into word or something I don’t know. It’s, it’s, it’s difficult. Which, which, which adds to the problem of trying to work out. Well, what would you want to do? It’s like I find it really interesting. Lots of people have said to me with the hypertext set, oh it would be really useful if we had all the organizations in there. And I said, well, why? And I said, well, I could look at the organizations. Why? And. Well, because it would be really useful. Y and it turns out that actually isn’t very useful because people move around. You have you have a you have an author with seven people. And so so our notion we have again, we have the simplistic notion that, that, you know although 1 or 2 people may work somewhere their entire career, that’s actually quite rare. So place and affiliation are actually far less useful than you’d imagine.
Frode Hegland: Mark, could you guys see my screen?
Mark Anderson: Yes we can.
Frode Hegland: Yeah yeah yeah I see. No no no no, not my desktop. I’m going to try to I don’t. I’m going to try to share my view here. I think it’s this one. Because it goes. My screen goes black while this happens. Okay.
Speaker8: Yes.
Frode Hegland: It’s trying to.
Mark Anderson: Control screen goes goes goes black just as you’re trying to use it.
Frode Hegland: It’s working. Leon, please go ahead and please tell me if you will never change. Okay. Right now you see it, right?
Speaker8: Yep. Yeah.
Leon van Kammen: Yes, yes, we’re seeing it.
Frode Hegland: Okay, so this is This is what I’ve done. Look at this.
Speaker8: To go.
Leon van Kammen: That is very cool. This is multiple pages or.
Speaker8: Yeah. It’s a it’s multiple pages.
Frode Hegland: I’m just move it so I don’t have this screen. So to me this is really sharp reading I can read this super comfortable.
Speaker8: The thing is.
Frode Hegland: It couldn’t fit in the space and we can’t easily cover with this. So that’s why we just made it like this. And that’s why I asked them to try to get the visual matter, to just not show it. And with that knowledge, we should have an easy way to say that. By the way, can you see my hand?
Mark Anderson: Like we can. Yes we can.
Speaker8: Yeah. It’s cool.
Frode Hegland: There’s references. If that’s indicated, even by the end user, that should be able to disappear from this view, but have it separate so you can do the thing I was talking about to come across a reference. You tap on it and you see here what it is. All these things are possible easily here.
Mark Anderson: When you open that reference, what are you going to do with the the the paper?
Speaker8: Now I’m saying it.
Frode Hegland: Would be separate, like you would have. Maybe the paper here and.
Mark Anderson: Yeah. No, I just wonder the question was genuine and open ended in the sense that it’s something else. I’ve been trying to sort of take more of a note of, because I’m aware that actually that looking at another paper itself is a whole actually grab bag. About the only common factor is it involves opening another, opening another document. But actually what one’s doing, it’s interesting because a lot of the time you probably don’t. If you had some more information stored, you probably wouldn’t need to open the document. So I mean, an interesting one is often you’re opening the document just so you can get the visual recognition of the cover page, say, oh, that’s the one I thought it was. Yes. You know, I can tell by the typeface it’s 20 years old. That’s the one which is telling you probably that your memory isn’t as great as it used to be, but it’ll be interesting. I just I just don’t know. I haven’t seen anyone who’s done any that, you know, someone who’s done a sort of study of what people are actually doing there because it’s really quite pertinent to this. And we can we can make we’re imagining we are in the process of making all sorts of cool interactions where we basically traverse from document to document, based on the fact that we know they’re linked by citation. The next thing is to get into the point of, okay, but what were they there? I mean, the truth is most people cite really badly. So you follow the citation. Oh, great. And now you’ve got a 300 page book from which some words may not have come. So there’s, there’s the again, there’s some interesting stuff to drill down to in this.
Speaker8: But yeah.
Frode Hegland: I think definitely we need to Fantasized that this is the most incredible environment the world can ever see. You know, we’re inventing that. So when it comes to even the metadata, I think we can really say. If people follow our way of doing it, this is the benefits they get, right? This is the right thing in this space. Look at the size of this thing. Can, of course, make it bigger.
Mark Anderson: Yeah. No, I was I was pushing down on the fact that it’s slightly more granular to say that if you’re going to do this you will get a benefit and you will get the benefit if you actually bother to, to site more closely. Trouble is, I don’t think I’m not sure a lot of people would really like that because they may or may not have read the document they’re citing.
Leon van Kammen: Oh, I know another killer feature. Throat for author. Okay. I would love. When I saw that six page article, I was thinking like, oh, I’ve always wanted to have a sort of to enforce to the user that they, they read all these six pages sort of next to each other because they’re really about they’re referring to each other. I would love to build a PDF, which somehow allows me to specify that it’s supposed to be rendered six pages at a at a time. So if you do next page, then you will get the the, the next six pages. In other words, I would love to write a book which is designed to be read for six pages or a technical article with infographics, where on page one and two I have text three and for the infographic and on the last two pages some more text so that you can make a complete basically a spatial document. It’s 2D, but it’s. That would be really epic. And I think you already got the viewer part down. That was a reader, right?
Frode Hegland: So I will say the same thing to both of you in response to that. Let’s just do it. I mean, if we in September have a version of the article that we’re submitting for something else, and it’s written in a different way, presented in a different way, using different metadata, as long as it’s compelling when someone puts the headset on. You know, that’s a sales pitch. If we do it kind of like halfway. You know, then it’s just me. So, yeah, I think that’s a very good idea. From what I understand that you’re saying is very much. But Anderson Mark Anderson style of. You know, be slightly aware of what you’re writing for, what kind of medium, what kind of substrate, what kind of display. So if you can write it in a more clearly contextualized manner, which is what ACM documents tried to do, you know, there’s an abstract title and all of that, right, that we can absolutely do that. And, you know, I so desperately want to be able to have 3D things in documents that will appear traditionally fine, because one of our key things here, this has to somehow be savable in a traditional and then exploding in there. Yeah. And that doesn’t mean obviously there has to be a technical thing. It can be exactly what you’re saying it can be. Here are the different things. View it in this way. That was one of our kind of earliest discussions here. And I think we have confused our way of.
Mark Anderson: The exploding thing is, is, in a sense, just another riff on the idea of something knowing what it is. Its ability to deconstruct is based on the fact that it knows it’s not one monolithic thing. And obviously we’ve got to explore what that unpicks as. I mean, we know some headline things, but but, but that’s, you know, the first couple come for free sort of thing. But actually digging into the to to what is it, what does it mean to give us the richness we want? I suspect it’s it will be fun. But I was thinking when when you were saying about it sort of like having a six up page as if you’re working from a start, like a PDF, which is inherently basically has already been printed into page size blocks. You have that, you have blocks of a known size. So if you want to tessellate six of them together, then a display space should find that I imagine fairly easy to do because it’s not like the pages. It’s not like the pages are all different lengths, for instance, or different widths. They’re all going to be consistent sizes. So if you just wanted to, you know, have have three columns or two sort of thing your display space needs to say give me six pages and put them in in the six slots in this order.
Speaker8: This is why we.
Frode Hegland: Have two hour discussions, because now we’re 2.5 hours and we’re really digging.
Speaker8: Deep.
Frode Hegland: Right? I mean, you should be able to offer a document and do something as simple as this. This is a six page document. And then in XR you open your own document, put it in certain thing, and then you have an ML over there with some extra intelligence commenting on it. You have this and that. And then when you send the document you send a PDF. But in the PDF it states this is a expanded document. If you open it up in an XR environment, these other resources will be pulled in and this display will be shown. This is so basic and crucial. You know, this is we have to decide on what use cases to support for these kinds of interactions.
Mark Anderson: Well we have three use cases. I mean that are we saying we don’t have enough use cases or I thought we had three. And is the problem that the three we have aren’t written out enough? This is where I’m a bit confused.
Frode Hegland: The use cases we have, I think obviously Dini is completely right. She is the academic in the group, so to speak. Senior. But to me it was more research questions. I’ll show you. You know, I was preparing to write, editing and assessing a graduate seminar paper. I think this one is a bit niche for only professors.
Mark Anderson: Well, you funny enough, but it’s probably the most interesting because it’s it takes you down the rabbit hole. All the things you’ve talked about, like stuff you did with, talked to les about like where, you know, where where are the citations turning up. Because what, what you’re needing to do in that is so that the use case three is effectively doing is almost like an audit?
Speaker8: I agree.
Frode Hegland: It’s deep reading.
Mark Anderson: Sorry. No, it’s not deep reading. Actually, funnily enough.
Frode Hegland: Sorry, I didn’t mean deep reading like that. I just mean it is proper reading, you know, to comment back either for anyway.
Speaker8: Well, what’s.
Mark Anderson: What’s interesting in that one is in a sense that that one of the things when you’re doing that is you are actually in your mind trying to explain the wretched document to make it tractable to yourself. So actually, that’s really interesting. The second one is, is a simulation where, where I that to me is more pulling the bits together and doing the assemblage of these, these parts that you’re going to that will fit into the box. And I the first one I think was just, was just I missed it. It’s gone now. But a general reading the question is how many use cases have we got? How do we actually have the time and the resources for.
Speaker8: What?
Mark Anderson: Well, you know, there’s only so much time and there’s a limited amount of resources, certainly in terms of, you know, the programing.
Speaker8: This.
Frode Hegland: Is what you were talking about a little bit. Reimplement the document that’s
Speaker8: Sorry.
Frode Hegland: Who said that?
Mark Anderson: Yeah, I think that’s. Yes. So when you were showing sorry, when you were showing me the the use cases So, for instance, you know, actually looking at your thesis at the moment, trying to one of the things that it’s interesting. So I’m reading a quite a richly structured document whose structure I don’t yet get. So I don’t know where this thing was explained or if it’s going to be I don’t know, because I’m the reader and it’s all new to me. And this is and so the questions I have are not at this point in any form of a judgment on the document. They’re they are a sort of a list of things, I don’t know, which I’m trying to ascertain as part of the understanding, just just to read it. So I. That was sort of in my mind as you talk about the use case three, which is probably the most interesting of the three because it. It causes us to want to think about. What elements of the document do we want to get our hands on?
Mark Anderson: And explain to ourselves why? Because it’s normally the writing down of it. This goes back to my question about people say they want to have, you know, organizational information, but when you ask them why, normally they don’t come up with an answer. So and as I write a lot of documentation, I’ve also learned that writing it down generally teaches me I understood less than I thought I did about it. So that’s sort of what’s what’s in my mind’s eye in terms of trying to flesh out the use cases, whether we need more. Because we’ve got to watch how the work scales.
Leon van Kammen: But I think maybe the editing part might be it. We might benefit from limiting that to annotating and highlighting and notes instead of editing the document. Themselves, you’re not going to have fairness.
Mark Anderson: That’s what.
Speaker8: Was.
Leon van Kammen: Going to edit a hamlet. Hamlet document. I’m not going to edit that one.
Speaker8: No, no.
Mark Anderson: I think what Deeney was referring to was. Acting in an editor’s role. In other words, people are sending me stuff for approval. So I’m in a sense I it’s a it’s probably also akin to doing peer review of papers. So someone sends you something and you’re just in a sense going through does it make sense? Does it go here? And is it is it all there? Is it full of typos? So arguably you could say it’s a bit like number two and number three. It’s just that I think the interesting thing in the third point is that if you’re looking at more than looking at paragraph two and did it meet the rubric and do I give them three marks or four. So oh that’s interesting. Well, why do you know, why are all the references in the first section or being able to do something like I’m going to expect I’m going to expect the following things which I have in my library. Do they, you know, cross match. They there all sorts of interesting things that you can begin to do. No one will ask for them now because they can’t do them and they probably can’t imagine them. You know, because it wouldn’t, they wouldn’t sort of think to do that. And they will they will explain to you seriously why you’re wrong and that you should do it the old way. But I do think once, once there’s more metadata you know, some of this sort of comparison stuff that we, we want to do and either don’t do or do with difficulty will become will become easier, which actually frees up time to actually interact with the the story, as it were.
Frode Hegland: Let’s have a look at this and tell me if you agree. So hang on let me just lift it up. There we go.
Mark Anderson: Why are they deciding to put on their headset? Why don’t they just put it on? It’s sort of.
Speaker8: It’s the.
Mark Anderson: Extra. I find the extra words confusing because you’re doing it. You’re doing something to get to the place we’re going that actually isn’t relevant to the end point. Why not just say an academic is working and puts on their headset?
Speaker8: I think it’s we.
Mark Anderson: Don’t care why they’re going to do some work, it’s just clearer. It’s much crisper and clearer. Otherwise. Otherwise you get into lots of extra parenthetical clauses and the sentences are really hard to read and it’s not clear what you’re actually doing. So we could make we could cover that almost in one sentence.
Speaker8: Okay.
Frode Hegland: The point is to show that you do different words and different systems. Right. That’s really kind of I would.
Mark Anderson: I would take those out and list them. I can tell you, because I’ve been reading a lot of sentences like this. They’re really hard to read. They’re tiring to read, and they’re not clear if you if you want to have a list, make a list because that’s essentially what you’re doing and you will be much clearer to the reader. So we’re going to do this. We’re doing this work. And the reason is. Bum bum bum bum. You make a bulleted list. You can make another list if you want, but then then the then the things being listed are much, much clearer. So you’ve got about four things you want to do in there. So they’re going to put on their headset and they want to interact with their documents in a more extended environment. I don’t know what that means. They you.
Speaker8: Mean you don’t know.
Frode Hegland: What it means?
Mark Anderson: A more extended environment, right. So it. Yeah, but that’s you’re assuming that somebody I know because I spent time talking with you about this, but I’m just trying to say that. Sorry. I spend a lot of time writing for, you know, for for people who, you know exactly like myself, people who ought to be bright enough to understand but don’t. And I know because they write and they tell me in no uncertain terms that something I thought was was plain. So I’m not I’m not trying to be clever here. I’m, I’m reflecting something that I lived through all the time. And what it’s taught me is to write less and more succinct and, and and so less long texts, more structure. People can read it much more easily.
Leon van Kammen: Well, it’s the reader of this document, academic.
Mark Anderson: It’s anybody, and it’s probably people who know less about XR than we do, which is the key.
Leon van Kammen: Because it’s it’s a bit of a Yeah. So these these words fraud is using does make me excited. And so you know I guess. Yeah, maybe. Yeah, I, I totally see your point. Like, if I write a spec, then I also have to strip it from all kinds of exciting words or concepts. But then it’s incredibly not exciting to read. So, yeah, I guess.
Speaker8: I take the.
Mark Anderson: Point, but I, I, I also sort of having experimented with some of this is the thing is that.
Speaker8: It’s.
Mark Anderson: Excitement, excitement, people, you know, if that’s what people are coming for, they’re coming for excitement. They will. But, but, but but the excitement is easier when you have some idea of what’s going on. And I’m really what I’m thinking is, if I’m the person who made the grant, reading the report and thinking, did I get my money’s worth? I don’t actually want lots of hyperbole.
Frode Hegland: But this is not for that. This is for us.
Speaker8: This is.
Frode Hegland: We are the target users here. This is not for anyone else, right?
Speaker8: So what we need to understand.
Mark Anderson: We need to understand what we’re doing. I’m excited already. I don’t need to get more excited. I need to understand what I’m doing.
Speaker8: Okay, Mark.
Frode Hegland: This is not poetry. All I’m saying is that there’s a difference between working on a traditional display and working in an XR headset. That’s all covered in the first sentence. The reason that we are building the software is to use that space, because that’s because the limitation of webXR, if you want to work on a small document, you can do that on the vision natively because it’s crisp and clear, right? So the use case that we’re supporting here is to try to use the space as Anders developing it. And here it’s just saying they go to a web page and they have their own library. You know it’s really not that important. It’s just for our own clarity. What we need to agree on are these two things here. What are the sorts of things that should be able to do in there? Because that’s this is going to be our Bible.
Mark Anderson: Yeah. So in that second thing, the second two lines weren’t breaking out into discrete things. So you can see what they are instead of also this and then that and then something else. Because it’s much clearer to the reader. Who’s reading it for the first time. What it means it’s very easy to write a sentence and just stick. Keep putting commas in and ands, but they they are very unreadable.
Frode Hegland: Okay, I’m going to.
Speaker8: So if you.
Mark Anderson: If you where, if you take.
Speaker8: That second. No no no hang on.
Frode Hegland: I will do that. But hang on first I want to make sure we make Deenie happy. That’s obviously one of our jobs. Right. So I’m just going to copy her research things. And then I’m going to go back to that page and the use case. I’m going to paste it at the bottom of the page for our reference. Right. Bang bang. Right. Okay. So where are we talking while in the library above.
Mark Anderson: So in the, in that, in that second paragraph everywhere you put a comma select it and hit return. So turn it into a new line and you’ll begin to see what you’re actually writing about. So every and and every comma make a line break just so you understand. So you can see because there’s actually quite a lot in there. So we’ve got an author entitled A timeline, a Citation tree. And based on and based on other attributes of the document. Which doesn’t actually mean anything except including those listed that, you know, I don’t understand what that means.
Speaker8: You know what.
Frode Hegland: List? Don’t be pedantic. No, no.
Mark Anderson: I know, I’m sorry. I’m. Look, I don’t when I say to you, I don’t understand. I’m really not. I’m actually being totally honest. I’m trying to help. I’m trying to say that it doesn’t read on the page as it does in your mind. So. So we’re going to get we want to see their documents. But it’s like throwing everything in the wall. It’s.
Frode Hegland: I’m just while we’re working on this, I’m just this is to enable the user to a bit so that we connect the research questions to.
Speaker8: Sure. Yeah.
Frode Hegland: These two next ones are less than the library and more while reading, right?
Speaker8: You see when you.
Mark Anderson: Say based on author and title, but also in the timeline. So I think what you’re saying is you want to say you want to see them basically listed in some form based on author or by or by title. And I mean, the point is, but but also in the timeline, I think what you’re actually saying and it will be useful to have a timeline view as well, but that’s not actually how it reads on the page, is what I’m trying.
Frode Hegland: Not fine. Like we can do that now. Well then library, you should be able to show to choose how their documents. Are listed based on author and type. Well, I’d.
Mark Anderson: Say listed and then. Well, author and title is a type of list, I presume, or whether they’re separate. This is what happens when when it all agglomerated, it turns out it’s just less clear. So. The timeline is also a different sort of display. So. So by option, we’re already saying that we assume the default will be author and title. That’s what I read.
Frode Hegland: That’s fine.
Mark Anderson: No. Is that. Is that what you meant? I’m just saying. That’s what I read. From what? That’s what I understand from what’s written. If that isn’t, then, then I. Then let’s make it as it should be. So there’s nothing wrong with the option. I’m just trying to get to the point. So what you’re saying is they choose how to see.
Speaker8: That it’s.
Frode Hegland: Arbitrary. Whatever they had lost could be the one they’ve used. So I can happy to change it to. It should also be okay.
Mark Anderson: Right. So what we’re actually getting we’re getting into it. So now basically why don’t we first of all just list the types of things that we think without deciding which is what, just make a little list rather than try and put it into a into a sentence where it obscures the meaning. So when you get are listed based on colon new line and then and then just just humor me. Just make a list of those things and we can maybe mash it back into a sentence when we’ve worked out what we want in the list and what the constituents are, because otherwise it’s not clear.
Speaker8: I.
Frode Hegland: It’s just we have a lot of lists on the sites.
Mark Anderson: You know, we can we can deal with that later. I don’t worry about the formatting. What I really meant was just put a line break in so that we can see what are the what what what are the discrete things that we’re thinking of. So we want So we basically got four view specs. And I’m not suggesting the words we use, but what I take from this is, is we’re imagining there being probably four discrete view specs, all of which can probably be configurable.
Frode Hegland: But by the way of note, Disney doesn’t really like us referring to Doug too much, but she thinks it’s too much. I haven’t referred to him in months, but she brought it up recently. So Doug language with her. We should be, but that’s fine.
Mark Anderson: But we’re not going to write that on the page. I’m just.
Speaker8: Trying to agree.
Frode Hegland: With you. It is essentially a view spec.
Mark Anderson: So all right displays layouts. Call them what you will. So now we so. That that basically I mean there’s always scope for others and things, but but again, the danger is overloading it and then putting it so on so and so on at the end. So we think essentially that we, we can envisage at least four different display types. So you can now basically roll that up after the colon. You could, you can now graph based on. But based on other attributes of the document, doesn’t really tell me anything. If you mean a graph or map based on, you know, based on metadata in the documents, that’s enough to describe what you’re describing. I just think it’s shorter. Yeah. Now you’re making it longer. Now you’re adding in more confusing stuff that the reader has to understand. You had it. You know, you just need to say do you want to? Do you want to draw a distinction between a map and a graph? Is that even helpful? Why? Or you could just say, say a map graph is more technical. So I would say map. More people will understand it. And you can just say a map based on other attributes of the document. End of sentence. The rest of it is not needed to inform someone as to the use case. And then. Then you’ve got for now, you’ve got four short phrases that you can come and join. So you can if you remove the bullets. So after the colon.
Speaker8: The bullets are fine.
Frode Hegland: They really are.
Mark Anderson: Okay, fine.
Frode Hegland: Otherwise, in WordPress it goes all funny space or.
Mark Anderson: Yeah. Well, rather, you know, I hate having to use a WordPress site at the moment. Is it? So. So the second one is. So you should be able to toggle between these views and save them.
Speaker8: Yep. We.
Mark Anderson: Probably don’t need to put so many also’s in, it becomes a tick. It’s self-evident that if they need to be done, they need to be done. So the also is redundant.
Speaker8: The.
Mark Anderson: So Leon, what do you think? Because I’m conscious that you’re reading it in a second language. Which is another point about clarity.
Leon van Kammen: Well, I agree with the. Well, so far, everything is clear. I find it quite an exercise to pretend is as if I’m not part of this group and read it. I do agree that I like the sort of trimming the fat what has been done so far? I it does take me some extra. Mind power. I the limited mind power I have. I need to focus on one sentence. If there is a summary afterwards, which is sort of extending on it, that makes it a bit harder to to read. So I guess that’s what you meant with the comma, comma, comma, comma stuff.
Speaker8: Yeah.
Mark Anderson: And it’s the thing is it’s not it’s terrible because it always feels it feels like a sort of a critique of style. And that’s not really the issue. It’s it’s.
Speaker8: What what.
Mark Anderson: The core. The core thing is almost to write down the shortest possible description, and then you can turn it into sort of nicer prose. The temptation is in, in trying to cover all the things. Which we tend to know. We know the edge is it goes on the page. And actually it doesn’t need to because it’s as you get into, as you get into the scenario, it’s sort of evident if we need to have a manifest of particular things we could write a, we could write a sort of effectively turn the use case into more of a spec at a, at a time when that seems pertinent. But I think in just terms of, of us being able to quickly show people broadly, what we’re about. Because there’s only a limited amount that people read before they get bored, unless they’re interested and we don’t know if they’re interested.
Speaker8: Mark, this.
Frode Hegland: Is not for other people. This is for us. It’s just a list of things to refer to while developing.
Leon van Kammen: So. So maybe we should also be careful to not Like, I really think that Mark has Actually, I wish to hide. I wish I could hire Mark on a daily basis to do this with me, but I. Yeah, I guess we should also decide when should we have Mark really go full in and which texts are not as important? Or. Yeah, I’m not really sure what the criteria is, I just.
Mark Anderson: Or maybe it’s just I think I’m to be fair, I don’t think they’re unimportant. I think it’s the best way I can describe it is it’s just how tiring do you want it to be for the reader who doesn’t know this? And it’s terribly and it’s terribly difficult to do it for your own. I find it incredibly difficult to do for my own writing. I rely on, on a small number of people who I know will be absolutely red in tooth and claw. And they’re normally. Right, because it’s just it’s like doing your own maths homework. You keep making the same mistake on, on, on line one, and there’s no real error in it. It’s it’s because, you know, the words that you wrote down, they broadly make sense.
Leon van Kammen: Yeah, yeah, I think that’s totally true. And I think we have now.
Speaker8: So
Frode Hegland: We’ve written.
Leon van Kammen: It. We’ve reviewed it. Is this what do you think fraud is is now something which is more or less finished or.
Frode Hegland: Do you want more review? I think it’s fascinating how as you can see right now, it can’t be at the XR guy. When I asked him, is there anything to add to the list? He sent me this massive amount of text, which I’m grateful for, but it’s basically wasted because it’s just in a Twitter chat. But I’m going to turn it into an article now. I’m going to send it to him and say, what do you think? You know? You know, we should all think about it. But it’s a ridiculous way to communicate, right? So I think I think we’re basically done. So this is our website now. So research questions what Dini said. And then it says these questions should be addressed by the use case. Click on the use case reading while in the library should be able to do this while reading a document should be able to do that right. This is what you should be able to do currently if I go back out. And then back out. Sorry. And then user stories. These are nice things, right? I should be able to use my hands. I should be able to do this and that. This is where I’d like to keep adding as we think of things to their.
Leon van Kammen: Nice and short as well.
Frode Hegland: Yeah. And it’s to make the it’s to make the use case more special.
Leon van Kammen: Actually, I think the two of you are kind of like a golden duo, because I think the moment fraught is is writing wrote. Fraught can really write in an exciting way. And when he goes too far, then Mark Anderson jump jumps into the ring. And I think that that’s a really, really good combination.
Frode Hegland: That makes sense. Yeah. So I’m going to work a little bit more on this, disconnecting these things and getting these other people’s Kind of comments in here like Now let’s see this. This could very easily become an article, you know. So anyway, I’ll send some kind of result tomorrow morning.
Mark Anderson: Okay, I’m going to put the something very quickly that I explain why I’m going to put it in in the I mentioned the chat. I’ll put it in the, in the, in the slack is someone demonstrating a system called Dash, which has been made by Andy Van Dam and colleagues at Brown University since about 2015. There’s annoyingly there’s really little published about it. But it I think it’s being bankrolled by Microsoft and Adobe as far as I can work out. And the, the demo, there’s about an hour recording. It was a sort of tinderbox community meetup, but one of the guys who works on the project was showing it. So and it’s my sense is that it’s a bit like it’s, it’s a, it’s a, it’s a solution looking for a problem. But what was interesting is I found myself thinking of the conversation we had a while back and somebody it might have been Peter had suggested it would be useful if you had little affordances around something showing what they could do. And when I was watching the Dash demo, there was a lot of that going on, and I found it far less intrusive than I thought I would. So I made note to self, you know, don’t don’t be prematurely prejudiced about this as an approach. And I thought it’s there because it’s just, it’s something being used on a running system.
Mark Anderson: I don’t care to say. I mean, I don’t know what the system is really for, to be honest. It’s supposedly a sort of broadly web based hypertextual system. And I think mainly what they do is they get they constantly get new classes of students in and say. Put all your stuff here and they do. The problem is that with doing that is they’re never really trying to do any synthesis, which is where the hard part comes in when you’re doing sort of tools for thought and communication. It’s one thing to throw everything on the wall. It’s it’s another to turn it into something that someone can understand. But I don’t say that. I absolutely don’t see that to be harsh. I think it’s very interesting, and it’s been done by people with real sort of chops in this sort of broad domain. So I thought it might be interesting for that reason anyway. So I’ll, I’ll put that in. I’ll put a link in the slack. As I say, it’s just a recording of another meeting. You can probably run it at high speed and stop when you see something interesting. I don’t think you’re going to go away feeling massively enlightened about what Dash is, but I just thought the sort of the interface and the interaction was interesting. Oh, and it’s 2D.
Frode Hegland: I saw a YouTube I need to share with you before we go. You just reminded me. No, it’s not trance music, although I’ll be happy to share that too. Andy Clark.
Speaker8: Okay.
Frode Hegland: Just want to make sure it’s shared properly. We all basically know what he’s talking about, but I was listening to it last night and it was really. Yeah, very very good. Leon, your hand is still up. Hello?
Leon van Kammen: Yeah, yeah. I wanted to say something. How crazy is it for the. For the next future of textbook. To. That’s for example, I and somebody else together create a sort of like six page block in the book that the moment you open it in the reader, that you get this sort of almost like a presentation worth of, of the whole topic, what is being discussed as a sort of like a. And and that basically reader detects it from the PDF that that is supposed to be rendered as a six pager. How awesome would that be? Because that would create a completely different collaboration than just oh, can you write a page for this? Can you write a page? No, it’s more like you’re you’re building a some kind of experience with with another writer. I’m sorry. I cannot let go of this this thing you you showed me. It’s a very inspiring, almost disruptive new take at PowerPoint presentations and vertical scrolling websites. It’s a sort of is a very interesting new use case of PDF.
Speaker8: Mark.
Frode Hegland: Answer that question for me, please, sir.
Speaker8: What?
Leon van Kammen: Oh, it’s not a question. It’s a it’s like more rhetorical.
Frode Hegland: And the question was how cool would that be? And the whole that’s the whole point of doing this stuff. Yeah, it would be super cool. You know that. It’s that simple. I’m just saving this.
Leon van Kammen: I think I think is there going to be a new future of textbook?
Speaker8: Yes.
Frode Hegland: And we will.
Leon van Kammen: Then I will.
Frode Hegland: I’m supposed to.
Leon van Kammen: I will.
Speaker8: Say.
Leon van Kammen: Okay, because then I am going to basically commit to writing somehow a six pager, I either alone or together with somebody else. And then I will also submit a feature request to author or visual meta to basically.
Speaker8: For.
Leon van Kammen: Something in visual media where I can say at this page render six pages.
Mark Anderson: So your page will have to be a folder then. So when we get to your folder in just six pages.
Speaker8: I mean.
Frode Hegland: The truth of that is that that is absolutely my dream.
Speaker8: Yeah.
Frode Hegland: Right. And it is I think it is very, very important to do that.
Speaker8: Very, very.
Frode Hegland: Important to do that.
Leon van Kammen: But because for, for me, something like that makes it much more exciting to write something. Because of the final projection, I would then I would I would actually tweet it. I would tweet this screen with the look what I made and how you can check it in in XR, I, I don’t really see people screenshotting their one page of PDF because it has been done. Yeah, it’s nothing new under the sun, I guess. You can you also cannot really screenshot a PowerPoint or some kind of keynote because it’s only only one page. And with this you can really have like, bam. Super impressive.
Speaker8: I think if you’re.
Mark Anderson: At Leon, you see, you need a paper article that had pop up windows and cuts through. So you go on down the pages. Have you seen I don’t know why I’ve got it. The book of the Book of Trees. Somebody made a book, and it’s all done with cut outs that you read the book by looking through the cutouts. It’s. I’ll find it for another meeting. Stephen falls. It was only a small a small run of books. So you won’t find it easily. It’s on my it’s on my shelf of weird hypertext book hypertextual type books.
Leon van Kammen: Oh. Sounds awesome.
Speaker8: All right.
Frode Hegland: I have to go down to the family. And thank you very much. Thank you for today. That was a long session, but I’m very, very happy. Thank you very, very, very much. I look forward to seeing you on Monday. If we can arrange the time zones, if it fits. If not I’m sure we have other means for which we can communicate.
Speaker8: And safe.
Mark Anderson: Flight. Importantly.
Speaker8: Thank you. Bye, guys. Bye bye. Okay, bye.
Chat Log:
16:03:18 From Rob Swigart : Trouble getting AV
16:04:32 From Frode Hegland : https://public.3.basecamp.com/p/LFvQ3KFmXAA2XxsmoewRs6t6
16:08:02 From Mark Anderson : Replying to “JSON_outgoing.txt”
BBEdit reports Incorrectly formed UTF-8
16:09:12 From Frode Hegland : Mozilla Hubs
16:10:29 From Leon van Kammen : what was the name of this webxr community/group?
16:10:53 From Frode Hegland : Agenda: https://public.3.basecamp.com/p/LFvQ3KFmXAA2XxsmoewRs6t6
16:11:14 From Frode Hegland : https://public.3.basecamp.com/p/LFvQ3KFmXAA2XxsmoewRs6t6
16:12:54 From Frode Hegland : https://t.co/iznYuoNeCP
16:13:00 From Frode Hegland : For the weber
16:13:04 From Frode Hegland : Web XR group
16:15:13 From Frode Hegland : I will be available though!
16:15:14 From Frode Hegland : Monday 8 & Wednesday 10
Monday 15, & Wednesday 17
5AM Japan which is 1PM pacific, 4PM NY, 9PM UK and 10PM European.
16:19:04 From Mark Anderson : Sunday, I saw Andy van Dam’s DASH (c.2015–) demo-ed to the Tinderbox community. Meet was recorded so i’ll post a link to the Slack.
16:19:16 From Brandel Zachernuk : Reacted to “Sunday, I saw Andy v…” with 🤩
16:19:23 From Leon van Kammen : @Dene Grigar my github handle is ‘coderofsalvation’ (email:leonvankammen@gmail.com)
16:19:25 From Leon van Kammen : [This is an encrypted message]
16:21:26 From Frode Hegland : https://futuretextlab.info/use-case/
16:36:55 From Brandel Zachernuk : I’m sorry, I realized that my day starts 30 mins earlier than I had initially thought – I have to get ready for some stuff. This sounds really useful and substantive, I look forward to the result!
16:52:36 From Frode Hegland : HTML is just plain text with external resources.
16:53:05 From Frode Hegland : PDF is just plain text with options to render in a specific layout initially
16:54:29 From Mark Anderson : Pertinent that the hypertext system Dee is descibing != web. …and it is pre-Web.
16:57:29 From Dene Grigar : Oh yeah we are
16:58:22 From Mark Anderson : The problem we can’t overlook is that in areas like Medicine/Physics/Chemistry there is a need for a ‘fixed’ version. Be it print-text or digitally-malleable text, it needs a reference copy.
16:58:43 From Mark Anderson : PDF can be edited as it is Postcript inside!
16:59:12 From Frode Hegland : Also fully extractable if nicely laid out
17:00:52 From Mark Anderson : Documents – Fast vs. slow!
17:00:58 From Dene Grigar : +1
17:01:53 From Frode Hegland : Manuscript vs published work is indeed very different
17:02:12 From Dene Grigar : perhaps
17:07:44 From Frode Hegland : 2) XR Software. Coding for XR will be a learning experience where knowledge gained will be made public through the Symposium and Book. Our end goal for our software development is to allow a user to put on a XR headset, access their PDF library, read and interact with documents in XR and traditional systems, and export their work in traditional and useful formats. This workflow will be possible because of the integration of software we have already developed for macOS: www.augmentedtext.info.
17:07:51 From Frode Hegland : We have started experimenting with work in XR on a basic level, with our experiences available to try with any modern XR headset. This output has already produced surprises, such as how annoying it is to visualize connections if they overlap the user’s space unless it is very easy to toggle on and off and how moving a ‘mural’ with your hand does not produce motion sickness, but moving your body virtually with a joystick does––even though the experience is visually identical: https://thefutureoftext.org/2022/09/19/vr/
17:08:33 From Frode Hegland : 3) Metadata. The goal for the Visual-Meta approach is to provide more data for the user to interact with in an open way that will encourage an ecosystem of developers and users.
17:09:16 From Frode Hegland : PDF-HTML-JSON-Visual-Meta all as is useful
17:09:36 From Mark Anderson : I *think* the use cases are fleshing out what the document elements are that relate doing meaningful things in XR. Existing formats are essentially just constraints on that process.
17:19:09 From Andrew Thompson : Very much a side note not part of the current conversation: I’ve been looking through the JSON export a bit, is “isFaforite” a typo or something deliberately different than “isFavorite”?
17:28:25 From Frode Hegland : Likely typo!
17:28:29 From Rob Swigart : I have a friend who does critical editions of 16th century texts. Index, glossary, footnotes, translations, even handwriting analysis. What is the text vs. supporting apparatus in such a document. When quoting from books (fiction in my field) the edition is important because the reference is always to page number. Now though we can find a piece of text by searching for it, no page number needed. At the moment this is not easy.
17:29:06 From Frode Hegland : Reacted to “I have a friend who …” with 👍
17:29:24 From Rob Swigart : In this case what is the “true” text of a critical edition?
17:30:26 From Mark Anderson : Replying to “In this case what is…”
This is what I want to get at. As an author, what does that mean to you?
17:33:36 From Rob Swigart : I’m interested in the authoritative text, but even that is malleable. I have an edition of The Waste Land with Pounds edits which is fascinating because of what he cut. The authority then is both texts? My books now are published on Kindle, which means edits can be made any time, especially for typos or other glitches. But lately it meant a title change…
17:34:21 From Rob Swigart : My friend did a critical edition in the 90s that was born hypertext.
17:40:46 From Leon van Kammen : https://web.dev/articles/text-fragments
17:44:55 From Leon van Kammen : https://yourwebxr.app/index.html?important.pdf#:~:text=highligthed_text
17:50:35 From Frode Hegland : https://futuretextlab.info/current-testing/
17:54:47 From Frode Hegland : Back in 2 mins
18:05:18 From Rob Swigart : I do need to go now. Good to see weeds now, means real gardening.
18:15:37 From Frode Hegland : https://futuretextlab.info/use-case/ for the lists of things BTW: https://futuretextlab.info/use-case/
18:17:58 From Frode Hegland : Coudl you see my view?
18:55:58 From Frode Hegland : https://youtu.be/A1Ghrd7NBtk?si=bn3q3J5cKYqVgCVW