CC: 11 Dece 2023

16:59:09 Hello, Adam. Hello, Denny.
16:59:12 Good morning. How’s everybody?
16:59:15 Let’s see. What I have behind me, it is not the black wall that is outside.
16:59:22 It’s beautiful.
16:59:25 Not so much morning. And this side of the planet.
16:59:27 It’s beautiful. Let me shut the door here. Hmm.
16:59:37 And there’s my friend Rob Swigert. He’s coming to see me in tomorrow. He’s coming to see me tomorrow.
16:59:46 Oh, fantastic. And I’m coming to see you second week of January. It looks like.
16:59:54 Let me excited.
16:59:57 What’s the date?
16:59:57 Yeah, it will be good.
17:00:00 You’re coming in tomorrow. He’s flying in. The week of the fifteenth. For the onboarding.
17:00:08 Okay.
17:00:07 Yeah, I haven’t. I haven’t spoke yet.
17:00:12 After the 11 time to come.
17:00:08 We’re hitting the floor. We have a week of He’s coming in, he’s also going to Menlo Park.
17:00:18 I think you’re going up to Yeah, he’s gonna be in your nick of the woods after visiting me, I think.
17:00:26 You are?
17:00:26 Yes, I’m gonna go say, yes, I’m going to California.
17:00:30 Huh, need a place to stay?
17:00:32 10, that’s wonderful, but I usually stay in Berkeley. So I should be okay.
17:00:41 Where are you?
17:00:39 Okay. Menlo Park, Stanford.
17:00:44 Oh, just a touch more central. Just to touch.
17:00:49 Closer to SRI.
17:00:55 See.
17:00:52 Yes, absolutely. Right now.
17:00:59 Yep, right next door.
17:01:01 Les a.
17:01:04 I’ll be right back. I’m gonna put my other camp.
17:01:10 So I just wanted to show you the Emily and I. We’re in Norway now. Quick waves, Hello, they’re gonna try to be quiet.
17:01:21 And your happy holiday.
17:01:27 What hotel are you staying?
17:01:24 Oh, hey, can’t tell you, I’m ahead. Are they gonna?
17:01:30 What hotel are you at?
17:01:33 Oh no.
17:01:37 That’s your mother’s house.
17:01:39 Okay.
17:01:37 Oh, this is a home in Norway. No, this is, this is our house. Family family in summer house so to speak.
17:01:45 But we share for summers and quite clearly summer right now. So, you know, Please, freezing somewhere.
17:01:50 You’re a bargain. Right.
17:01:53 Well, it’s summer in Australia.
17:01:54 Bergen, right?
17:01:56 Yes, but yeah.
17:02:00 So, yeah, if there’s too much.
17:02:01 Ouais à toi à Stephen Ring Tud de la Comme Brachi. L’altitude.
17:02:12 13 h.
17:02:12 Thank you. We are very quite happy.
17:02:14 Well, know what I said to him when I when I talked to him I said oh, so little faith.
17:02:20 Yeah.
17:02:20 He said, oh, we’re not going to get this, Grant. I said, Yes, we are.
17:02:25 We’re gonna get this. I mean, all the signs are there. We’re not going to get this. Yes, we are.
17:02:32 Faith.
17:02:31 Anjib, Dale.
17:02:36 Okay, now what’s going on?
17:02:39 I was, you mean the beeping, that was my wife trying to find her phone. Everybody heard it around the world.
17:02:46 They’d lost phone beep heard around the world. Sorry, it’s just that it’s nice to have a summer house, access and probably, and almost going to be holistic.
17:02:54 But the problem is the internet is really expensive, so I’m using my phone for internet. So if I sit at the back of the house.
17:03:02 Then I get no internet back. That’s what I have to share with the family today.
17:03:06 Otherwise, it’s an insane amount of money and we’re here not so often here. So I apologize for the family engagement system.
17:03:15 But yeah, no, it’s nice. And it’s been really interesting. A lot of the private messages I’ve received.
17:04:04 And then, yeah, there’s been some other people now who. Are able to engage with us in a different way.
17:04:11 I’ll actually turn recording back on.
17:04:14 So yeah, there are some people who are now in kind of institutionally able to engage with us in different ways.
17:04:21 Oh, that’s very good. Essentially, as Dini said, we have been given this more clap.
17:04:30 So one of the key things, Adam was very good last week when we had an impromptu call is asking, what does it mean?
17:04:37 What does it mean? What does it mean? Thank you for being very insistent, Adam. And for those who weren’t there.
17:04:44 The analogy that we have is we have a flagpole now and you decide where you want to put your flag.
17:04:50 So we have to decide. What the community will be. Also how we will invite new people.
17:04:58 You saw I sent a thank you email today. Sorry it’s been a little much email lately of course.
17:05:02 And that’s what I’m trying to highlight there. Once I can have to ask, I go to the call.
17:05:10 Can I mention something? Well, he’s being quiet. Is that we want it one of the things we promised the Sloan Foundation is that we would expand this community.
17:05:17 And make it more diverse. They’re very keen on that. If we’re going to be building.
17:05:23 This kind of system for XR, although XR is pretty expensive the equipment. It still means open source, open web.
17:05:32 And so how can we bring in more people? To help us think through these things. This is not just us that are going to be using this eventually.
17:05:39 At some point everybody’s going to be using this. There was an article in CNN today about Apple Company and Tim Rice.
17:05:45 And him putting himself on the. Line with the Vision Pro. So I encourage you to look at it.
17:05:51 I’ll pull it up and drop it into this Slack channel, but essentially. He says some things about the use of this technology that I think is worth.
17:05:58 Looking at. So. But we won’t expand. So we want more people to come and do this on these Monday mornings.
17:06:04 That was an interesting Freudian’s lived there. You said Tim Rice.
17:06:09 I did, didn’t I?
17:06:14 Okay.
17:06:13 Now they’re always quite lovely. I also mentioned Slack channel. Do we have a slack channel now or?
17:06:20 We will have a Slack channel. Yes, all these things. We kick off in January. So this week is my last week of school exams are this week.
17:06:30 I’m leaving for New York on Thursday. I give back Sunday Monday morning the lab hits the ground running on setting up the systems.
17:06:36 So that when we when January hits, we’re ready to go. So we’ll be setting up Slack, base camp and all kinds of things for us to use.
17:06:45 Very good.
17:08:05 So we are Yeah, of course, Dave.
17:08:05 May I respond? Okay. So the lab is, it’s an interesting, project for my lab because we do a lot of preservation work and, and our art based up, right?
17:08:18 And so the question is, how does this fit? Well, one of the big. Tenets of our lab of my lab is accessibility.
17:08:25 How do we make things last over time? For as many people as we possibly can make it. So that’s the kind of the grounding of what we do.
17:08:33 So the preservation work is not always just want to preserve things, it’s how do we make this accessible to audiences in 100 years.
17:08:39 You know, what technologies will last will persist. Over the next decades. And so, you know. The product this project is interesting because it’s using web XR.
17:08:51 So open, right? Although it’s using a very expensive headset. But we’re moving away from proprietary software like unrelenting.
17:08:59 And unity, which is great because I don’t like proprietary systems. On the other hand, we’re also interested in.
17:09:06 How to work with people that are disabled. Neural divergent. How do we make these technologies?
17:09:12 Work for their needs so that it’s not, we’re not building and saying, oh, if you’re, if you’re hearing impaired, you need to flip the switch.
17:09:21 It’s like, no, no, it’s built from ground up everybody. We don’t have to make any special changes to these things to make it work, right?
17:09:29 And then the third one is that how do we make this available? To people of. Who don’t normally have access to these things.
17:09:38 It’s not just people of color, it could be anybody. And I live in a blue. Caller working class community.
17:09:45 My students are not elite families. But their elite thinkers. So how do we get them to have access to these things so that they can?
17:09:56 Use these tools to build and and create. And move forward in their generations to come, right? So I’m interested in this project from a lot of standpoint.
17:10:07 So of course the other thing is that I could see using this for art. There’s all kinds of things we can do a text and a VR environment for art.
17:10:12 But those are those are kind of sitting there and I think that’s one of the things that We heard back from the.
17:10:19 From the evaluators that they liked our DEI. And for me, it’s not about DEI.
17:10:26 It’s about accessibility. It’s about being kind and this is what we talked about. A week ago or so, right?
17:10:32 Just being kind to everybody, making sure everybody has access. So yeah, Michael. Students and my lab is full of students.
17:10:42 And in recent grads. As Rob knows. And they’re smart, they’re whip smart.
17:10:52 I’ll stop their photo.
17:10:54 Okay, so I’m gonna say the same but opposite
17:10:59 And the reason I’m excited about the Apple Headsets is because it’s expensive. Because we need to be a few years in the future.
17:11:08 You know, Alan Kay said, why don’t you spend as much money on your computer as you do in your car.
17:11:12 You too professional or a student trust me. I’ve been both. Hey young and not so young student so I know that telling you to invest when you’re a student is pretty ridiculous but you know that’s why this is exciting.
17:11:26 You know, we do expect in a few years that this will be you know, like a cheap iPhone.
17:11:32 Type of thing and a lot of students will have it and we need to live in that future. Make it to make that possible.
17:11:38 Yeah.
17:11:41 Now, a couple of other things. First thing is. We did for book one, have a competition for students.
17:11:48 And that was okay. And now we’re gonna have more of a competition. For students. So we need to over time in community define what that is.
17:11:57 We need to decide on Yeah, it’s basically going to be for the book. Excuse me.
17:12:05 That we want to make it clear doesn’t have to be like a master’s dissertation.
17:12:09 It can, it can even be a photograph of a paper mache toy if you want to do if it’s a student wants to do prototyping.
17:12:16 The only requirement is passion. There has to be an honest passion. So we’re not going to have a cash price.
17:12:23 Hi, that because then you will get parents involved who will push their kids. Because it has a certain kind of a cachet or whatever and you’re going to get a lot of rubbish.
17:12:33 So it has, you know, the main thing, like the first one we had the price was an hour meeting with Vent online.
17:12:39 Several signed books from the other authors who were part of that book. At that kind of stuff. So we will absolutely in January have a proper meeting on how to reach these students.
17:12:53 That’s important. And then there’s another thing which is going straight into Bras Tax as they call it.
17:13:03 So Peter, you the one who a long time ago said that visual meadows great, you should be able to throw it away.
17:13:10 Meaning temporary visual meta dependencies and that’s become more and more important. So I’m reminding everyone of that now, but also.
17:13:18 How important it is specifically for me to credit people. I like to invent myself. I like to take credit where it is due, no question.
17:13:29 I’m an artist. But I have a teacher. Hearts. I think it’s more exciting.
17:13:34 We’re in a community where we can properly say. Oh my god, what you thought of there?
17:13:39 I have never thought of it. You’re brilliant. So Peter, you’re fantastic. Example of coming in looking at something and changing it.
17:13:45 And then there’s one other thing. That is really relevant and I think I thought it was either Leon or Fabian and yes I can tell you a part.
17:13:55 But there was one thing that I discussed with one of you and it turns out it’s probably Leon.
17:14:00 And that is the notion of having of where I know it sounds like details, but this is not a detail.
17:14:07 This is so important of where the documents are gonna be. And somebody said. The software on your PC.
17:14:16 Mac, whatever, could be a server. Was for streaming was that you Mr. Leon?
17:14:25 I’m afraid that was, or my brain is as, as bad as yours, I don’t know.
17:14:34 I think it was probably both of you in conversations. But let’s not go back to our record.
17:14:41 You know, we have stored everything. Was it you, Adam?
17:14:41 I said that. I said that last week, yeah, yeah, we talked about this last meeting.
17:14:48 Okay, you’re here so okay now you should feel really bad at them you’re here so rarely that I completely forgot it was right.
17:14:54 So let me tell you why all of you. Obviously not you, Adam, and thank you. Why that is so important.
17:14:58 Rarely, I’ve been here. I’ve been here for 45 50 meetings. Of the 300 so
17:15:03 No, I know. I know I’m just saying it’s nice that you have an opportunity now to be here live because of your obligations on Monday’s and we’ll probably change the time but there we go.
17:15:16 So this is really important because
17:15:19 Hang on. Okay, sorry, work message, right. Here’s the thing. There has been a lot of argument within the community.
17:15:34 Yes, now. EDF, all of that stuff. And Adam solved L swoop. So this is what I propose we do.
17:15:42 If someone wants to read a document, we probably have to take the document to XOR. But one thing that it seems to me, we agree on is dealing with the library.
17:15:51 And XR is going to be a primary feature because you can have lots and lots of documents, right?
17:15:54 So I don’t mind. Serving from let’s say my software reader to XR Jason files or something I don’t think there’s anything wrong with that
17:16:07 So that means that whatever the head is best for XR. There should be a server. That can provide that.
17:16:15 Why should we make the XR do the heavy lifting? So having really thought about this this week, I think that Good.
17:16:24 I’m glad, SABBY and SCHOOL is hand up and Leon that we can ask you guys what is the absolute best way to get this data into XR.
17:16:30 And then we do the host software. Do the heavy lifting to provide that. It seems reasonable, right?
17:16:38 Okay.
17:16:40 Is a reader open source and is it multi platform?
17:16:54 I can’t work on it to be very, direct. I don’t have a Mac.
17:16:58 Look on it and you do that on the open board. I can’t touch it.
17:16:47 A Rareita is entirely owned by me. It’s one platform, Ha ha ha. And you should see that as a benefit because That No, No, no, but, Okay, I’m still answering your question.
17:17:06 You should still see that as a benefit. Because The idea is in my experience that if you have client server systems of any kind If you have the same team working on both, they tend to cheat without even realizing it.
17:17:20 Of how things communicate. The people doing my commercial reader stuff and not saying commercials, not even that commercial, it’s free and I don’t mind giving it up, that’s not a problem.
17:17:29 Good idea is that the protocol through which it communicates with what you do in XR. That’s what you decide on someone else implements it to make sure things do go through the API and it goes in a way that these other people understand.
17:17:43 Of course, we need to support other platforms. We need to figure out a way to do this for Linux.
17:17:49 We need to do it for Windows, no question. But I think in the in the sense of a case study having a locked in little thing on the Mac.
17:17:56 It’s not necessarily a bad idea. Now, let me make it clear. If it turns out it is a bad idea.
17:18:01 Happy to open source, Reader. Not against that at all. But I don’t think that’s initially useful.
17:18:15 Think of me as someone outside of the community, technically speaking. Thank you. The fact that my programmers have to be given the right documentation that anyone else would have.
17:18:25 It’s like they can test it. And okay, let’s put it this way.
17:18:31 Let’s work on that for a year. And then in a year when we’re done with this.
17:18:36 We have a discussion. If you guys feel I should open source and open up, I’ll do so.
17:18:41 I just think it’s a worthwhile and repeat myself, sorry, testing. Leon, over to you.
17:18:51 Thank you. Yeah, just to, you just, triggered some memories, maybe, Adam, Fabian, and I all said something in the direction of, you know, having the files on the letter.
17:19:04 Okay.
17:19:07 What I suggested, I think in the past, which was just a suggestion. Based on your you had some kind of video where you would basically you have a text on your laptop then you close your laptop and then you put on a headset.
17:19:26 Do you remember that video? So I remember that after seeing that I just had this brain fart of sort of you know shoehorning a tiny web server in reader or author doesn’t matter so that whatever you’re doing whenever you’re doing something you can press on a button which triggers a QR code which you can then you know, sort of scan with your headset and then immediately you’re sort of
17:19:57 you have that document in Ex because you’re on the same Wi-Fi. So to speak.
17:20:03 So that was one sort of like More or less frictionless idea to sort of transfer a sort of a web XR application to the headset.
17:20:16 Without needing any server infrastructure somewhere in the cloud which Yeah, might be, might add some extra layers of So that’s what I remember from.
17:20:30 From what you’ve told me just what you just told me. Yeah.
17:20:33 Yeah, that makes a lot of sense in this. I obviously see you right on 1 s. All I’m saying is that my passion for measure is for specific domain.
17:20:46 It should not slow down the development of this at all. Right, so if we can do some kind of a server way if we can do stuff.
17:20:53 Basically, we just need to optimize whatever the heck is best for Webex or Adam, please.
17:21:03 Whoops. That’s my mic on.
17:21:10 I, I think going to the web server of this one, the most, most platform. Cross platform ways you can do things.
17:21:19 No, and I think having a web server could be one of the most, one of the best way to preserve this as well for the long term.
17:21:28 So having a small web serving doing I don’t know if how much of heavy lifting it is to run the library, but it’s some lifting at least.
17:21:40 And that could be integrated into reader, but also run standalone without reader and on on PC or Linux or Anything I think that’s the most viable.
17:21:53 Called forward. In a way we need the web server anyway to serve Webex or content. Yeah.
17:22:01 Yeah, perfect. Adam, Alan and Brenda here, so I’m gonna repeat what you said and everyone said and that is lots of nice talk you have to rewind to watch the recording lecture anyway there has been some talk in the community by Leon Fabian and maybe Adam, I’m just kidding.
17:22:22 It turned out I was Adam. And saying that maybe a good way to get information into XR, libraries and documents.
17:22:27 Is by having a web server. And that web server could be in the client software. And I think that my passion for a visual meta should not slow things down.
17:22:37 It has many attributes in many places that are good. But to import a PDF into an extra environment and having it OCR or whatever to get the data out.
17:22:46 This of course planes silly. I just wanna make it clear that. You know, that’s understood.
17:22:52 So one approach we may want to do is on different platforms we have a basic reader like software. And that then speaks with Jason or whatever you guys turn out.
17:23:03 Make that you think makes most sense. To a Webex our clients so that at least the library information for thousands of documents.
17:23:12 Can be relatively instantly available. In that environment. I saw you nodding your heads, which is nice.
17:23:19 And Fabian is gonna come out of a bottle.
17:23:23 No, no. No, yes, all this makes sense. I agree with that. I’m just again saying that I can’t touch anything that’s not open source.
17:23:32 So indeed having a protocol and a format that allows to have Any number of implementation being closed source or not does make sense and I agree that in terms of ensuring the quality of the work that’s also a good practice.
17:23:47 So that’s not a problem. I also want to, highlight a little bit that, I mean, I’ve been using the word web dove quite a few times now.
17:24:00 It’s all school stuff, but I think in terms of managing a collection of document via the web, so having a web server being in video in any other implementation.
17:24:11 It’s a little bit annoying to set up, but once it’s and it’s not like complex basically they were Basically, turn key solution for this.
17:24:20 Allows to both read and write documents so you can imagine right reading a document and notating it or changing the document itself, no sidecar files.
17:24:30 And pushing it back or listing a set of documents in a directory or subject directories, etc.
17:24:39 So I’m just I don’t want to, I don’t have any love for web.
17:24:43 It’s not necessarily the best solution. But in terms of a lot of the things I hear again and again, I think it would already go quite a long way.
17:24:53 And that’s also why I did this demo during the How was it like 6 months ago, a little while ago?
17:25:02 It was as per usual, aesthetically. Boring and ugly but the list of Things were basically documented through web dev.
17:25:11 So I think that might. You will we might kick store the process through just let’s say a list of files and we applauding through a form and whatnot.
17:25:23 I think that would be reinventing the will quite a bit. So yeah, just an example of what might be a way to, to bootstrap and go relatively fast on the
17:25:36 Hmm. Yeah, yes, Webdab or some other technology that seems to make sense. Now, just show you guys something.
17:25:47 Writing all my secret chats with the coders. Making you a bit smaller. So I’m just gonna talk you through this.
17:25:55 You can see the screen, right?
17:25:57 I can only see if I’ll be answer can you not? Okay good. Right. So this is,
17:26:05 Some of this is possible now as a slideshow, so I’ll just go through it. So open a document like you can now run a prompt.
17:26:14 In this case, it’s the one of extract names. This is slightly on and I were experimenting with last week.
17:26:19 So it does all its magic thinking. Was back with the list. Now, first of all, this list is now editable.
17:26:28 So if there are any names in this document that are either wrong or unreasonable or not necessary, you just delete them at this point.
17:26:34 So this is how it works now. So this is not needing implementation, but what I think we should do, add a button at the bottom there that says save metadata.
17:26:44 And that does the Peter thing of adding another. You can see it mocked up here in other visual meta appendix with only that.
17:26:52 At the bottom of it it says what the prompt was and all kinds of stuff right So the way that I’ve been using this and testing.
17:27:03 It’s basically, I think there was another thing. I’m sorry I was a bit. Quick, can you still see my my screen?
17:27:12 Oh, okay. Sorry. I wasn’t paying attention. Is, so then we can do things like this.
17:27:20 So the user has their, specific names or keywords on the left. That they’ve added through whatever means they click on one.
17:27:28 And then whatever documents. Have that in their name metadata are highlighted. This is obviously the crudest possible way of doing it in the list on a small screen.
17:27:39 But I wanted to show you that because We’ve been talking quite a bit in the community about how to get AI to be usefully part of this.
17:27:47 And to make it just a find engine that’s a little clever. Might be one of the ideas.
17:27:53 So once you have this. Data and the documents. It is something that the software that created at No.
17:28:00 To extract. So that’s why I’m saying it could now quite easily be made available. Through a web server.
17:28:09 But that was basically that. So. Do we all, it seems like we all quite instantly agreed with trying the the web server.
17:28:22 Approach, Leon.
17:28:28 I, I think it’s a really cool idea because, Then, You can basically offload the.
17:28:38 The headset with, some, heavy work because For example, these open AI.
17:28:46 Web requests, etc. If that could happen on somebody’s laptop. Then the frame rate of the,
17:28:57 It’s easier to maintain a very fast frame rate on the headset. And perhaps you can just, sort of cast.
17:29:06 Just like a Chromecast, you can cast these kind of AI, find results to your, perhaps your space.
17:29:15 Of course you can have the document there, but having these Yeah, I think that’s a really cool idea actually.
17:29:22 I think it can save a lot of you know performance tweaking and just to give you an example usually when you do a lot of web requests within a web XR session, then sometimes it can be, it can cost some hiccups or And I think, you know, since you are a big fan of, elegant experiences and, etc, that could be a nice reason to sort
17:29:48 of offload. And yeah, purely use the headset for fast rendering and a nice experience.
17:29:58 Yeah, one of the key things with this approach, of course, is that all the documents would have whatever they the user wants.
17:30:05 Maybe they want to extract dates or whatever suits them. And because it is editable results if there’s nonsense they go through it and this is not intended for a million documents in the field.
17:30:16 This is their library. These are the things they care about. Because then hopefully we can get to a stage where we can have 100,000 keywords on your walls and they will make sense like Brandle showed us a year and a half ago.
17:30:28 Fabian and then Bristol.
17:30:30 Yeah, I don’t remember what prompted my thought earlier today. But, Yeah, I think it’s so I change I’ll reach trace my step.
17:30:43 You can see my low turn on and off again. I switch from an IoT platform called Web Things or the implementation for modsila web things.
17:30:55 So it’s to control physical devices. I have, example, so sensors like this. Motion sensor.
17:31:05 So I can track my family, but, in privacy respecting way, whatever other usage or bind the family tree, Christmas light when whatever is happening.
17:31:17 Which is completely pointless thus absolutely necessary I just for having fun really but the point is I switch from the Madilla web things to the Standard pretty much called home assistant.
17:31:32 And what home assistant does, just like web things from Matilda did. Is provide A set of URLs.
17:31:42 So basically a web API. So it means my light bulb literally has, a URL and I can contact my light bulb.
17:31:51 By sending the right message, let’s say to an off, turn on, change colors.
17:31:55 I have shortcuts for all this. All that convoluted explanation to say that I I tend to think as everything as a URL and that you basically have to do some routing.
17:32:11 You need to define the right path, something that is understandable, manageable, let’s say intellectually speaking and programmatically speaking and I think providing documents, a set of documents, URLs and making sure that How they were processed, for example, using summarization from whatever services, however it’s done, that’s really matter.
17:32:33 But providing URLs and providing ways to make them addressable and thus manipulable. I mean, makes a lot of sense to me and I don’t know if it’s a healthy obsession to see anything as an addressable endpoint.
17:32:49 A URL on API. And but I mean, so far it’s going more that way than the other way around.
17:32:57 Like I’m trying to make anything I have around me including so I give the light bulb example.
17:33:02 10 G ball things not just let’s say documents and that also means like a physical switch or rather a major switch in order to turn on and off devices that are otherwise let’s say not compatible with it.
17:33:18 So like how do you say? A what making the real world addressable is a continuation of this.
17:33:27 So I think it’s a Yeah, makes sense.
17:33:33 There was a lot pecked into that. Perfect to that brother. Okay, interesting.
17:33:39 I mean addressability is key. You can’t address something that smuggling. And therefore interact with it.
17:33:46 But you’re obviously going quite a bit further.
17:33:53 Right. So just to ask a few specific questions. Should we do webbeda? And it’s like.
17:34:04 Could you put a Right, a reference to probably and to that in the chat. One little behind on the chat I’ve been watching you guys.
17:34:14 Okay, if we’re going to have. The experience that I did in that little lockup video of working on something other computer closing the lid and the smell of the headset.
17:34:24 The computer will then not be online. So that’s something to really consider. Because the lid will be closed.
17:34:31 So we can you say that again please I didn’t catch it
17:34:36 In the little video that Emily filmed me at the coffee shop where I’m working on the computer.
17:34:41 And I close the lid and the information goes into Hey, our space because I’m closing the computer it means it’s no longer online.
17:34:50 So we can’t necessarily easily. Use a web that in the software we may still need a web host
17:35:02 So the headset itself can also be a web host. That’s that’s what I do when I’m let say flying or being offline and I want just the headset.
17:35:11 So it’s also an option. You keep the data synced, for example, but the headset itself can become a host and serve files to other devices.
17:35:24 Ready for grunt.
17:35:25 Right, okay. And what was that?
17:35:32 What was that you Peter? I didn’t hear.
17:35:35 Oh, I wanted to call everyone’s attention to the sidebar. Where I just dropped references.
17:35:42 2 a group called X Reality, which has an Academy. Oh sure.
17:35:46 Oh, okay. Peter, can we hold with that for a bit? Sorry. I thought you were referring to something to do with the serving thing.
17:35:56 Okay, sorry off that different topic.
17:35:55 Sorry, sorry. Yeah, I’m sorry, I met misunderstanding.
17:35:59 So. So, my understanding is you, you can keep them back working. I mean, you can keep many laptops working, when they closed our lead.
17:36:12 Mass concluded, I believe. Particularly if you have an external display plugin. Bit, real or virtual.
17:36:18 Yeah, yeah, yeah.
17:36:20 So you know I use my plugged into external display because I despise the ergonomic laptops.
17:36:29 So, so it should be possible to, to, come to some kind of arrangement like that.
17:36:35 It may be necessary to use some kind of explicit virtual projection system. But it may also not you know if the the Mac sort of remains as a web server then, then that should be, possible.
17:36:50 To, to write the like that. Okay.
17:36:55 Right, yeah, that’s how I use my mic too. So I have a dream of this.
17:37:00 Let me see if you agree with a particular dream and that is literally that little bit. So I’m working on such thing, probably my laptop because I’m away somewhere and oh I need to do this bigger thing.
17:37:11 But I need to close the laptop because otherwise it’s in my way. That’s how it is in a public location.
17:37:17 I don’t have any where really put it away. Close it, put my headset on.
17:37:22 So whatever the last few things I did on my laptop has to either be quickly synced with the headsets or there need to be a stable connection but there won’t be a display plug-in as such I think.
17:37:32 Do we all agree that that is a useful thing? Or is it more a matter of having a PC plugged in the corner at home?
17:37:39 Oh, probably, Yeah, and okay, there are some. Feelings on that.
17:37:49 So, I went on a holiday 9 months ago in the middle of literally nowhere in Belgium and I take all my device but I drop the internet connection as a mental and actual exercise and the point was to explore this kind of topic.
17:38:05 I put in the chat a specific issue in my repository and I believe that’s exactly what you were talking about.
17:38:13 Namely, you work on a device, you switch another, one is becoming inaccessible basically and you still need to keep on working.
17:38:20 So that’s what this specific issue is about. One trick, let’s say, if you use Web Dev or whatever, it doesn’t really matter is you try to sing the data whenever you’re Saving, committing, doing whatever you want and if your system your file system or whatever is properly distributed.
17:38:40 It should come for free. In the case of, yeah, manageable size documents on a working network.
17:38:50 That’s okay. When you start to have things like, oh, I want to run an LM that needs a specific hardware and everything, it becomes tricky and you might have to do some caching or say okay this service is not accessible but I think it is a proper question but there are some solutions for it.
17:39:10 It doesn’t Yeah. The issue there is an example. I personally, for example, did not implement it.
17:39:15 Right now in this tradition I have because Most cases I did find a like a pragmatic solution like regaining a connection or plugging back the device or postponing the work.
17:39:28 But yeah, I think it’s a valid question with existing solution. So. We can tackle that.
17:39:35 It would be amazing to have something decided today because I really don’t want to spend so much time figuring out the plumbing.
17:39:43 You know, we need to get the data into the Excel environment and spend a year working on just XR interactions.
17:39:49 Oh, I’m really grateful for the perspective here today, Leon.
17:39:55 Yeah, I would like to. I would like to be careful. With like, this sort of like pushing for a storage solution at this early point in time.
17:40:08 Could really sort of I, it feels a bit, yeah, maybe. Maybe I said enough.
17:40:20 Not storage transmission
17:40:23 And the feeling, okay, so I feel that. Visual Meta is still important because it’s very, very stable if nothing else.
17:40:31 So the idea is that someone will have a library somewhere on their PC, Mac, whatever. And they will have some sort of metadata.
17:40:38 It may not be the way we’re doing it, but some sort of metadata. And the key thing has to be when they put their headset on.
17:40:45 That metadata needs to be there. The data too. But I think we may. And I’m looking at Dini, Hare particularly, I think we may slightly reverse a little bit.
17:40:54 And say that yes, we do need to get a PDF in there. We do need to make it readable.
17:40:59 Good for the real interactions it’s probably going to be with the metadata of the people rather than the PDF itself.
17:41:07 So that’s why for you know like Peter is sitting in front of a Virtual Library there and we’ve talked about library, just imagine a couple of 1,000 PDFs.
17:41:17 That overtime a student or an academic has gone through they’ve done the sort of thing you just saw.
17:41:21 Extracted the kind of things that are relevant to them. If you do that on a 13 or 27 inch screen, it’s Nonsense.
17:41:31 But to do that room scale? It becomes really easy to make shapes out of this. But how we do that, how we hide and show things and how we cluster things.
17:41:39 That’s gonna require a ton of work. So I hope we can spend most of our effort on that.
17:41:45 Yes, oh, Leon, yes.
17:41:49 Yeah, I do agree, but, yeah, I would suggest to sort of let, let that.
17:41:58 You know if it’s so important then why try to limit ourselves to a particular idea we have right now at the journey at the beginning.
17:42:07 Why not just focus on the accessibility stuff? Like, focusing on. The how it will look in in in XR and imagining that we can just, you know, imagine that it was that visual meta was transferred through a web server, etc.
17:42:32 But we don’t perhaps really need that right now. If we focus on the, the top down, let’s say, how does the web XR client, even without communication from a MacBook.
17:42:45 Then, yeah, perhaps then we can slowly sort of think, okay, do we need to send a PDF?
17:42:54 And the visual meta in one go or over 2 channels like I think I think these kind of things will probably sort of show to itself what they want.
17:43:10 Okay, good points. First of all, I do not have a problem with The document and the metadata coming separately.
17:43:19 I don’t mind if you are, first of all, I do think that the library should be there instantly.
17:43:26 And in terms of working on the interactions and not the data transmission. I don’t agree with you there with the priority because I strongly feel that it has to be real data.
17:43:38 Because if it’s not real data, I mean having spent years and wasted massive amount of money in both author and reader.
17:43:46 Trying to come up with ideas. It’s only when I’m actually working and it’s actual knowledge for me.
17:43:51 I can know whether a view is useful or just a demo. So that’s really important. So that’s why I think that we can work on different ways to do the reading.
17:44:01 To send a PDF in we’re probably going to be doing different ways even horrible JPEG lots of different ways.
17:44:07 Get the metadata. If we just agree on adjacent form, for instance. That really suits Webex are we just Good nail in it, hanging on the wall.
17:44:20 So that’s the way it’s got to be. And then we could take her out. Way to get that stuff to Webex.
17:44:26 For the least amount of work from the from the user. Fabian.
17:44:32 I mean, it’s kind of the
17:44:37 Biggest challenge in my opinion it’s not even how we do it but what matters what should we focus on and indeed I think that’s the hard question when Leon is asking this is, okay, having a properly resilient system that is for tolerant and whatnot would be cool and I agree with that.
17:44:57 It even be would be useful. But would it be genuinely new? Or without cheating our way while still using real documents that Provider, a use case that Either for our ideal user or for one of us or all of us whatever is generally useful Can we take shortcuts basically?
17:45:19 Because we have limited resources being. Available brain time or even just energy time itself. So I think that’s the trick and that’s the trick.
17:45:31 That’s the challenge and it’s exactly like when you do I mean project management in general but prototyping when you do something genuine, in general, but prototyping when you do something genuine, and what are you focusing on?
17:45:40 What’s actually maybe She’s. And yeah, I don’t honestly have an answer for this.
17:45:47 I know that I that’s why I showed the issue before I do find this kind of like, file system interesting, useful.
17:45:55 Here honestly I would then also to focus on interaction like how do we manipulate a set of documents that itself as It’s not something that I’ve seen or heard and it’s something I’m generally curious about and I think is important.
17:46:09 So like yeah what’s the minimum we can live with without being ashamed of their underlying implementation to do the real work in my opinion.
17:46:21 I might told you guys lately how much I love you.
17:46:26 This is exactly half of why it’s Good. Yeah. Community. You know, we can agree on things and then some people will push further and some people will go back.
17:46:43 And I think that’s exactly what we have to do now. I completely agree. So let’s try to define where we can cut corners because we will have to cut corners.
17:46:52 Any company would have to. Number one thing that I’m not wanting to give up on is actual real use of data has to be there.
17:46:59 But if I think maybe what we do, we split. The entire project. Idea was into reading a document versus interacting with the library.
17:47:11 They almost have nothing in common. Of course they have some things in common but they are very very different.
17:47:16 So I see Brantles hand us up, which is great because For instance, when Brandle took the map from author and put that in XR, the way that works is the JSON file that Arthur already has made.
17:47:26 It’s dropped in. And if we If we have something, literally a message for the user saying.
17:47:34 Please note that your client software is robust. The XR experience may not be up to date. You know, we quite literally write it out for them.
17:47:44 You know, we may say if you’re missing something, have to reload. All of these things are fine because it’s not commercial software.
17:47:51 But there has to be an expectation that if they follow a few steps. They can’t put their headset on and it is their real library.
17:47:57 Randall, please.
17:47:59 Yeah, so thank you for reminding me. The way that that that, that, XR.
17:48:09 It’s, what I was gonna say before I went to say the cat as well is that, There’s a video game called from 1996 or so and it’s something that John Carmack did throughout his career for multiplayer games just that they’re, they’re never actually single-player games.
17:48:26 They’re always a client and a server. And so that, and it’s it’s a pattern I followed there as well and one I would recommend everywhere.
17:48:35 Is that, you, don’t do any processing on. But actually just turn, the server and the client into the same machine at the time when it’s a single use device into the same machine at the time when it’s a single use device into the same machine at the time when it’s a single use device and make sure that you’re always following sort of a networkable protocol across that gap in order
17:48:53 to make those things happen. So the way that the XR map thing happens is that, I ingest the reader document.
17:49:01 Yeah, the auto document. And ingest the first the the list of nodes and then the adjacency matrix.
17:49:10 And so it’s possible to be able to one load that kind of thing asynchronously and to be able to populate it as those things emerge.
17:49:20 I didn’t do that, but it’s possible to do so. And seconds to make sure that rather than rather than.
17:49:28 Acting on the the local sort of processing of identifying business nodes actually having to make sure that they are processed by way of.
17:49:40 Being sent as as a message as a signal through the system in that way means that any other client that has the ability to has the ability to respond to it.
17:49:53 Kind of do so as well. You know, I’m not a thing to do to make it more robust is to have sort of ad hoc sort of, tallying one.
17:49:59 Network states. So that you can cash a buffer and a set of commands that you know didn’t get to or any of the other ones that you know didn’t get to or any of the other ones.
17:50:11 You can actually make serverless kinds of things like that by saying like, I know that’s kinds of things like that by saying like, well, that’s kind of things like that by saying like, well, I know that’s kind of things like that by saying like, well, I know that so I’m client, A, this is can’t be, that’s kind of, say, I actually don’t know.
17:50:18 I know that they didn’t get messages at the rate of 10, so I’m going to call a copy of them.
17:50:22 So those kinds of things would allow you to create a period of peer. Based thing where there’s no distinction between client and server, but everybody must at least be a client answer ever.
17:50:32 So those are sort of lower level. Approaches that you can can use to make that kind of thing happen.
17:50:43 A question then.
17:50:40 Oh yeah. The ability to actually, sir, go ahead. Yeah.
17:50:46 I was just gonna say, are you saying that if in the scenario of someone in a coffee shop. They leave their laptop slightly open.
17:50:55 This will be a much easier thing to do.
17:50:59 We, I mean, so. I, yeah, the reality is that, so you could have a, you could, you could be connected to like the person on the in the cafe on their laptop could be you know doing one of 2 things that could be looking working entirely locally.
17:51:16 Or they could be working primarily locally. And just having something, even something very simple being sent to a co-located server somewhere that happens to be remote.
17:51:28 It’s my belief that the laptop has the ability to maintain. It’s entirety of it.
17:51:35 Activity once you close the lid, but even if it isn’t, then so then it would be possible for your your headmounted device to be able to hit just the laptop in the event that it’s closed.
17:51:45 But even if that’s not the case, it would be possible to then hit that server as well.
17:51:49 So.
17:51:49 And architecturally there will be no distinction between the servers and the client.
17:51:54 So if we then use, as you’re saying, have a server on the headset and on the laptop.
17:52:01 We could do something like the laptop saying You’re up to date. So then the headset can literally have a note that’s saying you can close the lid now if you want to.
17:52:11 For the sake of argument. Right. And then they work in the headset. And then they put it away and then the laptop could say somewhere your headset has sent everything back.
17:52:21 Okay.
17:52:25 Yeah.
17:52:23 You can put it away now or whatever. These are the kinds of cheating I don’t mind at all.
17:52:30 You know, I certainly don’t want to have to own some kind of a web server. Thing that is Amazon or WordPress that complicates it.
17:52:39 So we just say that devices have to be on during synchronization and then They resynchronize later, it’s not that much to ask the user, right?
17:52:47 Well, so that the issue is that synchronization is. I don’t wanna say trivially easy, easy, but it like this an incredible amount of infrastructure underpinning the speed and connectivity of the internet they said but if you use a Googleogle dog you know those are things for keystroke and and it’s not that it requires, you know, the Google scale big iron in
17:53:08 order to be able to do that. It just needs to do that for the number of Google Docs in flight at the moment.
17:53:13 So if you’re just dealing with, you know, your own set of, you know, single digit client, single digits, like, with, you know, tens of bytes per per kind of integration.
17:53:30 Then you know those things happen. Functionally for free now. You just, they just, they, it’s really ethic.
17:53:38 So the glitch, you know, page that I made for the, you know, the various NICE, the one that has the, like, it doesn’t have a save button.
17:53:46 And when you every time you press command desk, it has this snarky little message saying, oh, it’s automatically saved.
17:53:52 You’re obviously a very old person who’s not used to this. So, yeah, like, it’s, one of those problems that kind of, with the right kind of infrastructure on the right kind of.
17:54:08 Network infrastructure, which is, increasingly easy to achieve these days. It’s simply dissolved as an issue.
17:54:17 Right. Okay, good. So am I right in saying that this community, we’re happy to use Webdab at least as a test, may turn out itself when we do something else, but initially So we need to find a way to get from whatever software to X are I will invest in doing it from reader And you guys who are on the website will invest in telling me what to do.
17:54:42 And once we’ve done that, We’re kind of golden. One of the benefits I have with Reader is that as you know, there’s a function and reader that when you highlight the document.
17:54:54 And you’re close and you go to the library if you do a search, it will also search for highlighted text.
17:54:59 That means that this highlighted text is stored in some sort of database. I don’t actually know exactly what it is.
17:55:05 But it means it’s fully available to the reader software. Which means it should be trivial for my guys who take all of this other stuff, put it in there too.
17:55:15 Then implement the web. Dabs, solution you guys think about and I can see Adams about say something big.
17:55:23 And just make it happen for you, Adam, please.
17:55:27 You want me to do?
17:55:32 It was more a question of it like. Procedure or in because Dean is also running this and she has the team and we brainstorm a lot here and then it’s So it’s a matter of what they want to do also, I guess.
17:55:54 So it’s a bit complicated here, lots of lots of people and the people. Who’s doing much of the work is not in this video right now.
17:56:06 So we’re kind of the community part here. Yeah.
17:56:10 They will.
17:56:09 I can answer that even before Guinea and that is There’s things that are going to require specific knowledge.
17:56:19 Real work in testing and that’s going to be Guinea’s team. In so many communities I’ve been a part of and I’m sure you’ve had summer experience.
17:56:29 It’s really easy. To get stumbled on something early. And then go on and on and overthink it.
17:56:34 And my great mentor, Ed Le, in advertising, I remember we’re doing a pitch for some giant thing.
17:56:41 I was supposed to choose a background. I was a very lovely employee at the time over the summer. And I was trying to get the right this and the right that and he said you know Do it.
17:56:50 Just get it in there and then we can move on, right? When it comes to how to get the data in, I think it is actually for us.
17:56:56 I’m not saying any one of you should have to do any coding or spend any money or anything like that.
17:57:01 But unless Denny’s team disagrees, of course we will put this to them. Just getting the stuff there and then they can really focus on we all talk about the interactions in XR.
17:57:13 And that’s where we spend our money. Denny, was I right?
17:57:19 I’ve put a couple of things while you’re talking. So the person that’s doing this programming is Andrew Thompson and he met Froda.
17:57:27 At the Hypertext conference back in September. Andrew’s been working in my lab now. I think it’s been 3 years.
17:57:34 And he does a variety of programming activities and he’s currently working on the VR project. That we’re building for Rob Swiger.
17:57:43 In fact, he’s kind of leading that VR team for me. He’s got a lot of experience in programming.
17:57:50 He, lot of these students in my program started as computer science people. But wanted to switch into my program because we offer a more creative outlet.
17:57:58 He’s one of those kinda kiddos. He’s able to do pretty much whatever we wanted to do.
17:58:04 And if he doesn’t know it today, he’ll learn it tomorrow, which is what we teach in our program is how to.
17:58:09 You know, how to think the system rather than teach a particular software system. So we start onboarding him on Monday to get him, get get get computer setup for him.
17:58:22 Software. All the tech set up, I’m ordering him a new computer. These kinds of things so that he can hit the ground running in January before.
17:58:32 Froder shows up. The grant starts in January. There’s no funding until it turn the money transfers.
17:58:38 So I can’t pay Andrew until January, so I’m paying him through my lab right now.
17:58:43 To get him ready for the project. But once January hits and the funding is there. Then he’ll start full on on this project.
17:58:51 Monday. I get him kind of on the ground running. Which is good. And he’ll do, I think, whatever we’re wanting to do.
17:58:58 And he’s happy to experiment. The nice thing about working with young people is they really don’t have boundaries.
17:59:06 They they’re really just ready to try. So many different things and I like that about. About him especially.
17:59:14 So, in terms of, I’m afraid I haven’t read your actual grant.
17:59:20 Okay, so you’re what you’re expecting this to do. Oh yeah, hi, Bob.
17:59:27 Hi, Bob.
17:59:28 No.
17:59:27 Hopefully you saw all of them. Hopefully you saw all of the news that we’re talking about some of the logistics of building a.
17:59:39 Building a, the ability to. Process text and BR with some funding to do, to do that.
17:59:47 So, in terms of the highest level goals, and the, the people and resources, it’s, it’s this Andrew Thompson who’s going to be doing the entirety, the bulk of the programming and that the expectation is for them to take on.
18:00:01 The interaction.
18:00:02 No, we have 2 sources of programming money. One is for the web. And that’s gonna be Andrew.
18:00:11 And then we have another 12,000. A year for XR. Hi-level programming, that’s someone else.
18:00:16 Okay. Perfect.
18:00:18 It doesn’t have to be my lab, but it can be. I have. Are you nickels?
18:00:23 Nicholas for example that can do that but That I did not put in the grant as coming straight from my lab.
18:00:28 They’ll have to go through my lab to get paid. But I, I have right now about 8 people that are in the lab working for me.
18:00:37 That I’ve fund, right, through different grants and donations and stuff like that. And has been with me.
18:00:42 He’s one of those people who’s been with me for quite a while. But I bring in people constantly.
18:00:47 There’s a flow based on project. So right now I’m bringing in 2 young people. To work with Dina Larson on the Rob Swiger project that they’re doing together.
18:00:57 Exactly.
18:00:57 And they’re running through my leg funded through Dina’s money to do that project. So I have a lot of those kind of people, they get paid.
18:01:03 Okay.
18:01:04 Here’s the lab. One thing, one thing I’ll mention is that the university is going to be able to pay people.
18:01:10 Through stipends in honoraria. So I’m not worried about the 12,000 a year.
18:01:16 Person because that’s going to be straight funding but Andrews being paid as an employee. Because he’s been in a boy.
18:01:24 That makes sense, Randall.
18:01:24 Yeah.
18:01:26 Yeah, yeah, no, that’s great. So, so you have the, sort, of, the, with all to be able to, to find a range of people doing a range of different things.
18:01:34 And so that.
18:01:34 And there’s things, yeah, and there’s probably things Andrew can’t do that we wanted to do.
18:01:38 That’s where the extra money comes in.
18:01:40 Huh, cool. So yeah, so just before I finish, I’m, I think one of the most useful things will be it will be and having a web interface is gonna be useful for this is to be able to adjust.
18:01:54 Submit and surface a fairly wide array of resources in in terms of technical resources into a client process. Such that or a multiplicity of people and views have the ability to consume that in them in a range of ways.
18:02:09 You know, one of the things that that, Pro has, for example, is, voice input output in Webex R.
18:02:17 So you can do speech input output in Webex R. So you can do speech. That doesn’t exist.
18:02:21 That doesn’t exist in quest today. I, you know, you can do speech. That doesn’t exist in quest today.
18:02:23 I, you know, much to my frustration, I keep saying like, please, please do voice.
18:02:25 We can’t afford it. Whatever. So, so, that would be, you know, a useful thing if somebody wanted to, have, their own sort of original research or some kind of function within the context of documents, information, Webex or voice.
18:02:42 Where a lot of people may be wanting to invest more heavily in gesture and action. So, if there’s that, if there’s that breadth of being able to attach multiple people to the project, I think that also invites, you know, value in making sure that there is a range of ways that they can interface with it.
18:03:04 Exactly. And, the kind of complication of the opportunity that we have a thousand dollars a month.
18:03:11 On to be able to spend where it best goes as a complication I was. Actually not going to bring up too early because it may complicate things.
18:03:22 We are trying today to work out with the That would mean. And the thing is, if we now manage to relatively quickly establish this webdef protocol with metadata.
18:03:34 It means that let’s say Adam for instance, he designs to make it on that client. He decides to do all kinds of things and he wants to keep it on his own on his own name.
18:03:44 And just showed at the end of the year, that’s fine. Yes, Adam, as an example, also says, you know, here’s an aspect for something.
18:03:52 I would like to spend some significant time, can you pay me for at the time? And we’ll have to decide.
18:03:58 And. Course if we can we will. You know, we will pay for that time for whoever. That’s, that is why we wrote in the XR specialist.
18:04:09 It was just a title to cover the opportunities there because I don’t want anyone in this community to be out of pocket on this.
18:04:15 You know, I’ve been out of pocket for 12 years doing this. It’s not fun.
18:04:18 You know, I don’t want you guys to have to do that. But at the same time, there is quite a few of us to spread that money equally.
18:04:26 Be around every month doesn’t make any sense. And of Dini and me, we also get incredibly small salaries out of this.
18:04:33 It’s not that we could afford from that to pay for things either. But an alternative to this is also, let’s say, Deuce Adam as an example again.
18:04:40 He has spent some significant time doing something and he needs a piece of equipment to get to the next level.
18:04:45 You know it’s quite easy for us to consider yes that is a legitimate expense for the community. So right now we don’t know and this is why you that analogy last time for those of you were there.
18:04:57 It’s like this is a mast. And you guys get to decide in dialogue with us where you want to hoist your flag.
18:05:02 You want a big flag pulling us in the same direction? Do you want a smaller flag that’s just an article in the book pulling in a different direction?
18:05:09 It really is entirely up to you as long as we keep as we have been a really cousin positive dialogue going.
18:05:19 Hello, good.
18:05:16 Peter. Oh, actually, Brandon, does that address you? Sorry, I just wanted to ask.
18:05:25 Yeah, yeah, I think, given that there is a project with people. The sort of, actually the part allocated to it.
18:05:34 My, question was like, How many people what kind of people and what are the explicit sort of goals as they are established already for it?
18:05:43 And it sounds like, because we, we were saying like Andrew Thompson, you know, then a lot would depend on both.
18:05:49 That person’s skill level in a range of fields as well as any explicit polls that are associated with, you know, their employment frankly.
18:06:00 In terms of what, it is that we could do to make the. The project on the Prussian sort of skills and balance sort of all B set up for success.
18:06:13 But if it’s actually that there is a range of one has the ability to set up a number of people to kind of work with this, then there is a core infrastructure sort of aspect of it and then there are these subsequent kind of upfront and things.
18:06:29 So to that end, you know, I think there is a fairly substantial sort of But in terms of having people who have, you know, some minimal capacity to consume data and then transform it, people who have, you know, some minimal capacity to consume data and then transform it based on interaction.
18:06:40 But then to whatever extent is, necessary on the back hand side to be able to. Consume data, but it’s not really based on interaction areas.
18:06:48 It’s based on having a robust enough representation of what that information wants to be. Like on the server side in order to be able to kind of vend it, then to all of the other clients and make sure that those things can maintain.
18:07:02 My I will say that that job is a lot easier than back-end engineers make it sound when you’re dealing with you know one or 2 or 3 clients make it sound when you’re dealing with you know one or 2 or 3 clients.
18:07:11 It’s not that it’s an easy job. It’s that it’s not that it’s an easy job. It’s that it’s an easy job.
18:07:14 It’s that it’s an easy job at that scale. Back and people tend to be working with, you know, X 1,000 requests a second kind of thing.
18:07:21 That’s when it become challenging. We are not going to reach that and we shouldn’t aim to replicate server architectures that are sort of structured for that at this time because you can boil the ocean.
18:07:37 Thank you, Brendan.
18:07:40 Okay, one thing I was thinking is that since a lot of what we’ll be doing will be sit down We are an XR.
18:07:46 It would be really good if we had support for existing 6 degree of freedom, input, controller devices.
18:07:54 Then we could bypass a lot of the efforts of worrying about gestures. And work directly with.
18:08:01 You grab your little puck on the side of your desk and you can move a data cursor around in volume that goes out farther than the length of your arms and hands.
18:08:13 I’d really like to see. That is an option.
18:08:13 Okay. We’re not gonna do that. And I’m lovingly putting my hand out because That kind of 3D has been around for many, many decades.
18:08:29 Huge part of the benefit of XR. Is the fact that your head is never completely stable. So you always have a sense of death that you cannot replicate on the screen.
18:08:40 Plus there’s a sense of space and considering we have one year. We don’t even know how to get the data into this thing.
18:08:47 What you’re talking about is important.
18:08:47 I’m just talking using the 3D punk, the means of moving a cursor around in a 3 dimensional space using the vector controller as opposed to physically moving something around.
18:08:58 Otherwise I’d wind up replicating the functionality of the nice tangible pup. With recognition of where my hand is and have the hand be offsetting in a volume in order to create the same, it’s just the tangible of ford and seems at least in my mind make more sense if I want to be controlling a point within the volume that’s going to go out farther than the length of my
18:09:18 arm.
18:09:18 Sure, so to that end here, like, the one hard sort of, demand.
18:09:27 Not that I can make any here is that the client, not that I can make any here, is that the clients I’d be rendered, not that I can make any here, is that the clients I’d be rendered in through.
18:09:31 I, I see no reason why that should not have been the case. And, and Space Mouse, the threed connection device there.
18:09:40 People build a you know the ability to use a space mouse and through jazz. It also has all of the 6 degree of freedom controllers like the quest 1 2 3 for all controllers.
18:09:51 Or, I picked up. So, yeah. And if you want to be able to use the Space Mouse, then you can if you want to use a desktop computer with just a mouse in a keyboard you can.
18:10:05 In fact one of the one of the neat things I discovered this isn’t been through Jasper site but on an iPad you can use a you can use a to the Apple Pencil, but you could also get the tilt XY and the and the pressure and, portray that in 3 jazz as well.
18:10:19 So, you know, there’s a, there’s a huge multiplicity of, interactors available.
18:10:23 Another neat thing about it is that because it has its own tilt that you can have 2 people you can have somebody on tilt that you can have 2 people you can have somebody on a vision grow or an oculus quest and Also on an iPad at the same time and have that iPad represented in their video session with that pencil as I said.
18:10:40 So, there’s, there’s a, you know, whole range of ways that you can kind of interact with the system.
18:10:43 It’s web mediated in this way. And, and it’s made no harder.
18:10:48 Bye, doing that. So, yes, and.
18:10:53 Yeah, I’m still putting my foot down and the reason for it is of course twofold.
18:10:58 One is time. And focus that’s one reason. And the thing is, Currently, I’m writing a little piece on the Vision Pro saying I’m really annoyed by the fact that they expect us a set where their hands in our lap and twiddle our thumbs and that’s considered work.
18:11:16 That’s how I know that’s only the marketing material, Brandle. I know there are other interactions.
18:11:21 And then, of course, in the quest, you have laser beams. These are very limiting, but of course in both media the hand mapping is absolutely incredibly good.
18:11:30 You know, when I do use my quest, I can’t take the screen when I’m doing it.
18:11:35 I can move it where I want. So when it comes to issues for where to reach into a bigger volume of knowledge.
18:11:41 Even within the headsets, that is going to be a lot of research. Of how to do that.
18:11:47 I think that’s going to be a lot of our work. So to then introduce another piece of hardware requirement for the user.
18:11:53 And to start testing that I think within a year and you know thus as a community we meet once a week.
18:12:00 And the limited fronts we have, I just don’t think it’s reasonable. Because Peter, I’m like you, I will run off with things.
18:12:08 You know, I will look at that and then, oh my gosh, that’s not where we should be.
18:12:11 So I think it is useful. I think it is important and that for that reason I don’t think we can do it as part of this.
18:12:18 Does anyone else want to? Comment on the Dinny.
18:12:20 I would like to, we talk in our program, we talk in terms of stretch goals. And so if we make certain goals and we have time and the funding resources to do it, then we add these series of stretch goals in a priority.
18:12:36 So I would like to put on the table that we put together a list of stretch goals. That if we were able to achieve the things we promised Sloan that we will do.
18:12:46 We will go beyond that if there’s time and resources. I know that that from my lab’s perspective.
18:12:53 We’re always interested in what is next, right? So I, I have been part of this, a grant.
18:12:59 Proposal to do the things that. We’ve been talking about during this these labs but also to support Froda.
18:13:06 From a personal perspective, I’m interested in 4 things, one of which is how How art performance, a poetic art performance can happen with this technology that we’re building.
18:13:17 That’s. Know where I wanna take it one day. So I have my own particular stretch goals.
18:13:23 So I’m sure that these things that we’re talking about can be put on that list. And are worthy of attention and that would not be outside of our purview.
18:13:35 Yeah. I can agree with that, Danny, but I’m reacting a bit stronger than Dini because I also get excited by things and if I mentally start thinking about another interaction, I may just go nuts.
18:13:44 Hmm.
18:13:46 So that means that’s me being defensive. Oh, Kenny.
18:13:48 Yeah, may I recommend, why don’t we, I don’t know if we shared the most recent proposal copy and I have no problems with sharing it.
18:13:58 It’s not proprietary. And that way everyone can see, We’ve promised, right?
18:14:04 You can see what we have to deliver in the timeline of delivery. And then we could talk a little bit more.
18:14:11 I don’t know. In with a knowledge base about how to proceed from there.
18:14:17 Does that make sense to everybody?
18:14:20 Okay, great. Then we’ll send out the proposal to you all.
18:14:26 I’ll do that now.
18:14:27 Thank you.
18:14:30 And just to say one more thing is that, you know, usually when you put grants together and you submit.
18:14:36 There’s a 6 months wait before you find out you get the grant. We had no idea. If they were gonna turn it around in 2 weeks.
18:14:47 Never seen this before. And I’ve done other foundational grants. So this was really fast. And I was thinking that I would have Christmas they to kind of wind down and relax.
18:14:58 I got a book project that’s just getting released. I’ve been working on. And I’m going to start another book, I thought.
18:15:05 And so I really thought I was. Gonna have some downtime. So now I realize I’m gonna be spending all on Christmas.
18:15:10 Getting Andrew set up and getting everything ready for Phot’s visit. So, we’re moving pretty fast.
18:15:17 And I don’t want too much scope creep. At the beginning especially. So if we can make these things.
18:15:24 Extra, Landyap as we see in New Orleans.
18:15:28 American now it’s going to be interesting. Yeah, thank you for that theme.
18:15:31 Tina, are you from New Orleans? Sorry.
18:15:36 Alan right here.
18:15:33 I father, who asked me that question. Hi, Allen. I lived there 5 years. My father was head of a guide fishing business in the Schandler Islands.
18:15:48 Wow.
18:15:44 I would fly out there by seat plane and work with him on the boat on the weekends. Since I’ve been with him since, until he died, I mean, I was, so I’d go out with him.
18:15:55 I didn’t lived on the, so I’d go out with him. I lived on the Houston in Houston in the Gulf Coast area and then moved to New Orleans to be with him.
18:15:59 And I did that for 5 years until I realized I couldn’t drink the water. I was, I was actually dying from the water.
18:16:04 Yeah.
18:16:05 Water poisoning and so I had to leave. Makes me sad. And I would be back at least a couple of times a year for jazz vest and visit family.
18:16:14 I kept a house there until I don’t another 10 years. I didn’t get rid of the house for a long time.
18:16:20 Finally sold it.
18:16:20 Sorry, I had to ask. I was I was a born of Van Rouge, so I know.
18:16:25 Pretty well.
18:16:24 Oh yeah. I live in the, at first and then I moved uptown behind the Dominican.
18:16:32 So yeah, I love New Orleans. I don’t think I’ll go back because of the politics.
18:16:35 Yeah.
18:16:42 Alright, continue. Sorry, thank you.
18:16:36 It’s sad, but yeah, New Orleans is great. It just Sorry.
18:16:45 So for a lot more boring topic, I would personally not say no to anything as long as it’s not toxic.
18:16:53 So I would not discard any idea and some of the craziest weirdest I did don’t even make sense at the beginning then I was restored to unfold they do and some of the most brilliant one are actually bullshit as long as we stood as soon as we dig in rather.
18:17:08 So I would, doesn’t mean I would say yes to anything, but I would still like keep it to trace somewhere and then maybe 9 months down the line when it’s time to think if everything went well and we don’t hate each other and what not, then yeah, what’s coming next?
18:17:23 So I think some of the stuff that seems to out there right now, either at stretch goal or layered or whatever to put at the bottom of the list.
18:17:34 I would, as long as it’s not toxic, Keep them somewhere and sometimes also we keep on having the same ideas.
18:17:42 We just don’t use the same words or the what triggers it is a bit different. So.
18:17:51 Okay.
18:17:45 Having them written down without an instant no because it sounds too, although maybe nobody sound interested in it, but there is still like a kernel of excitement in there.
18:17:56 Is useful, but it’s in the end we’ll still have to prioritize say okay that’s interesting but invisible in that time frame or, oh yeah, maybe it’s too niche or whatnot, but still I would recommend to not throw away anything, you know, at least in terms of description.
18:18:13 Just put. It’s a not a top priority on this.
18:18:19 Yes, that’s very nice to say. However, I do think we need to have some framework around this.
18:18:26 And I think bringing in additional hardware, you know, a truck ball. Is very large expansion or what this is, especially if it’s considered.
18:18:36 Integration into an AR or even a 2D environment. You know, we could bring it up in a few months once things are going.
18:18:44 Peter, please note it down as a stretch goal. That’s fine. But it is literally beginning on a whole new device.
18:18:49 So Peter, I feel it’s okay for me to be very clear with you because you’ve had so many brilliant ideas that I didn’t understand and later did.
18:18:57 I think you know I hold you in high regards so It’s I’m not saying This is a silly idea not worth our consideration, I’m saying.
18:19:05 Yeah. Hey on.
18:19:08 I would like to say I just put in this chat that. I just imagine that we’re working on this project and we have these things we have to do to meet Sloan’s obligations, right?
18:19:26 Yeah.
18:19:20 But there’s going to be all these great opportunities for other things that we can expand into. And so I can imagine there’s other little smaller grants, other grants slow might fund a stage 2 grant.
18:19:33 For a list of things that we want. And when I was working on the Mellon Foundation grant some years ago.
18:19:38 I gave them the proposal and they wrote back and they said, Yeah, this is great. But where do you want to go with this after?
18:19:46 Okay.
18:19:46 So I gave them the whole scope. Okay, here, here, here, here, I got the grant because I had.
18:19:54 Let’s go.
18:19:53 These other things in mind. Yes, so that’s the I think if we start with that premise.
18:19:59 And we make a list of all the things as we’re working to get, oh my god, we can do this.
18:20:03 Can be this. Let’s keep that going and see where we can take this. For a long-term project.
18:20:11 Yes, that is fair enough. But I’m also saying it’s a comment here, another neat aspect.
18:20:18 The 3D connection tech is that one could theoretically have multiple controllers and use at the same time say each and one hand.
18:20:25 No, not now. Okay. Look, we’ve been doing this for 2 and a half years now, the XR thing.
18:20:33 In the beginning, there was some coding done. After that, done nothing. The fact that we now have a programmer who’s gonna be working on a relatively reasonable budget.
18:20:42 We still have to define what this is. And this project, and I’ve emailed all of you just now, the documentation.
18:20:49 It is getting 2 things into XR, a library and individual documents on demand. Being able to interact with MC connections, that is it.
18:20:58 No other hardware, you know, we’re gonna use the interactions that are that are in the systems that means and controls maybe the laser pointers maybe the controllers but if we’re going to do this it is Yeah, I know.
18:21:12 I see I see you writing not necessarily now just living integration books in place. I see you write that Peter and that’s good but We also, I mean, I see Bob Horn is back here today, which is very lovely.
18:21:23 And Bob Bug does a year ago saying, you know, what are you guys actually doing? Oh yeah, that’s fine.
18:21:29 I know you have to leave but just literally just another second. I think in these discussions now In the future we need to try to find a schedule or when are we talking about the future of text and general and when are we talking about this?
18:21:43 Because we do not want to shut down the kind of conversation I’m thinking you’re doing, Peter.
18:21:47 Absolutely not. But I do think that it needs to be more in the this is the day for Alan Laidlaw’s origami and Peter the Celko’s new threed things we shouldn’t drop that absolutely not But, you know, I am actually not very stressed by this project because I think bus we can do this.
18:22:06 But when it opens up in such huge areas.
18:22:13 Yeah.
18:22:12 Right. How many more minutes for you, Danny, before you have to rush?
18:22:19 Dina, you have a few more minutes to be with us, right?
18:22:22 I have about another 5. I’ve got to get boxed up. I’m taking a lot of things to campus today.
18:22:27 This is our last week on campus and I’ve got. Rob coming in tomorrow.
18:22:31 Excellent. Okay, Leon, and then we’ve got to figure out what Alan’s doing.
18:22:38 Yeah. I have actually some good news I think. I think you might not realize this throat but i see actually a reality where you and Peter walk hand-in-hand with a headset on.
18:22:55 Where you are using the hand controllers, throat and, Peter is using whatever he wants to use because And I think this was also the point, Brando was partially making that you know, if it’s Webex R it’s sort of it sort of comes with the package already because these input device interfaces they’re all sort of extending each other.
18:23:22 There’s a certain hierarchy in input devices and they all sort of use the same code.
18:23:32 So if we get the accessibility, right. Then it might just work out of the box.
18:23:41 Like if you plug in a mouse. Into a headset then there is some Some things happening already, done by the operating system and mapped to a web XR input device.
18:23:55 So I think, we can just, we can also just keep it out of the sort of, paid goals.
18:24:03 But yeah, I guess the unpaid goals. There’s lots to experiment on, which, could even help each other’s I think on both sides.
18:24:14 So I think it’s a sort of illusionary separation this bringing in new devices or not.
18:24:24 I think this is why we need to figure out a way to separate our conversations by a little bit of topic.
18:24:30 Because clearly you are right. There’s no question about that. Just for context, and I was getting some messages earlier from my programmers and reader, they’re supposed to have a new version last Monday.
18:24:42 Obviously they didn’t. The thing they’re trying to do now. Is in the library if there is an editor associated with the work.
18:24:49 So name, comma, editor. The reason for that is in my library because of the future of textbooks my name comes up a lot.
18:24:56 And I don’t think my name should come up for the future of text. That’s all of us.
18:24:59 So it should be FROM HAGLIN COMA editor. That’s taken them weeks to do.
18:25:05 Because of this that and the other. So what I mean is when I’m talking about focus, to get the interactions not to a commercially sellable, scalable thing, not at all.
18:25:16 But to make it so when someone with the right circumstances puts the headset on, it’s a really smooth experience.
18:25:21 It’s not a joke and you guys are the world programmers. First we do 90% of the work then we do the other 90% of the work.
18:25:28 I think we have to be laser focused on one part of the discussion. Maybe we start meeting 2 times a week for those who want to.
18:25:34 But the other one, we just continue as we did before and if we have crazy ideas we were down.
18:25:41 Oh, see you later, Minnie.
18:25:42 I was gonna say I agree with about the extra meetings. I’m gonna be putting in an extra 5 HA week.
18:25:47 Starting January one, but I think if we have a meeting to work on just the project. And a meeting on where we’re headed.
18:25:56 This is philosophically, spiritually, intellectually, all those things that would be good. Cause I think the other thing Photas talked about is the book that he wants to do on what this all means as for humanity.
18:26:07 So I think there’s that part of it too. So there’s There’s gonna need to be more meetings and that’s just for those of us that are in.
18:26:15 But yeah, I need to leave now, so I’m packing up. Bye everybody. Thank you for your good wishes and I’ve loved the ideas.
18:26:23 See you next Monday.
18:26:27 Hello.
18:26:24 He can’t see you, but same boy. They got, no, no, no, you can’t hear her is what I meant to say.
18:26:32 Okay, Dean, he’s got to go. See you later, Danny. Thank you.
18:26:33 Thank you everybody. Well, I’ll see you tomorrow.
18:26:36 Yeah, quickly. Where did you want to? No, you go talk to Mama. Okay.
18:26:44 I got we’re in a meeting right okay guys yeah we gotta keep It’s focused and I’m in a wide open conversation.
18:26:54 No question about that. Now, that D is out of the room. What is your suggestion, guys, for what I can do to make it happen that we can do this synchronization mechanism.
18:27:07 I want that done by the end of the year. If the foster piece of we do it again, fine, at least we’ve learned something, but until it’s the real dot and the real data, it’s not real.
18:27:18 What to do?
18:27:20 Right, so you need to send something over some kind of. Web protocol, be it web. Dev or web sockets.
18:27:30 And I haven’t written web data before. I imagine has. And, I don’t know whether web depth is sort of a real time synchronous protocol, in the same way that the socket base connection.
18:27:46 Is, or if it can be upgraded to that, but, it depends on the The timing of that’s fair enough.
18:27:55 I mean, I have also been, mostly to Lazy to do anything, directly.
18:28:00 So I’ve been using, you know, a socket. I think as the mechanism for these things rather than writing my own.
18:28:06 It’s, some raw real time connection. But yeah, so, you just need to get.
18:28:11 Stuff out of water. Into a web web based sort of platform oh yeah so one thing I was gonna say about that space master in, functionally it looks like I’ve used a space mouse before, but it looks like, 3 J and makes the integration of a Space mouse trivial.
18:28:35 On the other hand, Peter. The first contact with a 6 degree of freedom. Controller. Makes a space mouse completely obsolete.
18:28:45 So yes, it’s it’s easy to make. It’s completely unnecessary to make once you’re an XR because you had fixed off.
18:28:54 Controlers have that sort of ray based control, very, trivial.
18:28:59 And getting into just completely raw. Webmax are a hand tracking of the kind that vision pro has, it is pretty straightforward to just go like, oh, I have.
18:29:09 Array that corresponds to either the explicit prep space established within the thing or I’ve got a ray that I use which is perpendicular to the.
18:29:18 Kind of this ring here and I and I use the proximity of the fentrand or the thumb and foreprint guard to to indicate the sort of the pressingness of that, that thing and then you can, and navigate like that, you know, in a way that’s far, far superior to anything capable with this spacecraft.
18:29:46 So yes, but if the goal is for this to be in XR then, then fundamentally you don’t want to be bothering with a space mouth while you’re in XI, because you have this kind of range of motion available. Yeah.
18:29:49 Once again, so yeah. Yeah, I, send stuff over the network. Is the, is the answer?
18:29:57 Send something over the network in a way that a web system has the ability to concern.
18:30:04 Leon, I hope you have more specifics to flush out on that.
18:30:07 Yeah, I. To add to what Brandle is saying. I think a pragmatic approach would be to tell, your program or throat.
18:30:21 Create a small web socket server in a reader or author So, Dini’s program to, sort of, be able to consume that.
18:30:34 I don’t know what he’s gonna start with, but basically a web socket client. And you know have them send a ping or a pong message back and forth.
18:30:44 And then, perhaps try to get some, you can maybe send Jason over, but at least you have the real time, connection, there, which Brando was emphasizing.
18:30:58 And I think with web death, I know it a little bit and already from the start a pretty heavy authentication stuff.
18:31:08 Comes in so you need to have users and all these So I would sort of push that back for, maybe a second iteration or.
18:31:17 A bit further down the. The path, I would say that would be.
18:31:24 You might suggest. So just start with this web socket server on author, reader size and a web socket client on the, or perhaps both a web socket, server and client on both sides to sort of, because of the quick story.
18:31:45 Okay. On the authentication and stuff. Is there a way it can tell that these through the web and not through Apple technology?
18:31:55 That these devices are nearby so we don’t really need all that heavy authentication and user stuff.
18:32:03 Yes, if you’re on the same network, so.
18:32:08 Okay, so that means we can make that a requirement. You need to be on the same network for the synchronization to take place.
18:32:15 So that means they can discover it.
18:32:15 So that’s Sorry, that’s what I’ve been doing when I was away like in, in annoying Belgium is like, are you, it’s not a safe thing, it’s not to do in real life, but if your prototyping.
18:32:28 Honestly, I think it’s acceptable, namely that if the device on you the same network or the Wi-Fi then they have a bit more rights let’s say they were privileged so they can You can give them more things to do.
18:32:43 So yeah, that’s different.
18:32:43 So does that. If we decide and design a patient standard.
18:32:50 Let’s say and we have something and we have a web to have server on the PC type thing and one in the headset type thing.
18:33:00 Then at least for someone to use it. It’s as a prototype, but using it with their own data.
18:33:08 That’s fine, and then later on. Someone may come in and make a more. Secure version that has user passwords and all that stuff, is that right?
18:33:18 Yes, and if I were to give a grant and people start to re-implement the will for something unrelated, I would be pissed so I would indeed think we need to shun authentication and we need to remove bits that are Not actually valuable.
18:33:34 They’re important, but documentation with a big warning that said not production ready. In my opinion in that context makes sense and is better.
18:37:18 Yeah, Fabian, sorry.
18:37:21 In, yeah, one, okay, I heard a couple of suggestions. First to also echo Adam, give me the data.
18:37:32 Like, can you give me a zip file right now? With about a dozen PDFs.
18:37:38 And And that’s it. Like, can you store it on your server or GitHub or wherever?
18:37:46 I don’t care. In the sense of just starting the process, being hosted on an HTTP, HTTPS server with web, or with web socket or whatever endpoint in order to write the data back.
18:38:02 Like if you want to store the process, yeah, just. Let’s have a little tiny playful yet realistic data set.
18:38:10 So a set of documents. And eventually we can have some G in terms of either exposing the documents or additional metadata or savings or whatever.
18:38:22 But I think if that’s the game, let’s say without explicit protocol to follow. What the result would be.
18:38:32 That would be my suggestion. Like again, regardless, is it just HTTP or HTTPS rather with a We’d not even web socket, but just a say slash save URL where you can send whatever data you need as let’s say post or even get.
18:38:48 Whatever request, it’s enough to get the process going. And then we can share with each other different URLs with different versions, different implementation, etc.
18:38:58 That would be my recommendation. Having her foul in whatever format and then then we can store the set to attack it to be a little bit aggressive to start to bear with it rather and if it’s unclear if it’s the wrong format then we iterate we We document and we chat on it.
18:39:18 But yeah, that would be my recommendation. And ideally all this, they need like whenever somebody ask a question.
18:39:24 If it’s by example, GitHub then make an issue. Okay, it’s not the right format so I can read this and then yeah we can Move from there.
18:39:39 I’m having some problems hearing you now. Oh, you finished? Okay. Yeah, I will show you 2 things.
18:39:45 Number one. On the visual meta page there’s a implementation. You click on that. Then you click on AI.
18:39:55 And then here I have an example. And I have an embedded PDF. With that visual, that’s what we’re shown in the video.
18:40:06 This is something that we have to design together or rather you have to tell me if it’s wrong.
18:40:11 So that’s the first step. And then secondly if we now do package contents here so this is not a PDF this is the author thing Hey, look at this for instance.
18:40:27 This is what Randall deals with. So this didn’t have a lot of information, but we have Jason’s for that at that point, right?
18:40:37 Adam, please.
18:40:41 So a couple of points on the last one. Now, you had a document there with some sort of appended information where you actually modified the document.
18:40:54 I would absolutely not love to have my I have about 1,000 PDFs that are books and papers.
18:41:04 I would I don’t want them to be modified. My origin originals. So that’s one thing.
18:41:13 So the information I want to be kept. On the side or in a database or in the library in some sort of way so it doesn’t.
18:41:22 Kind of pollute my my originals that’s one thing. The other thing is to actually get a PDF.
18:41:31 Into XR, we have done that. Enough time so that is the actual Yeah, get to get the images, sort of say, yeah, with page flips.
18:41:44 That’s, that is not that hard. I will say extracted images from it and the text I don’t have pixel perfect text overlays yet, but I have it.
18:41:54 Good enough. For some text selection. You could do date basic text selection in the PDFs, but it’s not.
18:42:02 And it’s hard to get the font rendering, right? So you match the font of the text in the PDF and so on.
18:42:09 So that’s where I am at.
18:42:14 Right, so when it comes to the changing of the dock.
18:42:16 How are you? You did not raise your hand so I have the right to cut you there.
18:42:22 No, just for Adam and for any time we have this, I think it’s very important and it’s going to piece you off at them and also Leon and also Brendan also myself namely we cannot just say we’ve done that we need a URL for this.
18:42:37 To ideally a live demo or ideally a live demo, push the code, plus the documentation, etc.
18:42:45 I mean, you know the drill. And if we don’t, we need some kind of trace so that when we say we’ve done that, it’s not because I just trust you obviously I know that we did do that and it was interesting, but we need to build on top of that.
18:42:59 And even if the conclusion was like, oh, we done that and it was not good enough for XYZ reason.
18:43:06 And if somebody else, one of us or someone else need to build on top of this, we need to actually reference the, we’ve done that thing.
18:43:14 Yeah, I think that’s absolutely a fine thing to say. Hi, Mark. Hi.
18:43:19 Hi Mark.
18:43:22 Mark, I love you, but you’re not allowed to speak for minutes because we’re gonna talk about a PDF thing and you’re going to go off on one.
18:43:28 I’m gonna be fighting with Adam instead for a little bit. So Adam, the idea here is to keep the Webex our experience and the desktop experience.
18:43:38 Very, very separate but connected. So visual meta I think is extremely important because it’s the single most robust way you can admit metadata to a document because it’s the same layer layer as the contents.
18:43:52 I absolutely accept the notion that people will not want anything added to the appendix of their document. Course I don’t agree with it, but I accept it.
18:43:59 In my reader software, that is how it’s done. You have a basic visual meta and you also have the Temporary visual meadow of AI or XR information.
18:44:11 My defense would be 2 things. One, the robustness, but also 2, they can easily be stripped.
18:44:17 Just cut away the last few pages and nothing is lost. However, it is really, really important in keeping the headset and the desktop separate so that if you, for instance, wanted to write a webdab for your library, That stores the metadata.
18:44:32 Separately, that will be incredible. So your solution and mine can compete. That would be fantastic.
18:44:37 So that is why it’s so important we get this Jason thing web death thing whatever thing. Done so we can just out, you know, compete, say what makes sense for people.
18:44:51 But I’m not going to start storing things in resource forks at this point. And I don’t really want to put it in a database for the application itself only.
18:45:01 Even though someone will have to be there because, you know, they
18:45:03 So, where do we keep, where do we keep the library then? The library is not inside a single book.
18:45:11 It’s a totality of everything. And, So there has to be some sort of data store where you that serves documents, it keeps a list of the documents where they are in if we have different X or views.
18:45:25 So there there is data that is not. That does not have a home in a PDF somewhere unless we use a special library PDF for that.
18:45:34 But that’s quite convoluted.
18:45:38 Yes, absolutely. The way I look at it and I’m really grateful for this particular part of the conversation.
18:45:44 I look at this as being, as I said, 2 separate things and the funnel being really, really, really important.
18:45:51 In my world. The library is the user’s folder with PDFs. And the way that information goes to the headset is through this magic web dev, JSON encoded stuff.
18:46:03 And when it comes to the headset, the headset can be made by another company, they don’t even need to know about each other.
18:46:10 They just need to give each other permissions. That’s so, still, so important. So in my library, the metadata is stored for each single document.
18:46:19 So each single document has an appendix for the basics and then and again Peter for the millionth time I thank you.
18:46:26 There will be one single page appendix for one AI search for names. One single appendix for XR positions in space of elements and so on.
18:46:34 So there can be easily 20 pages of appendices. Unless you print them out, you probably will never notice.
18:46:41 But that is why if someone decides they hate breeder But let’s say through a dream, visual matter becomes the norm.
18:46:48 They can put it in someone else’s system and have lost no data. I really think that the data needs to be where it is.
18:46:56 Think he needs to be on with. So that is. Yeah, sorry, that’s my temp that announced.
18:47:04 Is that okay or are we ready for Ellen? I’m very happy to go a bit further, Adam, if you have further on this.
18:47:15 Yeah, I don’t think that’s a viable. Path forward for many types of information that we want to store.
18:47:22 About library collections and so on they don’t necessarily live inside a PDF they don’t have a natural place.
18:47:28 There, we will make, we will have rooms and spaces defined and they will, that’s data that has to live somewhere.
18:47:37 I believe and that and that needs to be We must find a way to store that in in a sort of a database or file or but it has to be stored somewhere and I don’t think.
18:47:50 Every PDF can know about the whole library. That’s on reasonable and more.
18:47:55 Oh, the complete I completely agree with you. And I’ve been fighting this internally for a long time where how do you refer to book in that library?
18:48:05 And one way is to store it as a PDF reference. It’s a bit pathetic, but that is one way of doing it.
18:48:12 Another way of doing exactly what you’re saying, having a database. I don’t mind that at all.
18:48:17 Right, what I’m talking about here is a library of PDFs for an academic user.
18:48:23 How we refer to other media. I’m extremely happy to have that discussion. And I’m absolutely not saying it has to be shown in the same way, but I think the use case here is primarily Academic PDFs.
18:48:45 What’s
18:48:48 He’s just gonna go.
18:48:48 Okay, alright. Yeah. That’s going off. So it is a known problem.
18:48:58 Except a problem. We have to decide. Whether we’re going to tackle that this year or whether we’re just going to say academic documents, nothing else for now.
18:49:06 If you want to tackle it, I’m extremely happy to be part of that dialogue though.
18:49:12 But you want to library, so where is that library store? They said, I think it’s a more.
18:49:19 This problem has to do with your project. If you want to library and some sort of organization of that library and not just a list from your folder.
18:49:28 It has, yeah, yeah. Sorry.
18:49:29 Can I put in here? Sorry, I just gotta, I gotta say this. Phrase very well. It does lead to some other stuff, but it deals directly with this.
18:49:41 I would. Request sincerely fraud that you think about what is the priority here. And I totally get that an app.
18:49:56 Or a interaction doesn’t feel real. Unless it’s with your own data. I get that.
18:50:01 But I also think that as evidence by the last 30 min or an hour. Seriously distracts what it is I think that’s at the core of what you’re trying to do.
18:50:14 And so I would, I would. Really think about you know even think about the creative selection in that book and the things that King Kostienda was not concerned with specifically in order to just get at how to type.
18:50:28 You know in in this little tiny device right What do we want to be focused on and and when I think about the grant and what I think about the nature of our conversations for the last few years.
18:50:41 It’s not about libraries, it’s not about live. It’s about How can we work effectively in this space?
18:50:50 How can we change our own paradigms, right? Now, I think channeling in live data is excellent.
18:50:58 As soon as possible, but I would really I really think it would be. Well, when you do prototypes like this, when you do spikes and sprints like this, you have to be laser focused.
18:51:09 On what is truly the most important thing and be able to sacrifice sometimes the most, you know, things that the babies right So to add to that a little bit more.
18:51:24 You know, I think that from anyone coming off the street and looking at this project. The first question is, why would I put on a helmet to look at a PDF?
18:51:34 Right? What can this new interface do that is 10 X beyond that will give me past the threshold or the barrier of putting a helmet on.
18:51:45 And then accessing a thing that is otherwise, you know, always there. The experience, the payoff has got to be so remarkably different, I would argue.
18:51:55 That it’s like the only way I can think is through this other medium. Right? I don’t think we’re gonna be able to get there if we’re thinking in terms of libraries and storage.
18:52:07 And even in terms of lists. Right, we have to really think. Quite a bit differently, which happens kind of naturally in our.
18:52:15 You know, weekly cadence conversations. But. For a lot of. Products that goes away immediately, the minute you have to start being practical.
18:52:26 The advantage of this grant is that it doesn’t have to be practical. As practical, right? It doesn’t have to make money, it doesn’t have to be industry ready, you can find product market fit.
18:52:35 And that’s pretty fantastic. So other stuff to say, but I’ll stop there.
18:52:41 Okay, so directly back on that. I do think the library is key. It will be nice to get a PDF in.
18:52:50 I really would like that especially to read with many pages or alcohol. Screen like for the inside all wall I would like to be able to extract the images But I really think, and this is very much back to one of Brandles very, very earliest things you showed us, you can have literally a million items in VR.
18:53:10 And it becomes manageable. So that’s why I think it’s the library because if you imagine
18:53:18 If you imagine someone like Mark who also has a lot of documents, let’s say Mark, forget Adam for a second because he has so many others just for the sake of this particular argument.
18:53:27 Mark says, okay, I want to try this system. And he opens into this new library software, or everything he has, and it does the basic metadata.
18:53:37 Adding to it, you know, author, title, date, that kind of stuff. Then he has an opportunity to go through the process we discussed earlier of adding further metadata, which suits him.
18:53:47 Might be names, might be something else. I really think that as long as we develop and I think we can do it relatively quickly some sort of solution just to take that data and just shoot it into the headset.
18:53:58 The beginning of working with huge volumes of text. And see how they connect. You know, just literally imagine having, you know, Washington state is one of the things on the wall.
18:54:12 Hell the map of America for instance. And then, you know, have lots of different things in the room.
18:54:15 If it comes manageable, I think we can see relationships we can’t otherwise. And the reason I’m selling this aspect of it is quite literally.
18:54:22 The use case of in a year. Or maybe at the Hypertech conference. We put someone’s headset on and yeah, they may start with our data because you know we’re not going to make them do that thing.
18:54:36 But it is data from a shared field. And there’s like literally, you know, they say my mind is literally expanded.
18:54:41 But are you using the term literally in all the ways? I cannot literally see connections. I could never see in any other medium.
18:54:49 Because the entire space is my workspace. And that’s it. I think that is the entire goal.
18:54:57 Do we want to have other media and PDF? Of course we do, Adam. Crucial.
18:55:02 But not for this first. I’m talking about, for instance, a case study could be the hypertext conference papers that Mark has worked on because he’s also made sure the metadata is clean.
18:55:12 Because in our field that is known work that a lot of people would have anyway. And then they can be shown how to later on have their own specific versions of it.
18:55:21 But I think the library will be more important than any other interaction we can.
18:55:29 And I do think that with the enthusiasm and intelligence of you guys, We’re going to find crazy ways to build the interactions there, you know, Brando had his crazy carpet.
18:55:41 You know, Adams had many, many ways of curving text. Once it’s there, okay, so I realized I changed what I’m saying.
18:55:49 I’m now not saying it has to be the end user’s document. But I’m saying at least for the demo it has to be in the end users field.
18:55:57 Right, that’s at least a basic beginning.
18:56:02 It just cannot be generic. I think we ran that. Yes, Peter, you’ve been very patient.
18:56:07 Please go.
18:56:07 Yeah, one thing I think we need to have is some sort of a globally unique identifier that we can stick in our documents.
18:56:16 So that we can use visual meta to indicate what alternate formats of the same document are and deal with the whole versioning issue.
18:56:24 And also have the ability to have some sort of standoff visual matter for instances where the underlying PDF is copyrighted.
18:56:34 We do not have the right to alter the underlying. PDF, but we still want to associate annotations with it.
18:56:38 So some mechanism there to be able to bind may just an automatically generate hash of an arbitrary PDF that we cannot touch.
18:56:46 To a hunk of visual meta in a standalone file, but it will be able to work with it in our system even though we don’t have the rights to play with the original document.
18:56:56 I think that for this at least if you have the document you have rights to do whatever you want with it.
18:57:02 This is for personal use. However, it just goes into what Adam was talking about, things that aren’t PDFs.
18:57:09 So if you have a PDF that is on someone else’s server that you don’t have the right to download or you’re talking about a book, we need a solution for that at some point.
18:57:16 Yes.
18:57:16 And also pay wall documents where unless somebody pays they can’t get a hold of the original PDF and we still want to apply visual meta.
18:57:24 To establish links and definitions and concepts and Do a nice little map view.
18:57:31 Right. Yeah.
18:57:38 I think Adam and then I think Alan took his hand up after Adam was getting ready to speak.
18:57:45 And what I was proposing was not to bring other kinds of media in there. But I like that, of course.
18:57:52 I like images and videos and sounds and yeah, other things but what i was talking about was the library If we have a library that there has to be some sort of representation, some sort of structure to that library.
18:58:11 At least visually, graphically, how do you, how is it, how have you, how have you organized it and that will be very different from a folder structure.
18:58:21 So that information needs to be somewhere on the And. So at some point we we have to decide how to.
18:58:31 Where the data live and how it lives. For some people it’s not unimportant, but but I think it’s quite important here.
18:58:41 And that can’t be inside a PDF, I think so.
18:58:42 Okay. I understand you better now. You know, this is the basic library that I’m experimenting with.
18:58:51 And and reader but there are interactions there such as these guys on top are pinned some are hidden So there is absolutely information about the structure of the library.
18:59:01 Thank you, Direct. I’ll keep thinking and Alan, please go ahead.
18:59:10 So, when I think about XR and what it could do
18:59:17 Yeah, that’s truly different. I think about something like this, right? This is a knowledge graph in curved space, if you will.
18:59:26 Right? It’s a not, but everything’s, these are the connections that would normally just be lines that are overlapping, but here, you know.
18:59:34 Of course it looks like a confusing mess when it’s on a flat screen, but when you’re actually looking through it, you can see.
18:59:40 Hey, the, I see how these things connect, right? This may not be the greatest example, but it’s something I threw together just for the hell of it, right?
18:59:49 And.
18:59:53 I. Would like. I think what’s exciting about XR and the sort of the screenlessness.
19:00:01 Illusion, cause of course it is a screen. Is this paradigm to get outside of, just getting outside the paradigm of lists and trees and 2D thinking.
19:00:14 And see if like this kind of thing is intuitive to your Einstein’s and your porn cares, right?
19:00:25 Hope I pronounce that correctly. This is how they think, right? Yeah, sorry.
19:00:31 Did you see your transcript? That’s brilliant.
19:00:35 The, this is, you know, they’re comfortable with, you know, sort of understanding space as being curved.
19:00:41 And surfaces and topologies, right? And right now it’s in the domain of esoteric.
19:00:49 Math, right? But I think it’s very likely that this stuff becomes pretty core to how we think because we have a track record of that.
19:00:56 You know, just a hundred, 200 years ago, there concepts that were arcane mathematics and now they’re part of our everyday life.
19:01:03 So Like I want to be on the race to getting to getting here with this. Strange. Undergirding, but
19:01:14 Yeah, that’s I don’t know if that’s part of the project, but. I think there’s.
19:01:23 A lot more that can be done to understand. How ideas linked together and this isn’t just entities and nodes.
19:01:32 So much. I mean, it’s a whole different topic to discuss what these dots actually are, but.
19:01:38 Yeah, there we go.
19:01:42 This is exactly where I want to spend our time, you know, getting the data in the XR space and building stuff like that.
19:01:48 Like Ted Nelson Zigzag as well is very complementary what you’re saying.
19:01:52 Absolutely super important. Peter.
19:01:56 Okay. I think it would be really helpful if We had an external file form. I have that sidecar file idea.
19:02:07 For the visual matter and I really would like to have The equivalent of a font DA Mover, so I could take a piece of visual meta from one file as an ordinary end user who isn’t familiar with the grammar and just drag that bit into another file.
19:02:21 And Let the system worry about the JSON formatting. And the bib tech formatting for me.
19:02:27 Otherwise, Only programmers are gonna wanna play with the guts to the visual metaphiles. And the the aim over model is just beautiful because you have 2 libraries that could have a copy of Fros library.
19:02:40 I could have a copy of Fabian’s library. I could decide that I want to pluck this reference from Fabian’s, this reference from frauds, stick them together in my library and that model of just being able to click on something in one column and click a button pointing to the other column to move it over to transfer that data and let the system handle the formatting and insulate the user from having to
19:02:57 deal with it. Otherwise, people are gonna look at visual meta and it’ll be the reaction of oh my god, that’s cold.
19:03:05 My brain shuts down when I see cold.
19:03:06 Peter, the end user is never expected to look the visual metad other than if they ever want to check something.
19:03:14 Completely agree with what you’re saying there.
19:03:17 Yeah, I put back a link that I should couple of. Weeks ago, or even I think made a I made a quick demonstration on yeah, I did make a very small demonstration with a PDF.
19:03:29 That had I think just the title on the threed page. Based on Kaleb web so Galeb is a popular tool on Linux to manage documents.
19:03:41 So that you can convert. HGML, whatever, things that are usually books like, but it doesn’t really matter so much.
19:03:51 So we can say documents taking a few shortcuts. And it shows them only in 2D which is the expected way let’s say to look at the library on a computer and I think what I was trying to evoke couple of weeks ago during the demo is indeed what does the library in 3D and eventually in XOR, look like and how it could be manipulated.
19:04:18 We can imagine that, reader or whatever. Software uses to manipulate the documents. Doesn’t just provide a web server, being web, web socket or HTTPS with whatever endpoint.
19:04:32 But provides calibre web but the modified version of it so that The bookshelf is a Webex or page.
19:04:40 With the content being many portable it’s very heckish definitely not the most optimal solution but i think in terms of Going from the discussion we’re having now to manipulating documents that are kind of on a shelf.
19:04:56 That’s the most direct pathway. I can imagine right now.
19:05:04 Right, so obviously we’re over time. Yes, lots of really important things came up.
19:05:13 I think maybe Adam. Your statement that a library is a thing it’s not just a list of documents is One thing really to.
19:05:22 Important to reflect on. So, Does anyone feel like having another meeting soon to just talk about?
19:05:32 Getting a few sample documents and then just developing a format for it or should we do that offline or is nobody interested or how should we deal with that?
19:05:41 I think it’s something we can Get done relatively quickly and that includes the atom. You know, library document or whatever we think it should be.
19:06:00 I don’t.
19:06:04 I would encourage, to think of. User interface or what we do first and work backwards. I feel today it has, we have started a bit in the opposite end to decide on the technologies and work forward.
19:06:21 And that is a recipe for catastrophe, I think. Or it’s as many potential pitfalls.
19:06:27 So I would encourage everyone to do napkin sketches or do a hand hand small speeches with illustrated with gestures whatever you feel comfortable with actually design what you would like to have.
19:06:48 Whether it’s picking books from your library or how you how your ideal see a way of annotating or or pulling a book.
19:06:58 Apart or whatever it is. How you would like to do that? Okay, that is the opportunity we have here here.
19:07:09 Usually you shouldn’t rip book support, but now in X or we have a great way of actually doing exactly that.
19:07:17 Throwing out pages away or duplicating pages or cutting out things and that is so we have a ample opportunity for for that and I would start that there to get something really in question or yeah.
19:07:32 I absolutely fundamentally agree, which is why I’ve been asking for the last year and a half for exactly that.
19:07:39 I think this is also good. A inspiration for us to do that. I think that in addition, we need to Really just a fine.
19:07:50 Cause all we’re talking about, let’s say. Okay, I’m going to say we’ve decided we’re gonna use Jason.
19:07:56 The library. And if anybody disagrees fine. But let’s say we have decided on Jason, all we need to do is define The basic structure of basic metadata and extended metadata.
19:08:07 Visual Meta, which is based on Bib Tape. Maps onto that quite well.
19:08:12 Right. And once we’ve done that. Then we’re beginning to have a reality for this.
19:08:17 And in terms of the user, those interactions, ignoring the user interaction with the actual stuff once you are in, because that’s, that should be 99% of this.
19:08:28 You know, I just want the 1% basic. I want you to be able to use. The library type software.
19:08:34 Whatever vendor, low still laptop. Or rather close it a little bit, put your headset on.
19:08:44 And to the environment, which is probably going to be a local URL or something, or maybe it’s launched from an app.
19:08:47 And you have access to that. That’s one we can really start to do what you’re talking about really, whether it’s a napkin or not.
19:08:57 It’s a reality that it’s In a way obscene isn’t easy and ready, but it really is also exciting and it shows how immature.
19:09:07 VR is for work.
19:09:09 But to do it, do I sense a slight reluctance to do the kind of imaginary work first here?
19:09:17 No, you’re not.
19:09:16 Where is it hard? Or what is it that stopped? Because we need to know what to do. Before they start to do it.
19:09:24 No, it’s not a, yeah, what you detect is a very strong frustration. Because I have written many scenarios and I’ve shared them in the journal and they’re in the book.
19:09:34 And in terms of feedback from the community there has been very, very little. And that’s absolutely fine.
19:09:40 Because of the time we work on. Right? The rhythm that we have. I don’t expect us to all comment on those things.
19:09:48 If people want to do that now, that is great because for one thing if you want if you get to a certain level let’s put in in the book I’m really really excited by that.
19:09:57 But I am concerned in the experience of our community of asking to think of scenarios. It doesn’t really happen and either, and even whether we do think of scenarios or not.
19:16:05 I know the recording was paused. We can on post now, right? I’ll just summarize in 5 s and give it over to you.
19:16:16 Yes.
19:16:15 So we just had a little bit of a discussion on to what degree to look at scenarios versus synchronization, what corp is to look at and we’re thinking maybe using the hypertext ACM documents as an initial bar corpus.
19:16:29 Fabian, please.
19:16:32 Yeah, I think, for example, using their hypertext corpus is interesting. Like I allegedly want to know more about it.
19:16:39 So for me that’s very good. Some of it, some metadata might be enriched or whatever.
19:16:46 Fine. What I think we need on top of that is also a user and a user who is not as passionate as we are.
19:16:55 Might not be as knowledgeable. As we are probably more knowledgeable about about other topics, maybe about hypertext itself.
19:17:03 Honest namely that when we go wherever in crazy land and it’s not efficient regarding their workflow to tell us basically.
19:17:15 And I would argue with some at least minimal experience of XR so that they were not like straight up saying, oh, a headset.
19:17:26 I don’t wanna touch this thing. But I think that would help us be pulled in the right direction.
19:17:31 And then there probably would tell us if it matters if it’s this or that format, this of that back end or the implementation, etc.
19:17:41 And for us then focusing on value basically.
19:17:45 Just a brief comment before I handed over to Peter there. The thing you saw earlier today about extracting names from a PDF.
19:17:54 I think this is a really important part of the process because the PDF in itself is quite dumb, obviously.
19:18:00 So I’m very happy to use Mark Anderson’s vast office. But you said. The or not.
19:18:08 If we agree on the basic formatting through which that would be communicated. You know, then we really have something because to only have documents where the title date and author name in XR is really limited.
19:18:22 To find what’s inside the documents. And that can be done in all kinds of AI ways. And it can be done in simpler ways.
19:18:29 Oh yeah, I see all kinds of head. Yeah, Peter.
19:18:34 Okay, from a technical perspective, I think we won’t factor out The formatting of the data.
19:18:42 From our goal of being able to transmit the data from the headset to the laptop and back.
19:18:48 So what I recommend is that the data packet being transferred consists of an identifier for a grammar.
19:18:56 And simple string with an attempt for why we’re sending this packet over and then a block of text.
19:19:04 Then the receiving system. Then consult the appropriate grammar and apply that parser to the block of text and of course from the intent string it knows why it was handed that data to work with in the first place.
19:19:18 That way, free to swap out arbitrary grammar specifications over time and it has no impact on the data transfer mechanism.
19:19:27 All the data transfer mechanism sending is a grammar name. And intending what to do with the data block coming in the grammar and the raw block of data.
19:19:36 And that basically will let whoever is working on the data transfer, the operating independently of whoever is working on our initial minimal grammar specification and add-on specification over time.
19:19:49 In this context, what?
19:19:49 Well, Peter, it’s a Yeah, we, You have to acknowledge that even theater teachers like me or should be able to work in this protocol even though I’m very technical.
19:20:03 So having kind of. Different grammar, and, it’s kind of, we’re leaning into computer science.
19:20:14 It’s kind of we’re leaning into a computer science if they’re, leaning into a computer science if they’re, territory now and the
19:20:17 Well, what you, what you’d see is an end user might be, this block is going to be the names grammar.
19:20:26 And our intent is to merge it on your side. And then we have the names which could be Jason or it could be YAML or it could be a markdown list.
19:20:39 Depending upon what the data format is. But the nice thing is it’s sort of factors that decision out.
19:20:45 You wouldn’t be thinking of it as the end users of course because the system would be generating whatever the appropriate format is.
19:20:51 But again, it lets the The protocol would be living independent of what the formatting of the underlying data is.
19:20:57 It would just be seen as a big hunk of text. The name of a grammar and then the software on either side would be using libraries and whatever programming language it was using to be parsing.
19:21:07 The text based upon that grammar. The grammar specification itself, a parsing expression grammar, absolutely makes the most sense because that gives us an unambiguous single parse.
19:21:20 Of whatever data we’re working with and That kind of a grammar formalism was designed to work with artificial languages as opposed to natural languages, so you don’t have the ambiguity.
19:21:31 Parching expression grammars are based upon ordered choice. There’s an order choice operator and whichever.
19:21:36 Matches first win.
19:21:35 Okay, okay, or, or, way over time. We just have PDFs. Right, that is the real world scenario.
19:21:48 See you later, Allen. The basic data here, it has to be, it, the grant was given for an academic audience.
19:21:56 And we don’t have access to raw data. So the PDFs are gonna be badly parsed and all kinds of stuff.
19:22:02 There’s going to be issues. So what we’re talking about, really. The 2 things, the document interactions with the document, which we haven’t really discussed today, and how to do.
19:22:11 How to get the stuff across. So I’ve decided to agree with all of you. I’m gonna pause really focusing on the transmission mechanism.
19:22:19 And we’re gonna start with one corpus and we’re gonna have that marked up as well as we can by Mark.
19:22:28 And then we can start to look at how to how to deal with that because Everybody who was ever done.
19:22:35 Oh, he did.
19:22:32 Mark, volunteer. Did you volunteer now, Mark? Throw to volunteer you. Wonderful.
19:22:37 I think so.
19:22:40 I’ll volunteer for designing parameters.
19:22:42 That’s the best, best kind of volunteering.
19:22:39 Yes. Yeah, well. It Mark’s already done it mostly so all I’m saying is They have been myriad ideas for doing clouds and stuff in XR.
19:22:56 It’s probably the most obvious solution for it. With lots of data points and all that. So what we’re really looking at here that I think is useful.
19:23:03 Is the basic metadata of a document. And on the client side, excuse me, on the on the computer side, desktop side.
19:23:12 For the user to decide which of the internal data is relevant. One way of being is just using AI to extract all names in the document just as an example.
19:23:22 And how we transmit that into the XR spaces of course part of our issue. Because then we can really start having a really rich multi-dimensional environment that is custom so that Peter may not want to have the same kind of things out of the document as Mark even though they’re the same documents.
19:23:40 And as if you if we get to the stretch goal to use that terminology of being able to highlight or annotate any of these documents when you open them.
19:23:49 And have that be back on your computer. Insanely amazing. But, but it.
19:23:54 So when you when you’re saying author, what the names or do you have a kind of a visual or an interaction that you and that pops up to you when you when you say that so if you have a PDF or a couple of PDFs with with lots of names floating around whoa or what would that be or look like to
19:24:17 Answer that with some principles. The principle is user interaction choice. Of selecting and choosing what to see and what to hide.
19:24:27 So that means that you go through the processes to analyze the document one by one or in bulk. And let’s say you extract a lot of place names.
19:24:36 You may then as the user in XR have a map on the wall. And they and you can have little dots on that map.
19:24:44 You may also choose to have portraits of any people mentioned on another world. And this is where I think it gets fun and interesting.
19:24:52 That because these are known to be something because when you extract names it’ll tell you this is a name rather than another kind of value and because you have an opportunity to validate whether you agree with this or not.
19:25:05 And of course there can be mistakes like with anything. We can start building really interesting interactions so you can, you know, look on the map, have it over here.
19:25:13 And then you can maybe have a time slider. You can see, you know, don’t show me this.
19:25:18 I’m sure that all these fun things can then begin to happen. But Mark, please.
19:25:22 And I was just really gonna say that, I, the current data that I pushed out has got all the data in there I do actually have I think it’s like I’ve got something like 2.8 A authors mapping onto 2.6.
19:25:34 Is it 2.6 K? Something like that. Actually, just, you know, people change their names and people change their names and, and, and, you know, people change their names, and people change their names, and, and, and, general, so, all these things we have to, we have to put in there.
19:25:47 I didn’t, the only reason the author stuff wasn’t out there is I I’m not clever enough to know how to I you know I didn’t know how it would usefully seem in.
19:25:59 I’m not clever enough to know how to, I, you know, I didn’t know how it would usefully seem in.
19:26:00 So I, what I’m very happy to do is perhaps correspond with, Adam and, and Peter just, will say, look, this is the sort of data I’ve got, how, meaningfully, Can I, for instance, connect it to what’s there?
19:26:10 I mean, essentially the database I’ve got has a 10 digit GUID figure. So I mean, I, everything at the end of the day, got a number attached to it and we can lash it up how we want but I’m totally open and you know within within what I can manage that in terms of getting to Tinderbox.
19:26:26 I’m very happy to try and mash it up in in you know, whatever format. Is useful.
19:26:31 The names are just the names as extracted from the original paper so in the early days there are people who had just have initials and then their names are lost lost to us in time.
19:31:39 It doesn’t get errors. You sort of think that what happened but Only left and right.
19:31:39 Right. Okay. I’m gonna turn recording back on.
19:31:45 So, and what we talked about when we weren’t recorded was a specific corpus of academic documents and how much we could take them into this.
19:31:53 So obviously this is a discussion and definitely we want to go further with this. And We still need a way for someone to have their own document put it into this this XR thing.
19:32:08 But I really found last week. 2 breakthroughs. One was the laptop software can be a server.
19:32:18 Hey, whatever. XR needs. But the other one was this act of manually using a, I analysis of a document to extract what I as a user want and have that be available.
19:32:31 I think that is absolutely crucial because in this amazing environment if it’s just what Mark has with due respect, because he’s only taking what A/C and gives him, it’s not enough.
19:32:41 The keywords are rubbish. Including my own keywords for ACM, you know, what do you write?
19:32:47 But if it’s extracted the kind of stuff, then we can start playing with the interactions.
19:32:50 So I can very easily see, Denny and I, of course, very involved in the A/C in community.
19:32:58 I can imagine when we present our workload based on next year. That it will be presented here and it will have at least this current year’s ACM documents in full.
19:33:06 That’s very reasonable. And we will have done these interactions from one user’s point of view.
19:33:11 And then show this is how they can do themselves. So that could be a good starting corpus.
19:33:16 Another work going on probably means that we may end up having, sort of basically, cleaned as in, no typos or missed detections, OCR, plain text of the documents, which is I think exciting for what we’re doing because That makes it a lot easier for us to probably do some of the interesting experiments we want to do.
19:33:36 It’s an accidental byproduct, but I’m, I’m working on that.
19:34:36 Current PDFs are okay. And we can get additional data from the source ACM if we want to.
19:34:44 That’s it.
19:34:47 Good. We need to wind down the conversation. I’m really, really grateful for your time today.
19:34:53 And, they spend real What is it, politicians say when they disagree? There’s been Frank in honesty.
19:35:03 Yes
19:35:05 I’ve got a dash. Before I do, there is a link in the site bar there to my, 822 paper that I only got access to an HTML form whilst going through the ATM tap system.
19:35:16 So you can’t publicly do this. But I had the presence of mind to grab a copy. I put it up there.
19:35:21 So again, it’s just an example of what we might be getting to. Anyway, I’m afraid I’ve got a hard cut, so I mustache. Thanks, everyone.
19:35:26 I’m sorry.
19:35:24 Yeah. Yep, yep. Say a letter, Mark. Adam for your earlier question.
19:35:32 Of course, the videos going up. The chat always also goes into a blog. So that will be available on Future Text.
19:35:38 Okay. Nice.
19:35:40 If I don’t if you don’t find it, I’ll try to email you if you don’t.
19:35:44 Point to third again.
19:35:48 And, have a good week. We will think about things until next week. Let’s see how far we get.