Frode Hegland: Right? So, yeah, for the sake of the recording Fabian was just talking about the importance of a grammar and like a snapping grammar that he has with his post-it notes and the environment. I was spending some time earlier today thinking about our work, and I came to the thing that. I’m just looking at my notes. The most core thing about text is its insistence on context. No word means anything by itself. You know, you got to know where it was uttered, where it was written and what’s around it. Right? So when you’re talking about a grammar, Fabian, that’s really, really important. And in the real world, imagine putting down a glass on the table and having full freedom. You’re going to be spending way too much, way too much time to make sure it doesn’t spill. So the notion of snapping in XR, I think, is really, really important. You know, this is something we’re going through with Andrew’s work. We’re in the beginning. You could do anything with the text. You picked it up and throw it around. So for us to decide on the right, let’s not call it constraints. Let’s call it scaffolding because that sounds more positive, right? But the right kind of scaffolding is important. And I love your use of the word grammar, Fabian, because. When I was working with Doug, he didn’t really use the term grammar. I kind of put it on him, I said, because I don’t know how. If you all know Doug’s command language, it was based on you write a sentence. It wasn’t like dos where the the command seem arbitrary. It was logical. It was like, open this document named so and so and then colon.
Frode Hegland: Oh meaning outline view. So you had all these options. You would learn after a while. So you could really quickly do things. And I said you have a full grammar. And he did agree with that. But. Andrew, one of the things we talked about earlier today is the next volume of the book, and I really want us to have the notion that the book is quite separate from all our software and the lab and all of that stuff, and what we should try to ask people to look beyond what they think they know. Right. And that really comes into this. You know. How do we even. Try to develop a new grammar for what this might be. You know, I was I looked at, sorry. My the vision was precarious on the bed. Don’t want, do not want that to crash in a meeting. Physically crash. So what I want us to try to invite people and I’m going to give you like. Actually, Fabian, could you please repost the link for Andrew of the. Why am I asking you to do it when I can do it just as easily as what my wife would say, right? To go beyond what we think we know. Right to push people a little bit. You know what in their field can they know? And what annoys me, I the Michael Benedict cyberspace book, there’s a lot of great stuff in there. But what annoys me is that the notion of XR, even though they talk about having a point of view in the images and things they do, there isn’t really a point of view.
Frode Hegland: You know, that’s kind of the in a sense, the human is missing. So what I’m saying and what you’re doing, Andrew, and what Fabian is doing, it’s both trying to figure out how things should quite literally snap together. And that is an incredibly important discussion. I don’t think any of us would have had that discussion until we actually tried these things. You know, it’s it’s been amazing. Anyway, that was just my little monologue there. Actually on the whole snapping thing. Because I had this breakthrough with author being listed in the macOS App Store on page one. My download numbers are obviously considerably better, so that means I’m telling my guys now I’m spending a ton of money, relatively speaking, on them. Because author is now iOS and visionOS as well. And I’m telling them to do things like, we don’t have the facility to deal with citations and iOS or visionOS. Yes, yet. So they’re now writing a protection around the citation. So on a mac, you click on the citation, you can edit it. It’s all cool. But if you do it on iOS now, you can edit the text inside and it breaks the whole thing. So we have to have a little protection. So we’re doing the important thing. But it is really shocking in a way how much tweaking has to be done, even for the visual feel of it. You know, just like. Yeah, it’s it’s interesting. So one more thing between you guys, Fabian and Andrew. Conceptually, where are we with sharing some kind of information? Jason Vishal Mehta, is there any is there anything useful? Fabian, at this point that could go between the environments?
Fabien Bentou: Hello. So the most pragmatic way, I’m just going to So the last season. I had it on the link. I guess that was this. This doesn’t work. In. So this link doesn’t work, but I’ll still share it. Because that. Maybe it’s an example I shared here. But what I do have that did work, and I think I have a branch for this. I guess called citations. Three weeks ago. Yeah. Okay. That’s the comet. And the branch itself with the. It’s a bit of a mess. But anyway at least it’s about. Correct. It’s at about that spot. So I made a decision. It’s not necessarily the best one, but it works, so at least it’s not an incorrect one. Is to use a csl JSON, which is the way basically to get some some citations or at least a list of documents, a library of, for example, Dois or titles or anything, let’s say. So it’s csl JSON. Yeah. I’ll put the. Link in there. It’s because it’s specific to academic citations and it’s JSON, which is easy in JavaScript to read and send around. It’s also very small. I so I have the example on my WebDAV server because that’s what I use to read and write from. And the example for example would be this. And I believe if we check. Yeah. Perfect. So if you check especially I’ll try again to share my screen and please let me know. If it displays anything.
Andrew Thompson: Yep, we can see it this time. Yeah.
Fabien Bentou: Super, super. So we have to clarify. It’s a web server and you can have files on it. So that’s super exciting. But you can save file on it, which is a lot more exciting. And you can save text files like this, a test, or you can save a JSON file. And here that JSON file has been.
Speaker4: Up and.
Fabien Bentou: With for example, I cannot solve alone by innovation, including dois and whatever, but what’s the little trick that I used is that there is The position here, I think. I’m not sure if that’s the right version, actually. But anyway, it’s fine. The point is, I added the position in space like this. Citation should appear at zero. So right in front of you on the x axis, two meters up on on the y axis and half a meter forward. So, like, right in front of your face, basically however you display it is.
Speaker4: Do you use.
Fabien Bentou: The title, the abstract field, the position I think she should be. In this supposed to be in a common section, so that it’s not rewritten over by another tool that’s not aware of X or data. But I think maybe positioned there right there was okay I forgot. No. Okay. Sorry. Here you can see at the bottom. So it’s a little bit of a mess. But I did not plan to present this. So here you can see the correct version. Namely, that note is a field in CSS algorithm that you put whatever you want in it. It’s not officially supported feature. For example, the property page is officially supported. But position is not. So if you open Zotero like I’m doing now, at least I’m trying. Let’s imagine that I did it. They will. Do. If I were to export this and write position as I did there on the left, it would delete it. But as a note. So if I take here a random paper this one might not have a note. I guess in this one, if I take the same one I can search on Kobe. No, it’s not in this one. Hello. Evolution of combinatorics in compositionality. And if I go in extras, I got this field. And this field is the position. So right in front of my face on x 1.8m. So about high level for me and half a meter forward basically. So this data is the data that you Andrew can read back and that software like Zotero are not going to delete. They might not know what to do. Like here they just display it. But that’s let’s say the trick. So yeah, that’s my pitch for using CSL, JSON and optionally WebDAV. But yeah, any URL could be as long as there is a JSON, a csl, JSON out we could exchange this way. Does it make sense?
Andrew Thompson: Yeah. Kind of. So I’m curious why CSL JSON as opposed to just JSON, which was what we were doing up until now.
Speaker4: Yeah.
Fabien Bentou: Because I did not want to reinvent a format and Zotero, which is open source and more popular than any tool I’ve done supports it.
Andrew Thompson: Okay. Yeah, we’ll have to like. Because I’m just developing what the group wants me to. So if the group wants me to switch from JSON to csv, JSON, I could put development time towards that. So I’m not sure if there’s a.
Speaker4: Is a.
Fabien Bentou: Type of, you know, it’s there is no if you just, like, save as JSON or load the JSON file. It’s compatible. It’s it’s it’s I don’t know the technical name. It’s it’s a format in JSON with a sub grammar of JSON. I forgot the name, to be honest. But technically speaking, saving and loading JSON as csv JSON will work. What I mean is those fields, let’s say abstract title list of documents has been specified so that the different software that supports CSL will want to work with. Just I made a decision for this one just because it looked like it encompasses usage.
Andrew Thompson: Yeah. That’s good. Yeah. As long as we match our like, our file names for things and put them in the same places, we should be able to, to have data that’s read by both software in different ways. Which is. Which is great.
Fabien Bentou: We can also agree on what’s like minimally acceptable. Like we don’t have to support CCGs in its entirety. Maybe title and the position in extra is enough. And in the future, if we want to support more than this, like, I don’t know, page number or maybe, for example, here, I just support the position in space. So let’s say there. But I don’t support the orientation like this to me. And this is the same, which is obviously wrong. Because yes, if I’m here and there, it should be oriented toward me, I guess. So I don’t support this. I assume it was like that.
Andrew Thompson: The software is always going to orient it towards the camera, which is we can.
Speaker4: Do.
Fabien Bentou: This, we can also do this. But let’s say to have a citation in space that both our software solution support, I would argue through this, the bare minimum is there.
Andrew Thompson: Bro you’re still Muted yeah.
Frode Hegland: So my question is obviously to do with visual meta. This kind of any kind of JSON could just be stuck inside a visual meta. So that shouldn’t be a problem. And you’ve now explained Fabian what CSL means. So Andrew, if you just have a look into CSL and if you think it’s reasonable to use this subset specifically, that would be very interesting now.
Andrew Thompson: Yeah, we we can use whatever we want. The only thing is we already developed it in JSON. So, like. I can. I can change stuff. That’s okay. It just depends on what you guys want me to put time towards.
Frode Hegland: Well, how significant. I mean, please spend a little bit of time having a look at it to see if you believe it’s a significant effort. And then we can look at prioritizing. I think Fabian is saying that it should be not that bad. Right. So but under that’s a call for you to make when you actually have a look at it, see how it fits in your world.
Andrew Thompson: Yeah, I guess it depends on how we want to like what specific aspect of the project we’re using it for. Because right now we have Jason used in two different ways in my lab, and we can change either of them. One is the library, which I think works quite well for saving all the information to the library.
Frode Hegland: Right. So.
Andrew Thompson: Well and then the other one is the saving and exporting of the.
Frode Hegland: So I think the workflow that we want, this is what I want. And I think this is what we all want. Please tell me if you disagree is again starting out in reader because that represents the outside world. And you’re going through a document and it’s a bit much. So you want a wider view, you click a button or something it sends to Safari in vision of course, whatever browser is available. Something, and that allows a webXR rendering to happen of something. Right. Obviously important questions here. Is it the full PDF? Is it The HTML version of the PDFs, the content, so to speak. You know, these are important questions. And once that is done, the person works on this wider space, multidimensional space at the end of the session. To be able to save that, send it back out to reader. So a new visual media gets created that all it has is the JSON, CSL or whatever, so that the next time someone goes into webXR from that button hit the layout is the same. That’s the kind of thing we’re looking at, right? At least.
Andrew Thompson: So you want to save it per document. Now? I thought we were saving per library.
Frode Hegland: Yeah, but.
Andrew Thompson: There’s been a change.
Frode Hegland: No, this this would be an addition. And the library turned. The way I see it is that the library? Jason stores two things one a list of items in the library. And there. Externally accessible metadata, meaning how to cite them. The reference section, that kind of stuff. So we can have a citation tree right. And then you have.
Andrew Thompson: And that’s that’s correct.
Frode Hegland: Yeah. That’s exactly. So then in addition to that, what we should have is the. Yeah, yeah. Okay. I’ll just keep calling it the Jason. It really doesn’t matter. But the the information about the spatial layouts. And this is where the discussion that we had with the Andrew, Ummi and Dini where Dini got a bit concerned about the timeline came up because what I’m talking about is at the back of the document, it really should state what its view is in Zor, whether it’s in your build or in Fabian’s build or something else. And A notion for that would be something as simple as put the reference section on the right, and then I want a small window with an abstract here. You know, that kind of stuff. Because.
Andrew Thompson: Okay, so. Sorry, I didn’t. I didn’t know you were so good. No, no, no.
Frode Hegland: That’s why I’m covering my mouth. So you can see that I’m shutting up for once in my life, okay?
Andrew Thompson: I’m gonna, like, basically say it back to you kind of, and you can verify stuff is interpret it. It sounds location. We no longer care about having multiple documents relative to one another. We just want one at a time. And that saves as meta visual meta at the back of the document.
Frode Hegland: No. Very important point, I think.
Andrew Thompson: What part got I got wrong?
Frode Hegland: I think Danny, who is our main academic use case, would like more than one document open.
Andrew Thompson: Right. We can still open multiple, but it sounds like you just want to save one document at a time. So if you load in another document, they then sort of overlap or replace one another because you’re saving it. If other document data on a document. That doesn’t make a lot of sense.
Frode Hegland: This just goes into the.
Andrew Thompson: We can save it as a workspace.
Frode Hegland: Yeah. This goes into the discussion we had in the beginning of today of kind of the grammar of a space. You know, this is something we talked about before, way before Sloan, of the notion of where is something in space, whereas a second thing in space. Hang on, I’ll just address that when Brundle’s here. By Brandel.
Speaker5: Right? So.
Frode Hegland: Your timing is impeccable. We’re addressing a special issue, and that is Fabian showed us amazing new work. And it’s accessible. Fabian, could you please put a link to that again in the chat so Brandon could try it? And we’re not talking to Andrew about the whole Jason flow. And the issue is that we have two almost competing notions. One of them is a Jason of a library, which contains the list of the documents, plus basic stuff, and then the structure of the library where documents are in relationship to each other. We also need to have. Json for each document. So let’s say you open up a document, you put abstract there references there. You have a space for it that should be saved when you leave the space and append it onto that document itself. Right. So then the question Andrew just brought up, if you have those two types of environments, what happens if you open two documents? Will they then just overlap? Which is a very interesting issue. So what? You probably. Yeah. Fabian, please take over.
Fabien Bentou: Yeah. No, but just very briefly. First I’ll have to go in five minutes because initially we’re just the two of us. So I kind of changed my schedule a bit. Yeah. Well, life. And second, it’s a different question when when fraud you ask before like ten minutes ago, you ask like could those two solutions, let’s say my virtual world and what Andrew’s has been working on talk to each other? My answer was straight up yes. And I gave one solution through the suggestion of how this could be now, what they would do with this data and how did you would be interpreted is still completely open to interpretation. And what’s the right way to do this? But as soon as at least we have one format that supported both ways then they can quote unquote, talk to each other. And I would argue if that can be done with minimal or at least no change from both side, then we can start after a lot more about like, yeah, how do we collaborate? What’s the data? But I would personally prefer like we just display the title with the in the right position. When I change in my software and I save it, then Andrew reloads it and see the result and vice versa. Only after this then yeah, we can discuss about what to show, what not to show. But yeah, the bare minimum is one item in place. Moved, saved, reflected in the other solution.
Frode Hegland: The thing is that
Andrew Thompson: Yeah. Right now that’s been. Sorry, Fred. No.
Frode Hegland: No. No. Please. Under.
Andrew Thompson: I think there’s a definite delay between Japan and here. Shoot. What was I even saying? Yeah. So what Fabian’s talking about right now. Since we’re talking about grammar is called a workspace in my environment, it’s it’s the layout of what’s in the space and where things are. And then there would be the library, which is separate. And now we’re talking about the idea of putting visual meta in each document. We’re just not sure what goes in there, if that includes workspace position data or if that is just bits of information like, hey, I appended a note to this. Is that is that correct?
Frode Hegland: Fabian, do you want to take that or shall I take that?
Fabien Bentou: Yes, very briefly, because I would argue with all politeness. Sounds like it’s not my problem. Like whatever you do on your end of the solution, however you display it, whatever format, if you display a potato per page. Yeah. I’m also fine with this, as long as whenever there is a change that is meaningful, let’s say moved in space, I get an update like the save file in a format that I can interpret like the one I suggested you. Yeah, you.
Speaker4: Can do.
Fabien Bentou: On your side, whatever you want. It’s like as long as there is this minimal understanding, let’s say a csv JSON with a URL and using the extra field. And I’m not again, not lobbying for this, I’m just putting that out because that’s the one version that was the quickest for me to implement. I proved, quote unquote, that it works. And I gave an example, but it’s no more than.
Frode Hegland: So, Fabian, that leads to the question before you go. Do you want to be able to send information about one document to our environment or shared environment, or do you want to also be able to send a workspace of, let’s say, 2 or 3 documents in space? Do you want to be able to send that as well?
Fabien Bentou: This is the case now, namely that I send the library a CSV. Json is a set of documents and hijacking the extra field to have their position in space. So it’s it’s it’s a set of documents with their I mean, it’s a set of metadata about documents, including the new extra field for their position in space.
Frode Hegland: But when Android receives this, he will clearly know inside that that there are different documents. Right. So they’re not glued together, so to speak.
Speaker4: Yes.
Frode Hegland: So you are, in other words, sending metadata about specific document plus metadata about the workspace.
Fabien Bentou: The position of each document within a workspace? Yes.
Frode Hegland: Okay, so that means that logically Andrew, in your library, the workspace, is that just an aspect of the library? Then it is right.
Fabien Bentou: Here, let’s say we can imagine there is a single workspace. And yeah, there could be there could be more. But the point of that earlier demo the one shown today, but the one that I linked on slack a couple of weeks ago, actually was to have a single set of documents within a single library, within a single workspace. Because I would argue once that’s been done, to do it multiple times, you. Quote unquote, just abstract it away. And I would say it’s relatively simple, and I would suggest not to think too much about it before a single exchange has been made, because, yeah, we can declare multiple libraries and multiple workspace. It’s just like I didn’t have the time to do that yet. I think in term of exchange, as soon as as long rather as the format is this example of format is agreed on. We can do more, but I would not argue to do more before we do this. Like minimum test to exchange.
Speaker5: Okay.
Frode Hegland: Will you be able to join us on Wednesday? I’m not sure if he heard me. Fabian, my connection is not great. Will you be able to join us on Wednesday?
Fabien Bentou: I’ll do my best. I’m not sure I’ll let you know.
Frode Hegland: Okay. Because this is a very, very important topic, obviously. And the fact that Andrew is integrating with not just my place, but two places, that means we have a much better opportunity for making something that’s not making some obvious mistake because we didn’t think about it.
Fabien Bentou: So just to argue, I really got to run. But here, it’s not as if we were, like, declaring 20 new industry standards. I think doing a couple of back and forth through usages, through tests of, like, what’s the library where we move it? I think it’s safer. And I ran. Sorry. Bye bye.
Speaker4: Thanks.
Brandel Zachernuk: Yeah, I’m with Fabian. That having the the concept of messages being passed back and forth between a couple of different conceptual frames of reference is a useful thing to commit lightly to and to have a flexible view of its form and function so that you can iterate it, and also so that there isn’t a full expectation of full compatibility out of the gate. Yeah. Being able to just play with two things, interacting with each other without necessarily having to be equivalent. So yeah it seems like that’s probably the more likely outcome in the near term. Anyway. Different apps aren’t going trivially, aren’t the same.
Frode Hegland: Yeah, absolutely. I’m not sure if Oh, yeah. I don’t think. Fabian. I’m just going to give you a link here. Brundle. Okay, so this is what Fabian has been working on. And it’s mostly self-evident. But a really cool feature is that if you grab a post-it by the top left hand corner and put it onto another post-its top left hand corner, they lock snap next to each other, which is really useful. So we were talking earlier about a grammar, also a scaffolding of how we interact. I’m going to text you that link as well Fabian. So when you’re in the headset, you just click on the text for me so you don’t need to. It’s got automatically there. There you are. Right. So. But because the thing is. Oh, yeah, a brand. I don’t know if you saw, but the author is now on the front page of the macOS App store in the US. Not here. So that’s nice. So now I have significantly more downloads, which means I can afford to pay a bit more for programing, which is nice. So if thought experiment for me or thought issue for me is I can do a lot native. With my app for someone to click a button and reader to want to go to webXR, it has to be really compelling. Because of course you click a button and what happens? You go to Safari. Then you have to click two more buttons. And that is actually an effort in the same way that people, even on a normal web, they don’t always click on a link because the loading time is an effort. It’s a barrier. So that’s why the discussion we’ve had on what’s the point of XHR is so crucial. And the point has to be to use the space. So what Andrew has been working on, I think is really, really perfect to use that whole space. And that’s why I’m so grateful for the discussion today, for not just have a default loading, but to have things in a more.
Speaker5: Mainline for.
Frode Hegland: The end user. Useful placement. Six. That’s what we’re talking about a bit earlier. I mean, okay, just for fun. One thing that. And this is just to to run past and see what you guys think. Imagine you’re reading a document in the headset and reader. And you want to see how it relates to other documents citation wise, right. Which is something we’ve looked at before and Adam and Mark has built. So you click a button, it opens in webXR. And because. Okay, here’s the thing. Andrew, this is really important. Sending something to. Are you there, by the way? My connection is a bit weird. I can’t see you move.
Andrew Thompson: The connection is not good. I can hear you. Mostly it’s cutting out occasionally, but I can understand you.
Speaker5: Okay? No.
Frode Hegland: It’s just you were frozen from my view because of my incoming. But is it better now that I’ve removed the whole video thing?
Andrew Thompson: Honestly, not really. But I can still understand you.
Frode Hegland: Shutting down some other services. You know, being in being in Japan, high tech being in the village, not so much. Right okay. So here’s the.
Brandel Zachernuk: Vignette out quickly. Yeah.
Frode Hegland: Yeah. So so here’s the thing that I realized that’s really important. When you when you receive into webXR Jason. It is not just the data, it’s also the view spec. Just like Doug Engelbart had. When you open a file, you specify a view spec meaning the in reader and equivalent apps, you should be able to click on a button that says open to see Citation Tree. Or open to see easy read layouts, etc.. So when you receive it in webXR, you know that the user doesn’t want to see this just in the default view. They want to go straight to see the citation layout or whatever it is. Does that make sense?
Andrew Thompson: Yeah, that makes sense. You can put any sort of data you want. Assuming we’re still using JSON. Having a view mode is is a good idea.
Speaker5: Okay.
Frode Hegland: Good. So that’s probably once.
Andrew Thompson: I figure out how to get the troika text rendering better, we can start working on having actual like the document content itself rendered, which I think will be a big step. It’s just that right and right now that would just destroy any sort of performance.
Speaker5: Yeah.
Frode Hegland: I’ve been talking to Fabian about him being able to spend significantly more time helping you on that particular issue. So if that all works out, if he has the time, as he hopes he will, that would be good.
Andrew Thompson: I’m sure I’ve stopped to look into potential options. Maybe some way to, like, pause the draw calls or something? I’m not really sure.
Frode Hegland: Yeah, that’s that’s above my pay grade, but. Yes, that sounds good. So Some of the takeaways from today’s calls are the notion of grammar. Fabian used it in a way that I wouldn’t. I’m very glad he did. And so now we’re talking about a grammar of opening. You want to be able to say open this as. I think that’s going to be really, really crucial because from my design philosophy, every time the user has to make a decision that adds to their fatigue. So if they can instantly say what they want and not have to open another environment and then respecify it, I think would be very useful. I think that means we can easily extend it for, for instance, someone can be in reader come across and the name Brandel Zachernuk. Sorry for pronunciation, Brandel. I’m sure your last name. I didn’t say it perfectly. Select that and then in a menu item, click. Open this in webXR and show me what this guy has worked on, for instance. So it’s not just a document that’s open. So that means that it must know your library, but that gives the search function or the view function for that corpus.
Andrew Thompson: Is the question there would be, does it do this search first, and then generate the library that contains all the search results, and then send that to the webXR environment? Or does it send the entire library and then do the search in the web browser?
Frode Hegland: The way that I think about it and Brandel, please contradict me if you want to. I see the webXR as being a powerful, free standing, full application. I think in a few years it will be. So I would like all the data to be there all the time. So it just sends the command and through whatever means the data is either sent or already there.
Andrew Thompson: I could. I kind of see it as like a viewing space. So if you’re sending like if you do a search, the way I see it, you would generate a library in reader. Now that has whatever results you want. And then that just sends the library over. It seems a little strange to me to make a specific search function that would only ever be used by, like, one software, but maybe this would be more universal.
Frode Hegland: What do you think, Brandel?
Brandel Zachernuk: I think two things about this. One is that what the webXR app is doesn’t have to be contained in just the webXR per se. So it could be that there is some other process or system that is backing it and kind of fielding data and things. If there turn out to be Computational issues and ceilings that people are hitting on a on a device that’s necessarily constrained in order to manage it. And so that’s something to be able to to use if necessary. The second is that when you have a negotiation between different applications, different runtimes, they’re they’re actually different value systems. In the same way that Fabian was using You say protocol? No. I sent a grammar. I would say that there are different protocols in the sense of almost a human protocol. The sort of ongoing negotiation about what kind of what kind of information is available and on what terms so that so that there could be subsequent kind of asking and answering for more data as the two two platforms talk to each about other about being able to develop a composite view. And hopefully that doesn’t sound like waffle. The idea that, yeah, like if you invoke the construction of a view on a, on a, on a webXR side that that it can then make multiple queries back to the other system to go, like, do you have anything about this kind of way of representing things? And if it says yes, then get that. And if it says no, then it then then it just sort of makes it up or whatever, you know, so that’s that’s, I think, a constructive thing to do. Does that make sense? Andrew, am I just.
Andrew Thompson: Yeah. Like conceptually, that makes sense. We’re definitely not there yet with the project. Oh, no. No, no no.
Brandel Zachernuk: And I wouldn’t I wouldn’t pretend to aim to be there in the foreseeable future. I just mean, that’s actually my just a long, long term vision for what inter-application sort of communication needs to entail. You know, the trusted application boundaries where you just sort of let them act as proxies for each other’s execution is inappropriate and incorrect. It’s not going to take us past the next couple of years of Of trust and safety breakdowns. Everything needs to be treated as though it’s as unsafe as the web. And so we might as well just sort of so but yeah, so, like one of the things that we can do to prepare for that is to actually think about what we want to do with consuming these heterogeneous input sources. So if Fabian system is different from yours, then that’s actually kind of good, because it means that you get a chance to think about what those conversions are. And if, if you want. That’s one of the things that universal scene description USD does relatively well is that it has reasonable closure over the. Obligation that that you as a consumer of data that you only partially understand, have to pass back that data that you didn’t understand or care about. So that you do what you want with the things that you can, but then you give stuff back in a way that makes it renders it recognizable. The help that USD per se is supposed to give to you specifically the developer is not there yet, so don’t don’t worry about having to integrate it. But my goal over the next year is to make sure that USD is clearly the right choice for being able to manage that kind of thing. So it’s not a definitely the definite in terms of what kind of role it can play for you. But yeah, it’s it’s a goal. So yeah.
Speaker5: Right.
Frode Hegland: Oh, video is back on.
Speaker5: Nice.
Frode Hegland: I heard you at the end there at Brandel. Are you talking about that for? Even for something like sharing between Fabian’s environment and what Andrew’s building?
Brandel Zachernuk: Maybe it depends on what kind of help one gets from USD. I fundamentally something like a plain text visualizer, a plain text representation like USD or binary. Is in principle not that different from JSON. In terms of the fact that once you have a robust understanding of what you mean by anything, you can say it in any language. What USD has JSON doesn’t. Is liver peas. This strength ordering and inheritance hierarchy kind of schema, which is similar to the CSS cascade. You know, the way that Cascading Style Sheets work is that they use the first letter to mean this inheritance hierarchy, cascade system. Otherwise it would just be style sheets. And so it’s an important concept in computing. And Jason doesn’t have that. Jason is obviously it’s not it’s not opinionated about what it is that it’s conveying. It’s just opinionated about where the braces go and the double quotes. So so you could do something USD, like with JSON or you could borrow philosophically from some other resolving resolution techniques, resolution. And so when you settle a fight or not, how many pixels are on screen. And that could be fine. And it might not even be for USD, but that’s, that is precisely the kind of job that USD was built to solve for Pixar. It’s just that it was primarily for computer graphics building, you know, shows like Moana and Big Hero six and stuff like that, rather than for passing around media about Media about hypertext or media in an academic context.
Frode Hegland: Okay. So that was very useful. I’m going to walk through something, Randall, and tell me if it’s appropriate what you’re suggesting, and also tell me if you agree with the use case. It’s based on the notion that in an external piece of software, you have some buttons that will send stuff. That’s what I’m going to call it, stuff to XR, right? So it will be. Library data. Document maybe. Document data. Maybe or both. And it’ll be the positioning and also an action, a view, an initial view. One button. I’m trying to be as simple as I can. One button when reading one document should be open up in webXR and there’s a default view. Similar buttons should be in the library. Open this library in webXR and I’ll do that too in the default view. Additionally, you should be able to do the thing we just discussed, such as open up in library as a citation view with this document in the middle. That’s that half. And then the other half is sharing between what Fabian is doing. Where? Let’s say you’ve been working on a document. You got all these amazing views of it scattered all over the place. That information you should there have a button, so to speak, to send it to Anders webXR. Where? It opens and then under immediately said, what happens if you already have a document in front of you, so to speak? You know. How is it seen in the workspace? And I’m wondering if. If you come into a new environment, the thing that you came with is selected. So you can easily move it into a position. So if it happens to be overlapping something, that’s okay. You just do this and there it is. I don’t know how should we deal with that, guys?
Speaker5: Well. Go ahead.
Andrew Thompson: Okay, I’ll be quick. I wonder if fundamentally we’re looking at the the concept a little bit off, instead of it being like, we have to make sure my software is compatible with Fabian since we’re both kind of just testing stuff. Maybe we should be, like. My software is compatible with the reader export when whichever form we decide that is and Fabian’s software is compatible with the reader export, which is the same thing, right? Like that means ours is now compatible. But instead of looking at how do we get his and my software to talk to each other, we just look at and we agree on some structure that we all reference. And then it can be both of ours work with each other, reader works with it. Any other tests? As long as the tests are built off of that structure everything will just kind of be able to send information to one another.
Frode Hegland: Okay. So yeah. So what you’re saying I agree with that. Yeah. What you’re saying is that. It’s not necessarily the same. Thing in all these different environments.
Frode Hegland: You know, the reason you would use these different environments is to do different things in them.
Andrew Thompson: Yeah, but the data is still passed in the same format. Maybe they send different amounts of data. Maybe they don’t care about certain bits. So Fabian’s export doesn’t care about I don’t know how like where the citation lines are, right? He doesn’t have lines. Whatever. So he just doesn’t include that data. But whatever the format is still carries over. That’s right. Yeah.
Brandel Zachernuk: And yeah. So you have you have sort of. I sort of think of it as two different questions here as well. It’s like you have you have your own application or runtime or workspaces scene graph. And that’s a you know, the scene graph has always been and probably always makes sense to be thought of as a as a pyramid, as a tree, you have a root and then you have things as children in it. And and then a leaf or some node of that same graph can be this imported data from one place or another. And depending on the constraints that are applied based on its presence, its position in a scene graph which could be negotiated, you could be like, I want this here, which corresponds broadly to this, this node or I, I want it to be down in this hierarchy, down onto where the table is or whatever. And that could change the representation, the appearance of something, so that you have in the same way that, like CSS today, is responsive, in that if you have an element and it has a maximum width of this, then you want it to go to three columns, and if it only has a maximum width of that, then it goes to two columns. I’m sketching things that are beyond the scope of what Andrew can produce with the Sloan grant.
Brandel Zachernuk: My apologies for that. But in, in abstract, like the, the idea that. Yeah, you not only is that data different between applications, but it may be different between different parts of your graph depending on where you want to put it. But it could also be live potentially. So rather than an export, what if you one of the cool benefits of this kind of philosophy is that once you have once you have the idea of a, of a live stage versus an export, then that can remain up to date in real time irrespective of heterogeneities in the visualization between the two to the two worlds, the two places or even two parts of the same scene graph on your own place. So you could have two paragraphs in different parts. You, you can have a live copy of two paragraphs and in your, in your one document, and one of them is constrained to be inline like a paper page, and the other one is blown up to be sort of parsed in visual syntactic text formatting. But they are the same paragraph. And so as you live updated then it changes. I think that’s not unreasonable as a goal. And also means that once you have that. Yeah, that that interchange, I think your your point, Andrew, about just agreeing on a format solves a lot of a lot of problems.
Speaker5: So I think.
Frode Hegland: Conceptually what we need to do is to invent a jokingly called it SDF Spatial Document Format, which should just be the means through which you can transmit a normal document plus spatial stuff. And we could maybe have a related one, spatial library formats, which is the scene slash workspace description. Right.
Andrew Thompson: I don’t know if we need to give it a custom like format though, because it’s using JSON or sdf JSON. If we if we go with Fabian’s route, they’re both based on JSON and we’re writing this in JavaScript. So it no real point in like making something new. But we should agree on the structure.
Frode Hegland: What I mean by the SDF is not to make a new format. As such, it is an approach. But let’s pretend we’re doing it. So what I mean is, for the sake of joking, we make the zdfinfo some website, and people go on there to find out what to do, and it would say something like we use for our special document format. It means we use JSON to encode this. You know, we use this to encode this in this way. The components of it are standard, standard, standard, standard. All we’re doing is how we have them deal with each other, because the library JSON needs to understand how to refer to the documents with this extra special stuff. And once we have a nice way of doing that, I think we’ve come conceptually quite far. So Andrew or Brandel, what is other than nicely nodding? Which is appreciated. Do you have any further thoughts on that?
Andrew Thompson: Now that makes sense. Having it be a structure rather than an actual format makes makes a lot more sense. Yeah, it is ten, so I have to run it just a second here, but I want to hear what Randall has to say.
Speaker5: Oh I.
Brandel Zachernuk: Know it was a straw man name, but Yeah. Saf does mean something in ust. It might. I’ll. I’ll look into it more, but it might actually already mean literally what you want. But. Yeah. I’m going to spend I need to spend more time for various reasons. Getting closer to to UST and its principles. And so I will be able to give you more details on whether it solves your problems or if it just adds to them within, within a couple of weeks.
Frode Hegland: All right. That’s good. Andrew, I’ll see you on Wednesday. Are you going to be in the lab? And Have you been using the vision? Has Denny brought it in? Because I have a few vision specific questions for you.
Andrew Thompson: She has brought it in. I don’t use it exclusively, but I do sometimes put it on and test.
Frode Hegland: Okay. It would be really interesting if you play around a little bit with the spatial video either on an iPhone or film there. And just because there is something interesting there that I think is related to our work somehow. Anyway, thank you for today, Andrew, and I look forward to Wednesday.
Speaker5: Okay.
Frode Hegland: Brandel, how much have you been playing with the spatial video? It’s. I think it’s really quite amazing.
Brandel Zachernuk: How a spatial video. I I don’t have an iPhone 15 Pro, but I you can take it just with a headset itself. And so I’ve done a little bit of that.
Frode Hegland: I’ve. Yeah. Some people say it’s better with the headset. The, the scale is better on the headset because the cameras are further apart, more human. You know, these these are obviously very close. So there is a scale issue. People seem bigger.
Speaker5: Because it’s like a.
Frode Hegland: Smaller creature looking. But yeah, it’s really, really fantastic. I highly recommend that you get an iPhone 15 and just start filming the family. You know, when I look back.
Speaker5: Yeah, I.
Frode Hegland: Think it’s crazy.
Brandel Zachernuk: Yeah. I would love to get, like, an iPhone 15 Pro or whatever comes after that. If I can get get get a good excuse. I’ve never bought an iPhone. I tend to get them for work. So. Because then I can install internal software on it as well. Yeah. And so you know, that’s same with all of my devices. They all have to actually have the thing with all of the, the bugs and and the problems that come from it. But I will try harder to do that this year because I think that it will be worthwhile to get whatever is bleeding edge and and by consequence causing all the problems. Yeah. So in terms of what the iPhone 15 Pro does, I believe that it, it doesn’t just go literally with the left eye, right eye of the two lenses. What I think it does is try to construct a disparity map so that it because it knows that there’s this disparity with this known sort of nominal interpupillary distance. It sets it out to something that is a more realistic human IPD. That said, that process is going to be flawed. So yeah.
Speaker5: Yeah.
Frode Hegland: No it does. It does an absolutely fantastic job. It’s just like with any kind of recording to learn the constraints of it. So, you know, not to be too close, not to be too far away. All of that stuff. Today we went to a here in the village. We ended up at a pet store where they had reptiles, and they just took the huge reptiles and just put it on us like in a zoo. It was amazing. So of course, I filmed that. And Yeah, it just looks incredible in the headset. And of course it’s version one. So it’s going to be better on this, that and the other way in the future. But yeah, just the stereo aspect is amazing.
Speaker5: It is.
Brandel Zachernuk: And the change from you on 1.1 or 1.2.
Frode Hegland: I am on the beta.
Brandel Zachernuk: Oh, the the current beta is very likely to be 1.2 then. Yeah, there there will be changes between that as well. So yeah, I totally agree.
Speaker5: That’s impressive stuff.
Frode Hegland: But yeah, it’s nice to be finally featured on the App Store, which means that I’m now congratulations to a lot of vision stuff as well. So. That’s. It’s exciting. Exciting times.
Brandel Zachernuk: Yeah. That’s wonderful. Congratulations on on on getting that top spot. It looks great on.
Frode Hegland: Yeah, yeah. The thing is, there’s an animation on top. I haven’t actually seen that because I’m not in the US, but that’s just basically things just moving. It doesn’t have much information, right?
Speaker5: That’s right. Yeah.
Brandel Zachernuk: It’s it’s pretty abstract information. It abstract film of of the, the dynamic typography. Yeah. If you don’t have I can make sure I’ll try to see if it’s. Still up and then recorded if it’s.
Frode Hegland: Yeah, that would be really cool, actually. I would appreciate that. And now I’m going to have to go to bed. It’s two.
Speaker5: Yes, indeed indeed.
Frode Hegland: Well, we’re going to Tokyo tomorrow morning. So another step.
Speaker5: Wonderful. Will you be with us on Wednesday?
Brandel Zachernuk: Yes, I think so.
Speaker5: Okay.
Frode Hegland: Yeah. Very good. Anyway, hi to the family. I could hear them randomly in the background. So, yes.
Brandel Zachernuk: We’re on holiday at a beach house.
Speaker5: Oh. Where like or something. Sorry.
Brandel Zachernuk: No. Down by Capitola.
Speaker5: Oh. Super nice.
Brandel Zachernuk: The TV and stuff, like. Yeah.
Speaker5: Super. Super nice. All right.
Frode Hegland: Thanks for coming by. See you soon.
Speaker5: Bye. Yeah. Enjoy. Talk to you.
Frode Hegland: Will do.