INTRO

There’s a LOT to talk about, so let’s first begin with saying that we have a lot of documentation, which is really convenient, because that means there’s a lot of proof of our work. To start off, I’ll link my final showcase presentation to the left here.

It’s a 2 part video, because I accidentally stopped the stream in the middle of the presentation. You live you learn, and now I know not to click the shiny red button called “Stop Stream”.’

In this presentation I showcased what NEOS was in a short time, but there’s still so much more I wanted to show and tell you about NEOS, so I’ll probably tell you here.

 

Of course, I’ll also show more about what the project was and how we designed our performances and experiments, but as for my part in this project, I’ll mostly show you how and what I made during this project, because it doesn’t include any specific coding practices, but more creator practices and how I iterated on my creations.

THE TEAM & ASSIGNMENT

To start off, let’s talk about who I worked with, and what the assignment was (and who gave us this assignment).
Our team consisted of:
a Theater student: Robin Krijgsman,
a narrative game designer: Myrna Min,
a level game designer: Joris Dingelstad,
a enviromental 3D artist: Youri Hoek,
a asset 3D artist: Randy Paulus,
and a Unitversity student of Computer Sciences: John Krijger
And last but not least myself: Pepijn Kok, VR gameplay programmer.

Together we worked on this project given by Joris Weijdom, a Mixed Reality Researcher and senior lecturer at HKU University of the Arts Utrecht. He gave us the assignment to “Create a meaningful and embodied mixed reality experience using NEOS VR and (serveral) physical locations for (minimal) 5 participants“.

Using our mixed diciplines together, we did some experiments, and created our final performance JUNK, and a EXPO hall which we created in NEOS and could be called the first official museum on the NEOS VR platform.

The Process

Our First day in NEOS

So, as already said and shown, we worked in a program called NEOS VR. And I have to already say, I LOVE WORKING IN NEOS!

Neos VR is a development engine available on steam for free (so download it RIGHT NOW!!) It’s similar to VRChat in the sense that it’s a community multiplayer experience, where people can meet and talk to each other though their virtual avatars, but I want to make clear that NEOS has so much more to offer than VRChat. NEOS is a Real time virtual development engine, which means that you can open NEOS and create anything you want real time with the use of “Components” and their programming language “Logix”. For more information about NEOS itself, see my presentation at the start of this page.

The first thing we did for our project is start learning and using NEOS. Joris gave us about 1 1/2 weeks to learn NEOS and create a Home for ourselves. He wanted us to have our own place to work and create things. Of course, I immediately got into learning NEOS by just experiencing it. I started off going to worlds, seeing what was already made, and I got obsessed by the idea of changing the player’s gravity, which was so much fun to do (kind of nauseating too, but still fun).

ME… But upside down sitting on the roof

The next thing we did was of course to make a classroom. A place where we could come together to have meetings, create stuff, and hang out in NEOS. We already had an assigned classroom from school, but since we won’t be going to school because of COVID-19, we wanted to have a virtual space where we could still meet. And I have to say: we got lucky with this project. We where able to meet, talk, and work together in VR almost every day! We could always meet and discuss our findings and designs with each other, and it was really convenient to work together every day.

Most of us mostly worked in VR, but if anyone was sick of putting on a heavy VR set on their face they could still work in desktop mode! Working in desktop mode doesn’t give you the full experience for your builds, but if it’s the only thing you can do then it’s not that bad.
Most of our Artist worked in desktop mode, because that way they could easily access their 3d modeling applications.

THE EXPERIMENTS:

once we got everything set up, we started with the EXPERIMENTS! we decided to first do these experiments to see what we could make in NEOS, and to research ways to make performances in VR. We did some experiments in real life, but we mostly made these experiment worlds in NEOS. We started off with groups, and I started off with Randy to make our “Breaking Reality” experiment.

This experiment was about us figuring out what we could do in VR that you couldn’t do in real life. I immediately started making a little game that used the gravity tool that NEOS has built in. This experiment didn’t really lead to anything big in our final product, but it was really fun to figure new stuff out and it got me more into thinking about different perspectives you can have on correlating environments.

In the meantime, Myrna and Youri worked on a Lightswitch experiment that used a lightswitch to change the environment every time you flicked it. Which they used to tell a story of a person moving in and starting a family in a small apartment room. And Robin and Joris worked on (my favorite) the Photogrammetry room. They got a photogrammetry from the interior of an old German castle, and they used this in combination with a very dark environment and only a torch as a light source to create a very interesting experience where nothing seems like what it is. They used real time acting and also created a very cool aesthetic which I loved!

Next up, we decided to split the groups again, to where I worked with Randy and Myrna on the “Escape Room“. We started off with using the gravity mechanic to maybe make an escape room where 2 players are in the same room, but only one of them could see while they where on the roof, and the other one was on the ground but was blind. And this way, they had to communicate with each other to figure out an escape room puzzle. After some small testing we figured it would be more interesting if the players would be in different rooms that sort of looked the same. They would still need to communicate with each other to figure out a puzzle, but the rooms had very small differences, which made it more of a challenge to explain everything to each other.

For this experiment, I was very proud of my creations, including: A working walkie talky, which took the audio stream of your voice and then send that to the other one. And a very dynamic chessboard puzzle, in which the designer (Myrna) could decide a color and a location of the correct chess piece and it’s location. And of course the invisible pins, which could only be seen if you looked through the special magnifying glass.

This experiment was also about integrating a Twitch Chat system, to where the Twitch chat could enter commands to make stuff happen in the Escape rooms. I worked together with John on this system, and we made it very dynamic. This meant that our designer would only have to enter a new command to a behavior, and the command would automatically work with Twitch. The system was hover very simple and I could show it in C# form:

Function OnTwichGetMessage(string message){ //<-- automatic function built in NEOS
         if(message[0] == "!"){
                  string[] messages = message.split(" ");
                  if(messages.Count > 1){
                           SendDynamicTrigger(slot = Root, name = message, value = messages[1]);
                  } else {
                           SendDynamicTrigger(slot = Root, name = message, value = null);
                  }
         }
}

Most of the functions we use in NEOS are very simple, and work really easy with their own interfaces and components. I do have to admit that none of these systems are very optimized in a real game engine, but it works really well in the NEOS system, because it gives more control over multiple Logix systems.

Our experiment was a big success in terms of mechanics, because everything work very well and many of us didn’t experience any lag. But in terms of the overall experience it failed, because we where too dependent on the control of the Twitch Chat and how many viewers we had. The testers also thought that it was kind of a weird experience that they depended on a higher power which they couldn’t see, couldn’t hear, and didn’t really know they where watching.

While we where working on our escape room, the rest of our group was working on their experiment. Robin and Joris started working with Youri on their “Pool party“ experience, where they made a classic local pool that users could go to and experience and be a part of a live performance. You could enter through the gate, and was assigned a role. Once you got this role, you had to live into it, and you where not allowed to leave until they let you. This experiment was interesting because it introduced the player being part of the performance, which we also used in our final performance. But, a bad part about the pool party experiment was that the performers where not allowed to move, and didn’t get a clear introduction to their role, which we changed in the final performance.

Next up was the Final performance, JUNK!
We iterated through many different ways to make the performance, and we got some great inspiration from Jason Moore’s MetaMovie project. Joris Weijdom had some connections with Jason, which gave us a quick look and a VIP experience of his project. We where also shown around NEOS by MedraVR, who brought us to interactive and performative worlds. After getting all of their inspiration, we decided to start on the real life technical tests, to see how we would make our performance a Mixed-Reality experience.

We got to school, and started setting up small tests. I was in charge of setting up everything technical, including the use of VR trackers, making an interface that shows the virtual space through a phone, and making a virtual screen that shows the real world. Every time the designers wanted something technical to happen, I would be the one to set it up (mostly) successfully. At this point, our theater student Robin went off and made a few little story’s about life and specifically perceiving certain “special” moments in life.

Long story short, we made all of the performances around those stories, because we got the feedback that they where very interesting and real interpretations of what those moment would be like to experience them yourself. The artist created some environments, the designers created the experiences for the stories, and I worked with John to set those experiences in action. At this point we didn’t have a clear goal, but we still had A LOT to do, because I was assigned as the Logix expert of the group. I was called multiple times to help people with their creations, and I was also responsible of all the technical systems in the real life performance area. I had to set up scenes for the light and sound systems, I set up the midi pad for the control of the experience, so we could control it real time, and I was fully responsible for the working of the VR headset.

At this point, it was just optimizing everything until the performance, and this was where the real stress came out. Some systems didn’t want to work, and sometimes audio wouldn’t play. It was what you call: “a real shitshow“. But magically, as we where 5 minutes away from the start of the real performance… everything worked, and it would so throughout the entire performance. I was really happy that everything worked as how I set it up, and everyone was proud of what we had made.

At this point, it was just optimizing everything until the performance, and this was where the real stress came out. Some systems didn’t want to work, and sometimes audio wouldn’t play. It was what you call: “a real shitshow“. But magically, as we where 5 minutes away from the start of the real performance… everything worked, and it would so throughout the entire performance. I was really happy that everything worked as how I set it up, and everyone was proud of what we had made.

To finalize our project, we decided to create a “little” showcase of our performance in NEOS. We wanted to make a museum like building which showcased all our experiments and our final performance for our school’s exposure show. And again, I was set in charge of everything technical, and optimizing the crap out of this world.

The EXPO world was a disaster at first, with about 3000 assets you had to load in before you could even get to the entrance. Luckily, I got some help from Medra and his friends to find every little asset and optimize it in a wat that the a user could still see the expo and not crash (spoiler: some still did, it was a really heavy world for the NEOS system, with too much to load for some people).I could tell you more about the Expo, but I’d much rather you check it out yourself! We documented everything! And you can find all of it in our museum which we’ll upload on the NEOS servers through the HKU. So if you want to check it out, DOWNLOAD NEOS and find our expo in the “published worlds” folder under the name: MR Expo – HKU Creator Hall

So that’s it! of course I didn’t explain everything we did in this project, but to sum up everything I did and created in this project: “I was the Logix Expert, and if anyone wanted something technical to happen, I was there to help.“ Of course I’m not the overall Logix expert, because there’s still so much more I can do with NEOS that I haven’t done before. But I can proudly say that I’m not quitting on NEOS and that I’m actually trying to get more into the community and use NEOS for my graduation project!

I hope this post told you enough about some of the processes of our project and how I loved to use it for my creations, and I hope to see you sometime in VR! you can always send a contact request through NEOS to “AtlasTitanium“ to talk to me, and ask me more!

Copyright © 2021 Pepijn Kok — Escapade WordPress theme by GoDaddy