The Power of Presence: Creating Immersive Worlds That Connect

While celebrating more than 50 years of being one of the oldest festivals for innovation in the field of computer graphics, SIGGRAPH understands the value of sparking conversations about the content of the future, and carving the way for the next-gen of experiences.

The Festival will feature several opportunities for attendees to get a closer glimpse at the future. One of the highlights The VR Theater experiences is a world-class showcase of exemplary virtual reality storytelling and a staple SIGGRAPH program since 2017.

As SIGGRAPH’s VR Theater prepares to announce its prestigious “Best in Show” award for immersive experiences. TechvangArt wanted to find out more about how VR Theater started, how it evolved, the new trends related to storytelling, and what the future of XR will be.

As the festival embarks on its 50th anniversary and 51st conference, TechvangArt sits down for a captivating dialogue with SIGGRAPH 2024 VR Theater Director, Yangos Hadjiyannis to uncover the unique vision behind the selection process, exploring the trends driving innovation and what lies ahead for XR.

Let’s hang out and talk about the story of VR Theater, about past, present, discover the coolest new ways to tell stories in immersive ways, and peek into the awesome future of XR!

TVA: Can you share the origins of the VR Theater at SIGGRAPH? What inspired its creation, and how has it developed over time?

Yangos Hadjiyannis: There used to be a place at SIGGRAPH called the VR Village that was basically a space where people would see the development of VR. By 2017, Pol Jeremias had the idea of having a space dedicated for viewing VR narrative pieces that are concentrated in one room. In 2018, myself and Larry Bafia were given the task to imagine the next evolution of it. So playing with the name of the VR theater, we thought to bring the theater back to the name. The questions were:
-> HOW DO WE design spaces that basically remove the barriers of entry or acceptance of the mind to forget about the technology and just think of the content?
-> HOW DO WE make it more of a show so that when you enter space, your mind is relaxed in a way that you can travel to other realities?
-> HOW DO WE showcase, and inspire the public around this new way of storytelling?

The idea was to combine these three things and make VR a collective experience that people can watch together and feel that they’re part of something larger.

And one of the things what drove us to make this is that we were frustrated by how we were seeing VR displayed in 2018:

• We would go to exhibitions and it would be in a badly lit space.
• People would wear the headsets, but it was kind of clunky to set up you had to go through computer screens to select the content 
• And the most important thing was that when they were wearing the headset, especially women would tell us that they would feel very exposed – that when they were wearing the headset everybody was looking at them and they couldn’t see them back, so it felt uncomfortable.

All this felt more of a chore versus an experience. I think that drew us a lot to think of all these aspects in priming the mind to be ready for an immersive experience. 

DESIGNING PLACES OF CONNECTIONS

In 2018, we started with the idea that first of all, we created a nice space with red curtains to create the idea of a theater, with two big front doors – and when you first open those doors, there’s a big surprise on what’s the space inside. It was almost like a disruptive theatrical layout circularly with a panoramic screen that had animations that started from the beginning of the screen towards the centre, as well as Spatial Sound starting from the from the entrance to the end of the of the of the room, with the idea to push people organically to go and populate the space and sit around this seats without us guiding them – almost like this space calls for people to see the particular seats. That was a really nice way to make people feel that they are in a ritual. And then everything that was projected on the panorama would expand as soon as you put on the headset.

In 2019 we had a particle system, and when you put on the headset the world that was displayed on the projections would expand and then we would go into these particles.

Every year we choose particular themes as a launcher to begin the flight in a way that people feel connected to. I like to call them like places of connection. We want to design in a way to make people feel they belong in a place.

For example in 2019, when people were brought into this particle sea, they could use their controllers to gather the particles and then splash them on each other, almost like splashing water when you’re a kid in a pool to remind them that joy of a childhood memory when you spend time with your friends. And then, at some point , there was a big Gong, and everybody had to gather their particles and share them in the center to start the show. Almost like an Olympic ceremony.

We were really trying to get the idea of connection- everybody being there and watching the pieces together. And from an outside perspective, when you look at all these people in one space, you witness this beautiful choreography of people dancing as they move their headset at the particular points of the narrative. There is a beautiful human motion, human dance that happens. And I like the synchronicity- that feeling that everybody’s in a particular world at the same time.

Yangos Hadjiyannis: What sets VR Theater apart is the collective experience. Usually we have around 35 to 45 headsets depending on the year, and the goal is to showcase a selected curation and ignite the audience’s imagination as we celebrate the best narrative. 

VR Theater stands out from other venues in a way that the curated pieces that we have in the main show are viewed by over like 1000 visitors in five days. Which compared to other venues is unheard of – the spotlight that our contributors get with particular pieces  is unmatched. 

Also, the whole show is designed in a way to take you to other worlds, which includes what I I love to call ‘flight attendants’. They help people put on the headset, and feel comfortable with the ultimate goal of forgetting about the technology. I think that’s one of the things that SIGGRAPH does really well with the brilliant volunteers that makes that whole thing happen. They are well trained to welcome you, to make sure that the hygiene of the headsets is top notch. They know the pieces how to navigate it, if there’s anybody who gets stuck. And it’s all geared towards first time VR users because we know that even in 2024, a lot of people haven’t tried VR.

And at the end, we have a reflection space that Julia Read the director of last year introduce- The Metaverse Lounge was called. This year, the space is called The Reality Lounge. Once you watch the collective theater together, you can go to this space to sit with us and other creators and chat about the experiences. So there is let’s say a creative debrief. I think people really liked it last year. We also had the opportunity to do some features with talks with the directors of the movies that we’re shown. It’s an intimate space for us to explore, get feedback. Our contributors always love the feedback on their particular pieces. This is another important part, because as creators have high exposure, they can get valuable feedback in a constructive way of course, on how to improve a piece.  We were always asked by contributors after the show, “what do people like, what didn’t they?” 

Lastly, I think what we’re embracing this year is that we’re expanding the way that stories are told by introducing mixed reality.  We are working with the Ringling College of Art and Design to create a mixed reality world with a mixed reality garden. As soon as you put the work into our space that has this physical grip representation of a tree, and as soon as you put on your headset, you’ll see that the whole space comes to life with a virtual garden appearing all around you. And around the tree, you’ll see all these elements of the garden. So we’re combining the physical and the virtual space with the idea that now we can tell stories in all of the realities in a way that the physical and the virtual matter – and we have a few pieces also that are in mixed reality. We want to put the ground and  the logistics, the lighting, all things that make the transition from one reality to the other to be as smooth as possible as a standard for the years to come.

TVA: Reflecting on the past few years, how have you observed the evolution of the VR Theater? What significant changes or trends have shaped its current form?

Yangos Hadjiyannis: I think in the early years, a lot of pieces were done in a way that were exploratory of VR, and a lot of times when asked “why did you do it in VR?”, the answer was “because I just want to try it out” versus “why VR is the medium that serves this story”. But, throughout the years, we have been experiencing an incredible maturity and understanding of the other strengths and weaknesses of the medium. And I think contributors have pushed the level really high to understand and utilize the medium, to tell unique stories in a way that is only achievable in VR. So for example, one of the things that I find that makes VR stand out is the idea of presence. Compared to any other medium, any flat screen, VR has a way of making you feel that you’re actually there, your presence is acknowledged and in some cases you can also affect the story. You move with your body in the environment, so your body has a volume that affects the story. And also because everything is happening all around you, you have the freedom to look at particular points of this story, the way that you choose. Some of the things that are unique. Besides it is a great empathy engine and a way to embody particular situations that you couldn’t in, in real life.

Another aspect that I found that has been changing is the duration – in the last few years has dramatically increased. We used to have five pieces each 5-10 minutes, but now we have 30 to 45 minutes of experience. Also, we’ve seen pieces that combine a lot of different techniques from mixing 360 videos with real time graphics to volumetric capture, to clever and sophisticated interactions where you’re asked to do particular things with your hands or your head to really empathize with the hero of the story.
In addition, the pieces particularly in the last two years have important subjects that reflect the state of the world compared with the first generation of VR stories that were more fantastical, more out of this world. But the last few years we had a lot more grounded pieces about particular events that have happened, that people wanted to immortalize or pieces that wanted to bring in the spotlight on particular events like Ukraine. Or this year, we have some fantastic pieces for example one called Draw for Change, which is about the femicide in Mexico and the wave of resistance that women are doing to change the policies in Mexico and save their lives, and a piece of gratitude and resilience, called spots of light that depicts the journey of a human story of war; of someone that loses their sight, regains it through an operation and sees his loved ones and then loses it again
It feels like VR now is reflecting the pulse of what’s happening, which was not as much the case before.

THE SELECTION FROM THIS YEAR

TVA: And about the selection process, what factors influence the selection process for the jury? 

Yangos Hadjiyannis: There are few factors, that jury is looking at:

1. One of them is that technology innovation is used in a visual and auditory manner. So how did the creators use for example real -time graphics or other techniques to showcase? 
2. The second part is the appropriate use of VR as a storytelling medium –  is VR the appropriate medium to tell their story and what techniques have they used in VR to connect with the viewer? 
3. Third part is new, and it relates if it reflects us as today- there are pieces that sometimes come from more indie spaces that talk more about subjects that affect our daily life and society. 
4. Last part is that is there something that moves me forward? Is it giving a glimpse of what’s to come both in technique and storytelling and in visual techniques and audio techniques, as well as including the use of hands or other haptics to tell the story forward? 

Those are the four main categories that have also subcategories underneath to create a weighted score. We also have each juror write their deliberations of each piece, so if any contributor wants to get feedback, we always give them feedback. 

TVA. That is indeed interesting for directors and creators to get real feedback, because usually, it is a ‘yes’ or ‘no’ answer. OK, and looking forward, what do you envision for the future of storytelling within emerging mediums like VR, AR, and AI? How do you think these technologies will redefine narratives and the way we tell stories?

ADVANCED HEADSETS

Yangos Hadjiyannis The effect of being transported to other places gets super magnified with the new generation of headsets because of the clarity of resolution and amazing spatial audio. Even if you just look at older content in the headset, they just feel more real, more present. There is an increase in resolution, but also in the sophistication level of real time graphics with the advanced processing that these headsets have. 

As well as, creators know better how to pace the story, grab attention and really create the emotional moments that are starting to become in their own way unique to VR. We get to those moments when one can say “oh, I did this in VR, and I remember it this particular way. And if I watched the same version in a flat way, I would not get that”. So it’s hard to recreate those differentiators that made you watch the particular show in a next generation headset, it really creates a unique memory that when you compare it to a flat medium, there’s a noticeable difference. 

MIXED REALITY

Part of that is also the deliberate use of mixed reality as a way to connect us with the world around us. So how do we use virtual elements augmented in our own space as vessels to tell a story in a meaningful and unforgettable way? I’ve been engaged in many mixed reality pieces lately, and what I find interesting is when I engage with them for a long time, when I take off the headset, my mind misses those lively virtual elements all around me. It almost feels that we’re moving to a space where our real life will be imbued with stories and the medium to watch them is XR. So, how do we create this meaningful connection between the spaces that we live in with  particularly designed real time environments and stories?

New frontiers: AI+XR = Marriage

The third part is the use of different ways of input to to even feel more present. With the new headset, you use your hands in a way that you can grab virtual objects, it just feels that you’re crossing that barrier of the physical and the virtual when you grab an object and you move it around with your own hands. You get these surprises as the world around you changes, I think those are things that will be super magnified with the advent of AI and generative environments in XR.
This makes me believe that the marriage between AI and XR is inevitable and it’s going to be one to look out for. Because this brings us to follow our holodeck dreams where we wear the headset and we can tell, “let’s go to Spain, I want to meet Picasso and show him my drawings!”.  The interface is XR and uses its power to really transport you to a place, then we use AI to create the artificial environment and also create AI-driven characters like Picasso for example, and then using our own hands to draw and then giving you a valuable critique on how you could improve for example your drawing.
This is just an example of all these stories we have seen in Star Trek in our youth that has inspired a lot of technological advancements that have happened so far. In my mind, I see more and more spaces, more stories like that. And at one point, we don’t even need an author. In the next 10 years, I could see a movement towards that.

TVA: So you think that the directors and creators will be off?

Yangos Hadjiyannis: No. I just think that there will be even more space and freedom in designing spaces and experiences for people to explore and discover surprising visitors… The joy of AI is not to get the perfect outcome; even now when you create with DALL-E or Chat-GPT, it is about to discover creatively when you ‘let go’ and enjoy the mistake that the AI did,  as part of the art versus you trying the perfect form. () To give the emergence of randomness as part of how you experience art, I think it’s interesting on a creative level that could allow us to think of other possibilities. And of course, the best combination is always Human with AI together, and AI being not as a replacement, but two co-pilots.

TVA: In your opinion, what elements do festivals sometimes overlook that could enhance audience immersion and engagement? Is it related to the physical space, the types of stories presented, or perhaps something else entirely? 

Yangos Hadjiyannis: I like to say that the “experience always starts before the experience”. Making sure that the space where the XR is displayed is very important. The physical space affects how your mind perceives what you’re going to experience. So, try to make it more like a show, make it more ritualistic, design it for belonging, as a place to connect with others. And as such it removes the attention from the technology and helps us concentrates on the content. Also, when you make the space more fun, as well when you encourage play, the dance, the choreography, and when you make people feel connected in something beyond themselves. Especially in the world that we live in today, I think of it a little bit as a social medicine to go into the spaces and feel that you’re part of something larger.

^{_{All photos are copyrighted by SIGGRAPH and Yangos Hadjiyannis and may be used by the press only for the purpose of news or editorial coverage}}