Making Waves in VR

What would it feel like to navigate a virtual environment manifesting the collective dream of a large artistic community? That is the question the lucid dreamers at Oscillations seek to answer. Oscillations is a movement and media arts production company capitalizing on the latest developments in VR and AR, neuroscience, and the performing arts to create a new wave of immersive content. In an imagination-constrained field of so-called “AAA” VR content characterized by crass corporate marketing, first-person Pixar knockoffs and brain-dead zombie fests, Oscillations’ idiosyncratic work is a stimulating cocktail of aesthetic idealism, scientific rigor and business acumen.

I had the opportunity to pose a raft of questions to two of Oscillations’ co-founders, Danielle Perszyk (Consule Scientae) and Brendan Lewis (Consul Pro Ars), on everything from their first encounters with VR, their takes on the current “VR ecosystem,” and their hopes for the future of VR. Their replies are thoroughly thought-provoking.

 

“I suspect that the road to the medium’s full potential will be paved

with bold new ideas, not by throwing franchises at it.”

– Brendan Lewis

 

KEVIN: Tell us more about your background and your first encounter with VR?

DANIELLE: I just got my PhD in cognitive psychology. I study the evolutionary and developmental origins of human communication and consciousness. I first encountered VR as a researcher: VR is used in both neuroscience and philosophy to study concepts of self and embodiment. You can do some really strange things to your brain – like have an “out-of-body” experience – in VR!

BRENDAN: As an acrobat and dancer, I’ve been watching how social media platforms like Instagram, YouTube, and Tumblr are changing how artists create, share, and monetize their craft. The son of two ballet dancers, I grew up on stories of the Ballet Russes and its remarkable contribution to 20th century art. They brought together the best dancers, composers, choreographers, fashion designers, and painters during the Belle Epoque in Paris to create works of “total art.” I am firmly convinced that the impact of social media on global creative communities suggests the preconditions for another cultural phenomenon like the Ballet Russes.

VR/AR originally presented itself as a solution to a problem: How can we bring together the brightest artists from around the world in an affordable and scalable way that doesn’t restrict the creativity and the nomadic lifestyle of these influencers? A standing company model just wouldn’t work. We started exploring VR and AR as creative mediums in 2016. It was the creative potential that got us hooked.

 

KEVIN: What interests you the most about virtual reality and immersive media?

DANIELLE: I’m fascinated by how immersive media can “trick” the brain. You can create environments that are photorealistic, but then manipulate subtle features like shadows, coloring, or geometry to create the illusion of impossible physical environments. This can simulate dreaming. Then, if we use BCI and machine learning to enable users to control environments with their thoughts and emotions, we can simulate lucid dreaming, and even shared lucid dreaming.

I’m also interested in using immersive media to “close the interpersonal gap.” I think one of the most beautiful tragedies is that we can never truly know what it’s like to be someone else, to experience the inner world of another human. This is part of what motivates artists to create: we want to hijack others’ minds to make them see what we see and feel what we feel. VR as an artistic medium can help get us closer to achieving that, and in doing so we can improve interpersonal relationships and cross-cultural understanding.

Finally, I’m intrigued by the possibility of using immersive media to explore “thought hypotheses.” What would Chicago look like if it had lots of vertical farms? What aesthetics in public spaces would support psychological well-being? We can simulate cities of the future, for example, and “try them on” to collectively decide on how to beautify our shared spaces and motivate voters and policy makers to move toward progressive solutions for society.

BRENDAN: For me there are two ways to answer this: self-indulgently and outwardly reflective. The former bleeds into the latter. I’m excited by XR (cross reality) and immersive media because of the creative challenges offered by new mediums and their technologies. The limitations unlock creativity in the same way that exercises in constrained writing do. For example, as a choreographer, one of the first things I wanted to do in the XR space was use volumetric capture on movement artists and create stage charts and formations that would deliver an experience both on a fixed axis in 3DoF and in room-scale 6DoF, depending on which platform you were using to take in the art. I’m excited about design and choreography that considers the limitations and capabilities of multiple technologies and platforms at the same time.

Until the XR markets mature, the industry needs experimental artists willing to work with limitations to express compelling ideas. Artists know how to say a lot and do a lot with limited resources. I think the experimental stuff is always going to excite me the most. I don’t get excited about the big studio properties showing up in the early days of VR content. I suspect that the road to the medium’s full potential will be paved with bold new ideas, not by throwing franchises at it. Franchises will be great for location-based VR, which could help the immersive entertainment industry grow, but the Snow White, Toy Story or Final Fantasy of VR will be something we’ve never seen before.

Possibly what excites me the most is how brain-computer interface (BCI) and machine learning integrations are going to work their way into VR’s future. I will consider it my biggest success and contribution to the arts and society if our productions and research at Oscillations can set the course for virtual environments and experiences that respond to our thoughts, emotions, and imagination, like how the afterlife functions in the film What Dreams May Come. In my opinion, that’s the fullest potential of VR as an entertainment technology and artistic medium. I’m excited about the world-making potential of VR, and I’m hopeful about platforms and builder’s tools like Sansar and Sumerian. The Oscillations dream is a BCI-driven platform that conforms to the creative vision of the user.

KEVIN: You’ve assembled an impressive array of artists, filmmakers, scientists and performers as collaborators in your immersive media experiences. Tell us more about your approach to engaging and coordinating such a diverse group of talent. What are the challenges and what are the rewards?

BRENDAN: At Oscillations, we’re working with 80+ of the world’s most talented performers, designers, visual artists, and other creatives. I used to question what kind of creative I was, and what my role as the artistic director was, given the caliber of our artists. They’ve risen to stardom on social media – to the top of their cultural niche – by sincerely being some of the best in the world at what they do. Who am I to direct such talent? How can we develop a method to tease out an Oscillations terroir? Are these professionals interested in VR as a medium? Do they want to contribute to the technological development and scientific research? Do they even want to be part of a movement?

It occurred to me that I’m something of a collage artist, using social media and creative subcultures in place of magazine clippings and virtual environments in place of a canvas. We’re working on a few use-case partnerships right now, and it’s always fun to think about how to mix the Oscillations secret sauce into someone else’s meal. But our flagship content ideas just hit me in the head as they fell from the sky in the middle of my stretching and handstand practices. In cognitive psychology this is called “opportunistic assimilation:” your subconscious is assembling random bits of stored information in new ways, trying to solve problems, and then one day the assimilation or solution arrives in your conscious awareness nearly-formed and seemingly out of nowhere. You hear artists talk about this “lightning bolt” all the time. One of our advisors, Andrew Razeghi, has a great TED talk about this.

These flagship productions are consistent with the idea that a Ballet Russes-like phenomenon is just waiting to happen, given the way social media is influencing the visual and performing arts right now. And the visual and performing arts are perfectly suited for VR in a few ways. They don’t try to be like other products such as video games, movies, or television. I personally feel VR is still a ways away from being able to compete with these traditional categories on their own merits. They’re a bit more like theater and ballet in structure, but still very much their own product; something completely new and designed uniquely for VR. We originally called them “Dreamscapes,” and pulled back on that when we heard about the location-based VR company by the same name. I also like “Hypnagogiques,” which is sort of French for the same thing and a subtle nod to Erik Satie’s “Gymnopedies”… but let’s face it, the marketing folks would laugh me out of the room.

These pieces basically ask the questions, “What would it look like to navigate a virtual environment meant to feel like the collective dream of this large artistic community? What would a world fashioned out of incredible talent and imagination look and feel like?” We’ve been inspired by the musings of Scott Ross and Jaron Lanier when they talk about the promise of VR being in “world-making” and “shared lucid dreaming.”

Anyway, Danielle and I started recruiting collaborators and talent in 2016, and it turns out the questions I asked above – the ones we started with – were just a case of “imposter syndrome.” Oscillations is taking subcultural trends in the arts and kicking them into overdrive. It’s about building a diverse and creative community and finding the partners and resources to empower that community. It’s about pushing the limits of technology. It’s about bringing new customers to a nascent and fragile new industry. Its about telling the story of the early days of VR, to the demographic most interested in the consumer tech, through a branded, transmedia ecosystem featuring influencers with a masterful command of attention economics. Its about starting a conversation between artists and scientists that’s never been had.

Oscillations is all of these things, and our artists really understood that right away. We’ve built a sense of family and community in the short time we’ve been working together. When they looked at our flagship content ideas, to my surprise an overwhelming number of them found the “collage artistry” of the work to be somewhat obvious. I got a lot of responses like, “Yes, this is the evolution of what’s been happening on social media. This makes perfect sense. This is what’s next and this is what I want to do next in my career.” That’s been the general feeling surrounding our artistic aspirations; its what has made it easy to recruit collaborators. Artists want to push limits, discover new things, and create. We’re driven to do it. This is the underlying theme of Danielle’s research. So our ambitions have attracted a humbling amount of enthusiasm and support from the creative community.

Once you build a sense of community, coordination is just about mutual respect and cooperation. We’re committed to providing the tools for our artists to grow and prosper. My hope for VR, and technology in general, is that it empowers and enriches the arts. Our founding fathers wrote very clearly that their hope for the future was a degree of prosperity that allowed the average citizen to pursue the arts and explore their own creativity. Beverly Sills once said, “Art is the signature of civilizations.” I love that quote. So what do we want the signature of the present day to look like? What is all this civilization for if not to strive to be surrounded by beauty and inspiration? It can’t just be to procreate and exchange goods and services… that’s depressing.

Civilization doesn’t have to be pragmatic, gaudy, or resigned. I like the remnants of Deco architecture in cities like Chicago and New York. It reminds me that civilization can be aspirational. So to answer your question fully, the personal reward for the work I do at Oscillations is the sense of belonging to a community building something together. In terms of a civic-mindedness, the greatest reward for our efforts would be conferring an aspirational civic orientation to our audience. As Danielle mentioned, VR can allow us to preview the kind of world we want to live in.

KEVIN: The application of neuroscience and BCI to Oscillations’ work is fascinating. Where are you now, and where would you like to be?

DANIELLE: We’re actually called Oscillations because I study neural oscillations, or brainwaves. Brainwaves can tell us things like whether people are bored or engaged. We can measure them with little sensors that pick up the electricity emitted by people’s brains. We can then integrate this brain data, in realtime, with a virtual reality experience (VRE) so that certain artistic elements of the VRE (e.g., color, music) are modulated by users’ brainwaves. Even a few brain-controlled elements can make a big difference in perceived interactivity of a VRE.

Right now, brainwaves are probably the most robust way to reliably allow people to control things with their minds. If we combine several different physiological measures, such as brainwaves, eye tracking, skin conductance, and all other sorts of incredible new bio measures that are currently being innovated at places like the MIT Media Lab, we can allow VR users to more precisely control VREs with their thoughts and emotions. But what we’ll be able to do in a decade or so will be truly amazing. Neuroscientists are constantly discovering more fine-grained brain signatures, and right now they’re working on “reading” what’s in people’s imaginations (e.g., whether somebody is thinking about a face). If we combine these neural “read outs” with machine learning – which is, of course, also making unbelievable strides – we’ll be able manifest immersive environments that very closely resemble what’s in a person’s minds eye. And if we prime users to think of certain things with narrative storytelling and mental associations, we’ll be able to do so even more effectively.  

KEVIN: What is your view of the current “VR ecosystem?” What is working and what is lacking?

BRENDAN: I’m excited about entrepreneurs and developers thinking creatively in XR. Rene Pinnell at Kaleidoscope is a great guy and he’s working tirelessly to build and support a beautiful community of creators. Kaleidoscope throws the most aspirational and encouraging events. Now they’re working on a funding platform; VR desperately needs this. Travis Cloyd at Observe Media is doing pioneering work in the cinematic VR space; he’s applying his film industry savoir-faire to build innovative revenue models and distribution channels. Creatively speaking, I love everything Monochrome Paris represents.

I think there is some content out there that points to how cool VR will be in the future. Nature videos, concerts, space and deep ocean tourism, the blurring lines between cinematic and gaming, etc. These experiences leave me wanting VR to instantly advance ten or twenty years. It’s great for trade show demos, but maybe not home consumption.

I think I’m paraphrasing or recontextualizing a quip I read somewhere when I say that VREs should be designed to make you forget that you’re in VR. I worry that content should be focusing on how compelling we can be with the tools that we have to create experiences right now, and not focused on making virtual realities that don’t quite live up to their full promise. I really haven’t been swept away in VR the way I know can be done with the technology we have available to create with today; I’m always thinking about how much more immersive or advanced or less glitchy it could be.

I think VR is like hoverboards (at least to Gen-X and late millennials who liked Back to the Future). We want the technology so badly, and we want it to be amazing right away. The expectation bar is set high. I don’t think this was the case for developments in video games or animation. The public had no prior expectations. Maybe I’m wrong, but I don’t think there was the same kind of hype and cultural mythology that there is for VR at the advent of Nintendo or Pixar. This supports the argument I made earlier about the highly artistic and experimental content being at least as important as the big Hollywood and gaming industry IP. It’s just as likely that an idea from left field is going to be the breakthrough hit. Perhaps we’re even more likely to get swept away if the type of product, experience, or subject matter doesn’t offer us any comparison or references outside the experience that keeps us mentally tethered to reality. Maybe the first hit will be a hit because it was designed specifically for VR by creators with no filmmaking or game industry experience, and no access to expensive IP. Maybe VR is meant to be something completely different than film and games. Its worth leaving our minds open and asking the question anyway.

By the same token I worry about the other end of the spectrum, which is too much user-generated content. I worry about early releases, bug-riddled experiences, and the general lack of curation of the VR ecosystem. I read an article the other day about Nintendo’s seal of quality model being a possible necessity for VR in response to this problem. It gave me pause to consider.

Maybe all of these issues are actually just one issue: a money issue. It’s a middle market for content creators that I’m advocating: something between the marketing gimmicks for big entertainment IP and the vast ocean of polygonal zombie shooters. This is exactly what market forces won’t provide for without revenue models for independent content creators.

The Hype Cycle has left content investors feeling burned because VR didn’t follow the growth trajectory of mobile phones. There’s a patience and dedication problem when it comes to VR investment. The ones still investing in VR are not seriously looking at content. Entertainment is viewed as a “hit making” industry… a big gamble in a buyer’s market with no reliable method for prediction or scalability. This accusation applies to legacy media with its established distribution channels, gatekeepers, and red tape, but not to third-wave digital media. Virality can be formulaic, merit-based, or both. I’m thinking of licensers and programmers like Jukin Media and DanceOn. Try explaining that to anyone over 35, though.

The VR industry is really neglecting an opportunity to leverage social media to tell its story. For the smaller studios, third-wave digital media can be a good source of income, too. The content ecosystem right now isn’t targeting the right demographics in my opinion. Monochrome Paris writes in their manifesto that designing for Millennials is paramount. I couldn’t agree more. Hopefully Oscillations and other influencer-driven VR developers can fill in this gap. It’s got to be done carefully, though. You can’t just throw an influencer into anything and expect your numbers to move. I’ve seen some of that in VR. The stories have to be genuine and natural. Disingenuous celebrity and influencer marketing has a stink to it. Millennials and younger can smell the difference. I’m really excited about Pixvana joining the fray. Their platform could make such strategies easier.

VR needs a hit, but nobody wants to take risks, so we end up with an increasingly impressive entertainment technology with nothing to entertain us except a few short films, a lot of movie trailers, and too many zombie shooters. Venture capital is still hopeful and curious about VR, but the venture formula can’t stomach the risk yet; VR going mainstream is still a big question mark. Venture capital in general reminds me of Richard Pryor in THE WIZ, announcing the color of the week to the citizens of Emerald City. Everything turns red, green, or gold and everyone starts dancing and signing a song about the new color. Venture can be a lot like that: “Its VR… now its fintech… no wait, sorry, blockchain!” And just like the Wizard of Oz, there’s often not a whole lot behind the curtain to justify the prognostication, so the tendency is to stick to what you know, play it safe, and follow the crowd.

We don’t need more platforms or hardware as much as the experiences consumers will enjoy once they show up. I think it’s too early to expect the creators and developers to shoulder the burden and just figure it out. Content investment from the studios, the hardware companies, and the entertainment-themed venture capitalists needs to be braver and more creative. Take this with a pinch of salt, I’m certainly biased… Oscillations is super weird. But then, I think the far-out, highly imaginative, world-making experiences is where VR will always outshine other entertainment technologies. We need to design with this in mind, especially now when consumers are still wondering what VR offers that’s so much better than movies, TV, and games to justify VR prices.

So who’s going to step up and take risks? Who’s going to take content seriously? Developers drive adoption. Hits always establish and define new entertainment technologies, once the technology is good enough. VR is ready.

KEVIN: Where do you see Oscillations five years from now?

BRENDAN: Oscillations may turn out to be the most beautiful failure I can think of. The business world does not forgive ideas that can’t be explained in 10 words or less, and Oscillations cannot be summed up as something like “Uber for Dogs.” Luckily, the VR community is really open-minded, and like I mentioned earlier, there’s a whole cultural mythology and magic around it. It attracts some of the most intelligent, passionate, and creative people I’ve ever met, from technologists and scientists to entrepreneurs and artists. Among the dreamers and the optimists, there is a real sense that we’re in this together and we’re coming up together in exciting times. It’s cliché by now to say so, but I can’t help wondering if this is what it felt like to be a pioneer in the silent film era. I sincerely hope all the great people I’ve met are given the opportunity to work in the space: to really get in there and tinker and do the exploratory research. It will produce better results than mediocre versions of products and IP that are more satisfying in other mediums.

Maybe the VR winter is truly thawing, and five years from now we’ve encountered an astonishing mixture of luck and opportunity. If we’ve executed successfully, Oscillations will have started a movement in the arts and sciences, perhaps on its way to becoming a household name – a Ballet Russe for the 21st century. We will have brought the attention of millennials and iGen to XR art and entertainment, instilling the idea among our demographic that VR and AR are the most exciting creative mediums of their lifetime. In so doing we look to leverage the revenue models from third-wave digital media to support our content and research.

If we’re really rolling, we will be collaborating with an ever-expanding pool of creatives and bringing their audiences to brand partners. We will be the go-to for artistic use cases for new technologies and innovations in the immersive space. In five years I hope Oscillations has a highly creative and inspiring portfolio of experiences in VR and AR. If the revenue models succeed and the road map is satellite-linked, then we will direct the resources at our disposal toward developing our BCI platform until VR is synonymous with imagination.