91桃色

On this episode of Living Live with 91桃色, host Tyler Kern was joined by Gideon Ferber, 91桃色 Director of Product Management and Business Development for Ross Virtual Solutions, and Chris Mollomo, 91桃色 Manager, Virtual Design Group,鈥疪oss Virtual Solutions.

The trio tackled the sometimes-intimidating topic of rapidly advancing technology 鈥� though the sheer pace of today鈥檚 live production environment is overwhelming on its surface, paying careful attention to how these innovations can benefit your live productions can pay tremendous dividends.

And some of the most exciting, Ferber said, are new applications for virtual and augmented reality.

鈥淲e鈥檙e talking about virtual solutions, whether that鈥檚 virtual sets or augmented reality,鈥� he said. 鈥淚t used to be quite a hassle to build a set. Looking 10 to 15 years ago, it was extremely hard. Engines weren鈥檛 powerful enough. 鈥� Things have evolved and are much easier.鈥�

In fact, Ferber said, the entire, end-to-end workflow is now relatively simple, offering live production operations a way to overcome traditional 鈥渢oo-expensive, too-complicated鈥� thinking.

In the case of the virtualization of productions, Mollomo said the benefits are immediate and enormous.

鈥淎t the end of the day, you end up saving money, because you鈥檙e not having to maintain these hard pieces,鈥� he said. 鈥淵ou鈥檙e able to have many different environments in one studio. You can pare down the amount of cameras you鈥檙e using, if you wish. There are a lot of advantages to a virtual studio.鈥�

AUTHOR: Tyler Kern
CONTRIBUTOR: Chris Kelly

 

---

Podcast Transcript

Tyler Kern:

Technology has been moving at a rapid pace these days and for some that can be a scary topic to tackle. But with the right guidance and solutions, technology can make a huge difference and make life a lot easier. Joining me to talk about how this fits in with TV production is Gideon Ferber. He’s the Director of Product Management and Business Development for Ross Virtual Solutions. Gideon, thank you so much for being here.

Gideon Ferber:

Absolutely. My pleasure.

Tyler Kern:

Absolutely. And then we also have Chris Mallomo, he’s the Manager of Virtual Design Group for 91桃色. Chris, thank you for being here as well.

Chris Mollomo:

Thanks for having me.

Tyler Kern:

Absolutely. Gideon, let’s start off here. Tell me a little bit more about how advancing technology can make life easier in these production settings and how have things really evolved over the last several years to make this easier to implement?

Gideon Ferber:

We’re talking about virtual solution, whether it’s virtual sets or augmented reality, and it used to be quite a hassle to build a set up. If we’re looking at, I don’t know, 10, 15 years ago, it was extremely hard and engines weren’t powerful enough. You needed at least three, four, five different companies involved to create one set. These years, things have evolved and are much easier. So there are more and more companies in the market that can actually provide you a full end to end solution. Tracking devices are much cheaper to begin with and easier to operate. The entire end to end workflow is now pretty simple. This is something that we can do even remotely. We can guide customers to set up their equipment and prep everything over the phone. So all of a sudden it’s much more accessible to broadcaster that in the past thought that, “Oh this is way too complicated, way too expensive. We can’t approach it.”

Tyler Kern:

So as Gideon is talking about that added accessibility, Chris, I was wondering if you could help us understand some of the advantages of virtual productions? What does this provide and what does this allow that, that maybe customers didn’t previously have access to?

Chris Mollomo:

So I think at the end of the day, you end up saving money because you’re not having to maintain these hard pieces. You’re able to have many different environments in one studio. You can pair down the amount of cameras you’re using if you wish. So there’s a lot of advantages to a virtual studio.

Gideon Ferber:

Tyler, one more comment here. You can also use virtual studio to reduce your downtime between shows. So if, with physical sets, I have to get my crew to roll in equipment between shows if I want to change the look and feel of the studio. With virtual, I can just click a single button and now I’m loading a completely different set and I have a completely different look. So I can go into commercial break with one set and come back from commercial break on a completely different look. So I’m saving time between shows, which means I can produce more content over the same hour, same amount of hours a day.

Gideon Ferber:

Another thing to keep in mind and that also relates a bit to the previous question. That’s another change in technology. So one of the reasons that many broadcasters avoided virtual for many years is the well, computer game feel that graphics has in the past. Over the last, I would say three years, three and a half years, something like that, the Unreal Four Game Engine, all of a sudden made a huge breakthrough into the market and being the lead, in my opinion, some people prefer Unity, but I’m not going into personal preferences. My opinion, Unreal is probably the strongest game today in terms of render quality.

Gideon Ferber:

So all of a sudden in the last three years, I can now create hyper-realistic looking content. So it’s now not only that I can save, as Chris said, save space, save money, save time. I also eliminate the barrier of non-realistic looking virtual content. So now I have hyper-reality content that is all still extremely efficient. So the combination is extremely powerful one. And one last mention here. It seems like that almost every vendor, broadcast graphics vendor in the market, adopted Unreal as their go to solution for virtual augmented graphics. So, we’ve leveled the playing field between different vendors. We are all using the same quality, the same engine and the same graphic capabilities. Yeah. So it made life pretty simple for well, for the end customer.

Tyler Kern:

Yeah, absolutely. And one of the things that, it seems that’s standing out to me about what you’re saying is, that what you’re enabling is for creative people to now have these tools at their disposal. That it doesn’t take up a ton of time, like it used to, and it’s easier to work with, easier to implement into a workflow. So now you’re giving creative people the tools to actually do better storytelling and create better content right? Isn’t that the end goal of all of this? Is that there’s better content, more content and it’s more easy to create.

Gideon Ferber:

Absolutely. Yeah, absolutely. Well, let’s take just an example. A customer that we’re working with in the Netherlands, I won’t name names, but they now built a studio with 13 cameras, purely running on Unreal, full virtual set and absolutely amazing studio. Now, if we go just five years ago, that project would have been incredibly expensive, incredibly complex and this customer, well is using Panasonic PTZ cameras. And now you have tracking, you have a 4k complete [inaudible 00:06:27] video pipeline and unlimited creativity and yeah, the sky is the limit. It’s pretty amazing.

Chris Mollomo:

I think it’s a good point. As Gideon pointed out, all the major players in this industry are leveraging the Unreal Game Engine, which gives us near photo realistic rendering in real time, which is fantastic. So all of the people in this space essentially are leveraging that render. So what becomes important is your workflow around those systems. And as Ross works on our workflows for news environments, what is nice is that Unreal, from a creative standpoint, is working on workflows to make it easier for creative people to get their content into the Unreal Game Engine.

Chris Mollomo:

They’ve developed data smith, which helps people working in Cinema 4D or 3D Max to take their native files and bring them right into Unreal and have your lighting and materials and everything come through, making that process much easier. In the past, we had to bake all of our lighting into our textures and reapply them in previous render engines, which took a lot of time. And if you needed to change something, you needed to rebake your whole environment. With Unreal, everything’s in real time. If I need to move an asset or move something, I can move it right away. And I’m right into my show. So it gives you a lot more flexibility.

Gideon Ferber:

Yeah. That’s a really, really good point. Again, kind of going to the basic features of the Unreal Engine, as Chris said, things that were impossible up until three, four years ago, dynamic reflections, talent reflections in the studio, realtime talent reflection, realtime [inaudible 00:08:11], dynamic textures. It sounds funny, but things today, it’s almost a norm. If up until five years ago, I had to fake a talent reflection by using the same video page, flipping it upside down, trying to mask it to create some sort of reflection. Today takes, what 10 seconds? I’m applying trader on a table or on the surface. And it automatically generates the realtime reflection in the engine. So yeah, the Unreal Four platform is a real game changer on the creative side.

Tyler Kern:

Yeah, absolutely, absolutely. That sounds like it’s a massive time saver and it is very convenient for users. Chris, as you have conversations with clients and with customers, how do you help them understand and walk them through the process of finding the solution that works best for them in this world?

Chris Mollomo:

It’s a good question. There’s a lot of different flavors of using a virtual studio. You could have a trackless environment where you’re having a fixed camera on your talent. And then you’re bringing that talent into a virtual environment and using virtual camera moves in that virtual space. You can have a fully tracked studio, which is something that Ross does all the time and specializes in, where you can have a multi camera setup. Each camera is giving you it’s positional data, pan, tilt, zoom, and focus. And we’re feeding that into the render engine and we’re compositing the talent into that scene. So they become part of the scene and you see their real reflections and shadows in that environment. So it really depends on what the end goals of the client are. We generally meet with the clients, we discuss what they’re trying to achieve and steer them towards what product or what products might be best for them. And what they’re trying to do.

Gideon Ferber:

One culture, and it’s kind of funny, sometimes if we take it to the technical side of things, some customers just miss the bigger picture. We had the conversation with the customer and we explained everything on the technical side and what’s going to be achieved and how can we modify the design to their likings, and eventually the project just fell apart. When we started digging a little bit deeper, apparently they didn’t like the color yellow. And the samples we shared with them had yellow in it. So it’s a really fine art. I need to, on one hand customize your, the way you deliver the vision to the customer, to meet their expectations. On the other end, at the end of the day, it’s up to them. We can provide the technology, we can assure them that it works, but they need to be able to think outside the box and really push boundaries a little bit more than, I like this physical desk. So let’s copy that and put it in virtual.

Chris Mollomo:

Yeah. Gideon that’s a really good point because it’s hard to, as people are transitioning from using hard sets and doing traditional broadcast to using virtual, there’s something that happens where it takes a while, a learning curve to understand how to leverage these technologies and get the most out of it. With leveraging the Unreal Four Game Engine and what we’re doing with Voyager at 91桃色, the sky’s limit in regards to what you can do creatively. You’re not bound by the laws of physics or by the studio space that you have. You can really create any kind of environment you can imagine.

Chris Mollomo:

It’s such a departure from the way we’ve traditionally done broadcast television, that it’s hard for people to grasp that. So we, at the onset of every project, we make sure that the decision makers, the creative people that we’re talking to about what environments to create, really understand that they’re not, we don’t need to recreate their hard set studio and bring it into virtual. We can do something completely different. And that opens up a lot of ideas and really helps the client and helps the process to get the most out of what we’re trying to achieve in any project.

Tyler Kern:

Yeah. Chris, you mentioned that that learning curve there and the way that you play a part in this process of creation with your clients. How do you help them, I suppose, walk through all of the various aspects and all of the various capabilities of this technology? Because it does seem like at a certain point, if you were to go from a more traditional setup to a more virtual one that maybe the possibility seems so endless that it’s difficult to nail down exactly what you’re looking for. So how do you help them walk through that process and maybe educate them on some best practices?

Chris Mollomo:

Sure. With any client that we’re talking about doing a virtual production with, we share a lot of existing videos of other projects that we’ve done. So people can hopefully quickly get a sense of what can be achieved. And then we point out some of the, we’re always developing new methods and new ways of doing things. I mentioned, virtual camera moves earlier. That’s breaking from a tracking shot and being able to fly off in space into a virtual environment, show any aspect of it. Of course, that’s something that we can’t do in the physical world. So it’s really about a process of education, sharing previous projects we’ve done. Really trying to understand what they’re trying to achieve and what their target market is. Some clients want to be very traditional and recreate what they’ve done in a hard set and others are looking to, the sky’s the limit.

Chris Mollomo:

We have clients across the spectrum. We have corporate clients. We have people in broadcast doing news and weather and sports. We have E-sports, which is perfect for virtual studios because, we’re leveraging a game engine to have these near photo realistic environments. So people in the gaming community quickly can understand what can be achieved and how to match the look of their game or of their broadcast using a virtual studio. So it’s really about targeting what they’re trying to achieve, who their target market is and educating them to what’s out there, what’s been done and what can be done.

Tyler Kern:

Gideon, is there anything you want to add on to that last question?

Gideon Ferber:

I think again, as Chris said, some customers are more traditional, some are out there. But in the core you need to always go back to what the production is trying to achieve. So whether, what’s the actual story the show is about. So, you can design your studio in any way, shape or form you want. You probably need to tie it back to the basics of broadcast, right? I mean, if we’re talking about data integration, if we’re talking about blocking shots that will make sense on the broadcast show. So it’s on one hand and again, pushing the boundaries visually, but then maintaining the workflow of a traditional broadcast station.

Gideon Ferber:

So, if I need now to have, I don’t know, 15 Unreal designers to run my show, then obviously that’s not a viable pipeline. So it’s, again, it’s finding the balance between pushing the boundaries on the visual side, but keeping it down to earth on the workflow and control that anyone can actually run it. If I now need to retrain my entire station on Unreal, then again, that’s extremely expensive exercise. So again, that’s only another angle, I think, on what Chris said.

Tyler Kern:

Okay. So Gideon talk me through a little bit more about Ross’s workflows and how they can apply to these specific situations and these elements.

Gideon Ferber:

As I said earlier, different vendors, pretty much everybody adopted Unreal as the core engine behind virtual set and augmented reality. So really, the differentiation between most of the companies is how do you actually control this engine? So Unreal gives you immense power, but as Spiderman said, “With great power comes great responsibility.” So our approach is what it is we’re trying to do. We’re trying to keep the traditional broadcast workflow in place. Designers design, operators operate. Journalists work in their newsroom environment. Lighting directors, responsible for the lighting and chroma key usually. So we’re trying to keep the same structure in place. For example, and I know we have to keep it more generic, but just small example from the way we do it in Ross, the same control interface that we have for Unreal is used whether I want to use it in the manual workflow. And I have an operator triggering the different events at different times, or I can use the same software, same UI to create templates that would be exposed in most workflow for [inaudible 00:18:02].

Gideon Ferber:

Then each journalists have its own access through the plugin, in their workspace to any AR or VS, any augmented reality or virtual set, event or behavior during the production. So for them it’s a completely transparent workload, right? The same way that they add a lower third to their story, they can now add the virtual monitor dropping from the ceiling and they can populate it with content, whether it’s images or video clips or live sources. Type the title underneath, define the duration and that’s it. And that will just save it to their stories, the same way that they’ve been doing well, since the newsroom workflow was developed. So, that’s the philosophy that we are adopting. So it’s, the engine is extremely powerful, but we’re trying to bring it into the traditional workflow. So which would be easier to adopt for the broadcaster.

Tyler Kern:

So, Chris, one of the things that Gideon just mentioned that I think is really interesting, was that use of AR right? And this seems like a prime opportunity to blend a little bit more AR in with physical existing sets, right? So mixing of the two, is that something that happens on a regular basis?

Chris Mollomo:

Yeah, absolutely. Augmented reality, we do a lot of augmented reality productions. I mentioned earlier E-sports has been using AR for years now, for some of their tournaments in a variety of ways. Showing the different teams, what the scores are, having avatars from the game, walk on stage, things like that, but AR is great. We have a team that does Sunday night football for NBC, and they are using AR, 10 to 15 times a broadcast dropping data-driven stats onto the field and being able to do full camera moves around them while on the field. So AR is a great way to enhance your broadcast.

Chris Mollomo:

It’s a great way in a studio setting to add some elements to your existing hard set in either, however you want to do that. Usually it’s data driven graphics using weather or talking about elections or things like that, but it’s a really powerful way to take us out of PowerPoint mode, so to speak and be able to visualize data in a different way. So it really helps to enhance storytelling and offer a look that you traditionally haven’t been able to do.

Gideon Ferber:

Two more comments about AR because that’s actually a really good point and Chris, thank you for opening the door here. One, AR is probably the easiest way for broadcasters to go into this virtual world. You don’t have to change your pipeline. You don’t have to change your set. You don’t have to change any of your workflows. You are adding an overlay layer on your existing pipeline. So I don’t have to change cameras. I don’t have to change graphic engines. I can add an add on to an existing setup. So this is the, usually the easiest way to start exploring this virtual vertical.

Gideon Ferber:

Second thing about AR, which is also pretty interesting. That’s a very easy way to generate revenue. So depends on the market you’re in. Some places, obviously you can’t, but again, depends on regulation. You can very easily create sponsorships and logos and branding and change them per show or change them per segment. Think about, in physical sets, if I have to put a flower logo off for sponsor, now I’m stuck with it. I can’t change it, unless I bring someone into scrape the floor and put a new one in. With virtual capabilities, but I can do that with a click of a button. So I can rend each segment of my show in a different way. So, that’s another huge advantage.

Tyler Kern:

Wow. Well, it really does seem like there are a lot of benefits and a lot of added capabilities through this type of technology. I think one of the things that really came across from this interview is that this is not as difficult to implement maybe as it was in the past, and so people shouldn’t be scared of the virtual technology, but really embrace it a little bit more and see what the capabilities are moving forward.

Gideon Ferber:

Absolutely.

Tyler Kern:

Well, these are exciting developments. It’s been really fun to get to learn a little bit more about virtual studios and how they can be beneficial and the ways that you’re guiding your clients to having success with these products. And so Gideon, thank you so much for joining me today.

Gideon Ferber:

Absolutely. My pleasure. Thank you

Tyler Kern:

And Chris, thank you for joining us as well.

Chris Mollomo:

All right. Thank you so much. Thanks for having me.