MR360

Seamlessly composite interactive digital objects into live streaming 360 degree video in real time!

A toolkit for automating realistic lighting and shadows for digital objects composited into live streaming 360 degree video; perfect for creating exciting mixed reality experiences.

Innovation overview

MR360 is a toolkit which integrates into Unreal Engine [Unity version currently in development] making it easy to import a 360 degree video and create a realistic interactive experience for virtual reality.

Using Victoria University of Wellington's patent pending algorithm, the toolkit automates image based lighting and light detection resulting in realistic looking digital objects with shadows that composite seamlessly into a 360 degree video.

With no precomputation, the toolkit is suitable for use with streaming video, allowing interactive experiences to be brought to a variety of events such as live sports, or a live streamed conference etc.

Features

Easy Composition
The toolkit makes working with 360 degree video and compositing digital objects a breeze!
Realistic Lighting & Shadows
The toolkit automates image based lighting and light detection for shadows in real time from the 360 degree video. Swap the video out, and the IBL and shadows change to suit instantly!
Do it live!
The toolkit changes the lighting and shadows of digital objects in real time with no precomputation, this makes it perfect for use with live footage!

Applications

Telepresence for Business
Don't just make a Skype call, bring your collaborators into the room virtually and let them interact with digital objects which appear to be really there!
Live Sporting Events
Composite interactive elements into a live mixed reality sporting experience.
Turn viewers into participants, let them wave their flag, interact with digital mascots, or engage with more effective advertising!
Interior Design
Want your clients to see how that new sofa will look in their lounge before they buy it? Use MR360 to take your library of products into a customer's home with stunning realism!
Architecture
Film your site in various conditions and use MR360 to place your architectural model in a real scene and then study how natural light will affect your interior! Also perfect for showing clients.

Technology Diagram

Research leader

Associate Professor Taehyun Rhee

AProf Rhee (also known as TJ) joined Victoria University in 2012, after 17 years working in product development for Samsung. During his time with the consumer electronics conglomerate, he oversaw a 3D virtual prototyping and visualisation system that resulted in the creation of over 200 products—and reached out to leading universities in the United States to form relationships, and work with resea...

Contact person

Liam Sutton

Liam joined the Viclink team as an Assistant Commercialisation Manager on 1 April 2016. Although he only recently graduated from Victoria with a Bachelor of Architecture, Liam is no stranger to commercialisation, having started up his own business at the 2015-2016 Victoria Entrepreneur Bootcamp. Responsible for developing and formalising processes that will make the innovation pipeline more effici...

See Liam Sutton's technology portfolio