Mixed reality technologies provide real-time and immersive experiences,which bring tremendous opportunities in entertainment,education,and enriched experiences that are not directly accessible owing to safety or cost....Mixed reality technologies provide real-time and immersive experiences,which bring tremendous opportunities in entertainment,education,and enriched experiences that are not directly accessible owing to safety or cost.The research in this field has been in the spotlight in the last few years as the metaverse went viral.The recently emerging omnidirectional video streams,i.e.,360°videos,provide an affordable way to capture and present dynamic real-world scenes.In the last decade,fueled by the rapid development of artificial intelligence and computational photography technologies,the research interests in mixed reality systems using 360°videos with richer and more realistic experiences are dramatically increased to unlock the true potential of the metaverse.In this survey,we cover recent research aimed at addressing the above issues in the 360°image and video processing technologies and applications for mixed reality.The survey summarizes the contributions of the recent research and describes potential future research directions about 360°media in the field of mixed reality.展开更多
Virtual reality(VR)offers an artificial,computer generated simulation of a real life environment.It originated in the 1960 s and has evolved to provide increasing immersion,interactivity,imagination,and intelligence.B...Virtual reality(VR)offers an artificial,computer generated simulation of a real life environment.It originated in the 1960 s and has evolved to provide increasing immersion,interactivity,imagination,and intelligence.Because deep learning systems are able to represent and compose information at various levels in a deep hierarchical fashion,they can build very powerful models which leverage large quantities of visual media data.Intelligence of VR methods and applications has been significantly boosted by the recent developments in deep learning techniques.VR content creation and exploration relates to image and video analysis,synthesis and editing,so deep learning methods such as fully convolutional networks and general adversarial networks are widely employed,designed specifically to handle panoramic images and video and virtual 3 D scenes.This article surveys recent research that uses such deep learning methods for VR content creation and exploration.It considers the problems involved,and discusses possible future directions in this active and emerging research area.展开更多
基金supported by the Marsden Fund Council managed by Royal Society of New Zealand under Grant Nos.MFP-20-VUW-180 and UOO1724Zhejiang Province Public Welfare Technology Application Research under Grant No.LGG22F020009the Key Lab of Film and TV Media Technology of Zhejiang Province of China under Grant No.2020E10015.
文摘Mixed reality technologies provide real-time and immersive experiences,which bring tremendous opportunities in entertainment,education,and enriched experiences that are not directly accessible owing to safety or cost.The research in this field has been in the spotlight in the last few years as the metaverse went viral.The recently emerging omnidirectional video streams,i.e.,360°videos,provide an affordable way to capture and present dynamic real-world scenes.In the last decade,fueled by the rapid development of artificial intelligence and computational photography technologies,the research interests in mixed reality systems using 360°videos with richer and more realistic experiences are dramatically increased to unlock the true potential of the metaverse.In this survey,we cover recent research aimed at addressing the above issues in the 360°image and video processing technologies and applications for mixed reality.The survey summarizes the contributions of the recent research and describes potential future research directions about 360°media in the field of mixed reality.
基金supported by the National Natural Science Foundation of China(Grant Nos.61902012,61932003)supported by a Victoria Early-Career Research Excellence Award.
文摘Virtual reality(VR)offers an artificial,computer generated simulation of a real life environment.It originated in the 1960 s and has evolved to provide increasing immersion,interactivity,imagination,and intelligence.Because deep learning systems are able to represent and compose information at various levels in a deep hierarchical fashion,they can build very powerful models which leverage large quantities of visual media data.Intelligence of VR methods and applications has been significantly boosted by the recent developments in deep learning techniques.VR content creation and exploration relates to image and video analysis,synthesis and editing,so deep learning methods such as fully convolutional networks and general adversarial networks are widely employed,designed specifically to handle panoramic images and video and virtual 3 D scenes.This article surveys recent research that uses such deep learning methods for VR content creation and exploration.It considers the problems involved,and discusses possible future directions in this active and emerging research area.
