AI-Generated 3D Graphics to Deliver Augmented Reality Graphics: Improving Visual Storytelling with Machine Learning
How to cite
Abstract
The study analyzes the use of AI-generated 3D graphics to improve visual storytelling in augmented reality systems. The relevance of the work is due to the rapid development of machine learning technologies and their integration into various fields, including AR. The purpose of the study is to assess the potential of using AI to generate realistic and informative 3D graphics that contribute to more effective communication of context and enhance the immersion of AR content. The methods of analyzing scientific publications, comparative analysis of existing approaches and empirical testing of prototypes on a sample of 120 participants were used. The results demonstrate that AI models are capable of generating 3D objects that are not inferior to manually created ones in key parameters of realism (p<0.01), while providing a 47% increase in the speed of AR content development. A positive effect on user engagement (correlation coefficient 0.76) and understanding of the transmitted information (23% higher than the control group, p<0.05) were also revealed. The obtained data demonstrate significant potential for using AI-generated graphics to create a new level of AR experience. Prospects for developing the approach and its integration into existing development pipelines are discussed.
Keywords
References
Aberman K. Learning character-agnostic motion for motion retargeting in 2D // ACM transactions on graphics. 2019. Vol. 38. № 4. pp. 75:1-75:14.
Azuma R. A survey of augmented reality // Presence: teleoperators and virtual environments. 1997. Vol. 6. № 4. рр. 355-385.
Brock A., Donahue J., Simonyan K. Large scale GAN training for high fidelity natural image synthesis // ArXiv:1809.11096 (Cs, Stat). 2019.
Fišer J. StyleBlit: fast example-based stylization with local guidance // Computer graphics forum. 2016. Vol. 35. № 2. pp. 83-91.
Gatys L.A., Ecker A.S., Bethge M. Image style transfer using convolutional neural networks // 2016 IEEE conf. on computer vision and pattern recognition (CVPR). 2016. pp. 2414-2423.
Goodfellow I. Generative adversarial nets // Advances in neural information processing systems. 2014. Vol. 27.
Isola P. Image-to-image translation with conditional adversarial networks // 2017 IEEE conf. on computer vision and pattern recognition (CVPR). 2017. pp. 5967-5976.
Karras T. Progressive growing of GANs for improved quality, stability, and variation // ArXiv:1710.10196 (Cs, Stat). 2018.
Park T. Semantic image synthesis with spatially-adaptive normalization // 2019 IEEE/CVF conf. on computer vision and pattern recognition (CVPR). 2019. pp. 12294-12303.
Sangkloy P. Scribbler: controlling deep image synthesis with sketch and color // 2017 IEEE conf. on computer vision and pattern recognition (CVPR). 2017. pp. 6836-6845.
Wang T.-C. High-resolution image synthesis and semantic manipulation with conditional GANs // 2018 IEEE/CVF conf. on computer vision and pattern recognition. 2018. pp. 8798-8807.
Wang X. ESRGAN: Enhanced super-resolution generative adversarial networks // ECCV Workshops. 2018.
Wang X., Dunston P.S. Comparative effectiveness of mixed reality-based virtual environments in collaborative design // IEEE Transactions on systems, man and cybernetics. Part C. (Applications and reviews). 2011. Vol. 41. № 3. pp. 284-296.
Zhang R. Real-time user-guided image colorization with learned deep priors // ArXiv:1705.02999 (Cs). 2017.
Zhu J.-Y. Unpaired Image-to-Image translation using cycle-consistent adversarial networks // 2017 IEEE Inter. conf. on computer vision (ICCV). 2017. pp. 2242-2251.