When Does Sora Show:The Beginning of TAO to Imaginative Intelligence and Scenarios Engineering

DURING our discussion at workshops for writing“What Does ChatGPT Say:The DAO from Algorithmic Intelligence to Linguistic Intelligence”[1],we had expected the next milestone for Artificial Intelligence(AI)would be in the direction of Imaginative Intelligence(II),i.e.,something similar to automatic wordsto-videos generation or intelligent digital movies/theater technology that could be used for conducting new“Artificiofactual Experiments”[2]to replace conventional“Counterfactual Experiments”in scientific research and technical development for both natural and social studies[2]-[6].Now we have OpenAI’s Sora,so soon,but this is not the final,actually far away,and it is just the beginning.

Can Digital Intelligence and Cyber-Physical-Social Systems Achieve Global Food Security and Sustainability?

Plants sequester carbon through photosynthesis and provide primary productivity for the ecosystem. However, they also simultaneously consume water through transpiration, leading to a carbon-water balance relationship. Agricultural production can be regarded as a form of carbon sequestration behavior.From the perspective of the natural-social-economic complex ecosystem, excessive water usage in food production will aggravate regional water pressure for both domestic and industrial purposes. Hence, achieving a harmonious equilibrium between carbon and water resources during the food production process is a key scientific challenge for ensuring food security and sustainability. Digital intelligence(DI) and cyber-physical-social systems(CPSS) are emerging as the new research paradigms that are causing a substantial shift in the conventional thinking and methodologies across various scientific fields, including ecological science and sustainability studies. This paper outlines our recent efforts in using advanced technologies such as big data, artificial intelligence(AI), digital twins, metaverses, and parallel intelligence to model, analyze, and manage the intricate dynamics and equilibrium among plants, carbon, and water in arid and semiarid ecosystems. It introduces the concept of the carbon-water balance and explores its management at three levels: the individual plant level, the community level, and the natural-social-economic complex ecosystem level. Additionally, we elucidate the significance of agricultural foundation models as fundamental technologies within this context. A case analysis of water usage shows that, given the limited availability of water resources in the context of the carbon-water balance, regional collaboration and optimized allocation have the potential to enhance the utilization efficiency of water resources in the river basin. A suggested approach is to consider the river basin as a unified entity and coordinate the relationship between the upstream, midstream and downstream areas. Furthermore, establishing mechanisms for water resource transfer and trade among different industries can be instrumental in maximizing the benefits derived from water resources.Finally, we envisage a future of agriculture characterized by the integration of digital, robotic and biological farming techniques.This vision aims to incorporate small tasks, big models, and deep intelligence into the regular ecological practices of intelligent agriculture.

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

In the construction of Metaverses,sensors that are referred to as the“bridge of information transmission”,play a key role.The functionality and efficiency of today’s sensors,which operate in a manner similar to physical sensing,are frequently constrained by their hardware and software.In this research,we proposed the Parallel Sensing framework,which includes background,concept,basic methods and typical application of parallel sensing.In our formulation,sensors are redefined as the integration of real physical sensors and virtual software-defined sensors based on parallel intelligence,in order to boost the performance of the sensors.Each sensor will have a parallel counterpart in the virtual world within the framework of parallel sensing.Digital sensors serve as the brain of sensors and maintain the same properties as physical sensors.Parallel sensing allows physical sensors to operate in discrete time periods to conserve energy,while cloud-based descriptive,predictive,and prescriptive sensors operate continuously to offer compensation data and serve as guardians.To better illustrate parallel sensing concept,we show some example applications of parallel sensing such as parallel vision,parallel point cloud and parallel light fields,both of which are designed by construct virtual sensors to extend small real data to virtual big data and then boost the performance of perception models.Experimental results demonstrate the effective of parallel sensing framework.The interaction between the real and virtual worlds enables sensors to operate actively,allowing them to intelligently adapt to various scenarios and ultimately attain the goal of“Cognitive,Parallel,Crypto,Federated,Social and Ecologic”6S sensing.

Training and Testing Object Detectors With Virtual Images

In the area of computer vision, deep learning has produced a variety of state-of-the-art models that rely on massive labeled data. However, collecting and annotating images from the real world is too demanding in terms of labor and money investments, and is usually inflexible to build datasets with specific characteristics, such as small area of objects and high occlusion level. Under the framework of Parallel Vision, this paper presents a purposeful way to design artificial scenes and automatically generate virtual images with precise annotations.A virtual dataset named Parallel Eye is built, which can be used for several computer vision tasks. Then, by training the DPM(Deformable parts model) and Faster R-CNN detectors, we prove that the performance of models can be significantly improved by combining Parallel Eye with publicly available real-world datasets during the training phase. In addition, we investigate the potential of testing the trained models from a specific aspect using intentionally designed virtual datasets, in order to discover the flaws of trained models. From the experimental results, we conclude that our virtual dataset is viable to train and test the object detectors.

Integrated Inspection of QoM, QoP, and QoS for AOI Industries in Metaverses

With the rapid development of information technologies such as digital twin, extended reality, and blockchain,the hype around "metaverse" is increasing at astronomical speed. However, much attention has been paid to its entertainment and social functions. Considering the openness and interoperability of metaverses, the market of quality inspection promises explosive growth. In this paper, taking advantage of metaverses, we first propose the concept of Automated Quality Inspection(Auto QI), which performs integrated inspection covering the entire manufacturing process, including Quality of Materials, Quality of Manufacturing(Qo M), Quality of Products, Quality of Processes(Qo P), Quality of Systems, and Quality of Services(Qo S). Based on the scenarios engineering theory, we discuss how to perform interactions between metaverses and the physical world for virtual design instruction and physical validation feedback. Then we introduce a bottomup inspection device development workflow with productivity tools offered by metaverses, making development more effective and efficient than ever. As the core of quality inspection,we propose Quality Transformers to complete detection task,while federated learning is integrated to regulate data sharing.In summary, we point out the development directions of quality inspection under metaverse tide.

Exploring Image Generation for UAV Change Detection

Change detection(CD)is becoming indispensable for unmanned aerial vehicles(UAVs),especially in the domain of water landing,rescue and search.However,even the most advanced models require large amounts of data for model training and testing.Therefore,sufficient labeled images with different imaging conditions are needed.Inspired by computer graphics,we present a cloning method to simulate inland-water scene and collect an auto-labeled simulated dataset.The simulated dataset consists of six challenges to test the effects of dynamic background,weather,and noise on change detection models.Then,we propose an image translation framework that translates simulated images to synthetic images.This framework uses shared parameters(encoder and generator)and 22×22 receptive fields(discriminator)to generate realistic synthetic images as model training sets.The experimental results indicate that:1)different imaging challenges affect the performance of change detection models;2)compared with simulated images,synthetic images can effectively improve the accuracy of supervised models.

Parallel Driving with Big Models and Foundation Intelligence in Cyber-hysical-ocial Spaces

Recent years have witnessed numerous technical breakthroughs in connected and autonomous vehicles (CAVs). On the one hand, these breakthroughs have significantly advanced the development of intelligent transportation systems (ITSs);on the other hand, these new traffic participants introduce more complex and uncertain elements to ITSs from the social space. Digital twins (DTs) provide real-time, data-driven, precise modeling for constructing the digital mapping of physical-world ITSs. Meanwhile, the metaverse integrates emerging technologies such as virtual reality/mixed reality, artificial intelligence, and DTs to model and explore how to realize improved sustainability, increased efficiency, and enhanced safety. More recently, as a leading effort toward general artificial intelligence, the concept of foundation model was proposed and has achieved significant success, showing great potential to lay the cornerstone for diverse artificial intelligence applications across different domains. In this article, we explore the big models embodied foundation intelligence for parallel driving in cyber-physical-social spaces, which integrate metaverse and DTs to construct a parallel training space for CAVs, and present a comprehensive elucidation of the crucial characteristics and operational mechanisms. Beyond providing the infrastructure and foundation intelligence of big models for parallel driving, this article also discusses future trends and potential research directions, and the ?S?goals of parallel driving.

Sora for foundation robots with parallel intelligence:three world models,three robotic systems

This paper outlines the initial steps and basic framework for developing foundation/infrastructure robots/robotics based on foundation models and parallel intelligence,as well as the potential applications of new artificial intelligence(AI)techniques such as AlphaGO,ChatGPT,and Sora.