Integration of Manufacturing Services into Virtual Environments over the Internet

Virtual reality (VR) is a rapidly developing technology that has a wide spectrum of industrial and commercial applications. Networked (distributed or shared) virtual environments (VE) are of growing interest to modern manufacturing industry; a dominating use of networked virtual manufacturing environments (VMEs) is on-line visualisation and collaborative control of 3D information. This has to be supported by real-time data transfer. To meet a broad range of common requirements for Internet-based VE communications, particularly for virtual manufacturing and collaborative design and control, this paper presents a networked virtual environment system that is designed to support networked virtual design and manufacturing. The system is implemented with manufacturing message specification (MMS) standards so as to integrate a range of manufacturing services into networked VEs over the Internet.

A Hierarchical Framework for Visualising and Simulating Supply Chains in Virtual Environments

This paper presents research into applying virtual environment (VE) technology to supply chain management (SCM). Our research work has employed virtual manufacturing environments to represent supply chain nodes to simulate processes and activities in supply chain management. This will enable those who are involved in these processes and activities to gain an intuitive understanding of them, so as to design robust supply chains and make correct decisions at the right time. A framework system and its hierarchical structure for visualising and simulating supply chains in virtual environments are reported and detailed in this paper.

Generation and Control of Game Virtual Environment

In this paper, we present a framework for the generation and control of an Internet-based 3-dimensional game virtual environment that allows a character to navigate through the environment. Our framework includes 3-dimensional terrain mesh data processing, a map editor, scene processing, collision processing, and walkthrough control. We also define an environment-specific semantic information editor, which can be applied using specific location obtained from the real world. Users can insert text information related to the characters real position in the real world during navigation in the game virtual environment.

Realization of Reconfigurable Virtual Environments for Virtual Testing

This paper presents the design and implementation of reconfigurable virtual environments (VEs) for virtual testing. It proposes a hybrid design approach that is derived from a so-called integration and composition of the reconfiguration strategy. The designing process has thus evolved from binding virtual objects using reconfiguration rules within the context of virtual testing scenarios. Therefore reconfigurable virtual environments are established with improved flexibility and scalability, tailored to a wide range of virtual testing applications. Those virtual environments integrate virtual testing scenarios, data acquisition, databases, rule mapping and application interfaces, which yield modular testing functions and an open-ended system architecture with a set of extensible interface tools to realize data exchange within reconfigurable VEs. This enables virtual testing scenarios to be reconfigured interactively based on real time data and communication between virtual environments and real environments. A virtual testing application has been implemented using reconfigurable VEs. Keywords Virtual environment - virtual testing - reconfigurable virtual reality Wenyan Wu graduated from Dalian University of Technology, China, with BSc and MSc in 1988 and 1991 respectively. She earned her PhD degree from University of Derby, UK, in 2002. She had taught and researched in Harbin Institute of Technology, China and De Montfort University, UK. She is currently a senior lecturer in simulation and virtual reality at Staffordshire University, UK. Her research interests include computer graphics, Virtual Reality and Augmented Reality system, advanced interface, modelling and simulation, distribution system.Zhengxu Zhao BSc, MSc, PhD, CEng, CITP, MBCS, full Professor in Applied Computing, Director of Virtual Reality Centre. He is also a Cheng Kong Scholar Project Professor in Virtual Reality Technology and Director of Virtual Reality Research Centre at the Southeast University, P R China. Professor Zhao’s research interests include computing graphics and VR systems, industrial process simulation, CIM and manufacturing management systems.

A Mobile Agent-Based Prototype of HeterogeneousDistributed Virtual Environment Systems

Mobile agents provide a new method for the distributed computation. This paper presents the advantages of using mobile agents in a distributed virtual environment (DVE) system, and describes the architecture of heterogeneous computer's distributed virtual environment system (HCWES) designed to populate some mobile agents as well as stationary agents. Finally, the paper introduces how heterogeneous computer network communication is to be realized.

Real-time Virtual Environment Signal Extraction and Denoising Using Programmable Graphics Hardware

The sense of being within a three-dimensional （3D） space and interacting with virtual 3D objects in a computer-generated virtual environment （VE） often requires essential image, vision and sensor signal processing techniques such as differentiating and denoising. This paper describes novel implementations of the Gaussian filtering for characteristic signal extraction and waveletbased image denoising algorithms that run on the graphics processing unit （GPU）. While significant acceleration over standard CPU implementations is obtained through exploiting data parallelism provided by the modern programmable graphics hardware, the CPU can be freed up to run other computations more efficiently such as artificial intelligence （AI） and physics. The proposed GPU-based Gaussian filtering can extract surface information from a real object and provide its material features for rendering and illumination. The wavelet-based signal denoising for large size digital images realized in this project provided better realism for VE visualization without sacrificing real-time and interactive performances of an application.

Forecasting display technique of human in network virtual environment

The simulation of human in the distributed virtual environment is different from the simulation of vehicle. Due to the human’s many joints, to implement the real time performance of the interaction between the virtual human and the environment, the quantity of information broadcasted in the network will be increased. So with the growing of the quantity of virtual human added to the network, the network burthen would augment rapidly. Therefore, for the simulation of virtual human this paper divides organically the motion parts of the body, which makes those body parts that often interact with the environment are described according to the joints while other parts are described according to the action information. Simultaneity, this paper adopts Dead Reckoning to reduce the quantity of information needed by maintaining the reality motion of virtual human. The experiment indicates that the forecasting display technique can efficiently reduce the network burthen and enhance the interaction of virtual human.

Effects of virtual agents on interaction efficiency and environmental immersion in MR environments

Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.

The Trade-Off Between Performance and Security of Virtualized Trusted Execution Environment on Android

Nowadays,with the significant growth of the mobile market,security issues on the Android Operation System have also become an urgent matter.Trusted execution environment(TEE)technologies are considered an option for satisfying the inviolable property by taking advantage of hardware security.However,for Android,TEE technologies still contain restrictions and limitations.The first issue is that non-original equipment manufacturer developers have limited access to the functionality of hardware-based TEE.Another issue of hardware-based TEE is the cross-platform problem.Since every mobile device supports different TEE vendors,it becomes an obstacle for developers to migrate their trusted applications to other Android devices.A software-based TEE solution is a potential approach that allows developers to customize,package and deliver the product efficiently.Motivated by that idea,this paper introduces a VTEE model,a software-based TEE solution,on Android devices.This research contributes to the analysis of the feasibility of using a virtualized TEE on Android devices by considering two metrics:computing performance and security.The experiment shows that the VTEE model can host other software-based TEE services and deliver various cryptography TEE functions on theAndroid environment.The security evaluation shows that adding the VTEE model to the existing Android does not addmore security issues to the traditional design.Overall,this paper shows applicable solutions to adjust the balance between computing performance and security.

Stereoscopic Camera-Sensor Model for the Development of Highly Automated Driving Functions within a Virtual Test Environment

The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments and virtual sensor models. In the context of this, the present paper documents the development of a sensor model for depth estimation of virtual three-dimensional scenarios. For this purpose, the geometric and algorithmic principles of stereoscopic camera systems are recreated in a virtual form. The model is implemented as a subroutine in the Epic Games Unreal Engine, which is one of the most common Game Engines. Its architecture consists of several independent procedures that enable a local depth estimation, but also a reconstruction of a whole three-dimensional scenery. In addition, a separate programme for calibrating the model is presented. In addition to the basic principles, the architecture and the implementation, this work also documents the evaluation of the model created. It is shown that the model meets specifically defined requirements for real-time capability and the accuracy of the evaluation. Thus, it is suitable for the virtual testing of common algorithms and highly automated driving functions.

An interactive approach to investigate brightness perception of daylighting in Immersive Virtual Environments:Comparing subjective responses and quantitative metrics

Daylighting plays an eminent role in the performance of indoor environments and their occupants,thus necessitating the need to investigate daylight perception of potential occupants at early design stages.The present study introduces an interactive approach to collect and visualize brightness perception of daylighting in a large-scale immersive virtual environment using a game engine as a daylight simulation tool.The developed system allows users to explore building models freely at different day times set in virtual reality and report their perceptions in real time.Following a validation study(N=36)to investigate the consistency of brightness perceptions in a real environment and its virtual replica,a set of 24 participants were recruited to use the system to report their brightness perception in a virtual model of a daylit art museum,through snapshotting the scenes where they perceive as one of the following in terms of daylighting:(very dark,dark,bright,or very bright).Using an output of 419 snapshots,a"Perceptual Light Map"(PLM)was developed to visualize the collective brightness perception of participants as a heat map.Subjective responses were found to be positively correlated with four daylight metrics,with the highest correlation to mean luminance and the lowest to luminance ratio.The findings of this exploratory study represent a step towards a user-oriented supplement tool to the existing quantitative daylight metrics,validating game engines’adequacy as a daylight simulation tool,and illustrating the potentials of immersion and interaction principles for the perception of daylit spaces in virtual reality.

A hybrid brain-computer interface control strategy in a virtual environment

This paper presents a hybrid brain-computer interface (BCI) control strategy,the goal of which is to expand control functions of a conventional motor imagery or a P300 potential based BCI in a virtual environment.The hybrid control strategy utilizes P300 potential to control virtual devices and motor imagery related sensorimotor rhythms to navigate in the virtual world.The two electroencephalography (EEG) patterns serve as source signals for different control functions in their corresponding system states,and state switch is achieved in a sequential manner.In the current system,imagination of left/right hand movement was translated into turning left/right in the virtual apartment continuously,while P300 potentials were mapped to discrete virtual device control commands using a five-oddball paradigm.The combination of motor imagery and P300 patterns in one BCI system for virtual environment control was tested and the results were compared with those of a single motor imagery or P300-based BCI.Subjects obtained similar performances in the hybrid and single control tasks,which indicates the hybrid control strategy works well in the virtual environment.

An Orientation Update Message Filtering Algorithm in Collaborative Virtual Environments

Orientation update message filtering is an important issue in collaborativevirtual environments (CVEs). Dead-reckoning (DR) is a known effective mechanism for update messagefiltering. Yet, previous dead-reckoning techniques mainly focus on the update message filtering forpositions. The existing orientation dead-reckoning algorithms are based on fixed threshold values.The drawbacks of fixed thresholding for orientations (FTO) are discussed in this paper. We propose avariable thresholding for orientations (VTO) based on average recent angular velocity. The mainadvantage of the proposed VTO is the ability of balancing the number of state update messages andshift frequency of direction and speed of rotation.

Geovisualization with immersive virtual environments in theory and practice

Virtual reality(VR)is a frequently emphasized issue on the Digital Earth(DE)agenda.While current DE research is more engaged in the technical aspects of VR applications,this paper focuses on what is possible with immersive virtual environments(IVE)from the user’s perspective.After a brief discussion of spatial presence and embodiment in the context of IVE,both concepts will be merged into a geovisualization immersion pipeline(GIP)as a framework with which to systematically link technical and cognitive aspects of IVE.We will then analyze the general criteria that must be met by IVE in order to facilitate the experience of spatial presence.Adapting these criteria to the special requirements of geospatial data,a definition of geovisualization immersive virtual environments(GeoIVE)is formulated.Finally,the theoretical considerations of this paper are set into practice,using a GeoIVE of a coral reef ecosystem as an example.As we shall see,while GIS do not provide data models or data compatibility for direct visualization of GeoIVE on VR output devices,game engines can serve as middleware to fill this gap.

Creation and Evaluation of a Multi-sensory Virtual Assembly Environment

The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of information that is accepted and processed by the user＇s perception system. The increase of the useful feedback information may reduce the user＇s cognitive load, thus enhancing the user＇s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment （VAE） and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes： efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times （TCTs）, questionnaires, and human performance error rates （HPERs）, respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback （visual or auditory） or no feedback. The participants＇ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.

Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system

An Augmented virtual environment(AVE)is concerned with the fusion of real-time video with 3D models or scenes so as to augment the virtual environment.In this paper,a new approach to establish an AVE with a wide field of view is proposed,including real-time video projection,multiple video texture fusion and 3D visualization of moving objects.A new diagonally weighted algorithm is proposed to smooth the apparent gaps within the overlapping area between the two adjacent videos.A visualization method for the location and trajectory of a moving virtual object is proposed to display the moving object and its trajectory in the 3D virtual environment.The experimental results showed that the proposed set of algorithms are able to fuse multiple real-time videos with 3D models efficiently,and the experiment runs a 3D scene containing two million triangles and six real-time videos at around 55 frames per second on a laptop with 1GB of graphics card memory.In addition,a realistic AVE with a wide field of view was created based on the Digital Earth Science Platform by fusing three videos with a complex indoor virtual scene,visualizing a moving object and drawing its trajectory in the real time.

Measuring user preferences and behaviour in a topographic immersive virtual environment(TopoIVE)of 2D and 3D urban topographic data

This paper investigates user preferences and behaviour associated with 2D and 3D modes of urban representation within a novel Topographic Immersive Virtual Environment(TopoIVE)created from official 1:10,000 mapping.Sixty participants were divided into two groups:the first were given a navigational task within a simulated city and the second were given the freedom to explore it.A Head-Mounted Display(HMD)Virtual Reality(VR)app allowed participants to switch between 2D and 3D representations of buildings with a remote controller and their use of these modes during the experiment was recorded.Participants performed mental rotation tests before entering the TopoIVE and were interviewed afterwards about their experiences using the app.The results indicate that participants preferred the 3D mode of representation overall,although preference for the 2D mode was slightly higher amongst those undertaking the navigational task,and reveal that different wayfinding solutions were adopted by participants according to their gender.Overall,the findings suggest that users exploit different aspects of 2D and 3D modes of visualization in their wayfinding strategy,regardless of their task.The potential to combine the functionality of 2D and 3D modes therefore offers substantial opportunities for the development of immersive virtual reality products derived from topographic datasets.

Identifying subassemblies and understanding their functions during a design review in immersive and non-immersive virtual environments

Design review(DR)is a product development(PD)activity used to inspect the technical characteristics of a design solution.Immersive virtual reality(IVR)technology enables the presentation of spatial information and interaction with 3D CAD models inside an immersive virtual environment(IVE).Such capabilities have shown the potential to mitigate the cognitive load needed for the visual perception of spatial information and,consequently,enhance design understanding and DR performance.Thus,an increasing number of studies have explored the effect of IVR technology on DR activities in different domains.However,determining when the implementation of IVR technology rather than a conventional user interface for DRs in mechanical engineering PD projects will be beneficial remains unclear.Hence,a conceptual DR experimental study was conducted to investigate the differences in the ability of engineering students to identify mechanisms and understand their functions when a design solution for a technical system is presented in an IVE by IVR technology and in a non-immersive virtual environment(nIVE)by a conventional user interface(monitor display,keyboard,and mouse).Data were collected by performing DR tasks and having participants complete a prior experience questionnaire,presence questionnaire,and mental rotations test.Findings of the study indicate that IVR does not support an enhanced ability of engineering students to identify mechanisms and understand their functions compared with a conventional user interface.

Real-Time Optimal Reach-Posture Prediction in a New Interactive Virtual Environment

Human posture prediction is a key factor for the design and evaluation of workspaces, in a virtual environment using virtual humans. This work presents a new interface and virtual environment for the direct human optimized posture prediction （D-HOPP） approach to predicting realistic reach postures of digital humans, where reach postures entail the use of the torso, arms, and neck. D-HOPP is based on the contention where depending on what type of task is being completed, and human posture is governed by different human performance measures. A human performance measure is a physics-based metric, such as energy or discomfort, and serves as an objective function in an optimization formulation. The problem is formulated as a single-objective optimization （SO0） problem with a single performance measure and as multiobjective-optimization （MOO） problem with multiple combined performance measures. We use joint displacement, change in potential energy, and musculoskeletal discomfort as performance measures. D-HOPP is equipped with an extensive yet intuitive user-interface, and the results are presented in an interactive virtual environment.

3D Path prediction of moving objects in a video-augmented indoor virtual environment

Augmented virtual environments(AVE)combine real-time videos with 3D scenes in a Digital Earth System or 3D GIS to present dynamic information and a virtual scene simultaneously.AVE can provide solutions for continuous tracking of moving objects,camera scheduling,and path planning in the real world.This paper proposes a novel approach for 3D path prediction of moving objects in a video-augmented indoor virtual environment.The study includes 3D motion analysis of moving objects,multi-path prediction,hierarchical visualization,and path-based multi-camera scheduling.The results show that these methods can give a closed-loop process of 3D path prediction and continuous tracking of moving objects in an AVE.The path analysis algorithms proved accurate and time-efficient,costing less than 1.3 ms to get the optimal path.The experiment ran a 3D scene containing 295,000 triangles at around 35 frames per second on a laptop with 1 GB of graphics card memory,which means the performance of the proposed methods is good enough to maintain high rendering efficiency for a video-augmented indoor virtual scene.