The research on legged robots attracted much attention both from the academia and industry. Legged robots are multi-input multi-output with multiple end-e ector systems. Therefore,the mechanical design and control fra...The research on legged robots attracted much attention both from the academia and industry. Legged robots are multi-input multi-output with multiple end-e ector systems. Therefore,the mechanical design and control framework are challenging issues. This paper reviews the development of type synthesis and behavior control on legged robots; introduces the hexapod robots developed in our research group based on the proposed type synthesis method. The control framework for legged robots includes data driven layer,robot behavior layer and robot execution layer. Each layer consists several components which are explained in details. Finally,various experiments were conducted on several hexapod robots. The summarization of the type synthesis and behavior control design constructed in this paper would provide a unified platform for communications and references for future advancement for legged robots.展开更多
Robot’s living space Human-robot relationship Robot humanization Hybrid intelligence Human-robot integrated society Robots,as the creation of humans,became an irreplaceable component in human society.With the advance...Robot’s living space Human-robot relationship Robot humanization Hybrid intelligence Human-robot integrated society Robots,as the creation of humans,became an irreplaceable component in human society.With the advancement of technologies,robots have become more and more intelligent and have been widely used in many fields,such as disease diagnosis,customer services,healthcare for the older people,and so on.As robots made our lives much more convenient than ever before,they also brought many potential risks and challenges in technology,security,and ethic.To better understand the development of robots,we proposed a concept of a robot’s living space and analyzed the role of robots in our society.In this paper,we focus on setting a theoretical framework of the robot’s living space to further understand the human-robot relationship.The research in this paper contains three central aspects.First,we interpret the concept of the robot’s living space and the functions of each space.Second,we analyze and summarize the relative technologies which support robots living well in each space.Finally,we provide advice and improvement measures based on a discussion of potential problems caused by the developments of robots.With the trend of robots humanization and human-robot society integration,we should seriously consider how to collaborate with intelligent robots to achieve hybrid intelligence.To build a harmonious human-robot integrated society,studying the robot’s living space and its relationship with humans is the prerequisite and roadmap.展开更多
A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress ...A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress the direction of the gaze by analyzing images of the eyes,also known as eye patches.However,it is very difficult to construct a person-independent model that can estimate an accurate gaze direction for every person due to individual differences.In this paper,we hypothesize that the difference in the appearance of each of a person’s eyes is related to the difference in the corresponding gaze directions.Based on this hypothesis,a differential eyes’appearances network(DEANet)is trained on public datasets to predict the gaze differences of pairwise eye patches belonging to the same individual.Our proposed DEANet is based on a Siamese neural network(SNNet)framework which has two identical branches.A multi-stream architecture is fed into each branch of the SNNet.Both branches of the DEANet that share the same weights extract the features of the patches;then the features are concatenated to obtain the difference of the gaze directions.Once the differential gaze model is trained,a new person’s gaze direction can be estimated when a few calibrated eye patches for that person are provided.Because personspecific calibrated eye patches are involved in the testing stage,the estimation accuracy is improved.Furthermore,the problem of requiring a large amount of data when training a person-specific model is effectively avoided.A reference grid strategy is also proposed in order to select a few references as some of the DEANet’s inputs directly based on the estimation values,further thereby improving the estimation accuracy.Experiments on public datasets show that our proposed approach outperforms the state-of-theart methods.展开更多
Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out(POCO)around nuclear facilities each year,resulting in a trend towards robotic deployments to gain an impro...Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out(POCO)around nuclear facilities each year,resulting in a trend towards robotic deployments to gain an improved understanding during nuclear decommissioning phases.The UK Nuclear Decommissioning Authority supports the view that human-in-the-loop(HITL)robotic deployments are a solution to improve procedures and reduce risks within radiation characterisation of nuclear sites.The authors present a novel implementation of a Cyber-Physical System(CPS)deployed in an analogue nuclear environment,comprised of a multi-robot(MR)team coordinated by a HITL operator through a digital twin interface.The development of the CPS created efficient partnerships across systems including robots,digital systems and human.This was presented as a multi-staged mission within an inspection scenario for the hetero-geneous Symbiotic Multi-Robot Fleet(SMuRF).Symbiotic interactions were achieved across the SMuRF where robots utilised automated collaborative governance to work together,where a single robot would face challenges in full characterisation of radiation.Key contributions include the demonstration of symbiotic autonomy and query-based learning of an autonomous mission supporting scalable autonomy and autonomy as a service.The coordination of the CPS was a success and displayed further challenges and improvements related to future MR fleets.展开更多
Background The uncanny valley hypothesis states that users may experience discomfort when inter-acting with almost human-like artificial characters.Advancements in artificial intelligence,robotics,and computer graphic...Background The uncanny valley hypothesis states that users may experience discomfort when inter-acting with almost human-like artificial characters.Advancements in artificial intelligence,robotics,and computer graphics have led to the development of life-like virtual humans and humanoid robots.Revisiting this hypothesis is necessary to check whether they positively or negatively affect the current population,who are highly accustomed to the latest technologies.Methods In this study,we present a unique evaluation of the uncanny valley hypothesis by allowing participants to interact live with four humanoid robots that have varying levels of human-likeness.Each participant completed a survey questionnaire to evaluate the affinity of each robot.Additionally,we used deep learning methods to quantify the participants’emotional states using multimodal cues,including visual,audio,and text cues,by recording the participant-robot interactions.Results Multi-modal analysis and surveys provided interesting results and insights into the uncanny valley hypothesis.展开更多
Humanoid robots are designed and built to mimic human form and movement. Ultimately, they are meant to resemble the size and physical abilities of a human in order to function in human-oriented environments and to wor...Humanoid robots are designed and built to mimic human form and movement. Ultimately, they are meant to resemble the size and physical abilities of a human in order to function in human-oriented environments and to work autonomously but to pose no physical threat to humans. Here, a humanoid robot that resembles a human in appearance and movement is built using powerful actuators paired with gear trains, joint mechanisms, and motor drivers that are all encased in a package no larger than that of the human physique. In this paper, we propose the construction of a humanoid-applicable anthropomorphic 7-DoF arm complete with an 8-DoF hand. The novel mechanical design of this humanoid ann makes it sufficiently compact to be compatible with currently available narrating-model humanoids, and to be sufficiently powerful and flexible to be functional; the number of degrees of freedom endowed in this robotic arm is sufficient for executing a wide range of tasks, including dexterous hand movements. The developed humanoid arm and hand are capable of sensing and interpreting incoming external force using the motor in each joint current without conventional torque sensors. The humanoid ann adopts an algorithm to avoid obstacles and the dexterous hand is capable of grasping objects. The developed robotic ann is suitable for use in an interactive humanoid robot.展开更多
In the context of collaborative robotics,distributed situation awareness is essential for supporting collective intelligence in teams of robots and human agents where it can be used for both individual and collective ...In the context of collaborative robotics,distributed situation awareness is essential for supporting collective intelligence in teams of robots and human agents where it can be used for both individual and collective decision support.This is particularly important in applications pertaining to emergency rescue and crisis management.During operational missions,data and knowledge are gathered incrementally and in different ways by heterogeneous robots and humans.We describe this as the creation of Hastily Formed Knowledge Networks(HFKNs).The focus of this paper is the specification and prototyping of a general distributed system architecture that supports the creation of HFKNs by teams of robots and humans.The information collected ranges from low-level sensor data to high-level semantic knowledge,the latter represented in part as RDF Graphs.The framework includes a synchronization protocol and associated algorithms that allow for the automatic distribution and sharing of data and knowledge between agents.This is done through the distributed synchronization of RDF Graphs shared between agents.High-level semantic queries specified in SPARQL can be used by robots and humans alike to acquire both knowledge and data content from team members.The system is empirically validated and complexity results of the proposed algorithms are provided.Additionally,a field robotics case study is described,where a 3D mapping mission has been executed using several UAVs in a collaborative emergency rescue scenario while using the full HFKN Framework.展开更多
As a wearable and intelligent system, a lower limb exoskeleton rehabilitation robot can provide auxiliary rehabilitation training for patients with lower limb walking impairment/loss and address the existing problem o...As a wearable and intelligent system, a lower limb exoskeleton rehabilitation robot can provide auxiliary rehabilitation training for patients with lower limb walking impairment/loss and address the existing problem of insufficient medical resources. One of the main elements of such a human–robot coupling system is a control system to ensure human–robot coordination. This review aims to summarise the development of human–robot coordination control and the associated research achievements and provide insight into the research challenges in promoting innovative design in such control systems. The patients’ functional disorders and clinical rehabilitation needs regarding lower limbs are analysed in detail, forming the basis for the human–robot coordination of lower limb rehabilitation robots. Then, human–robot coordination is discussed in terms of three aspects: modelling, perception and control. Based on the reviewed research, the demand for robotic rehabilitation, modelling for human–robot coupling systems with new structures and assessment methods with different etiologies based on multi-mode sensors are discussed in detail, suggesting development directions of human–robot coordination and providing a reference for relevant research.展开更多
Shared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance.However,prior work that seeks to find the optimal shared control ratio needs an a...Shared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance.However,prior work that seeks to find the optimal shared control ratio needs an accurate human model,which is usually challenging to obtain.In this study,the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient(DDPG)(TD3X)-based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally.The robot's states,shared control ratio in the previous time step,and human's control input is used as inputs to the reinforcement learning(RL)agent,which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model.Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio.Furthermore,to accelerate the training process and protect the robot,a navigation demonstration policy and a safety guard are developed.A neural network(NN)structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed.In addition,an extended DAGGER(DAGGERX)human agent is developed for training the RL agent to reduce human workload.Robot simulations and experiments with humans in the loop are conducted.The results show that the DAGGERX human agent can simulate real human inputs in the worst-case scenarios with a mean square error of 0.0039.Compared to the original TD3 agent,the TD3X-based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment.The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment,while the performance is equally good in the complex environment because of the use of an advanced human agent.In the human subject tests,participants'average perceived workload was significantly lower in shared control than that in exclusively manual control(26.90 vs.40.07,p=0.013).展开更多
基金Supported by National Natural Science Foundation of China(Grant No.U1613208)
文摘The research on legged robots attracted much attention both from the academia and industry. Legged robots are multi-input multi-output with multiple end-e ector systems. Therefore,the mechanical design and control framework are challenging issues. This paper reviews the development of type synthesis and behavior control on legged robots; introduces the hexapod robots developed in our research group based on the proposed type synthesis method. The control framework for legged robots includes data driven layer,robot behavior layer and robot execution layer. Each layer consists several components which are explained in details. Finally,various experiments were conducted on several hexapod robots. The summarization of the type synthesis and behavior control design constructed in this paper would provide a unified platform for communications and references for future advancement for legged robots.
基金supported by the Key the Lab of Information Network Security of Ministry of Public Security(The Third Research Institute of Ministry of Public Security)and Civil Aviation Joint Funds of the National Natural Science Foundation of China(Grant No.U1633121).
文摘Robot’s living space Human-robot relationship Robot humanization Hybrid intelligence Human-robot integrated society Robots,as the creation of humans,became an irreplaceable component in human society.With the advancement of technologies,robots have become more and more intelligent and have been widely used in many fields,such as disease diagnosis,customer services,healthcare for the older people,and so on.As robots made our lives much more convenient than ever before,they also brought many potential risks and challenges in technology,security,and ethic.To better understand the development of robots,we proposed a concept of a robot’s living space and analyzed the role of robots in our society.In this paper,we focus on setting a theoretical framework of the robot’s living space to further understand the human-robot relationship.The research in this paper contains three central aspects.First,we interpret the concept of the robot’s living space and the functions of each space.Second,we analyze and summarize the relative technologies which support robots living well in each space.Finally,we provide advice and improvement measures based on a discussion of potential problems caused by the developments of robots.With the trend of robots humanization and human-robot society integration,we should seriously consider how to collaborate with intelligent robots to achieve hybrid intelligence.To build a harmonious human-robot integrated society,studying the robot’s living space and its relationship with humans is the prerequisite and roadmap.
基金supported by the Science and Technology Support Project of Sichuan Science and Technology Department(2018SZ0357)and China Scholarship。
文摘A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress the direction of the gaze by analyzing images of the eyes,also known as eye patches.However,it is very difficult to construct a person-independent model that can estimate an accurate gaze direction for every person due to individual differences.In this paper,we hypothesize that the difference in the appearance of each of a person’s eyes is related to the difference in the corresponding gaze directions.Based on this hypothesis,a differential eyes’appearances network(DEANet)is trained on public datasets to predict the gaze differences of pairwise eye patches belonging to the same individual.Our proposed DEANet is based on a Siamese neural network(SNNet)framework which has two identical branches.A multi-stream architecture is fed into each branch of the SNNet.Both branches of the DEANet that share the same weights extract the features of the patches;then the features are concatenated to obtain the difference of the gaze directions.Once the differential gaze model is trained,a new person’s gaze direction can be estimated when a few calibrated eye patches for that person are provided.Because personspecific calibrated eye patches are involved in the testing stage,the estimation accuracy is improved.Furthermore,the problem of requiring a large amount of data when training a person-specific model is effectively avoided.A reference grid strategy is also proposed in order to select a few references as some of the DEANet’s inputs directly based on the estimation values,further thereby improving the estimation accuracy.Experiments on public datasets show that our proposed approach outperforms the state-of-theart methods.
基金Engineering and Physical Sciences Research Council,Grant/Award Numbers:EP/P01366X/1,EP/V026941/1,EP/W001128/1Small Business research initiative,Grant/Award Number:C/2064382。
文摘Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out(POCO)around nuclear facilities each year,resulting in a trend towards robotic deployments to gain an improved understanding during nuclear decommissioning phases.The UK Nuclear Decommissioning Authority supports the view that human-in-the-loop(HITL)robotic deployments are a solution to improve procedures and reduce risks within radiation characterisation of nuclear sites.The authors present a novel implementation of a Cyber-Physical System(CPS)deployed in an analogue nuclear environment,comprised of a multi-robot(MR)team coordinated by a HITL operator through a digital twin interface.The development of the CPS created efficient partnerships across systems including robots,digital systems and human.This was presented as a multi-staged mission within an inspection scenario for the hetero-geneous Symbiotic Multi-Robot Fleet(SMuRF).Symbiotic interactions were achieved across the SMuRF where robots utilised automated collaborative governance to work together,where a single robot would face challenges in full characterisation of radiation.Key contributions include the demonstration of symbiotic autonomy and query-based learning of an autonomous mission supporting scalable autonomy and autonomy as a service.The coordination of the CPS was a success and displayed further challenges and improvements related to future MR fleets.
基金Supported by the National Research Foundation,Singapore under its International Research Centers in Singapore Funding InitiativeInstitute for Media Innovation,Nanyang Technological University(IMI-NTU)。
文摘Background The uncanny valley hypothesis states that users may experience discomfort when inter-acting with almost human-like artificial characters.Advancements in artificial intelligence,robotics,and computer graphics have led to the development of life-like virtual humans and humanoid robots.Revisiting this hypothesis is necessary to check whether they positively or negatively affect the current population,who are highly accustomed to the latest technologies.Methods In this study,we present a unique evaluation of the uncanny valley hypothesis by allowing participants to interact live with four humanoid robots that have varying levels of human-likeness.Each participant completed a survey questionnaire to evaluate the affinity of each robot.Additionally,we used deep learning methods to quantify the participants’emotional states using multimodal cues,including visual,audio,and text cues,by recording the participant-robot interactions.Results Multi-modal analysis and surveys provided interesting results and insights into the uncanny valley hypothesis.
文摘Humanoid robots are designed and built to mimic human form and movement. Ultimately, they are meant to resemble the size and physical abilities of a human in order to function in human-oriented environments and to work autonomously but to pose no physical threat to humans. Here, a humanoid robot that resembles a human in appearance and movement is built using powerful actuators paired with gear trains, joint mechanisms, and motor drivers that are all encased in a package no larger than that of the human physique. In this paper, we propose the construction of a humanoid-applicable anthropomorphic 7-DoF arm complete with an 8-DoF hand. The novel mechanical design of this humanoid ann makes it sufficiently compact to be compatible with currently available narrating-model humanoids, and to be sufficiently powerful and flexible to be functional; the number of degrees of freedom endowed in this robotic arm is sufficient for executing a wide range of tasks, including dexterous hand movements. The developed humanoid arm and hand are capable of sensing and interpreting incoming external force using the motor in each joint current without conventional torque sensors. The humanoid ann adopts an algorithm to avoid obstacles and the dexterous hand is capable of grasping objects. The developed robotic ann is suitable for use in an interactive humanoid robot.
基金This work has been supported by the ELLIIT Network Organization for Information and Communication Technology,Sweden(Project B09)and the Swedish Foundation for Strategic Research SSF(Smart Systems Project RIT15-0097)The first author is also supported by an RExperts Program Grant 2020A1313030098 from the Guangdong Department of Science and Technology,China in addition to a Sichuan Province International Science and Technology Innovation Cooperation Project Grant 2020YFH0160.
文摘In the context of collaborative robotics,distributed situation awareness is essential for supporting collective intelligence in teams of robots and human agents where it can be used for both individual and collective decision support.This is particularly important in applications pertaining to emergency rescue and crisis management.During operational missions,data and knowledge are gathered incrementally and in different ways by heterogeneous robots and humans.We describe this as the creation of Hastily Formed Knowledge Networks(HFKNs).The focus of this paper is the specification and prototyping of a general distributed system architecture that supports the creation of HFKNs by teams of robots and humans.The information collected ranges from low-level sensor data to high-level semantic knowledge,the latter represented in part as RDF Graphs.The framework includes a synchronization protocol and associated algorithms that allow for the automatic distribution and sharing of data and knowledge between agents.This is done through the distributed synchronization of RDF Graphs shared between agents.High-level semantic queries specified in SPARQL can be used by robots and humans alike to acquire both knowledge and data content from team members.The system is empirically validated and complexity results of the proposed algorithms are provided.Additionally,a field robotics case study is described,where a 3D mapping mission has been executed using several UAVs in a collaborative emergency rescue scenario while using the full HFKN Framework.
基金the National Natural Science Foundation of China(Grant Nos.91848104,91748201,and 52105004)。
文摘As a wearable and intelligent system, a lower limb exoskeleton rehabilitation robot can provide auxiliary rehabilitation training for patients with lower limb walking impairment/loss and address the existing problem of insufficient medical resources. One of the main elements of such a human–robot coupling system is a control system to ensure human–robot coordination. This review aims to summarise the development of human–robot coordination control and the associated research achievements and provide insight into the research challenges in promoting innovative design in such control systems. The patients’ functional disorders and clinical rehabilitation needs regarding lower limbs are analysed in detail, forming the basis for the human–robot coordination of lower limb rehabilitation robots. Then, human–robot coordination is discussed in terms of three aspects: modelling, perception and control. Based on the reviewed research, the demand for robotic rehabilitation, modelling for human–robot coupling systems with new structures and assessment methods with different etiologies based on multi-mode sensors are discussed in detail, suggesting development directions of human–robot coordination and providing a reference for relevant research.
文摘Shared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance.However,prior work that seeks to find the optimal shared control ratio needs an accurate human model,which is usually challenging to obtain.In this study,the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient(DDPG)(TD3X)-based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally.The robot's states,shared control ratio in the previous time step,and human's control input is used as inputs to the reinforcement learning(RL)agent,which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model.Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio.Furthermore,to accelerate the training process and protect the robot,a navigation demonstration policy and a safety guard are developed.A neural network(NN)structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed.In addition,an extended DAGGER(DAGGERX)human agent is developed for training the RL agent to reduce human workload.Robot simulations and experiments with humans in the loop are conducted.The results show that the DAGGERX human agent can simulate real human inputs in the worst-case scenarios with a mean square error of 0.0039.Compared to the original TD3 agent,the TD3X-based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment.The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment,while the performance is equally good in the complex environment because of the use of an advanced human agent.In the human subject tests,participants'average perceived workload was significantly lower in shared control than that in exclusively manual control(26.90 vs.40.07,p=0.013).