Computer vision-based six layered ConvNeural network to recognize sign language for both numeral and alphabet signs

People who have trouble communicating verbally are often dependent on sign language,which can be difficult for most people to understand,making interaction with them a difficult endeavor.The Sign Language Recognition(SLR)system takes an input expression from a hearing or speaking-impaired person and outputs it in the form of text or voice to a normal person.The existing study related to the Sign Language Recognition system has some drawbacks,such as a lack of large datasets and datasets with a range of backgrounds,skin tones,and ages.This research efficiently focuses on Sign Language Recognition to overcome previous limitations.Most importantly,we use our proposed Convolutional Neural Network(CNN)model,“ConvNeural”,in order to train our dataset.Additionally,we develop our own datasets,“BdSL_OPSA22_STATIC1”and“BdSL_OPSA22_STATIC2”,both of which have ambiguous backgrounds.“BdSL_OPSA22_STATIC1”and“BdSL_OPSA22_STATIC2”both include images of Bangla characters and numerals,a total of 24,615 and 8437 images,respectively.The“ConvNeural”model outperforms the pre-trained models with accuracy of 98.38%for“BdSL_OPSA22_STATIC1”and 92.78%for“BdSL_OPSA22_STATIC2”.For“BdSL_OPSA22_STATIC1”dataset,we get precision,recall,F1-score,sensitivity and specificity of 96%,95%,95%,99.31%,and 95.78%respectively.Moreover,in case of“BdSL_OPSA22_STATIC2”dataset,we achieve precision,recall,F1-score,sensitivity and specificity of 90%,88%,88%,100%,and 100%respectively.

Enhancing undulation of soft robots in granular media:A numerical and experimental study on the effect of anisotropic scales

Generating efficient locomotion in granular media is important,although it is difficult for robots.Inspired by the fact that sand vipers usually have saw-like scales,in this study,we design a soft undulation robot with tangential anisotropic friction to enhance the undulation performance of soft robots in granular media.A mathematical model was derived and numerical simulations were conducted accordingly to investigate the effectiveness of tangential friction anisotropy for undulation gait generation in granular media.In particular,we introduce a pseudo-rigid-body dynamics model consisting of links and joints while simulating the pneumatic actuation method to more closely approximate the response of soft robots.Moreover,a soft snake-like robot was fabricated,and its forward and reverse undulations were compared in two sets of controlled experiments.The consistency between the experimental results and the numerical simulations confirms that tangential anisotropic friction induces a propulsive effect in undulation,thereby increasing the robot's locomotion speed.This discovery provides new insights into the design of undulation robots in granular environments.2024 The Author(s).Published by Elsevier B.V.on behalf of Shandong University.This is an open access articleunder the CCBY license(http://creativecommons.org/licenses/by/4.0/).

Controlling a peristaltic robot inspired by inchworms

This study presents an innovative approach in soft robotics,focusing on an inchworm-inspired robot designed for enhanced transport capabilities.We explore the impact of various parameters on the robot’s performance,including the number of activated sections,object size and material,supplied air pressure,and command execution rate.Through a series of controlled experiments,we demonstrate that the robot can achieve a maximum transportation speed of 8.54 mm/s and handle loads exceeding 100 g,significantly outperforming existing models in both speed and load capacity.Our findings provide valuable insights into the optimization of soft robotic design for improved efficiency and adaptability in transport tasks.This research not only contributes to the advancement of soft robotics but also opens new avenues for practical applications in areas requiring precise and efficient object manipulation.The study underscores the potential of biomimetic designs in robotics and sets a new benchmark for future developments in the field.

Defect detection and repair algorithm for structures generated by topology optimization based on 3D hierarchical fully convolutional network

Customized 3D-printed structural parts are widely used in surgical robotics.To satisfy the mechanical properties and kinematic functions of these structural parts,a topology optimization technique is adopted to obtain the optimal structural layout while meeting the constraints and objectives.However,topology optimization currently faces some practical challenges that must be addressed,such as ensuring that structures do not have significant defects when converted to additive manufacturing models.To address this problem,we designed a 3D hierarchical fully convolutional network(FCN)to predict the precise position of the defective structures.Based on the prediction results,an effective repair strategy is adopted to repair the defective structure.A series of experiments is conducted to demonstrate the effectiveness of our approach.Compared to the 2D fully convolutional network and the rule-based detection method,our approach can accurately capture most defect structures and achieve 89.88%precision and 95.59%recall.Furthermore,we investigate the impact of different ways to increase the receptive field of our model,as well as the trade-off between different defect-repairing strategies.The results of the experiment demonstrate that the hierarchical structure,which increases the receptive field,can substantially improve the defect detection performance.To the best of our knowledge,this paper is the first to investigate 3D defect prediction and repair for topology optimization in conjunction with deep learning algorithms,providing practical tools and new perspectives for the subsequent development of topology optimization techniques.

Bioinspiration review of Aquatic Unmanned Aerial Vehicle(AquaUAV)

The performance of Aquatic Unmanned Aerial Vehicle(AquaUAV)has always been limited so far and far from practical applications,due to insufficient propulsion,large-resistance structure etc.Aerial-aquatic amphibians in nature may facilitate the development of AquaUAV since their excellent amphibious locomotion capabilities evolved under long-term natural selection.This article will take four typical aerial-aquatic amphibians as representatives,i.e.,gannet,cormorant,flying fish and flying squid.We summarized the multi-mode locomotion process of common aerial-aquatic amphibians and the evolutionary trade-offs they have made to adapt to amphibious environments.The four typical propulsion mechanisms were investigated,which may further inspire the propulsion design of the AquaUAV.And their morphological models could guide the layout optimization.Finally,we reviewed the state of art in AquaUAV to validate the potential value of our bioinspiration,and discussed the futureprospects.

Variable stiffness methods of flexible robots for minimally invasive surgery:A review

With high flexibility and slim body,flexible robots have been widely used in minimally invasive surgery because they can safely reach the lesion deep inside the human body through small incisions or natural orifices.However,high stiffness of robot body is also required for transferring force and maintaining the motion accuracy.To meet these two contradictory requirements,various methods have been implemented to enable adjustable stiffness for flexible surgical robots.In this review,we first summarize the anatomic constraints of common natural tracts of human body to provide a guidance for the design of variable stiffness flexible robots.And then,the variable stiffness methods have been categorized based on their basic principles of varying the stiffness.In the end,two variable stiffness methods with great potential and the moving strategy of variable stiffness flexible robots are discussed.

Continuous adaptive gaits manipulation for three-fingered robotic hands via bioinspired fingertip contact events

The remarkable skill of changing its grasp status and relocating its fingers to perform continuous in-hand manipulation is essential for a multifingered anthropomorphic hand.A commonly utilized method of manipulation involves a series of basic movements executed by a high-level controller.However,it remains unclear how these primitives evolve into sophisticated finger gaits during manipulation.Here,we propose an adaptive finger gait-based manipulation method that offers real-time regulation by dynamically changing the primitive interval to ensure the force/moment balance of the object.Successful manipulation relies on contact events that act as triggers for real-time online replanning of multifinger manipulation.We identify four basic motion primitives of finger gaits and create a heuristic finger gait that enables the continuous object rotation of a round cup.Our experimental results verify the effectiveness of the proposed method.Despite the constant breaking and reengaging of contact between the fingers and the object during manipulation,the robotic hand can reliably manipulate the object without failure.Even when the object is subjected to interfering forces,the proposed method demonstrates robustness in managing interference.This work has great potential for application to the dexterous operation of anthropomorphic multifingered hands.

Adaptive patient-cooperative compliant control of lower limb rehabilitation robot

With the increase in the number of stroke patients,there is a growing demand for rehabilitation training.Robot-assisted training is expected to play a crucial role in meeting this demand.To ensure the safety and comfort of patients during rehabilitation training,it is important to have a patient-cooperative compliant control system for rehabilitation robots.In order to enhance the motion compliance of patients during rehabilitation training,a hierarchical adaptive patient-cooperative compliant control strategy that includes patient-passive exercise and patient-cooperative exercise is proposed.A low-level adaptive backstepping position controller is selected to ensure accurate tracking of the desired trajectory.At the high-level,an adaptive admittance controller is employed to plan the desired trajectory based on the interaction force between the patient and the robot.The results of the patient-robot cooperation experiment on a rehabilitation robot show a significant improvement in tracking trajectory,with a decrease of 76.45%in the dimensionless squared jerk(DSJ)and a decrease of 15.38%in the normalized root mean square deviation(NRMSD)when using the adaptive admittance controller.The proposed adaptive patient-cooperative control strategy effectively enhances the compliance of robot movements,thereby ensuring the safety and comfort of patients during rehabilitation training.

LiDAR-based estimation of bounding box coordinates using Gaussian process regression and particle swarm optimization

Camera-based object tracking systems in a given closed environment lack privacy and confidentiality.In this study,light detection and ranging(LiDAR)was applied to track objects similar to the camera tracking in a closed environment,guaranteeing privacy and confidentiality.The primary objective was to demonstrate the efficacy of the proposed technique through carefully designed experiments conducted using two scenarios.In Scenario I,the study illustrates the capability of the proposed technique to detect the locations of multiple objects positioned on a flat surface,achieved by analyzing LiDAR data collected from several locations within the closed environment.Scenario II demonstrates the effectiveness of the proposed technique in detecting multiple objects using LiDAR data obtained from a single,fixed location.Real-time experiments are conducted with human subjects navigating predefined paths.Three individuals move within an environment,while LiDAR,fixed at the center,dynamically tracks and identifies their locations at multiple instances.Results demonstrate that a single,strategically positioned LiDAR can adeptly detect objects in motion around it.Furthermore,this study provides a comparison of various regression techniques for predicting bounding box coordinates.Gaussian process regression(GPR),combined with particle swarm optimization(PSO)for prediction,achieves the lowest prediction mean square error of all the regression techniques examined at 0.01.Hyperparameter tuning of GPR using PSO significantly minimizes the regression error.Results of the experiment pave the way for its extension to various real-time applications such as crowd management in malls,surveillance systems,and various Internet of Things scenarios.

Multi-gait snake robot for inspecting inner wall of a pipeline

In the field of pipeline inner wall inspection,the snake robot demonstrates significant advantages over other inspection methods.While a simple traveling wave or meandering motion will suffice for inspecting the inner wall of small-diameter pipes,comprehensively and meticulously inspecting the inner wall of large-diameter pipes requires the snake robot to adopt a helical gait that closely adheres to the inner wall.Our review of existing literature indicates that most research and development on the helical gait of snake robots has focused on the outer surface of cylinders,with very few studies dedicated to developing a helical gait specifically for the inspection of the inner wall of pipes.Therefore,in this study,we propose a helical gait that is suitable for the inner wall of pipes and meets the requirements of gas pipeline engineering.The helical gait is designed using the backbone curve method.First,we create a mathematical model for a circular helix curve with constant curvature and torsion,ensuring it is applicable to a snake robot prototype in a laboratory environment.Subsequently.we calculate the joint angles required for two conical spiral curves with variable curvature and torsion,establish a new model,and define the physical significance of the specific parameters.To ensure the feasibility of the proposed gait,we conduct experiments involving meandering and traveling wave motions to verify the communication and control between the host computer and the snake robot.Building upon this foundation,we further validate the mathematical model of the complex helical motion gait through simulation experiments.Our findings provide a theoretical basis for realizing helical movement with a real snake robot.

Image format pipeline and instrument diagram recognition method based on deep learning

In this study,we proposed a recognition method based on deep artificial neural networks to identify various elements in pipelines and instrumentation diagrams(P&ID)in image formats,such as symbols,texts,and pipelines.Presently,the P&ID image format is recognized manually,and there is a problem with a high recognition error rate;therefore,automation of the above process is an important issue in the processing plant industry.The China National Offshore Petrochemical Engineering Co.provided the image set used in this study,which contains 51 P&ID drawings in the PDF.We converted the PDF P&ID drawings to PNG P&IDs with an image size of 8410×5940.In addition,we used labeling software to annotate the images,divided the dataset into training and test sets in a 3:1 ratio,and deployed a deep neural network for recognition.The method proposed in this study is divided into three steps.The first step segments the images and recognizes symbols using YOLOv5+SE.The second step determines text regions using character region awareness for text detection,and performs character recognition within the text region using the optical character recognition technique.The third step is pipeline recognition using YOLOv5+SE.The symbol recognition accuracy was 94.52%,and the recall rate was 93.27%.The recognition accuracy in the text positioning stage was 97.26%and the recall rate was 90.27%.The recognition accuracy in the character recognition stage was 90.03%and the recall rate was 91.87%.The pipeline identification accuracy was 92.9%,and the recall rate was 90.36%.

Computer-controlled ultra high voltage amplifier for dielectric elastomer actuators

Soft robotics is a breakthrough technology to support human-robot interactions.The soft structure of a soft robot can increase safety during human and robot interactions.One of the promising soft actuators for soft robotics is dielectric elastomer actuators(DEAs).DEAs can operate silently and have an excellent energy density.The simple structure of DEAs leads to the easy fabrication of soft actuators.The simplicity combined with silent operation and high energy density make DEAs interesting for soft robotics researchers.DEAs actuation follows the Maxwell-pressure principle.The pressure produced in the DEAs actuation depends much on the voltage applied.Common DEAs requires high voltage to gain an actuation.Since the power consumption of DEAs is in the milli-Watt range,the current needed to operate the DEAs can be neglected.Several commercially available DC-DC converters can convert the volt range to the kV range.In order to get a voltage in the 2-3 kV range,the reliable DC-DC converter can be pricy for each device.This problem hinders the education of soft actuators,especially for a newcomer laboratory that works in soft electric actuators.This paper introduces an entirely do-it-yourself(DIY)Ultrahigh voltage amplifier(UHV-Amp)for education in soft robotics.UHV-Amp can amplify 12 V to at a maximum of 4 kV DC.As a demonstration,we used this UHV-Amp to test a single layer of powdered-based DEAs.The strategy to build this educational type UHV-Amp was utilizing a Cockcroft-Walton circuit structure to amplify the voltage range to the kV range.In its current state,the UHV-Amp has the potential to achieve approximately 4 kV.We created a simple platform to control the UHV-Amp from a personal computer.In near future,we expect this easy control of the UHV-Amp can contribute to the education of soft electric actuators.

Locomotion control of a rigid-soft coupled snake robot in multiple environments

The versatile motion capability of snake robots offers themselves robust adaptability in varieties of challenging environments where traditional robots may be incapacitated.This study reports a novel flexible snake robot featuring a rigid-flexible coupling structure and multiple motion gaits.To better understand the robot's behavior,a bending model for the soft actuator is established.Furthermore,a dynamic model is developed to map the relationship between the input air pressure and joint torque,which is the model base for controlling the robot effectively.Based on the wave motion generated by the joint coupling direction function in different planes,multiple motion gait planning methods of the snake-like robot are proposed.In order to evaluate the adaptability and maneuverability of the developed snake robot,extensive experiments were conducted in complex environments.The results demonstrate the robot's effectiveness in navigating through intricate settings,underscoring its potential for applications in various fields.

Flexible head-following motion planning for scalable and bendable continuum robots

Continuum robots,which are characterized by high length-to-diameter ratios and flexible structures,show great potential for various applications in confined and irregular environments.Due to the combination of motion modes,the existence of multiple solutions,and the presence of complex obstacle constraints,motion planning for these robots is highly challenging.To tackle the challenges of online and flexible operation for continuum robots,we propose a flexible head-following motion planning method that is suitable for scalable and bendable continuum robots.Firstly,we establish a piecewise constant curvature(PCC)kinematic model for scalable and bendable continuum robots.The article proposes an adaptive auxiliary points model and a method for updating key nodes in head-following motion to enhance the precise tracking capability for paths with different curvatures.Additionally,the article integrates the strategy for adjusting the posture of local joints of the robot into the head-following motion planning method,which is beneficial for achieving safe obstacle avoidance in local areas.The article concludes by presenting the results of multiple sets of motion simulation experiments and prototype experiments.The study demonstrates that the algorithm presented in this paper effectively navigates and adjusts posture to avoid obstacles,meeting the real-time demands of online operations.The average time for a single-step solution is 4.41×10^(-5) s,and the average tracking accuracy forcircular paths is 7.8928mm.

Deep learning-based semantic segmentation of human features in bath scrubbing robots

With the rise in the aging population,an increase in the number of semidisabled elderly individuals has been noted,leading to notable challenges in medical and healthcare,exacerbated by a shortage of nursing staff.This study aims to enhance the human feature recognition capabilities of bath scrubbing robots operating in a water fog environment.The investigation focuses on semantic segmentation of human features using deep learning methodologies.Initially,3D point cloud data of human bodies with varying sizes are gathered through light detection and ranging to establish human models.Subsequently,a hybrid filtering algorithm was employed to address the impact of the water fog environment on the modeling and extraction of human regions.Finally,the network is refined by integrating the spatial feature extraction module and the channel attention module based on PointNet.The results indicate that the algorithm adeptly identifies feature information for 3D human models of diverse body sizes,achieving an overall accuracy of 95.7%.This represents a 4.5%improvement compared with the PointNet network and a 2.5%enhancement over mean intersection over union.In conclusion,this study substantially augments the human feature segmentation capabilities,facilitating effective collaboration with bath scrubbing robots for caregiving tasks,thereby possessing significant engineering application value.

Swimmer with submerged SiO_(2)/Al/LiNbO_(3) surface acoustic wave propulsion system

Acoustic propulsion system presents a novel underwater propulsion approach in small scale swimmer.This study introduces a submerged surface acoustic wave(SAW)propulsion system based on the SiO_(2)/Al/LiNbO_(3) structure.At 19.25 MHz,the SAW propulsion system is proposed and investigated by the propulsion force calculation,PIV measurements and propulsion measurements.3.3 mN propulsion force is measured at 27.6 V_(pp).To evaluate the miniature swimmer,the SAW propulsion systems with multiple frequencies are studied.At 2.2 W,the submerged SAW propulsion system at 38.45 MHz demonstrates 0.83 mN/mm^(2) propulsion characteristics.At 96.13 MHz and 24 V_(pp),the movements of miniature swimmer with a fully submerged SAW propulsion system are recorded and analyzed to a maximum of 177 mm/s.Because of miniaturization,high power density,and simple structure,the SAW propulsion system can be expected for some microrobot applications,such as underwater drone,pipelinerobotand intravascularrobot.

NP-MBO:A newton predictor-based momentum observer for interaction force estimation of legged robots

Swift perception of interaction forces is a crucial skill required for legged robots to ensure safe human-robot interaction and dynamic contact management.Proprioceptive-based interactive force is widely applied due to its outstanding cross-platform versatility.In this paper,we present a novel interactive force observer,which possesses superior dynamic tracking performance.We propose a dynamic cutoff frequency configuration method to replace the conventional fixed cutoff frequency setting in the traditional momentum-based observer(MBO).This method achieves a balance between rapid tracking and noise suppression.Moreover,to mitigate the phase lag introduced by the low-pass filtering,we cascaded a Newton Predictor(NP)after MBO,which features simple computation and adaptability.The precision analysis of this method has been presented.We conducted extensive experiments on the point-foot biped robot BRAVER to validate the performance of the proposed algorithm in both simulation and physical prototype.

Structure design and coordinated motion analysis of bionic crocodile robot

Crocodiles,one of the oldest and most resilient species on Earth,have demonstrated remarkable locomotor abilities both on land and in water,evolving over millennia to adapt to diverse environ-ments.In this study,we draw inspiration from crocodiles and design a highly biomimetic crocodile robot equipped with multiple degrees of freedom and articulated trunk joints.This design is based on comprehensive analysis of the structural and motion characteristics of real crocodiles.The bionic crocodile robot has a problem of limb-torso incoordination during movement.To solve this problem,we used the D-H method for both forward and inverse kinematics analysis of the robot's legs and spine.Through a series of simulation experiments,we investigated the robot's motion stability,fault tolerance,and adaptability to environments in two motor patterns:with and without spine and tail movements.The experimental results show that the bionic crocodile robot exhibits superior motion performance when the spine and tail cooperate with the extremities.This study not only demonstrates the potential of biomimicry in robotics but also underscores the significance of understanding how nature's designs can inform and enhance technological innovations.

A new active rehabilitation training mode for upper limbs based on Tai Chi Pushing Hands

Robot-assisted rehabilitation is a crucial approach to restoring motor function in the limb.However,the current training trajectory lacks sufficient theoretical or practical support,and the monotony of single-mode training is a concern.Tai Chi Pushing Hands,a beneficial and effective daily exercise,has been shown to improve balance function,psychological state,and motor function of the upper extremities in patients recovering from stroke.To address these issues,we propose a new active rehabilitation training that incorporates Tai Chi Pushing Hands movements and yin-yang balance principles.The training trajectory and direction are encoded by the velocity field and consist of two processes:yang(push)and yin(return).During yang,the limb actively pushes the robot to move,while during yin,the limb actively follows the robot’s movement.To provide necessary assistance,an admittance controller with self-adaptive parameters is designed.In addition,we introduce two indexes,the‘Intention Angle’(ϖ)and the time ratio(Γ),to evaluate motion perception performance.Our experiment was conducted on a 4-degree-of-freedom upper limb rehabilitation robot platform,and the subjects were separated into a familiar group and an unfamiliar group.The experiment results show that the training could be completed well no matter whether the subject is familiar with Tai Chi Pushing Hands or not.The parameters and the movement of the robot can be adjusted based on the interactive force to adapt to the ability of the subject.

TWrist:An agile compliant 3-DoF tensegrity joint

Tensegrity structures,with their unique physical characteristics,hold substantial potential in the field of robotics.However,the very structures that will give tensegrity robots potential advantages over traditional robots also hold long term challenges.Due to the inherent high redundancy of tensegrity structures and the employment of tension elements,tensegrity robots exhibit excellent stability,compliance,and flexibility,although this also results in lower structural deformation efficiency.Existing research has endeavoured to enhance the motion performance of tensegrity robots,exploring diverse approaches such as actuation schemes,structure design,aligned with control algorithms.However,the physical constraints of the elements in such structures and the absence of suitable controllers impede further advancements in the usefulness of tensegrity robots.This paper presents a novel design based on an under constrained transition region design and a tailored control approach based on inverse kinematics,improving the motion performance of the proposed novel tensegrity joint.Through this approach,the tensegrity joint,while preserving the advantages of compliance and flexibility expected from tensegrity structures,offers three degrees of rotational freedom,mirroring the controllability of conventional rigid-body joints.The results demonstrate the capability of tensegritybased robotic joints to provide flexible actuation under situations demanding high compliance.The integration of structure design with a tailored control approach offers a pioneering model for future development of tensegrity robots,underscoring the practical viability of tensegrity structures in the realm of robotics.