A Multi-Task Deep Learning Framework for Simultaneous Detection of Thoracic Pathology through Image Classification

Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.

Deep reinforcement learning based multi-level dynamic reconfiguration for urban distribution network:a cloud-edge collaboration architecture

With the construction of the power Internet of Things(IoT),communication between smart devices in urban distribution networks has been gradually moving towards high speed,high compatibility,and low latency,which provides reliable support for reconfiguration optimization in urban distribution networks.Thus,this study proposed a deep reinforcement learning based multi-level dynamic reconfiguration method for urban distribution networks in a cloud-edge collaboration architecture to obtain a real-time optimal multi-level dynamic reconfiguration solution.First,the multi-level dynamic reconfiguration method was discussed,which included feeder-,transformer-,and substation-levels.Subsequently,the multi-agent system was combined with the cloud-edge collaboration architecture to build a deep reinforcement learning model for multi-level dynamic reconfiguration in an urban distribution network.The cloud-edge collaboration architecture can effectively support the multi-agent system to conduct“centralized training and decentralized execution”operation modes and improve the learning efficiency of the model.Thereafter,for a multi-agent system,this study adopted a combination of offline and online learning to endow the model with the ability to realize automatic optimization and updation of the strategy.In the offline learning phase,a Q-learning-based multi-agent conservative Q-learning(MACQL)algorithm was proposed to stabilize the learning results and reduce the risk of the next online learning phase.In the online learning phase,a multi-agent deep deterministic policy gradient(MADDPG)algorithm based on policy gradients was proposed to explore the action space and update the experience pool.Finally,the effectiveness of the proposed method was verified through a simulation analysis of a real-world 445-node system.

Scale adaptive fitness evaluation‐based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning

Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.

A Novel IoT Architecture, Assessment of Threats and Their Classification withMachine Learning Solutions

The Internet of Things(IoT)will significantly impact our social and economic lives in the near future.Many Internet of Things(IoT)applications aim to automate multiple tasks so inactive physical objects can behave independently of others.IoT devices,however,are also vulnerable,mostly because they lack the essential built-in security to thwart attackers.It is essential to perform the necessary adjustments in the structure of the IoT systems in order to create an end-to-end secure IoT environment.As a result,the IoT designs that are now in use do not completely support all of the advancements that have been made to include sophisticated features in IoT,such as Cloud computing,machine learning techniques,and lightweight encryption techniques.This paper presents a detailed analysis of the security requirements,attack surfaces,and security solutions available for IoT networks and suggests an innovative IoT architecture.The Seven-Layer Architecture in IoT provides decent attack detection accuracy.According to the level of risk they pose,the security threats in each of these layers have been properly categorized,and the essential evaluation criteria have been developed to evaluate the various threats.Also,Machine Learning algorithms like Random Forest and Support Vector Machines,etc.,and Deep Learning algorithms like Artificial Neural Networks,Q Learning models,etc.,are implemented to overcome the most damaging threats posing security breaches to the different IoT architecture layers.

Machine learning with active pharmaceutical ingredient/polymer interaction mechanism:Prediction for complex phase behaviors of pharmaceuticals and formulations

The high throughput prediction of the thermodynamic phase behavior of active pharmaceutical ingredients(APIs)with pharmaceutically relevant excipients remains a major scientific challenge in the screening of pharmaceutical formulations.In this work,a developed machine-learning model efficiently predicts the solubility of APIs in polymers by learning the phase equilibrium principle and using a few molecular descriptors.Under the few-shot learning framework,thermodynamic theory(perturbed-chain statistical associating fluid theory)was used for data augmentation,and computational chemistry was applied for molecular descriptors'screening.The results showed that the developed machine-learning model can predict the API-polymer phase diagram accurately,broaden the solubility data of APIs in polymers,and reproduce the relationship between API solubility and the interaction mechanisms between API and polymer successfully,which provided efficient guidance for the development of pharmaceutical formulations.

Machine Learning Security Defense Algorithms Based on Metadata Correlation Features

With the popularization of the Internet and the development of technology,cyber threats are increasing day by day.Threats such as malware,hacking,and data breaches have had a serious impact on cybersecurity.The network security environment in the era of big data presents the characteristics of large amounts of data,high diversity,and high real-time requirements.Traditional security defense methods and tools have been unable to cope with the complex and changing network security threats.This paper proposes a machine-learning security defense algorithm based on metadata association features.Emphasize control over unauthorized users through privacy,integrity,and availability.The user model is established and the mapping between the user model and the metadata of the data source is generated.By analyzing the user model and its corresponding mapping relationship,the query of the user model can be decomposed into the query of various heterogeneous data sources,and the integration of heterogeneous data sources based on the metadata association characteristics can be realized.Define and classify customer information,automatically identify and perceive sensitive data,build a behavior audit and analysis platform,analyze user behavior trajectories,and complete the construction of a machine learning customer information security defense system.The experimental results show that when the data volume is 5×103 bit,the data storage integrity of the proposed method is 92%.The data accuracy is 98%,and the success rate of data intrusion is only 2.6%.It can be concluded that the data storage method in this paper is safe,the data accuracy is always at a high level,and the data disaster recovery performance is good.This method can effectively resist data intrusion and has high air traffic control security.It can not only detect all viruses in user data storage,but also realize integrated virus processing,and further optimize the security defense effect of user big data.

Quafu-RL:The cloud quantum computers based quantum reinforcement learning

With the rapid advancement of quantum computing,hybrid quantum–classical machine learning has shown numerous potential applications at the current stage,with expectations of being achievable in the noisy intermediate-scale quantum(NISQ)era.Quantum reinforcement learning,as an indispensable study,has recently demonstrated its ability to solve standard benchmark environments with formally provable theoretical advantages over classical counterparts.However,despite the progress of quantum processors and the emergence of quantum computing clouds,implementing quantum reinforcement learning algorithms utilizing parameterized quantum circuits(PQCs)on NISQ devices remains infrequent.In this work,we take the first step towards executing benchmark quantum reinforcement problems on real devices equipped with at most 136 qubits on the BAQIS Quafu quantum computing cloud.The experimental results demonstrate that the policy agents can successfully accomplish objectives under modified conditions in both the training and inference phases.Moreover,we design hardware-efficient PQC architectures in the quantum model using a multi-objective evolutionary algorithm and develop a learning algorithm that is adaptable to quantum devices.We hope that the Quafu-RL can be a guiding example to show how to realize machine learning tasks by taking advantage of quantum computers on the quantum cloud platform.

An Interpretable Light Attention-Convolution-Gate Recurrent Unit Architecture for the Highly Accurate Modeling of Actual Chemical Dynamic Processes

To equip data-driven dynamic chemical process models with strong interpretability,we develop a light attention–convolution–gate recurrent unit(LACG)architecture with three sub-modules—a basic module,a brand-new light attention module,and a residue module—that are specially designed to learn the general dynamic behavior,transient disturbances,and other input factors of chemical processes,respectively.Combined with a hyperparameter optimization framework,Optuna,the effectiveness of the proposed LACG is tested by distributed control system data-driven modeling experiments on the discharge flowrate of an actual deethanization process.The LACG model provides significant advantages in prediction accuracy and model generalization compared with other models,including the feedforward neural network,convolution neural network,long short-term memory(LSTM),and attention-LSTM.Moreover,compared with the simulation results of a deethanization model built using Aspen Plus Dynamics V12.1,the LACG parameters are demonstrated to be interpretable,and more details on the variable interactions can be observed from the model parameters in comparison with the traditional interpretable model attention-LSTM.This contribution enriches interpretable machine learning knowledge and provides a reliable method with high accuracy for actual chemical process modeling,paving a route to intelligent manufacturing.

Vision-based multi-level synthetical evaluation of seismic damage for RC structural components: a multi-task learning approach

Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.

Convective Storm VIL and Lightning Nowcasting Using Satellite and Weather Radar Measurements Based on Multi-Task Learning Models

Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.However,most existing works on Arabic ABSA content separately address them,assume that aspect terms are preidentified,or use a pipeline model.Pipeline solutions design different models for each task,and the output from the ATE model is used as the input to the APC model,which may result in error propagation among different steps because APC is affected by ATE error.These methods are impractical for real-world scenarios where the ATE task is the base task for APC,and its result impacts the accuracy of APC.Thus,in this study,we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a singlemodel.This paper integrates themulti-task model,namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification(LCF-ATEPC)and Arabic Bidirectional Encoder Representation from Transformers(AraBERT)as a shred layer for Arabic contextual text representation.The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism(LCF)to capture the interactive information between an aspect and its context.Moreover,data augmentation techniques are proposed based on state-of-the-art augmentation techniques(word embedding substitution with constraints and contextual embedding(AraBERT))to increase the diversity of the training dataset.This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA.Extensive experiments were conducted on the original and combined datasets(merging the original and augmented datasets).Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques.Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer(AR-LCF-ATEPC-Fusion)and the proposed data augmentation word embedding-based method(FastText)on the combined dataset.

Multi-Task Deep Learning with Task Attention for Post-Click Conversion Rate Prediction

Online advertising has gained much attention on various platforms as a hugely lucrative market.In promoting content and advertisements in real life,the acquisition of user target actions is usually a multi-step process,such as impres-sion→click→conversion,which means the process from the delivery of the recommended item to the user’s click to the final conversion.Due to data sparsity or sample selection bias,it is difficult for the trained model to achieve the business goal of the target campaign.Multi-task learning,a classical solution to this pro-blem,aims to generalize better on the original task given several related tasks by exploiting the knowledge between tasks to share the same feature and label space.Adaptively learned task relations bring better performance to make full use of the correlation between tasks.We train a general model capable of captur-ing the relationships between various tasks on all existing active tasks from a meta-learning perspective.In addition,this paper proposes a Multi-task Attention Network(MAN)to identify commonalities and differences between tasks in the feature space.The model performance is improved by explicitly learning the stacking of task relationships in the label space.To illustrate the effectiveness of our method,experiments are conducted on Alibaba Click and Conversion Pre-diction(Ali-CCP)dataset.Experimental results show that the method outperforms the state-of-the-art multi-task learning methods.

A General Linguistic Steganalysis Framework Using Multi-Task Learning

Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.

A Requirement Driven Learning Management Architecture Based on BPEL

With the prosperity of the Intemet, e-learning has been greatly improved. By supporting multiple learners and multiple roles in a learning activity, the IMS Leaming Design （LD） specification provides a collaborative scenario for participants. However, IMS LD provides insufficient support for interaction among learning activities and can not dynamically integrate learning resources to meet the continually changing service requirements. In this paper, a Business Process Execution Language （BPEL） enhanced requirement driven learning management architecture to address the issues of personalize adaptive learning was proposed. It models the learning activity by combining IMS LD with BPEL and matches optimal learning sequence based on Case-based reasoning （CBR） method. By providing expandable secure learning sequences flexibly, it satisfies the different actual demands for personalize learning

Hybrid Deep Learning Architecture to Forecast Maximum Load Duration Using Time-of-Use Pricing Plans

Load forecasting has received crucial research attention to reduce peak load and contribute to the stability of power grid using machine learning or deep learning models.Especially,we need the adequate model to forecast the maximum load duration based on time-of-use,which is the electricity usage fare policy in order to achieve the goals such as peak load reduction in a power grid.However,the existing single machine learning or deep learning forecasting cannot easily avoid overfitting.Moreover,a majority of the ensemble or hybrid models do not achieve optimal results for forecasting the maximum load duration based on time-of-use.To overcome these limitations,we propose a hybrid deep learning architecture to forecast maximum load duration based on time-of-use.Experimental results indicate that this architecture could achieve the highest average of recall and accuracy(83.43%)compared to benchmark models.To verify the effectiveness of the architecture,another experimental result shows that energy storage system(ESS)scheme in accordance with the forecast results of the proposed model(LSTM-MATO)in the architecture could provide peak load cost savings of 17,535,700 KRW each year comparing with original peak load costs without the method.Therefore,the proposed architecture could be utilized for practical applications such as peak load reduction in the grid.

Multi-tasking to Address Diversity in Language Learning

With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately in varied contexts and with different uses of the language. To attain this, the teacher is tasked with designing, monitoring and processing language learning activities for students to carry out and in the process learn by doing and reflecting on the learning process they went through as they interacted socially with each other. This paper describes a task named"The Fishbowl Technique"and found to be effective in large ESL classes in the secondary level in the Philippines.

Modelling Mobile-X Architecture for Offloading in Mobile Edge Computing

Mobile Edge Computing(MEC)assists clouds to handle enormous tasks from mobile devices in close proximity.The edge servers are not allocated efficiently according to the dynamic nature of the network.It leads to processing delay,and the tasks are dropped due to time limitations.The researchersfind it difficult and complex to determine the offloading decision because of uncertain load dynamic condition over the edge nodes.The challenge relies on the offload-ing decision on selection of edge nodes for offloading in a centralized manner.This study focuses on minimizing task-processing time while simultaneously increasing the success rate of service provided by edge servers.Initially,a task-offloading problem needs to be formulated based on the communication and pro-cessing.Then offloading decision problem is solved by deep analysis on taskflow in the network and feedback from the devices on edge services.The significance of the model is improved with the modelling of Deep Mobile-X architecture and bi-directional Long Short Term Memory(b-LSTM).The simulation is done in the Edgecloudsim environment,and the outcomes show the significance of the proposed idea.The processing time of the anticipated model is 6.6 s.The following perfor-mance metrics,improved server utilization,the ratio of the dropped task,and number of offloading tasks are evaluated and compared with existing learning approaches.The proposed model shows a better trade-off compared to existing approaches.

Nuclear mass based on the multi-task learning neural network method

The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 nuclei(Z ≥ 8, N ≥ 8) released in the latest Atomic Mass Evaluation AME2020 and the deviations between the fitting result of the liquid drop model(LDM)and data from AME2020 for each nucleus were obtained.To compensate for the deviations and investigate the possible ignored physics in the LDM, the MTL-ANN method was introduced in the model. Compared to the single-task learning(STL) method, this new network has a powerful ability to simultaneously learn multi-nuclear properties,such as the binding energies and single neutron and proton separation energies. Moreover, it is highly effective in reducing the risk of overfitting and achieving better predictions. Consequently, good predictions can be obtained using this nuclear mass model for both the training and validation datasets and for the testing dataset. In detail, the global root mean square(RMS) of the binding energy is effectively reduced from approximately 2.4 MeV of LDM to the current 0.2 MeV, and the RMS of Sn, Spcan also reach approximately 0.2 MeV. Moreover, compared to STL, for the training and validation sets, 3-9% improvement can be achieved with the binding energy, and 20-30% improvement for S_(n), S_(p);for the testing sets, the reduction in deviations can even reach 30-40%, which significantly illustrates the advantage of the current MTL.

EM-Gaze:eye context correlation and metric learning for gaze estimation

In recent years,deep learning techniques have been used to estimate gaze-a significant task in computer vision and human-computer interaction.Previous studies have made significant achievements in predicting 2D or 3D gazes from monocular face images.This study presents a deep neural network for 2D gaze estimation on mobile devices.It achieves state-of-the-art 2D gaze point regression error,while significantly improving gaze classification error on quadrant divisions of the display.To this end,an efficient attention-based module that correlates and fuses the left and right eye contextual features is first proposed to improve gaze point regression performance.Subsequently,through a unified perspective for gaze estimation,metric learning for gaze classification on quadrant divisions is incorporated as additional supervision.Consequently,both gaze point regression and quadrant classification perfor-mances are improved.The experiments demonstrate that the proposed method outperforms existing gaze-estima-tion methods on the GazeCapture and MPIIFaceGaze datasets.