An Empirical Study on Automated Test Generation Tools for Java:Effectiveness and Challenges

Automated test generation tools enable test automation and further alleviate the low efficiency caused by writing hand-crafted test cases.However,existing automated tools are not mature enough to be widely used by software testing groups.This paper conducts an empirical study on the state-of-the-art automated tools for Java,i.e.,EvoSuite,Randoop,JDoop,JTeXpert,T3,and Tardis.We design a test workflow to facilitate the process,which can automatically run tools for test generation,collect data,and evaluate various metrics.Furthermore,we conduct empirical analysis on these six tools and their related techniques from different aspects,i.e.,code coverage,mutation score,test suite size,readability,and real fault detection ability.We discuss about the benefits and drawbacks of hybrid techniques based on experimental results.Besides,we introduce our experience in setting up and executing these tools,and summarize their usability and user-friendliness.Finally,we give some insights into automated tools in terms of test suite readability improvement,meaningful assertion generation,test suite reduction for random testing tools,and symbolic execution integration.

Iterative Android automated testing

With the benefits of reducing time and workforce,automated testing has been widely used for the quality assurance of mobile applications(APPs).Compared with automated testing,manual testing can achieve higher coverage in complex interactive Activities.And the effectiveness of manual testing is highly dependent on the user operation process(UOP)of experienced testers.Based on the UOP,we propose an iterative Android automated testing(IAAT)method that automatically records,extracts,and integrates UOPs to guide the test logic of the tool across the complex Activity iteratively.The feedback test results can train the UOPs to achieve higher coverage in each iteration.We extracted 50 UOPs and conducted experiments on 10 popular mobile APPs to demonstrate IAAT’s effectiveness compared with Monkey and the initial automated tests.The experimental results show a noticeable improvement in the IAAT compared with the test logic without human knowledge.Under the 60 minutes test time,the average code coverage is improved by 13.98%to 37.83%,higher than the 27.48%of Monkey under the same conditions.

Contribution of Automated Antigen Tests, the LumiraDx Ag Test in the Response during the Second Wave of the COVID-19 Pandemic in Bangui

Context and objective: The COVID-19 pandemic has become a major public health problem and has mobilized many innovative means of diagnosis. The Central African Republic is not spared. The emergence of variants and their impact require health monitoring despite the obligation of vaccination. The purpose of this campaign was to determine the circulation of pending second-wave variants. Patients and Methods: A second mass screening campaign took place from 02 to 22 July 2021 in the main land and river entry points of Bangui (Exit North-PK12, Exit South-PK9, Port Beach) and at the LNBCSP. Antigenic and RT-PCR tests carried out on nasopharyngeal samples made it possible to select strains which were finally sequenced. Results: Of 2687 participants included in the study, 53 (1.97%) were positive for SARS-CoV-2. Thirteen (1.53%) were male and 40 (2.18%) female. The analyses carried out on the LumiraDx analyzer were positive for 109 samples against 53 on the RT-PCR. The prevalence was higher in the most tested age groups (30 to 50 years) with two clusters identified. B.1.617.2 (Delta) variants were predominant (57%). Conclusion: SARS-CoV-2 continues to circulate. The acquisition of automated antigenic tests (LumiraDx&#174) with sensitivity and specificity close to those of the reference test (RT-PCR) will allow better mass diagnosis for an optimization of the surveillance of COVID-19 in our countries with limited resources. The predominance of the B.1.617.2 (Delta) variant would suggest a third wave in the Central African Republic.

Automated Testing of Web Applications Using Combinatorial Strategies

Recently, testing techniques based on dynamic exploration, which try to automatically exercise every possible user interface element, have been extensively used to facilitate fully testing web applications. Most of such testing tools are however not effective in reaching dynamic pages induced by form interactions due to their emphasis on handling client-side scripting. In this paper, we present a combinatorial strategy to achieve a full form test and build an automated test model. We propose an algorithm called pairwise testing with constraints （PTC） to iraplement the strategy. Our PTC algorithm uses pairwise coverage and handles the issues of semantic constraints and illegal values. We have implemented a prototype tool ComjaxTest and conducted an empirical study on five web applications. Experimental results indicate that our PTC algorithm generates less form test cases while achieving a higher coverage of dynamic pages than the general pairwise testing algorithm. Additionally, our ComjaxTest generates a relatively complete test model and then detects more faults in a reasonable amount of time, as compared with other existing tools based on dynamic exploration.

Model-based automated testing of JavaScript Web applications via longer test sequences

JavaScript has become one of the most widely used languages for Web development.Its dynamic and event-driven features make it challenging to ensure the correctness of Web applications written in JavaScript.A variety of dynamic analysis techniques have been proposed which are,however,limited in either coverage or scalability.In this paper,we propose a simple,yet effective,model-based automated testing approach to achieve a high code-coverage within the time budget via testing with longer event sequences.We implement our approach as an open-source tool LJS,and perform extensive experiments on 21 publicly available benchmarks.On average,LJS is able to achieve 86.5%line coverage in 10 minutes.Compared with JSDEP,a state-of-the-art breadth-first search based automated testing tool enriched with partial order reduction,the coverage of LJS is 11%-19%higher than that of JSDEP on real-world large Web applications.Our empirical findings support that proper longer test sequences can achieve a higher code coverage in JavaScript Web application testing.

Novel Software Automated Testing System Based on J2EE

Software automated testing is one of the critical research subjects in the field of computer application. In this paper, a novel design of architecture called automated testing system (ATS) is proposed. Based on techniques relating to J2EE including MVC design pattern, Struts framework, etc, ATS can support any black-box testing business theoretically with relevant APIs programmed using Tcl script language beforehand. Moreover, as the core of ATS is built in Java, it can work in different environments without being recomplied. The efficiency of the new system is validated by plenty of applications in communication industry and the results also show the effectiveness and flexibility of the approach.

Automated Flight Test and System Identification for Rotary Wing Small Aerial Platform Using Frequency Responses Analysis

This paper proposes an autopilot system that can be used to control the small scale rotorcraft during the flight test for linear-frequency-domain system identification. The input frequency-sweep is generated automatically as part of the autopilot control command. Therefore the bandwidth coverage and consistency of the frequency-sweep are guaranteed to produce high quality data for system identification. Beside that, we can set the safety parameters during the flight test （maximum roll/pitch value, minimum altitude, etc.） so the safety of the whole flight test is guaranteed. This autopilot system is validated using hardware in the loop simulator for hover flight condition.

AWeb Application Fingerprint Recognition Method Based on Machine Learning

Web application fingerprint recognition is an effective security technology designed to identify and classify web applications,thereby enhancing the detection of potential threats and attacks.Traditional fingerprint recognition methods,which rely on preannotated feature matching,face inherent limitations due to the ever-evolving nature and diverse landscape of web applications.In response to these challenges,this work proposes an innovative web application fingerprint recognition method founded on clustering techniques.The method involves extensive data collection from the Tranco List,employing adjusted feature selection built upon Wappalyzer and noise reduction through truncated SVD dimensionality reduction.The core of the methodology lies in the application of the unsupervised OPTICS clustering algorithm,eliminating the need for preannotated labels.By transforming web applications into feature vectors and leveraging clustering algorithms,our approach accurately categorizes diverse web applications,providing comprehensive and precise fingerprint recognition.The experimental results,which are obtained on a dataset featuring various web application types,affirm the efficacy of the method,demonstrating its ability to achieve high accuracy and broad coverage.This novel approach not only distinguishes between different web application types effectively but also demonstrates superiority in terms of classification accuracy and coverage,offering a robust solution to the challenges of web application fingerprint recognition.

An Automated Penetration Semantic Knowledge Mining Algorithm Based on Bayesian Inference

Mining penetration testing semantic knowledge hidden in vast amounts of raw penetration testing data is of vital importance for automated penetration testing.Associative rule mining,a data mining technique,has been studied and explored for a long time.However,few studies have focused on knowledge discovery in the penetration testing area.The experimental result reveals that the long-tail distribution of penetration testing data nullifies the effectiveness of associative rule mining algorithms that are based on frequent pattern.To address this problem,a Bayesian inference based penetration semantic knowledge mining algorithm is proposed.First,a directed bipartite graph model,a kind of Bayesian network,is constructed to formalize penetration testing data.Then,we adopt the maximum likelihood estimate method to optimize the model parameters and decompose a large Bayesian network into smaller networks based on conditional independence of variables for improved solution efficiency.Finally,irrelevant variable elimination is adopted to extract penetration semantic knowledge from the conditional probability distribution of the model.The experimental results show that the proposed method can discover penetration semantic knowledge from raw penetration testing data effectively and efficiently.

Search-Based Software Test Data Generation for Path Coverage Based on a Feedback-Directed Mechanism

Automatically generating test cases by evolutionary algorithms to satisfy the path coverage criterion has attracted much research attention in software testing.In the context of generating test cases to cover many target paths,the efficiency of existing methods needs to be further improved when infeasible or difficult paths exist in the program under test.This is because a significant amount of the search budget(i.e.,time allocated for the search to run)is consumed when computing fitness evaluations of individuals on infeasible or difficult paths.In this work,we present a feedback-directed mechanism that temporarily removes groups of paths from the target paths when no improvement is observed for these paths in subsequent generations.To fulfill this task,our strategy first organizes paths into groups.Then,in each generation,the objective scores of each individual for all paths in each group are summed up.For each group,the lowest value of the summed up objective scores among all individuals is assigned as the best aggregated score for a group.A group is removed when no improvement is observed in its best aggregated score over the last two generations.The experimental results show that the proposed approach can significantly improve path coverage rates for programs under test with infeasible or difficult paths in case of a limited search budget.In particular,the feedback-directed mechanism reduces wasting the search budget on infeasible paths or on difficult target paths that require many fitness evaluations before getting an improvement.

A Method of Minimum Reusability Estimation for Automated Software Testing

Through reusing software test components, automated software testing generally costs less than manual software testing. There has been much research on how to develop the reusable test components, but few fall on how to estimate the reusability of test conlponents for automated testing. The purpose of this paper is to present a method of minimum reusability estimation for automated testing based on the return on investment （ROI） model. Minimum reusability is a benchmark for the whole automated testing process. If the reusability in one test execution is less than the minimum reusability, some new strategies must be adopted ill the next test execution to increase the reusability. Only by this way, we can reduce unnecessary costs and finally get a return on the investment of automated testing.

Cloud Platform Based Automated Security Testing System for Mobile Internet

With respect to security, the use of various terminals in the mobile Internet environment is problematic.Traditional terminal testing methods cannot simulate actual testing environments; thus, the test results do not accurately reflect the security of terminals. To address this problem, we designed and developed a cloud platform based automated testing system for the mobile Internet. In this system, virtualization and automation technology are utilized to integrate mobile terminals into the cloud platform as a resource, to achieve a novel cloud service called Testing as a Service(Taa S). The system consists of three functional modules: web front-end module, testing environment module, and automated testing module. We adopted the permeable automated testing tool Metasploit to perform security testing. In our test experiments, we selected 100 apps with diverse vulnerability levels, ranging from secure to vulnerable, to perform a series of functional tests. The experimental results show that this system can correctly test both the number of vulnerable apps and their corresponding vulnerability levels. As such, the designed system can flexibly configure various testing environments for different testing cases or projects, and thereby perform security testing automatically.