Big Data Testing Techniques:Taxonomy,Challenges and Future Trends

Big Data is reforming many industrial domains by providing decision support through analyzing large data volumes.Big Data testing aims to ensure that Big Data systems run smoothly and error-free while maintaining the performance and quality of data.However,because of the diversity and complexity of data,testing Big Data is challenging.Though numerous research efforts deal with Big Data testing,a comprehensive review to address testing techniques and challenges of BigData is not available as yet.Therefore,we have systematically reviewed the Big Data testing techniques’evidence occurring in the period 2010–2021.This paper discusses testing data processing by highlighting the techniques used in every processing phase.Furthermore,we discuss the challenges and future directions.Our findings show that diverse functional,non-functional and combined(functional and non-functional)testing techniques have been used to solve specific problems related to Big Data.At the same time,most of the testing challenges have been faced during the MapReduce validation phase.In addition,the combinatorial testing technique is one of the most applied techniques in combination with other techniques(i.e.,random testing,mutation testing,input space partitioning and equivalence testing)to find various functional faults through Big Data testing.

Generating of Test Data by Harmony Search Against Genetic Algorithms

Many search-based algorithms have been successfully applied in sev-eral software engineering activities.Genetic algorithms(GAs)are the most used in the scientific domains by scholars to solve software testing problems.They imi-tate the theory of natural selection and evolution.The harmony search algorithm(HSA)is one of the most recent search algorithms in the last years.It imitates the behavior of a musician tofind the best harmony.Scholars have estimated the simi-larities and the differences between genetic algorithms and the harmony search algorithm in diverse research domains.The test data generation process represents a critical task in software validation.Unfortunately,there is no work comparing the performance of genetic algorithms and the harmony search algorithm in the test data generation process.This paper studies the similarities and the differences between genetic algorithms and the harmony search algorithm based on the ability and speed offinding the required test data.The current research performs an empirical comparison of the HSA and the GAs,and then the significance of the results is estimated using the t-Test.The study investigates the efficiency of the harmony search algorithm and the genetic algorithms according to(1)the time performance,(2)the significance of the generated test data,and(3)the adequacy of the generated test data to satisfy a given testing criterion.The results showed that the harmony search algorithm is significantly faster than the genetic algo-rithms because the t-Test showed that the p-value of the time values is 0.026<α(αis the significance level=0.05 at 95%confidence level).In contrast,there is no significant difference between the two algorithms in generating the adequate test data because the t-Test showed that the p-value of thefitness values is 0.25>α.

Obtaining 2D Soil Resistance Profiles from the Integration of Electrical Resistivity Data and Standard Penetration Test (SPT) and Light Dynamic Penetrometer (DPL) Resistance Tests—Applications in Mass Movements Studies

In Brazil and various regions globally, the initiation of landslides is frequently associated with rainfall;yet the spatial arrangement of geological structures and stratification considerably influences landslide occurrences. The multifaceted nature of these influences makes the surveillance of mass movements a highly intricate task, requiring an understanding of numerous interdependent variables. Recent years have seen an emergence in scholarly research aimed at integrating geophysical and geotechnical methodologies. The conjoint examination of geophysical and geotechnical data offers an enhanced perspective into subsurface structures. Within this work, a methodology is proposed for the synchronous analysis of electrical resistivity geophysical data and geotechnical data, specifically those extracted from the Light Dynamic Penetrometer (DPL) and Standard Penetration Test (SPT). This study involved a linear fitting process to correlate resistivity with N10/SPT N-values from DPL/SPT soundings, culminating in a 2D profile of N10/SPT N-values predicated on electrical profiles. The findings of this research furnish invaluable insights into slope stability by allowing for a two-dimensional representation of penetration resistance properties. Through the synthesis of geophysical and geotechnical data, this project aims to augment the comprehension of subsurface conditions, with potential implications for refining landslide risk evaluations. This endeavor offers insight into the formulation of more effective and precise slope management protocols and disaster prevention strategies.

Quantitative Analysis of Seeing with Height and Time at Muztagh-Ata Site Based on ERA5 Database

Seeing is an important index to evaluate the quality of an astronomical site.To estimate seeing at the Muztagh-Ata site with height and time quantitatively,the European Centre for Medium-Range Weather Forecasts reanalysis database(ERA5)is used.Seeing calculated from ERA5 is compared consistently with the Differential Image Motion Monitor seeing at the height of 12 m.Results show that seeing decays exponentially with height at the Muztagh-Ata site.Seeing decays the fastest in fall in 2021 and most slowly with height in summer.The seeing condition is better in fall than in summer.The median value of seeing at 12 m is 0.89 arcsec,the maximum value is1.21 arcsec in August and the minimum is 0.66 arcsec in October.The median value of seeing at 12 m is 0.72arcsec in the nighttime and 1.08 arcsec in the daytime.Seeing is a combination of annual and about biannual variations with the same phase as temperature and wind speed indicating that seeing variation with time is influenced by temperature and wind speed.The Richardson number Ri is used to analyze the atmospheric stability and the variations of seeing are consistent with Ri between layers.These quantitative results can provide an important reference for a telescopic observation strategy.

Optimal Test Points Selection Based on Multi-Objective Genetic Algorithm

A new approach to select anoptimal set of test points is proposed. The described method uses fault-wise table and multi-objective genetic algorithm to find the optimal set of test points. First, the fault-wise table is constructed whose entries are measurements associated with faults and test points. The selection of optimal test points is transformed to the selection of the columns that isolate the rows of the table. Then, four objectives are described according to practical test requirements. The multi-objective genetic algorithm is explained. Finally, the presented approach is illustrated by a practical example. The results indicate that the proposed method can efficiently and accurately find the optimal set of test points and is practical for large scale systems.

Analysis of Incomplete Data of Accelerated Life Testing with Competing Failure Modes

Data obtained from accelerated life testing （ALT） when there are two or more failure modes, which is commonly referred to as competing failure modes, are often incomplete. The incompleteness is mainly due to censoring, as well as masking which might be the case that the failure time is observed, but its corresponding failure mode is not identified. Because the identification of the failure mode may be expensive, or very difficult to investigate due to lack of appropriate diagnostics. A method is proposed for analyzing incomplete data of constant stress ALT with competing failure modes. It is assumed that failure modes have s-independent latent lifetimes and the log lifetime of each failure mode can be written as a linear function of stress. The parameters of the model are estimated by using the expectation maximum （EM） algorithm with incomplete data. Simulation studies are performed to check＇model validity and investigate the properties of estimates. For further validation, the method is also illustrated by an example, which shows the process of analyze incomplete data from ALT of some insulation system. Because of considering the incompleteness of data in modeling and making use of the EM algorithm in estimating, the method becomes more flexible in ALT analysis.

Impact of elliptical boreholes on in situ stress estimation from leak-off test data

We developed an inversion technique to determine in situ stresses for elliptical boreholes of arbitrary trajectory. In this approach, borehole geometry, drilling-induced fracture information, and other available leak-off test data were used to construct a mathematical model, which was in turn applied to finding the inverse of an overdetermined system of equations.The method has been demonstrated by a case study in the Appalachian Basin, USA. The calculated horizontal stresses are in reasonable agreement with the reported regional stress study of the area, although there are no field measurement data of the studied well for direct calibration. The results also indicate that 2% of axis difference in the elliptical borehole geometry can cause a 5% difference in minimum horizontal stress calculation and a 10% difference in maximum horizontal stress calculation.

TESTING FOR VARYING DISPERSION OF LONGITUDINAL BINOMIAL DATA IN NONLINEAR LOGISTIC MODELS WITH RANDOM EFFECTS

In this paper, it is discussed that two tests for varying dispersion of binomial data in the framework of nonlinear logistic models with random effects, which are widely used in analyzing longitudinal binomial data. One is the individual test and power calculation for varying dispersion through testing the randomness of cluster effects, which is extensions of Dean(1992) and Commenges et al (1994). The second test is the composite test for varying dispersion through simultaneously testing the randomness of cluster effects and the equality of random-effect means. The score test statistics are constructed and expressed in simple, easy to use, matrix formulas. The authors illustrate their test methods using the insecticide data (Giltinan, Capizzi & Malani (1988)).

Multi-objective optimization and experiment of nylon cord rubber in expandable packer

Nylon cord rubber has the advantages of small residual deformation and is easy to lift and lower the tubing string in low-permeability oil and gas reservoirs.However,it is associated with low-pressure resistance and poor sealing performance.To enhance the performance of nylon cord rubber,a three-dimensional numerical model of the nylon cord rubber was established and its accuracy experimentally determined.The Plackett-Burman test,the Steepest climbing test and the Response surface method were used to acquire the polynomial response surface model connecting structural parameters with bearing and sealing pressure.Using genetic algorithms,optimal structural parameters of nylon cord rubber were determined depending on field operation.The reliability of the optimized results was verified by laboratory tests.It was shown that after optimization,the bearing capacity of the expandable packer increased by 25%while the sealing performance increased by 66%.In addition,the bearing pressure was 70 MPa while the sealing pressure was 50 MPa.These measurements effectively met the on-site requirements of high-pressure and fine fracturing in low-permeability oil and gas reservoirs.

Inference and optimal design on step-stress partially accelerated life test for hybrid system with masked data

Under Type-Ⅱ progressively hybrid censoring, this paper discusses statistical inference and optimal design on stepstress partially accelerated life test for hybrid system in presence of masked data. It is assumed that the lifetime of the component in hybrid systems follows independent and identical modified Weibull distributions. The maximum likelihood estimations(MLEs)of the unknown parameters, acceleration factor and reliability indexes are derived by using the Newton-Raphson algorithm. The asymptotic variance-covariance matrix and the approximate confidence intervals are obtained based on normal approximation to the asymptotic distribution of MLEs of model parameters. Moreover,two bootstrap confidence intervals are constructed by using the parametric bootstrap method. The optimal time of changing stress levels is determined under D-optimality and A-optimality criteria.Finally, the Monte Carlo simulation study is carried out to illustrate the proposed procedures.

Goodness-of-fit tests for multi-dimensional copulas:Expanding application to historical drought data

The question of how to choose a copula model that best fits a given dataset is a predominant limitation of the copula approach, and the present study aims to investigate the techniques of goodness-of-fit tests for multi-dimensional copulas. A goodness-of-fit test based on Rosenblatt＇s transformation was mathematically expanded from two dimensions to three dimensions and procedures of a bootstrap version of the test were provided. Through stochastic copula simulation, an empirical application of historical drought data at the Lintong Gauge Station shows that the goodness-of-fit tests perform well, revealing that both trivariate Gaussian and Student t copulas are acceptable for modeling the dependence structures of the observed drought duration, severity, and peak. The goodness-of-fit tests for multi-dimensional copulas can provide further support and help a lot in the potential applications of a wider range of copulas to describe the associations of correlated hydrological variables. However, for the application of copulas with the number of dimensions larger than three, more complicated computational efforts as well as exploration and parameterization of corresponding copulas are required.

The Trail Making Test in Morocco: Normative Data Stratified by Age and Level of Education

The purpose of this study is to design a Moroccan Trail Making Test B;explore the effects of age, education and gender on the performance of the Trail Making Test (TMT);and provide normative information in Moroccan subjects. Our normalization study was conducted on 348 subjects (156 female and 192 male). The subjects were classified into four groups based on age (18 - 39 years, 40 - 59 years, 60 - 69 years and ≥70 years), and three groups based on educational level (3 - 6 years, 7 - 10 years and ≥11 years). The data were analyzed using descriptive statistics through SPSS. The results displayed that increasing age and decreasing levels of education significantly result in a decreased performance on the Trail A, Moroccan Trail B and English Trail B. Only 229 bilingual subjects among the 348 initial subjects completed both versions of Trail B. There was no significant difference on performance between Moroccan Trail B and English Trail B for these subjects.

Analysis of video head impulse test saccades data in patients with vestibular migraine or probable vestibular migraine

Objective:Saccades accompanied by normal gain in video head impulse tests(vHIT)are often observed in patients with vestibular migraine(VM).However,they are not considered as an independent indicator,reducing their utility in diagnosing VM.To better understand clinical features of VM,it is necessary to understand raw saccades data.Methods:Fourteen patients with confirmed VM,45 patients with probable VM(p-VM)and 14 agematched healthy volunteers were included in this study.Clinical findings related to spontaneous nystagmus(SN),positional nystagmus(PN),head-shaking nystagmus(HSN),caloric test and vHIT were recorded.Raw saccades data were exported and numbered by their sequences,and their features analyzed.Results:VM patients showed no SN,PN or HSN,and less than half of them showed unilateral weakness(UW)on caloric test.The first saccades from lateral semicircular canal stimulation were the most predominant for both left and right sides.Neither velocity nor time parameters were significantly different when compared between the two sides.Most VM patients(86%)exhibited small saccades,around 35%of the head peak velocity,with a latency of 200e400 ms.Characteristics of saccades were similar in patients with p-VM.Only four normal subjects showed saccades,all unilateral and seemingly random.Conclusions:Small saccades involving bilateral semicircular canals with a scattered distribution pattern are common in patients with VM and p-VM.

Using Genetic Algorithm as Test Data Generator for Stored PL/SQL Program Units

PL/SQL is the most common language for ORACLE database application. It allows the developer to create stored program units (Procedures, Functions, and Packages) to improve software reusability and hide the complexity of the execution of a specific operation behind a name. Also, it acts as an interface between SQL database and DEVELOPER. Therefore, it is important to test these modules that consist of procedures and functions. In this paper, a new genetic algorithm (GA), as search technique, is used in order to find the required test data according to branch criteria to test stored PL/SQL program units. The experimental results show that this was not fully achieved, such that the test target in some branches is not reached and the coverage percentage is 98%. A problem rises when target branch is depending on data retrieved from tables;in this case, GA is not able to generate test cases for this branch.

Structural damage identification using test static data based on grey system theory

A new structural damage identification method using limited test static displacement based on grey system theory is proposed in this paper. The grey relation coefficient of displacement curvature is defined and used to locate damage in the structure, and an iterative estimation scheme for solving nonlinear optimization programming problems based on the quadratic programming technique is used to identify the damage magnitude. A numerical example of a cantilever beam with single or multiple damages is used to examine the capability of the proposed grey-theory-based method to localize and identify damages. The factors of meas-urement noise and incomplete test data are also discussed. The numerical results showed that the damage in the structure can be localized correctly through using the grey-related coefficient of displacement curvature, and the damage magnitude can be iden-tified with a high degree of accuracy, regardless of the number of measured displacement nodes. This proposed method only requires limited static test data, which is easily available in practice, and has wide applications in structural damage detection.

A Test Pattern Identification Algorithm and Its Application to CINRAD/SA(B) Data

A variety of faulty radar echoes may cause serious problems with radar data applications,especially radar data assimilation and quantitative precipitation estimates.In this study,＂test pattern＂ caused by test signal or radar hardware failures in CINRAD （China New Generation Weather Radar） SA and SB radar operational observations are investigated.In order to distinguish the test pattern from other types of radar echoes,such as precipitation,clear air and other non-meteorological echoes,five feature parameters including the effective reflectivity data percentage （Rz）,velocity RF （range folding） data percentage （RRF）,missing velocity data percentage （RM）,averaged along-azimuth reflectivity fluctuation （RNr,z） and averaged along-beam reflectivity fluctuation （RNa,z） are proposed.Based on the fuzzy logic method,a test pattern identification algorithm is developed,and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm.Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed.The statistical results show that the test pattern identification algorithm performs well,since the test pattern is recognized in most cases.Besides,the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.

DAMAGE DETECTION IN BUILDINGS USING A TWO-STAGE SENSITIVITY-BASED METHOD FROM MODAL TEST DATA

Many multi-story or highrise buildings consisting of a number of identical stories are usually considered as periodic spring-mass systems. The general expressions of natural frequencies, mode shapes, slopes and curvatures of mode shapes of the periodic spring-mass system by utilizing the periodic structure theory are derived in this paper. The sensitivities of these mode parameters with respect to structural damages, which do not depend on the physical parameters of the original structures, are obtained. Based on the sensitivity analysis of these mode parameters, a two-stage method is proposed to localize and quantify damages of multi-story or highrise buildings. The slopes and curvatures of mode shapes, which are highly sensitive to local damages, are used to localize the damages. Subsequently, the limited measured natural frequencies, which have a better accuracy than the other mode parameters, are used to quantify the extent of damages within the potential damaged locations. The experimental results of a 3-story experimental building demonstrate that the single or multiple damages of buildings, either slight or severe, can be correctly localized by using only the slope or curvature of mode shape in one of the lower modes, in which the change of natural frequency is the largest, and can be accurately quantified by the limited measured natural frequencies with noise pollution.

Processing the rig test data of an air filling twin-tube shock absorber

A separation method is proposed to design and improve shock absorber according to the characteristics of each force. The method is validated by rig test. The force data measured during rig test is the resultant force of damping force, rebound force produced by pressed air, and friction force. Different characters of damping force, air rebound force and friction force can be applied to seperate each force from others. A massive produced air filling shock absorber is adopted for the validation. The statistic test is used to get the displacement-force curves. The data are used as the input of separation calculation. Then the tests are carried out again to obtain the force data without air rebound force. The force without air rebound is compared to the data derived from the former tests with the separation method. The result shows that this method can separate the damping force and the air elastic force.

Compression Design and Test of Water Vapor Receiving Data by Beidou Satellite

Water vapor monitoring system by Beidou satellite is a new detection system in meteorological department, which makes receiving amount of detected data and data storage and transmission pressure increase. Here, we try to use data compression to relieve pressure. Compres- sion software of water vapor monitoring system by Beidou satellite can be designed into three components： real-time compression software, check compression software and manual compression software, which respectively completes the compression tasks under real-time receiving, in-time check and separate compression, thereby forming a perfect compression system. Taking the design of manual compression software as guide,and using c language to develop,compression test of original receiving data is conducted. Test result proves that the system can carry out batch auto- matic compression, and compression rate can reach 30% ,which can reach the target of saving space in a degree.

An EFSM-Based Test Data Generation Approach in Model-Based Testing

Testing is an integral part of software development.Current fastpaced system developments have rendered traditional testing techniques obsolete.Therefore,automated testing techniques are needed to adapt to such system developments speed.Model-based testing(MBT)is a technique that uses system models to generate and execute test cases automatically.It was identified that the test data generation(TDG)in many existing model-based test case generation(MB-TCG)approaches were still manual.An automatic and effective TDG can further reduce testing cost while detecting more faults.This study proposes an automated TDG approach in MB-TCG using the extended finite state machine model(EFSM).The proposed approach integrates MBT with combinatorial testing.The information available in an EFSM model and the boundary value analysis strategy are used to automate the domain input classifications which were done manually by the existing approach.The results showed that the proposed approach was able to detect 6.62 percent more faults than the conventionalMB-TCG but at the same time generated 43 more tests.The proposed approach effectively detects faults,but a further treatment to the generated tests such as test case prioritization should be done to increase the effectiveness and efficiency of testing.