A Study of EM Algorithm as an Imputation Method: A Model-Based Simulation Study with Application to a Synthetic Compositional Data

Compositional data, such as relative information, is a crucial aspect of machine learning and other related fields. It is typically recorded as closed data or sums to a constant, like 100%. The statistical linear model is the most used technique for identifying hidden relationships between underlying random variables of interest. However, data quality is a significant challenge in machine learning, especially when missing data is present. The linear regression model is a commonly used statistical modeling technique used in various applications to find relationships between variables of interest. When estimating linear regression parameters which are useful for things like future prediction and partial effects analysis of independent variables, maximum likelihood estimation (MLE) is the method of choice. However, many datasets contain missing observations, which can lead to costly and time-consuming data recovery. To address this issue, the expectation-maximization (EM) algorithm has been suggested as a solution for situations including missing data. The EM algorithm repeatedly finds the best estimates of parameters in statistical models that depend on variables or data that have not been observed. This is called maximum likelihood or maximum a posteriori (MAP). Using the present estimate as input, the expectation (E) step constructs a log-likelihood function. Finding the parameters that maximize the anticipated log-likelihood, as determined in the E step, is the job of the maximization (M) phase. This study looked at how well the EM algorithm worked on a made-up compositional dataset with missing observations. It used both the robust least square version and ordinary least square regression techniques. The efficacy of the EM algorithm was compared with two alternative imputation techniques, k-Nearest Neighbor (k-NN) and mean imputation (), in terms of Aitchison distances and covariance.

Development of a Quantitative Prediction Support System Using the Linear Regression Method

The development of prediction supports is a critical step in information systems engineering in this era defined by the knowledge economy, the hub of which is big data. Currently, the lack of a predictive model, whether qualitative or quantitative, depending on a company’s areas of intervention can handicap or weaken its competitive capacities, endangering its survival. In terms of quantitative prediction, depending on the efficacy criteria, a variety of methods and/or tools are available. The multiple linear regression method is one of the methods used for this purpose. A linear regression model is a regression model of an explained variable on one or more explanatory variables in which the function that links the explanatory variables to the explained variable has linear parameters. The purpose of this work is to demonstrate how to use multiple linear regressions, which is one aspect of decisional mathematics. The use of multiple linear regressions on random data, which can be replaced by real data collected by or from organizations, provides decision makers with reliable data knowledge. As a result, machine learning methods can provide decision makers with relevant and trustworthy data. The main goal of this article is therefore to define the objective function on which the influencing factors for its optimization will be defined using the linear regression method.

Preconditioned iterative methods for solving weighted linear least squares problems

A class of preconditioned iterative methods, i.e., preconditioned generalized accelerated overrelaxation （GAOR） methods, is proposed to solve linear systems based on a class of weighted linear least squares problems. The convergence and comparison results are obtained. The comparison results show that the convergence rate of the preconditioned iterative methods is better than that of the original methods. Furthermore, the effectiveness of the proposed methods is shown in the numerical experiment.

Calculation of Significant Wave Height Using the Linear Mean Square Estimation Method

Significant wave height is an important criterion in designing coastal and offshore structures.Based on the orthogonality principle, the linear mean square estimation method is applied to calculate significant wave height in this paper.Twenty-eight-year time series of wave data collected from three ocean buoys near San Francisco along the California coast are analyzed.It is proved theoretically that the computation error will be reduced by using as many measured data as possible for the calculation of significant wave height.Measured significant wave height at one buoy location is compared with the calculated value based on the data from two other adjacent buoys.The results indicate that the linear mean square estimation method can be well applied to the calculation and prediction of significant wave height in coastal regions.

Least Squares Method from the View Point of Deep Learning

The least squares method is one of the most fundamental methods in Statistics to estimate correlations among various data. On the other hand, Deep Learning is the heart of Artificial Intelligence and it is a learning method based on the least squares. In this paper we reconsider the least squares method from the view point of Deep Learning and we carry out the computation thoroughly for the gradient descent sequence in a very simple setting. Depending on the values of the learning rate, an essential parameter of Deep Learning, the least squares methods of Statistics and Deep Learning reveal an interesting difference.

Estimation of mass transfer coefficient in ozone absorption by linear least square fitting and Simplex search methods

For physical ozone absorption without reaction,two parametric estimation methods,i.e.the common linear least square fitting and non-linear Simplex search methods,were applied,respectively,to determine the ozone mass transfer coefficient during absorption and both methods give almost the same mass transfer coefficient.While for chemical absorption with ozone decomposition reaction,the common linear least square fitting method is not applicable for the evaluation of ozone mass transfer coefficient due to the difficulty of model linearization for describing ozone concentration dissolved in water.The nonlinear Simplex method obtains the mass transfer coefficient by minimizing the sum of the differences between the simulated and experimental ozone concentration during the whole absorption process,without the limitation of linear relationship between the dissolved ozone concentration and absorption time during the initial stage of absorption.Comparison of the ozone concentration profiles between the simulation and experimental data demonstrates that Simplex method may determine ozone mass transfer coefficient during absorption in an accurate and high efficiency way with wide applicability.

ON THE BREAKDOWNS OF THE GALERKIN AND LEAST-SQUARES METHODS

The Galerkin and least-squares methods are two classes of the most popular Krylov subspace methOds for solving large linear systems of equations. Unfortunately, both the methods may suffer from serious breakdowns of the same type: In a breakdown situation the Galerkin method is unable to calculate an approximate solution, while the least-squares method, although does not really break down, is unsucessful in reducing the norm of its residual. In this paper we first establish a unified theorem which gives a relationship between breakdowns in the two methods. We further illustrate theoretically and experimentally that if the coefficient matrix of a lienar system is of high defectiveness with the associated eigenvalues less than 1, then the restarted Galerkin and least-squares methods will be in great risks of complete breakdowns. It appears that our findings may help to understand phenomena observed practically and to derive treatments for breakdowns of this type.

A SELF-ADAPTIVE ALGORITHM FOR NONLINEAR LEAST SQUARES WITH LINEAR CONSTRAINTS

An algorithm for solving nonlinear least squares problems with general linear inequality constraints is described.At each step,the problem is reduced to an unconstrained linear least squares problem in a subs pace defined by the active constraints,which is solved using the quasi-Newton method.The major update formula is similar to the one given by Dennis,Gay and Welsch (1981).In this paper,we state the detailed implement of the algorithm,such as the choice of active set,the solution of subproblem and the avoidance of zigzagging.We also prove the globally convergent property of the algorithm.

Alternate Broyden's Method for Solving Linear Least Squares Problem with Multiple Right-Hand Sides

In this paper, we extend the alternate Broyden's method to the multiple version fbi solving lincar leastsquarc systems with multiple right-hand sides. We show that the method possesses property of a finite tcrmination.Some numerical cxperiments are gi von to inustrate the effectiveness of the method.

Seismic displacement demand prediction in non-linear domain: Optimization of the N2 method

In Europe, computation of displacement demand for seismic assessment of existing buildings is essentially based on a simplified formulation of the N2 method as prescribed by Eurocode 8(EC8). However, a lack of accuracy of the N2 method in certain conditions has been pointed out by several studies. This paper addresses the assessment of effectiveness of the N2 method in seismic displacement demand determination in non-linear domain. The objective of this work is to investigate the accuracy of the N2 method through comparison with displacement demands computed using non-linear timehistory analysis(NLTHA). Results show that the original N2 method may lead to overestimation or underestimation of displacement demand predictions. This may affect results of mechanical model-based assessment of seismic vulnerability at an urban scale. Hence, the second part of this paper addresses an improvement of the N2 method formula by empirical evaluation of NLTHA results based on EC8 ground-classes. This task is formulated as a mathematical programming problem in which coefficients are obtained by minimizing the overall discrepancy between NLTHA and modified formula results. Various settings of the mathematical programming problem have been solved using a global optimization metaheuristic. An extensive comparison between the original N2 method formulation and optimized formulae highlights benefits of the strategy.

A NEW SOLUTION MODEL OF NONLINEAR DYNAMIC LEAST SQUARE ADJUSTMENT

The nonlinear least square adjustment is a head object studied in technology fields. The paper studies on the non derivative solution to the nonlinear dynamic least square adjustment and puts forward a new algorithm model and its solution model. The method has little calculation load and is simple. This opens up a theoretical method to solve the linear dynamic least square adjustment.

ANALYSIS OF NONLINEAR PIEZOELECTRIC CIRCULAR SHALLOW SPHERICAL SHELLS BY DIFFERENTIAL QUADRATURE ELEMENT METHOD

The static behavior of piezoelectric circular spherical shallow shells under both electrical and mechanical loads is studied by using the differential quadrature element method (DQEM). Geometrical nonlinearity effect is considered. Detailed formulations and procedures are given for the first time. Several examples are analyzed and accurate results are obtained by the DQEM. Based on the results in this paper, one may conclude that the DQEM is a useful tool for obtaining solutions of structural elements. It can be seen that the shell shape may be theore tically controlled and snap through may occur when the applied voltage reaches a critical value even without mechanical load for certain geometric configurations.

Low-complexity signal detection for massive MIMO systems via trace iterative method

Linear minimum mean square error(MMSE)detection has been shown to achieve near-optimal performance for massive multiple-input multiple-output(MIMO)systems but inevitably involves complicated matrix inversion,which entails high complexity.To avoid the exact matrix inversion,a considerable number of implicit and explicit approximate matrix inversion based detection methods is proposed.By combining the advantages of both the explicit and the implicit matrix inversion,this paper introduces a new low-complexity signal detection algorithm.Firstly,the relationship between implicit and explicit techniques is analyzed.Then,an enhanced Newton iteration method is introduced to realize an approximate MMSE detection for massive MIMO uplink systems.The proposed improved Newton iteration significantly reduces the complexity of conventional Newton iteration.However,its complexity is still high for higher iterations.Thus,it is applied only for first two iterations.For subsequent iterations,we propose a novel trace iterative method(TIM)based low-complexity algorithm,which has significantly lower complexity than higher Newton iterations.Convergence guarantees of the proposed detector are also provided.Numerical simulations verify that the proposed detector exhibits significant performance enhancement over recently reported iterative detectors and achieves close-to-MMSE performance while retaining the low-complexity advantage for systems with hundreds of antennas.

Analytical Solutions of System of Non-Linear Differential Equations in the Single-Enzyme, Single-Substrate Reaction with Non-Mechanism-Based Enzyme Inactivation

A closed form of an analytical expression of concentration in the single-enzyme, single-substrate system for the full range of enzyme activities has been derived. The time dependent analytical solution for substrate, enzyme-substrate complex and product concentrations are presented by solving system of non-linear differential equation. We employ He’s Homotopy perturbation method to solve the coupled non-linear differential equations containing a non-linear term related to basic enzymatic reaction. The time dependent simple analytical expressions for substrate, enzyme-substrate and free enzyme concentrations have been derived in terms of dimensionless reaction diffusion parameters ε, λ1, λ2 and λ3 using perturbation method. The numerical solution of the problem is also reported using SCILAB software program. The analytical results are compared with our numerical results. An excellent agreement with simulation data is noted. The obtained results are valid for the whole solution domain.

A NEW LEAST SQUARE ALGORITHM FOR LINEAR PROGRAMMING

By attacking the linear programming problems from their dual side,a new general algorithm for linear programming is developed.At each iteration,the algorithm finds a feasible descent search direction by handling a least square problem associated with the dual system,using QR decomposition technique.The new method is a combination of pivot method and interior-point method.It in fact not only reduces the possibility of difficulty arising from degeneracy,but also has the same advantages as pivot method in warm-start to resolve linear programming problems.Numerical results of a group of randomly constructed problems are very encouraging.

NON-LINEAR DYNAMIC BEHAVIOR OF THERMOELASTIC CIRCULAR PLATE WITH VARYING THICKNESS SUBJECTED TO NONCONSERVATIVE LOADING

The non-linear dynamic behaviors of thermoelastic circular plate with varying thickness subjected to radially uniformly distributed follower forces are considered. Two coupled non-linear differential equations of motion for this problem are derived in terms of the transverse deflection and radial displacement component of the mid-plane of the plate. Using the Kantorovich averaging method, the differential equation of mode shape of the plate is derived, and the eigenvalue problem is solved by using shooting method. The eigencurves for frequencies and critical loads of the circular plate with unmovable simply supported edge and clamped edge are obtained. The effects of the variation of thickness and temperature on the frequencies and critical loads of the thermoelastic circular plate subjected to radially uniformly distributed follower forces are then discussed.

Non-Linear Mathematical Model of the Interaction between Tumor and Oncolytic Viruses

A mathematical modeling of tumor therapy with oncolytic viruses is discussed. The model consists of two coupled, deterministic differential equations allowing for cell reproduction and death, and cell infection. The model is one of the conceptual mathematical models of tumor growth that treat a tumor as a dynamic society of interacting cells. In this paper, we obtain an approximate analytical expression of uninfected and infected cell population by solving the non-linear equations using Homotopy analysis method (HAM). Furthermore, the results are compared with the numerical simulation of the problem using Matlab program. The obtained results are valid for the whole solution domain.

Double Elzaki Transform Decomposition Method for Solving Non-Linear Partial Differential Equations

In this paper, we discuss a new method employed to tackle non-linear partial differential equations, namely Double Elzaki Transform Decomposition Method (DETDM). This method is a combination of the Double ELzaki Transform and Adomian Decomposition Method. This technique is hereafter provided and supported with necessary illustrations, together with some attached examples. The results reveal that the new method is very efficient, simple and can be applied to other non-linear problems.

Research on the Model of Linear Data Fitting Method

By using the method of least square linear fitting to analyze data do not exist errors under certain conditions, in order to make the linear data fitting method that can more accurately solve the relationship expression between the volume and quantity in scientific experiments and engineering practice, this article analyzed data error by commonly linear data fitting method, and proposed improved process of the least distance squ^re method based on least squares method. Finally, the paper discussed the advantages and disadvantages through the example analysis of two kinds of linear data fitting method, and given reasonable control conditions for its application.

Optimized CUDA Implementation to Improve the Performance of Bundle Adjustment Algorithm on GPUs

The 3D reconstruction pipeline uses the Bundle Adjustment algorithm to refine the camera and point parameters. The Bundle Adjustment algorithm is a compute-intensive algorithm, and many researchers have improved its performance by implementing the algorithm on GPUs. In the previous research work, “Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm using GPUs,” the authors demonstrated first the Bundle Adjustment algorithmic performance improvement by reducing the mean square error using an additional radial distorting parameter and explicitly computed analytical derivatives and reducing the computational burden of the Bundle Adjustment algorithm using GPUs. The naïve implementation of the CUDA code, a speedup of 10× for the largest dataset of 13,678 cameras, 4,455,747 points, and 28,975,571 projections was achieved. In this paper, we present the optimization of the Bundle Adjustment algorithm CUDA code on GPUs to achieve higher speedup. We propose a new data memory layout for the parameters in the Bundle Adjustment algorithm, resulting in contiguous memory access. We demonstrate that it improves the memory throughput on the GPUs, thereby improving the overall performance. We also demonstrate an increase in the computational throughput of the algorithm by optimizing the CUDA kernels to utilize the GPU resources effectively. A comparative performance study of explicitly computing an algorithm parameter versus using the Jacobians instead is presented. In the previous work, the Bundle Adjustment algorithm failed to converge for certain datasets due to several block matrices of the cameras in the augmented normal equation, resulting in rank-deficient matrices. In this work, we identify the cameras that cause rank-deficient matrices and preprocess the datasets to ensure the convergence of the BA algorithm. Our optimized CUDA implementation achieves convergence of the Bundle Adjustment algorithm in around 22 seconds for the largest dataset compared to 654 seconds for the sequential implementation, resulting in a speedup of 30×. Our optimized CUDA implementation presented in this paper has achieved a 3× speedup for the largest dataset compared to the previous naïve CUDA implementation.