Actor-Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking

This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process.A reinforcement learning(RL)agent successfully tracks an interface between two liquids,which is often a critical variable to track in many chemical,petrochemical,metallurgical,and oil industries.This method utilizes less than 100 images for creating an environment,from which the agent generates its own data without the need for expert knowledge.Unlike supervised learning(SL)methods that rely on a huge number of parameters,this approach requires far fewer parameters,which naturally reduces its maintenance cost.Besides its frugal nature,the agent is robust to environmental uncertainties such as occlusion,intensity changes,and excessive noise.From a closed-loop control context,an interface location-based deviation is chosen as the optimization goal during training.The methodology showcases RL for real-time object-tracking applications in the oil sands industry.Along with a presentation of the interface tracking problem,this paper provides a detailed review of one of the most effective RL methodologies:actor–critic policy.

AN ADAPTIVE FAST INTERFACE TRACKING METHOD

An adaptive numerical scheme is developed for the propagation of an interface in a velocity field based on the fast interface tracking method proposed in [2]. A multiresolution stategy to represent the interface instead of point values, allows local grid refinement while controlling the approximation error on the interface. For time integration, we use an explicit Runge-Kutta scheme of second-order with a multiseale time step, which takes longer time steps for finer spatial scales. The implementation of the algorithm uses a dynamic tree data structure to represent data in the computer memory. We briefly review first the main algorithm, describe the essential data structures, highlight the adaptive scheme, and illustrate the computational efficiency by some numerical examples.

Jet formation and penetration mechanism of W typed shaped charge

Existing classical shaped charges are well known for their longer jets capable of achieving large hole depth to hole diameter ratios in metallic targets. However, in some situations, there arises demand to obtain 1：1 ratio for hole depth to hole diameter which is beyond normal shaped charges capability. A new variant of shape charge, named W typed shape charge （WSC）, is proposed in this paper, which can meet the demand of 1：1 ratio, and is based on the geometry that can produce annular jets upon proper initiation scheme. In this paper, we present formation and penetration results of WSC based on three different schemes. We also show that not all WSC designs can form annular jets, only annularly initiated WSC, which also fulfils the ＂Internal-External Liners Equal-Impulse＂ criterion, has the capability to form annular jet. The experimental and numerical results show that when the ratio between annular initiation ring diameter and the charge diameter is 0.75, an annular jet is formed, which was also supported by high speed photographs performed in vacuum. 2D numerical simulations are performed with indigenously developed simulation software, where Eulerian approach with multi-material interface tracking algorithm is utilized, to find various mechanisms involved during jet formation process. The calculation results are found in good agreement with the experimental results, indicating that the interface treatment algorithm proposed in this paper can not only deal with large deformation problem, but also depict clearly the variation of materials interface. It is especially suitable for simulation of the process from liner collapse to formation of shaped charge jet.

NUMERICAL SIMULATION OF THE FLOW WITH CONTACT LINES

The paper proposes a physical model for the motion of the contact line and the gas-liquid interface. The local motion of the contact line at the solid wall is assumed and the interface between gas and liquid is traced by a level function. The finite volume method and staggered grids are used to solve the governing equation numerically. The motion of the water column in a vertical pipe is computed and the results are in good agreement with experimental data.

Application of Gridless Method to Simulation of Compressible Multi-Material Flows

The least-square gridless method was extended to simulate the compressible multi-material flows. The algorithm was accomplished to solve the Arbitrary Lagrange-Euler( ALE) formulation. The local least-square curve fits was adopted to approximate the spatial derivatives of a point on the base of the points in its circular support domain,and the basis function was linear. The HLLC( Harten-Lax-van Leer-Contact) scheme was used to calculate the inviscid flux. On the material interfaces,the gridless points were endued with a dual definition corresponding to different materials. The moving velocity of the interface points was updated by solving the Riemann problem. The interface boundary condition was built by using the Ghost Fluid Method( GFM).Computations were performed for several one and two dimensional typical examples. The numerical results show that the interface and the shock wave are well captured,which proves the effectiveness of gridless method in dealing with multi-material flow problems.

A Hybrid Immersed Interface Method for Driven Stokes Flow in an Elastic Tube

We present a hybrid numerical method for simulating fluid flow through a compliant,closed tube,driven by an internal source and sink.Fluid is assumed to be highly viscous with its motion described by Stokes flow.Model geometry is assumed to be axisymmetric,and the governing equations are implemented in axisymmetric cylindrical coordinates,which capture 3D flow dynamics with only 2D computations.We solve the model equations using a hybrid approach:we decompose the pressure and velocity fields into parts due to the surface forcings and due to the source and sink,with each part handled separately by means of an appropriate method.Because the singularly-supported surface forcings yield an unsmooth solution,that part of the solution is computed using the immersed interface method.Jump conditions are derived for the axisymmetric cylindrical coordinates.The velocity due to the source and sink is calculated along the tubular surface using boundary integrals.Numerical results are presented that indicate second-order accuracy of the method.

Numerical studies of penetration problems by an improved particle method

A particle mapping transportation algorithm was proposed on the basis of the particle-in-cell method.The particles with rectangular influence domains were employed in the transportation algorithm to reduce the numerical fluctuations.Based on the error analysis in the process of particle motion computation,a prediction-correction algorithm was introduced to improve the computational accuracy.Furthermore,the performance of the particle mapping transportation method was evaluated by using the rotation,the slotted disk and the shear advection tests,and the results were compared with other interface reconstruction methods.Finally,the hemispherical projectile penetration into a steel target was numerically simulated.The results showed that the proposed method produced less numerical fluctuations and exhibited clear material interfaces,which indicated that it is accurate and effective.

Marker-Based,3-D Adaptive Cartesian Grid Method for Multiphase Flow Around Irregular Geometries

Computational simulations of multiphase flow are challenging because many practical applications require adequate resolution of not only interfacial physics associated with moving boundaries with possible topological changes,but also around three-dimensional,irregular solid geometries.In this paper,we highlight recent efforts made in simulating multiphase fluid dynamics around complex geometries,based on an Eulerian-Lagrangian framework.The approach uses two independent but related grid layouts to track the interfacial and solid boundary conditions,and is capable of capturing interfacial as well as multiphase dynamics.In particular,the stationary Cartesian grid with time dependent,local adaptive refinement is utilized to handle the computation of the transport equations,while the interface shape and movement are treated by marker-based triangulated surface meshes which freely move and interact with the Cartesian grid.The markers are also used to identify the location of solid boundaries and enforce the no-slip condition there.Issues related to the contact line treatment,topological changes of multiphase fronts during merger or breakup of objects,and necessary data structures and solution techniques are also highlighted.Selected test cases including spacecraft fuel tank flow management and liquid plug flow dynamics are presented.

A Moving-Mesh Finite Element Method and its Application to the Numerical Solution of Phase-Change Problems

A distributed Lagrangian moving-mesh finite element method is applied to problems involving changes of phase.The algorithm uses a distributed conservation principle to determine nodal mesh velocities,which are then used to move the nodes.The nodal values are obtained from an ALE(Arbitrary Lagrangian-Eulerian)equation,which represents a generalization of the original algorithm presented in Applied Numerical Mathematics,54:450–469(2005).Having described the details of the generalized algorithm it is validated on two test cases from the original paper and is then applied to one-phase and,for the first time,two-phase Stefan problems in one and two space dimensions,paying particular attention to the implementation of the interface boundary conditions.Results are presented to demonstrate the accuracy and the effectiveness of the method,including comparisons against analytical solutions where available.