Multi-camera calibration method based on minimizing the difference of reprojection error vectors

In order to achieve a high precision in three-dimensional（3D） multi-camera measurement system, an efficient multi-cameracalibration method is proposed. A stitching method of large scalecalibration targets is deduced, and a fundamental of multi-cameracalibration based on the large scale calibration target is provided.To avoid the shortcomings of the method, the vector differencesof reprojection error with the presence of the constraint conditionof the constant rigid body transformation is modelled, and mini-mized by the Levenberg-Marquardt （LM） method. Results of thesimulation and observation data calibration experiment show thatthe accuracy of the system calibrated by the proposed methodreaches 2 mm when measuring distance section of 20 000 mmand scale section of 7 000 mm × 7 000 mm. Consequently, theproposed method of multi-camera calibration performs better thanthe fundamental in stability. This technique offers a more uniformerror distribution for measuring large scale space.

Optimization of the forearm angle for arm wrestling using multi-camera stereo digital image correlation: A preliminary study

This study analyzes the function of different muscles during arm wrestling and proposes a method to analyze the optimal forearm angle for professional arm wrestlers.We built a professional arm-wrestling platform to measure the shape and deformation of the skin at the biceps brachii of a volunteer in vivo during arm wrestling.We observed the banding phenomenon of arm skin strain during muscle contraction and developed a model to evaluate the moment provided by the biceps brachii.According to this model,the strain field of the area of interest on the skin was measured,and the forearm angles most favorable and unfavorable to the work of the biceps brachii were analyzed.This study demonstrates the considerable potential of applying DIC and its extension method to the in vivo measurement of human skin and facilitates the use of the in vivo measurement of skin deformation in various sports in the future.

New multi-camera calibration algorithm based on 1D objects

A new calibration algorithm for multi-camera systems using 1D calibration objects is proposed. The algorithm inte- grates the rank-4 factorization with Zhang (2004)'s method. The intrinsic parameters as well as the extrinsic parameters are re- covered by capturing with cameras the 1D object's rotations around a fixed point. The algorithm is based on factorization of the scaled measurement matrix, the projective depth of which is estimated in an analytical equation instead of a recursive form. For more than three points on a 1D object, the approach of our algorithm is to extend the scaled measurement matrix. The obtained parameters are finally refined through the maximum likelihood inference. Simulations and experiments with real images verify that the proposed technique achieves a good trade-off between the intrinsic and extrinsic camera parameters.

Object Matching Across Multiple Non-overlapping Fields of View Using Fuzzy Logic

An approach based on fuzzy logic for matching both articulated and non-articulated objects across multiple non-overlapping field of views (FoVs) from multiple cameras is proposed. We call it fuzzy logic matching algorithm (FLMA). The approach uses the information of object motion, shape and camera topology for matching objects across camera views. The motion and shape information of targets are obtained by tracking them using a combination of ConDensation and CAMShift tracking algorithms. The information of camera topology is obtained and used by calculating the projective transformation of each view with the common ground plane. The algorithm is suitable for tracking non-rigid objects with both linear and non-linear motion. We show videos of tracking objects across multiple cameras based on FLMA. From our experiments, the system is able to correctly match the targets across views with a high accuracy.

Hyper 360—Towards a Unified Tool Set Supporting Next Generation VR Film and TV Productions

We describe four fundamental challenges that complex real-life Virtual Reality (VR) productions are facing today (such as multi-camera management, quality control, automatic annotation with cinematography and 360˚?depth estimation) and describe an integrated solution, called Hyper 360, to address them. We demonstrate our solution and its evaluation in the context of practical productions and present related results.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions.Traditional tracking methods(e.g.,marking each animal with dye or surgically implanting microchips)can be invasive and may have an impact on the social behavior being measured.To overcome these shortcomings,video-based methods for tracking unmarked animals,such as fruit flies and zebrafish,have been developed.However,tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns.In this study,we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network(R-CNN)deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology.The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior.The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.

A unified multi-view multi-person tracking framework

Despite significant developments in 3D multi-view multi-person (3D MM) tracking, current frameworks separately target footprint tracking, or pose tracking. Frameworks designed for the former cannot be used for the latter, because they directly obtain 3D positions on the ground plane via a homography projection, which is inapplicable to 3D poses above the ground. In contrast, frameworks designed for pose tracking generally isolate multi-view and multi-frame associations and may not be sufficiently robust for footprint tracking, which utilizes fewer key points than pose tracking, weakening multi-view association cues in a single frame. This study presents a unified multi-view multi-person tracking framework to bridge the gap between footprint tracking and pose tracking. Without additional modifications, the framework can adopt monocular 2D bounding boxes and 2D poses as its input to produce robust 3D trajectories for multiple persons. Importantly, multi-frame and multi-view information are jointly employed to improve association and triangulation. Our framework is shown to provide state-of-the-art performance on the Campus and Shelf datasets for 3D pose tracking, with comparable results on the WILDTRACK and MMPTRACK datasets for 3D footprint tracking.

3D Surface Reconstruction of Coarse Aggregate Particles from Occlusion-Free Multi-View Images

Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.

Multi-camera systems for rehabilitation therapies:a study of the precision of Microsoft Kinect sensors

This paper seeks to determine how the overlap of several infrared beams affects the tracked position of the user, depending on the angle of incidence of light, distance to the target, distance between sensors, and the number of capture devices used. We also try to show that under ideal conditions using several Kinect sensors increases the precision of the data collected. The results obtained can be used in the design of telerehabilitation environments in which several RGB-D cameras are needed to improve precision or increase the tracking range. A numerical analysis of the results is included and comparisons are made with the results of other studies. Finally, we describe a system that implements intelligent methods for the rehabilitation of patients based on the results of the tests carried out.

Collaborative Tracking Method in Multi-Camera System

Visual tracking has been a popular task in computer vision in recent years,especially for long-term tracking.A novel object tracking framework is proposed in this paper.For surveillance cameras with overlapping areas,the target area is divided into several regions corresponding to each camera,and a simple re-matching method is used by matching the colors according to the segmented parts.For surveillance cameras without overlapping areas,a time estimation model is employed for continuously tracking objects in different fields of view(FoVs).A demonstration system for collaborative tracking in real time situation is realized finally.The experimental results show that compared with current popular algorithms,the proposed approach has good effect in accuracy and computation time for the application of continuously tracking the pedestrians.

Biological Eagle-eye Inspired Target Detection for Unmanned Aerial Vehicles Equipped with a Manipulator

Inspired by eagle eye mechanisms,the structure and information processing characteristics of the eagle′s visual system are used for the target capture task of an unmanned aerial vehicle(UAV)with a mechanical arm.In this paper,a novel eagle-eye inspired multi-camera sensor and a saliency detection method are proposed.A combined camera system is built by simulating the double fovea structure on the eagle retina.A saliency target detection method based on the eagle midbrain inhibition mechanism is proposed by measuring the static saliency information and dynamic features.Thus,salient targets can be accurately detected through the collaborative work between different cameras of the proposed multi-camera sensor.Experimental results show that the eagle-eye inspired visual system is able to continuously detect targets in outdoor scenes and that the proposed algorithm has a strong inhibitory effect on moving background interference.

Geometric Calibration Algorithm of Polarization Camera Using Planar Patterns

Polarization pattern provides additional information besides spectral signatures. It can be used in many applications, such as navigation, defect detection and object identification. A novel polarization camera composed of four synchronized cameras is proposed, and it can realize real-time polarization measurement. This study particularly concentrates on the geometric calibration of the system. The projection model is analyzed and the multi-camera calibration algorithm is proposed. Firstly, each camera is calibrated separately using planar patterns, and then the geometric calibration algorithms are performed. Due to the geometrical constraint, a global optimization method results in smaller estimation uncertainties. A mean rotation error of 0.025?and a mean translation error of 0.26 mm are achieved after geometric calibration. The images are rectified to establish a correspondence among cameras and are combined to acquire the polarization measurement. The polarization pattern of the skylight is measured by the system and the results are consistent with the previous studies.

Minimizing Resource Cost for Camera Stream Scheduling in Video Data Center

Video surveillance service, which receives live streams from IP cameras and forwards the streams to end users, has become one of the most popular services of video data center. The video data center focuses on minimizing the resource cost during resource provisioning for the service. However, little of the previous work comprehensively considers the bandwidth cost optimization of both upload and forwarding streams, and the capacity of the media server. In this paper, we propose an efficient resource scheduling approach for online multi-camera video forwarding, which tries to optimize the resource sharing of media servers and the networks together. Firstly, we not only provide a fine-grained resource usage model for media servers, but also evaluate the bandwidth cost of both upload and forwarding streams. Without loss of generality, we utilize two resource pricing models with different resource cost functions to evaluate the resource cost： the linear cost function and the non-linear cost functions. Then, we formulate the cost minimization problem as a constrained integer programming problem. For the linear resource cost function, the drift-plus-penalty optimization method is exploited in our approach. For non-linear resource cost functions, the approach employs a heuristic method to reduce both media server cost and bandwidth cost. The experimental results demonstrate that our approach obviously reduces the total resource costs on both media servers and networks simultaneously.