基于ClickHouse的版本化数据迁移方法

针对ClickHouse进行数据迁移过程中存在的业务开发周期较长、数据空窗和集群性能下降的问题,提出了一种版本化的数据迁移方法。首先采用参数化配置模式提升开发人员在不同业务场景下的开发效率;其次,利用ClickHouse中的原生ATTACH方法在源数据表和目标表之间构建一张版本表,保证数据迁移过程对用户无感知;接着,通过数据预处理以及对集群状态的实时监控,选择负载最小的副本方法来减少集群负担;此外,还加入验数逻辑和分片级的数据回滚功能来保证数据准确性。在广泛使用的业务生产场景中进行亿级数据的迁移测试对比,结果表明,该方法优于市面上最先进的技术,在硬件设备相同的情况下数据迁移时间缩短90%以上。

京东零售基于ClickHouse的增量刷岗方法

刷岗是对某个最小存货单位(SKU)对应的岗位等维度字段发生变更,按照最新的SKU岗位等维度数据回溯、覆写的过程,刷岗这一行为在典型的电商企业(京东(JD)、淘宝)较为常见。针对京东零售业务场景下明细表和维表数据量庞大导致刷岗操作的运行时间过长的问题,提出一种基于ClickHouse的增量刷岗方法。首先,将维度表加载为ClickHouse字典表,采用明细表关联ClickHouse字典表的方法进行刷岗;其次,采用增量刷岗方法取代传统全量刷岗的形式,不仅提升刷岗效率,同时减少刷岗带来的集群资源消耗;最后,加入验数逻辑和并发控制机制保证数据准确性和集群的稳定。将该技术与传统的刷岗技术在实际业务生产场景中进行亿级数据的刷岗测试对比,实验结果表明,在硬件设备相同的情况下,提出的增量刷岗方法刷岗时间缩短80%,集群资源(CPU、内存)的使用减少50%,显著提高刷海量数据岗效率。

Gas desorption characteristics of the high-rank intact coal and fractured coal

The objective of this work is to study the gas desorption characteristics of the high-rank intact coal and fractured coal.The gas adsorption,mercury porosimetry and gas desorption experiments were carried out in this study.Then,the theories of thermodynamics,diffusion mechanism and desorption kinetics were used to estimate the gas desorption characteristics.The results of gas adsorption experiments show that the initial isosteric adsorption heat of the intact coal is greater than that of the fractured coal,indicating that the gas molecules desorb more easily from fractured coal than intact coal.Using the mercury porosimetry,we find that the diffusion channels of fractured coal are more developed than those of intact coal.The difficult diffusion form dominates in the intact coal during the gas diffusing,while the easy diffusion form dominates in the fractured coal.The results of gas desorption experiments show that the initial gas desorption volume and velocity of the fractured coal are both greater than those of the intact coal.Using the Fick diffusion law,the study calculates the gas diffusion coefficients of the intact coal and fractured coal.The diffusion coefficients of the fractured coal are 2 times and 10 times greater than those of the intact coal at the time of 0-120 and 0-10 min,respectively.

New double yield surface model for coarse granular materials incorporating nonlinear unified failure criterion

A new double-yield-sarface （DYS） model was developed to characterize the strength and deformation behaviors of coarse granular materials （CGMs）. Two kinds of deformation mechanisms, including the shear and compressive plastic deformation, were taken into account in this model, These two deformation mechanisms were described by the shear and compressive yield functions, respectively. The Lode angle dependent formulations of proposed model were deduced by incorporating a 3D nonlinear unified failure criterion. Some comparisons were presented between the numerical predictions of proposed model and test data of true triaxial tests on the modeled rockfills. The model predictions are in good agreement with the test data and capture the strain hardening and plastic volumetric dilation of CGMs. These findings verify the reasonability of current DYS model, and indicate that this model is well suited to reproduce the stress-strain-volume change behavior of CGMs in general.

Experimental and simulation study of nylon 6 solid–liquid extraction process

The solid–liquid extraction process of nylon 6 to eliminate small molecules, i.e., caprolactam(CL), cyclic dimers(CD) and cyclic trimers(CT), is investigated in detail by both batch extraction experiments and numerical simulations. In the batch extraction experiments, due to the small molecules attaching to the polymeric surface, the basic physical mechanism shifts from surface diffusion to internal diffusion as the extraction proceeded. The experimental data are well reproduced by a diffusion model consisting of two distinct steps, characterized as surface diffusion and internal diffusion. Furthermore, based on the established mass transfer mechanism and diffusion model of the two distinct steps, the equilibrium constants and internal diffusion coefficients of CL, CD and CT are acquired. An industrial countercurrent extraction tower is further simulated. It is found that the extraction efficiency of CL can be significantly improved by increasing the temperature at the bottom portion of the tower. The elimination of CD, which can be greatly promoted by a high-concentration CL-water solution, is controlled by mass transfer resistance, whereas the removal of CL is mainly affected by the equilibrium.

An experimental investigation on decoupling performance for a lateral axis micromachined gyroscope with varying environmental parameters

This paper carries out an experiment study of decoupling performance for a novel lateral axis micromachined gyroscope with varying environmental parameters. The non-ideal mathematical model for the coupling mechanism of the gyroscope is estab-lished through the gyro dynamic response matrix. The coupling components varying with ambient pressure and temperature induced by stiffness coupling, damping coupling and electrostatic force coupling are semi-analytically discussed. The overall coupling ratio is evaluated via experiments in the custom-built installation. The testing results show that the decoupling per-formance of the gyroscope is sensitive to the environmental parameters and all the non-ideal errors are determined as a function of ambient pressure and temperature. The coupling error varies from 0.05% to 0.25% within the pressure range of 100 Pa-100 kPa. The characteristics of coupling with temperature are measured from 20℃ to 100℃ with a variation from 0.35% to 0.41%. The results also indicate that within the range of measured ambient pressure and temperature, the minimum coupling ratio occurs at 100 Pa and room temperature. The overall performance of the gyroscope is tested under the pressure of about 2000 Pa and room temperature to achieve a relatively low coupling ratio. The scale factor is measured to be 7.8 mV （°）-1 s-1 with nonlinearity about 0.45% in the full-scale range of 600 （°） s-1. The short-term bias stability is approximately 0.06 （°）s-1 （1σ） for 20 min with noise equivalent angular rate evaluated to be 0.077 （°） s-1 Hz-1/2.

Two interannual variability modes of the Northwestern Pacific Subtropical Anticyclone in boreal summer

Using the reanalysis data and 20th century simulation of coupled model FGOALS_gl developed by LASG/IAP, we identified two distinct interannual modes of Northwestern Pacific Subtropical Anticyclone （NWPAC） by performing Empirical Orthogonal Function （EOF） analysis on 850 hPa wind field over the northwestern Pacific in summer. Based on the associated anoma- lous equatorial zonal wind, these two modes are termed as ＂Equatorial Easterly related Mode＂ （EEM） and ＂Equatorial Westerly related Mode＂ （EWM）, respectively. The formation mechanisms of these two modes are similar, whereas the maintenance mechanisms, dominant periods, and the relationships with ENSO are different. The EEM is associated with E1 Nifio decaying phase, with the anomalous anticyclone established in the preceding winter and persisted into summer through local positive air-sea feedback. By enhancing equatorial upwelling of subsurface cold water, EEM favors the transition of ENSO from E1 Nifio to La Nifia. The EWM is accompanied by the E1 Nifio events with long persistence, with the anomalous anticyclone formed in spring and strengthened in summer due to the warm Sea Surface Temperature anomalies （SSTA） forcing from the equatorial central-eastern Pacific. The model well reproduces the spatial patterns of these two modes, but fails to simulate the percentage variance accounted for by the two modes. In the NCEP reanalysis （model result）, EEM （EWM） appears as the first mode, which accounts for 35.6% （68.2%） of the total variance.

Discussion on research methods of bacterial resistant mutation mechanisms under selective culture——uncertainty analysis of data from the Luria-Delbrück fluctuation experiment

Whether bacterial drug-resistance is drug-induced or results from rapid propagation of random spontaneous mutations in the flora prior to exposure, remains a long-term key issue concerned and debated in both genetics and medicinal fields. In a pio-neering study, Luria and Delbruck exposed E. coli to T1 phage, to investigate whether the number of resistant colonies fol- lowed the Poisson distribution. They deduced that the development of resistant colonies is independent of phage presence. Similar results have since been obtained on solid medium containing antibacterial agents. Luria and Delbruck＇s conclusions were long considered a gold standard for analyzing drug resistance mutations. More recently, the concept of adaptive mutation has triggered controversy over this approach. Microbiological observation shows that, following exposure to drugs of various concentrations, drug-resistant cells emerge and multiply depending on the time course, and show a process function, incon-sistent with the definition of Poisson distribution （which assumes not only that resistance is independent of drug quantity but follows no specific time course）. At the same time, since cells tend to aggregate after division rather than separating, colonies growing on drug plates arise from the multiplication of resistant bacteria cells of various initial population sizes. Thus, statisti-cal analysis based on equivalence of initial populations will yield erroneous results. In this paper, 310 data from the Lu- ria-DelbNck fluctuation experiment were reanalyzed from this perspective. In most cases, a high-end abnormal value, resulting from the non-synchronous variation of the two above-mentioned time variables, was observed. Therefore, the mean value cannot be regarded as an unbiased expectation estimate. The ratio between mean value and variance was similarly incompara-ble, because two different sampling methods were used. In fact, the Luria-Delbrtick data appear to follow an aggregated, rather than Poisson distribution. In stmnnary, the statistical analysis of Luria and Delbruck is insufficient to describe rules of resistant mutant development and multiplication. Correction of this historical misunderstanding will enable new insight into bacterial resistance mechanisms.