应用程序属性感知的Yarn资源调度模型研究

Hadoop应用程序存在计算密集属性和调度时间属性,但是Hadoop大数据平台集成的第二代资源管理器Yarn内置的三种资源调度器无法将相同属性的应用程序均衡分配到计算节点上,导致部分节点负载过高,出现严重的计算任务长尾效应。文中提出了一种应用程序属性感知的Yarn负载均衡调度模型——APB Scheduler。APB Scheduler自动感知应用程序属性,将相同属性应用程序的Container按照动态资源计划均衡分配到集群计算节点上,并使用NSGA-Ⅲ算法完成最优分配方案计算。实验结果表明,APB Scheduler解决了相同属性应用程序的Container分配倾斜问题,大幅提升了集群的性能和稳定性。

Scalable Core–Sheath Yarn for Boosting Solar Interfacial Desalination Through Engineering Controllable Water Supply

Tailoring water supply to achieve confined heating has proven to be an effective strategy for boosting solar interfacial evaporation rates.However,because of salt clogging during desalination,a critical point of constriction occurs when controlling the water rate for confined heating.In this study,we demonstrate a facile and scalable weaving technique for fabricating core-sheath photothermal yarns that facilitate controlled water supply for stable and efficient interracial solar desalination.The core-sheath yarn comprises modal fibers as the core and carbon fibers as the sheaths.Because of the core-sheath design,remarkable liquid pumping can be enabled in the carbon fiber bundle of the dispersed superhydrophilic modal fibers.Our woven fabrics absorb a high proportion(92%)of the electromagnetic radiation in the solar spectrum because of the weaving structure and the carbon fiber sheath.Under one-sun(1 kW·m^(-2))illumination,our woven fabric device can achieve the highest evaporation rate(of 2.12kg·m^(-2)·h^(-1) with energy conversion efficiency:93.7%)by regulating the number of core-sheath yarns.Practical application tests demonstrate that our device can maintain high and stable desalination performance in a 5 wt%NaCl solution.

The Influence of Elimination of Roving Frame on Ring Spun Cotton Yarn Characteristics

Generally, ring spun yarns are manufactured from roving which is produced by roving frame. In this paper, an experiment has been done producing ring spun cotton yarn directly from finisher drawn sliver eliminating the roving frame. Total 3 types of yarn with the various linear density of 8 Ne, 10 Ne & 12 Ne were produced using a roving frame and without using a roving frame. In the next step, physical and mechanical properties of those yarns including unevenness, imperfections, hairiness & tenacity were investigated. The result showed that ring spun cotton yarns produced from sliver exhibited inferior physical and mechanical properties compared with samples from the conventional ring spinning system.

Dual-Ion Co-Regulation System Enabling High-Performance Electrochemical Artificial Yarn Muscles with Energy-Free Catch States

Artificial yarn muscles show great potential in applications requiring low-energy consumption while maintaining high performance. However, conventional designs have been limited by weak ion-yarn muscle interactions and inefficient “rocking-chair” ion migration. To address these limitations, we present an electrochemical artificial yarn muscle design driven by a dual-ion co-regulation system. By utilizing two reaction channels, this system shortens ion migration pathways, leading to faster and more efficient actuation. During the charging/discharging process, PF_6~- ions react with carbon nanotube yarn, while Li~+ ions react with an Al foil. The intercalation reaction between PF_6~- and collapsed carbon nanotubes allows the yarn muscle to achieve an energy-free high-tension catch state. The dual-ion coordinated yarn muscles exhibit superior contractile stroke, maximum contractile rate, and maximum power densities, exceeding those of “rocking-chair” type ion migration yarn muscles. The dual-ion co-regulation system enhances the ion migration rate during actuation, resulting in improved performance. Moreover, the yarn muscles can withstand high levels of isometric stress, displaying a stress of 61 times that of skeletal muscles and 8 times that of “rocking-chair” type yarn muscles at higher frequencies. This technology holds significant potential for various applications, including prosthetics and robotics.

Yarn Quality Prediction for Small Samples Based on AdaBoost Algorithm

In order to solve the problems of weak prediction stability and generalization ability of a neural network algorithm model in the yarn quality prediction research for small samples,a prediction model based on an AdaBoost algorithm(AdaBoost model) was established.A prediction model based on a linear regression algorithm(LR model) and a prediction model based on a multi-layer perceptron neural network algorithm(MLP model) were established for comparison.The prediction experiments of the yarn evenness and the yarn strength were implemented.Determination coefficients and prediction errors were used to evaluate the prediction accuracy of these models,and the K-fold cross validation was used to evaluate the generalization ability of these models.In the prediction experiments,the determination coefficient of the yarn evenness prediction result of the AdaBoost model is 76% and 87% higher than that of the LR model and the MLP model,respectively.The determination coefficient of the yarn strength prediction result of the AdaBoost model is slightly higher than that of the other two models.Considering that the yarn evenness dataset has a weaker linear relationship with the cotton dataset than that of the yarn strength dataset in this paper,the AdaBoost model has the best adaptability for the nonlinear dataset among the three models.In addition,the AdaBoost model shows generally better results in the cross-validation experiments and the series of prediction experiments at eight different training set sample sizes.It is proved that the AdaBoost model not only has good prediction accuracy but also has good prediction stability and generalization ability for small samples.

Are yarn quality prediction tools useful in the breeding of high yielding and better fibre quality cotton(Gossypium hirsutum L.)?

Results The population had large variations for lint yield,fibre properties,predicted yarn properties,and composite fibre quality values.Lint yield with all fibre quality traits was not correlated.When the selection was conducted first to keep those with improved fibre quality,and followed for high yields,a large proportion in the resultant populations was the same between selections based on Cottonspec predicted yarn quality and HVI-measured fibre properties.They both exceeded the selection based on FQI and Background The approach of directly testing yarn quality to define fibre quality breeding objectives and progress the selection is attractive but difficult when considering the need for time and labour.The question remains whether yarn prediction tools from textile research can serve as an alternative.In this study,using a dataset from three seasons of field testing recombinant inbred line population,Cottonspec,a software developed by the Commonwealth Scientific and Industrial Research Organisation(CSIRO)for predicting ring spun yarn quality from fibre properties measured by High Volume Instrument(HVI),was used to select improved fibre quality and lint yield in the population.The population was derived from an advanced generation inter-crossing of four CSIRO conventional commercial varieties.The Cottonspec program was able to provide an integrated index of the fibre qualities affecting yarn properties.That was compared with selection based on HVI-measured fibre properties,and two composite fibre quality variables,namely,fibre quality index(FQI),and premium and discount(PD)points.The latter represents the net points of fibre length,strength,and micronaire based on the Premiums and Discounts Schedule used in the market while modified by the inclusion of elongation.PD points.Conclusions The population contained elite segregants with improved yield and fibre properties,and Cottonspec predicted yarn quality is useful to effectively capture these elites.There is a need to further develop yarn quality prediction tools through collaborative efforts with textile mills,to draw better connectedness between fibre and yarn quality.This connection will support the entire cotton value chain research and evolution.

Performance Improvement through Novel Adaptive Node and Container Aware Scheduler with Resource Availability Control in Hadoop YARN

The default scheduler of Apache Hadoop demonstrates operational inefficiencies when connecting external sources and processing transformation jobs.This paper has proposed a novel scheduler for enhancement of the performance of the Hadoop Yet Another Resource Negotiator(YARN)scheduler,called the Adaptive Node and Container Aware Scheduler(ANACRAC),that aligns cluster resources to the demands of the applications in the real world.The approach performs to leverage the user-provided configurations as a unique design to apportion nodes,or containers within the nodes,to application thresholds.Additionally,it provides the flexibility to the applications for selecting and choosing which node’s resources they want to manage and adds limits to prevent threshold breaches by adding additional jobs as needed.Node or container awareness can be utilized individually or in combination to increase efficiency.On top of this,the resource availability within the node and containers can also be investigated.This paper also focuses on the elasticity of the containers and self-adaptiveness depending on the job type.The results proved that 15%–20%performance improvement was achieved compared with the node and container awareness feature of the ANACRAC.It has been validated that this ANACRAC scheduler demonstrates a 70%–90%performance improvement compared with the default Fair scheduler.Experimental results also demonstrated the success of the enhancement and a performance improvement in the range of 60%to 200%when applications were connected with external interfaces and high workloads.

Detection of Residual Yarn in Bobbin Based on Odd Partial Gabor Filter and Multi-Color Space Hierarchical Clustering

In an automatic bobbin management system that simultaneously detects bobbin color and residual yarn,a composite texture segmentation and recognition operation based on an odd partial Gabor filter and multi-color space hierarchical clustering are proposed.Firstly,the parameter-optimized odd partial Gabor filter is used to distinguish bobbin and yarn texture,to explore Garbor parameters for yarn bobbins,and to accurately discriminate frequency characteristics of yarns and texture.Secondly,multi-color clustering segmentation using color spaces such as red,green,blue(RGB)and CIELUV(LUV)solves the problems of over-segmentation and segmentation errors,which are caused by the difficulty of accurately representing the complex and variable color information of yarns in a single-color space and the low contrast between the target and background.Finally,the segmented bobbin is combined with the odd partial Gabor’s edge recognition operator to further distinguish bobbin texture from yarn texture and locate the position and size of the residual yarn.Experimental results show that the method is robust in identifying complex texture,damaged and dyed bobbins,and multi-color yarns.Residual yarn identification can distinguish texture features and residual yarns well and it can be transferred to the detection and differentiation of complex texture,which is significantly better than traditional methods.

Study on the Presence of Filament Yarn in Jamdani Saree in Bangladesh

Jamdani weaving is one of the oldest heredities of Bangladesh. From the beginning 100% cotton yarn was used to produce high quality jamdani saree. The weavers were the finest with weaving skills. Higher yarn count yarns were used to weave the jamdani saree. In course of time at present manmade fibres are also used to produce jamdani saree. The use of filament yarn may have eased the manufacturing difficulties, but the jamdani saree is missing its originality without 100% cotton. In this project, random jamdani saree sample was collected to identify the fibre composition. Samples of filament were also collected from the manufacturer and tested. It was evident that instead of cotton yarn in warp and weft silk and polyester filament yarn were used.

Spark on Yarn模式的电信大数据处理平台

针对电信大数据处理系统中存在的问题,文章提出一种基于Spark on Yarn模型的SY-TPP。在SY-TPP平台上,应用Hadoop2.0 Yarn标准,并利用Spark分布式存储技术,将SY-TPP系统的数据在内存中进行集中处理。以分级聚类算法为案例,对SY-TPP平台的开发过程进行了详细的分析。实验结果表明,TPP平台上的GB级用户可以在半个工作日内完成数据处理,而32个实体节点的SYTPP系统的速度比相同配置下的Map Reduce平台提高了10.25倍。

基于YARN集群的计算加速部件扩展支持

以GPU和Intel MIC为代表的计算加速部件已在科学计算、图形图像处理等领域得到了广泛的应用,其在基于云平台的高性能计算及大数据处理等方向也具有广泛的应用前景.YARN是新一代Hadoop分布式计算框架,其对计算资源的分配调度主要针对CPU,缺少对计算加速部件的支持.在YARN中添加计算加速部件需要解决多个难点,分别是计算加速部件资源如何调度以及异构节点间如何共享问题、多个任务同时调用计算加速部件而引起的资源争用问题和集群中对计算加速部件的状态监控与管理问题.为了解决这些问题,提出了动态节点捆绑策略、流水线式的计算加速部件任务调度等,实现了YARN对计算加速部件的支持,并通过实验验证了其有效性.

基于YARN规范的智能电网大数据异常检测

文章描述了Hadoop早期版本在处理智能电网大数据上的不足,同时分析了YARN规范对Map-Reduce进行改进后对电网大数据处理的优点。文章详细讨论了YARNDPP平台中智能电网大数据处理的编码与实现及YARN-DPP的优势,并以IEEE 118节点的电网作为智能电网大数据异常检测的案例程序,对YARN-DPP平台的硬件环境与软件运行环境进行配置。实验结果表明,针对海量的智能电网状态安全大数据异常检测的程序,YARN-DPP平台具有较好的吞吐量与加速比,可以满足现代智能电网大数据异常检测的需求,在计算速度上比单机串行计算及Map-Reduce计算要快。

YARN平台上的并行主题标引算法

文档主题标引是当前个性化智能检索的重要前提,但面对大规模海量数据资源时,主题标引也成为性能瓶颈。当前在Map Reduce框架上设计实现的主题标引算法,通常存在启动任务耗时长,中间数据过多地进行磁盘IO等缺陷。为了解决此类问题,采用YARN(yet another resource negotiator)作为底层分布式资源管理平台,选择更加合适的计算框架来改善计算性能。针对文档主题标引算法计算步骤多、阶段性强的特点,选择有向无环图(directed acyclic graph,DAG)计算模型进行算法实现,避免不必要的作业拆分,从而减少中间结果的磁盘IO。另外,考虑到Map Reduce的排序策略耗时较多,而有些计算无需对结果排序,故可以改用基于Hash的数据归约策略来提高计算性能,但这又会带来随机读的问题。利用固态硬盘高速随机读的特性,设计相应的优化计算策略来解决随机读的问题。通过实验对比发现,以YARN为底层管理平台,在此基础上选择合适的计算框架并加以优化,可以有效改善分布式计算的性能。

Yarn架构下基于GA的Web日志挖掘技术

提出了一种面向TB级别日志文件挖掘需求的日志挖掘技术。采用MapReduce编程模型设计并实现了一种基于子种群联姻策略的GA,并部署到Yarn架构上,使Yarn架构与GA有效结合。在算法Partition阶段,采用Round-Robin策略代替原有的哈希方法,使各个子种群基因更加趋向均匀分布,增加了子种群收敛效率和结果准确性,同时均衡各个节点运行负载,提高了系统性能。经测试,应用该技术的挖掘结果平均准确度达到93%以上,效率提升接近33%。

Full‑Fiber Auxetic‑Interlaced Yarn Sensor for Sign‑Language Translation Glove Assisted by Artificial Neural Network

Yarn sensors have shown promising application prospects in wearable electronics owing to their shape adaptability, good flexibility, and weavability. However, it is still a critical challenge to develop simultaneously structure stable, fast response, body conformal, mechanical robust yarn sensor using full microfibers in an industrial-scalable manner. Herein, a full-fiber auxetic-interlaced yarn sensor(AIYS) with negative Poisson’s ratio is designed and fabricated using a continuous, mass-producible, structure-programmable, and low-cost spinning technology. Based on the unique microfiber interlaced architecture, AIYS simultaneously achieves a Poisson’s ratio of-1.5, a robust mechanical property(0.6 c N/dtex), and a fast train-resistance responsiveness(0.025 s), which enhances conformality with the human body and quickly transduce human joint bending and/or stretching into electrical signals. Moreover, AIYS shows good flexibility, washability, weavability, and high repeatability. Furtherly, with the AIYS array, an ultrafast full-letter sign-language translation glove is developed using artificial neural network. The sign-language translation glove achieves an accuracy of 99.8% for all letters of the English alphabet within a short time of 0.25 s. Furthermore, owing to excellent full letter-recognition ability, real-time translation of daily dialogues and complex sentences is also demonstrated. The smart glove exhibits a remarkable potential in eliminating the communication barriers between signers and non-signers.

Analysis on the Characterization of the Twist Insertion Level and Its Effect of the Compact and Ring Spun Yarns

In this paper, the difference of the twist insertion levels of the compact and ring spun yarns of the same count is analyzed based on the relationship of the yarn twist factor to the twist angle and the bulk density. It is proposed that the relationship of the twist angle, twist level, and the yarn diameter should be synthetically considered when evaluating the twist insertion level of the compact yarn. The twist angle of the compact yarn is smaller than that of the ring yarn with the same twist insertion level. This results from the ordered fiber arrangement and compact structure of the compact yarn. Experiment was conducted to verify the conclusions. It is also discovered that the selection of the twist factor of the compact yarn, which can be usually lower than that of the ring yarn by 10%-15%, can be determined based on the yarn tensile strength.

Radial Yarn Movement in Yarn Duct of Interlacer

In order to find the relations between the production performance of interlacer and its cross-sectional shapes of yarn duct,radial yarn movement in various yarn ducts of interlacer and its effect on performance of interlaced yarn were studied in this paper.Seven interlacers,which were different in the cross-sectional shape of yarn duct,were made.Yarn motion in yarn duct of interlace was observed with optical fibers and photo-sensors.Performance of the interlaced yarn was evaluated by the number and the strength of tangles.Experimental results disclose the relations between the yarn motions and the properties of the interlaced yarn.The interlacer with round yarn duct is capable of producing an interlaced yarn with bigger number of tangles,since it can make yarn transverse frequently the axis of air jet issuing from air jet nozzle.The interlacer with cornered shape of yarn duct makes yarn subjected to the action of compressed air for a longer time,which results in larger strength of tangles.

Prediction of Rotor Spun Yarn Strength Using Adaptive Neuro-fuzzy Inference System and Linear Multiple Regression Methods

This paper presents a comparison study of two models for predicting the strength of rotor spun cotton yarns from fiber properties. The adaptive neuro-fuzzy system inference （ANFIS） and Multiple Linear Regression models are used to predict the rotor spun yarn strength. Fiber properties and yam count are used as inputs to train the two models and the count-strength-product （CSP） was the targel. The predictive performances of the two models are estimated and compared. We found that the ANFIS has a better predictive power in comparison with linear multiple regression model. The impact of each fiber property is also illustrated.

Advances in Electrospun Nanofiber Yarns

In order to increase the application area of nanofibers,electrospun nanofiber yarns have drawn attention of many researchers around the globe.Once the production method of nanofiber yarn is mature enough to be universally accepted,many new gates of applications will open to the world.In this review,different electrospinning techniques of electrospun nanofiber yarns are divided into needle electrospinning and needleless electrospinning.Considering yarn twist as an important mechanism,needle electrospinning technique is further categorized into mechanical,electrical and field flow twisting methods.Moreover,parameters influencing the mechanical properties of electrospun nanofiber yarns are investigated.Methods of improving mechanical properties of nanofiber yarns are addressed,including hot-water-bath treatment,addition of carbon nanotubes(CNTs)and introducing regulators.Finally,applications of electrospun nanofiber yarns in different fields of smart textile and bioengineering are summed-up.In summary,challenges encountered in the industrialization of nanofiber yarns and future prospects are anticipated.

一种YARN和Spark框架的网格聚类方法

分布式计算为大数据的处理提供一种新的平台,能有效提升算法的执行速度。在DBSCAN算法基础上提出一种数据分网格算法,该算法将每个分区上的数据集划分成以Eps半径为边长的单元格数据块,将查找Eps邻域的范围缩小到数据对象的八个相邻单元格之内,从而提高查找Eps邻域的速度及聚类速度,具有较好的加速比和扩展率。同时还优化分区聚类合并方法。