In recent years,the demand for real-time data processing has been increasing,and various stream processing systems have emerged.When the amount of data input to the stream processing system fluctuates,the computing re...In recent years,the demand for real-time data processing has been increasing,and various stream processing systems have emerged.When the amount of data input to the stream processing system fluctuates,the computing resources required by the stream processing job will also change.The resources used by stream processing jobs need to be adjusted according to load changes,avoiding the waste of computing resources.At present,existing works adjust stream processing jobs based on the assumption that there is a linear relationship between the operator parallelism and operator resource consumption(e.g.,throughput),which makes a significant deviation when the operator parallelism increases.This paper proposes a nonlinear model to represent operator performance.We divide the operator performance into three stages,the Non-competition stage,the Non-full competition stage,and the Full competition stage.Using our proposed performance model,given the parallelism of the operator,we can accurately predict the CPU utilization and operator throughput.Evaluated with actual experiments,the prediction error of our model is below 5%.We also propose a quick accurate auto-scaling(QAAS)method that uses the operator performance model to implement the auto-scaling of the operator parallelism of the Flink job.Compared to previous work,QAAS is able to maintain stable job performance under load changes,minimizing the number of job adjustments and reducing data backlogs by 50%.展开更多
Virtualization is the key technology of cloud computing. Network virtualization plays an important role in this field. Its performance is very relevant to network virtualizing. Nowadays its implementations are mainly ...Virtualization is the key technology of cloud computing. Network virtualization plays an important role in this field. Its performance is very relevant to network virtualizing. Nowadays its implementations are mainly based on the idea of Software Define Network (SDN). Open vSwitch is a sort of software virtual switch, which conforms to the OpenFlow protocol standard. It is basically deployed in the Linux kernel hypervisor. This leads to its performance relatively poor because of the limited system resource. In turn, the packet process throughput is very low.In this paper, we present a Cavium-based Open vSwitch implementation. The Cavium platform features with multi cores and couples of hard ac-celerators. It supports zero-copy of packets and handles packet more quickly. We also carry some experiments on the platform. It indicates that we can use it in the enterprise network or campus network as convergence layer and core layer device.展开更多
Integral to the urban ecosystem,greening trees provide many ecological benefits,but the active biogenic volatile organic compounds(BVOCs)they release contribute to the production of ozone and secondary organic aerosol...Integral to the urban ecosystem,greening trees provide many ecological benefits,but the active biogenic volatile organic compounds(BVOCs)they release contribute to the production of ozone and secondary organic aerosols,which harm ambient air quality.It is,therefore,necessary to understand the BVOC emission characteristics of dominant greening tree species and their relative contribution to secondary pollutants in various urban contexts.Consequently,this study utilized a dynamic enclosure system to collect BVOC samples of seven dominant greening tree species in urban Chengdu,Southwest China.Gas chromatography/mass spectrometry was used to analyze the BVOC components and standardized BVOC emission rates of each tree species were then calculated to assess their relative potential to form secondary pollutants.We found obvious differences in the composition of BVOCs emitted by each species.Ficus virens displayed a high isoprene emission rate at31.472μgC/(gdw(g dry weight)·hr),while Cinnamomum camphora emitted high volumes of D-Limonene at 93.574μgC/(gdw·hr).In terms of the BVOC emission rates by leaf area,C.camphora had the highest emission rate of total BVOCs at 13,782.59μgC/(m^(2)·hr),followed by Cedrus deodara with 5466.86μgC/(m^(2)·hr).Ginkgo biloba and Osmanthus fragrans mainly emitted oxygenated VOCs with lower overall emission rates.The high BVOC emitters like F.virens,C.camphora,and Magnolia grandiflora have high potential for significantly contributing to environmental secondary pollutants,so should be cautiously considered for future planting.This study provides important implications for improving urban greening efforts for subtropical Chinese urban contexts,like Chengdu.展开更多
The data stream processing framework processes the stream data based on event-time to ensure that the request can be responded to in real-time.In reality,streaming data usually arrives out-of-order due to factors such...The data stream processing framework processes the stream data based on event-time to ensure that the request can be responded to in real-time.In reality,streaming data usually arrives out-of-order due to factors such as network delay.The data stream processing framework commonly adopts the watermark mechanism to address the data disorderedness.Watermark is a special kind of data inserted into the data stream with a timestamp,which helps the framework to decide whether the data received is late and thus be discarded.Traditional watermark generation strategies are periodic;they cannot dynamically adjust the watermark distribution to balance the responsiveness and accuracy.This paper proposes an adaptive watermark generation mechanism based on the time series prediction model to address the above limitation.This mechanism dynamically adjusts the frequency and timing of watermark distribution using the disordered data ratio and other lateness properties of the data stream to improve the system responsiveness while ensuring acceptable result accuracy.We implement the proposed mechanism on top of Flink and evaluate it with realworld datasets.The experiment results show that our mechanism is superior to the existing watermark distribution strategies in terms of both system responsiveness and result accuracy.展开更多
基金supported by the National Key Research and Development Program of China(2020YFB1506703)the National Natural Science Foundation of China(Grant No.62072018)+1 种基金the State Key Laboratory of Software Development Environment(SKLSDE-2021ZX-06)the Fundamental Research Funds for the Central Universities.
文摘In recent years,the demand for real-time data processing has been increasing,and various stream processing systems have emerged.When the amount of data input to the stream processing system fluctuates,the computing resources required by the stream processing job will also change.The resources used by stream processing jobs need to be adjusted according to load changes,avoiding the waste of computing resources.At present,existing works adjust stream processing jobs based on the assumption that there is a linear relationship between the operator parallelism and operator resource consumption(e.g.,throughput),which makes a significant deviation when the operator parallelism increases.This paper proposes a nonlinear model to represent operator performance.We divide the operator performance into three stages,the Non-competition stage,the Non-full competition stage,and the Full competition stage.Using our proposed performance model,given the parallelism of the operator,we can accurately predict the CPU utilization and operator throughput.Evaluated with actual experiments,the prediction error of our model is below 5%.We also propose a quick accurate auto-scaling(QAAS)method that uses the operator performance model to implement the auto-scaling of the operator parallelism of the Flink job.Compared to previous work,QAAS is able to maintain stable job performance under load changes,minimizing the number of job adjustments and reducing data backlogs by 50%.
文摘Virtualization is the key technology of cloud computing. Network virtualization plays an important role in this field. Its performance is very relevant to network virtualizing. Nowadays its implementations are mainly based on the idea of Software Define Network (SDN). Open vSwitch is a sort of software virtual switch, which conforms to the OpenFlow protocol standard. It is basically deployed in the Linux kernel hypervisor. This leads to its performance relatively poor because of the limited system resource. In turn, the packet process throughput is very low.In this paper, we present a Cavium-based Open vSwitch implementation. The Cavium platform features with multi cores and couples of hard ac-celerators. It supports zero-copy of packets and handles packet more quickly. We also carry some experiments on the platform. It indicates that we can use it in the enterprise network or campus network as convergence layer and core layer device.
基金supported by the National Natural Science Foundation of China(No.21906108)the Fundamental Research Funds for the Central Universities(No.YJ201937)+1 种基金Chengdu Science and Technology Bureau(No.2020-YF09-00051-SN)the Sichuan"1000 Plan"Scholar Program
文摘Integral to the urban ecosystem,greening trees provide many ecological benefits,but the active biogenic volatile organic compounds(BVOCs)they release contribute to the production of ozone and secondary organic aerosols,which harm ambient air quality.It is,therefore,necessary to understand the BVOC emission characteristics of dominant greening tree species and their relative contribution to secondary pollutants in various urban contexts.Consequently,this study utilized a dynamic enclosure system to collect BVOC samples of seven dominant greening tree species in urban Chengdu,Southwest China.Gas chromatography/mass spectrometry was used to analyze the BVOC components and standardized BVOC emission rates of each tree species were then calculated to assess their relative potential to form secondary pollutants.We found obvious differences in the composition of BVOCs emitted by each species.Ficus virens displayed a high isoprene emission rate at31.472μgC/(gdw(g dry weight)·hr),while Cinnamomum camphora emitted high volumes of D-Limonene at 93.574μgC/(gdw·hr).In terms of the BVOC emission rates by leaf area,C.camphora had the highest emission rate of total BVOCs at 13,782.59μgC/(m^(2)·hr),followed by Cedrus deodara with 5466.86μgC/(m^(2)·hr).Ginkgo biloba and Osmanthus fragrans mainly emitted oxygenated VOCs with lower overall emission rates.The high BVOC emitters like F.virens,C.camphora,and Magnolia grandiflora have high potential for significantly contributing to environmental secondary pollutants,so should be cautiously considered for future planting.This study provides important implications for improving urban greening efforts for subtropical Chinese urban contexts,like Chengdu.
基金This work was supported by National Key Research and Development Program of China(2020YFB1506703)the National Natural Science Foundation of China(Grant No.62072018).
文摘The data stream processing framework processes the stream data based on event-time to ensure that the request can be responded to in real-time.In reality,streaming data usually arrives out-of-order due to factors such as network delay.The data stream processing framework commonly adopts the watermark mechanism to address the data disorderedness.Watermark is a special kind of data inserted into the data stream with a timestamp,which helps the framework to decide whether the data received is late and thus be discarded.Traditional watermark generation strategies are periodic;they cannot dynamically adjust the watermark distribution to balance the responsiveness and accuracy.This paper proposes an adaptive watermark generation mechanism based on the time series prediction model to address the above limitation.This mechanism dynamically adjusts the frequency and timing of watermark distribution using the disordered data ratio and other lateness properties of the data stream to improve the system responsiveness while ensuring acceptable result accuracy.We implement the proposed mechanism on top of Flink and evaluate it with realworld datasets.The experiment results show that our mechanism is superior to the existing watermark distribution strategies in terms of both system responsiveness and result accuracy.