Systematic Review on Ground-Based Cloud Tracking Methods for Photovoltaics Nowcasting

Renewable energies are highly dependent on local weather conditions, with photovoltaic energy being particularly affected by intermittent clouds. Anticipating the impact of cloud shadows on power plants is crucial, as clouds can cause partial shading, excessive irradiation, and operational issues. This study focuses on analyzing cloud tracking methods for short-term forecasts, aiming to mitigate such impacts. We conducted a systematic literature review, highlighting the most significant articles on cloud tracking from ground-based observations. We explore both traditional image processing techniques and advances in deep learning models. Additionally, we discuss current challenges and future research directions in this rapidly evolving field, aiming to provide a comprehensive overview of the state of the art and identify opportunities for significant advancements in the next generation of cloud tracking systems based on computer vision and deep learning.

Advances in Deep-Learning-based Precipitation Nowcasting Techniques

Precipitation nowcasting,as a crucial component of weather forecasting,focuses on predicting very short-range precipitation,typically within six hours.This approach relies heavily on real-time observations rather than numerical weather models.The core concept involves the spatio-temporal extrapolation of current precipitation fields derived from ground radar echoes and/or satellite images,which was generally actualized by employing computer image or vision techniques.Recently,with stirring breakthroughs in artificial intelligence(AI)techniques,deep learning(DL)methods have been used as the basis for developing novel approaches to precipitation nowcasting.Notable progress has been obtained in recent years,manifesting the strong potential of DL-based nowcasting models for their advantages in both prediction accuracy and computational cost.This paper provides an overview of these precipitation nowcasting approaches,from which two stages along the advancing in this field emerge.Classic models that were established on an elementary neural network dominated in the first stage,while large meteorological models that were based on complex network architectures prevailed in the second.In particular,the nowcasting accuracy of such data-driven models has been greatly increased by imposing suitable physical constraints.The integration of AI models and physical models seems to be a promising way to improve precipitation nowcasting techniques further.

Application of Multi-Scale Tracking Radar Echoes Scheme in Quantitative Precipitation Nowcasting

A new radar echo tracking algorithm known as multi-scale tracking radar echoes by cross-correlation （MTREC） was developed in this study to analyze movements of radar echoes at different spatial scales. Movement of radar echoes, particularly associated with convective storms, exhibits different characteristics at various spatial scales as a result of complex interactions among meteorological systems leading to the formation of convective storms. For the null echo region, the usual correlation technique produces zero or a very small magnitude of motion vectors. To mitigate these constraints, MTREC uses the tracking radar echoes by correlation （TREC） technique with a large ＂box＂ to determine the systematic movement driven by steering wind, and MTREC applies the TREC technique with a small ＂box＂ to estimate small-scale internal motion vectors. Eventually, the MTREC vectors are obtained by synthesizing the systematic motion and the small-scale internal motion. Performance of the MTREC technique was compared with TREC technique using case studies： the Khanun typhoon on 11 September 2005 observed by Wenzhou radar and a squall-line system on 23 June 2011 detected by Beijing radar. The results demonstrate that more spatially smoothed and continuous vector fields can be generated by the MTREC technique, which leads to improvements in tracking the entire radar reflectivity pattern. The new multi-scMe tracking scheme was applied to study its impact on the performance of quantitative precipitation nowcasting. The location and intensity of heavy precipitation at a 1-h lead time was more consistent with quantitative precipitation estimates using radar and rain gauges.

Methods of Lightning Nowcasting Based on Radar Echo Extrapolation Technology

An improved echo extrapolation technology( MOD-COTREC) was introduced firstly,and then two plans for lightning nowcasting based on MOD-COTREC and both isothermal radar reflectivity and MOD-COTREC were proposed based on the technology. Afterwards,the two plans for lightning nowcasting were tested by a case respectively. It is concluded that during the process of lightning nowcasting singly based on MOD-COTREC,the appearance and disappearance of lightning are not considered,and only lightning position is predicted when lightning density is constant,so the prediction error is big. The plan for lightning nowcasting based on both isothermal radar reflectivity and MOD-COTREC is still at an experimental stage,and the nowcasting products of cloud-to-ground lightning based on the plan are very different from the actual density and position of cloud-to-ground lightning,so it needs to be improved further.

Lightning Nowcasting with an Algorithm of Thunderstorm Tracking Based on Lightning Location Data over the Beijing Area

A thunderstorm tracking algorithm is proposed to nowcast the possibility of lightning activity over an area of concern by using the total lightning data and neighborhood technique.The lightning radiation sources observed from the Beijing Lightning Network(BLNET)were used to obtain information about the thunderstorm cells,which are significantly valuable in real-time.The boundaries of thunderstorm cells were obtained through the neighborhood technique.After smoothing,these boundaries were used to track the movement of thunderstorms and then extrapolated to nowcast the lightning approaching in an area of concern.The algorithm can deliver creditable results prior to a thunderstorm arriving at the area of concern,with accuracies of 63%,80%,and 91%for lead times of 30,15,and 5 minutes,respectively.The real-time observations of total lightning appear to be significant for thunderstorm tracking and lightning nowcasting,as total lightning tracking could help to fill the observational gaps in radar reflectivity due to the attenuation by hills or other obstacles.The lightning data used in the algorithm performs well in tracking the active thunderstorm cells associated with lightning activities.

Development of typhoon driven wave nowcasting model in Southeast China Sea

Using optimal interpolation data assimilation of observed wave spectrum around Northeast coast of Taiwan Island, the typhoon driven wave nowcasting model in Southeast China Sea is setup. The SWAN (simulating waves nearshore) model is used to calculate wave field and the input wind field is the QSCAT/NCEP (Quick Scatterometer/National Centers for Environmental Prediction) data. The two-dimensional wavelet transform is applied to analyze the X-band radar image of nearshore wave field and it reveals that the observed wave spectrum has shoaling characteristics in frequency domain. The reverse calculation approach of wave spectrum in deep water is proposed and validated with experimental tests. The two-dimensional digital low-pass filter is used to obtain the initialization wave field. Wave data during Typhoon Sinlaku is used to calibrate the data assimilation parameters and test the reverse calculation approach. Data assimilation corrects the significant wave height and the low frequency spectra energy evidently at Beishuang Station along Fujian Province coast, where the entire assimilation indexes are positive in verification moments. The nowcasting wave field shows that the present model can obtain more accurate wave predictions for coastal and ocean engineering in Southeast China Sea.

Summary on Applications of Stratiform Clear Air Echo in Nowcasting

It was difficult to probe the clear air echo by the general traditional radar for echo's weak intensity.Therefore,its investigation was less because of the restrictions of probe technique and data.In recent years,with the probe tools improving,more clear air echoes were probed,and the relative investigations were more and more.However,most investigations stayed in the theory at present,and the relative literatures about its application in the practical forecast work were few.For a new generation of Doppler radars' powers and sensitivities were all high,they were put into service successively in China.People could observe more and more the clear air atmospheric echoes in the daily business.Its Doppler radar velocity provided the important basis for daily short-term predication and had very important indication meaning for the nowcasting of seasons which were spring,summer and fall.It was important to forecast the precipitation,especially the abrupt rainstorm by using the symptom of clear air echo which was probed by the new generation of Doppler radar products.Therefore,the advances on clear air echo research at home and abroad were summarized simply.

Evaluation of the Added Value of Probabilistic Nowcasting Ensemble Forecasts on Regional Ensemble Forecasts

Ensemble forecasting systems have become an important tool for estimating the uncertainties in initial conditions and model formulations and they are receiving increased attention from various applications.The Regional Ensemble Prediction System(REPS),which has operated at the Beijing Meteorological Service(BMS)since 2017,allows for probabilistic forecasts.However,it still suffers from systematic deficiencies during the first couple of forecast hours.This paper presents an integrated probabilistic nowcasting ensemble prediction system(NEPS)that is constructed by applying a mixed dynamicintegrated method.It essentially combines the uncertainty information(i.e.,ensemble variance)provided by the REPS with the nowcasting method provided by the rapid-refresh deterministic nowcasting prediction system(NPS)that has operated at the Beijing Meteorological Service(BMS)since 2019.The NEPS provides hourly updated analyses and probabilistic forecasts in the nowcasting and short range(0-6 h)with a spatial grid spacing of 500 m.It covers the three meteorological parameters:temperature,wind,and precipitation.The outcome of an evaluation experiment over the deterministic and probabilistic forecasts indicates that the NEPS outperforms the REPS and NPS in terms of surface weather variables.Analysis of two cases demonstrates the superior reliability of the NEPS and suggests that the NEPS gives more details about the spatial intensity and distribution of the meteorological parameters.

THE INFLUENCE OF CLOUD PARAMETERIZATION ADJUSTMENT USING REFLECTIVITY OF DOPPLER ON NOWCASTING WITH GRAPES MODEL

In this study, we attempted to improve the nowcasting of GRAPES model by adjusting the model initial field through modifying the cloud water, rain water and vapor as well as revising vapor-following rain water. The results show that the model nowcasting is improved when only the cloud water and rain water are adjusted or all of the cloud water, rain water and vapor are adjusted in the initial field. The forecasting of the former(latter) approach during 0-3(0-6) hours is significantly improved. Furthermore, for the forecast for 0-3 hours, the latter approach is better than the former. Compared with the forecasting results for which the vapor of the model initial field is adjusted by the background vapor with those by the revised vapor, the nowcasting of the revised vapor is much better than that of background vapor. Analysis of the reasons indicated that when the vapor is adjusted in the model initial field, especially when the saturated vapor is considered, the forecasting of the vapor field is significantly affected. The changed vapor field influences the circulation, which in turn improves the model forecasting of radar reflectivity and rainfall.

A modified stochastic model for LS+AR hybrid method and its application in polar motion short-term prediction

Short-term(up to 30 days)predictions of Earth Rotation Parameters(ERPs)such as Polar Motion(PM:PMX and PMY)play an essential role in real-time applications related to high-precision reference frame conversion.Currently,least squares(LS)+auto-regressive(AR)hybrid method is one of the main techniques of PM prediction.Besides,the weighted LS+AR hybrid method performs well for PM short-term prediction.However,the corresponding covariance information of LS fitting residuals deserves further exploration in the AR model.In this study,we have derived a modified stochastic model for the LS+AR hybrid method,namely the weighted LS+weighted AR hybrid method.By using the PM data products of IERS EOP 14 C04,the numerical results indicate that for PM short-term forecasting,the proposed weighted LS+weighted AR hybrid method shows an advantage over both the LS+AR hybrid method and the weighted LS+AR hybrid method.Compared to the mean absolute errors(MAEs)of PMX/PMY sho rt-term prediction of the LS+AR hybrid method and the weighted LS+AR hybrid method,the weighted LS+weighted AR hybrid method shows average improvements of 6.61%/12.08%and 0.24%/11.65%,respectively.Besides,for the slopes of the linear regression lines fitted to the errors of each method,the growth of the prediction error of the proposed method is slower than that of the other two methods.

Modelling the ZR Relationship of Precipitation Nowcasting Based on Deep Learning

Sudden precipitations may bring troubles or even huge harm to people’s daily lives.Hence a timely and accurate precipitation nowcasting is expected to be an indispensable part of our modern life.Traditionally,the rainfall intensity estimation from weather radar is based on the relationship between radar reflectivity factor(Z)and rainfall rate(R),which is typically estimated by location-dependent experiential formula and arguably uncertain.Therefore,in this paper,we propose a deep learning-based method to model the ZR relation.To evaluate,we conducted our experiment with the Shenzhen precipitation dataset.We proposed a combined method of deep learning and the ZR relationship,and compared it with a traditional ZR equation,a ZR equation with its parameters estimated by the least square method,and a pure deep learning model.The experimental results show that our combined model performsmuch better than the equation-based ZRformula and has the similar performance with a pure deep learning nowcasting model,both for all level precipitation and heavy ones only.

A Novel Method for Precipitation Nowcasting Based on ST-LSTM

Precipitation nowcasting is of great significance for severe convective weather warnings.Radar echo extrapolation is a commonly used precipitation nowcasting method.However,the traditional radar echo extrapolation methods are encountered with the dilemma of low prediction accuracy and extrapolation ambiguity.The reason is that those methods cannot retain important long-term information and fail to capture short-term motion information from the long-range data stream.In order to solve the above problems,we select the spatiotemporal long short-term memory(ST-LSTM)as the recurrent unit of the model and integrate the 3D convolution operation in it to strengthen the model’s ability to capture short-term motion information which plays a vital role in the prediction of radar echo motion trends.For the purpose of enhancing the model’s ability to retain long-term important information,we also introduce the channel attention mechanism to achieve this goal.In the experiment,the training and testing datasets are constructed using radar data of Shanghai,we compare our model with three benchmark models under the reflectance thresholds of 15 and 25.Experimental results demonstrate that the proposed model outperforms the three benchmark models in radar echo extrapolation task,which obtains a higher accuracy rate and improves the clarity of the extrapolated image.

Convective Storm VIL and Lightning Nowcasting Using Satellite and Weather Radar Measurements Based on Multi-Task Learning Models

Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.

Segmentation and Classification of Individual Clouds in Images Captured with Horizon-Aimed Cameras for Nowcasting of Solar Irradiance Absorption

One important aspect of solar energy generation especially in inter-tropical sites is the local variability of clouds. Satellite images do not have temporal resolution enough to nowcast its impacts on solar plants, this monitoring is made by local cameras. However, cloud detection and monitoring are not trivial due to cloud shape dynamics, the camera is a linear and self-adjusting device, with fish-eye lenses generating a flat image that distorts images near the horizon. The present work focuses on cloud identification to predict its effects on solar plants that are distinct for every site’s climatology and geography. We used RASPBERY-PI-based cameras pointed at the horizon to allow observation of clouds’ vertical distribution, not possible with a unique fish-eye lens. A large number of cloud image identification analyses led the researchers to use deep learning methods such as U-net, HRnet, and Detectron. We use transfer learning with weights trained over the “2012 ILSVRC ImageNet” data set and architecture configurations like Resnet, Efficient, and Detectron2. While cloud identification proved a difficult task, we achieved the best results by using Jaccard Coefficient as a validation metric, with the best model being a U-net with Resnet18 using 486 × 648 resolution. This model had an average IoU of 0.6, indicating a satisfactory performance in cloud segmentation. We also observed that the data imbalance affected the overall performance of all models, with the tree class creating a favorable bias. The HRNet model, which works with different resolutions, showed promising results with a more refined segmentation at the pixel level, but it was not necessary to detect the most predominant clouds in the sky. We are currently working on balancing the dataset and mapping out data augmentation transformations for our next experiments. Our ultimate goal is to use such models to predict cloud motion and forecast the impact it will have on solar power generation. The present work has contributed to a better understanding of what techniques work best for cloud identification and paves the way for future studies on the development of a better overall cloud classification model.

Predictive value of red blood cell distribution width and hematocrit for short-term outcomes and prognosis in colorectal cancer patients undergoing radical surgery

BACKGROUND Previous studies have reported that low hematocrit levels indicate poor survival in patients with ovarian cancer and cervical cancer,the prognostic value of hematocrit for colorectal cancer(CRC)patients has not been determined.The prognostic value of red blood cell distribution width(RDW)for CRC patients was controversial.AIM To investigate the impact of RDW and hematocrit on the short-term outcomes and long-term prognosis of CRC patients who underwent radical surgery.METHODS Patients who were diagnosed with CRC and underwent radical CRC resection between January 2011 and January 2020 at a single clinical center were included.The short-term outcomes,overall survival(OS)and disease-free survival(DFS)were compared among the different groups.Cox analysis was also conducted to identify independent risk factors for OS and DFS.RESULTS There were 4258 CRC patients who underwent radical surgery included in our study.A total of 1573 patients were in the lower RDW group and 2685 patients were in the higher RDW group.There were 2166 and 2092 patients in the higher hematocrit group and lower hematocrit group,respectively.Patients in the higher RDW group had more intraoperative blood loss(P<0.01)and more overall complications(P<0.01)than did those in the lower RDW group.Similarly,patients in the lower hematocrit group had more intraoperative blood loss(P=0.012),longer hospital stay(P=0.016)and overall complications(P<0.01)than did those in the higher hematocrit group.The higher RDW group had a worse OS and DFS than did the lower RDW group for tumor node metastasis(TNM)stage I(OS,P<0.05;DFS,P=0.001)and stage II(OS,P=0.004;DFS,P=0.01)than the lower RDW group;the lower hematocrit group had worse OS and DFS for TNM stage II(OS,P<0.05;DFS,P=0.001)and stage III(OS,P=0.001;DFS,P=0.001)than did the higher hematocrit group.Preoperative hematocrit was an independent risk factor for OS[P=0.017,hazard ratio(HR)=1.256,95%confidence interval(CI):1.041-1.515]and DFS(P=0.035,HR=1.194,95%CI:1.013-1.408).CONCLUSION A higher preoperative RDW and lower hematocrit were associated with more postoperative complications.However,only hematocrit was an independent risk factor for OS and DFS in CRC patients who underwent radical surgery,while RDW was not.

A convolutional recurrent neural network for strong convective rainfall nowcasting using weather radar data in Southeastern Brazil

Strong convective systems and the associated heavy rainfall events can trig-ger floods and landslides with severe detrimental consequences.These events have a high spatio-temporal variability,being difficult to predict by standard meteorological numerical models.This work proposes the M5Images method for performing the very short-term prediction(nowcasting)of heavy convective rainfall using weather radar data by means of a convolutional recurrent neural network.The recurrent part of it is a Long Short-Term Memory(LSTM)neural network.Prediction tests were performed for the city and surroundings of Campinas,located in the Southeastern Brazil.The convolutional recurrent neural network was trained using time series of rainfall rate images derived from weather radar data for a selected set of heavy rainfall events.The attained pre-diction performance was better than that given by the persistence forecasting method for different prediction times.

Short-Term Household Load Forecasting Based on Attention Mechanism and CNN-ICPSO-LSTM

Accurate load forecasting forms a crucial foundation for implementing household demand response plans andoptimizing load scheduling. When dealing with short-term load data characterized by substantial fluctuations,a single prediction model is hard to capture temporal features effectively, resulting in diminished predictionaccuracy. In this study, a hybrid deep learning framework that integrates attention mechanism, convolution neuralnetwork (CNN), improved chaotic particle swarm optimization (ICPSO), and long short-term memory (LSTM), isproposed for short-term household load forecasting. Firstly, the CNN model is employed to extract features fromthe original data, enhancing the quality of data features. Subsequently, the moving average method is used for datapreprocessing, followed by the application of the LSTM network to predict the processed data. Moreover, the ICPSOalgorithm is introduced to optimize the parameters of LSTM, aimed at boosting the model’s running speed andaccuracy. Finally, the attention mechanism is employed to optimize the output value of LSTM, effectively addressinginformation loss in LSTM induced by lengthy sequences and further elevating prediction accuracy. According tothe numerical analysis, the accuracy and effectiveness of the proposed hybrid model have been verified. It canexplore data features adeptly, achieving superior prediction accuracy compared to other forecasting methods forthe household load exhibiting significant fluctuations across different seasons.

Modeling injection-induced fault slip using long short-term memory networks

Stress changes due to changes in fluid pressure and temperature in a faulted formation may lead to the opening/shearing of the fault.This can be due to subsurface(geo)engineering activities such as fluid injections and geologic disposal of nuclear waste.Such activities are expected to rise in the future making it necessary to assess their short-and long-term safety.Here,a new machine learning(ML)approach to model pore pressure and fault displacements in response to high-pressure fluid injection cycles is developed.The focus is on fault behavior near the injection borehole.To capture the temporal dependencies in the data,long short-term memory(LSTM)networks are utilized.To prevent error accumulation within the forecast window,four critical measures to train a robust LSTM model for predicting fault response are highlighted:(i)setting an appropriate value of LSTM lag,(ii)calibrating the LSTM cell dimension,(iii)learning rate reduction during weight optimization,and(iv)not adopting an independent injection cycle as a validation set.Several numerical experiments were conducted,which demonstrated that the ML model can capture peaks in pressure and associated fault displacement that accompany an increase in fluid injection.The model also captured the decay in pressure and displacement during the injection shut-in period.Further,the ability of an ML model to highlight key changes in fault hydromechanical activation processes was investigated,which shows that ML can be used to monitor risk of fault activation and leakage during high pressure fluid injections.

A Time Series Short-Term Prediction Method Based on Multi-Granularity Event Matching and Alignment

Accurate forecasting of time series is crucial across various domains.Many prediction tasks rely on effectively segmenting,matching,and time series data alignment.For instance,regardless of time series with the same granularity,segmenting them into different granularity events can effectively mitigate the impact of varying time scales on prediction accuracy.However,these events of varying granularity frequently intersect with each other,which may possess unequal durations.Even minor differences can result in significant errors when matching time series with future trends.Besides,directly using matched events but unaligned events as state vectors in machine learning-based prediction models can lead to insufficient prediction accuracy.Therefore,this paper proposes a short-term forecasting method for time series based on a multi-granularity event,MGE-SP(multi-granularity event-based short-termprediction).First,amethodological framework for MGE-SP established guides the implementation steps.The framework consists of three key steps,including multi-granularity event matching based on the LTF(latest time first)strategy,multi-granularity event alignment using a piecewise aggregate approximation based on the compression ratio,and a short-term prediction model based on XGBoost.The data from a nationwide online car-hailing service in China ensures the method’s reliability.The average RMSE(root mean square error)and MAE(mean absolute error)of the proposed method are 3.204 and 2.360,lower than the respective values of 4.056 and 3.101 obtained using theARIMA(autoregressive integratedmoving average)method,as well as the values of 4.278 and 2.994 obtained using k-means-SVR(support vector regression)method.The other experiment is conducted on stock data froma public data set.The proposed method achieved an average RMSE and MAE of 0.836 and 0.696,lower than the respective values of 1.019 and 0.844 obtained using the ARIMA method,as well as the values of 1.350 and 1.172 obtained using the k-means-SVR method.

Physics Guided Deep Learning-Based Model for Short-Term Origin–Destination Demand Prediction in Urban Rail Transit Systems Under Pandemic

Accurate origin–destination(OD)demand prediction is crucial for the efficient operation and management of urban rail transit(URT)systems,particularly during a pandemic.However,this task faces several limitations,including real-time availability,sparsity,and high-dimensionality issues,and the impact of the pandemic.Consequently,this study proposes a unified framework called the physics-guided adaptive graph spatial–temporal attention network(PAG-STAN)for metro OD demand prediction under pandemic conditions.Specifically,PAG-STAN introduces a real-time OD estimation module to estimate real-time complete OD demand matrices.Subsequently,a novel dynamic OD demand matrix compression module is proposed to generate dense real-time OD demand matrices.Thereafter,PAG-STAN leverages various heterogeneous data to learn the evolutionary trend of future OD ridership during the pandemic.Finally,a masked physics-guided loss function(MPG-loss function)incorporates the physical quantity information between the OD demand and inbound flow into the loss function to enhance model interpretability.PAG-STAN demonstrated favorable performance on two real-world metro OD demand datasets under the pandemic and conventional scenarios,highlighting its robustness and sensitivity for metro OD demand prediction.A series of ablation studies were conducted to verify the indispensability of each module in PAG-STAN.