Assessment of ECMWF’s Precipitation Forecasting Performance for China from 2017 to 2022

This study used the China Meteorological Administration(CMA)three-source fusion gridded precipitation analysis data as a reference to evaluate the precipitation forecast performance of the European Centre for Medium-Range Weather Forecasts(ECMWF)model for China from 2017 to 2022.The main conclusions are as follows.The precipitation forecast capability of the ECMWF model for China has gradually improved from 2017 to 2022.Various scores such as bias,equitable threat score(ETS),and Fractions Skill Score(FSS)showed improvements for different categories of precipitation.The bias of light rain forecasts overall adjusted towards smaller values,and the increase in forecast scores was greater in the warm season than in the cold season.The ETS for torrential rain more intense categories significantly increased,although there were large fluctuations in bias across different months.The model exhibited higher precipitation bias in most areas of North China,indicating overprediction,while it showed lower bias in South China,indicating underprediction.The ETSs indicate that the model performed better in forecasting precipitation in the northeastern part of China without the influence of climatic background conditions.Comparison of the differences between the first period and the second period of the forecast shows that the precipitation amplitude in the ECMWF forecast shifted from slight underestimation to overestimation compared to that of CMPAS05,reducing the likelihood of missing extreme precipitation events.The improvement in ETS is mainly due to the reduction in bias and false alarm rates and,more importantly,an increase in the hit rate.From 2017 to 2022,the area coverage error of model precipitation forecast relative to observations showed a decreasing trend at different scales,while the FSS showed an increasing trend,with the highest FSS observed in 2021.The ETS followed a parabolic trend with increasing neighborhood radius,with the better ETS neighborhood radius generally being larger for moderate rain and heavy rain compared with light rain and torrential rain events.

Verification of Subseasonal-to-Seasonal Forecasts for Major Stratospheric Sudden Warmings in Northern Winter from 1998/99 to 2012/13

This study reports verification results of hindcast data of four systems in the subseasonal-to-seasonal(S2S)prediction project for major stratospheric sudden warmings(MSSWs)in northern winter from 1998/99 to 2012/13.This report deals with average features across all MSSWs,and possible differences between two MSSW types(vortex displacement and split types).Results for the average features show that stratospheric forecast verifications,when further averaged among the four systems,are judged to be successful for lead times around 10 d or shorter.All systems are skillful for lead times around 5 d,whereas the results vary among the systems for longer lead times.A comparison between the MSSW types overall suggests larger forecast errors or lower skill for MSSWs of the vortex split type,although the differences do not have strong statistical significance for almost all cases.This limitation is likely to at least partly reflect the small sample size of the MSSWs available.

Western North Pacific Tropical Cyclone Intensity Guidance Evaluations Using an Alternative Verification Technique

In this study,six intensity forecast guidance techniques from the East China Regional Meteorological Center are verified for the 2008 and 2009 typhoon seasons through an alternative forecast verification technique.This technique is used to verify intensity forecasts if those forecasts call for a typhoon to dissipate or if the real typhoon dissipates.Using a contingency table,skill scores,chance,and probabilities are computed.It is shown that the skill of the six tropical cyclone intensity guidance techniques was highest for the 12-h forecasts,while the lowest skill of all the six models did not occur in 72-h forecasting.For both the 2008 and 2009 seasons,the average probabilities of the forecast intensity having a small error(6 m s-1) tended to decrease steadily.Some of the intensity forecasts had small skill scores,but the associated probabilities of the forecast intensity errors > 15 m s-1 were not the highest.

Long-term Prediction and Verification of Rainfall Based on the Seasonal Model

Using the seasonal cross-multiplication trend model, monthly precipitation of eight national basic weather stations of Shaanxi Province from 2005 to 2010 was predicted, and the forecast results were verified using the rainfall scoring rules of China Meteorological Administration. The verification results show that the average score of annual precipitation prediction in recent six years is higher than that made by a professional forecaster, so this model has a good prospect of application. Moreover, the level of making prediction is steady, and it can be widely used in long-term prediction of rainfall.

Evaluation of Tianji and ECMWF high-resolution precipitation forecasts for extreme rainfall event in Henan in July 2021

The extreme rainfall event of July 17 to 22, 2021 in Henan Province, China, led to severe urban waterlogging and flood disasters. This study investigated the performance of high-resolution weather forecasts in predicting this extreme event and the feasibility of weather forecast-based hydrological forecasts. To achieve this goal, high-resolution precipitation forecasts from the Tianji weather system and the forecast system of the European Centre for Medium-Range Weather Forecasts (ECMWF) were evaluated with the spatial verification metrics of structure, amplitude, and location. The results showed that Tianji weather forecasts accurately predicted the amplitude of 12-h accumulated precipitation with a lead time of 12 h. The location and structure of the rainfall areas in Tianji forecasts were closer to the observations than ECMWF forecasts. Tianji hourly precipitation forecasts were also more accurate than ECMWF hourly forecasts, especially at lead times shorter than 8 h. The precipitation forecasts were used as the inputs to a hydrological model to evaluate their hydrological applications. The results showed that the runoff forecasts driven by Tianji weather forecasts could effectively predict the extreme flood event. The runoff forecasts driven by Tianji forecasts were more accurate than those driven by ECMWF forecasts in terms of amplitude and location. This study demonstrates that high-resolution weather forecasts and corresponding hydrological forecasts can provide valuable information in advance for disaster warnings and leave time for people to act on the event. The results encourage further hydrological applications of high-resolution weather forecasts, such as Tianji weather forecasts, in the future.

A Review on Aspects of Climate Simulation Assessment

This paper reviews some aspects of evaluation of climate simulation, including the ITCZ, the surface air temperature （SAT）, and the monsoon. A brief introduction of some recently proposed approaches in weather forecast verification is followed by a discussion on their possible application to evaluation of climate simulation. The authors suggest five strategies to extend the forecast verification methods to climate simulation evaluation regardless significant differences between the forecasts and climate simulations. It is argued that resolution, convection scheme, stratocumulus cloud cover, among other processes in the atmospheric general circulation model （AGCM） and the ocean-atmosphere feedback are the potential causes for the double ITCZ problem in coupled models and AGCM simulations, based on the system- and component-level evaluations as well as the downscaling strategies in some recent research. Evaluations of simulated SAT and monsoons suggest that both coupled models and AGCMs show good performance in representing the SAT evolution and its variability over the past century in terms of correlation and wavelet analysis but poor at reproducing rainfall, and in addition, the AGCM alone is not suitable for monsoon regions due to the lack of air-sea interactions.

Evaluation of WRF-based Convection-Permitting Multi-Physics Ensemble Forecasts over China for an Extreme Rainfall Event on 21 July 2012 in Beijing

On 21 July 2012,an extreme rainfall event that recorded a maximum rainfall amount over 24 hours of 460 mm,occurred in Beijing,China. Most operational models failed to predict such an extreme amount. In this study,a convective-permitting ensemble forecast system（CEFS）,at 4-km grid spacing,covering the entire mainland of China,is applied to this extreme rainfall case. CEFS consists of 22 members and uses multiple physics parameterizations. For the event,the predicted maximum is 415 mm d^-1 in the probability-matched ensemble mean. The predicted high-probability heavy rain region is located in southwest Beijing,as was observed. Ensemble-based verification scores are then investigated. For a small verification domain covering Beijing and its surrounding areas,the precipitation rank histogram of CEFS is much flatter than that of a reference global ensemble. CEFS has a lower（higher） Brier score and a higher resolution than the global ensemble for precipitation,indicating more reliable probabilistic forecasting by CEFS. Additionally,forecasts of different ensemble members are compared and discussed. Most of the extreme rainfall comes from convection in the warm sector east of an approaching cold front. A few members of CEFS successfully reproduce such precipitation,and orographic lift of highly moist low-level flows with a significantly southeasterly component is suggested to have played important roles in producing the initial convection. Comparisons between good and bad forecast members indicate a strong sensitivity of the extreme rainfall to the mesoscale environmental conditions,and,to less of an extent,the model physics.

Statistically Extrapolated Nowcasting of Summertime Precipitation over the Eastern Alps

This paper presents a new multiple linear regression（MLR） approach to updating the hourly, extrapolated precipitation forecasts generated by the INCA（Integrated Nowcasting through Comprehensive Analysis） system for the Eastern Alps.The generalized form of the model approximates the updated precipitation forecast as a linear response to combinations of predictors selected through a backward elimination algorithm from a pool of predictors. The predictors comprise the raw output of the extrapolated precipitation forecast, the latest radar observations, the convective analysis, and the precipitation analysis. For every MLR model, bias and distribution correction procedures are designed to further correct the systematic regression errors. Applications of the MLR models to a verification dataset containing two months of qualified samples,and to one-month gridded data, are performed and evaluated. Generally, MLR yields slight, but definite, improvements in the intensity accuracy of forecasts during the late evening to morning period, and significantly improves the forecasts for large thresholds. The structure-amplitude-location scores, used to evaluate the performance of the MLR approach,based on its simulation of morphological features, indicate that MLR typically reduces the overestimation of amplitudes and generates similar horizontal structures in precipitation patterns and slightly degraded location forecasts, when compared with the extrapolated nowcasting.

THE PERFORMANCE OF GLOBAL MODELS IN TC TRACK FORECASTING OVER THE WESTERN NORTH PACIFIC FROM 2010 TO 2012

Forecasts of tropical cyclone(TC)tracks from six global models during 2010 and 2012 were assessed to study the current capability of track forecast guidance over the western North Pacific.To measure the performance of the global model forecasts,a series of statistical evaluations of track forecasts up to 120 h were carried out,including the mean,median,percentile distribution,regional distribution,relative position,correlation analysis,and binned analysis.Results showed that certain improvements have been made for the six global models in their prediction accuracy and stability in the past three years.Remarkably,stepped decreases in the values of each quantile were found at all lead time levels from 2010 to 2012 for NCEP-GFS.An analysis of the regional distribution of position errors showed that a high-latitude region,low-latitude region(which covers mostly the TC genesis region)and the South China Sea are the three main areas within which large errors tend to concentrate.The majority of the models show their own respective characteristics of systematic bias at each lead time,as established through the relative position analysis results.Only the results of NCEP-GFS and CMA-T639 did not show any obvious systematic bias in the three-year study period.Binned analyses indicated that the prediction accuracy and stability of most of the models were better for strong TCs than for weak TCs at short lead time levels.It was also found that the models tend to perform better for initially large TCs,or for those with weak vertical wind shear at lead times shorter than 48 h.The results demonstrate the heavy reliance of forecast errors upon the initial characteristics of a TC or its environmental conditions.

Verification of SPE probability forecasts at the Space Environment Prediction Center(SEPC)

In space weather forecasting, forecast verification is necessary so that the forecast quality can be assessed. This paper provides an example of how to choose and devise verification methods and techniques according to different space weather forecast products. Solar proton events(SPEs) are hazardous space weather events, and forecasting them is one of the major tasks of the Space Environment Prediction Center(SEPC) at the National Space Science Center of the Chinese Academy of Sciences. Through analyzing SPE occurrence characteristics, SPE forecast properties, and verification requirements at SEPC, verification methods for SPE probability forecasts are identified, and verification results obtained. Overall, SPE probability forecasts at SEPC exhibit good accuracy, reliability, and discrimination. Compared with climatology and persistence forecasts, the SPE forecasts are more accurate. However, the forecasts for SPE onset days are substantially underestimated and need to be considerably improved.

TROPICAL CYCLONE FORECAST VERIFICATION BY INDIA METEOROLOGICAL DEPA RTMENT FOR NORTH INDIAN OCEAN: A REVIEW

The tropical cyclone(TC)forecast verification procedure followed by India Meteorological Department(IMD)is reviewed herewith as compared to the standard prescribed by World Meteorological Organisation.The limitations in the present procedure and the future scope are presented and analysed.The IMD has considerably increased its efforts in recent years in the areas of TC forecasting and verification.Many facilities have been developed to enable a detailed assessment of the global model’s performance in the forecasting of TCs as well as operational TC forecast performance.Further development of these facilities is being undertaken by IMD.There are grey areas in TC forecast verification of IMD in the field of calculation of forecast error in surface wind radii in geographical quadrants,which is yet to start.Also there is need for improvement in verification of storm surge,wave height,coastal inundation forecasts and heavy rainfall forecasts by calculating the skill parameters and also by improving the analysis data required for verification of forecasts.Similarly there is need for verification of strike probability as well as dynamical cone of uncertainty as given by ensemble prediction system.

PERFORMANCE OF TROPICAL CYCLONE FORECAST IN WESTERN NORTH PACIFIC IN 2016

The forecasts of tropical cyclones(TC) in 2016 from 5 official guidances, 5 global models, 3 regional models and 6 ensemble systems were assessed to study the current capabilities of track and intensity forecasts for the western North Pacific. In 2016, the position errors for each official agency were under 85, 150 and 250 km at the lead times of 24, 48, and 72 h, respectively,indicating the performance of track forecasts was a little worse than that in 2015. For each lead time, decreases were seen for each quantile value of the global models from 2010 to 2015; however, this progress in forecasts was stagnated or was reversed in 2016, especially for long lead times.A new error tracking tool,called a "Track Error Rose",was used to visualize the spatial distributions of the track forecast error relative to the observed TC center. The results show that as lead time increases, the moving speed of most global model TC forecasts becomes slower than those of the observations, and the largest track error often appears to the south of the observation position. In 2016, JMA-GSM, NCEP-GFS, STI-GRAPES and UKMO-MetUM made considerable progress in their intensity forecasts at lead times of 24 and 48 h, and the EPS intensity forecasts made significant progress compared to those of 2015.

Rainfall forecast errors in different landfall stages of Super Typhoon Lekima (2019)

The rainfall forecast performance of the Tropical Cyclone(TC)version Model of Global and Regional Assimilation PrEdiction System(GRAPESTCM)of the China Meteorological Administration for landfalling Super Typhoon Lekima(2019)is studied by using the object-oriented verification method of contiguous rain area(CRA).The major error sources and possible reasons for the rainfall forecast uncertainties in different landfall stages(including near landfall and moving further inland)are compared.Results show that different performance and errors of rainfall forecast exist in the different TC stages.In the near landfall stage the asymmetric rainfall distribution is hard to be simulated,which might be related to the too strong forecasted TC intensity and too weak vertical wind shear accompanied.As Lekima moves further inland,the rain pattern and volume errors gradually increase.The Equitable Threat Score of the 24 h forecasted rainfall over 100 mm declines quickly with the time-length over land.The diagnostic analysis shows that there exists an interaction between the TC and the mid-latitude westerlies,but too weak frontogenesis is simulated.The results of this research indicate that for the current numerical model,the forecast ability of persistent heavy rainfall is very limited,especially when the weakened landing TC moves further inland.

EVALUATION OF TROPICAL CYCLONE FORECASTS FROM OPERATIONAL GLOBAL MODELS OVER THE WESTERN NORTH PACIFIC IN 2013

Tropical cyclones(TCs)forecasts from seven global models in 2013 were assessed to study the current capability of track and intensity forecast guidance over the western North Pacific.Analysis of along-and cross-track error revealed stepped decreases in the values of each quantile at each lead time level by showing the annual track error distribution from 2010 to 2013,particularly in the ECMWF-IFS,NCEP-GFS and UKMO-Met UM models.The TC propagation direction was much easier to handle for most of the global models;however,the propagation speed seemed to be more closely linked to the inner-core dynamics and thus processes that take place at smaller spatial scales.A new model evaluation tool,‘track error rose’,was used to analyze the models’systematic error in the track forecast using the same concepts as the‘wind rose’.The results showed that as the lead time increased,most of the global models forecast a TC moving speed that was slower than observations and the largest track error often appeared around the rear direction of the observation position.Another new model evaluation tool,the Taylor diagram,was used to evaluate the intensity predictions from the global models.A Taylor diagram provides a way of plotting standard deviation,centered root mean square,and the correlation coefficient on a two-dimensional graph,indicating how closely a predicted TC intensity matches observations.This made it easy to distinguish the intensity forecast performance of the seven global models and determine which models were in relatively good agreement with observations.Furthermore,it also provided a statistical measure of the correlation between modeled and observed TC intensity,offering a practical way of assessing and summarizing model capability.

Application of an Improved Analog-Based Heavy Precipitation Forecast Model to the Yangtze–Huai River Valley and Its Performance in June–July 2020

Precipitation extremes,such as the record-breaking Meiyu characterized by frequent occurrences of rainstorms that resulted in severe flooding over the Yangtze-Huai River valley(YHRV)in June-July 2020,are always attracting considerable interest,highlighting the importance of improving the forecast accuracy at the medium-to-long range.To elevate the skill in forecasting heavy precipitation events(HPEs)with both long and short durations,the Key Influential Systems Based Analog Model(KISAM)was further improved and brought into operational application in 2020.Verification and comparison of this newly adapted analog model and ensemble mean forecasts from the ECMWF at lead times of up to 15 days were carried out for the identified 16 HPEs over the YHRV in June-July 2020.The results demonstrate that KISAM is advantageous over ECMWF ensemble mean for forecasts of heavy precipitation≥25 mm day-1 at the medium-to-long(6-15-day)lead times,based on the traditional dichotomous metrics.At short lead times,ECMWF ensemble mean outperforms KISAM due largely to the low false alarm rates(FARs)benefited from an underestimation of the frequency of heavy precipitation.However,at the medium-to-long forecast range,the large fraction of misses induced by the high degree of underforecasting overwhelms the fairly good FARs in the ECMWF ensemble mean,which partly explains its inferiority to KISAM in terms of the threat score.Further assessment on forecasts of the latitudinal location of accumulated heavy precipitation indicates that smaller displacement errors also account for a part of the better performance of KISAM at lead times of 8-12 days.

PERFORMANCE OF TROPICAL CYCLONE FORECAST IN WESTERN NORTH PACIFIC IN 2015

The operational track and intensity forecast errors of tropical cyclones(TCs) over the western North Pacific in 2015 were evaluated on the basis of RSMC-Tokyo's "best-track" dataset. The results showed that position errors for each official agency were under 80 km, 130 km, 180 km, 260 km and 370 km at 24, 48, 72, 96 and 120 hr lead time. Stepped decreases in the values of each quantile were made at every lead times and have been made by global models from 2010 to 2015, especially for long lead time. The results of the Track Forecast Integral Deviation(TFID) show a clearly decreasing trend for most global models, indicating that the TC forecast tracks became increasingly similar to the observations. In 2015, the intensity forecast skill scores for both global and regional models were almost negative. However, the skill of EPSs' intensity forecasting has made significant progress in the past year.

Performance of tropical cyclone forecasts in the western North Pacific in 2017

The forecasts of tropical cyclones(TC)in 2017 from five official guides,six global models,six regional models and six ensemble systems were assessed to study the current capability of track and intensity forecasts for the western North Pacific.The results show that the position errors for official agencies were under 100,165,265,335 and 425 km at the lead times of 24,48,72,96 and 120 h,respectively.As the forecast lead times increased,the forecasted TCs propagated,on average,too slow for most official guides.It is encouraging to note that all the models had positive skill scores,there is an overall upward trend in the skill scores of the models during from 2010 to 2017.Furthermore,both global and regional models’intensity forecast skill was increasing year by year from 2010 to 2017.For the ensemble prediction systems(EPSs),ECMWF-EPS was the best forecast system for the lead time less than 72 h,beyond the 72 h,the best EPS belong to NCEP-GEFS.

Analysis of a Heavy Rainfall Event over Beijing During 21-22 July2012 Based on High Resolution Model Analyses and Forecasts

The heaviest rainfall over 61 yr hit Beijing during 21-22 July 2012.Characterized by great rainfall amount and intensity,wide range,and high impact,this record-breaking heavy rainfall caused dozens of deaths and extensive damage.Despite favorable synoptic conditions,operational forecasts underestimated the precipitation amount and were late at predicting the rainfall start time.To gain a better understanding of the performance of mesoscale models,verification of high-resolution forecasts and analyses from the WRFbased BJ-RUCv2.0 model with a horizontal grid spacing of 3 km is carried out.The results show that water vapor is very rich and a quasi-linear precipitation system produces a rather concentrated rain area.Moreover,model forecasts are first verified statistically using equitable threat score and BIAS score.The BJ-RUCv2.0forecasts under-predict the rainfall with southwestward displacement error and time delay of the extreme precipitation.Further quantitative analysis based on the contiguous rain area method indicates that major errors for total precipitation（〉 5 mm h^（-1）） are due to inaccurate precipitation location and pattern,while forecast errors for heavy rainfall（〉 20 mm h^（-1）） mainly come from precipitation intensity.Finally,the possible causes for the poor model performance are discussed through diagnosing large-scale circulation and physical parameters（water vapor flux and instability conditions） of the BJ-RUCv2.0 model output.

Evaluation of forecast performance for Super Typhoon Lekima in 2019

The predictions for Super Typhoon Lekima(2019)have been evaluated from official forecasts,global models,regional models and ensemble prediction systems(EPSs)at lead times of 1–5 days.Track errors from most deterministic forecasts are smaller than their annual mean errors in 2019.Compared to the propagation speed,the propagation direction of Lekima(2019)was much easier to determine for the official agency and numerical weather prediction(NWP)models.The National Centers for Environmental Prediction Global Ensemble Forecast System(NCEP-GEFS),Japan Meteorological Agency Global Ensemble Prediction System(JMA-GEPS)and Meteorological Service of Canada Ensemble System(MSC-CENS)are underdispersed,and the Shanghai Typhoon Institute Typhoon Ensemble Data Assimilation and Prediction System(STI-TEDAPS)is overdispersed,while the ensemble prediction system from European Centre for Medium-Range Weather Forecasts(ECMWF)shows adequate dispersion at all lead times.Most deterministic forecasting methods underestimated the intensity of Lekima(2019),especially for the rapid intensification period after Lekima(2019)entered the East China Sea.All of the deterministic forecasts performed well at predicting the first landfall point at Wenling,Zhejiang Province with a lead time of 24 and 48 h.

Super-Parameterization in GRAPES: The Construction of SP-GRAPES and Associated Preliminary Results

Super-parameterization（SP） aims to explicitly represent deep convection within a coarse resolution global model by embedding a cloud resolving model（CRM） in each column of the mother model. For the first time, we implemented the SP in a mesoscale regional weather model, the Global/Regional Assimilation and Pr Ediction System（GRAPES）. The constructed SP-GRAPES uses a two-dimensional（2D） CRM in each grid column. A control and two SP simulations are conducted for the Beijing ＂7.21＂ heavy rainfall event to evaluate improvements in GRAPES using SP. The SP-run-I is a basic SP run delivering microphysics feedback only, whereas the SP-run-II delivers both microphysical and cloud fraction feedbacks. A comparison of the runs indicates that the SP-run-I has a slightly positive impact on the precipitation forecast than the control run. However, the inclusion of cloud fraction feedback leads to an evident overall improvement, particularly in terms of cloud fraction and 24-h cumulative precipitation. Although this is only a preliminary study using SP-GRAPES, we believe that it will provide considerable guidance for follow-up studies using SP in China.