Evaluation of Cosmo Model Precipitation Forecast for Heavy Rainfall Events Over Nigeria

Abstract

Accurate precipitation forecast from Numerical Weather Prediction models could be a useful tool in the issuance of early warning for extreme weather-related events such as flooding. Analysis of rainfall events over Nigeria is challenged by a lot of factors, ranging from lack of good radar coverage and sparse population of rain gauge stations to inconsistency in the recording of rainfall amounts from the available stations. This article evaluated the precipitation forecast of the COSMO model, which is simulated and used at the Nigerian Meteorological Agency. The evaluation was done in terms of categorical and Quantitative Precipitation Forecast for four heavy rainfall events that caused severe flooding in some cities in Nigeria in the months of August and September 2018. Precipitation forecasts from the COSMO model were compared with observed precipitation at both station and gridded observation points using eyeball verification, categorical statistics, and Taylor diagrams. Categorical Statistics showed that in all four cases studied, the model recorded accuracy and Critical Success Index (CSI) values of over 50%. However, further analysis revealed that location errors and underestimation of heavy rainfall events in some areas were the main sources of forecast uncertainties for most of the days evaluated.

Keywords

Numerical weather prediction, COSMO model, Extreme events, Precipitation forecast, Taylor diagram

Introduction

Forecasting of convective precipitation is a central issue in current meteorology [1]. Numerical Weather Prediction (NWP) Models have been used for over seven decades to make precipitation forecast and other atmospheric variables. Though the use of these models has brought considerable improvement in precipitation forecasting, the evaluation of the performance of these models over a particular place is key to ascertaining their reliability and their ability to reasonably predict atmospheric variables over such areas. Verification of meteorological and oceanographic forecasts is essential for monitoring accuracy, understanding errors, and making improvements in forecasting systems [2]. Kostopoulou et al. [3] noted that validation of model estimates with observed data is thus considered essential to assess the reliability of modeled data at regional scales.

A number of studies have assessed NWP precipitation forecast by evaluating them against gauge observations. For example, Damrath et al. [4] evaluated the Quantitative Precipitation Forecast from the German Weather Service (DWD) using long-time verification statistics against 240 gauge stations over 7 years in Germany and Switzerland, including the Frequency Bias Index (FBI) and the True Skill Statistics (TSS), and presented examples of application to flood events. They also identified that wrong parameterization of convective precipitation could lead to poor input to hydrological models in the case of summer time flash floods. In Africa, Canonical Correlation Analysis [5] was carried out using real-time predictions of rainfall in July-September 1997 and January-March 1998, respectively for the Sahel and southern Africa at 1 month lead period. Cross-validation was used to estimate the skill in the forecasts and Heidke skill (S) score was obtained showing reasonable skill. In Nigeria, considering skillful weather forecasting as a strategy for adapting food production to a variable and changing the climate, existing products of four weather forecasting organizations with interests in West Africa were assessed using the observed weather during the period from 1996 to 2000. The weather forecasting organizations concerned are NOAA (USA), Met Office (UK), CNRS (France) and the Nigerian Central Forecasting Office [6]. It was established in the study that better rainfall forecasts could be achieved with higher resolution sea-surface temperature anomaly data and the inclusion of more predictor variables, especially those of a synoptic nature. Evaluation of NWP precipitation forecast using gauge observation is likely to give a poor result in areas having a poor density of gauge stations. This is the case with Nigeria having about fifty-six gauge stations.

Convective activities could produce very intense rainfall that often leads to river and flash floods causing significant loss of life and property, soil erosion and other socioeconomic issues [7]. In Nigeria, heavy rain is one of the major causes of floods and other hydrological disasters during the summer monsoon season [8]. The forecast for such events no doubt is beneficial for preventing disasters and reducing damage. Several cases of flooding were recorded in several cities of Nigeria in the year 2018 leading to significant loss of life and property. One aim of this paper is to provide weather forecast evaluation results from the COSMO model used in Nigeria. Also, this article aims to obtain major information on the accuracy and reliability of convective precipitation forecast of the COSMO model based on the analysis of four flood events that occurred in some cities of Nigeria [9] in the year 2018. The findings could be very useful in early warning and crisis management in Nigeria and neighboring countries for natural events such as flash floods caused by heavy precipitation.

Materials and Methods

Materials

The data used in this study consist of observed rainfall data collected from 56 gauge stations across Nigeria and the COSMO model precipitation forecast for selected days of heavy rainfall in 2018 (Table 1). The available gauge stations are unevenly distributed and sparse as seen in Figure 1, this usually brings a limitation to the quality of data. The initial data and lateral boundary conditions of the COSMO model were collected from DWD which are used to initialize the regional model.

Table 1. Observed rainfall data from 56 gauge stations across Nigeria and COSMO model precipitation forecast for selected days of heavy rainfall in 2018.

Figure 1: Map of the study area showing the locations of the meteorological stations.

Methods

A joint distribution contingency table was used to compute accuracy, bias and Critical Success Index (CSI) for the categorical forecast (Table 2).

Table 2. Joint distribution Contingency table for categorical verification scores.

The verification scores that are obtainable from the contingency table above are defined below:

Accuracy: The level of agreement between forecast and observed

Bias: Compares the forecast and observed the frequency of YES events

Critical Success Index (CSI): Looks at how well the forecast yes events corresponds with the observed yes events

Also, Eyeball verification, Histogram and Taylor diagrams were used to determine the level of relationship between forecast and observed events in areas that recorded heavy precipitation.

The Taylor diagram is a very useful model evaluation tool. Taylor diagram [10] provides a graphical way to summarize how closely the simulated rainfall matches the observed data. In this type of polar diagram, the angular coordinate corresponds to the correlation coefficient(r) between simulated and observed data. The correlation coefficients between the variables are usually from -1.0 to +1.0. Thus, a single quadrant is used throughout with r values ranging from 0.0 to +1.0. These are marked along the widest arc joining the tips of the two axes. Note that variable with a low or negative value of correlation would not show in the positive quadrant. The radial coordinate gives information about the Standard Deviation (SD) of the results for each experiment. The purple points on the abscissa axis or the origin represent the observation having ‘r’ of 1 (as expected). The Centered Root-Mean Square Errors (CRMSE) between the experiments and observation are proportional to the distance between this reference and the simulation points. In conclusion, Taylor diagram is a powerful tool that summarizes three statistics: standard deviations, correlation coefficient, and RMSE, giving a rapid, concise, and easy visual point of view between model output and observation

Root Mean Square Error (RMSE)

This provides a good overall measure of how close the modeled values are compared to the predicted values. It combines the spread of individual errors.

The (Pearson) Correlation Coefficient (r) is a measure of the strength of the linear relationship between two variables. For r=1, it implies that there is a perfect linear relationship with a positive slope between the two variables. For r=-1, it means that there is a perfect linear relationship with a negative slope between the two variables. However, if r=0 then, no linear relationship between the variables. r indicates the extent to which patterns in the model match those in the observations.

Results and Discussion

Categorical Statistics

Table 3 reveals that in all four cases studied, the model recorded accuracy and CSI values of over 50%. The highest accuracy value of 79% was recorded for the rainfall event of 6th September 2018. This implies that a large fraction of the COSMO forecast for the days under study were correct. Also, the BIAS values show a fair relationship between the forecast frequency of “yes” events and the observed frequency of “yes” events.

Table 3. Verification scores of the categorical forecast of the COSMO model for selected heavy rainfall events over Nigeria.

Eyeball Verification

Figure 2 shows that the COSMO model gave a forecast of heavy precipitation over parts of Northern Nigeria while Gauge observation shows that heavy precipitation occurred but was placed southerly when compared with COSMO forecast.

Figure 2: COSMO model forecast top and gauge observation (bottom) on 05/08/2018 over Nigeria.

From Figure 3, it can be seen that the COSMO model expected the light to moderate rainfall over Northern Nigeria and heavy rainfall over some parts of southern Nigeria. Gauge observation showed heavy rainfall over Northeastern and Southern Nigeria.

Figure 3: COSMO model forecast top and gauge observation (bottom) on 28/08/2018 over Nigeria.

Figure 4 shows that heavy rainfall was expected over some parts of Central Nigeria while moderate to light rainfall was expected elsewhere by the model. The gauge observation shows that heavy rainfall occurred over parts of Northern and Southern Nigeria.

Figure 4: COSMO model forecast top and gauge observation (bottom) on 29/08/2018 over Nigeria.

The model precipitation forecast Shows heavy rainfall in most parts of northern Nigeria and some sections in the south. Gauge observation also shows heavy rainfall over most part of the country (Figure 5).

Figure 5: COSMO model forecast top and gauge observation (bottom) on 06/09/2018 over Nigeria.

Taylor Diagram Analysis

Taylor diagram analysis from Figure 6 showed that there was a good spatial correlation between observed and forecast rainfall over the South-west (SW) region of Nigeria on August 29 and September 6, 2018, Elsewhere the correlation was poor. Also, a good correlation was recorded over South-eastern (SE) and Northern Nigeria on the 5^th and 28^th August respectively.

Figure 6: Taylor diagrams showing COSMO simulated rain rate (mm/day) with respect to the rain gauge (mm/day) for 05^th, 28^th, 29^th August and 6^th September 2018.

The histogram shows (Figure 7) that 84% of the heavy rainfall events studied was underestimated by the COSMO model. However, about 15% of the model forecast of extreme rainfall was accurate.

Figure 7: Histogram showing COSMO simulated rain rate (mm/day) with respect to the rain gauge (mm/day) for the selected days in 2018.

Summary and Conclusion

This study evaluated the performance of the COSMO model for four heavy rainfall events that occurred in Nigeria on 05^th, 28^th, 29^th August and 5^th September 2018. The model forecasts were compared directly with gauge observations from 56 stations. The results showed that in terms of the categorical forecast of precipitation events, the model performed well having recorded Accuracy and Critical Success Index (CSI) values of over 50% in all four cases studied. Also, the Taylor diagram showed a fair spatial correlation between forecast and observed events in some parts of the country. However, eyeball verification and Histogram analysis showed spatial misplacement of heavy rainfall events and underestimation of precipitations amounts in most heavy rainfall areas. These were the main source of forecast uncertainties. It is therefore recommended that more experiments and simulations were conducted using different Physics in the COSMO model for several heavy rainfall events over Nigeria. Also, the introduction and use of convective allowing models in addition to the existing COSMO model could improve the country’s capability in forecasting heavy rainfall events.

References

1. Saur D. Evaluation of the accuracy of numerical weather prediction models: advances in intelligent systems and computing. Artif Intell Persp App. 2015;347:181-191.
2. Ebert E, et al. Progress and challenges in forecast verification. Meteorol App. 2013;20:130-139.
3. Kostopoulou E, et al. Evaluation of a regional climate model using in situ temperature observations over the Balkan Peninsula. Tellus A. 2009;61:357-370.
4. Damrath U, et al. Operational quantitative precipitation forecasting at the German weather service. J Hydrol. 2000;239:260-285.
5. Thiaw WM, et al. Predictions of African rainfall on the seasonal timescale. J Geophys Res Atmos. 1999;104:31589-31597.
6. Adejuwon JO, et al. Skill assessment of the existing capacity for extended-range weather forecasting in Nigeria. Int J Climatol. 2004;24:1249-1265.
7. Papadopoulos A, et al. Improving convective precipitation forecasting through assimilation of regional lightning measurements in a mesoscale model. Am Meteorol Soci. 2005.
8. Yusuf MA, et al. Origin and residence time of shallow groundwater resources in Lagos coastal basin, south-west Nigeria: An isotopic approach. Heliyon. 2018;4:e00932.
9. Tesi GO, et al. Lagdo dam flood disaster of 2012: An assessment of the concentrations, sources, and risks of pahs in floodplain soils of the lower reaches of river Niger, Nigeria. J Environ Qual. 2016;45:305-314.
10. Taylor KE. Summarizing multiple aspects of model performance in a single diagram. J Geophys Res. 2001;106:7183-7192.