ISSN: 2347-7830

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Evaluation of Cosmo Model Precipitation Forecast for Heavy Rainfall Events Over Nigeria

Chibuzo N Agogbuo1*, Olubi C Adedamola1, Benjamin O Alabi1, Mbah Obiageri1 and Oloruntoba Bamidele2

1National Weather Forecasting and Climate Research Center, Abuja, Nigeria

2African Centre of Meteorological Application and Development, Niamey, Nigeria

*Corresponding Author:
Chibuzo N Agogbuo
National Weather Forecasting and Climate Research Center
Abuja, Nigeria
E-mail: [email protected]

Received date: 03/07/2019; Accepted date: 16/07/2019; Published date: 19/07/2019

Visit for more related articles at Research & Reviews: Journal of Ecology and Environmental Sciences

Abstract

Accurate precipitation forecast from Numerical Weather Prediction models could be a useful tool in the issuance of early warning for extreme weather-related events such as flooding. Analysis of rainfall events over Nigeria is challenged by a lot of factors, ranging from lack of good radar coverage and sparse population of rain gauge stations to inconsistency in the recording of rainfall amounts from the available stations. This article evaluated the precipitation forecast of the COSMO model, which is simulated and used at the Nigerian Meteorological Agency. The evaluation was done in terms of categorical and Quantitative Precipitation Forecast for four heavy rainfall events that caused severe flooding in some cities in Nigeria in the months of August and September 2018. Precipitation forecasts from the COSMO model were compared with observed precipitation at both station and gridded observation points using eyeball verification, categorical statistics, and Taylor diagrams. Categorical Statistics showed that in all four cases studied, the model recorded accuracy and Critical Success Index (CSI) values of over 50%. However, further analysis revealed that location errors and underestimation of heavy rainfall events in some areas were the main sources of forecast uncertainties for most of the days evaluated.

Keywords

Numerical weather prediction, COSMO model, Extreme events, Precipitation forecast, Taylor diagram

Introduction

Forecasting of convective precipitation is a central issue in current meteorology [1]. Numerical Weather Prediction (NWP) Models have been used for over seven decades to make precipitation forecast and other atmospheric variables. Though the use of these models has brought considerable improvement in precipitation forecasting, the evaluation of the performance of these models over a particular place is key to ascertaining their reliability and their ability to reasonably predict atmospheric variables over such areas. Verification of meteorological and oceanographic forecasts is essential for monitoring accuracy, understanding errors, and making improvements in forecasting systems [2]. Kostopoulou et al. [3] noted that validation of model estimates with observed data is thus considered essential to assess the reliability of modeled data at regional scales.

A number of studies have assessed NWP precipitation forecast by evaluating them against gauge observations. For example, Damrath et al. [4] evaluated the Quantitative Precipitation Forecast from the German Weather Service (DWD) using long-time verification statistics against 240 gauge stations over 7 years in Germany and Switzerland, including the Frequency Bias Index (FBI) and the True Skill Statistics (TSS), and presented examples of application to flood events. They also identified that wrong parameterization of convective precipitation could lead to poor input to hydrological models in the case of summer time flash floods. In Africa, Canonical Correlation Analysis [5] was carried out using real-time predictions of rainfall in July-September 1997 and January-March 1998, respectively for the Sahel and southern Africa at 1 month lead period. Cross-validation was used to estimate the skill in the forecasts and Heidke skill (S) score was obtained showing reasonable skill. In Nigeria, considering skillful weather forecasting as a strategy for adapting food production to a variable and changing the climate, existing products of four weather forecasting organizations with interests in West Africa were assessed using the observed weather during the period from 1996 to 2000. The weather forecasting organizations concerned are NOAA (USA), Met Office (UK), CNRS (France) and the Nigerian Central Forecasting Office [6]. It was established in the study that better rainfall forecasts could be achieved with higher resolution sea-surface temperature anomaly data and the inclusion of more predictor variables, especially those of a synoptic nature. Evaluation of NWP precipitation forecast using gauge observation is likely to give a poor result in areas having a poor density of gauge stations. This is the case with Nigeria having about fifty-six gauge stations.

Convective activities could produce very intense rainfall that often leads to river and flash floods causing significant loss of life and property, soil erosion and other socioeconomic issues [7]. In Nigeria, heavy rain is one of the major causes of floods and other hydrological disasters during the summer monsoon season [8]. The forecast for such events no doubt is beneficial for preventing disasters and reducing damage. Several cases of flooding were recorded in several cities of Nigeria in the year 2018 leading to significant loss of life and property. One aim of this paper is to provide weather forecast evaluation results from the COSMO model used in Nigeria. Also, this article aims to obtain major information on the accuracy and reliability of convective precipitation forecast of the COSMO model based on the analysis of four flood events that occurred in some cities of Nigeria [9] in the year 2018. The findings could be very useful in early warning and crisis management in Nigeria and neighboring countries for natural events such as flash floods caused by heavy precipitation.

Materials and Methods

Materials

The data used in this study consist of observed rainfall data collected from 56 gauge stations across Nigeria and the COSMO model precipitation forecast for selected days of heavy rainfall in 2018 (Table 1). The available gauge stations are unevenly distributed and sparse as seen in Figure 1, this usually brings a limitation to the quality of data. The initial data and lateral boundary conditions of the COSMO model were collected from DWD which are used to initialize the regional model.

S. NO. Station Name Latitude Longitude 5th August 28th August 29th August 6th Sept
Observed Cosmo Observed Cosmo Observed Cosmo Observed Cosmo
1 Abeokuta 7.2 3.33 0 0 0.7 1.36719 1.1 0.417969 2.6 0.679688
2 Abakiliki 5.4 7.9 40,2 6.82812 45.2 18.9062 10.3 12.6406 8.5 16.5156
3 Abuja 9.25 7 8.3 9.89453 0 29.4805 6.6 84.4844 20.4 202.375
4 Ado-Ekiti 7.6 5.2 0 0.351562 1.5 7.71094 3.3 2.46875 0.5 5.23438
5 Akure 7.28 5.23 0 0.164062 27.2 9.84375 2.9 5.47656 15.8 5.14062
6 Asaba 6.23 6.82 0 0.28125 98.6 2.42969 4.1 7.23047 13.2 1.375
7 Awka 6.2 7.1 0 0.642578 57.1 8.70312 0 8.83008 41.7 17.8906
8 Bauchi 10.28 9.82 95.6 9.32812 10.4 7.19531 0 15.9609 3.8 114.148
9 Bida 9.1 6 49.7 2.14844 0 5.95703 1.4 14.1719 0 66.5859
10 Benin 6.33 5.6 0 0.826172 59.2 5.45312 56.7 15.9141 46.4 43.2031
11 Calabar 4.97 8.35 4.4 7.3125 0 18.3672 0 12.4727 112.8 5.58594
12 CR 4.75 8.35   2.25586 0 2.94922 0 6.26367 0 7.82031
13 Dutse 11.7 9.3 23 93.6523 7 9.40625 0 13.3125 22.1 72.2969
14 Eket 4.4 7.95 0 19.2051 0 16.582 16.8 18.1738 3.2 35.5859
15 Enugu 6.5 7.55 0.6 -0.00391 1.4 7.98438 0.4 0 14.8 2.46875
16 Gombe 10.28 11.17 0 14.8125 30.1 3.77344 14.2 6.41016 44.7 48.8047
17 Gusau 12.17 6.77 1.8 8.19141 0 7.65625 0 12.4062 44.7 12.3516
18 Ibadan 7.43 3.9 0 0 0 4.55469 0 2.49609 3.7 9.625
19 NCent 8.18 9.75 0 2.92383   11.9492 0 7.1582 0 7.57031
20 Ikeja 6.85 3.33 0 0 2.3 9.57031 0 0.568359 30.2 1.25
21 Ikom 5.97 8.72 0 6.375 1.2 27.8438 9.6 11.7031 28.2 7.07031
22 Ilorin 8.48 4.58 0 0 46.2 24.6055 36.2 3.25 130.1 28.6016
23 Ijebu Ode 6.83 3.93 0.2 0 2 9.23438 20.5 6.11133 12.2 3.97656
24 Iseyin 7.96 3.06 0 0 3.4 0.632812 0 0.972656 7.7 6.32812
25 Jalingo 8.9 11.4 0 10.5234 13.3 14.3438 0 31.5527 50.5 54.0703
26 Jos 9.87 8.9 0 16.8047 1.1 17.4258 33.1 31.1445 23.9 55.25
27 Kaduna 10.6 7.45 0 9.84375 21.3 3.70703 0 12.9316 25 42.0391
28 Kano 12.05 8.53 2.2 57.2344 0 5.52344 0 15.7734 21.5 74.5703
29 Katsina 13.02 7.68 0 5.55469 0 13.2188 21.8 11.8594 15.1 0.875
30 SSou 5.6 5.8 0 1.32812 0 1.30469 105 7.09766 131.6 41.0469
31 Lafia 8.5 8.5 0 10.4395 0 24.0781 0 27.5957 8.6 16.1797
32 Lagos 6.52 3.3 0 0 0 3.17188 3.5 1.6582 46.1 0.757812
33 Lokoja 7.8 6.73 1.2 0.867188 0 2.75391 5.6 7.70312 22.6 30.4609
34 Maiduguri 11.85 13.08 8 11.2031 0 6.23047 0 12.2266 19 24.6953
35 Makurdi 7.73 8.53 15 7.50781 7.1 15.4219 2.6 14.2383 2.8 22.8594
36 Minna 9.56 6.54 2.8 8.69141 0 26.8711 3.1 26.3672 22.5 38.4062
37 Nguru 12.88 10.47 0 43.6484 30.1 11.2344 54.2 3.85938 0 20.0703
38 Obudu 6.7 9.2 5.6 0.03125 7.1 136.703 28.3 5.90234 0.5 9.75781
39 Ogoja 6.7 8.8 0 0.117188 16 71.5117 4.8 4 0 26.1016
40 Ondo 7.1 4.83 0 0 0.7 3.6875 0 5.42969 19.8 7.80469
41 Onne 4.7 7.2 0.2 1.04688 0 13.3984 0.6 0.740234 8.8 2.89844
42 Oshogbo 7.8 4.5 0 0 2 0.609375 0 2.05664 26.7 19.6016
43 SWes 7.82 4.5 0 0 0 0.609375 0 2.05664 42.6 19.6016
44 Owerri 5.48 7.03 0 3.75 37.6 3.91797 0 5.72656 65.2 10.1328
45 Port Harcourt 4.85 7.12 0.2 1.44531 0 12.7383 11 0.001953 31.2 0.179688
46 Potiskum 11.7 11.03 1.4 21.0762 0 10.8672 0.3 15.2422 9.8 91.7188
47 Oshodi 6.58 3.32 0 0 0 9.57031 6 0.568359 15.5 1.25
48 Shaki 8.35 3.47 0 0 4 12.6211 0 1.16406 8.2 9.40625
49 Sokota 12.92 5.2 1.8 1.21094 1.1 6.41797 0 5.58594 27.6 6.97656
50 Umuahia 5.5 7.5 5.1 6.16406 0 9.86719 55 6.86914 27.5 15.5156
51 Usi Ekiti 7.7 5.3 1.2 0.304688 1.5 8.91406 0 3.25 0.3 7.32812
52 Uyo 5.05 7.95 10.3 5.16797 41.6 11.6055 0.4 11.1582 16.8 15.7344
53 Warri 5.52 5.73 8.3 0.015625 0 0 3.8 8.31445 40.7 43.5391
54 Yelwa 11 4.5 8.7 2.45312 11.5 10.7109 65.3 7.94336 17.1 78.7188
55 Yola 9.23 12.47 0 2.63281 0 11.9492 33 8.12891 0 33.1719
56 Zaria 11.07 7.75 2.5 31.6641 53.7 30.3164 29.3 24.2969 35.1 52.4531

Table 1. Observed rainfall data from 56 gauge stations across Nigeria and COSMO model precipitation forecast for selected days of heavy rainfall in 2018.

environmental-sciences-study-area

Figure 1: Map of the study area showing the locations of the meteorological stations.

Methods

A joint distribution contingency table was used to compute accuracy, bias and Critical Success Index (CSI) for the categorical forecast (Table 2).

Forecast   Yes No Total
Yes Hits false alarm forecast yes
No Misses correct negatives forecast no
Total observed yes observed no total

Table 2. Joint distribution Contingency table for categorical verification scores.

The verification scores that are obtainable from the contingency table above are defined below:

Accuracy: The level of agreement between forecast and observed

image

Bias: Compares the forecast and observed the frequency of YES events

image

Critical Success Index (CSI): Looks at how well the forecast yes events corresponds with the observed yes events

image

Also, Eyeball verification, Histogram and Taylor diagrams were used to determine the level of relationship between forecast and observed events in areas that recorded heavy precipitation.

The Taylor diagram is a very useful model evaluation tool. Taylor diagram [10] provides a graphical way to summarize how closely the simulated rainfall matches the observed data. In this type of polar diagram, the angular coordinate corresponds to the correlation coefficient(r) between simulated and observed data. The correlation coefficients between the variables are usually from -1.0 to +1.0. Thus, a single quadrant is used throughout with r values ranging from 0.0 to +1.0. These are marked along the widest arc joining the tips of the two axes. Note that variable with a low or negative value of correlation would not show in the positive quadrant. The radial coordinate gives information about the Standard Deviation (SD) of the results for each experiment. The purple points on the abscissa axis or the origin represent the observation having ‘r’ of 1 (as expected). The Centered Root-Mean Square Errors (CRMSE) between the experiments and observation are proportional to the distance between this reference and the simulation points. In conclusion, Taylor diagram is a powerful tool that summarizes three statistics: standard deviations, correlation coefficient, and RMSE, giving a rapid, concise, and easy visual point of view between model output and observation

Root Mean Square Error (RMSE)

This provides a good overall measure of how close the modeled values are compared to the predicted values. It combines the spread of individual errors.

image

The (Pearson) Correlation Coefficient (r) is a measure of the strength of the linear relationship between two variables. For r=1, it implies that there is a perfect linear relationship with a positive slope between the two variables. For r=-1, it means that there is a perfect linear relationship with a negative slope between the two variables. However, if r=0 then, no linear relationship between the variables. r indicates the extent to which patterns in the model match those in the observations.

Results and Discussion

Categorical Statistics

Table 3 reveals that in all four cases studied, the model recorded accuracy and CSI values of over 50%. The highest accuracy value of 79% was recorded for the rainfall event of 6th September 2018. This implies that a large fraction of the COSMO forecast for the days under study were correct. Also, the BIAS values show a fair relationship between the forecast frequency of “yes” events and the observed frequency of “yes” events.

Rainfall date Accuracy (%) Critical Success Index (CSI) (%) Bias
August 5, 2018 69.7 54 1.48
August 28, 2018 59 57 1.6
August 29, 2018 53 51 1.7
September 6, 2018 79 79 1.1

Table 3. Verification scores of the categorical forecast of the COSMO model for selected heavy rainfall events over Nigeria.

Eyeball Verification

Figure 2 shows that the COSMO model gave a forecast of heavy precipitation over parts of Northern Nigeria while Gauge observation shows that heavy precipitation occurred but was placed southerly when compared with COSMO forecast.

environmental-sciences-COSMO-model

Figure 2: COSMO model forecast top and gauge observation (bottom) on 05/08/2018 over Nigeria.

From Figure 3, it can be seen that the COSMO model expected the light to moderate rainfall over Northern Nigeria and heavy rainfall over some parts of southern Nigeria. Gauge observation showed heavy rainfall over Northeastern and Southern Nigeria.

environmental-sciences-gauge-observation

Figure 3: COSMO model forecast top and gauge observation (bottom) on 28/08/2018 over Nigeria.

Figure 4 shows that heavy rainfall was expected over some parts of Central Nigeria while moderate to light rainfall was expected elsewhere by the model. The gauge observation shows that heavy rainfall occurred over parts of Northern and Southern Nigeria.

environmental-sciences-top-gauge

Figure 4: COSMO model forecast top and gauge observation (bottom) on 29/08/2018 over Nigeria.

The model precipitation forecast Shows heavy rainfall in most parts of northern Nigeria and some sections in the south. Gauge observation also shows heavy rainfall over most part of the country (Figure 5).

environmental-sciences-forecast-top

Figure 5: COSMO model forecast top and gauge observation (bottom) on 06/09/2018 over Nigeria.

Taylor Diagram Analysis

Taylor diagram analysis from Figure 6 showed that there was a good spatial correlation between observed and forecast rainfall over the South-west (SW) region of Nigeria on August 29 and September 6, 2018, Elsewhere the correlation was poor. Also, a good correlation was recorded over South-eastern (SE) and Northern Nigeria on the 5th and 28th August respectively.

environmental-sciences-Taylor-diagrams

Figure 6: Taylor diagrams showing COSMO simulated rain rate (mm/day) with respect to the rain gauge (mm/day) for 05th, 28th, 29th August and 6th September 2018.

The histogram shows (Figure 7) that 84% of the heavy rainfall events studied was underestimated by the COSMO model. However, about 15% of the model forecast of extreme rainfall was accurate.

environmental-sciences-rain-rate

Figure 7: Histogram showing COSMO simulated rain rate (mm/day) with respect to the rain gauge (mm/day) for the selected days in 2018.

Summary and Conclusion

This study evaluated the performance of the COSMO model for four heavy rainfall events that occurred in Nigeria on 05th, 28th, 29th August and 5th September 2018. The model forecasts were compared directly with gauge observations from 56 stations. The results showed that in terms of the categorical forecast of precipitation events, the model performed well having recorded Accuracy and Critical Success Index (CSI) values of over 50% in all four cases studied. Also, the Taylor diagram showed a fair spatial correlation between forecast and observed events in some parts of the country. However, eyeball verification and Histogram analysis showed spatial misplacement of heavy rainfall events and underestimation of precipitations amounts in most heavy rainfall areas. These were the main source of forecast uncertainties. It is therefore recommended that more experiments and simulations were conducted using different Physics in the COSMO model for several heavy rainfall events over Nigeria. Also, the introduction and use of convective allowing models in addition to the existing COSMO model could improve the country’s capability in forecasting heavy rainfall events.

References