This record is being processed for inclusion into GeoRef. It may not yet have been indexed, given a translated title, or checked by a GeoRef editor.

Proper scoring systems with definite connections to information values of tsunami warnings

Saved in:
Online Access: Get full text
Authors:Hayashi, Yutaka
Author Affiliations:Primary:
Meteorological Research Institute, Japan
Volume Title:Japan Geoscience Union meeting 2014
Source:Abstract - Japan Geoscience Union Meeting, Vol.2014; Japan Geoscience Union meeting 2014, Yokohama, Japan, April 28 - May 2, 2014. Publisher: Japan Geoscience Union, Tokyo, Japan
Publication Date:2014
Note:In English
Summary:Necessary conditions which newly introduced method can improve forecast are, existing proper scoring system, and that the new method marks better score than the present method do. Up to now, these scoring system have never applied to tsunami warning system. Some scoring rules being applied widely to binary forecasts in weather forecasting, such as having precipitation or not, have close connections to change of utility for users. These scores are based on assumption that all user know their cost to make counter measures (C) and loss in case of no counter measure (L). When the forecast says the event will occur, and all users are assumed to make counter measures. In addition, a simple probability density distribution of U(-C)/U(-L) is assumed for cost-loss model, where U is the utility function. In general, a score is calculated by using a targeted dataset, e.g., a fixed period of time, and frequencies: occurrence of targeted phenomena is forecasted and observed (hit: Na), forecasted but not occurred (false alarm Nb), not forecasted but occurred (misdetection: Nc), and not forecasted and not occurred (hit: Nd). For example, equitable threat score (ETS = (Na-K)/(Na+Nb+Nc-K), where K = (Na+Nb)(Na+Nc)/(Na+Nb+ Nc+Nd) ) is one of their scoring system. In this paper, suitable scoring rules for tsunami warnings are derived by considering the characteristics of tsunami warnings and following assumptions. (1) Scores can be defined without Nd, because counting Nd does not make sense for tsunami warning. (2) In case of tsunami warning, users of forecasts can select actions to take a counter measure or not. In case of no warning, users do not take a counter measure. Change of utilities are U(-C) and U(-L) for taking a counter measure and for when a phenomenon happens without a counter measure, respectively. (3) All users know the fault alarm ratio (FAR = Nb/(Na+Nb)) of the warning, their utilities for each condition (U(-C), U(-L)), and then their rational decision-making choose the option so that their expectation of utility (Ex(U)) become maximum. Here, if U(-L)/U(-C) <FAR/(1-FAR) is satisfied, not taking a counter measure is the more reasonable decision. According to this assumption, larger the FAR is, larger the cost-loss ratio is, warning become easier to be ignored. (4) Assuming three types of probability density functions on x=U(-C)/U(-L). a) Uniform model: f(x)=1, b) Low-cost model: f(x)=2-2x, and (c) High-cost model: f(x)=2x for the range of 0 = x = 1. (5) The scores are set to be proportional to the information value of the warning. Here, ΔU can be calculated as the integral corresponding to each distribution of (4) and utilities of selected actions at the Na+Nb warnings based on the rational decision-making described in (3). Besides, if there were not for warning system, users should have lose utility as much as -U(-L) at every event. Then, V =-ΔU/((Na+Nc)U(-L)). Scores corresponding to models a)-c) in (4) are derived as follows. a) V=Na 2/(2(Na+Nb)(Na+Nc)). For good warning which satisfies both Na >> Nb and Na >> Nc, the score can be approximated to V = CSI/2, where CSI = Na/(Na+Nb+Nc) is threat score or critical success index. b) V=(2/3)(1-FAR)(1-M)(1+M/2), where M = Nc/(Na+Nc) is missing ratio. For warnings with few misdetection which satisfies Nc (Na, the score can be approximated to be V = (2/3)(1-FAR)(1-M/2). (c) V=(1-FAR)2(1-M)/3. The proper score system thus changes according to the cost-loss ratio, which have close relation to preparedness. It is necessary to choose suitable forecast method using proper scoring system which is corresponding to a social structure. In the meeting, the author would like to discuss also on the problem for the practical application of the scoring systems.
Subjects:Decision-making; Early warning systems; Geologic hazards; Natural hazards; Ocean waves; Prediction; Statistical analysis; Tsunamis; Warning systems
Record ID:713189-3
Copyright Information:GeoRef, Copyright 2021 American Geosciences Institute.
Tags: Add Tag
No Tags, Be the first to tag this record!
Be the first to leave a comment!
You must be logged in first