Copyright (c) 2009 All rights reserved. http://works.bepress.com/j_scott_armstrong Recent documents in J. Scott Armstrong en-us Sun, 31 May 2009 06:46:01 PDT 3600 http://works.bepress.com/j_scott_armstrong/139 http://works.bepress.com/j_scott_armstrong/139 Wed, 17 Dec 2008 11:45:11 PST We assessed three important criteria of forecastability--simplicity, certainty, and variability. Climate is complex due to many causal variables and their variable interactions. There is uncertainty about causes, effects, and data. Using evidence-based (scientific) forecasting principles, we determined that a naïve "no change" extrapolation method was the appropriate benchmark. To be useful to policy makers, a proposed forecasting method would have to provide forecasts that were substantially more accurate than the benchmark. We calculated benchmark forecasts against the UK Met Office Hadley Centre's annual average thermometer data from 1850 through 2007. For 20- and 50-year horizons the mean absolute errors were 0.18°C and 0.24°C. The accuracy of forecasts from our naïve model is such that even perfect forecasts would be unlikely to help policy makers. We nevertheless evaluated the Intergovernmental Panel on Climate Change's 1992 forecast of 0.03°C-per-year temperature increases. The small sample of errors from ex ante forecasts for 1992 through 2008 was practically indistinguishable from the naïve benchmark errors. To get a larger sample and evidence on longer horizons we backcast successively from 1974 to 1850. Averaged over all horizons, IPCC errors were more than seventimes greater than errors from the benchmark. Relative errors were larger for longer backcast horizons. Kester C. Green Forecasting http://works.bepress.com/j_scott_armstrong/138 http://works.bepress.com/j_scott_armstrong/138 Wed, 17 Dec 2008 11:45:04 PST Prior research offers a mixed view of the value of expert surveys for long-term election forecasts. On the positive side, experts have more information about the candidates and issues than voters do. On the negative side, experts all have access to the same information. Based on prior literature and on our experiences with the 2004 presidential election and the 2008 campaign so far, we have reason to believe that a simple expert survey (the Nominal Group Technique) is preferable to Delphi. Our survey of experts in American politics was quite accurate in the 2004 election. Following the same procedure, we have assembled a new panel of experts to forecast the 2008 presidential election. Here we report the results of the first survey, and compare our experts' forecasts with predictions by the Iowa Electronic Market. Randall J. Jones Jr. Forecasting http://works.bepress.com/j_scott_armstrong/137 http://works.bepress.com/j_scott_armstrong/137 Wed, 17 Dec 2008 11:44:59 PST Prior research found that people's assessments of relative competence predicted the outcome of Senate and Congressional races. We hypothesized that snap judgments of "facial competence" would provide useful forecasts of the popular vote in presidential primaries before the candidates become well known to the voters. We obtained facial competence ratings of 11 potential candidates for the Democratic Party nomination and of 13 for the Republican Party nomination for the 2008 U.S. Presidential election. To ensure that raters did not recognize the candidates, we relied heavily on young subjects from Australia and New Zealand. We obtained between 139 and 348 usable ratings per candidate between May and August 2007. The top-rated candidates were Clinton and Obama for the Democrats and McCain, Hunter, and Hagel for the Republicans; Giuliani was 9th and Thompson was 10th. At the time, the leading candidates in the Democratic polls were Clinton at 38% and Obama at 20%, while Giuliani was first among the Republicans at 28% followed by Thompson at 22%. McCain trailed at 15%. Voters had already linked Hillary Clinton's competent appearance with her name, so her high standing in the polls met our expectations. As voters learned the appearance of the other candidates, poll rankings moved towards facial competence rankings. At the time that Obama clinched the nomination, Clinton was ahead in the popular vote in the primaries and McCain had secured the Republican nomination with a popular vote that was twice that of Romney, the next highest vote-getter. J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/136 http://works.bepress.com/j_scott_armstrong/136 Wed, 17 Dec 2008 11:44:53 PST One hundred and thirty-nine principles are used to summarize knowledge about forecasting. They cover formulating a problem, obtaining information about it, selecting and applying methods, evaluating methods, and using forecasts. Each principle is described along with its purpose, the conditions under which it is relevant, and the strength and sources of evidence. A checklist of principles is provided to assist in auditing the forecasting process. An audit can help one to find ways to improve the forecasting process and to avoid legal liability for poor forecasting. J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/135 http://works.bepress.com/j_scott_armstrong/135 Fri, 21 Nov 2008 10:08:41 PST Wensley (1994) makes three key points. First, it is worthwhile to conduct empirical studies of the value of management techniques. Second, managers probably misuse portfolio methods. Third, the Armstrong and Brodie study is flawed. We agree with all three points. J. Scott Armstrong http://works.bepress.com/j_scott_armstrong/134 http://works.bepress.com/j_scott_armstrong/134 Mon, 09 Jun 2008 12:10:30 PDT Calls to list polar bears as a threatened species under the United States Endangered Species Act are based on forecasts of substantial long-term declines in their population. Nine government reports were written to help U.S. Fish and Wildlife Service managers decide whether or not to list polar bears as a threatened species. We assessed these reports based on evidence-based (scientific) forecasting principles. None of the reports referred to sources of scientific forecasting methodology. Of the nine, Amstrup, Marcot, and Douglas (2007) and Hunter et al. (2007) were the most relevant to the listing decision, and we devoted our attention to them. Their forecasting procedures depended on a complex set of assumptions, including the erroneous assumption that general circulation models provide valid forecasts of summer sea ice in the regions that polar bears inhabit. Nevertheless, we audited their conditional forecasts of what would happen to the polar bear population assuming, as the authors did, that the extent of summer sea ice would decrease substantially during the coming decades. We found that Amstrup et al. properly applied 15 percent of relevant forecasting principles and Hunter et al. 10 percent. Averaging across the two papers, 46 percent of the principles were clearly contravened and 23 percent were apparently contravened. Consequently, their forecasts are unscientific and inconsequential to decision makers. We recommend that researchers apply all relevant principles properly when important public-policy decisions depend on their forecasts. J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/133 http://works.bepress.com/j_scott_armstrong/133 Wed, 06 Feb 2008 06:58:03 PST Jørgensen's paper examines the application of one of the most well established findings in forecasting--namely, the superior accuracy of quantitative models in comparison to judgmental forecasts. Models improved the accuracy in 72% of the 136 studies in the meta-analysis by Grove, Zald, Lebow, Snitz, and Nelson (2000). However, in Jørgensen's meta-analysis, which is restricted to forecasts of software effort, models were superior for only 38% of the studies. J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/132 http://works.bepress.com/j_scott_armstrong/132 Wed, 06 Feb 2008 06:55:51 PST J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/131 http://works.bepress.com/j_scott_armstrong/131 Wed, 16 Jan 2008 11:39:09 PST Sophisticated extrapolation techniques have had a negligible payoff for accuracy in forecasting. As a result, major changes are proposed for the allocation of the funds for future research on extrapolation. Meanwhile, simple methods and the combination of forecasts are recommended. J. Scott Armstrong Forecasting http://works.bepress.com/j_scott_armstrong/130 http://works.bepress.com/j_scott_armstrong/130 Wed, 16 Jan 2008 11:39:04 PST It seems trivial to point out that one of the major goals of the International Institute of Forecasters is to communicate research findings. In particular, the IIF tries to foster communication among researchers, between researchers and practitioners, across nationalities, and across disciplines. We have two major vehicles for this: the annual symposiums and the journal. This editorial examines the results that we have had to date with our journals. J. Scott Armstrong Forecasting