A great example of a historical forecasting methodology is PECOTA. PECOTA forecasts future performance for a player by first going back in time and finding historical players that are “similar”. It then uses information on how those historical players evolved over their careers to project how the current player will evolve. So far, this approach is as good as it can get for forecasting player performance.
Not all forecasts are this obviously historical, but all forecasts really are about intelligent selection of historical comparators.
This key relationship indicates why forecasts will always need both quantitative and qualitative components. Quantitative components — from numerical data that describes the past — is key to anchoring estimates of magnitude in an objective way.
Qualitative components are necessary to adjust for limitations in the data and to accommodate the possibility that this time “really is different”. Frequently, historical data exists because of convenience or some other business purpose; rarely is the historical data directly applicable to the current problem. A significant degree of wisdom is required to judge when “this time really is different”, as is perhaps obvious.
This post may appear to be a truism, but I've found that model interpretation and forecasting errors frequently stem from a lack of appreciation regarding the relationship between history and prediction. Curiosity, energy, and time are all required to investigate the past in a comprehensive way. It is difficult — even in retrospect — to identify key causes for historical events. It is exponentially more difficult to select and measure which of those causal relationships will be the key drivers in the future.
Companies would do well to keep in mind that forecasts are as much about the past as they are about the future. The better you know where you've been, and why, the better you will be able to navigate where you will be.
 Even quantitative measures are susceptible to subjective interpretation and biases that influence the selection of the data. Nevertheless, quantitative evaluation helps provide a measure of dispassion, if used wisely.
[Technical Post] On Friday, a Vice President in our company asked me what our definition of Economic Capital was. I responded that we defined “economic capital” as the amount of capital necessary to cover unexpected losses at the 99% confidence level. That is total and complete gibberish. I have no idea what it means, I just mirror the sentence structure used by others. For examples, see Investopedia, and other sources.
Investopedia also provides a standard graphical representation, produced below:
Below I will describe why my definition is gibberish, I will contrast this to what we are really trying to say, and I’ll close by saying that this is more than a semantics problem.
“The Machine Knows” is a classic Office episode that teaches us about Actuarial modeling in the following clip:
There are many lessons embedded in this two minutes.
1) Be careful with model interpretation. Michael wanted to interpret the result literally; be warned, if you do this, you may get very wet.
2) Models are only guides. The Map is Not the Territory; the model is not reality. From the clip, it seems possible that the GPS system (aka the Map, aka the Model) was wrong and Michael was following it into disaster (like these individuals did in real life). Even if the GPS was correct, either way the clip illustrates that reality itself needs to be paid attention to, regardless of what you believe any model says.
2-alternate) Michael fell prey to the Reification Fallacy, one of the most prevalent and powerful modeling fallacies.
3) If you can’t understand a model, then be warned that disastrous results may follow. Sometimes, models erroneously embolden you. Always be humble when interpreting model results, and be open to contrary evidence. All models are wrong; some are useful.
4) Don’t be a passenger in a car driven by someone who takes his models literally. Unless you make your living in disaster recovery.
Tomorrow I’ll post on the analysis of risk, and how that could be applied to this video.
Russ Roberts recently interviewed Sam Altman, of YCombinator. In this EconTalk episode, Sam offered the following insights into what level of planning he expects to see from those who apply to become part of YCombinator:
Sam Altman: I’ve never written [a business plan] in my life. At the stage that we are operating at, it’s irrelevant. Like financial projections also we never look at. … We would rather them spend the time working on their product, talking to users. What we care about is: Have you built a product? Have you spoken to users? Can we see that? Can we talk about where it may involve?
I think this is exactly right. As actuaries, our first instinct is to measure, quantify, and plan. However, you don’t have to have a detailed financial plan before engaging in an activity. What you have to have is a rational basis for believing that the activity has substantial merit. The level of modeling and projections must be related to (a) the ability of the forecaster to model accurately, and (b) the relative cost of producing the forecast. There is art in knowing when to model and when not to.
For young start-ups, the ability to forecast accurately is low, and the cost of forecasting is high, especially the opportunity cost. Further, if the business case has merit, the value proposition has to be easy to explain or it won’t take off. Business plans and pro formas should properly be viewed as a means of communication, not an end of themselves. And frequently the idea and business prospects can be best communicated in words, with examples, or with simple math that demonstrates scalability. And the simplest communication vehicle is frequently the most persuasive.
Most Catholic institutions affected by the recent contraceptive ruling fund their own health benefit plans. This means that there is no “insurer” available to pass the cost of that coverage on to, even in a shell-game sort of way (see prior posts on large and small employers that purchase insurance). When the Administration announced its compromise for the relatively insignificant fully-insured market, it didn’t offer any compromise for the much larger set of religious self-funded plans. Instead, they announced an intention to figure out how to compromise:
The Departments intend to develop policies to achieve the same goals for self-insured group health plans sponsored by non-exempted, non-profit religious organizations with religious objections to contraceptive coverage.
In this past week’s Advanced Notice of Proposed Rule-Making (ANPRM), Health and Human Services (HHS) began the brainstorming process:
For such religious organizations that sponsor self-insured plans, the Departments intend to propose that a third-party administrator of the group health plan or some other independent entity assume this responsibility. The Departments suggest multiple options for how contraceptive coverage in this circumstance could be arranged and financed in recognition of the variation in how such self-insured plans are structured and different religious organizations’ perspectives on what constitutes objectionable cooperation with the provision of contraceptive coverage.
These options (beginning on page 16,507) can be summarized as follows:
1) Use drug rebates;
2) Use fees paid by the religious organization nominally designated for another purpose, such as disease management fees;
3) Use funds from a private, non-profit entity to be specified later;
4) Receive a “reinsurance contribution” fund rebate or tax credit (this only “works” for 2014-2016);
5) Use the federal Office of Personnel Management designate a national, private insurer that would offer this stand-alone coverage;
6) Give the national plan a “credit” so they wouldn’t have to pay their entire Exchange fee bill;
This is an incredibly weak set of ideas. These boil down into the following “pass the hot potato” funding sources:
a) The religious institution itself (ideas 1 and 2)
b) The third-party administrator itself out of profits (ideas 1 and 2)
c) The individual market via reduced reinsurance payments and/or the general US Treasury (idea 4)
d) An unspecified, wealthy benefactor (idea 3)
e) A benevolent, national, private insurer (idea 5)
f) Each of the individual Exchanges through reduced user fees (idea 6)
The administration is between a rock and a hard place here. It is worth working through the mechanics of each of the above ideas, however, to illustrate the full breadth of the problem.
Under healthcare reform, women’s preventive services will be covered at no direct cost to women. The Heritage foundation writes:
In addition, mandated coverage of preventive services with no cost-sharing will increase health care costs, since cost of services will simply be passed from the insurer to the patient through higher premiums.
This is only entirely true in the large group market. In the individual and small group markets, other health benefits may end up being cut to compensate.
The key is in the restriction that *every* plan (on the Exchange or otherwise) must meet a particular actuarial value, say 60%. What that means is that health plans must pay, on average, 60% of the cost of covered services.
Let’s assume that the Obama administration rules boost preventive service utilization up to 10% of total services (a nice round number). Let’s further assume that these services must be paid for at 100%. In order to comply with an overall actuarial value of, say, 60%, health plans will need to cover all non-preventive services at a rate of 50/90=55.6%.
In this manner, premiums for the 60% actuarial value plan don’t go up. What you have done is to take from the sick and give to the healthy. Specifically, the health plan can’t even pay 60% of the cost of other diagnostic imaging, cancer treatments, etc. They have to reduce their effective cancer coverage in order to boost the detection of that coverage via preventive services.
[Note: through indirect and highly speculative conversations, I understand that HHS may be considering an actuarial value model that is so crude as to not be able to capture the above effects. To the extent our federal government is not going to calculate actuarial values correctly with respect to this issue, then the Heritage analysis may end up being more correct in all markets.]
Earlier this month, HHS issued the final rule for the reinsurance program under the Affordable Care Act (ACA). I believe it will be highly controversial once it becomes understood. I’ll start with a backgrounder and then move to a short discussion of how this is a tax on states, how it may adversely impact employees of state governments, and how the government is proposing to selectively administer these taxes depending on the religious beliefs of those involved. This post is necessary background to fully understand the Administration’s proposed “compromise” on contraceptive coverage, which I am covering in a series of posts beginning here.