Problems with Probability

Why we don't know what we talk about when we talk about probability has been revisited by its original author Nassim Taleb in a recent publication on his Fooled by Randomness portal. Great claims are being made in this paper that perhaps we should ban the use of probability and sometimes the best discoveries seem to occur when we explore dynamics at their extremities. This might just be the case here as well.

In this short post, we take a look at the recent paper "Problems with Probability" published by Nassim Taleb.

A source of our failure

It is possible that we actually go wrong in school, that the source of our commercial failure begins in the academy of statistics. I am not saying that we should ban statistics although Nassim Taleb believes we might just be better off if we did, however if you are to teach probability theory, you will do well by simplifying it to make it easy to comprehend but at the same time you will instill the wrong kind of thinking in your pupil.

There are several levels of thinking (more than four for what it's worth), the raw probability for an occurrence of X is a slim chance of success or failure (depending on what you are looking for or trying to avoid), it is easy to understand but in a complex causal network it will fail us as a ruler for decision making.

Fooled by Randomness (click to read) | Nassim Taleb

Yet many institutions and certainly more than a handful of banks base their risk framework decisions on the probability of X occurring. Value at Risk which is a common measure of downside used by the banking community is fundamentally built on this premise of speculation. We also know for a fact that during the credit crisis, VaR did not effectively estimate the shortfall of capital banks would inevitably lack and then fail from.

The greater issue is not only can we accept that X is a range of outcomes but that our utility function and risk aversion for X will change as we move through a range of X's.  Our payoff, not even but our appetite for exposure may become entirely negative as we travel into the tail of the  distribution of X.  The correlation between utility and the outcome of X may not remain consistent either across the distribution of X.  In fact, it may not remain consistent overtime although this paper doesn't directly explore a time linked aspect of inequality.

In mathematics, Jensen's inequality relates the value of a convex function of an integral to the integral of the convex function. It has been proven that inequality appears in many forms depending on the context but simply the inequality states that the convex transformation of a mean is less than or equal to the mean after convex transformation.

Jensen's Inequality Explained | Jensen's Inequality

So where to next

Well throwing probability out of the window without a replacement may serve some people more than others. It may just furnish some of those managers that lead us into the Global Financial Crisis the freedom of thinking for endorsing another novel parameterisation function that suites one payoff and not another. A payoff worse than privatizing profits and socializing losses, if that is possible.

What can't continue is level [I] thinking and level [II] thinking is fraught with huge error as we know from experience. The diversity the US, Middle East, Europe, Japan and Asia have suffered from over the last twenty four months is telling us one thing; that the door of opportunity is opening for work to address a level [IV] type of thinking and in a practical manner.

The world is crying out for the evolution of risk based statistics for businesses and that doesn't mean refining accuracy but creating a mechanism for applying a dimension for connecting value, utility, payout and randomness (all functions are random and clustered, their clustering is a distribution) and, we need to track this through time.