Saturday, August 14, 2010

What are we to do with Sharpe ratio?

I wrote several times before how useless Sharpe ratio is for certain types of strategies: see here and here. Not only is a high Sharpe ratio quite useless in telling you what damage extreme events can do to your equity, a low Sharpe ratio is also quite useless in telling you what spectacular gain your strategy might enjoy in the event of a catastrophe. I came across another brilliant example of the latter category in the best-selling book "The Big Short", where the author tells of the story of the fund manager Mike Burry.

Mike Burry started buying credit default swaps in 2005, essentially an insurance policy on mortgage-backed securities, betting that there will be widespread defaults on mortgages. Of course, we now know how this story would turn out: Mike Burry made $750 million in 2007 alone.  But there was nothing but pain for the fund manager and his investors in 2005-2006, since they had to pay an annual premium of 8% of the portfolio.  Investors who measured the performance of this strategy using Sharpe ratio, without knowing the details of the strategy itself, would be quite justified to think that it was an utter disaster prior to 2007. And indeed, many of them lost no time in trying to pull out their investments.

So what are we to do with Sharpe ratio, with its inherent reliance on Gaussian distributions? Clearly, it is useful for measuring high frequency strategies which you can count on to generate consistent returns every day, but which has limited catastrophic risks. But it is less useful for measuring statistical arbitrage strategies that hold positions over multiple days, since there may well be substantial hidden catastrophic risks in these strategies that would not be revealed by their track record and standard deviation of returns alone. As for strategies that are designed to benefit from catastrophes, such as Mike Burry's CDS purchases or Nassim Taleb's options purchases, it is completely useless. If I were to allocate my assets over different hedge funds, I would be sure to include some funds in the first category to generate cash flows for my daily needs, as well as funds in the last category to benefit from the infrequent black-swan events. As for the funds in the middle category, I am increasingly losing my enthusiasm.

18 comments:

Alexandre Rubesam said...

Hey Ernie, how do you mean you're losing your enthusiasm about stat-arb strategies with horizons of several days?

Jez Liberty said...

completely agree, the Sharpe ratio has many flaws.

One of them is the use of arithmetic returns, which can give false view of multi-period profitability.

Same thing applies to the Information Ratio (which is an improve version of the Sharpe ratio - choice of benchmark).

Here is an improved version using geometric return: the Geometric Information Ratio, which gives a better performance measure for trading strategies and managers (as seen here)

Jonathan Shore said...

Penalizing deviation in the upside (as the sharpe ratio does) is ridiculous. I have a strategy that takes advantage of extreme situations, so the return has periodic jumps (upward) and little downside risk.

Sortino does a much better job of providing a useful picture of the upside of the strategy than Sharpe.

Unfortunately, like the Sharpe ratio, it makes no allowance for possible risks, only those that have been observed. Still, I would like to see Sortino used in place of Sharpe as a metric.

Ernie Chan said...

Hi Alexandre,
I am wary of strategies that hold position for multiple days because it has been shown in recent years that they are highly vulnerable to tail risks as well as the non-stationarity of fundamental factors.
Ernie

Ernie Chan said...

Jez,
Thanks for telling us about this measure!
Ernie

Ernie Chan said...

Jonathan,
Yes, if everyone can agree to switch to reporting Sortino ratio instead of Sharpe ratio, it would be a great thing. Unfortunately, as long as most people are talking about Sharpe ratio, we still have to report Sharpe ratio for benchmarking purposes.
Ernie

Edouard d'Archimbaud said...

Sharpe ratio isn't useful for measuring every high frequency strategy performance. A condition is having a symmetric gain distribution.

It wouldn't make sense to use a sharpe ratio with a high frequency trend follower because trend followers usually have a strongly asymmetrical gain distribution (take a look at Bouchaud's paper).

Anonymous said...

Hello Ernie,
In your book you used cumprod(1+retvect)-1; in calculating the max drawdown. I believe this would be known as the cumulative return. You also used cumsum(pnl(testset)) to just simply add the returns in your pairs trading code. I was not familiar with just summing returns, especially daily returns. I notice these different methods can give very different results when compared with each other, and also they are very different from the basic profit loss return calculation. Why do you use sum of daily returns? and when is it proper to use these different methods. Thank you.

Ernie Chan said...

Edouard,
Thanks for the reference to the paper on trend following strategies. However, I would like to make 2 points:
1) The paper does not refer specifically to a high frequency strategy. In fact, their simulation is done on daily FX data.
2) Even assuming that per period returns on HF trend following strategy also follows such skewed distribution, it is unclear that the daily return (accumulated over many intraday period returns), will show the same skewness. For investors, it is the distribution of daily returns that matter, not that of the intraday returns.

Ernie

Ernie Chan said...

Hi anon,
Cumulative compounded returns should be used when one is holding a position without rebalancing over multiple periods (e.g. days). It should also be used if one is determining the order size (or market value) based on Kelly's formula, i.e. increasing market value when equity increases and vice versa.

Cumulative simple returns should be used if you exit your position at the end of every period, or if you rebalance your position to keep the capital the same at the end of every period, or when you keep the order size/market value the same at the beginning of every period irrespective of equity (i.e. not using Kelly formula to size the order/market value).

I use the cumulative simple returns for a lot of backtests because it relieves me from explaining whether I follow Kelly sizing, and if so, at what leverage.

Ernie

Alexandre Rubesam said...

Ernie, do you have any references related to your comment?

I run some pairs strategies with horizons of several days and I have noticed that kind of behaviour from time to time, especially when there is a big rally. I've tried many ways of heding this risk but to no avail. Do you have any ideas from your experience? Thanks, great blog!

Ernie Chan said...

Alexandre,
You can read the book The Quants about the meltdown of many quant funds in 2007.

I believe there is no cure to this illness if one holds multiple days, except diversification of strategies and assets.

Ernie

Jesse said...

Thanks for answering our questions?
I have noticed that when calculating the Sharpe ratio you include every data point of the pnl vector, even though at times pnl(t) might be zero because there is no trade open. I am wondering if under any circumstances, is it legitimate when calculating the Sharpe ratio to only average in days where we have a return. After all, the reason for not having a return in this case is because a position is not actually open, we are on the sidelines waiting for a signal to enter.

Ernie Chan said...

Jesse,
We should indeed include zero position and zero return days in calculating Sharpe ratio. After all, the average annualized returns will most definitely include zero-position day, otherwise it would not be representative of the total return of the year. As average returns go, so go Sharpe ratios.
Ernie

Jesse said...

Hello Ernie,
On the subject of calculating returns for a market neutral strategy, in your book, in the section regarding the calculation of Sharpe ratios for a market neutral strategy, there is the following formula:
netRet = (dailyret – dailyretSPY) / 2 it says we should divide by 2 because twice as much capital is used. (long position and short position)
A couple pages later on the subject of Pair Trading of GLD and GDX the pnl=sum(lag1(positions).*dailyret,2) ,So these returns are added but they are not divided by 2. I see this will not change the Sharpe ratio calculation, but it does change the cumulative return by a factor of 2. This pairs trading code is calculating these returns with only 1 price per day (the closing price), so the program must be assuming we are holding these positions overnight. I want to know if there is some reason we don’t need to divide these pnl’s returns by 2?

Thanks again

Ernie Chan said...

Hi Jesse,
pnl=sum(lag1(positions).*dailyret,2) is a calculation of daily P&L, it is not a calculation of daily return. The daily ret is pnl divided by total capital, which in this case is 2.
And yes, the strategy I backtested assumed holding position overnight.
Best,
Ernie

Anonymous said...

Alexandre,

Some people think that upside volatility in the observed sample suggests there will be downside volatility in the future data you haven't seen yet. If you believe this then this feature of the sharpe ratio is sensible gives a conservative approach.

RAJEEV SINGH said...

This is very nice article and providing very important information about trading and also focused on strategies for trading, i think which is very helpful.