0

I noticed that some traders use backtesting to evaluate their trading strategies. What I don't understand is: how do they separate the predictive power of their trading strategy from the general market trend when interpreting the backtest results?

For example, long-only trading strategies will likely show good performance during rising markets, even if some of those strategies have no predictive power at all. Similarly for short-only trading strategies during declining markets. There must somehow be a way to remove the effect of market trends if the predictive power of a strategy is to be measured. I imagine that the real life situation is more complex than what I have just illustrated, because some strategies involve both long and short positions (at different times).

Flux
  • 17,043
  • 8
  • 68
  • 124
  • 1
    Using a long enough amount of time should alleviate that - test 50 years, not only 5. – Aganju May 22 '21 at 15:51
  • Does this answer your question? How can I determine if my stock picking performance was due to luck or due to skill? – nanoman May 22 '21 at 17:17
  • @nanoman No, this question is considerably more focused on the process of backtesting. Considering the nature of Bob Baerker's answer to this question, I would say that this question is not a duplicate of the one you are suggesting. – Flux May 22 '21 at 17:24
  • @nanoman By the way, I was the OP of that question too. I have carefully read all the answers there. The answers there are not specific to trading and backtesting. – Flux May 22 '21 at 17:26
  • I think my answer to that question applies here. True, that question assumed a real trading track record. In the case of backtested performance, the usual caveats apply (evaluation should be done on out-of-sample data that were not used to train the strategy). I am suggesting that you could use the backtested results in this question in place of the actual results assumed in the other question. As long as the backtest is reliable, the analysis is the same. – nanoman May 22 '21 at 20:40

3 Answers3

1

I'm not sure that I agree that a strategy has 'predictive power'. That's probably a separate debate which I'm going to sidestep.

Long strategies perform well in up markets and short strategies perform well in down markets. So either you have to have something that suggests what type of market you're in (easier said than done), or you have to have a mix of long and short strategies, or you have to have something that is agnostic to direction (trading volatility).

FWIW, 25+ years ago I went through a technical analysis phase with a group of clever people. I'll spare you the details/conclusions but one thing we all had in common we had was that we all used a software program that could backtest any strategy that we could formularize.

The backtesting provided 3 sets of stats: long only, short only, and both. Now while those results didn't quantify the amount that the general market trend contributed, it surely indicated its affect.

Bob Baerker
  • 76,304
  • 15
  • 99
  • 175
  • "we all used a software program ..." — MetaStock? Did you have a successful technical analysis phase? – Flux May 22 '21 at 16:20
  • LOL. BUSTED! I owned the first release of Metastock and a good number of the subsequent upgrades. Another good program was Technifilter which enabled you to run your trading strategy against a database, providing a list of stocks each day that met your screening criteria. In terms of P&L, I did not have a successful technical analysis phase. Nor was it unsuccessful. However, it did provide a lot of market awareness and was the foundation of my evolution into other strategies. – Bob Baerker May 22 '21 at 16:35
  • Are these backtesting software useful for those who are not doing technical analysis? – Flux May 22 '21 at 16:54
  • "In terms of P&L, I did not have a successful technical analysis phase ..." — Do you happen to know the reason(s) for your lack of success in technical analysis? What made you abandon technical analysis? – Flux May 22 '21 at 16:59
  • I can't imagine how backtesting software could be used for anything other than backtesting TA. I still use Metastock but not for its intended purposes. I've subscribed to Thomson Reuters EOD data for 20+ years and it is stored in Metastock files. I use a macro driven routine that exports the data and updates 75+ spreadsheets in a few minutes. I'd bet that there are more expedient ways to do this nowadays but I have no need to fix what ain't broke. It wouldn't be worth the time involved. – Bob Baerker May 22 '21 at 17:26
  • In order for TA to work, you have to form a cohesive trading strategy that has an edge. We never found that. Does one exist? It might, but given that we never found one, I dunno. I had to look elsewhere for that. Personally, I think that they've built an industry based more on hype than performance. The same holds true for trading courses, newsletter subscriptions, etc. – Bob Baerker May 22 '21 at 17:27
  • So are you using an old version of MetaStock? What version? Did you learn MetaStock's "formula language" to do all the analysis you need? I'm curious because I may someday want to try out these things. – Flux May 22 '21 at 17:44
  • I don't know which version I'm using but it is ancient. I have several from 1.0 up to either 7 or 9 (?). My last upgrade was 15-20 years ago. Yes, I learned MetaStock's "formula language". It wasn't difficult. But again, that's what existed 15-25 years ago. I have no idea what evolution has occurred since or what current software offers. At that time, Supercharts (a distilled version of Trade Station) was a more sophisticated analysis program so you might look at that as well as Trade Station if you head down this path. – Bob Baerker May 22 '21 at 18:08
  • Thank you for sharing your experiences. "I went through a technical analysis phase with a group of clever people" — Was it in-person, or was it some kind of online community/forum? – Flux May 22 '21 at 18:59
1

There must somehow be a way to remove the effect of market trends if the predictive power of a strategy is to be measured.

This is the meaning of Alpha. Your returns minus the market return equal Alpha.

As Larry Page wrote in announcing Google's new holding company, Alphabet:

For Sergey and me this is a very exciting new chapter in the life of Google—the birth of Alphabet. ... We also like that it means alpha‑bet (Alpha is investment return above benchmark), which we strive for!

EDS
  • 428
  • 3
  • 10
Orange Coast- reinstate Monica
  • 7,990
  • 11
  • 22
1

Yours is a question that is rampant in the industry, especially in quantitative finance. Let me put it this way: no company will ever offer a product with poor back tested results.

Any predicative power of a backtest, in my opinion, must be tested in different market environments so as to separate the power of the strategy from the prevailing market trends at the time. For example, can you get a picture of how the strategy did in periods of both rising and falling rates? How about periods where certain factors (growth/value/dividend, size, etc.) or certain sectors or subindustries (finance/consumer discretionary/technology, etc.) did well?

EDS
  • 428
  • 3
  • 10