12/5 – Potential shortfalls in testing results

I would figure that a 57% annual return [see yesterday’s post] is excellent and that every single hedge fund manager would be happy with that.  [But maybe hedge fund managers would only be happy with that if that return was made on a sufficient amount of trading capital.  Maybe if the trading capital was only $100k, then the manager would expect a far greater return.  Remember that large trading capitals often restrict the type of strategies that a fund manager can use.] However, now I am going to play devil’s advocate and argue why these testing results are not a realistic estimate of live trading results:

  1. Testing assumes that every single session is traded, and that each session is traded from start to finish.  This is not realistic.  Traders need holidays, and sometimes breaks during the day, or some other appointments might crop up.
  2. News Impact – testing assumes that targets and stops are honored equally during news releases.  In reality this is not the case, because of widening bid-ask spreads.
  3. Slippage in some events is not taking into account in testing- e.g. widening of spreads during daily rollovers.Testing
  4. The benefit of being able to “look-back” benefit can creep into testing exercises.
  5. Psychological factors at play in live trading cannot be simulated during testing.  Most of the time such factors would have a negative, rather than a positive impact.
  6. Testing has no limit for positions open simultaneously.  With this strategy there will sometimes be cases where multiple setups occur at the same time.  A recent example was GBP strength.

Call me negative, but the above are valid reasons for why live results are likely to be lower/smaller than the testing results.  (Note I have already thought of some ways to quantify and/or mitigate the above issues.)

However I can also think of two reasons for why the live trading results could in fact be bigger & higher than the testing implies:

  1.  I can trade a larger number of instruments – for now, I have tested only 31 instruments.  How many additional instruments can I find that lend themselves to this trading strategy?  By lending themselves I am basically referring to an instrument’s typical transaction costs relative to the size of its price moves.  Secondly the instrument must be trading 24 hours a day throughout the working week so that the EMA’s are not distorted by market closures (for example, the DAX only trades from 7am to 9pm, and then experiences a gap at the start of each session).  What instruments could these be? Additional cross fx rates such as the Swedish Krone, the Norwegian Krone, the Mexican Peso, the Brazilian Real, the Hong Kong Dollar, the Chinese Yuan, Natural Gas, treasury bonds – the list goes on.  The cut-off is a maximum 10% transaction cost.  I may need to shop around to find the best spreads – e.g. FXCM offers 3.5 on Silver, whereas CMC offers 2.5.
  2. In live trading, more attention and concentration is given to the charts – it’s likely that more setups will be spotted in live trading than in testing.

Ok, so I thought of variables that are likely to decrease and increase the performance in live trading over testing.  All in all, what will be the net impact?

Let’s go and see!

This entry was posted in Testing, Trend-Following Strategy. Bookmark the permalink.

2 Responses to 12/5 – Potential shortfalls in testing results

  1. Diane says:

    Great post!! Diane

    Liked by 1 person

  2. Pingback: 15/6: Strategy Development & Testing: Nitty Gritty Stuff | Trick or Trade - the story of a currency trader...

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.