Over the last week, my new “testing engine” produced a sample slightly over 3,000 test trades. In this post I am detailing my findings and thoughts for taking things forward.
You are likely guessing (correctly!) that I will probably not stop writing about trading strategy testing any time soon!
With that in mind…..
Building an Excel-based testing engine
Since that time, I have developed a basic testing model in Excel that enables me to back-test, on a large scale, an initial version of a trend-following trading strategy. My Excel knowledge is reasonable, though I tend to use primarily front-end formulas rather than Visual Basic programming. I have built the models such that within one Excel file I can back-test one year’s worth of historical prices in either a long or a short direction. Each file runs to approx 40MB in size.
My two data sources are two brokers with whom I have accounts – thus, FXCM for Contracts for Difference (CFD) products [not available to US residents], and TradeStation for futures data. It took me a good fortnight to translate the trading strategy (in its current format) into Excel formulas. [Aside from trading in the oil futures for a couple of hours each day, my work days were devoted to the spreadsheets.]
Top efforts have been spent to avoid common back-testing traps such as over-fitting, ignoring transaction costs, ignoring the bid-ask spread, or accidentally using data from the future to make a current decision. That said, I can identify two weaknesses in the existing approach. Firstly, I have not provided for scheduled Tier-1 news events in the testing, such as NFP, central bank meetings, UK 9.30am data, Weekly Oil Inventories and so on. Ideally I will want to introduce a black-out period for opening a new trade within x minutes of a news release, as well as close an open position y minutes prior to a release. My assumption is that widening of bid-ask spreads during news releases will in the long-run make trading results worse. Secondly, I have not checked the accuracy of the data that i am using.
On the plus side, a colleague provided me with an MT4 script to allow me to quickly plot all executed trades onto the MT4 chart, simply by using entry and exit points listed in the testing spreadsheets. This serves well for spotting errors and, later on, for refinement ideas.
Initial Testing Results
After being happy with the models, I then spent another whole week running the models through a range of instruments. As usual, this took more time than I had expected. However, now that I have come out the other end of the first “batch” of testing, here are the results that were produced:
The three tables relate to the same underlying data, but simply slices the data in different ways. The top table breaks the trades down by asset class, the second by the product used for the trades (either CFDs or futures contracts), whilst the third breaks the trades down into the years that the back-tests were completed in [there was no deliberate approach for choosing the back-test periods, other than trying to cover a mix of years and market environments].
The results are expressed in units of risk [for more discussion on units of risk, search on the Risk Management category] and a single unit of risk has been used for each trade [this is a further questionable assumption and may not be realistic in live trading]. The P&L is the final result of the testing. The transaction costs are equal to the bid-ask spread and commission – the impact of the strategy is the difference between the final P&L and the transaction costs. The overall return on risk (not shown above) is negative 1.65% (-53.18 divided by 3,229).
The total sample comes to 3,200 trades – not a small number. That said, what is small and what is big, really depends on who one asks!
The data above indicates that the strategy’s parameters (subject to methodology weaknesses mentioned above] have a positive impact on the trading results. This is because the results seem to be more favorable than a random selection – which in the long run, should produce losses equal to the transaction costs. However this could simply down to variance within the sample – I have not determined whether the idea that the strategy adds value is statistically significant – it probably is not, given that the strategy’s ‘edge’ is barely equal to half of the transaction costs – hopefully one of the maths guys here at the hedge fund will help me do that.
The next step
The present combination of parameters does not produce positive results. Hence, I will need to change the parameters. The trend-following strategy has three main components for assessing the setups – these are:
- Definition of Trend – when to look for long trades, when to look for short trades, and when to stay neutral
- Within trending conditions, the identification of a pullback.
- In the case of such a pullback, identifying the situation where the pullbacks appears to have finished and the trend to continue.
Each of these three components contain possibilities for amending the setup parameters.
Additionally, there is the question of execution and trade management – such as, if the setup occurs then where is the entry placed, where is the target placed? And the stop? And how will the stop be managed? These will also influence the result of the test trades.
My gut feel is that the first point on the list has the most potential for making refinements producing significant impacts. I will start on that this coming week – am planning to plot a large number of the test trades on the MT4 charts to help me judge whether I would feel comfortable trading the strategy’s suggested entries in real time. Additionally, I will ask why the strategy may miss other price scenarios which I would be comfortable using as trade signals. I will then try to formulate my thoughts into more Excel code and re-run the models for some of the instruments over the same time periods.
In doing so, I will be careful to avoid over-fitting. Hence I will not make a hundred different adjustments until I can finally see a positive result. If after making a few refinements, and only where those seem reasonable and logical, the results are still not positive, then I will have to discard that particular approach and work on a different idea.
As an illustration – the above setup deals only with entering into a trend using a pullback. There are of course also other techniques for trend-following, such as entering on the break of swing highs/lows. This would be a different approach, the models would be reworked significantly and I’d be back to square one.
The ideal/target outcome
Hopefully the next few weeks will pan out something along the lines of:
- Back-Testing (the in-sample) produces positive net returns and is statistically significant
- Out-sample confirms the existence of the edge
- Demo Trading in ForexTester confirms edge [at this point I would be entering into the first column of Lance Begg’s diagram]
- Strategy is deployable in a live trading conditions and performance in line with the back-tested results
- Meanwhile, hopefully the CL (crude oil) futures trading will continue to be as profitable as it has been in the first six weeks, and I can slowly increase the position sizing
I will update my progress on the blog here as usual.