@Cameron, all 3 strategies analyzed your past 5 years and with their outcome being judged equivalent could be expressed as: Σ(H(strat1).*ΔP) ≈ Σ(H(strat2).*ΔP) ≈ Σ(H(strat3).*ΔP). However, the average profit per trade is not equivalent and not equally significant.

To get the average profit per trade is relatively simple, you just divide the generated profit by the number of trades: Σ(H(strat1).*ΔP)/10, Σ(H(strat2).*ΔP)/250 and Σ(H(strat3).*ΔP)/1000. This will totally change the picture and the considerations as to which strategies should be put forward.

I would go for a mix of strat2 and strat3, totally ignoring strat1 as statistically insignificant. However, for strat2 and especially strat3, I would evaluate the impact of commissions on their respective design since strat3 will have 4 times more commissions to pay compared to strat2 and 100 times more than strat1.

Which one would I trust the most as representative of what to come? Strat3 with its much greater number of trades, more than enough to say that the results are not by chance alone and that there is something of value there. It is more difficult to design strategies with large number of trades and make their average significant.

Should commissions be insignificant, I would go for something like 2/3 strat2 and 1/3 strat3, or simply go all out with strat2 based on the limited data presented.

But that is just my humble opinion.

Good trading.

1000 trades option. No way algo's can predict market movement over a longer time frame since all fundamentals have been destroyed by the FED since 2009. Unless of course you are part of the cartel with inside information. Now if you know were the price imbalances are then you can use those 1000 trades very effectively. https://youtu.be/_WrMWK-FJzU Edward Griffin inside track.

Statistically, compute non-parametric confidence bands for Sharpes or whatever statistics you use. These bands will be wider for fewer samples. Take some confidence level - 95% for example - and take values of tail Sharpes at those confidence levels. Then your question converts to how much money am I willing to give up for tail Sharpe improvement. Simple answer will be if you use mean returns instead of Sharpes, in which case this becomes direct $ comparison.

Lets look at this problem differently. Suppose you had a business and you had 3 options, cater to very few customers (2/year) a lot of customers (200/year) or somewhere in between (50/year) what kind of business would you rather be in ?

I would rather not be in a position that losing 1 customer would have caused a 50% loss in business in that one year. Assuming all costs to be the same, I would rather invest in strategy that allows for a greater probability for correction/recapture of a losing trade. With that in mind, assuming costs are the same and that past performance does not guarantee future performance, I would go with 1000 trades over 5 years. As far as commissions go, if the total number shares traded are the same, commissions should have no bearing on the decision. One other point of interest that should be considered if the above condition is true, market impact of the individual trades would be less as more trades are executed with smaller size per trade.

You should consider using all of them in parallel while applying an appropriate risk management. In simple words the less statistically significant the strategy is the less fund would be for it. Naturally a strategy that made 10 trades over 10 years is statistically questionable...

Cameron, considering running all three strategies. From your brief description, it appears that each one would address varying types of volatility, the timing of which is unpredictable. That is precisely what I do with my scripts. Best...

Statistical significance as a way to quantify the goodness of fit is probably meaningless on a sample of 10 trades. I would use a different approach to identifying the viability of these strategies. For example, it is entirely possible that the 10-trade strategy reflects a set of very profitable opportunities that arise very infrequently - the designer of the strategy who knows what went into the design would be able to look in detail at each simulated trade and determine whether that's the case. Another point to consider is where the profits of the 1000-trade case come from: are they well dispersed indicating something meaningful has been found or are themselves somehow concentrated with majority of PnL coming from a few large positives.

Overall I prefer a collection of strategies with profitable yet infrequent trades, where I'm convinced of each one's relevance, over a more generic "shotgun" approach with thousands of potentially profitable transactions. The key here is personal conviction and it may be different for each one. Highly concentrated businesses are not unusual but they do require a high touch approach compared to more mass market strategies.

Assuming either zero transaction cost or that the transaction costs are accounted for in the performance, then you would have the most faith in strategy 3 since it has the most statistically significant performance. Assuming they all had the same volatility and correlation to each other, mathematically your best option would be to invest in all three strategies and weight that investment by the square root of the number of trades, ie sqrt(10):sqrt(250):sqrt(1000). In effect your portfolio should be 2/3rds strategy 3, 1/3rd strategy 2, and a tiny bit of strategy 1.

Another question is how you account for risk (if at all). I believe 'true' Sharpe equals total return divided by total risk. The latter equals daily volatility times sqrt(N) where N is the number of days your money is in the market since there is no risk while your capital is in cash. Then your strategies may have very different N and hence different Sharpe even if they have the same total return. Bootstrapping your returns may give you some sense of whether your strategies are really close in terms of total return. But in the end of the day I agree with practitioners suggesting splitting your capital among several comparable strategies.

Very good explanation Dr. Alec!

So many people argue that statistical significance cannot be inferred from a sample of 10. This is a bad argument. If the distribution spread is relatively narrow it can be very significant. I.e. If I make 10 trades and they each yield a return of in between 3.9 and 4.1% what is the probability that trade 11 will return -1%? In any event if the Sharpe ratio is the same and the populations normally distributed then it really doesn't matter bar the comments of Alec Schmidt about "true" Sharpe ratio. Perhaps you could consider that if you don't get a better risk adjusted return with increased risk then why accept the increased spread in the distribution?

Alec - I like your idea of true Sharpe, thank you.

Graeme - Why do you say "square root of the number of trades"? If that is a long answer perhaps you could direct me to a site for further reading.

Lasse - I too appreciate small samples whilst observing that most other people do not. However I think differently about them. In regards to your question, "what is the probability that trade 11 will return -1%?" I would answer that it could be reasonably high and that is precisely the signal that the market state has changed. In other words some people use large samples to try to force stationarity. That may be futile. But using small rolling samples, alongwith an assumption of non-stationarity, you might find sample lengths that consistently pick up profitable signals of changes in market state. These are temporary-stationary processes.

Thanks everyone.

Under the hypothesis of normale distribution on the profit per trade, the sharpe ratio follows a student distribution, whatever the number of observation, we can always compute the confidence level. If only the iid hypothesis is hold, it converge to a normal distribution, and a big number of observation is needed to compute the approximate confidence level.

Cameron - "square root of the number of days" (not trades). It is usually assumed that returns follow random walk. Its standard deviation is proportional to sqrt(time). If we deal with daily returns, we can measure time in number of days. Note that for consistency, total returns for various strategies should be estimated for the same period of time. If some strategy keeps money in the market at the end of this period, you should transform them into cash. I discuss some aspects of back-testing in my book:

www.amazon.com/Financial-Markets-Trading-Introduction-Microstructure/dp/0470924128/ref=sr_1_3?ie=UTF8&s=books&qid=1304549595&sr=1-3

which has rich bibliography on the subject.

* Return = Asset Turn * Margin so trading volume should not matter - does anybody care when stock picking?

* never look at the investment standalone

How does each investment correlate to the rest of my assets?

3) if you want to look standalone, then apply a PCA model to extract factors driving risk/return profile. Did the low turnover model ride a single factor wave?

I saw profitable trading system (fx/futures markerts ) which made 100 & more deals per day .