Sorry for interruption, but the discussion starts to sound quite like something I've tried to discuss some time ago in this group: about the premises on which trading decisions are based. Genetic algos, machine learning, some years ago neural networks, artificial intelligence and so on are not new; only the name change according to the latest trends in vogue. I wonder why almost no quant wants to even try to understand what actually happens in the market and then trade along with these processes, and prefer to replace this relatively simple task with designing complex methods.

Besides other issues, the result of such an algorithm will not necessarily correspond to any live market process. Therefore you won't be able to tell a temporary drawdown from end of life of a strategy, because you've never known why it brought you money.

Alex Krishtop three of your comments ago you mentioned exit plans and the fact the target price did not qualify as an exit plan. I totally agree with you on that fact and many of the others you have stated. We appear to be on the same page on a number of strategies. Our trading group uses a TSL (Trailing Stop Loss) as our exit plan it allows price to move downward a certain amount before triggering the exit of the position. It takes some of the volatility out of the price movement and allows the position to move thru some corrections without premature exiting to make profits.

Alex, I am with you regarding simple systems, but as the market evolve one needs to tweak a system. One might also want to research new ideas. If a system starts deteriorating it might be because one of indicator in it started giving more false signals. One of the indicators I used to relay alot on many years ago was NYSE TRIN. In the past anything above 1.5 used to be bearish especially if the value is elevated and is rising. Nowadays, the market can keep on rising with a 1.8 reading and rising.

So, yes you would want an light system, but as you assess it's performance you want to have tricks in your back-pocket to try to fix it by possible replacing some of the indicator with other or with new thresholds or new ways of looking at the data.

Muhammad, the "market", or the price time series are the result of quite a complex superposition of actions by various market participants. At certain times some of them are more active, at certain time — others. If you try to suggest a single strategy which aims to describe that final superstition in all its complexity, then indeed you need to "tweak" it somehow because of the changing proportions of market participants activities.

However we can go another way: instead of trying to figure out a "one size fits all" strategy, we construct many simple strategies, each of them describing a single process in the market, and then run a portfolio of strategies. If we are adequate with our hypotheses about market participants and what they do, and if we understand how to rebalance such a portfolio to keep it adequate to changes in the market regime, then we can achieve a remarkably stable performance.

And a brief note on indicators. When we use any indicator, we absolutely need to understand what it actually represents, and not in meaningless terms like "overbought", "oversold" or "trend", but in terms of real, physical processes going on there in the market. The we use only those indicators which help us describe the very process which takes place in this particular market, at this particular time. Within this r&d paradigm we simply don't have "false" or "true" signals.

Yes Alex, I think we are in agreement. And in reality of course we don't take the signal from one indicator alone. Many indicators need to agree, the daily trend also needs to agree.

When I gave the example of the TRIN it was just to make the discussion more concrete.

But assuming I have a system that was doing well, but most recent results have been deteriorating. If I were to just test dollar results I would just ditch the entire system for something else vs if I had a way of testing it at the indicator level. Like in removing one indicator and testing the past results to most recent, without TRIN for example, I might find out past and recent results are similar but with it, past results are better than the most recent.

Discussions always tend to move to the area participants are more comfortable with. I guess ultimately the author of the question should pull it back to the original subject and that is an art by itself. You don't want to push too hard but rather guide the discussion to encourage more participation.

I will reiterate the original question, how can we measure the participation of each indicator toward the success or failure of a trading signal? Once you know your most "potent" indicators you might be able to discover others or at least it would give you more insight towards your system. It might also make it easier to design other systems.

@Muhammad, testing signals is relatively easy. But is should be done on a representative market sample and over a sufficiently long time period to give it statistical significance. Surely testing over a couple of years over only a few securities is not enough.

What is the value of signal A compared to signal B?

Say you take the whole S&P100 over the past 20 years as testing ground. It would be representative and of sufficient duration. This would generate a price matrix P of 500,000 EOD prices, or on a minute time scale, a P matrix of size: 195,000,000 prices. Are you ready for some math, because you will need it.

Put both signals in the same trading strategy: Σ(H(signal A + signal B).*ΔP) and perform your long term simulation. It will be sufficient to comment out a signal's buy and sell orders to produce: Σ(H(signal A + signal B).*ΔP) - Σ(H(signal B).*ΔP) = Σ(H(signal A).*ΔP). It will also say if signal B added some value or not.