Search
× Search
Saturday, December 21, 2024

Archived Discussions

Recent member discussions

The Algorithmic Traders' Association prides itself on providing a forum for the publication and dissemination of its members' white papers, research, reflections, works in progress, and other contributions. Please Note that archive searches and some of our members' publications are reserved for members only, so please log in or sign up to gain the most from our members' contributions.

OUTLIERS DETECTION

photo

 private private,

 Sunday, February 1, 2015

What are good methods to detect outliers in technical analysis indicators? I mean I don’t have any prior information about the distribution of those indicators and I need a method that doesn’t depend on distribution. Are modified z-score or adjusted boxplot goods?


Print

7 comments on article "OUTLIERS DETECTION"

photo

 Jeremy Roseberry, President at Granite Capital, LLC

 Tuesday, February 3, 2015



Have you tried normalizing the indicators? It won't help you detect outliers but it will normalize your indicator to automatically adjust to changing volatility levels which can be helpful when optimizing and backtesting.


photo

 Larry Kase, Financial Analyst and Hedge Fund Principal

 Wednesday, February 4, 2015



As one baptized into the business during the age of dinosaurs I lean toward the traditional basic principles whenever oddities occur. As mightily as I attempted to rationalize adjustments to accommodate outliers numerous times in the past, post incident experience reminded me that there are no outliers. Prints are prints. We are not permitted to normalize indicators. Doing so corrupts the data rather than purifies it. A perceived outlier may actually represent situations such as island reversals or lead into continuation gaps or become part of diamond reversals or many other incidents. However, they are there and must be recognized and incorporated into the work. Failure to do so imposes our personal judgment into the importance or lack thereof. When an incident occurs, no know that it is an outlier since no one knows what happens next. Assuming forecasts can be made based upon judgment on a look back assumes that all such incidents will produce the same result in the future. Experience and history instructs us otherwise.


photo

 Graeme Smith, Investment Manager at The Tourists Portfolio

 Saturday, February 7, 2015



The first step has to be to get an idea of the distribution. Whether it is approximately normal or log normal will make a big difference. If the outliers don't affect the mean & sd too much you can probably just use these. Alternatively I think the robust method is to take the median, plus or minus 2.5 times the distance to the 25/75 quantile. However neither of these will work if your data is, for example, log normal and you haven't already treated it. So the first step has to be understanding the distribution.


photo

 Larry Kase, Financial Analyst and Hedge Fund Principal

 Saturday, February 7, 2015



Thanks Graeme, absolutely true. Interestingly, the numbers you mentioned are in the neighborhood of those found useful. Also, it was quickly determined that not only was understanding the distribution essential, the distribution segment needed to be relevant to the asset management policy and style. in this case short time slices were irrelevant to the primary strategy employed. Upon further study the comfort zone for trading was larger than expected. Eventually, distribution covering a minimum of 3 years was the election. The election followed a long slog of trial and observation largely incorporating the elements and considerations you mentioned. Since adoption the comfort level improved consistently solidifying the relevance for the particular purpose. The mathematical and statistical computations turned out to be the easier aspect of the pursuit. The difficult challenge was suitability and fulfilling the expectations for the work product. The math will do as told. It produced the results and turned to me saying, "okay, I did my part. What are you going to do with it." That was where the real work began. .


photo

 Graeme Smith, Investment Manager at The Tourists Portfolio

 Saturday, February 7, 2015



Also Daniele, carry out the statistical tests others have advised. But preface your understanding of the results by saying "in a normal universe this would apply". In general it won't. The tests are still useful, but can be dangerous since most financial data doesn't exist in what would be considered a well behaved "normal" universe.


photo

 private private,

 Saturday, February 7, 2015



Thanks Graeme and Larry for your views.

I'm testing different outlier detection methods and i'll choose the most economical one.


photo

 Larry Kase, Financial Analyst and Hedge Fund Principal

 Saturday, February 7, 2015



Regarding technical analysis the old school always comes out in me. There are no outliers. All prices count. Technical analysis is more art than science. The number of tools increased exponentially over the past several years but the first stop in technical analysis is the chart reading in the simplest form possible which includes the absence of moving averages. Then the additional layers are added in order to better view and understand what may be occurring. Regarding statistical analysis, there are measurable anomalies. Among the few considered significant to the work here is entry into tails and the dynamic stop alerts above and below current price. The points are noted and then back to the chart. Again, there is that nettlesome issue of selecting the relevant time slice. Statistical behavior is easily measured. Then there is the matter of what it all means and how to apply it. Be careful with the testing. The measurements are relatively easy. However, placing the data within the context of orders that can and would have been executed is critical. Virtually all methods fail to account for this essential element.

Please login or register to post comments.

TRADING FUTURES AND OPTIONS INVOLVES SUBSTANTIAL RISK OF LOSS AND IS NOT SUITABLE FOR ALL INVESTORS
Terms Of UsePrivacy StatementCopyright 2018 Algorithmic Traders Association