Yes, I think analytics and big data have ample opportunity to use in algorithmic/quant trading. They can be used in various ways, either directly to the generation of alpha (e.g., analysis and prediction of time series, sentiment, news analysis, non-linear regression, etc) as well as indirectly to enhance the "classical" strategy. Our research shows that such methods are unlikely to change a bad strategy in a good one, but it can improve the performance of one that already has a positive expected value. Look at http://ssrn.com/abstract=2462169 .

Regards,

Andrzej

Hi Ananth,

I was interviewed on this topic in Buenos Aires two weeks before their second sovereign default this year, I stress tome examples (by no means the universe it is limited to such examples), but you may find some relevant examples. The interview is in Spanish with subtitles in English:

http://vimeo.com/102090117

Another example is order book aggregation data (particularly using fragmented markets), just in about a dozen G7 currency FX pairs I daily collect and process over 20-60 Gb of high frequency order book data across different venues (way larger for equities). If you sample it, a classical relational database works (i.e. MySQL) but as you collect other data sources and join them you'd better move to big data particularly if you will apply distributed machine learning algorithms on top of it.

Hope it helps!

Best regards,

Andres

I see big data as an infrastructure. You may not reading it for ultra low frequency trading. But you will start needing it at the point your database starts taking a few seconds to respond to your queries. I don't see analytics as part of big data alone. You will perform analytics, that is explore correlations, slice and dice data, plot probability distributions etc. regardless of whether you are using big data infra.

thanks for your reply guys. how about different asset classes specially

Fixed income pricing etc ?

If you do algorithm and quant trading you definetly need to store some data, for research or backtesting or optimisation.

Now, if the amount of data that you are using is reasonable (can be handle by one machine in terms of storage or processing speed) you are not in the Big Data world and other tools might a better match for you.

The big data tools are there to help people to solve one problem: I cannot process or store my data on one machine. A lot of those tools have other advantages that are not related to Big Data so they might still be a good fit for you.

The choice of the tools that you use should only be driven by your real needs.

Yes, it can help, but this requires sophisticated hardware.

What about the analysis of options market data? Volume of data is very high compared to equities. Many strikes, many maturities means a lot of data, especially for index options (e.g. SPY, ES)

Andreas why we can use cloud ? , I don't get it why sophisticated hardware you can cluster the machines

In my experience around 2000 when I started in this field it was very expensive and difficult to process and store a huge amount of data. The computational time and the storage capacity had been two main limits that restricted the strategies.

The today server and the storage capacity are today accessible to a wide range of players in the industry, at a cheap cost compared with the past and I think this is the only way to drive into the complexity and the speed of the today market.

The future is more different sources connected, more volume processed and everything has to be done very fast.

Compare your mobile phone in 2000 with your smartphone in your hand and the evolution is similar.

Data and analytics are very important in any medium term to long term trade. How you analyse data and also mine data is extremely important.

From a practical point of view however data is only important if it is correctly used. If you are running many short term trades, how you analyse and run your correlations etc and what you actually are looking for in the algo is critical. For a longer term view, in the Funds we run for example big data does not matter, but we do take a view of at least a quarter with the result that our 5 yr 11% and 3 yr 8.6% and 1 year 8% as a credit fund, however our outlook is very different to most.