Search
× Search
Friday, January 10, 2025

Archived Discussions

Recent member discussions

The Algorithmic Traders' Association prides itself on providing a forum for the publication and dissemination of its members' white papers, research, reflections, works in progress, and other contributions. Please Note that archive searches and some of our members' publications are reserved for members only, so please log in or sign up to gain the most from our members' contributions.

Big Data and Deep Learning, a technology revolution in trading or yet another hype?

photo

 Vasily Nekrasov, Senior Risk Analyst and Model Developer at Total Energie Gas GmbH

 Sunday, October 23, 2016

Summary: *BigData and DeepLearning are popular buzz words nowadays. But the number of the genuine success stories is relatively small. *In trading the BigData technology is mostly accociated with automatic analysis of the news and sentiment in social networks. But unless you are Google or Reuters, you will never be the one who gets the news first. Additionally, a market reaction both to news and sentiment is often vague and amorph. *Large deep neural networks closely resemble a human brain, which also has a lot of neurons, interconnected in many layers. But it doesn’t mean a breakthrough to a real artificial intellegence: all is not gold that glitters. *A positive side: trading is only a part of the financial world. Likely, BigData + DeepLearing has a high potential in adjacent areas like risk profiling and credibility analysis. So IMO there is more hype than opportunities. However, I would be happy to hear the opposite opinions (esp. with concrete examples of success stories).


Print

21 comments on article "Big Data and Deep Learning, a technology revolution in trading or yet another hype?"

photo

 Eric Williams, Business Development Analyst at XPO Logistics, Inc.

 Sunday, October 23, 2016



Largely the success of high frequency traders is their ability to "pick off" order flow with techniques that if a human used on a physical trading floor would have them barred from the exchange.


photo

 David Edenfield, SR CONSULTANT - Trusted Advisor, Program/Project Management, Strategy, Process, Testing, Robotic Proces Automation (RPA)

 Tuesday, October 25, 2016



I don't agree Eric... I think maybe you have a limited view of HFT opportunities...


photo

 Tibor Komoroczy, CEO, Skunkworks LLC, DTQuant.com

 Tuesday, October 25, 2016



All I can say is people who buy headlines usually end up selling newspapers. The world of quantitative finance needs to get to an understanding of how markets function. You should be asking yourself does human nature change?


photo

 Oscar Cartaya, Private Investor

 Tuesday, October 25, 2016



Hello Vasily, your article strikes what I think is the correct combination of excitement and doubt about the capabilities of AI and deep learning. I want to emphasize that it can be done, BUT, not a a vast array of machines and algos that make original contribution and breakthroughs. Nope. The key is to use the machines for simple tasks that the machine does better and faster than we can, and avoid their use in those tasks the machines have trouble with. Machines cannot abstract data and reach conclusions that were not there before, they also cannot differentiate similar data that correlates well with one another into specific discontinuous things. They cannot think and reach conclusions like we do. But they are very good and very fast at matching patterns and find correlations. In a limited way, machines can be a great help, however they are just not capable of original thinking. Anyone who says otherwise is just expressing his own opinions and wishful thinking.


photo

 Vasily Nekrasov, Senior Risk Analyst and Model Developer at Total Energie Gas GmbH

 Wednesday, October 26, 2016



Oscar Cartaya, thank you for a detailed comment which I completely agree.

I am, myself, very convinced that one should "outsource" the routine computational tasks to the machines.

As to those, who claim "revolution with deep learning + bigdata", IMO there is not only wishful thinking behind but also a deliberate attempt to deceive (unquilified) investors. That's why I say: don't let them lure you with buzzwords, watch the real-life track record at first.


photo

 Jitendra Shah, Consultant at RBS Markets & International Banking

 Wednesday, October 26, 2016



Really interesting to analyse and code up. Not sure how any of this would look to a regulator... An algo around he-said she-said.


photo

 Chad Bryant, Senior Data Scientist specialising in Deep Learning and Neural Networks

 Wednesday, October 26, 2016



Oscar Cartaya I do disagree with you somewhat. Machines certainly can't think like us nor reach the same conclusion that we do. But they can reach DIFFERENT conclusions because they "see" patterns we cannot. These conclusions can be better or worse than a humans. In the example of AlphaGo, the system reached conclusions BETTER than the human to be able to beat him. These were likely conclusions different to what the human player made. Original thinking? No. But much of what we do is not original thinking either. We are simply applying past experience and patterns to a current situation. For example - driving. We follow rules and use patterns to drive, no original thinking there.

As for your last sentence, really, isn't what you do expressing your own opinion?


photo

 Oscar Cartaya, Private Investor

 Wednesday, October 26, 2016



OK Chad, so machines can go around analyzing everything and renaming everything if you wish. Oh do I wish we could avoid this but here it goes. In Genesis one of the main attributes of creation was naming. That is exactly what we human have been doing for eons, naming things, actions, etc... A machine can discern patterns much faster than a human can, it can find hidden correlations much faster than humans can. Is this better? In Epidemiology, which is the basis for a lot of what we know about human disease, there are strict rules to separate correlation from causation. Can we be satisfied with something that spews correlations faster than anything else. Where is the causation? You can make a mountain out of a molehill with big data correlations, but is any of this causative of anything? I know you are entitled to your own opinions but really can you say that machine found patterns and correlations are real knowledge? Just asking.


photo

 Holger Gebauer, MD

 Thursday, October 27, 2016



Do you need funding?

Have you thought of getting a legitimate loan provider where you would like to re-invest after much dividends? Grab this life time financial offer and give your business, education, homes and construction a lift.

We Offer 2% Loans, Business Loan, Venture Capital, Home Loan, Business Loan, Working Capital. if you are interested contact us via email :customer.services@capitalfinanceno1.com


photo

 Chad Bryant, Senior Data Scientist specialising in Deep Learning and Neural Networks

 Thursday, October 27, 2016



Oscar Cartaya Finding causation is hard. I never said a machine can do it. Heck, most humans have a difficulty in even understanding what that means. Knowledge is something entirely different. You can find hidden patterns and that is new knowledge. DL is good at that. Can they determine causation? Hard to say, not in most cases likely. If it is a timing issue - which came first (cause then effect) sure, it can provide probabilities.

I never said that DL was better just different. Because it sees things without human bias (usually caused by emotion, distraction, or even something someone doesn't what to see), it can provide information to humans that humans may not see themselves.

I harken back to my point about driving. An automated car can drive using DL developed systems because there is no need to do anything other than find patterns - when to stop, how fast to go, where a pedestrian is.

I'm also not sure what your point of "renaming" was - can you clarify?


photo

 Oscar Cartaya, Private Investor

 Thursday, October 27, 2016



The issue about renaming is that a machine has no real (in human terms) concept of the identitiy of things, it cannot identify patterns or whatever it is that do not fit something placed in its data banks. In other words, they have no concept of reality, just numbers and figures or quantifiable shades. Machines cannot have a concept of discovery of anything because they cannot identify it or name it, or even know it is different in many levels than just two different patterns are. Machines have no consciousness or reality. They do not think, they just compute and refine pattern recognition. Machines are not intelligent if you define intelligence in human terms. To define intelligence in machine terms you would have to measure things like speed of recognition of specific patterns, or false identification rates. They can be fabulous in this way, but not intelligent, not based upon reality (as humans know), and certainly not capable of unsupervised action in a non restricted environment.


photo

 Tibor Komoroczy, CEO, Skunkworks LLC, DTQuant.com

 Thursday, October 27, 2016



I will say AI programming is programming thinking. Garbage in Garbage out its like Frankenstein putting in the evil brain instead of the good. Then, of course, you need to understand time. Numbers can identify the reality of extremes of human emotions. Machines can cover subject become scholars. Street smarts mixed with understanding how markets function. Making Financial Service products is one of the Skunkworks Businesses.


photo

 Oscar Cartaya, Private Investor

 Thursday, October 27, 2016



Like you say Tibor Garbage in Garbage out. I would like to find out how you can make machines into scholars and teach them street smarts. Do not mean to offend but I detect a lot of garbage in those statements.


photo

 Chad Bryant, Senior Data Scientist specialising in Deep Learning and Neural Networks

 Friday, October 28, 2016



Oscar Cartaya I would have to disagree (assuming I understand your thought) about your renaming theory. If I give it a bunch of data - say cardiac data - it spins away and gives me groupings - maybe labels them 1, 2, and 3. It can tell us many characteristics about those groups just named unlike we would. We can make it more "real" to us by getting an understanding of those characteristics. Think about a child. If you put a child in a situation with things it has never encountered before, it does things similar to what a computer does. If it doesn't know what an elephant is, it may liken it to some other animal. Once we tell it is an elephant it goes into their memory (their "databank"). Humans will be able to generalize and work faster to identify a second elephant vs a machine. As far as intelligence - yes, I agree that a machine is not intelligent in the same way as a human (just like an ant for example) but you can compare its intelligence (however defined) to other machines.


photo

 Oscar Cartaya, Private Investor

 Saturday, October 29, 2016



This is enjoyable. OK let's take the cardiac data. There is no reason to collect cardiac data unless it can be applied to treatment of heart disease. You could do it for pure research purposes but what is the use if it cannot be readily applied to cardiac disease categories and treatments. Let's say that you can divide the level of myocardial muscle contraction into three groups. Unless you can associate these with specific disorders, or demonstrate they are a universal finding applicable to all diseases, you are nowhere in terms of how to apply this to therapeutic approaches. I liked your approach of the child and the elephant, I suppose that if an AI was given a set of grayscale pixels corresponding to an elephant during a trial to distinguish cats from dogs, the machine might label the elephant as either a cat or a dog. Your point on non human intelligence is valid, but not if it means that we have to rethink everything in terms of machine or ant specific intelligence and naming.


photo

 Ron Jaenisch, Author, Andrews and Babson Technical Analysis Expert

 Monday, October 31, 2016



Yep it really works. We use it daily.....to trade the markets.https://youtu.be/RSpDEPUPPPQ


photo

 Vasily Nekrasov, Senior Risk Analyst and Model Developer at Total Energie Gas GmbH

 Monday, October 31, 2016



Ron Jaenisch, a YouTube video is unfortunately not a proof :)

As I e.g. explained the scam behind binary options (), I also shown that the numerous "success stories" of YouTube need not even be falsified, it is enough to show just successful trades (rather than a complete trading history).

A genuine proof is e.g. a track record on a third-party(!) server.


photo

 Chad Bryant, Senior Data Scientist specialising in Deep Learning and Neural Networks

 Monday, October 31, 2016



Oscar Cartaya This is exactly my point with the cardiac data... the ML/DL finds three groups. It can provide me with patient IDs for patients in each group. I can then examine the patients in each group and see their status and any diagnosis. In my previous work we used simple k-means clustering to find patients who were in a group that had to be re-hospitalized or had sudden deaths. As new patients came in with similar symptoms and the clustering placed them in that group, we could direct the treatment as required to prevent sudden death. Other clusters aligned with other diagnosis to allow treatments of those patients as needed. Since the testing can focus in on certain areas, those patients are likely treated faster for their conditions. Those groups might never be since by a human, especially with hundreds of patients.


photo

 Oscar Cartaya, Private Investor

 Monday, October 31, 2016



This is advanced Chad, but you go nowhere if it is not reimbursed by Medicare or the insurance plans. As far as research goes, identifying clusters appears like a fine approach. There may be issues if you start identifying these clusters in a cross cut of different diagnosis categories. In other words, if you can apply this to say cardiomyopathies, which are known to cause sudden death, then you are OK. If you have a bunch of other disease categories in the same cluster, or even worse if you include people with no cardiac disease in the sudden death group, it is likely not to be OK as payment is based on disease categories and severity. Placing someone with no diagnosis or symptoms into a sudden death cluster (which may happen) will cause problems with payment. Suggest using the clusters only within specific diagnosis categories, or a separate cluster per diagnosis. After all, no money no research. I would suggest getting someone with knowledge of ICD codes into the group.


photo

 Chad Bryant, Senior Data Scientist specialising in Deep Learning and Neural Networks

 Tuesday, November 1, 2016



Oscar Cartaya lol - yes, the study is being run by a Cardiac Institute in Canada - no need for medicare or anything ;-) But yes, we have really only been working with a prototype and are looking to move forward to use it to reduce treatment times at this point by identifying the likely cluster that a patient is in and the probably tests that would be most useful, not as a prognosis tool in and of itself.


photo

 Oscar Cartaya, Private Investor

 Wednesday, November 2, 2016



That sounds reasonable to me, wish you all the best.

Please login or register to post comments.

TRADING FUTURES AND OPTIONS INVOLVES SUBSTANTIAL RISK OF LOSS AND IS NOT SUITABLE FOR ALL INVESTORS
Terms Of UsePrivacy StatementCopyright 2018 Algorithmic Traders Association