The key difference between machine learning and deep learning, according to R. Sawhney, the author of the posted article, is the following: "If a Deep Learning approach were applied to the same facial recognition challenge, a developer would simply load the image pixels, represented numerically, directly to the model itself. No prior domain experience of facial recognition would be required and the program would not need to be pre-loaded with data on eye, nose or face measurements. Instead, a type of Artificial Neural Network (ANN), inspired by the workings of the human brain, would be used to automatically find the right features from the data to create the model." This is, at best wishful thinking. The machine using deep learning might be able to construct correlations and similarities between pixel arrangements, but to go to the recognition of a face, any face, would not be possible since the machine by itself has not concept of "face" as something distinct and identifiable.

There is no difference: deep learning is just one of many machine learning methods. It refers to using a neural network with many internal layers. Usually, a network with more than 3 layers is referred to as "deep learning". The distinction is made because deep networks require special methods of training.

Johann I know there is no real difference between the two techniques, but the author of the article that is referred to here makes deep learning into some kind of magical technique where, without any prior instruction or knowledge, the machine can make out images out of grayscale pixel dumps. This is crazy the machine can find correlations between pixel groupings as well as patterns. However without prior comparison data showing specific (labelled or named) images it could not recognize a face from a cat.

That's certainly right. There's no Holy Grail in machine learning yet, not even deep learning.

Opposite to the previous comments, there is a major difference difference between the DL, and ALL other ML techniques.

1. All ML methods require a prescribed feature extraction which are fed into a ML algorithm.

Contrary to it, the DL doesn't require any prior feature extraction. The amazing power of the DL is its ability to learn the filters which extract the "correct" features.

Yes - with the DL "the developer would simply load the images" and "no prior domain experience of facial recognition" is required.

2. No a conventional network with 3, neither with 23, layers is as a Deep Learning. DL isn't just a number of layers - it's a principally different paradigm.

OK so the developer loads the pictures of the faces (assuming it is image recognition or classification that is desired) and the DL does the rest. The DL creates its own rules. This may be fine when you limit the data universe to faces. It is a vast jump from this to let the DL, fed random data, decide what to do with it. Take the market for example do you propose that raw data fed into the DL will be turned into a trading system, even a bad trading system, with no instructions or limits or filters provided to the DL? I am interested in knowing your opinion.

Deep learning is just a subset of machine learning. Google have showed how powerful it can be, with AlphaGo. But it is naive to use it on the stock market. Deep learning can recognize a picture of a leopard as a leopard. Which is an amazing achievement. But much of the reason it can recognize a leopard is because a leopard never changes its spots. Deep learning, or any form of neural network doesn't work in the market because in the market the leopard regulalarly changes its spots.

Any prediction is based on the assumption that there is at least some non-zero correlation between the past and the future - otherwise no prediction is possible. If you are a member of this group, then I assume you believe that such a correlation exists 

About a DL – there is no limitations using it for the market, feeding a raw data into the system, but the system itself is not free of parameters, of course.

I will explain:

A classical ML approach is based on the following scheme:

Training:

1. get a data (for example a time series) -> calculate features (for example: moving average for 5 time units (ma5), ma10, ma20,…, volatility for 5 time units (v5), v10, v20,…, momentum,… etc.)

2. set a desirable output (for example: an asset next move is UP or DOWN)

3. train the ML system according to the training data (input features, and a desirable output).

Prediction:

4. get a new data -> calculate the features -> feed into the ML system -> get prediction

(Cont.)

Deep Learning:

A DL is working with a similar scheme, except feature calculation step. During training a DL system will learn the filters which it uses for a feature calculation. Thus one skips a vague and a tedious step of the deciding which features to use, but the system itself will pick the best features it needs.

Having training the DL system, the prediction step operates as follows:

5. get a new (raw) data -> feed into the ML system -> get prediction

So you are really feeding a raw data into your system, but the system itself has a lot of internal parameters such as: no. of layers, layers size, no. of filters and the size, etc.

Hope it helps :)

As @Johann Johann Christian Lotter rightly said, there is no difference. Deep learning is in a sense closer to the human brain but all that glitters is not gold.

Joseph Levitas, "shallow" neural network are also capable to unsupervised learning, so there is no principle difference.

Recently I have written an essay on subject:

https://letyourmoneygrow.com/2016/10/23/big-data-and-deep-learning-a-technology-revolution-in-trading-or-yet-another-hype/

Vasily,

1. I didn’t say anything about an unsupervised learning. My explanation referred to the supervised (CNN) learning only.

2. The MAIN reason DL (CNN) is so successful is not because of the depth, but because they can automatically extract the right features needed for the task. You think having an automatic feature extractor makes no difference?

3. I liked your critical review and especially the note: “in trading those who know do not speak (and those who speak often do not know)”, and the closing sum-up part which is absolutely correct.

Just one comment: you probably meant “breakthrough” rather than “breakthru“.

Joseph, I assume you are talking about DL used by CNN (the media news network) and would like to know how you (or they) measure the success of their DL applications. I do know that CNN is quite successful as a media outfit. However this success have a lot to do with identification of what people want to hear or see. As such a mechanism to tailor news to perceived demand may be very successful. However when you have something infinitely more fluid and variable than preference, can such models be effective? Take for example the electoral model used by the NYT (I believe CNN cooperated in this model with the NYT but I am not sure about this). It was making a prediction as late as the election date for Mrs. Clinton having an 85% chance of winning the election with a predicted most likely outcome of 322 electoral votes. You do realize that the final outcome of the election is likely to show Mr. Trump winning by over 300 electoral votes. Is this a successful use of DL in your opinion?

Using your own words Joseph, a DL that can automatically extract the features needed for the task sound very enticing. But like in the example above can be miles away from coming with a valid prediction. Why is this, simply because the most likely features to be used in a task may or may not be the important features that determine the outcome of whatever it is that is being studied. Now take the market, do you really think anyone, anywhere, can devise a DL that can automatically extract the feature needed for the task and give a reasonably successful outcome in trading? First of all, keep in mind that the market, like an election, has levels of complexity that approach infinity. The DL can find a path or paths that may be sucessful, but it may totally fall on its face by choosing the apparently necessary features necessary, which in reality are nothing but mirages in the desert, and missing the real causative features (which oftentimes are completely unknowable) that determine results

Oscar Cartaya CNN = Convolutional Neural Network

1. CNN is indeed Convolutional Neural Network, as Robert Remen has commented already.

2. Of course the a specific prediction might be wrong. Any trading algorithm is significant (or insignificant) in a statistical content only.

3. I am even not claiming that one can trade with the DL. What I am saying is just that with the DL one can feed a raw data into an algorithm with-no any previous feature calculations.

Vasily,

You probably have missed my previous question :) , so I repeat it: "Do you think having an automatic feature extractor makes no difference?"

I have not tried to apply DL learning in stock or forex market. I used and I am using feed-forward networks involving various supervised training algorithms. I believe that in ML you have more options in terms of optimization (e,g selecting best combinations of inputs) and it may be perform better. Furthermore, you can use Committee of Neural Networks (CNNs), a collection of different neural networks that together decide and vote on a given example, hoping that errors would be canceled out as there are several experts . Combining several neural networks can lead to significant improvement. In fact the committee can often do better than the best single neural network. These structures can be static or dynamic. In my research, I used Multiples of Committees of Neural Networks (MCNNs). Multiples means many CNNs where each CNN is training in different output. The outputs are selected in such way to map the future price changes (short and long term), combining with data mining algo

Has anyone done any work regarding how many data samples do you require to train a deep neural network ? I have only seen https://www.quora.com/What-is-the-recommended-minimum-training-dataset-size-to-train-a-deep-neural-network (which sugges you need P^2 if P is the number of weights in your NN) -- hence why Google uses millions of pictures.

OK Robert and Joseph, CNN means convolutional neural network. Since I have never dealt with any of these I cannot comment on them.