Artificial intelligence is having a significant impact on our lives — from political campaigns to shows recommended on Netflix. Even when we type on our mobile phone keyboard, there’s a deep learning algorithm that is learning our style to predict and autocorrect words.

Before beginning this article, let’s clear up some definitions.

Reference: Deep Learning Book by Ian Goodfellow

Artificial Intelligence (AI) is a general term to describe algorithms that consume data to produce output. A straightforward, but counter-intuitive, example here would be Technical Analysis based trading. A human creates rules that are applied to financial market data to produce a trading decision.

So anybody who does technical analysis is actually working with artificial intelligence!

Machine Learning(ML) is a subset of AI. For example, linear regression solver in Microsoft Excel is an ML algorithm. More sophisticated models are Support Vector Machine, Tree-based Classifiers.

Deep Learning is a particular type of machine learning algorithms that use artificial neural networks (ANNs) as its calculation blocks.

Deeper we go into Artificial Intelligence algorithms — lesser the number of rules a human has to create.

Technical analysis requires all trading rules to be created by a human. In deep learning-based trading, humans make almost no rules. This difference is fundamental and significant.

Many players in financial markets use “Artificial Intelligence” or “AI Engine” in their marketing. Majority of them use algorithms that require human-created rules and not necessarily deep learning. It is, unfortunately, misleading.

We have some easy-to-use simulators for various machine and deep learning algorithms on our website if you’re curious to learn more.

Rephrasing the question: Do we need deep learning in financial markets?

Another way to ask this is — do we need automated trading rules or are manual (human) rules enough?

There are multiple parts to this answer.

Let’s contrast financial markets to computer vision. Computer vision is teaching an algorithm to identify objects in a photo/video. For decades researchers were hard at work in creating mathematical equations (rules) to do this task. For example, to identify cats in a photo, a researcher will write rules for cat ears, nose, whiskers, eyes and other body parts. There are infinite possibilities when taking a cat’s photo — in sunlight, in the dark, behind another object, wearing a cap — and the problem quickly becomes unmanageable. It’s impossible to finish this problem successfully. This task is just for cats, we then have dogs, humans and 8.7 million other species that are on our planet.

In 2012, there was a breakthrough, and a deep learning algorithm could identify objects with 85% accuracy. In 8 years, this accuracy is now more than 99% — performing better than humans.

Similarly, in financial markets, we have very talented folks who are trying to write mathematical equations (rules) to understand it better. Consider Bank of International Settlements (BIS), which is like a central bank to all the central banks around the world. It has an active research division that tries to understand our economy and its behaviour.

However, this problem is even more complicated than computer vision because of the immense noise and complexity of our economies. Most research at BIS is still theoretical.

Another challenge in financial markets is the rapid explosion at which information spreads. When the news was slow to travel, some traders and fund managers would make a profit on it before others knew about it.

This phenomenon is known as information asymmetry. As information spreads faster, this asymmetry reduces. We quantified the effect of this in the graph below :

10-year performance of equity-oriented mutual funds vs NIFTY 50

Source of this data is AMFI. Over 10 years, fund managers of equity mutual funds outperformed the market (also called alpha = α) by only 1.5%. Considering that mutual funds charge 2% management fees, it would be much better to invest somewhere else like ETFs.

This analysis also uncovers an underlying problem. With ever-increasing data, fund managers and trader will soon find it impossible to discover the right investing & trading rules manually.

Deep learning algorithms as a tool is excellent in identifying patterns efficiently in a large volume of data. The biggest challenge is creating a neural network architecture, and it’s training procedure that is capable of working with financial market data.

Almost all breakthroughs in this field have been for computer vision (autonomous driving, cancer detection), text and audio (smartphone assistants). A plug and play approach by using popular deep learning algorithms doesn’t work.

Properly trained neural networks are capable of identifying useful patterns (trading rules) even from noisy market data. To do this requires intensive R&D and only a few companies are willing to put in the time and resources.

Artistic visualisation of finding patterns — Not an actual representation of neural networks

Therefore a deep learning approach is the future of financial markets analysis. This will, no doubt, generate higher profits. However, a more meaningful impact would be in making better economic policy decisions around the world.