Deep learning is the driving force behind recent machine learning/AI breakthroughs. Deep learning has its routes in artificial neural network(ANN), a machine learning technique that has been in existence for many many years. Artificial neural network was inspired by the biological neural network (aka brain) with artificial neurons and synapses. Once again the great computer/mathematics genius Alan Turing is at the forefront of the invention of the artificial neural network concept (See Unorganized machine). However its early application was quite limited due to the constraints of limited computer processing capacity and limited data available at that time.
Things started to change in 2006 with computers becoming more and more powerful and data becoming more and more easily available and the discovery of enhanced algorithms now branded as deep neural networks or deep learning. At the core of this breakthrough is the promise of replacing handcrafted training data with efficient algorithms for unsupervised or semi-supervised learning.The deep learning algorithm is exposed to an environment where it trains itself continually using trial and error. The machine learns from past experience and tries to capture the best po knowledge to make accurate decisions.
Neural networks/deep learning currently provides very promising results to many difficult problems in computer vision, speech recognition, natural language processing, language translation and drug discovery. The recent winning of Google’s AlgophaGo over a Go World champion is a strong testament of the power of deep learning.
At this moment, all the deep learning algorithms are not yet generic enough, which means the machine used in AlphaGo to conquer the Go cannot be used to solve other problems.
The next big breakthrough in machine learning and AI will come from a more generic algorithm that can tackle a broader range of problems with less customization, which means we will need to create a general-purpose machine that can be easily tuned to specific tasks.
If you are interested in this topic, I highly recommend a book called The Master Algorithm written by a machine learning professor named Pedro Domingos from University of Washington.
Another interesting post related to this topic is this link from wired.
This youtube called the art of neural network is quite comprehensible for non-technical audience.
An interesting observation I noticed is at least two pioneers with breakthrough achievements in the deep learning/neural network sphere have a background in Psychology. They are Geoffrey Hinton and Andrew Ng’s PHD adviser Michael Jordan. Both got a Psychology Bachelor degree first before moving on to computer/AI. This probably gives an indication that people with multi-disciplinary background probably are key in the future breakthroughs of the master algorithm/general-purpose machine.
