Jurgen Schmidhuber’s 1990 Reinforcement Learning and Planning based on an agent consisting of a controller C and a world model M¹ is a great and widely applicable concept that found its latest implementation in DeepMind’s muZero² project.

A controller with internal RNN model of the world

M learns to predict the consequences of C’s actions. C learns to use M to plan ahead for several time steps, selecting action sequences that maximize predicted cumulative reward.

Humans develop a mental model of the world based on what they are able to perceive with their limited senses. The decisions and actions we make are based on this internal model. Jay Wright…

What is the technology and algorithm behind the Tesla FSD Beta release ?

According to Tesla’s AI Director Andrey Karpathy the technique is Imitation Learning:

While you are driving a car what you’re actually doing is you are entering the data because you are steering the wheel. You’re telling us how to traverse different environments… We train a neural network on those trajectories, and then the neural network predicts paths just from that data. So really what this is referred to typically is called Imitation Learning¹— we’re taking human trajectories from the real world.

Andrey Karpathy adds little bit more…

I presented at http://Risk.Net Machine Learning in Finance Course: Neural Nets and Reinforcement Learning ( Day 3 ) https://training.risk.net/machine-learning/event-agenda


Many thanks to the organizers, Lucy Taylor and Fenella Murray from Infopro Digital.

Some surprising course poll results: most of the attendees do not use NLP in their organizations. 30% use ML for credit scoring.

Some not so surprising results: nobody expects self-driving cars in the next five years.

RL is not really¹ a solved area and particularly there are very simple problems which will break it .

Today, we introduce the Reformer, a Transformer model designed to handle context windows of up to 1 million words, all on a single accelerator and using only 16GB of memory.

When Transformer was announced back in 2017, it created a major shift in NLP towards large language models. Transformer based models like BERT are now a standard part of NLP toolkit ( as demonstrated in Kaggle competitions, for example ). Still, BERT and likes are far from being a final take in solving NLP tasks like summarization, question answering etc.

BERT can effectively cope only with short contexts ( even…

LSTM is the key algorithm that enabled major ML successes like Google speech recognition and Translate¹. It was invented in 1997 by Hochreiter and Schmidhuber as an improvement over RNN vanishing/exploding gradient problem. LSTM can be used to model many types of sequential data² — from time series data to continuous handwriting and speech recognition³,⁸. What is it that makes LSTM so versatile and successful⁹ ?

LSTM cell has the ability to dynamically modify its state⁴ on each new input ( time step ). Past experience shapes how new input will be interpreted i.e. …

A recently released BERT paper and code generated a lot of excitement in ML/NLP community¹.

BERT is a method of pre-training language representations, meaning that we train a general-purpose “language understanding” model on a large text corpus ( BooksCorpus and Wikipedia), and then use that model for downstream NLP tasks ( fine tuning )¹⁴ that we care about.

Models preconditioned with BERT achieved better than human performance on SQuAD 1.1 and lead on SQuAD 2.0³. BERT relies on massive compute for pre-training ( 4 days on 4 to 16 Cloud TPUs; pre-training on 8 GPUs would take 40–70 days i.e…

A whole new software ( TensorFlow, PyTorch, Kubernetes¹) and hardware¹³ ( TPU, GPU, FPGA ) stack⁹ is being built or put together around the needs of Machine Learning community¹⁰ ¹². TensorFlow created that whole weird signal² , followed by PyTorch and other frameworks. In short order we’ve seen the merge of Keras and TensorFlow, Theano leaving the scene, further confirmation of utility of main ML³ algos⁴, addition of Eager execution to TensorFlow as an answer to PyTorch’s dynamic/imperative execution model; announcement of Swift for TensorFlow, courtesy of Chris Lattner’s salto from Apple via Tesla to Google; three generations of TPUs…

Recently open-sourced decaNLP model and code make it possible to easily deploy sophisticated integrated question answering system. While the idea of unified NLP architecture is not new ( Collobert/Weston 2008 paper got ICML 2018 Test of Time award; transfer across multiple learning tasks is explored in 1998 Lifelong Learning Algorithms by S. Thrun ), decaNLP provides direct insight into the latest research directed towards general NLP models and practical uses. …

Machine Learning / AI is undoubtedly becoming a part of mainstream IT. Companies outside of FANG circle are slowly experimenting and incorporating ML elements into its IT landscape ( we’re in the third inning of what’s going to be a seven-game series ). As Goldman Sachs Martin Chavez said, one of the more interesting ML/AI applications is Natural Language Processing. Companies are quite interested in sentiment analysis ( item A2 — sentiment / news — in JP Morgan analysis below holds almost the same potential value as A1, structured transactional data).

Bloomberg is a very strong player in financial NLP…

Ranko Mosic

Machine Learning / AI Consultant ranko.mosic@gmail.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store