Newsletter

JavaScript eats the world: deep learning and notebooks edition

Apr 25 2018 · by

Google recently announced TensorFlow.js, an open-source library for running machine learning in the browser, and a successor to the deeplearn.js library. While the majority of machine learning work is unlikely to shift to JavaScript anytime soon, the examples included on the TensorFlow.js site do a good job of showing the promise of machine learning models that run in the browser.

A short GIF of the Teachable Machines demo showing Grant raising his hand and the model responding with a GIF of a cat waving.

Teachable Machine lets you train a model to help you wave at cats.

Our favorite example is Teachable Machine, which walks you through a training process using images from your webcam to trigger response GIFs. It shows how training in the browser can help the model adapt to different contexts. For example, if you want your model to spot when a user raises their hand, a pre-trained model might have trouble if the user is sitting in a room surrounded by mannequins. Because you can train the Teachable Machine model with specific examples of both “raised hand” and “unraised hand,” there’s a good chance it will perform well in your weird mannequin room (a very specific situation). A related webcam-based example is the work Oz Ramos is doing to build a system for navigation using facial gestures, to help people with mobility impairments use the web.

Another advantage of having deep learning code running in the browser is that you can open up the code itself for people to interact with. Better JavaScript deep-learning tools mean more web-based interactive deep-learning explainers: explainers like Minsuk Kahng’s Deep Learning Tutorial, which shows how to use deeplearn.js to build a model for the Iris dataset. Because all of the code for the system is exposed and editable, viewers cans tweak different parameters and easily view the results.

A screenshot form a section of Minsuk Kahng's Deep Learning Tutorial. It shows the model code and a slider for specifying the number of neurons in the hidden layer.

Minsuk Kahng’s deeplearning.js node lets you edit the code and immediately view the results.

Kahng’s tutorial is built on the new JavaScript notebook site Observable. Made by D3.js creator Mike Bostock and others, Observable makes it easy to build interactive and modifiable JavaScript examples of all types. Along with sites like Codepen and Glitch, Observable is leading a renaissance of interactive code examples and explainers. Grant has used Observable for everything from using three.js for 2D data visualization to finding out how long it takes a browser to say different numbers out loud.

More from the Blog