We end 2017 with a round-up of some of the research, talks, sci-fi, visualizations/art, and a grab bag of other stuff we found particularly interesting, enjoyable, or influential this year.
Stars from around the world. Image credit: Kyle McDonald
2017 saw renewed interest in using the human brain as an inspiration for machine learning — perhaps most notably, Dynamic Routing Between Capsule Networks by Sabour, Frosst, and Hinton.
Somewhat to counter this trend (and Hinton’s criticism of stochastic gradient descent based on its biological implausibility) and to highlight necessary foundational work, we recommend two papers from Arpit, Jastrzębski, and collaborators: Memorization in Deep Neural Networks and The Factors Influencing Minima in SGD (with practical suggestions for developers) are beautiful examples of foundational work the field needs now that neural networks have proven to be effective applied problem solvers.
Work on influence functions and adversarial examples helps us understand both what neural networks learn and their fragility. As more models are pushed into production, it’s more important to understand the vulnerabilities. Indeed our own research built on separate academic work that highlighted the importance and value of of model interpretability.
On the infrastructure and engineering side, we’re excited about serverless in general, and PyWren in particular. PyWren is a tool which uses a serverless backend (AWS Lambda in this case) to offer data scientists a simple, inexpensive and scalable mapreduce. PyWren is presented in Occupy the cloud: distributed computing for the 99%. The upsides and downsides of serverless are examined in Serverless Computing: Economic and Architectural Impact.
Image credit: Aaron Levin at !!Con
Aaron Levin’s Finding Friends in High Dimensions: Locality-Sensitive Hashing For Fun and Friendliness! was a fast (and funny!) introduction to an important algorithm. Aaron delivered this talk at !!Con, which is hands-down our favorite programming conference.
Joel Grus’s Livecoding Madness - Let’s Build a Deep Learning Library was dazzling.
N.K. Jemisin completed her Broken Earth trilogy with The Stone Sky, and it was the best sci-fi we read this year. While not directly related to AI, it explores consequences of a society shaped by technology inherited from the past.
Autonomous by Annalee Newitz is an interesting look at bio-hacking and robot relationships.
Image credit: Music VAE
Kyle McDonald’s twitter account is not just about visualization (e.g., his dispatches from NIPS were fascinating), but we particularly love it when he shares his work looking at data. This thread on how people draw everyday objects varies by country, and this thread showing music tempo by year as a heatmap to demonstrate the arrival of the drum machine, were highlights.
Zach Lieberman’s visual sketches have been great, especially the AR stuff.
Mario Kligemann does interesting work with Generative Adversarial Networks, among other things.
Image credit: Universal Paperclips
We thought about putting the web (and now mobile) game Universal Paperclips in the sci-fi section of this list. It’s a fun game to discover and beat, but it’s also a fascinating look at the infamous Paperclip Maximizer artificial intelligence thought experiment.
Data & Society’s report on Media Manipulation and Disinformation Online shows how extremist groups attempt to hack the attention economy. And in a year during which harassment and discrimination was never far from the news, Susan Fowler’s powerful testimony on her time at Uber and Leah Fessler’s Your company’s Slack is probably sexist were especially important to our community. We urge everyone who works in tech to reflect on these pieces.
Finally, we wish you all a very happy holidays and look forward to sharing our work with you next year!
More from the Blog
Nov 30 2017
Earlier this month The New York Times published an article on Building A.I. That Can Build A.I.. There is a lot of excitement about AutoML due to the scarcity of machine learning (ML) talent: By some estimates, only 10,000 people worldwide have the education, experience and talent needed to build the complex and sometimes mysterious mathematical algorithms that will drive this new breed of ...
Jan 22 2018
by — While we commonly associate recommendation systems with e-commerce, their application extends to any decision-making problem which requires pairing two types of things together. To understand why recommenders don’t always work as well as we’d like them to, we set out to build some basic recommendation systems using publicly available data. Data The first ingredient for building a recommendati...
Jan 29 2019
by — UMAP explorer: an interactive visualization of the MNIST data set We’re in the middle of work on our next report, Learning with Limited Labeled Data, and the accompanying prototype. For the prototype’s front-end we wanted to be able visualize and explore the embedding of a large image data set. Once you get into the tens of thousands of points, this can be a challenge to do in the browser. T...