Dec 20, 2017 · post
Highlights of 2017
We end 2017 with a round-up of some of the research, talks, sci-fi, visualizations/art, and a grab bag of other stuff we found particularly interesting, enjoyable, or influential this year.
Stars from around the world. Image credit: Kyle McDonald
Research
2017 saw renewed interest in using the human brain as an inspiration for machine learning — perhaps most notably, Dynamic Routing Between Capsule Networks by Sabour, Frosst, and Hinton.
Somewhat to counter this trend (and Hinton’s criticism of stochastic gradient descent based on its biological implausibility) and to highlight necessary foundational work, we recommend two papers from Arpit, Jastrzębski, and collaborators: Memorization in Deep Neural Networks and The Factors Influencing Minima in SGD (with practical suggestions for developers) are beautiful examples of foundational work the field needs now that neural networks have proven to be effective applied problem solvers.
Work on influence functions and adversarial examples helps us understand both what neural networks learn and their fragility. As more models are pushed into production, it’s more important to understand the vulnerabilities. Indeed our own research built on separate academic work that highlighted the importance and value of of model interpretability.
On the infrastructure and engineering side, we’re excited about serverless in general, and PyWren in particular. PyWren is a tool which uses a serverless backend (AWS Lambda in this case) to offer data scientists a simple, inexpensive and scalable mapreduce. PyWren is presented in Occupy the cloud: distributed computing for the 99%. The upsides and downsides of serverless are examined in Serverless Computing: Economic and Architectural Impact.
Talks
Image credit: Aaron Levin at !!Con
Aaron Levin’s Finding Friends in High Dimensions: Locality-Sensitive Hashing For Fun and Friendliness! was a fast (and funny!) introduction to an important algorithm. Aaron delivered this talk at !!Con, which is hands-down our favorite programming conference.
Joel Grus’s Livecoding Madness - Let’s Build a Deep Learning Library was dazzling.
Sci-fi
N.K. Jemisin completed her Broken Earth trilogy with The Stone Sky, and it was the best sci-fi we read this year. While not directly related to AI, it explores consequences of a society shaped by technology inherited from the past.
Autonomous by Annalee Newitz is an interesting look at bio-hacking and robot relationships.
Visualization/Art
Image credit: Music VAE
Kyle McDonald’s twitter account is not just about visualization (e.g., his dispatches from NIPS were fascinating), but we particularly love it when he shares his work looking at data. This thread on how people draw everyday objects varies by country, and this thread showing music tempo by year as a heatmap to demonstrate the arrival of the drum machine, were highlights.
Speaking of the drum machine, the Google Brain team got great-sounding results with Music VAE.
Zach Lieberman’s visual sketches have been great, especially the AR stuff.
Mario Kligemann does interesting work with Generative Adversarial Networks, among other things.
Everything else
Image credit: Universal Paperclips
We thought about putting the web (and now mobile) game Universal Paperclips in the sci-fi section of this list. It’s a fun game to discover and beat, but it’s also a fascinating look at the infamous Paperclip Maximizer artificial intelligence thought experiment.
Data & Society’s report on Media Manipulation and Disinformation Online shows how extremist groups attempt to hack the attention economy. And in a year during which harassment and discrimination was never far from the news, Susan Fowler’s powerful testimony on her time at Uber and Leah Fessler’s Your company’s Slack is probably sexist were especially important to our community. We urge everyone who works in tech to reflect on these pieces.
Finally, we wish you all a very happy holidays and look forward to sharing our work with you next year!
