We end 2017 with a round-up of some of the research, talks, sci-fi, visualizations/art, and a grab bag of other stuff we found particularly interesting, enjoyable, or influential this year.
Stars from around the world. Image credit: Kyle McDonald
2017 saw renewed interest in using the human brain as an inspiration for machine learning — perhaps most notably, Dynamic Routing Between Capsule Networks by Sabour, Frosst, and Hinton.
Somewhat to counter this trend (and Hinton’s criticism of stochastic gradient descent based on its biological implausibility) and to highlight necessary foundational work, we recommend two papers from Arpit, Jastrzębski, and collaborators: Memorization in Deep Neural Networks and The Factors Influencing Minima in SGD (with practical suggestions for developers) are beautiful examples of foundational work the field needs now that neural networks have proven to be effective applied problem solvers.
Work on influence functions and adversarial examples helps us understand both what neural networks learn and their fragility. As more models are pushed into production, it’s more important to understand the vulnerabilities. Indeed our own research built on separate academic work that highlighted the importance and value of of model interpretability.
On the infrastructure and engineering side, we’re excited about serverless in general, and PyWren in particular. PyWren is a tool which uses a serverless backend (AWS Lambda in this case) to offer data scientists a simple, inexpensive and scalable mapreduce. PyWren is presented in Occupy the cloud: distributed computing for the 99%. The upsides and downsides of serverless are examined in Serverless Computing: Economic and Architectural Impact.
Image credit: Aaron Levin at !!Con
Aaron Levin’s Finding Friends in High Dimensions: Locality-Sensitive Hashing For Fun and Friendliness! was a fast (and funny!) introduction to an important algorithm. Aaron delivered this talk at !!Con, which is hands-down our favorite programming conference.
Joel Grus’s Livecoding Madness - Let’s Build a Deep Learning Library was dazzling.
N.K. Jemisin completed her Broken Earth trilogy with The Stone Sky, and it was the best sci-fi we read this year. While not directly related to AI, it explores consequences of a society shaped by technology inherited from the past.
Autonomous by Annalee Newitz is an interesting look at bio-hacking and robot relationships.
Image credit: Music VAE
Kyle McDonald’s twitter account is not just about visualization (e.g., his dispatches from NIPS were fascinating), but we particularly love it when he shares his work looking at data. This thread on how people draw everyday objects varies by country, and this thread showing music tempo by year as a heatmap to demonstrate the arrival of the drum machine, were highlights.
Zach Lieberman’s visual sketches have been great, especially the AR stuff.
Mario Kligemann does interesting work with Generative Adversarial Networks, among other things.
Image credit: Universal Paperclips
We thought about putting the web (and now mobile) game Universal Paperclips in the sci-fi section of this list. It’s a fun game to discover and beat, but it’s also a fascinating look at the infamous Paperclip Maximizer artificial intelligence thought experiment.
Data & Society’s report on Media Manipulation and Disinformation Online shows how extremist groups attempt to hack the attention economy. And in a year during which harassment and discrimination was never far from the news, Susan Fowler’s powerful testimony on her time at Uber and Leah Fessler’s Your company’s Slack is probably sexist were especially important to our community. We urge everyone who works in tech to reflect on these pieces.
Finally, we wish you all a very happy holidays and look forward to sharing our work with you next year!
More from the Blog
Nov 22 2017
by — As you are preparing for your Thanksgiving meal, just know that a robotic arm is holding the spoon at the Institute for Culinary Education (ICE); progress is relentless. “The Chef Watson cookbook is a revolutionary display of the creative collaboration of man and machine.” Cognitive Cooking with Chef Watson, culinary and cognitive creativity at your fingertips. Perhaps you should try the Acorn ...
Jan 22 2018
by — While we commonly associate recommendation systems with e-commerce, their application extends to any decision-making problem which requires pairing two types of things together. To understand why recommenders don’t always work as well as we’d like them to, we set out to build some basic recommendation systems using publicly available data. Data The first ingredient for building a recommendati...
Apr 19 2018
by — Today we are launching a mini-site featuring our collection of short stories inspired by new developments in machine learning. Beginning with our fourth report, we started including a science-fiction story along with the technical and strategic overviews that are the bulk of each report. Using these stories, we can look at the technologies we profile from a different angle and explore their c...