Last night was the New York Times’ Open Source Science Fair. It kicked off with a keynote speech by our own Hilary Mason.
Photo by Chrys Wu: http://t.co/m7G7G3g1ih
After a dinner meet-and-greet session, the attendees and exhibitors got down to business, showing off their open source software projects, science fair style.
Fast Forward Labs was an exhibitor. Micha Gorelick impressed the crowd with our realtime stream analysis prototype, CliqueStream.
CliqueStream visualizes the landscape of conversation on Reddit and Twitter, in realtime. This type and scale of analysis would generally be handled by a high-powered cluster and would take upwards of ten minutes to complete. But by using probabilistic algorithms, we are able to analyze the streaming data in realtime on a laptop.
The probabilistic modules underlying CliqueStream are used to distill stream data by counting it and later selectively “forgetting” it; this is the key to speedy analysis in small memory. These appropriately named modules are open source.
CliqueStream was just one among many great open source projects at the Fair. Two of our favorites were the New York Public Library Labs’ book cover generator and MIT’s App Inventor software.
- The New York Public Library Labs’ 10PRINT book cover generation algorithm creates cover art for digital books in its collection, many of which are public domain titles that never had cover art.
- MIT’s App Inventor software is a teaching tool that allows kids to develop Android apps.
The Fast Forward Labs team had a great time learning about these and the other presenters’ work. Thanks to New York Times Labs for a great event!
More from the Blog
Jun 23 2015
Celebrating our one year anniversary!
Aug 5 2015
On March 25, 1909, Wilbur Wright (of the Wright brothers) told a reporter at the Cairo, Illinois bulletin that “no airship will ever fly from New York to Paris.” As with most quotes inherited from the past, people often misinterpret Wright’s quote as reactionary because they read it out of context. He continues: “What limits the flight is the motor. No known motor can run at the requisite spee...
Oct 4 2017
by — We’ve started work on our next prototype. While the design is still evolving, we’re pretty sure one element of it will be a visualization of tens of thousands of data points, clustered through a dimensional reduction algorithm (most likely using T-SNE). For the past week I’ve been exploring how to render that many points in the browser and I wanted to document some early lessons in this post, s...