As we’ve progressed through our private beta, one of the most fruitful activities has been talking with folks in different user communities about their data projects. This has had a profound impact on our product vision and roadmap (including a few major pivots). A number of themes have come up again and again, so we thought it would be useful to distill this down to a few of the most significant points. We’ll cover the major themes in this post and then turn to some insights we’ve gained from different user...

Joe Germuska is Executive Director at the Knight Lab at Northwestern University, or as he calls it, “Chief Nerd”. Starting out as a programmer, he got into journalism in 2009 while working at the Chicago Tribune developing news applications and interactive data sites, such as “Crime in Chicagoland”, which aimed to make all reported crime data in the Chicagoland area searchable and more transparent. Joe was also the project lead for Census Reporter, a project funded by the Knight News Challenge that makes it easier for journalists and the public...

Giannina Segnini is the James Madison Visiting Assistant Professor of Journalism at the Graduate School of Journalism at Columbia University and the Director of the Master of Science Data Concentration Program. Prior to her academic career, Giannina worked as the editor of the investigative unit at La Nacion in Costa Rica. She has been involved in dozens of noteworthy investigative projects over the years, from building the offshore leaks database to assisting with shipping and trade data in the Panama Papers. Her work has garnered a long list of accolades...

Chris Groskopf is a reporter on the Things team at Quartz. Previously, he has worked on NPR’s visuals team and the news app team at the Chicago Tribune. Throughout his career as a data journalist, Chris has spent a lot of time building tools for other journalists, from the CSV kit to the PANDA Project to his most recent project, Agate, a data analysis library for Python. When not designing and refining data tools, he writes articles explaining issues from why it’s difficult to rig a US election to how...

Jonah Newman is a database reporter for the Chicago Reporter. His recent joint project on police misconduct settlements is an ambitious endeavour to make these cases more transparent by providing lawsuit data going back to 2012 in an easily searchable and sortable database. Jonah’s criminal justice coverage in the City of Chicago has included stories on racial profiling by the University of Chicago police department, and data visualizations on youth opinions on police and gun violence. Previously a database reporter at the Chronicle of Higher Education in Washington DC, Jonah...

After ten fantastic interviews with data journalists about their work, one point is already clear – data literacy is an increasingly essential skill. Matt Keifer of the Chicago Reporter puts it succinctly: "Data is getting to be ubiquitous. … I think that ‘data journalism’ is an anachronism waiting to happen. Just call it journalism." Daniel Hertz, Senior Fellow at the City Observatory, looks at open data when exploring a new story: "It usually starts with a question and you realize that this is an answerable empirical question, and the numbers are publicly available,...

Matt Kiefer is a data editor for The Chicago Reporter, an investigative news outlet that focuses on race, poverty, and income inequality in Chicago and beyond. Previously also at the Chicago Sun-Times and a Freedom of Information Act (FOIA) researcher at the Better Government Association, a Chicago-based investigative journalism non-profit organization, Matt spent two years as a software programmer before returning to the field of journalism where he now contributes to key data-driven reporting such as the Reporter’s recent “Settling for Misconduct”. In your own words, what is data journalism? Data is...

If you haven’t already heard of Chi Hack Night, you probably should. Originally known as Open Gov Hack Night, this weekly gathering of open data aficionados, data scientists and civic technology mavens is a fixture in the Chicago data ecosystem. The community shares a common mission “to build, share and learn about tools to create, support, and serve the public good.” When it started in 2012, the first meeting was an intimate affair attended by four people. Fast forward four years and renamed Chi Hack Night, the weekly event hosted by Derek...

In my previous post, I explored commonalities found in any given "data project." Here we'll run the first of the five gauntlets: data access. A few moons ago, we visited a customer to talk about some analytics he wanted to explore. Big company. Big ideas. Big IT infrastructure. The idea sounded great and we were happy to help 'em out. "All we need is the XYZ file," we said. Long pause. "Well, I can ask IT to pull an extract, but it will probably require an IT project. I'll call you next year." The phrase...

At this point, we've all heard the news. Data is growing at an exponential rate. Data is the new oil. Data is a strategic advantage. Data is big. Data is small. Certainly Goldilocks believes her porridge data is jusssst right. Meh. Data is simply a means to an end. So, how does one go from "data is super awesome" to actually getting value from it? Well, it's a long and winding road, often called "a data project." I recently came across a nice article discussing the process of building a predictive application with machine...