Last week at the 2013 CAR Conference, DocumentCloud's Lead Developer Ted Han joined a panel discussing how to get the widest impact out of a software project. 

Reporter's Lab has a thorough write-up of the panel, addressing the questions how much code is really re-used, and how to measure the success of a software project.

Success for an open-source project is different from a commercial one. As Han said during the panel, the big difference between commercial and open source start-up projects is “less a question of how you operate and more a question of your values.” 

The ...

Read more ...

If you’re working from a Mac computer or getting into truly large datasets, Access may not be a viable option as a database manager. (It doesn’t work on Macs, and there are row limitations.) A good alternative is MySQL, an open-source database manager, which Alex Richards of the Chicago Tribune taught on Friday.

MySQL essentially installs a server that runs in the background on your computer. But if you don’t really understand what that means, don’t let it scare you. There are a handful of secondary applications that let you interface with MySQL in a way ...

Read more ...

Knight-Mozilla OpenNews yesterday announced its 2013 class of fellows, who will work as developers and technologists embedded in newsrooms around the world. Fellows spend a year writing code along with reporters, designers and newsroom developers to create new tools for journalism. Fellows for 2013 and their placements include:

• Brian Abelson, The New York Times
• Manuel Aristarán, La Nacion
• Annabel Church, Zeit Online
• Stijn Debrouwere, The Guardian
• Friedrich Lindenberg, Spiegel Online
• Sonya Song, Boston Globe
• Mike Tigas, ProPublica
• Noah Veltman, BBC

Learn more about the fellows from Knight-Mozilla OpenNews.

By Anna Boiko-Weyrauch
@AnnaBoikoW

Syntax error. What does this bit of code do? Syntax error. Let’s go back to the source. Syntax error. Maybe try this?

After two hours of educated guesses, trial, error and some friendly help, Pam Dempsey, of cu-citizenaccess.org, and I had finally scraped our first bit of text: the word “2011” from a page of Illinois nursing home inspection records.

The ScraperWiki session on Thursday night was aimed at “liberating” a number of data sets from their online prisons by working late into the night scraping real websites.

The session started with a two-hour ...

Read more ...

Your average CAR geeks - especially the old timers - follow a predictable route in tools they use for data analysis and sharing.

You start with Microsoft Excel spreadsheets, and everything’s going fine. But then someone tells you about relational databases, and suddenly you notice all of the things you can’t easily do in Excel.

Step up to Microsoft Access database manager and pretty soon you’re joining tables right and left, slipping terms like "Group By" and "normalization" into conversations and generally feeling pretty good about yourself.

But at some point, someone in your newsroom looks over your shoulder ...

Read more ...

Data entry hell.

We’ve all been there. It’s the place where a team of 17 enters information off of handwritten expense forms into a database. It’s the no-man’s land where your summer interns type in the results of three months’ worth of records requests. It’s you combining summer camp listings from no fewer than 12 spreadsheets, each with its own adorable little quirks and styles.

Often, newsrooms tackling these kinds of monster projects turn to the least-common denominator method in the name of convenience. Generally, that means using Microsoft Excel spreadsheets or an Access database ...

Read more ...

For journalists hoping to analyze geographic data, the cost of geographic information system (GIS) software can be prohibitive. Fortunately, there are open-source, free solutions available for cost-cutting journalists who want to do spatial analysis.

Pursuing an open-source option is easier said than done. Often, open-source software can be difficult for the lay journalist to install and even begin to understand.

PostGIS for PostgreSQL database manager offers a solution that is free, robust and easy to use — assuming you know what you're doing.

Granted, when I received the assignment to install PostGIS for PostgreSQL on my Mac to handle spatial ...

Read more ...

Gary Sherman's Desktop GIS: Mapping the Planet with Open Source Tools

When journalists are looking for software, they usually greet the words "open source" in one of two ways: with confusion, because open source software is still a daunting mystery; or with delight, because the software is available for free.

Open source software is simply software that is available at no cost and has its source code available to the public. A network of users and developers constantly enhances and expands the program.

Journalists doing CAR have been using robust open-source tools for editing documents, analyzing spreadsheet data and ...

Read more ...

By now you may have read that EveryBlock, a Knight Foundation-funded project, has released its source code to the public (here's a browsable version). Getting a chance to look under the hood is a great opportunity to see how other folks tackle some of the tasks we all face, or are likely to. The first thing to note is that the code has the GPL license, which means that if you incorporate any of it into an application you're building and then release that code, it will need to be under the terms of the GPL as well ... Read more ...

I had the privilege of speaking on a panel with Sarah Cohen and Steve Doig last week in Baltimore about the future of computer-assisted reporting. Whoever thought I even belonged in the same room as those two gave me way more credit than I deserved. But in preparing for that panel, I got to thinking: What skills and software tools are we going to be using in 10 years? What skills should we start learning now if we want to be prepared for the future? Or better yet: What types of problems in newsgathering and investigations could technology best help ... Read more ...