The awesome developers at The New York Times’ Open blog have just posted aboutNYT_Transformer, a tool for converting between various data formats (XML, comma-separated files) and data storage mediums (flat files, databases).
This isn’t the first time such a conversion utility has been written–Babel comes to mind. The NYT_Transformer has a few perks in its favor, however, namely that it seems to be used in heavy production at The New York Times, and that it allows you to convert between databases and flat files (nice touch!).
Written in php, the tool is geared toward web applications. An immediate thought, aside from batch jobs for various internal projects, would be to use the tool for a greater purpose: a data converter for data.gov.
While data.gov is a nice start at centralizing the directory of the U.S. government’s raw data feeds, it lacks a utility for converting between various data formats. Given that the sunlight foundation is currentlyrunning a competitionto build tools on top of data.gov, this is the perfect opportunity to go meta and build a data.gov browser with automatic format conversion. That would help standardize the site a bit, and would be a nice signal for the folks at data.gov as to the file formats that people actually want for various datasets. If you’re interested in working on this project, let me know!