nsf.gov - National Science Foundation (NSF) Discoveries - Brown Dog: A search engine for the other 99 percent (of data) - US National Science Foundation (NSF)

" ... That's because digital data is often trapped in outdated, difficult-to-read file formats and because metadata--the critical data about the data, such as when and how and by whom it was produced--is nonexistent. Led by McHenry, a team at NCSA is working to change that. Recipients in 2013 of a $10 million, five-year award from the National Science Foundation (NSF), the team is developing software that allows researchers to manage and make sense of vast amounts of digital scientific data that is currently trapped in outdated file formats. The NCSA team, in partnership with faculty at the University of Illinois at Urbana-Champaign, Boston University and the University of Maryland, recently demonstrated two services to make the contents of uncurated data collections accessible. The first service, the Data Access Proxy (DAP), transforms unreadable files into readable ones by linking together a series of computing and translational operations behind the scenes. Similar to an Internet gateway, the configuration of the Data Access Proxy would be entered into a user's machine settings and then forgotten. From then on, data requests over HTTP would first be examined by the proxy to determine if the native file format is readable on the client device. If not, the DAP would be called in the background to convert the file into the best possible format readable by the client machine. In a demonstration at the Brown Dog Early User Workshop in July 2014, McHenry showed off the tool's ability to turn obscure file formats into ones that are more easily viewable. [Watch a video of the demo.] The second tool, the Data Tilling Service (DTS), lets individuals search collections of data, possibly using an existing file to discover other similar files in the data. Once the machine and browser settings are configured, a search field will be appended to the browser where example files can be dropped in by the user. Doing so triggers the DTS to search the contents of all the files on a given site that are similar to the one provided by the user ..."