Website Search
Find information on spaces, staff, and services.
5.1 Defining Obsolescence and Sustainability
Technology changes quickly. With that change, certain previously preferred file formats and media will stop being used in favor of other, more advantageous ones—this is called obsolescence.
This is a concern when working with research data because we know that, as researchers, our current methods for creating, storing, and sharing data could potentially become obsolete and be replaced by new formats.
To help thwart obsolescence as best we can, we want to try to ensure that we pick the most sustainable options for our data over time. Sustainability in this case means formats or media that are less susceptible to changes in technology. Below you’ll find some tips for this.
Best practices for reducing the impact of changing technology:
In general, pick file formats that:
- Are non-proprietary: the format is not owned by a company or manufacturer. If a file can only be opened in a certain software, it’s likely that it’s a proprietary format. The .DOCX file type you may be familiar with, produced by Microsoft Word, is a proprietary format. Common proprietary formats often have new versions and software upgrades required to use the files. Over time, new versions stop accounting for the ability to open old versions of that format, and you can lose file information or the ability to read the file entirely.
- Have seen wide adoption in your discipline: Sometimes there isn’t a non-proprietary option for you to use and that’s okay. If a file format is widely used in your field, it’s likely that it will continued to be supported and openable into the future.
- Have a history of backward compatibility: new versions of the software are still able to support old file versions.
Physical devices and media:
Physical devices and media have a lifespan. While devices we use for storing data have improved exponentially over the last twenty years, expect external hardware devices to have a lifespan of 3 to 5 years. Plan on migrating your files every few years if you use an external hard drive to prevent data loss!
5.2 File Formats
| Type of Data | Recommended Formats | Acceptable Formats |
|---|---|---|
| Tabular data with extensive metadata variable labels, code labels, and defined missing values | SPSS portable format (.por) delimited text and command (‘setup’) file (SPSS, Stata, SAS, etc.) structured text or mark-up file of metadata information, e.g. DDI XML file | proprietary formats of statistical packages: SPSS (.sav), Stata (.dta), MS Access (.mdb/.accdb) |
| Tabular data with minimal metadata column headings, variable names | comma-separated values (.csv) tab-delimited file (.tab) delimited text with SQL data definition statements | delimited text (.txt) with characters not present in data used as delimiters widely-used formats: MS Excel (.xls/.xlsx), MS Access (.mdb/.accdb), dBase (.dbf), OpenDocument Spreadsheet (.ods) |
| Geospatial data vector and raster data | ESRI Shapefile (.shp, .shx, .dbf, .prj, .sbx, .sbn optional) geo-referenced TIFF (.tif, .tfw) CAD data (.dwg) tabular GIS attribute data Geography Markup Language (.gml) | ESRI Geodatabase format (.mdb) MapInfo Interchange Format (.mif) for vector data Keyhole Mark-up Language (.kml) Adobe Illustrator (.ai), CAD data (.dxf or .svg) binary formats of GIS and CAD packages |
| Textual data | Rich Text Format (.rtf) plain text, ASCII (.txt) eXtensible Mark-up Language (.xml) text according to an appropriate Document Type Definition (DTD) or schema | Hypertext Mark-up Language (.html) widely-used formats: MS Word (.doc/.docx) some software-specific formats: NUD*IST, NVivo and ATLAS.ti |
| Image data | TIFF 6.0 uncompressed (.tif) | JPEG (.jpeg, .jpg, .jp2) if original created in this format GIF (.gif) TIFF other versions (.tif, .tiff) RAW image format (.raw) Photoshop files (.psd) BMP (.bmp) PNG (.png) Adobe Portable Document Format (PDF/A, PDF) (.pdf) |
| Audio data | Free Lossless Audio Codec (FLAC) (.flac) | MPEG-1 Audio Layer 3 (.mp3) if original created in this format Audio Interchange File Format (.aif) Waveform Audio Format (.wav) |
| Video data | MPEG-4 (.mp4) OGG video (.ogv, .ogg) motion JPEG 2000 (.mj2) | AVCHD video (.avchd) |
| Documentation and scripts | Rich Text Format (.rtf) PDF/UA, PDF/A or PDF (.pdf) XHTML or HTML (.xhtml, .htm) OpenDocument Text (.odt) | plain text (.txt) widely-used formats: MS Word (.doc/.docx), MS Excel (.xls/.xlsx) XML marked-up text (.xml) according to an appropriate DTD or schema, e.g. XHMTL 1.0 |
Check Your Understanding
Open, non-proprietary file formats are in less danger from obsolescence. Which of these four file formats would you use for the following types of research data?
- .csv
- .flac
- .txt
- .tif
Images
Correct! .tif or TIFF is a more sustainable format for images. Some image formats can compress the image and lose some of the image information. The TIFF format helps avoid that.
Incorrect. The correct answer is D) .tif (or TIFF), which is a more sustainable format for images. Some image formats can compress the image and lose some of the image information. The TIFF format helps avoid that.
Spreadsheets
Correct! .csv or Comma Separated Value is a plain text, more sustainable format for spreadsheets or tabular data. Similarly to the text formats mentioned above, other tabular data formats like .xlsx from Excel are proprietary and rely on specific software to read them.
Incorrect. The correct answer is A) .csv, or Comma Separated Value, which is is a plain text, more sustainable format for spreadsheets or tabular data. Similarly to the text formats mentioned above, other tabular data formats like .xlsx from Excel are proprietary and rely on specific software to read them.
Text
Correct! .txt or plain text is much more sustainable for text based files. Other textual file formats like .docx from Microsoft Word include extra proprietary information and rely on specific software to read them.
Incorrect. The correct answer is C) .txt, or plain text, which is much more sustainable for text based files. Other textual file formats like .docx from Microsoft Word include extra proprietary information and rely on specific software to read them.
Audio
Correct! .flac or Free Lossless Audio Codec is a more sustainable format for audio files, though you may often see .wav and .mp3 also recommended.
Incorrect. The correct answer is B) .flac, or Free Lossless Audio Codec, which is a more sustainable format for audio files. You may often see .wav and .mp3 also recommended.
References
[1] UK Data Service, “Recommended formats,” University of Essex, University of Manchester and Jisc. Retrieved from https://www.ukdataservice.ac.uk/manage-data/format/recommended-formats.