Integrating data and analysis technologies within leading environmental research infrastructures: Challenges and approaches

Research output: Contribution to journalArticle

Abstract

When researchers analyze data, it typically requires significant effort in data preparation to make the data analysis ready. This often involves cleaning, pre-processing, harmonizing, or integrating data from one or multiple sources and placing them into a computational environment in a form suitable for analysis. Research infrastructures and their data repositories host data and make them available to researchers, but rarely offer a computational environment for data analysis. Published data are often persistently identified, but such identifiers resolve onto landing pages that must be (manually) navigated to identify how data are accessed. This navigation is typically challenging or impossible for machines. This paper surveys existing approaches for improving environmental data access to facilitate more rapid data analyses in computational environments, and thus contribute to a more seamless integration of data and analysis. By analysing current state-of-the-art approaches and solutions being implemented by world‑leading environmental research infrastructures, we highlight the existing practices to interface data repositories with computational environments and the challenges moving forward. We found that while the level of standardization has improved during recent years, it still is challenging for machines to discover and access data based on persistent identifiers. This is problematic in regard to the emerging requirements for FAIR (Findable, Accessible, Interoperable, and Reusable) data, in general, and problematic for seamless integration of data and analysis, in particular. There are a number of promising approaches that would improve the state-of-the-art. A key approach presented here involves software libraries that streamline reading data and metadata into computational environments. We describe this approach in detail for two research infrastructures. We argue that the development and maintenance of specialized libraries for each RI and a range of programming languages used in data analysis does not scale well. Based on this observation, we propose a set of established standards and web practices that, if implemented by environmental research infrastructures, will enable the development of RI and programming language independent software libraries with much reduced effort required for library implementation and maintenance as well as considerably lower learning requirements on users. To catalyse such advancement, we propose a roadmap and key action points for technology harmonization among RIs that we argue will build the foundation for efficient and effective integration of data and analysis.

Details

Authors
  • Robert Huber
  • Claudio D'Onofrio
  • Anusuriya Devaraju
  • Jens Klump
  • Henry W. Loescher
  • Stephan Kindermann
  • Siddeswara Guru
  • Mark Grant
  • Beryl Morris
  • Lesley Wyborn
  • Ben Evans
  • Doron Goldfarb
  • Melissa A. Genazzio
  • Xiaoli Ren
  • Barbara Magagna
  • Hannes Thiemann
  • Markus Stocker
Organisations
External organisations
  • University of Bremen
  • University of Queensland
  • National Ecological Observatory Network
  • Deutsches Klimarechenzentrum GmbH
  • Australian National University
  • Austrian Federal Environmental Agency
  • Institute of Geographical Sciences and Natural Resources Research Chinese Academy of Sciences
  • Commonwealth Scientific and Industrial Research Organisation (CSIRO)
  • Leibniz Information Centre For Science And Technology (TIB)
Research areas and keywords

Subject classification (UKÄ) – MANDATORY

  • Information Studies

Keywords

  • Data analysis environments, Data service providers, Research infrastructures, Scientific data analysis
Original languageEnglish
Article number101245
JournalEcological Informatics
Volume61
Publication statusPublished - 2021 Mar
Publication categoryResearch
Peer-reviewedYes