For the gap analysis, we consulted several experts, who provided invaluable input. We would like to thank them here:
For geospatial and weather data
Simon Cox (CSIRO Australia, Research Data Alliance)
Giovanni L'Abate (CRA Italy)
Ben Schaap (GODAN Secretariat)
Dan Brickley (Google, schema.org)
For farm and crop management data
Soonho Kim (IFPRI, ICASA standards)
Andres Ferreyra (AgGateway)
Hugo Besemer (Wageningen UR)
Francesco Benincasa (RDA Weather, Climate and Air Quality IG, Earth Sciences department of the Barcelona Supercomputing Center)
Allard de Wit (Alterra)
Christopher Brewster (TNO Netherlands)
The main findings of the gap analysis can be summarized as follows:
First, a caveat: according to most experts, the biggest challenges with weather data are related to issues of data availability, discoverability, quality, coverage and documentation; the lack of standardization is a challenge but less impeding than the above.
Both for weather data and for farm management data, standardization of variable names across the different communities and even within the same community is an issue. Some work has been done, from the Climate and Forecast Metadata Convention designed for NetCDF to the publication of BUFR and GRIB codes in RDF, but there is no naming convention that is universally used, nor mappings or cross-walks between variables used in different systems.
Generalizing the issue above, there are many different code lists used by different authorities (e.g. EU, WMO, Met Offices). Some correspondences / mappings have been developed, but mostly ad hoc for specific platforms and not in a reusable way.
Given the varied landscape of data providers and related formats, intermediaries still have to do most of the work converting, processing, re-purposing the data between the different steps in the data value chain.
Besides the official WMO network, many different platforms have been created for different consumers (researchers, FMIS developers, weather service developers, farmers, journalists) with very different data outputs.
Some experts noted that there is no way yet to use standards to clarify and enforce data rights and data ownership at each stage of the data value chain.
Based on this gap analysis as well as a “developer experience” review based on the farm management use case, we put together a set of key recommendations.
1) Using standards to improve discovery
Improving use of metadata standards to help catalogue and describe data published: e.g. encourage use of DCAT, DCAT-AP, STAT-DCAT AP, Data Cube, WMO metadata profiles.
Perhaps considering the option of creating a weather data application profile.
2) Usability of standards: developer documentation and tooling
This is in response to the issue of variety of formats used and the lack of documentation and clarity about formats and standards, which could be improved for different users:
for data publishers, provide a detailed evaluation and comparison of the existing formats with guidance on when and where to use them to publish data, and which conventions are useful;
for non-specialist data consumers, provide good introductory guidance on how to parse and use specific data formats within modern frameworks;
for both publishers and consumers, invest in open source tooling, e.g. parsers, validators and converters, to help developers publish and use the data;
creating some community support: e.g. explore third-party, community-led supporting activities, e.g. Question & Answering (Q&A) Services, to help data publishers identify relevant standards and understand how to apply them in the publication of their data and to connecti standards authors with the communities that publish and consume data.
3) Usability of standards: shared and linked vocabularies
This is in answer to the challenge of having many different code lists and no agreed lists of variables; work could be done to improve this, e.g.:
Identifying 2-3 key code lists that should be published in more linkable, versatile formats, at an authoritative domain (e.g. Climate and Forecast Convention, BUFR/GRIB codes, ICASA variables, AgGateway variables…).
Linking key code lists and providing web services for cross-walks.
Both the gap analysis and the recommendations will be soon published.
Work in the Standards focal area will now continue towards a) transforming the recommendations into technical specifications and then into pilot services; b) following the same cycle for the two new thematic topics selected by the project: nutrition data and land data.
Let’s talk about some of the challenges of opening data. 1) Too expensive! 2) We don’t have the technical knowledge! 3) We don’t have the storage space! 4) We don’t have the time! GODAN works with many organizations in all sectors facing these challenges, but there’s one challenge that comes...Open
A Global Open Data for Agriculture (GODAN) partner report has highlighted the need for open access and open data at universities to help solve world hunger and malnutrition. The report, 'Open Access and Open Data at PUSH Universities' , was...Read more
The Global Open Data for Agriculture and Nutrition (GODAN, Global Forum on Agricultural Research and Innovation (GFAR) and the Technical Centre for Agricultural and Rural Cooperation ACP-EU (CTA) will be running a final Webinar on Farmers’ access to data ...Read more
GODAN website is licensed under a Creative Commons Attribution 4.0 International License. This license applies to all contents on the website except where expressedly indicated on the page or in the document and except for contents aggregated from other websites, in which case the original licensing applies.