Zimmerman and the IATI Secretariat are pleased to provide an update on the progress of the IATI Datastore project.
Over the last few months we have worked intensively to make a huge number of improvements to the tool, in response to community feedback during the testing phase, and to satisfy remaining requirements from the TOR. We’ve fixed bugs and added the required enhancements that enable the tool to be as robust and dependable as possible. We are working towards a release date of 22nd September.
What work has been completed?
A non-exhaustive list of these items include:
- 0.00 values not being output
- ‘Default’ elements being farmed to transactions
- Location output not matching published data
- Multiple description types not output
- Validator Integration
- Schema-valid XML output
- Several additional filters
- Streamlined parsing process
- Output CSV columns in Schema order
Remaining Requirements and official launch
As existing users will know, all items mentioned above have been implemented and are available to use now.
The only remaining requirement for the product to be fully launched is for sector and recipient-country expansion functionality to be added to the csv output. Sector and recipient-country expansion allows, if required by the user, the ability to repeat each activity or transaction (depending on the row-output choice) over multiple rows, for each sector or recipient-country that is reported.
The core functionality is already available on the tool, but the Zimmerman team is now working to make this available for all filters, and ensuring that all fields can be returned, as required.
Once the work on the sector and recipient-country expansion feature is completed, the tool will be ready for launch. The purpose of the launch is to actively promote the use of the IATI Datastore both to existing and new IATI stakeholders within the international development and humanitarian sectors. A comprehensive User Guide for the Query Builder will also be published at the time of launch.
As mentioned on a previous Discuss post, an important requirement is that queries that are made to the ‘old’ IATI Datastore API are redirected to the new one, with the equivalent query parameters used. This will ensure that any regular queries that utilise the existing tool can continue to return corresponding data from the new tool.
The timeline for deprecating the old Datastore and implementing the redirects is as follows:
From New Datastore communications Launch until 6 months after:
- Old Datastore remains live, as is.
- “Redirect version” of Old Datastore is also available for use. The “Redirect version” of the Old Datastore will provide users with the corresponding query to the New Datastore. This should help with any migrations before the Old Datastore is deprecated
6 months after Datastore official launch
- Old Datastore is deprecated and no longer available
- “Redirect version” of Old Datastore is deprecated and no longer available
- Any queries to the Old Datastore automatically redirect to a corresponding query on the new Datastore.
Post Launch Requirements
Following the launch, we will continue to work on implementing the following requirements previously identified and keep the Community updated as to when these will be released .
- Allow elements that can be returned at either transaction or activity level to be returned within the same query - #2081
- Add aggregated transaction data to the activity output - #2025
- Allow the user to choose between basic, complete, or custom csv output - #2295
- Normalise transaction values to USD - #2219
- Enable return of Validation results from the Datastore - #2296
As with all of our services, the IATI Secretariat will continue to monitor and respond to user needs with any necessary updates in the future, and will undertake consultations with the wider IATI Community about potential changes to the datastore and validator integration, including on topics currently under review on Discuss.
Get in touch
As ever, if you have further questions for the IATI Secretariat and Zimmerman, do post them here or email email@example.com.
IATI Secretariat & Zimmerman Team