Dear IATI Community,

Our IATI Datastore at lets you search, browse and download activities - but did you know we also have budgets and transactions data available?

Datastore Functions

Most people query activities via the Datastore website or API. When you do this you can get the original XML, a JSON representation of the XML and a representation of all the fields on the activity, including budgets and transactions.

But a few people will also be querying budgets or transactions directly. You can do this from the Datastore website by selecting that option in the download box, or via the API by changing the “collection” parameter. This will return data with one object per budget or transaction. When you do this via the API you can filter on any attribute of the budget or transaction or its associated activity.

Details of the bug’s scope

However, recently we discovered issues when querying budgets and transactions directly. These existing bugs cause an incorrect number of budgets or transactions records, or values for budgets or transactions to be reported on the wrong budget or transaction within an activity.  This specific release will fix these bugs.

Note that these bugs only affect people that query budgets and transactions directly in the Datastore. These bugs do not affect budget or transaction data in activity records returned from querying activities.

Changes we will make

To fix these bugs, we will be replacing part of the code that prepares data to be inserted into the Datastore. This code is also used for preparing Activity data, and will result in minimal changes to how Activities are stored. Because of this, we will be testing this change very carefully before releasing it.

The changes people will notice when querying activity data concerns empty or blank values - sometimes, for a field you would get a list of values that included some empty values. In the future, these empty values will not be there, and only values with actual data will be there. We don’t know of any use case that relies on these empty values, so we don’t think this will cause any problems. (If you have a use case that relies on these empty values, your code probably has the same bugs we have been working on and you should contact us to discuss that).

When querying budget and transaction data directly people will notice the same removal of empty or blank values, and fixes to the bugs we have identified.

Any questions? Contact the IATI Secretariat!

These changes are now in testing. We’ll update this post as we release them. Tip: click 'Follow Content' at the top of this post to receive a notification whenever we update this post.

If you think you may be affected by this or have any questions, let us know here or via our support email address :

Comments (4)

James (IATI Secretariat)
James (IATI Secretariat)

Hello, testing is going well and we hope to release this soon.

A note about date time parsing. In the IATI standard, it says that date or datetime fields should be xsd:date or xsd:datetime formatted, which is defined in

Firstly, this definition allows for years larger than 9999. But the new system won't handle years larger than 9999. We think this is ok, as we haven’t heard of a use case for data having values with years so big.

Secondly, a note on incorrectly formatted values:

If date and datetime values are formatted correctly with a year less than 10,000 the new system will process them just like the old system.

If values are formatted incorrectly but are still obvious, we will try and understand them anyway and convert them to a standard format.

But the way we handle these incorrectly formatted date values changes between the old system and new system, with the result that some incorrectly formatted values that the old system processed will not be processed by the new system and vice versa.

We don't make any promises about how we handle incorrectly formatted values; we will try our best, but this may change at any point.

If you are worried about date or datetime values in your data being understood, our advice will always be to make sure they are formatted as the standard says.

The IATI validator or CoVE tools will help with that by telling you if you use data with incorrectly formatted values. or

James (IATI Secretariat)
James (IATI Secretariat)

As well as the above work, this release also contains a small fix to the data in the "iati_json" field you can request via the API. Previously, some free text data was omitted when it was alongside children tags. This will also be fixed with this release. Details are at

We are hoping to start releasing this in a few days.

James (IATI Secretariat)
James (IATI Secretariat)

This release has now started. Any new data updated from now on will be processed using the new release.

We will reprocess all the existing data and when this is complete, we can say these issues have been fully fixed. We will do this reprocessing in batches to make sure the processing of any newly updated data is not held up for too long.

Any documents that are deleted from the Registry will still be deleted as normal - this release or reprocessing will not hold that up.

We’ll update when this is complete.

James (IATI Secretariat)
James (IATI Secretariat)

This release has now finished and all the data has been reprocessed.

As always, if you see any issues or have any questions do get in touch.

Please log in or sign up to comment.