DQI Consultation: 1.3 Timeliness - Spend Updates and 1.4 Timeliness - Active Publishers

Moderators

Anna Whitson - IATI Secretariat
Outreach, Partnerships and Engagement Specialist • UNDP / IATI Secretariat

Data Quality Index Consultation - sub-section 1.3 Timeliness - Spend Updates and 1.4 Timeliness - Active Publishers

Instructions for submitting your feedback

1. Read through the proposed methodology for this measure and / or download the PDF at the bottom of this page;

2. Share your feedback through the comment- box below, consider the guiding questions in your comments and include the question number in your response;

3. And finally you can suggest track-changes or add comments directly on the specifics of the methodology of the Timeliness and Validation measures - go to this DQI Live-Editing-Page

Proposed Measure - 1.3. Timeliness - SPEND UPDATES

Please find below the proposed methodology for this measure. Only active activities* will be assessed in the Timeliness measures.

*Active activities refer to activities which have an actual-start-date in the past and an actual-end-date in the future. If actual start and end dates are not present, the planned start and end dates will be used.

DEFINITION assess if an organisation has published timely spend (transaction + expenditure) updates.

OBJECTIVE / EXPLANATION

Data quality objective: Publication of timely spend data in all activities.
Based on the methodology, this is measuring whether all activities contain spend transactions for the month being assessed. For example, if a publisher has 100 activities in March, it will assess the percentage of those activities that contain at least one disbursement or expenditure transaction with a transaction date in March.
As such, this will motivate publishers to ensure that all of their activities contain at least one spend transaction.
Bigger picture, the goal is to motivate publishers to publish timely spend transactions in all of their activities.

OUTPUT

Percentage

METHODOLOGY

Count the number of activities with disbursement and / or expenditure transactions for each of the past 12 months.

Divide by total number of activities per month.

Please find below a visualisation for this proposed measure. Do note that this has been created to help participants picture what the DQI could look like. It is not final, nor part of the proposed methodology.

Guiding question - please refer to the question number when you respond via the comment box below!

1. Should the total activity count (denominator) only include active activities or all activities?

If only active activities, this would be assessed using the activity dates.

2. Should aggregates also be assessed and visualised?

For example, to assess the percentage of activities that contain spend transactions for each quarter or for the year?

Proposed Measure - 1.4 Timeliness - ACTIVE PUBLISHERS

Please find below the proposed methodology for this measure.

DEFINITION assess if an organisation has updated their data within the past 12 months.

OBJECTIVE / EXPLANATION

Data quality objective: reduction in the number of publishers failing to update their data at least annually.
Based on the methodology, this is measuring whether an organisation has made any update to its file in the past 12 months. It will help data users to easily identify organisations that are inactive and thereby unlikely to have recent or good quality data.
As such, this will motivate publishers to update their data so that they can be moved from inactive to active.

OUTPUT

Binary flag Active / Inactive.

METHODOLOGY

Calculate difference between current last-updated-date and stored last-updated-date;
If difference is 12 months or greater, mark the publisher as inactive.

Guiding questions - please refer to the question number when you respond via the comment box below.

3. Should this measure rely on each publisher's reporting of the last updated date? Any other suggestions?

4. Should the information on whether a publisher is active or inactive be added to all pages (with various different measures) of the DQI visualisation?

Webinar

For each discussion, the IATI Secretariat will organise a webinar to explain the proposed methodology, answer questions and further explain how to engage.

Please find an overview of the most frequently asked questions of the Timeliness and Validation webinar here.
Missed the DQI Webinar on Data Completeness held on March 30? Watch the recordings here or read the summary here!

BACK TO MAIN DQI-PAGE

Files

Spend updates and active publishers.pdf68.69 KB

Comments (8)

Anna Whitson - IATI Secretariat • 1 year ago Moderator

Dear members of the IATI community,

As moderator of this consultation on the forthcoming IATI Data Quality Index, a warm welcome to you all! Thank you in advance for your inputs, which will no doubt provide invaluable as we work toward a DQI that supports our publishers to better understand and improve the quality of the data they publish. On behalf of the Secretariat, again, welcome!

-Anna Whitson; Outreach, Partnerships and Engagement Specialist, IATI Secretariat

Yohanna Loucheur • 1 year ago

Questions 1 & 2: this indicator should be removed, as it would say nothing about data quality.

"measuring whether all activities contain spend transactions for the month being assessed."

The objective is to "motivate publishers to ensure that all of their activities contain at least one spend transaction" every month. This is not part of the IATI standard expectations, and definitely not realistic.

If data users are expecting all activities to have transactions every month (if we have evidence for this, could it be shared please?), perhaps we need to devote more energy explaining how development and humanitarian cooperation work.

Regarding "active activities" (also known in the IATI Standard as status code 2, Implementation), please note that "actual-end-date" cannot be (well, should not be) in the future. This is probably meant to say "planned-end-date".

leo stolk • 1 year ago

Less is often more, less disbursement can be (and is often) a reflection of higher quality program implementation (at least for an INGO), more trust in next tier implementing organizations, more mature relationship, more local leadership. More transactions means piecemeal funding of implementing partners or INGO direct implementation.

So i do not understand the logic of this ruleset.

Anna Whitson - IATI Secretariat • 1 year ago Moderator

Good point! Any thoughts from the brain trust behind the proposed methodology, Sarah McDuff - IATI Secretariat and Amy Silcock ?

leo stolk • 1 year ago

Active org check threshold should be quarterly in my opinion. Is updating once per year a sign of being 'active'?

Anna Whitson - IATI Secretariat • 1 year ago Moderator

Good question! The "rule of thumb" and best practice for publishing within IATI is quarterly, but there are many publishers who don't (or aren't able to) meet this threshold (including smaller organizations). Therefore, it would be useful to hear from you and other members of the community where the bar should be!

leo stolk • 1 year ago

Just to add an ingredient to this discussion. I sympathize with smaller organisations too. But the frame that they, publishing via Aidstream, face special difficulties to update their smaller dataset is not necessarily correct. Organizations with medium or larger datasets, may find it more cumbersome to update quarterly, than those with small number of activities

Amy Silcock • 1 year ago

Thanks to everyone for their contributions here. With my data use hat on: the availability of more real time data in IATI (compared to other sources like OECD) is where considerable value is derived by data users. However, data users continually face challenges in knowing how complete or current the data published is. In addition, spend data, based on considerable user research and our engagement with data users, is essential for many of IATI’s data users. As such, this measure (and the Spend Updates measure (1.3)) are intended to help assess the timeliness and predictability of the provision of spend data. It's well understood that organizations may not have spend every month and the objective is not to penalize organizations for their disbursement schedules but rather to enable data users to be able to understand how current the spend data is (at any point in time – to build understanding/trust/confident in the data they are analyzing) and whether this information is provided in a timely fashion (e.g. how long after the end of the calendar year has spend been provided for the last quarter). The measure simply provides this information; how this measure is weighted will determine how publishers are actually assessed on this measure and this will be determined in phase 3 of this consultation where we very much welcome your inputs on how we can ensure a useful measure that is fairly assessing publishers timeliness in the provision of spend data.

Please log in or sign up to comment.