Pathways to compliance: updates from our working group

We spun up two working groups at the first meeting of the new COUNTER Advisory Committee earlier this year. One was on generative and agentic AI, and the other on pathways to compliance. Earlier in September we wrote about AI, so this post describes progress in the Pathways group.

We want to tackle part of our second strategic objective, which is about growing our community to increase adoption and use of COUNTER metrics worldwide. Specifically, we’re looking at the call for us to “reduce barriers to compliance for report providers… who lack technical expertise” and who are not currently able to become compliant. 

Eligibility

Our scope of work is to create a structure that will allow eligible publishers to take steps towards being compliant with the Code of Practice. We agreed that the Pathway needs well-defined steps, and each step has to be an improvement on non-compliance. We’ll recognise and record the steps within the COUNTER Registry. We also agreed that there needs to be a clear definition of eligibility. As a starter for ten, we are likely to use the definition of small publisher from Section 9.3 of the Code, and restrict the Pathway to new publishers who aren’t already offering R5 or R5.1-compliant reports. And we also agreed that joining the Pathway should mean the publisher committing to becoming fully compliant with the next breaking release of the Code (not before 2030).

Aspects of compliance

The Code explains how raw usage logs are processed, what the COUNTER metrics are, how COUNTER reports are structured, and our technical validation and audit processes. 

The first aspect of compliance, data processing, is not up for discussion in this project. To be on the Pathway, usage data must be processed per the Code. 

Validation is the next step. We were unanimous that all publishers on the Pathway need to be able to successfully pass the technical structural checks applied by the COUNTER Validator. However, one of the biggest barriers to compliance is the cost of audit, so publishers on the Pathway will be exempt from audits for a limited time. 

When it came to metrics, we had more to discuss. All publishers on the Pathway will need to provide usage metrics (investigations and requests). But what about searches and denials? There’s a lot of variety in how search functionality works and therefore in search metrics, so we think it would be sensible to allow publishers to leave out search metrics. We know denial metrics are useful, but likely less valuable than usage metrics. Might they also be excluded? Precisely how to shape the Pathway, therefore, will depend on the results of our consultation.

Similarly up for consultation are report structure and formats. We agreed that dropping requirements for standard views would be sensible. (We leave Tasha’s delight at this decision to your imagination). The biggest challenge in shaping the Pathway is, surprisingly, the question of report delivery and format. Some small publishers have said that delivering JSON files through the COUNTER API (sushi) is a technical hurdle they cannot currently overcome. At the same time, some large consortia say that collecting tabular reports manually is too time consuming to be practical. We’ll be focusing a lot of attention on this topic in our consultation later this year. 

Get involved

If you’ve got a specific issue that you’d like us to consider, please email pathways@countermetrics.org to let us know. We’ll announce the consultation later this year on the blog and social media. And of course, keep an eye on our newsletters for updates!

The working group

The COUNTER team is very grateful for the enthusiasm and engagement of the Pathways working group members: Orazio Cappello, Karen Harker, Heather Heckman, Meghan Lenahan, Stuart Maxwell, and Bernd Oberknapp.