Metrics: Handle With Care

At the ALPSP and OASPA conferences in September, I said something that, as the Executive Director of a metrics standard, surprised a lot of people:

“Metrics can be dangerous, and should be handled with care.”

What I mean by that is that a number sitting on a spreadsheet (or in a JSON report) is, in and of itself, not very informative.

Whether a journal recorded 5,000 Unique Item Requests in August 2026, or a database reported 10,000 searches, raw numbers in isolation fail to tell a meaningful story. The true value of metrics emerges only when they are given context. That may be when they are compared across platforms, tracked over time to reveal trends, or synthesised with surrounding operational, financial, and qualitative data.

The Standalone Number Illusion

It is so very easy for us to fall into the trap of assessing the value of content we’ve bough based on a data snapshot. But static numbers don’t make for evidence-based decisions.

COUNTER-compliant publishers and technology providers carefully process their raw data according to the requirements of the Code of Practice. But even COUNTER metrics, such as Total Item Investigations or Searches Platform, often reflect user interface design rather than actual user intent. For instance, a website that automatically renders full text HTML of a journal article rather than forcing users to a simple abstract page is great for user experience – but we all know that many users will immediately choose to download the PDF without ever looking at that full text. That one user action has generated 2 Total Item Investigations and 2 Total Item Requests. Without deduplicating these events into 1 Unique Item Investigations and 1 Unique Item Requests, the standalone Total counts risk distorting actual engagement.

My mantra (plea?) to never use standard views applies here, too. Relying solely on pre-filtered summaries that omit crucial context, such as open access usage and emerging AI discovery patterns can make it appear that engagement is falling when overall usage is actually stable or growing.

Trend Lines and Indicators

One of the ways metrics become actionable is when we evaluate them as a continuous time-series. Tracking over time transforms static data points into usable, informative trend information. A journal with 20,000 Unique Item Requests per year in 2026 might seem to be a poor investment, compared with another journal reporting twice that number. But if you can show that usage for the smaller journal has grown from 2,000 requests in 2024, that upward trajectory can be a powerful signal of increasing need for the journal among your user community. Conversely, the higher-volume journal may be reporting a steady year-over-year decline. In combination, the two trends may indicate a shift in your organisation’s research focus.

Tracking metrics over time also allows libraries and publishers to evaluate the impact of intentional interventions. For example, a library may choose to promote a tool like GetFTR, and want to know whether usage has increased now that people can get access to subscription content while they are off-campus. Or a publisher could invest in platform usability upgrades (AI or not!) and want to know whether it’s had the desired impact on usage.

One of the reasons we aimed to keep the changes from Release 5 to Release 5.1 as small as possible was to minimise the disruption to this kind of trend tracking.

The Multi-Dimensional View

Usage metrics are most useful when we pair them with contextual, financial, and qualitative inputs. Using multiple dimensions of assessment reduces the risks that we’ll come to flawed conclusions.

We can start with the basics: cost per use. Subscription mean (relatively) little without usage, and usage means (equally relatively) little without price. Combining financial figures with normalised usage data yields the classic cost per use calculation. For those who’ve forgotten, that’s your annual subscription spend divided by the number of Unique Item Requests for the year. Evaluating cost per use trends over time provides a clear, objective benchmark for renewal and cancellation evaluations.

But we can go beyond simple finances, by integrating qualitative feedback. Metrics should complement, not replace, human judgment. By taking on board faculty feedback about their engagement with a title, such as editorial board participation, authorship, or society membership, we can get at a more holistic picture of the value of a subscription.

If you want to know more about how to combine COUNTER with other quantitative data, register for the COUNTER Academy.

Key Takeaways

• Standalone usage counts, just like all metrics, lack essential context and don’t lend themselved to good, evidence-based decision making.
• Monitor usage over multi-year periods to capture growth trajectories and seasonal patterns.
• Combine normalised COUNTER usage metrics with pricing, qualitative feedback, and research impact measures for a complete assessment.
• And of course, when you are using metrics, make sure they are based on normalised, community-governed standards (such as COUNTER!) so that comparisons remain fair and accurate.