Common validation errors: learning from others' mistakes
17 October 2024It’s been about a month since our new Validation Tool for R5.1 became publicly available (time flies!). Thus far, nine report providers – representing over a third of publishing platforms listed in the COUNTER Registry – have run reports through the new Validation Tool. With nearly 2000 validation attempts in the last week, we thought it might be helpful to dig through the results and highlight trends in the most serious errors with the COUNTER community.
Fatal Errors
We’re very pleased to see that just 5% of validation attempts returned Fatal Errors. Most commonly, these Fatal Errors crop up where the report provider has either uploaded a report header with no body, or the file encoding is not UTF-8. Both easy things to fix! The third type of Fatal Error is not quite such a quick fix: these are syntax errors in the JSON files, suggesting report providers might be trying to manually create JSON files to test instead of encoding data structures. Please learn from our efforts – the Technical Advisory Group can absolutely promise that it’s easier to do it right than to fix the problems arising from manually created JSON files! The Stoplight server has great information for development teams looking to get the COUNTER JSON schema right.
Critical Errors
The next most serious category of Validation Tool result, Critical Errors, are much more common at 41% of all validation attempts, with silly mistakes such as missing Institution_Name or other required fields in the report header accounting for a good number of them. Other trends we are seeing are:
- Missing required fields in report headers, including missing Institution_Name and incorrect Report_Name (e.g. Title Master Report instead of Title Report). Section 3.2 outlines precisely what you should be including in report headers.
- Missing or wrong Exceptions in reports without usage (e.g. Exception 3040 instead of 3030, 3031 or 3020). Appendix D of the Code of Practice lists handling errors and exceptions and how to use them.
- Duplicate or multiple Report_Items for the same item or parent, leading to double counting.
- And of course, poor metadata: for example, a missing Parent_Data_Type in the Item Report where other parent information (such as a book title) is provided, or using a custom Data_Type that isn’t in the Code of Practice. Check out Section 3.3 if you need a refresher on the COUNTER Data_Types.
Validating custom reports
The Validation Tool doesn’t support custom extensions at the moment. We know there are a few report providers looking to validate custom reports, and our advice to them would be to start by validating the reports without the custom extensions!