Blog post - OA Switchboard
Reporting Made Easy
Our partners share their insights on the rationale behind the need for reporting
4 May 2021 by Yvonne Campfens, Executive Director OA Switchboard
with contributions from Stacey Burke (American Physiological Society), Colleen Campbell (Max Planck Digital Library, ESAC, OA2020), Todd Carpenter (NISO), Helen Dobson (Jisc), Matthew Goddard (Iowa State University), Marten Stavenga (John Benjamins Publishing Company) and Ivo Verbeek (Elitex)
Open Access (OA) output is growing year-on-year (see OASPA's annual survey and ESAC's Market Watch), and there is widespread belief that research will function better if results are made openly available to the community. This growth necessitates, as a priority, the development of infrastructure to streamline communication between stakeholders and enable the fulfilment of OA publication-level arrangements. The founding partners of the OA Switchboard felt that for OA to be supported as the predominant model of publication, a practical solution was needed, mission-driven and thriving on collaboration.
Today, the OA Switchboard is live, supports many use cases, and can be called on as and when needed, or integrated into stakeholders’ systems and workflows. With launch customers and founding partners we are currently collaborating on a number of use cases as a priority. The first one is ´Reporting Made Easy´, and is all about reporting from publishers to institutions/consortia and funders.
We took a step back to speak with some of our partners about the rationale behind the need for a standardised, structured and validated data format, delivering real-time, situational authoritative data from the source. We’re grateful they agreed to share their views and experiences. These partners are Stacey Burke (American Physiological Society), Colleen Campbell (Max Planck Digital Library, ESAC, OA2020), Todd Carpenter (NISO), Helen Dobson (Jisc), Matthew Goddard (Iowa State University), Marten Stavenga (John Benjamins Publishing Company) and Ivo Verbeek (Elitex).
We asked our interviewees to answer five questions:
What is the underlying need for ‘reporting’?
What is the minimum set of metadata required to achieve that goal?
What sources (systems) capture and manage these (meta)data? Is it possible to extract the data?
What makes a ‘standard’? What’s the benefit of a ‘standard’? How to get there?
How does the OA Switchboard make reporting ‘easy’? How does it work, end-to-end and real-time?
Read a small selection of what they have to say, followed by the full interview responses below.
“Structured data and reports are an essential basis for managing the transition of the scholarly journal publishing market to open access-based business models”, says Colleen Campbell. Helen Dobson adds: “Institutional reporting on publication outputs, particularly open access outputs, is frequently requested by both external stakeholders, such as funders, and internal stakeholders...” For a society, Stacey Burke says: “Academic publishing is an essential part of our mission… Ensuring the Society can support the publishing needs of our community with the OA pathway they need to meet funder requirements and make their published research more widely available requires accurate data and precise connections to that data.” Marten Stavenga: “For a publisher with a portfolio in social sciences and humanities, a sustainable economic model to transform the subscription-based portfolio to OA is not straightforward. In order to model our options internally, to discuss them with our library clients and to manage the roll-out, we need data and reports for both internal and external use.”
For a minimum set of metadata to achieve the goals, Matthew Goddard states: “While standards like the ESAC recommendations provide a surer guide to what will serve the community as a whole, the elements we require are article title and DOI, journal title and ISSN, corresponding author name and e-mail address, and cost where applicable.” For Jisc, Helen Dobson adds to this list: article type, institution of the corresponding author, funder data and license. The current ESAC recommendations also include: complete statement of the author’s affiliation to the paying institution, date acceptance and publication.
On the question regarding source systems, Todd Carpenter says: “One challenge is that there are a lot of legacy systems in our ecosystem—both on the publisher and library side—that were not built to talk to other systems, particularly in the way that the demands of an OA exchange might require.” Marten Stavenga adds: “We’ve started manually, but will want to automate the collection, combination and transformation into JSON. There are simply too many different sources and too many different types of data.” Stacey Burke: “Societies operate differently and many technology solutions were established to support membership business so the systems did not 'talk' to each other. APS was able to assist in this data conversation through the integration of GRID...”
“Over the last months we’ve helped various publishers connect to the OA Switchboard via a so-called ‘custom connector’” says Ivo Verbeek. “This involved sourcing and combining data from not only different systems, but also of varying nature... The biggest challenges were finding common identifiers across the sources and the (lack of) interoperability of legacy systems. PIDs help and enable!”
Todd Carpenter from NISO says: “A core feature that makes a standard a standard, is its application. Whether a technology is formally approved or not, the key in determining whether something is a ‘standard’ is the extent to which people use the technology.... A core requirement is that common need, or common business rationale which drives the standard’s use by stakeholders.” Helen Dobson adds: “Agreement and usefulness 'make' a standard, such as COUNTER.”
“OA Switchboard engages all of the stakeholders…” says Colleen Campbell. “...there is obviously the operational element: OA Switchboard facilitates the creation of a common language… and a common way of communicating (via shared infrastructure). This brings transparency and, ultimately, the economic benefits of efficiency and cost-effectiveness.” Helen Dobson adds: “OA Switchboard also contributes to the process… to get standardisation of reporting. It funnels knowledge and experience, and takes a global approach, based on best practices.”
“The OA Switchboard makes reporting easy in a number of ways. Real time – on any given day, we can know where we stand with each publisher connected to the Switchboard… Standardized – OA Switchboard’s standardized messaging protocol means that we don’t need to spend time synthesizing widely variable reporting formats, or manually filling in the gaps of missing information. Flexible –the reporting data goes where we want it” as Matthew Goddard sums it up.
Read on for their full answers to our five questions!
What is the minimum set of metadata required to achieve that goal?
What sources (systems) capture and manage these (meta)data? Is it possible to extract the data?
What makes a ‘standard’? What’s the benefit of a ‘standard’? How to get there?
How does the OA Switchboard make reporting ‘easy’? How does it work, end-to-end and real-time?
and read also: Summary - Reporting Made Easy by the OA Switchboard
1. What is the underlying need for 'reporting'?
Todd Carpenter: Transforming an entire industry is never an easy task and one that is bound to create a great deal of disruption and unintended consequences, along with the hoped-for positive outcomes. Massive pools of resources will need to be redirected and many institutions will add to their burden as others, possibly the entire ecosystem as a whole, benefit. Data related to how this process is functioning will be an essential feature of understanding what is happening in this transition, how it is working, what implications it may have, and what sort of course-corrections may be necessary as scholarly communications shifts from a model where the consumers of that information pay, to one in which the creators of that information pay, either directly or indirectly.
Specifically for advancing open access and transformative agreements, or subscribe to open deals, the entire community of publishers, librarians, funders and researchers have a symbiotic need for similar data. These data prove that they are doing the things they say they are, the financial elements of that exchange, and the impact of those expenditures, each for their own purposes and each to suit their own missions or goals. Librarians need to justify their expenses to serve their constituencies, publishers to validate and improve their business, funders to assess their contributions and the trends they hope to drive, and the researchers to most broadly share their results. The question is how to do each of these interlocking elements get measured.
Matthew Goddard: Consistent, accurate and timely reporting across all of our open access agreements is crucial for assessing their value. This not only informs our strategy with respect to particular agreements, but also provides key insights into the various open access funding models we’re exploring and allows us to demonstrate the value to the university of the library’s open access initiatives.
Helen Dobson: Institutional reporting on publication outputs, particularly open access outputs, is frequently requested by both external stakeholders, such as funders, and internal stakeholders, such as University Senior Management teams. External reports are most often required to ensure and demonstrate compliance with the funder requirements and to evidence claimed expenditure. Internal reports are required for more varied purposes. It may be to evidence a business case for a subscription to a transitional agreement; to evidence the need for budget support; to demonstrate and understand the research output of the institution or to analyse the funder support secured by the institution. Reporting on publication outputs is also crucial for consortia to identify publishers with whom an agreement may be of benefit to the consortium, understand the value of transitional agreements for their members and the impact of them on the transition to open access. By understanding the publication output of an agreement at the consortium level it is possible to ensure transitional agreements are constraining the costs of reading access and publications and that the agreement is meeting the needs of the members i.e. consortium authors are still publishing with the publisher, and the benefit of the agreement is equitable across the consortium. By capturing historic publication patterns alongside transitional agreement output it is also possible to track and evaluate the transition to open.
Colleen Campbell: Structured data and reports are an essential basis for managing the transition of the scholarly journal publishing market to open access-based business models. They are needed not only for assessing the progress of (transformative) deals in delivering on their stated objectives, but they also form the foundation for negotiating subsequent agreements that are ever more ambitious and incisive. With good quality, authoritative and structured data, institutions and consortia can monitor the impact of their agreements locally, conduct bibliometric analysis to better understand the needs and expectations of their authors, and the research community at large can monitor scholarly journal portfolios as they make a definitive shift to open access.
Stacey Burke: Academic publishing is an essential part of our mission as a society at APS. Ensuring the Society can support the publishing needs of our community with the OA pathway they need to meet funder requirements and make their published research more widely available requires accurate data and precise connections to that data. To this end we not only need to connect data internally, but also connect data externally. This same data gives the Society the ability to provide our researcher’s institutions the accurate information they need to enter into transformative agreements like our Read, Publish & Join model. Of course, all of this requires data validation that flows throughout the publication process from submission to publication and beyond to institutions and funders. Getting all this right, and being able to report on it to all stakeholders accurately, is critical to help our community accelerate growth and collaboration through OA and while initiating sustainable progress for the Society.
Marten Stavenga: For a publisher with a portfolio in social sciences and humanities, a sustainable economic model to transform the subscription-based portfolio to OA is not straightforward. In order to model our options internally, to discuss them with our library clients and to manage the roll-out, we need data and reports for both internal and external use.
2. What is the minimum set of metadata required to achieve that goal?
Colleen Campbell: The ESAC recommendations for Open Access and Transformative Agreements were developed in 2017, at the 2nd ESAC Offsetting Workshop attended by libraries, funders and publishers from seven European countries, the United States and Japan. And, as the library community and publishers alike gain more experience in the mechanics of supporting authors in openly disseminating their research articles, the importance of metadata has come ever more into focus. In order to manage the administration of central institutional or consortium-level agreements with scholarly publishers and track the scholarly communication services provided to authors, the ESAC recommendations require publishers to provide machine readable reporting data including lists of eligible, rejected and opt-out articles, as well as data on eligible articles have been published and accounting details such as APC payment dates. The current ESAC recommendations designate the following metadata to be provided in publisher reports to the customer institution/consortium: Name and email address of the author who is affiliated to the paying institution (must be the corresponding author); Complete statement of the author’s affiliation to the paying institution (e.g. university, institute, department); Funding organisation (research funder and grant ID); Date of acceptance; Date of publication; Journal title; ISSN; Article title; Article type; DOI and link to the published article; Amount due; Discounts and discount group (if applicable); CC license.
Helen Dobson: We require the following metadata fields to undertake the analysis of the agreement: DOI, Journal title, ISSN, article type, corresponding author, institution of the corresponding author, funder data, licence. In 2019 Jisc participated in a study focused on analysing Knowledge Exchange countries' agreements with major publishers... to identify what article-level metadata consortia and academic institutions request from publishers and whether publishers deliver this... To promote the consistent delivery of article-level metadata by publishers to consortia and academic institutions, the KE Monitoring OA group repurposed the article-level metadata check-list as a template for publishers to use as a reporting tool.
Matthew Goddard: While standards like the ESAC recommendations provide a surer guide to what will serve the community as a whole, the elements we require are article title and DOI, journal title and ISSN, corresponding author name and e-mail address, and cost where applicable.
Todd Carpenter: When people seek to define a minimum set of metadata, three core features must be considered: What fundamentally do I want to describe? What data is available to describe that thing? and What are the sources and availability of those data? With OA publication, the thing in this case is the article (or other research output, if one is to be broad in one’s definition of the landscape). Its key attributes are the author, its publication (and therefore the publisher), the publication’s associated costs, as well as the business relationship with the author’s affiliation, and/or potentially the author’s funding body. Some of these items have identifiers such as the author’s ORCID, the institution’s ROR/GRID/ISNI, the publication’s DOI/ISSN, and possibly the funder. Beyond this, the date of the transaction and the license terms, ideally expressed using the NISO ALI standard, are important.
3. What sources (systems) capture and manage these (meta)data? Is it possible to extract the data?
Stacey Burke: To start out we needed to get some of our internal systems to collaborate with universal & industry standard solutions that could validate our data from our publishing processes and connect to the Society membership data. Societies operate differently and many technology solutions were established to support membership business so the systems did not 'talk' to each other. APS was able to assist in this data conversation through the integration of GRID, a unique institutional identifier database, in our submission and production systems as well as in our membership management system. However, to add further complexity, each of our transformative agreements are unique to the institutions and these systems did not connect to our librarian partners or to the funders that require OA as part of their grants. To get proper data flowing internally and externally, there would be a lot of merging and spreadsheets to accomplish what the OA switchboard was able to help in one simple report.
Marten Stavenga: You can’t do this manually! We’ve started manually, but will want to automate the collection, combination and transformation into JSON. There are simply too many different sources and too many different types of data. It is not as if these data are ever in any single system. At least not in the short term. If the automation is done, metadata is automatically available machine-readable for the recipient.
Helen Dobson: Data is captured from the publisher for Jisc Transitional Agreements which is usually exported by the publisher from their editorial and financial management systems. The editorial systems provide the article-level metadata whilst the financial management systems help in identifying who is the payee institution in the agreement rather. Some publishers have recently developed new systems which capture both the data together, but we understand from some publishers that it is quite difficult to join the editorial and financial data. We do also utilise other data sources such as Crossref, CORE, and PubMed which all have accessible and free APIs. These APIs allow us to use the unique DOI to query the API and return any missing metadata we may have not received from the publisher in their initial report or where we want to investigate further, i.e., has the article been deposited in a repository or enhancing funder data to the Crossref standard.
Todd Carpenter: One challenge is that there are a lot of legacy systems in our ecosystem—both on the publisher and library side—that were not built to talk to other systems, particularly in the way that the demands of an OA exchange might require. While research shows an acceleration of new implementations in the past 20 years, the appetite to change is low, while the need for open systems and APIs is growing. A variety of protocols and intermediary systems have developed over the years that address this challenge including formats like the DOI, KBART, ALI and COUNTER, each with functionality that can be used for exchanging elements of the required information to report on open access transactions. One barrier is related to where these data reside. Some of relevant information might exist in an editorial system, such as information about the authors, whereas other data might exist in a business system, such as a customer relationship management (CRM) system, which are most likely not designed to interact.
Additionally, identifiers such as ORCID, Crossref's Funder ID, and ROR are important connectors that will improve the eventual reporting requirements, yet are not universally adopted. Leveraging all these existing standards will speed implementation.
Ivo Verbeek: Over the last months we’ve helped various publishers connect to the OA Switchboard via a so-called ‘custom connector’. This involved sourcing and combining data from not only different systems, but also of varying nature. Think of editorial and payment systems, JATS XML at article level, client-level deal information, etc. The biggest challenges were finding common identifiers across the sources and the (lack of) interoperability of legacy systems. PIDs help and enable!
4. What makes a 'standard'? What's the benefit of a 'standard'? How to get there?
Todd Carpenter: A core feature that makes a standard a standard, is its application. Whether a technology is formally approved or not, the key in determining whether something is a ‘standard’ is the extent to which people use the technology. This can be achieved via various routes, for instance if it is obvious the technology makes everyone more efficient and people drive its use; if a major stakeholder requires implementation; or, if a major customer demands it. What will drive the community to that state varies. A core requirement is that common need, or common business rationale which drives the standard’s use by stakeholders. Getting to a formal standard is a slightly different process—though they can overlap—by which the community seeks to achieve a desired outcome (i.e., common use/application) through a consensus process.
Agreeing on a common approach usually involves a process of exploration, and then coming to agreement around what is technologically feasible, practically implementable, and makes business sense for both parties.
Helen Dobson: Agreement and usefulness 'make' a standard, such as COUNTER. Although relatively simple in one sense, it provides 'consistent, credible, and comparable usage data.' The ostensible benefit of having and/or providing a standard is the relative simplification of the information that is provided.
5. How does the OA Switchboard make reporting 'easy'? Hoe does it work, end-to-end and real-time?
Stacey Burke: The Society needs integration of systems across the association for everyone’s sake-we need it to be: frictionless for authors, verifiable for institutions and easy for APS staff. We started our ‘Read, Publish & Join’ program with one partner, Iowa State University, and the implementation was entirely manual. The OA Switchboard enables us to meet the reporting requirements of our clients without manual intervention. It enabled us to advance our OA program at scale and seal more partnerships, most recently with Jisc.
Matthew Goddard: The OA Switchboard makes reporting easy in a number of ways. Real time – on any given day, we can know where we stand with each publisher connected to the Switchboard. There’s no need to wait weeks or months to find out after the fact how our investment has been used. Standardized – OA Switchboard’s standardized messaging protocol means that we don’t need to spend time synthesizing widely variable reporting formats, or manually filling in the gaps of missing information. Flexible –the reporting data goes where we want it. There’s no need even to log into a separate system because the messages are automatically delivered to Oable, our OA agreements workflows dashboard from Knowledge Unlatched.
Colleen Campbell: OA Switchboard engages all of the stakeholders in the scholarly publishing cycle together around the shared principle of open dissemination of research results for the benefit of scholarship. As it is a collaborative initiative, there is also an important educational component in enabling the different stakeholders to see how their own processes connect to those of the others. Then, there is obviously the operational element: OA Switchboard facilitates the creation of a common language (which is a huge challenge but utterly vital in the process of transition) and a common way of communicating (via shared infrastructure). This brings transparency and, ultimately, the economic benefits of efficiency and cost-effectiveness.
Helen Dobson: Reporting and communication are streamlined among three main stakeholders involved in an open access publication: publishers, funders, and institutions. One of the major pain points for our institutions is the difference in reports received and the range of reporting requirements. Another is the accessibility of the data that is needed to compile and produce a report. OA Switchboard can provide a relatively easy flow of trusted information among those involved. This would provide an institution all that they needed to comply with funder policies, for example. We don’t want to make it harder for publishers, and OA Switchboard relieves them from the need to use external sources for their own data. OA Switchboard also contributes to the process with all stakeholders and NISO, to get standardisation of reporting. It funnels knowledge and experience, and takes a global approach, based on best practices.
Summary - Reporting Made Easy by the OA Switchboard
Think of the OA Switchboard as COUNTER, but for reporting on OA-related publication level information. Like the COUNTER Code of Practice (providing technical specs and instructions), the OA Switchboard provides documentation and message structure details in the repository. Our version of their ‘friendly’ (plain language) guides are our ‘description message structure in relation to UI’ and our example report with explanation of data fields; and, where the COUNTER validation tool proved to be a game-changer, OA Switchboard has built-in (basic) automated validation and routing of messages.
We drive shared infrastructure and standardisation, via a central information exchange hub as well as standard messaging protocol for stakeholders to safely exchange real-time, situational, authoritative data about OA publications. As a neutral intermediary, the OA Switchboard complements existing systems and drives interoperability, leveraging existing PIDs. It connects parties and systems, streamlining communication and the neutral exchange of OA related publication-level information. We do technology and service development collaboratively and open source and operate a self-sustaining, not-for-profit business model and keep fees as low as possible.
In short, reporting via the OA Switchboard means:
Structured data format (JSON and Excel) and automated validation (technical check) against the standard schema.
Feeds data automatically into existing funder, institution and library systems for further integration, processing and analysis. One API connection. Automated routing (delivered how and where the recipient wants to receive it) – the many-to-many ‘hub’ concept.
Being part of an industry-wide, global, collaborative initiative with all stakeholders at the table. Shared infrastructure. Standardisation. For the whole ecosystem to work better for everyone: learn, adjust and progress.
Join us! We’re live!
Contact: yvonne.campfens@oaswitchboard.org
OA Switchboard - central information exchange hub - the independent intermediary, connecting parties and systems, streamlining communication and the neutral exchange of OA related publication-level information, and ensuring a financial settlement can be done.
OA Switchboard is run from the new Stichting OA Switchboard, founded by OASPA in October 2020 and OASPA is supportive and actively involved through a strategic partnership. https://oaspa.org
