Overlay Journal Infrastructure for Meteorological Sciences (OJIMS) Project
Principal Investigator: Dr Alan Gadian
Sponsor: Joint Information Systems Committee
Start Date: 1 March 2007
End Date: 28 February 2009
Value to Leeds: £81,000
Institute: Institute for Climate and Atmospheric Science
Abstract
This proposal presents a partnership between the Royal Meteorological Society (RMetS) and the National Centre for Atmospheric Science (NCAS, led at the University of Leeds, and including the British Atmospheric Data Centre, BADC) which will exploit the data repository at the BADC along with the expertise of the RMetS to develop the mechanisms which could support both a new Journal of Meteorological Data and an Open-Access Repository. This new concept would include four components: (1) a new open access discipline-specific document repository based at the BADC, (2) the existing BADC data repository, (3) a new overlay journal in which "articles" link peer-reviewed documents to peer-reviewed datasets, and (4) an overlay journal framework that would provide links to highly regarded "kite-marked" papers via the repository (either to the repository contents or the version of record held by an original journal publisher.)
The work would build on the previously JISC funded CLADDIER project and take the next steps towards making these two classes of overlay journal (a "data" journal and a "really useful papers" journal) possible. The eventual goal for both journals would be for the RMetS to pick them up into their stable of activities, but it is important to note that neither journal can become a reality until the work is completed, and it is possible to identify a sustainable business model that would guarantee the longevity of these journals on a not-for-profit basis. Such evaluation would consist of two phases: an analysis phase, and an approval phase. While the approval phase is out of the scope of this project, the RMetS will undertake work within the scope of this project to identify what possible models exist and to recommend a method for practical implementation. The analysis would of course also be available as a product of the project available to other learned societies or groups considering such activities. The mechanisms built within the project would be open source and available to all.
These two new overlay journal activities would address two key issues. Firstly, the peer review process that provides both scrutiny and validation of academic endeavour is generally only applied to the final conclusions and interpretations of a body of work. While in some fields and nations, underlying datasets are available, generally these are not always tightly coupled to the publications that result, nor have they themselves been reviewed. Where these results are of significant importance, either within the academic field, or because work has legal or policy implications, this becomes a problem. Clearly the existence of a peer review process undertaken by and independent learned society such as the RMetS coupled with persistence of the data in repositories will address this issue.
The second key issue is that while it will be easy to create a discipline repository, it will be difficult to get it populated. While NCAS as a distributed body of (primarily) university staff needs such an entity, and is in a position to expect (even mandate) NCAS funded staff to use it, the real success of such a repository would be if it was populated by a much wider community from the UK, Europe and even further afield. Clearly there might be initial difficulties associated with IPR, but we foresee that both individual authors and journal publishers would see the benefit of submission if such submission was a necessary (but probably not sufficient) entry criteria for assessment by the RMetS and inclusion in the proposed overlay journal of "kite-marked" papers (recall that the links from the overlay journal would be to the primary versions held by the publishers). Exactly how the kite-mark process would occur is to be studied in this project, but the resulting journal should provide a new way of focusing community attention on papers of special merit, regardless of the original journal location. It would also act both as a mechanism to encourage repository population, and as a mechanism for to encourage publishers to accept the merits of open access.
One of the exemplar of the issues comes from the climate sciences: Mann et al (Geophysical Research Letters, vol. 26 no. 6, pp759-762, 1999) published a reconstruction of the last thousand years' climate, which amongst other things demonstrated that the global temperature increase over the last few decades was unique during that period. The resulting graphic became known as the "hockey-stick" and appeared on a significant amount of material from the Intergovernmental Panel on Climate Change. While only a very small component of the material used by the IPCC to draw their conclusions, the work suffered sustained attacks by the sceptic community because the datasets and algorithms themselves had not been published and subject to peer review. The resulting damage to the integrity of the processes (peer review and the IPCC) has taken years to resolve, and arguably is still not resolved, since even though the majority of the work has been replicated, there are questions about the data themselves which are difficult to resolve since the original material is no longer available.
The introduction to the CLADDIER proposal rehearsed other reasons for establishing peer review data journals such as the desirability of making dataset production assessable as one of the metrics of a future Research Assessment Exercise.