Reviewing for JOSE

Firstly, thank you so much for agreeing to review for the Journal of Open Source Education (JOSE)—we’re delighted to have your help! Follow this guide to help you complete your review, and feel free to ask editors for more help using any medium (email, Twitter, comments on review issues, etc.)

JOSE review principles

JOSE accepts two types of submissions, and your role as reviewer will be different depending on the type. For software submissions, you will download, build, test and assess the software quality. For learning modules, you will download and read, test code components, and assess content and pedagogy.

For both types of submissions, your goal as reviewer is to help authors improve the quality of the submission to the standard for acceptance into JOSE. Categorical rejections are expected only in cases of out-of-scope or otherwise ineligible submissions. If authors decide they cannot meet your requests for improvements, however, they can withdraw their submission.

JOSE provides a checklist for review. You will check off each item, as you proceed with the review, adding comments and suggestions in the Review issue thread. You may also post specific issues on the main repository of the submitted software or module. Be sure to post links between the JOSE Review issue and the submission repository’s issue tracker, as needed.

JOSE Conflict of Interest Policy

The definition of a conflict of Interest in peer review is a circumstance that makes you “unable to make an impartial scientific judgment or evaluation.” (PNAS Conflict of Interest Policy). JOSE is concerned with avoiding any actual conflicts of interest, and being sufficiently transparent that we avoid the appearance of conflicts of interest as well.

As a reviewer, COIs are your present or previous association with any authors of a submission: recent (past four years) collaborators in funded research or work that is published; and lifetime for the family members, business partners, and thesis student/advisor or mentor. In addition, your recent (past year) association with the same organization of a submitter is a COI, for example, being employed at the same institution.

If you have a conflict of interest with a submission, you should disclose the specific reason to the submissions’ editor. This may lead to you not being able to review the submission, but some conflicts may be recorded and then waived, and if you think you are able to make an impartial assessment of the work, you should request that the conflict be waived. For example, if you and a submitter were two of 2000 authors of a high energy physics paper but did not actually collaborate. Or if you and a submitter worked together 6 years ago, but due to delays in the publishing industry, a paper from that collaboration with both of you as authors was published 2 year ago. Or if you and a submitter are both employed by the same very large organization but in different units without any knowledge of each other.

Declaring actual, perceived, and potential conflicts of interest is required under professional ethics. If in doubt: ask the editors.

JOSE aim and scope

We’re witnessing a swelling of communities that develop and share tools and practices in computationally enabled teaching and learning. The founding editors of JOSE have been participating in these communities, and recognized a need for a publication to house these scholarly products in a citable form.

Several journals already exist that publish academic papers reporting on education research: the systematic collection and analysis of data on teaching methods and student learning, and related qualitative studies. JOSE has a different scope. We’re focused on the rich and growing work applying computational methods to enhance teaching and learning. For example, a whole new genre of open educational resources (OER) has sprouted out of creating and sharing Jupyter notebooks, in a variety of subjects benefitting from a computational treatment. At the same time, members of open-source communities are creating software tools to automate processes (like grading), facilitate access to computing for learning (via cloud systems), or otherwise enhance teaching and learning with computing. These are valuable contributions, independently of their possible use in education research, and their creators merit publication and citation credit for them.

JOSE also aims to disseminate the best practices in creating these scholarly objects, and increase quality via peer review. We want to be a formal, yet open and collegial vehicle for sharing accumulated knowledge in using computing to teach and learn.