How We Rate
The SpotOn™ Rubric
The reviews and ratings SpotOn uses to evaluate digital content are based upon a rubric developed initially through a modified Delphi survey process, which included national academic content and technology experts, as well as through focus group interviews of central Ohio teachers, principals, and national publishers. This initial rubric covered only core courses, but formed the foundation for the SpotOn Master Rubric, which can be applied to eight product types including courses, units, lessons, games, apps, and adaptive learning resources.
The development and validation of the SpotOn Rubric was facilitated through The Ohio State University by content and methodology experts from the College of Education and Human Ecology. The Rubric was designed to ensure that each resource evaluated received an objective and consistent review, addressing criteria that are specific to each resource type. The SpotOn rubric applies criteria over four dimensions:
- Content Quality
If you are interested in learning more about how we rate, you may view an abbreviated version of the SpotOn Rubric. If you have additional questions, please contact us.
The Review Process
Each resource was independently evaluated by multiple reviewers to ensure objectivity and review consistency. For each Dimension in the rubric, reviewers assigned the resource a value of 1 to 4, on a 4-point scale and provided written commentary to support their scoring.
Upon completion of the initial review, additional steps were completed to ensure consistency and resolve any scoring discrepancies. A subsequent editorial process was used to aggregate scores and synthesize reviewer comments. This process ensured that reviews adhered to the rubric, were accurate and factual, and provided evidence to support the ratings.
Weightings and Ratings
During the course of a review, reviewers assigned scores to each of the criteria outlined in the SpotOn rubric. These criteria scores were then weighted and averaged to create the SpotOn Dimension scores. The SpotOn Dimension scores were then combined through weighted averaging to create an overall SpotOn Score. The weightings for the SpotOn Dimensions and overall SpotOn Score are based on input provided through central Ohio administrator and teacher feedback sessions.
In addition to the written comments for each Dimension, all SpotOn reviews include an overall editorial summary and a list of the product's strengths and weaknesses. Furthermore, the comprehensive review includes product images and information supplied by the content provider, in order to provide educators with the information they need for evaluating a classroom resource.