1. (Optional) Set Agreement Range Threshold
Agreement range thresholds are only applicable to assessments dispatched via the Collective Review tool and assessments of the exact same student submissions by multiple Assessors (Inter-rater Reliability, Assessor Pool distribution to multiple members, or manual transfer of submission to additional assessors).
If the same file or submission is sent to multiple Assessors for assessment and criterion scoring agreement is required or desired, the agreement range threshold setting will generate new assessments to the same Assessors to complete until agreement is reached.
Agreement is determined by the assessment criterion scores selected by each assessor, for a given file or submission, falling within the selected range threshold.
Agreement assessments will be available for reporting in Results Analytics and in the Agreement Reporting Tool, regardless of whether or not all assessors have reached agreement.
1. Click on the Main Menu icon.
2. Select Curriculum.
3. Select Assessment Instruments.
Locate the Assessment Instrument to which you would like to enable and apply an Agreement Range Threshold.
4. Click on the Assessment Instrument and select Edit.
5. Click on the Settings tab.
6. Select Enable Agreement Assessments, under the 'Collective Review' Agreement heading.