1. (Optional) Set Agreement Range Threshold
Agreement range thresholds are only applicable to assessments dispatched via the Collective Review tool and assessments of the exact same student submissions by multiple Assessors (Inter-rater Reliablity, Assessor Pool distibution to multiple members, or manual transfer of submission to additional assessors).
If the same file or submission is sent to multiple Assessors for assessment and criterion scoring agreement is required or desired, the agreement range threshold setting will generate new assessments to the same Assessors to complete until agreement is reached.
Agreement is deteremined by the assessment criterion scores selected by each assessor, for a given file or submission, falling within the selected range threshold.
Step 1: Access the Assessment Instruments Screen
1. Click on the Main Menu icon.
2. Select Curriculum.
3. Select Assessment Instruments.
Step 2: Edit the Assessment Instrument
Locate the Assessment Instrument to which you would like to enable and apply an Agreement Range Threshold.
4. Click on the Assessment Instrument and select Edit.
Step 3: Enable Agreement Assessments
5. Click on the Settings tab.
6. Select Enable Agreement Assessments, under the 'Collective Review' Agreement heading.
Step 4: Select Agreement Range Threshold
7. Select the Agreement Range Threshold. New assessments will be generated when the agreement range threshold exceeds the scoring range selected.
8. Click Save Settings at the top right corner of the screen.