Assessing Agreement Assessments
If a file has been sent to multiple assessors for assessment and the completed assessments are in disagreement (i.e the selected criterion scores exceed the designated scoring variance threshold), a new assessment pending assessment will automatically be sent to the same assessors. This process will continue until the assessors reach an "agreement" in their scoring (i.e. the selected criterion scores fall within the designated scoring variance threshold).
If you have completed an assessment on an assessment instrument where a scoring variance threshold has been designated, you may receive a new pending assessment to complete if your scoring selections are not in agreement with other assessors who have also scored the same work sample.
These assessments are identified with a "Disagreement" flag. You will also see a number next to the flag, based on how many assessments you have completed on this work sample in order to reach agreement.
1. Click on the assessment and select the Assess option.
2. The criterion whose score exceed the scoring variance threshold designated by the administrator will be identifed with a "Disagree" flag.
3. If appropriate, select a new score for the criteria with a "Disagree" flag. Click HERE to review the steps to complete an assessment, if you are not familiar.
4. Click Save to complete the assessment.
Once you and the other assessors assessing the same work sample reach a scoring agreement, you will stop receiving "Disagreement" pending assessments.