1.2 Create Assessment Instrument
This lesson will walk you through the various types of Assessment Instruments available within your Chalk & Wire system, and will provide you with links to learn how to create those Instruments. Some of the Instrument types are listed as sub-types as they are simply another way to use the main type of Instrument.
Assessment Instruments are linked to Tables of Contents in order to allow for the submission and assessment of work. They are also linked to Outcome Set sections in order to show how the Outcomes are being met through the course curriculum and to track student progress and learning.
Conventional Assessment Instrument
A conventional Assessment Instrument has an internal score assigned to each level of the Instrument. The Instrument score in a conventional instrument is computed as a mean of all individual criterion scores, taking into consideration the weighting applied to individual criteria. The 'weight' applied to a criterion determines the number of times the criterion score is included in the calculation of the mean. If all the criteria have the same weight, regardless of what the weight is, it is the same as weighting them all at '1'.
A conventional Assessment Instrument has multiple performance levels based on the default performance levels that you have set up in your system. There should be a descriptive label of the expectations for each performance level and each level of the rubric is assigned an internal score.
There are two types of conventional Instruments: analytical and holistic. Your system should include a balanced mix of these types, in order to adequately measure all aspects of the complex skills being evaluated.
These Instruments have multiple criteria, and usually specify the attributes of each level of performance for each criterion. These are more difficult to write than holistic Instruments, but are invaluable to student progress as they already contain key feedback for improvement. In this regard, care should be taken to write them so that students will understand the level descriptors. At first, this may seem to make assessment slower, but not if the performance level descriptions are robust, reducing the need for the assessor to write repetitive general comments. They are also very powerful tools for program improvement, as they provide granular statistics about every dimension of a performance. You are encouraged to use the embedded tools for informally tracking the reliability of Instruments once they are in use, as well for conducting formal inter-rater reliability studies.
Holistic Instruments have broad multifaceted descriptions and a single criterion. They are most effectively used for summative assessments in which the assessor is considering overall performance on an end product or complex process task. Take care to include detailed descriptions of expectations in the performance levels to inform student development and increase the objectivity of the scoring tool.
Additive Assessment Instrument
An additive Assessment Instrument resembles a conventional rubric in format. The difference between conventional and additive instruments is the way the score is computed. The additive instrument generates a score that is the sum of criteria scores. Individual criteria are assigned a maximum score. Generally, additive instruments do not use a not-applicable (N/A) level, however an N/A may be added if the instrument includes developmental tiers.
Checklist Assessment Instrument
While checklists have little statistical value, they are useful to assure that requirements have been met. Checklists can be either Assessments Instruments or forms. If you would like to create a non-scored checklist, use a non-scored checklist for keeping track of submissions of required documents, numeric record logs, etc. Use the Form type 'Portfolio/ Assessment Form' and select the typed in numerical value response type. Results from the form submissions can be viewed by respondent on the 'View Form' screen.
There are two types of checklists: Met/Not Met Checklists and Additive Checklists.
This is a special type of additive instrument, one that has no performance levels. Each criterion is assigned a score and the instrument total is the sum of the criteria scores.
Formula Assessment Instrument
A formula instrument aggregates individual criterion scores from one or more conventional instruments. Formula rubrics are useful in aggregating data related to individual concepts across instruments, programs, and other groupings. For example, cross-campus or inter-program writing or communication skills may be of interest. A formula instrument can aggregate data from individual criteria related to communication from a variety of instruments without requiring additional assessment.
While these have little statistical value, they are useful to assure that key behaviors have taken place. Use only one criterion and two levels '0' and '1'. Report these assessments out separately from other assessments, and do not link these checklists to standards. This type of 'gate-keeping' may also require a second tier of assessment. In that case use the escalation process to advance the submission to another level of assessment (i.e. a different instrument).
A Self-Evaluation Assessment Instrument is created using the 'Conventional' scoring type and is completed by students rather than by assessors. Once a student completes the self-evaluation, the completed assessment is automatically transferred to the designated assessor for review.