評鑑的(一致性)比較

在工作坊中，一件學生作業被老師和多位同學評鑑是很平常的事。 若有使用範例，那教師在學生開始評鑑之前，會先將它們做一次評鑑。而來自學生們的作業 也會被教師(若非全部，至少其中部份)，以及某些學生評鑑。
在工作坊中，不但重視學生的綜合能力(作業表現)，也重視其評鑑能力(評鑑他人作業)，所以學 生評鑑教師提供的範例，以及評鑑同學作品時的表現也都納入最後分數中，最後分數的其餘部 分則取決於作業本身的品質。
綜合能力(作業品質)和評鑑能力(評鑑範例與同儕互評)兩種能力的比例，在工作坊結束前要設定好。 學生在評鑑能力表現上的分數是依據他們對作業的評分與教師的評分的一致程度來計算。 (若該作業剛好教師沒評到，就使用同儕評鑑的平均值為基準)。

學生的評鑑與教師的評鑑之間的一致性，是依據每一規準上分數的差異來計算(實際上是使用差異的平方， 以避免累加時正負相互抵銷)。這些差異的平均數必須被轉換成有意義的分數，差異的平均數愈大，代表此學生的評鑑能力愈差。 這 "評鑑的(一致性)比較"選項允許教師控制如何將比較結果轉換成評鑑能力表現的分數。

To get some idea on what effect this option has, take the (fairly simple) case of an assessment which has ten Yes/No questions. For example the assessment might use questions like "Is the chart correctly formatted?", "Is the calculated profit $100.66?", etc. Assume there are ten such questions. When the "Very Lax" setting is chosen, prefect agreement between the student's and teacher's assessment gives a grade of 100%, if there is only one question which does not match the grade is 90%, two disagreements give a grade of 80%, three disagreements 70%, etc.. That might seem very reasonable and you might be thinking why is this option called a "Very Lax" comparison. Well, consider the case of a student doing a completely random assessment where the answers of the ten questions are simply guessed. On average this would result in five of the ten questions being matched. So the "monkey's" assessment would get a grade of around 50%. The situation gets a little more sensible with the "Lax" option, then the random assessment gets around 20%. When the "Fair" option is chosen, random guessing will result in a zero grade most of the time. At this level, a grade of 50% is given when the two assessments agree on eight questions of the ten. If three questions are in disagreement then the grade given is 25%. When the option is set to "Strict" having two questions out of sync gives a grade of 40%. Moving into the "Very Strict" territory a disagreement in just two questions drops the grade to 35% and having a single question in disagreement gives a grade of 65%.

This example is sightly artificial as most assessments usually have elements which have a range of values rather than just Yes or No. In those cases the comparison is likely to result in somewhat higher grades then the values indicated above. The various levels (Very Lax, Lax, Fair...) are given so that the teacher can fine tune the comparisons. If they feel that the grades being given for assessments are too low then this option should be moved towards the "Lax" or even "Very Lax" choices. And alternatively, if the grades for the student's assessments are, in general, felt to be too high this option should be moved to either the "Strict" or "Very Strict" choices. It is really a matter of trial and error with the best starting point being the "Fair" option.

During the course of the workshop the teacher may feel that the grades given to the student assessments are either too high or too low. These grades are shown on the exercise's Administration Page.若是這樣，教師可以改變這一選項的設定 ，並重新計算學生在評鑑表現上的分數。重新計算的工作可以透過點選工作坊的管理 頁面的"重新計算學生評鑑表現" 來進行。它可以在工作坊結束前任何時候進行都沒關係。