評鑑的分析

這個分析是針對範本的評鑑，以及同儕評鑑階段對於作業的評鑑。它企圖從從教師和學生的所有評鑑中，選出比較好的評鑑 。這些"好的"評鑑將被用來計算最後分數。

這一分析工作若包含有教師評鑑，就會做得更好。因為教師的評鑑可以當做基準來判斷學生評鑑的好壞。教師不需要 評鑑每一個範例和每一提交的作業，但是為了讓這分析有意義，教師所做的評鑑次數最好要比平均每個學生做的評鑑次數還要多。 教師做越多的評鑑，教師對於這分析的結果就能越有信心。

這一分析通常會做好幾次，每次都改變一個或多個選項。 這一分析是由出現在這頁上端的三個選項來控制。

1. The Loading for Teacher Assessments sets the weighting to given the teacher's assessments compared to the students' assessments in the error analysis stage. 如果教師要使他們自己的評鑑結果在學生同儕評鑑中起主導的作用， 那教師要使自己在"誤差表"上，成為平均誤差最小的評鑑者。如果教師不是清單上的第一個，那教師評鑑的 加權量要增加，直到教師的平均誤差達到最低。這樣就表示教師的評鑑起了主導作用，而學生打分數愈像教師的， 就會越接近誤差表的頂端。而排在表的底端的學生，表示他們打的分數與教師打的分數(還有那些排在頂端的同學)差異最大，. The more assessments that are available from the teacher the more likely that this option will not have to be used to force the teacher to the top of the table. Note that this option does not apply a weighting factor the teacher's assessments when they used in the calculation of final grades. In that calculation the teacher assessments have the same weight as the student assessments. So for example if a student's submission is graded at 41% by the teacher and 45% and 55% by their peers the final grade given to the submission is (41% + 45% + 55%) / 3, that is 47%.
2. The Weight for Grading of Assessments is used in the calculation of the Final Grade. A simple formula is used to calculate a student's "Grading Performance". It is the proportion of "good" assessments that student have done compared to the maximum number of assessments open to them. So, if for example, the assignment asks the students to do 3 assessments of the example submission and 5 peer assessments and the student does 7 assessments and 1 of those is dropped from the analysis (see below), then their grading performance is (7 - 1)/8, that is 75%. The final grade for the assignment a weighted combination of this grading performance and the grade given to their submission (or best grade if they made more than one submission). The grade for the submission is always given a weight of 1. So setting this option to say, 0.5, means that the two grades are added together in the proportion 0.5:1 or 33% of the grading performance and 66% of the grade of the submission.
3. The Percentage of Assessments to drop determines the number of the assessments which are to be excluded when calculating the final grades. This number can be set in one of two ways.
   • Given the way the Grading Performance is calculated each student could, if they assessed all the work allocated to them, achieve full marks (for this element) if no assessments are dropped. If the teacher wishes to have a more reason average grade then setting this option to 30% would result in the average Grading Performance of about 70% (again if all students graded all the assessments open to them).
   • Alternatively the number of assessments to drop might be set such that the remaining "good" assessments result in the Average Errors being constrained to some reasonable value. These are the percentages given in the fourth column of the Error Table. For example, it may be thought that all the student assessments should (on average) lie within the 20% range. Then the analysis is repeated a number of times adjusting the number of assessments to drop until the figures in this column all lie within a particular limit.

In addition to the Error Table the analysis lists the grades of all assessments and the final grades given to the students. This table should be inspected to see if the results are reasonable. In particular if many assessments are dropped then some submissions may left unassessed and the student's final grade will be far too small. The analysis does given the number of submissions at the top of page and again just before the Grades Table. These two numbers should be same. If there are one or more unassessed submissions and the teacher does not want to decrease the number of dropped assessments then those submissions should be assessed by the teacher and the analysis repeated. It is important that all submissions are assessed at least once in the final stage of the analysis that is when the final grades are calculated.

There is a balance between the number of assessments dropped and the overall final grade. The more assessments dropped the lower the final grades are likely to be. However, if poor assessments are not dropped then students may complain about the quality of the assessments which determine the grade for their work. Provided there are enough assessments by the teacher to dominate the analysis without too much forcing, then it would seem reasonable to drop somewhere between 15% and 30% of the assessments.

注意，這個分析要花很長的時間，因為它包含了迭代反覆的歷程。要等很久是可預料的。