Impartial group assessment. Using peer review and economic theory to grade groups fairly.
In a clear case of practicing what he preaches, Dr. Heinrich Nax has applied game theory to his teaching practice. After lecturing on game theory for several years, he realised that his methods for teaching, more specifically, for assessing did not follow the very theories he was espousing and so he set out to correct this incongruence.
In his course «Controversies in Game Theory» students work in groups and are assessed based on a group project. Social tensions can develop between individual and collective interests in group interactions. One such tension, free-riding, when one person rides the coat-tails of other hard-working group members is well known. There are however additional potential problems when assessing group work such as collusion on grades in cases of peer review. To eliminate these tensions, Dr. Nax decided to implement a mechanism from economic theory to his assessments.
What triggered this approach?
Previously Dr. Nax gave the same final mark to everybody in a particular group regardless of their individual efforts as these could not reliably be assessed. From a game theory perspective this constituted a big temptation for free-riding and Dr. Nax decided to devise something that would incentivize individual efforts but without giving up the benefits of group work altogether.
What exactly did he do?
Influenced by the article Impartial division of a dollar by Clippel, G., Moulin, H., and Tideman, N. (2008), Dr. Nax and his colleague Sven Seuken implemented the article’s mechanism in a blockchain start-up company. The mechanism enables a group to split their financial earnings through peer review between the group members. Group members decide internally what a fair allocation of earnings should be. So he decided to try the mechanism in an educational setting where the “earnings” become the finite amount of points the group works towards, the total of which is determined by the grade he allocates to the group’s total project.
The key idea of the mechanism is that individual group members don’t evaluate their own performance and therefore don’t decide how many points they themselves have “earned”. Instead they allocate relative contributions to the other group members. So in a group of three, if student A thinks group member B did twice as much work as fellow group member C, then B should receive twice as many points as C. Using a specific formula (described in the paper) all three group members reports are then aggregated anonymously to make sure the resulting grades cannot be manipulated. In other words, student A only receives the (aggregated) amount of points, that their colleagues think student A deserves.
What were the results (for student learning)?
Not only was Dr. Nax convinced that the quality of the group projects improved, but the students were happier as well. They believed that the marking was much more fair. It is unclear if this grading method decreased free-riding, however students felt that freeriders did receive lower grades, thus increasing student satisfaction in comparison to grading methods where all members of the group receive the same grade, regardless of effort or contribution.
To see this mechanism in action, visit the Spliddit website which features a demonstration tool. Those interested in learning more should read the original paper or this second (less math-based) follow-up paper or contact Dr. Nax for further information. Dr. Nax is working on a tool to make his grading plan available to other professors.
Posted on
in Featuring teaching
