Prof. Joordens, professor of psychology at University of Toronto, developed a piece of software called peerScholar, an online service that makes it practical to use peer assessment as an anonymized, averaged grading tool
It's easy to ask students to complete assignments. Getting them to think critically about the work they've done is a different matter entirely.
This is a challenge in small classrooms as well as very big ones. Steve Joordens, a professor of psychology and the director of the Advanced Learning Technologies Lab at the University of Toronto's Scarborough campus faces first-year classes of no less than 1,800 students, and was looking for a way to make the process of assessing papers less of a rote, one-way activity, in which the work was done, a grade is assigned – and, with any luck, glanced at by the student – and that's the end of it.
One way of doing this is by bringing students into the grading process. Peer evaluation is a time-honoured tactic, but it has its logistical drawbacks. Prof. Joordens' solution was a piece of software called peerScholar, an online service that makes it practical to use peer assessment as an anonymized, averaged grading tool.
The software lets students submit their assignments digitally. Then, their work is anonymously distributed to a number of peers – say, five or six. Each of those peers gives a grade, but also feedback, which is a skill unto itself that universities don't tend to teach.
"People are very good at criticizing, but they're not very good at doing it in a way that suggests a positive approach," says Joordens.
With peerScholar, giving feedback becomes part of the course, and students give each other grades on the quality of feedback they receive. Joordens says that his lab's research indicates that peer grading is a reliable substitute for expert opinion. Students who don't like the mark their peers assign can ask to have their work re-graded by a teacher, though Joordens cautions them that mark will be final. Some students see their marks go up, but others see them drop. In fact, Joordens says that in the end, the average mark change for re-graded papers is nothing at all.
"The average of a peer grade is very comparable to the grade a single graduate-level TA would give," he says.
The software's origins date back through the last decade at U of T, but only now are its creators and owners finalizing a licensing model that would allow them to effectively sell to other institutions. Joordens has licensed the system through Pearson Canada – a large textbook distributor – and, after some years spent working through the challenges of being a small part of a big company, the software is entering the market. It's already in use at schools like Queen's and University of British Columbia in Canada, and at several American colleges. They are also marketing a version for primary and secondary schools under the brand name "Cogneeto."
To Joordens, the software is part of a broader project to teach cognitive skills that go beyond acquiring and retaining knowledge. "The analogy I like to give is, you can go to a one-hour seminar about karate. You can learn a lot about it. But if you want to learn to do karate, you cannot learn to do karate in an hour. You need a framework where you can practice."
Not everyone saw it that way. At peerScholar's own home at U of T, the union representing the university's teaching assistants viewed peer assessment as a form of "cheap labour," and launched complaints against Joordens. When an arbitrator sided with the union, Joordens modified the software so that students would provide feedback prior to getting an actual grade from teaching assistants. So far, the episode has yet to repeat itself at other schools.
While the software was designed for traditional written assignments, the evolving educational landscape is seeing innovative uses for it. A forthcoming class will ask students to submit audio instead of written assignments – they'll conduct oral presentations by speaking into their cellphones, and the audio will be distributed to peers for evaluation just as text would be. And, in another project, Joordens recently used it to grade a massively open online course with 65,000 students.
"The bolts stayed on," he says.
Follow us @GlobeSmallBiz, on Pinterest and Instagram
Join our Small Business LinkedIn group
Add us to your circles
Sign up for our weekly newsletter