Students from the United States to Norway are being assigned grades by computer models they don't understand
Coronavirus is changing the world in unprecedented ways. Subscribe here for a briefing twice a week on how this global crisis is affecting cities, technology, approaches to climate change, and the lives of vulnerable people.
By Avi Asher-Schapiro
NEW YORK, Aug 12 (Thomson Reuters Foundation) - As millions of students in Britain await exam results this month, a row has erupted over algorithms used to calculate their grades after exams were cancelled due to the new coronavirus.
In a last-minute change following an outcry in Scotland, education minister Gavin Williamson said on Wednesday that students set to receive their A level marks - used to enter university - on Thursday could use the results of earlier tests.
The credibility of new grading algorithms has come into doubt not just in Britain's national exams, but also for the International Baccalaureate (IB) programme used in more than 150 countries.
HOW DOES A GRADING ALGORITHM WORK?
In Britain, teachers were asked to submit to exam boards the grades they thought each student would have achieved, and to rank their pupils in order.
These data points for each student were then put through a statistical model, including the historic performance of their school, to determine whether the teachers' grades were more severe or generous than expected and matched previous years.
WHAT'S WRONG WITH THAT?
Relying on algorithms to determine results has the potential to reproduce - or even exacerbate - existing patterns of inequality for low-income and minority students, according to some education researchers and statisticians.
Critics say poorer students risk being unfairly penalised because they are judged on the track record of their schools rather than individual performance.
WHY THE UPROAR IN SCOTLAND?
Scotland on Tuesday promised to restore 75,000 grades that were downgraded by its moderation system to the original levels set by teachers.
The algorithm had reduced the pass rate for Highers - which 18-year-olds use to gain a place at university - by 15% for pupils from the most deprived backgrounds, while those for the wealthiest pupils only fell by 7%.
"We believe we have delivered fairness to learners, through a consistent, evidence-based approach in the absence of exams," a spokesman for the Scottish Qualifications Authority, which oversees the process, said in emailed comments.
CAN ALGORITHMS LEAD TO BIAS?
There was also an outcry in July when results for the IB - sat by 160,000 students globally each year - were released.
Most IB students live in the United States, where critics said the algorithm could lead to bias against minorities and low income students.
Almost 60% of public schools offering IBs are "Title I" schools which have significant low-income populations, according to the programme.
The IB model took into account historic data from each school, raising questions about whether students in poor communities, with less resources, might be unfairly downgraded.
"Racial, gender, socioeconomic or any other sort of bias has no place at the IB," a spokesman said in a statement.
"We continue to work closely with the regulators and experts that helped shape and approve the models that assessed student results."
CAN STUDENTS CHALLENGE THEIR RESULTS?
In Norway, IB students complained to the local data protection authority (DPA), which enforces laws requiring Norwegians' personal data - including grades - be processed fairly, accurately and transparently.
One student, Sigurd Salvesen, told the Thomson Reuters Foundation that he was assigned an IB grade of 30 out of 45, which fell short of the 34 he needed to take up an offer from the University of Edinburgh in Scotland.
"I don't feel like I was graded based on my own work or achievements," he said, adding that he feared the algorithm assigned him a low grade because of his school's past poor performance.
In a draft decision, the DPA said on Friday it planned to order the IB Organisation to issue fresh grades without reference to "school context" or "historical data" as these had no connection to individual academic achievements.
"The name of the school or its geographical location is not a relevant element in awarding grades," it said.
"This may have discriminatory effects as the model can potentially be of prejudice to certain socio-economic groups ... students from not-so-well-performing areas would be penalised just for going to school in that area, which is clearly unfair."
The DPA has given the IB Organisation until Aug. 21 to submit its response before issuing a final decision.
WHAT HAPPENS NEXT?
IB students in Britain are also hoping for new grades after the exam watchdog, Ofqual, asked the IB Organisation for more information about its grading process.
Meanwhile, A level results for England, Wales and Northern Ireland will be released on Thursday, followed by GCSE results for 16-year-olds on Aug. 20.
Britain's Royal Statistical Society, a professional body, has called for a "full and open appraisal" of the methods used, suggesting it would be fairer to include individual students' historic performance rather than that of their school.
"If the underlying process is flawed then, possibly the whole thing is flawed," said Guy Nason, chairman of the Royal Statistical Society's research section, adding that it was hard to judge the method as it still has not been made public.
Related stories:
Asia school closures for coronavirus expose digital divide
Schools challenged to teach climate change as students join Greta strikes
OPINION: Don't let girls' education be another casualty of the coronavirus
(Reporting by Avi Asher-Schapiro @AASchapiro, Editing by Katy Migiro. Please credit the Thomson Reuters Foundation, the charitable arm of Thomson Reuters, that covers the lives of people around the world who struggle to live freely or fairly. Visit http://news.trust.org)
Our Standards: The Thomson Reuters Trust Principles.
