Pointing out the frailties of AI and automated decision-making (ADM) in education is not to imply that human decision-making is somehow substantially better. Teachers and educational administrators – as with all humans throughout history – have always made biased, illogical and bad decisions. Education is full of occasions when those in positions of authority might benefit from a little extra advice or a nudge in the right direction.
Yet being ‘no worse than a human’ is no reason to justify the adoption of flawed automated decision-making technology in classrooms or any other educational setting. As Frank Pasquale (2021) puts it, “We don’t have to choose between biased AI and biased humans. We can regulate to improve the AI that is complementing humans.…not to stop AI entirely”.
Indeed, the main problem with the steady increase of AI-decision making into classrooms is our general willingness to presume that a computer is unbiased, objective and rational in its decision-making. Whereas students and parents are often quite keen to challenge their teachers’ judgement (especially when it comes to matters of grading), there is still a notable reticence to push-back against computerised outputs.
Educational ADM therefore benefits from people’s computer-related credulity – something that Sherry Turkle once described as humans ‘hav[ing] learned to take things at interface value’. This mentality of having to follow ‘what the computer says’ is compounded by the lack of transparency surrounding most ADM systems. If students or teachers try to interrogate why decisions have been reached, algorithmic systems and logics usually prove to be incredibly difficult to discern.
On the flipside, there are some advantages to human-led decision making. As Hannah Davis (2021) points out, human decision-making might not be perfect but at least takes place on slower and smaller-scale that algorithmic systems, and includes some scope for personal accountability. In short, there is only so much that a human teacher can decide to do, and generally they will be held responsible for these decisions. As such, the main way to improve decision-making in education is make more efforts to support the decision-making and integrity of educational professionals. Clearly, teachers are not perfect, yet attempting to rectify this through the implementation of imperfect technological equivalents will only exacerbate the problem.