Gabe Teninbaum, a professor at Suffolk College Regulation Faculty, is looking on ExamSoft to repair a critical bug with its test-taking software program: failure to acknowledge faces. It’s an issue that may delay take a look at takers — or bar them from beginning their exams altogether — and per reviews, it disproportionately impacts folks with darkish pores and skin tones.
ExamSoft’s software program information college students whereas they full distant exams and displays for indicators of educational dishonesty. Teninbaum’s report addresses an ExamSoft characteristic referred to as ExamID, which goals to confirm that test-takers are who they are saying they’re. The primary time a pupil logs into their examination portal, they add a photograph of themselves (their “baseline image”); they’re then prompted to take one other selfie earlier than starting future exams, which the software program checks towards their unique picture.
Research has found that facial-recognition algorithms constantly make extra errors in figuring out Black faces than they do white ones. And whereas these research didn’t deal with ExamSoft particularly, it doesn’t seem like an exception. Again in September, a number of non-white exam-takers instructed the New York Times that the software program couldn’t determine them as a result of “poor lighting” — an issue that Teninbaum, who has gentle pores and skin, wasn’t capable of replicate.
The @ExamSoft software program cannot “acknowledge” me as a result of “poor lighting” regardless that I am sitting in a properly lit room. Beginning to suppose it has nothing to do with lighting. Fairly positive all of us predicted their facial recognition software program would not work for folks of shade. @DiplomaPriv4All
— Alivardi Khan (@uhreeb) September 8, 2020
@ExamSoft I FINALLY found out ON MY OWN four hours later ( nonetheless on maintain for an EXAM SOFT chat rep) HOW TO GET TO MY MOCK EXAM… ONLY TO BE TOLD THE SYSTEM CANT RECOGNIZE ME. I AM IN A BOARD ROOM WITH ALL OF THE LIGHTS TURNED ON AND THE SUN WAS STILL UP. MAKE IT MAKE SENSE. pic.twitter.com/7K3yKT1y4n
— Futureesq1990 (@futureesq1990) September 17, 2020
Early this fall, Teninbaum got down to discover a repair. He believes such errors add undue stress to an already hectic time interval. “These are students who are about to take a high-stakes exam with a lot on the line, and that is very unwelcome,” Teninbaum mentioned in an interview with The Verge.
“Any time you go into an exam you just want to focus on the exam,” he added. “You don’t want to feel like you have these added challenges.”
Teninbaum additionally believes that optics matter; colleges owe it to marginalized college students to not depend on a class of software program that’s recognized to be discriminatory. “Students deserve to feel that their institution is doing what it can to protect their rights, interests, and dignity,” he says.
In his report, which is forthcoming in The Journal of Robotics, Synthetic Intelligence, and Regulation, Teninbaum outlines the workaround he discovered.
He means that colleges assign each pupil an equivalent generic, baseline picture. Then, he proposes, they need to ask ExamSoft to allow “deferred identification,” a characteristic constructed into the software program that permits college students to proceed with exams even when identification fails. This characteristic is principally hidden — it’s not talked about anyplace on ExamSoft’s web site (a minimum of, not that I may discover). Solely ExamSoft can flip it on.
In tandem, these tweaks will trigger ExamSoft to misidentify each test-taker. However they’ll nonetheless have the ability to proceed with their exams — ExamSoft will ship the selfies to the varsity afterwards, and instructors can manually confirm everybody. “We know who our students are,” Teninbaum says. “We can make sure the students are who they say they are and avoid subjecting students to these sorts of challenges.”
He additionally means that ExamSoft make the “deferred identification” characteristic accessible to clients. “The reporter urges ExamSoft to build this into a feature by which institutions can simply toggle on/off, thereby bypassing ExamID until such time that the technology matures into one that does not discriminate,” his report reads.
Teninbaum hopes these modifications will final past the COVID-19 pandemic, and can assist college students really feel extra snug taking distant lessons. “It’s going to be a growing problem as people get more and more online for their education,” he says.
Even so, he’s solely mounted a part of the issue. College students have skilled a range of hiccups with ExamSoft’s proctoring software program. Over 3,000 individuals who used the platform to take California’s bar examination in October had their movies flagged for potential rule violations — practically 36 p.c of candidates who took the net examination. Customers reported audio points, and different technical glitches as properly.
A bunch of six US senators — together with Richard Blumenthal, Elizabeth Warren, and Corey Booker — wrote an open letter to ExamSoft in December, highlighting potential harms to college students of shade and college students with disabilities, amongst quite a few different issues.
ExamSoft didn’t instantly reply to a request for remark.