The State Bar of California has disclosed that some multiple-choice questions in a problem-plagued bar exam were developed with the aid of artificial intelligence.

The legal licensing body said in a news release Monday that it will ask the California Supreme Court to adjust test scores for those who took its February bar exam.

“The debacle that was the February 2025 bar exam is worse than we imagined,” Mary Basick, assistant dean of academic skills at the University of California, Irvine, Law School, told the Los Angeles Times. “I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”

In February, the new exam led to complaints after many test-takers were unable to complete their bar exams. The online testing platforms repeatedly crashed before some applicants even started. Others struggled to finish and save essays, experienced screen lags and error messages and could not copy and paste text, the Times reported earlier.

  • LastYearsIrritant@sopuli.xyzEnglish
    161·
    2 days ago

    I’m perfectly fine with non-lawyers, and/or LLMs writing the questions. What was stupid was that it wasn’t independently validated and proofread by a third party.

    “The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam,” she said. “They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”

    It’s a bar exam. One of the most important tests these people will take in their entire career, and they half assed the implementation.