The dramatic drop in the passage rate for the July 2012 Michigan bar exam has been attributed, at least in part, to the decision by the Michigan Board of Law Examiners to stop scaling the raw essay scores of the test, which has precipitated considerable discussion about the meaning and point of scaling in testing. The best explanation I've read about bar exam scaling is this, from The Faculty Lounge:
- Scaling helps ensure that differences in scores from component to component, or from administration to administration, are due to differences in performance, rather than to differences in the relative difficulty of questions, or of the graders. The NCBE uses questions from prior exams to scale the raw scores on the MBE. Forty-six states use the distribution of the final MBE scores in their states to scale raw scores on the state components of their Bar exams.
- Forty-five of the scaling states combine the scaled scores on each of the components, and then compare the combined score to the state's minimum passing score.
The post, which focuses on the Illinois decision to ratchet up the passing score of that state's bar exam between now and 2015, points out that the widespread adoption of both scaling and combined-score grading means that the primary determinant of the relative difficulty of a particular state's bar exam is the state's minimum passing score, or cut score. Michigan's deviation from that practice makes it more difficult to determine the relative difficulty of the Michigan bar exam.