Great to see a medical board seriously thinking about and discussing open-book exams. This post from the American Board of Pediatrics does a nice job laying out the pros and cons of such a system. Two important issues they raise:

Previous exams have “ample psychometric data in support of [their] fairness, validity, and reliability”—Although any new testing format would need to undergo a tremendous amount of testing, this should in no way hinder improving an exam. The above statement means that the ABP board exam has good internal validity [1] but has no bearing on its external validity [2]. Board exams, at all levels, need to reflect the real-world practice of medicine. Kevin Pho MD wrote eloquently about this over 3 years ago on his blog.

Cheating—This is a solvable problem by controlling the testing environment. Test-takers could be limited to a select number of commonly used resources, not the full internet [3]. Or, all internet traffic for the examinees could be monitored. The potential for cheating should not be a significant barrier to changing exam formats. Even with the current exam mechanisms, cheating happens.

I previously wrote about why exactly I think we need open-book testing in medical eduction. The practice of medicine is about finding the right answer for your patient, not how you find that answer.

That is, how well the exam tests what it is supposed to test and performs year-to-year between similarly skilled test-takers. ↩
How well the exam tests someone’s skill at actually practicing medicine. ↩
Admittedly, such a system would have its own issues, most notably implicit endorsement of certain clinical resources. ↩