On top of the other issues discussed, I think that giving effective oral exams is hard for the same reasons that interviewing is hard. It’s a test of the subject’s ability to quickly and confidently say things that sound roughly correct. Some people stumble over their words and cannot effectively speak and think completely accurately in the time required to give a reasonable answer without an uncomfortable pause.

Now, these issues could be mitigated by asking each person the exact same questions and taking careful notes of their responses, but then you’re just back to a bad essay that can’t be revised, edited, planned, or recollected as easily as a real essay.