Let's think about it logically. If you were to actually test the users instead of asking them 'which of these do you know?', the ONLY option is multiple choice since analysing text-field input from the user to determine if their definition is correct is at best extremely difficult, and more than likely impossible ... right ?
I agree that the quality of the result is highly dependent on the honesty
of the person tested, but with multiple choice, wouldn't you be able to
deduct the meaning of the word from choices you are given?
E.g.: One of the words I encountered in the test was 'terpsichorean'. While I knew that Terpsichore is one of the Muses, I did not know which one and left the box unticked. Had there been multiple choices, I might have guessed the correct solution.
I ticked 'terpsichorean', because it made me think "dreamy travelogue writing of a scenic beach with either terpsichorean sky or sea, it means X looks a pale shade of blue-green".
Google tells me it means dancing so I was way way off (maybe conflating turquoise and cerulean?).
But I have no way of knowing how many words that I feel comfortable defining are actually nowhere near correct, so to be any kind of accurate, they need to do some verification of correctness. All 'honesty' means is 'don't deliberately cheat' not 'don't be dumb'.
I had "discomfit" as one of my words that I wasn't clear of the definition of, I was pretty close when I looked it up but couldn’t have guaranteed it. It's probably easily confused with discomfort ... which made me think that this needs to be a little more tested. Commonly misread words could easily inflate scores.
However, I think a multiple choice test could also inflate scores unless the definitions were very cunningly constructed.
I scored 75-80th percentile (32,800) which surprised me. It seems quite a lot of words, for one. For another I consider my vocab' to be very good and I don't think I'm being bigheaded in that. Ergo I expected to be ranked higher.
On the second page there was an entire column of words of which I recognised only three sufficiently to provide a guaranteed accurate definition. One of that column was terpischorean, another tatterdemalion.
Whilst looking up tatterdemalion I found little use of it after the 1930s except as a proper noun (a Marvel Comics character for example). What I did find however is that Google Books is useless for finding dates. One citation from an author Sir Edward Bulwer Lytton is given a date of 1999. That's a reprint date, the author died in the 19th century.
Actually, the SAT also employs multiple choice, and quite successfully. You see, multiple choice can not only give you hints, but can also be used to lead you astray. In the end, it balances out.
True, but it's still much better than just relying on people submitting accurate results without even testing them ... at least with multiple choice everyone is being tested to approximately the same metric, and you can get a more accurate percentile
To my defense, they are both derived from 'deducere', to lead away, and 'were not distinguished in sense until the mid 17th cent' according to my system’s dictionary
In all the exams I had in my life, from 1st grade to BS in CS, I only had one multiple choice test.