This is actually not surprising since word Frequencies more or less follow a power law probability distribution. With a vocabulary of even 1,000 words you are very well covered. Anyone who engages in a discussion with words that are not very common is doing the very opposite of trying to communicate.
I choose words on the basis of how precisely they convey my desired meaning - which may exist at multiple levels, eg using esoteric words when talking about esoteric words. It seems to me somewhat anti-elitist of you to decry those who enjoy taking such care.
(I scored 35,300; of the words I recognized only about six were ones I don't either read or use with some regularity. I read a fair amount of literature though.)