If I am not mistaken hasn't there been studies that show intelligence more about brain wrinkles than volume? Hence the memes about smooth brains (implying someone is dumb).
Although 50,000 humans inside a football stadium are not 50,000 times smarter than a single human. Indeed taken as a single entity, its intelligence is probably less than the average person. Collective intelligence likely peaks in the single digit number of coordinators and drops off steeply beyond a few dozen.
We're still here, so bigger brains alone might not be the reason.