Natural resources, Earth's capacity to sustain our growth, and later on, speed of light (unless we break the FTL, there's only so fast we can expand our economy). Every exponential growth in physical world (as opposed to mathematical theory) has a limit at some point.
Be creative: unbounded exponential growth would mean, for instance, that at some point in the future the global yearly energy consumption would be higher than what E=mc^2 suggests for m=mass of the earth.