Holding it back compared to what? I come from a CLR background and look enviously at the jvm ecosystem all the time. Haskell, BEAM, etc are almost all a step down from jvm once you factor in tooling.
I work with both, so I do agree with you with my C# hat on.
On the other hand with my Java hat on, I look enviously to proper unsigned types, value types, explicit SIMD, and the language support for low level coding optimizations.
Still looking forward for Valhalla and Panama to arrive.