I had a similar (though unrelated) problem early on with Golang writing a port scanner, which forced me to hoist my own event loop (build around an explicit select call) into my program: timeout or not, I needed lots and lots of OS file descriptors, and by relying on Golang's own event loop to close stale ones on timeouts, I'd invariably run out.
I ask because I wrote a port scanner professionally once, and I got away with using one fd for network I/O. I can only think of one case where you end up using a lot of fds and that is if you're using TCP sockets and connect(2) where you'd end up with one fd/tested port, and that's not good.
Similar unrelated problem here, too, on the sole machine now running the Heartbleed test.
After recompiling with Go 1.4 (from Go 1.2 - apparently the network poller was rewritten?) I'd start running out of file descriptors really fast, all leaking deep in net/http.ReadRequest.
Turns out http.DefaultServer doesn't have any default ReadTimeout, so when a client goes away before finishing a request I'd lose the goroutine and the file descriptor with it.
So yeah, you probably also want to explicitly set http.Server.ReadTimeout if you are not doing weird streaming/"comet" stuff.
That's related to my problem, but not the same; what I needed was control over when an fd/goroutine would be destroyed. Explicit timeouts (at least in 1.0, but I think still) couldn't give me that. I needed an obscene number of fds, though.
