Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I just tested os.Readdir() on a friend's budget Windows laptop which has a consumer grade SATA SSD running on the slow cygwin over the ancient NTFS format.

os.Readdir() fetched 10.000 file names into a slice in 0.2s.

That's more than fine for most use cases and certainly doesn't look like "one of the stupidest, most broken interfaces I've ever seen".

Specially given that it's not the only way to iterate over files of a directory in Go. There's filepath.Walk() for one.



filepath.Walk() isn't much better:

> The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient.

If it's fine for "most use cases," that's all well and good. Just implement the correct interface, which is fine for ALL use cases, and then implement this sugar on top of that.


They are aware: https://github.com/golang/go/issues/16399

Seems like they postponed incorporating fastwalk into the standard library because it would break API: https://godoc.org/golang.org/x/tools/internal/fastwalk

Perhaps until then one could use libraries that expose fastwalk:

https://github.com/karrick/godirwalk

https://github.com/s12chung/fastwalk


2 ms per 10 files is a huge amount of time.


You're right! I was executing time go run main.go

I now built the exe and ran:

    real    0m0.199s
    user    0m0.000s
    sys     0m0.014s
So 0.2s for 10k files on the worst possible hardware/software scenario I could find nearby. Edited my original comment. Thanks!


You're welcome!

It is still a ton of time, about an order of magnitude more than optimal if the real/sys time split is to be believed.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: