Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

As someone that has been on the other end, I can tell you devs don’t want to use selenium or inspect requests to reverse engineer your UI and wish there were more clean APIs.

Have you tried making your UI more challenging to scrape and adding a simple API that requires free registration?



So much this!

I work in E-Commerce and (needless to say) we scrape a lot of websites. Due to our growth and the increase in scrapers we require, I’ve been writing a proposal to a higher up to talk to our biggest competitors to all set up a public API that batches the data for a smaller amount of requests.

It would save everyone quite some traffic and effort.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: