Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Insta definitely uses IP blocks to force login walls, but I don’t think it’s anti-scraping, because the limit seems to be like 3 images before you get redirected to log in - it’s way too low and too much of a pain in the ass to normal users to be intentionally anti-scraping and unintentionally anti-logged-out-users.


The fact that it pushes anonymous viewers to log in or sign up is definitely a positive (for their conversion rate metric, if nothing else), and that might be the main reason the number of images is so low. But the specific mechanism of IP blocking that is used to achieve restriction - that is anti-scraping, pure and simple.


... except, that "pro" scrapers use parallelized calls from rolling ip addresses. It is one of the only ways to scrape things like Youtube... so I hear!


Nobody is going to be able to stop motivated, well-resourced, professional scrapers. IP blocks are effective for stopping most trivial scraping.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: