Be a good Maven citizen — Don't scrape the whole damn central repository!


August 28, 2008 By Jason van Zyl

For the last few nights we have had some particularly bad Maven citizens averaging almost 3000 connections in a 10 minute period. This particular abuser emanates from the Netherlands and this is the first time I have completely blocked an IP permanently from the central repository (I’m sure they will just use another one but it’s a start). I’ve started trying to track down who exactly the IP (82.94.207.11) belongs to. I have a pretty good suspicion who it is.

We’re happy to work with groups who want to mirror the repository using rsync provided you are using that mirror to service other Maven users. Trying to scape Maven central cripples the primary feeder to all the mirrors and the synchronization with other open source organizations we work with. By opening 3000 connections to central and scraping it you screw every other Maven user on the planet you idiots. Anyone who knows me knows how rabidly tenacious I am and I will track down every IP you have and create blacklist that every Maven mirror and Maven repository manager will just drop connections to. If you make the life of Maven users more difficult then necessary I will find a way to do the same to you.