In this short, two-minute excerpt from my interview with Brian Fox, he describes the steps Sonatype took to mitigate the load and bandwidth problem which were affecting the Central Maven Repository late last year. While short, this audio contains some very useful information for anyone facing similar traffic problems. You’ll hear him talk about the switch from Apache httpd to nginx.
[media id=3 width=320 height=70]
Yesterday, in Central Maven Repository Traffic: Investigation and Analysis, I wrote about the analysis involved in tracking down the increasing load on Central. By identifiying some misbehaving tools, we were able to reduce the traffic from a 98 Mbps average down to 60-80 Mbps. In this post, I discuss the next step toward a Central Maven repository that can scale to meet the load generated by the millions of developers using an ecosystem of tools which rely on the Central Maven Repository. Continue reading
For several months starting in August, the use of the Maven Central Repository has seen a dramatic increase in traffic. This is a great indicator of the adoption of Maven and other tools that recognize the value of a shared binary repository. It also set in motion a series of upgrades and changes required to keep the system going strong for everyone. We’re written about some of the steps we’ve taken in the last few months to preserve Maven Central as a public resource, and I wanted to provide more details for those of you who are interested in the numbers behind these changes.
The first change occurred in August, when we moved Central to a new 100 Mbps connection. This temporarily solved most of our availability problems, but the load continued to increase and Central started running out of httpd worker threads. For a while we played a cat and mouse game of bumping up the workers. Read on to find out how we ultimately solved this problem and made Central more stable and available for the world of developers it serves.
