Other titles tested: The Yumqueal. The Yum strikes back. Yum Harder. Yum reloaded...
A large part of my first year at Sonatype has been spent working on Yum support for Nexus Repository Manager (NXRM) and as we have just released Yum Hosted support, I thought I deserved to spend a morning coming up with cheesy blog post titles I thought now would be a good time to share what it’s like to develop a new format.
Let’s rewind a bit first. When I joined Sonatype, NXRM was one big team and the work spanned the entirety of the product. We decided that, to help us scale, we would split into “tiny” teams, each with a specialised focus. For me this was a great change because I became a member of the newly created Formats team, specializing in adding and maintaining format support. I count myself very lucky to be a member of this team and also a part of the wider NXRM team, the problems are fun to solve and the people are awesome, really awesome.
When we start work on a new format we start by having a play, learning what is like to be a user of that format. In this case we grabbed a CentOS VM and started running some yum commands trying to answer the following questions:
- What commands are there?
- How do users currently create a yum repository?
- How is yum configured?
- How might we configure yum to talk to a NXRM repository?
It is at this point that it went from learning how a user uses Yum to reverse engineering how the format works. Yum is open source so we grabbed the code and started reading, we scoured documentation, we added proxies to capture the http requests, we examined the metadata to see how it was structured and how it was related and, we looked at the yum implementation in NXRM2. Some of the questions we were trying to answer were:
- Does it even have metadata?
- What metadata did we need?
- Does it cache the metadata?
- What packages is it grabbing?
- Where is it grabbing the packages from?
It is a rewarding process to go through because each time you look you discover something new but at the same time it is tough. Trawling through code in a language that isn’t the one you use every day comes with a big learning curve and reading through 20 year old documentation can be frustrating. All that effort though leads to a moment where it clicks into place and you feel like you have enough knowledge to build it in NXRM; that is a great feeling.
As you may know, one of the ways format clients like yum find packages to download is to first fetch (what we call) metadata files which act like a catalog of packages. With Yum Hosted, one of the challenges was how to generate this metadata. The way to build a Yum repository without a repository manager would be to drag your RPMs into a folder and run createrepo which would create the metadata files. So one possibility we had was that we could tap into a local install of createrepo (which is how NXRM2 worked). We decided to go a different route and build createrepo from the ground up. This meant that we don’t have a dependency on an external tool, making concurrent access safer and also making the NXRM yum implementation platform independent.
The next step in the process was my favourite part. Building it. We took everything we had learned from reverse engineering the tools and started work on Yum. This is where I get the most enjoyment, seeing all of the leg work come to life.
And that gets us to where we are today. Release of Hosted (cue dramatic music). This, for me, is the most exciting—but also nerve-racking—part of the process. Exciting because it is a great feeling to see people using your work; nerve-racking because I hope it does what everyone needs.
Give it a go and let us know what you think!
