To enable users to scan their applications, we provide an executable JAR with a graphical user interface. With this interface users are a few clicks away from results. But, even with this GUI, some users want to be able to use Insight’s Application Health Check from the command-line because sometimes “clicking” isn’t the most effective way to get something done. If you’re building your application using Apache Maven, you probably already have a terminal window open to invoke its build phases. So, while you’re in there, adding or updating some dependencies in your POM and repackaging your application, why not check whether this dependency update introduced some security vulnerability or license issue, especially if it’s as easy as adding another goal to your command line? Meet the Application Health Check Maven Plugin:

mvn package com.sonatype.insight:ahc:run -D ahc.email=my.name@mycompany.com

Right after all artifacts making up your application have been built, the ahc:run goal will collect their fingerprints and send them to the Insight service. The Insight service will match these fingerprints against a database of OSS licensing and security vulnerability data and identify potential problems. A few minutes after the plugin has uploaded the data, you receive an email with a link to your free Application Health Check report.

[INFO] --- ahc:1.21.2:run (default-cli) @ my-application ---
...
[INFO] Scan completed in 4 seconds
[INFO] Number of directories: 0
[INFO] Number of archives: 37
[INFO] Number of files: 3017
[INFO] Number of errors: 0
[INFO] Uploading scan to https://insight.sonatype.com/
[INFO] Report information will be emailed to my.name@mycompany.com 
       from insight-notification@sonatype.com
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

Notice that you didn’t need to modify your POM to run the plugin. Nothing had to change. In fact, the plugin runs as well without a POM, prompting you for the WAR/EAR/ZIP/TAR.GZ/etc. to be scanned. So whenever you’re in a terminal window and have Maven installed, Application Health Check is right at your fingertips to tell you about security or license issues.

Of course, if you use the plugin on a regular basis to check your apps without having to pass in the full groupId and artifactId of the plugin you just have to make a few tweaks to your Maven settings file. Add the following settings file:

${user.home}/.m2/settings.xml

Then, enter the XML below in it:

<settings>
...
  <pluginGroups>
    <pluginGroup>com.sonatype.insight</pluginGroup>
...
  </pluginGroups>
  <profiles>
    <id>insight</id>
    <properties>
      <ahc.email>my.name@mycompany.com</ahc.email>
    </properties>
  </profiles>
  <activeProfiles>
    <activeProfile>insight</activeProfile>
  </activeProfiles>
</settings>

That blob of XML makes mvn package ahc:run a no-brainer. If you have a project and you want to get started tracking OSS licenses and vulnerabilities, this is the way to get started. We’ll scan your project and then send you an email with the results of the scan.

The plugin has a few more optional parameters e.g. to exclude proprietary packages or customize the report label. Just check out our knowledge base for the details.

…and, the build takes forever the first time you run it. Correction, the build takes forever every time you run it because it is just that big, and because you have the sort of environment that demands you always check for snapshot updates. Welcome to the reality of using Maven on a very large-scale project.

What happens when a new programmer rolls up and runs the build for the first time? What happens every night when you clear a CI build’s local repository? Maven downloads the Internet. That’s what happens. No really, Maven downloads MBs of dependencies from a repository, and it does so one-by-one pulling dependency metadata, POMs, and binaries down from a repository. It makes hundreds (if not thousands of requests) back to a repository.

Wasting Time

I’d like to put forward the idea that this is a problem with the design of the tool. Maybe it wasn’t a problem years ago, but it is today. I’m both busy and impatient, and I can see a better way. If the repository already has artifacts, metadata, and dependency information why not save some time. Maven (or Gradle or Ant or whatever your poison) should make a single request, passing in a collection of GAV coordinates and the repository should be able to calculate all of the dependency information sending back a compressed archive of everything a build would need.

This. I want this:

Making Maven more like Git

Let’s look to Git as an inspiration for how Maven should work going forward. Git makes a single request to a server, the server bundles up everything Git needs, compresses it and then sends down an entire repository. Quick, no waiting for XML parsing on the client side, no pages and pages of “Downloading XYZ…” on your screen, just a simple status that updates you on compression progress and a single network interaction. Git does it right, Maven does it wrong. Let’s fix it.

I shopped this idea around to a few people. Some of the responses have been positive, but a few people have had a strong negative reactions. (“This is unnecessary.”) I really only care about this for selfish reasons. I have a large project, I don’t really want to have to sit next to another new developer and apologize for Maven. In fact, on this same project we checkout a 300 MB git repository and it is fast as could be, then I have to show them how to run the Maven build… we start it and then we go for a long coffee break during which I apologize for Maven:

“Maven isn’t like Git, it takes time to make those requests. Maven has to download POM files, artifacts, metadata. Then it has to create a bunch of dependency graphs and sort out conflicts. Once those conflicts have been solved it then has to go get all the artifacts. One by one. I apologize, listen, the coffee’s on me, ok?”

The Difference: Latency is annoying.

I did some quick metrics with a real project. Actually it is more like a megaproject. This project has an insane number of internal dependencies. Running the build with a prepopulated local repo takes approximately 3 minutes. Running the build without a prepopulated local repo (against a populated Nexus repo) takes approximately 11 minutes and it downloads about 120 MB of dependencies. That 7 minute difference is the download time (over a 100 MBps connection) plus the latency required to setup HTTP connections, parse XML, and calculate dependencies.

Think about that: it took Maven 8 minutes to calculate dependencies for all projects (and all Maven plugins) and then download 120 MB. All the files, all the POMs, all the metadata was already present in Nexus. Let’s use a 120 MB download over a 10 MBps link to establish our baseline – if all we were doing was throwing this file down to the client, it should take ~10 seconds. I’m going to be generous and assume that it would take nexus 20 seconds to calculate all of the dependency information on the server-side (assuming that it didn’t have to retrieve anything from a proxy repo). Let’s them assume that compression of stream of files would add another 5 seconds of overhead.

This is where I’d like to go. Shift the calculation of all dependencies to the server side, send down an archive, and get rid of this back and forth between Maven and the repository. I think we can take this 8 minute Maven dependency mess and turn it into a 35 second process that involves 25 seconds of waiting and 10 seconds of transfer.

This should be easy, who’s up for the challenge?

I think we can do this without having to muck with Maven. I’m not a big fan of the Maven codebase (it’s a monster), and I’d rather not break into the tool itself. I think we could just write a simple CLI wrapper that would interact with a Nexus REST service. All this CLI interface would have to do is parse a pom.xml, gather a collection of dependencies and Maven plugins, and then send a request to Nexus. Nexus would then take this collection of GAV coordinates alongside the name of a single repository group to use for artifact resolution. Nexus would calculate the full list of transitive dependencies and then stream it back as a compressed collection of files to the CLI wrapper.

At this point, the CLI wrapper just unpacks everything and drops it in ~/.m2/repository. Maven would then run as normal, not even aware of the antics of what I’m calling “Maven Rides the Lightning”.

Note, this is not my idea. Nambi Sankaran brought this idea up during a training class several weeks ago, and I very much want to see this happen. Nambi has created a Github repository for this project and if you are interested it would be a good opportunity to get some like minded people together and start thinking about how this could be done. Don’t get too excited, there’s nothing in that repo yet, but I know we have all the pieces available to make this happen. (Plus, I would really like to see some other people create a good open source Nexus plugin.)

Participant: “Could you give us some guidance for migrating Ant projects to Maven? Is there a process that you recommend to make it easier.”

My (honest) answer: “If it’s a complex project, it won’t be an easy battle. Before you go down this road you need to understand what you are signing up for. It can be very complex, you may end up interrupting an active development cycle, and once you evaluate all of your options you might find it easier to first migrate to a repository manager. Maven’s not the big win, moving a repository is.”

In other words, I often find myself trying to discourage swapping build tools just for the sake of swapping build tools. While I do believe that Maven is preferable to Ant, I think that the build space often suffers from a belief that the grass really is greener on the other side. It might be, but is it greener enough to justify that work stoppage that is involved in taking a big important project and moving it to a different build technology? Often the answer is no. If it isn’t related to making money, switching a build system is often the last thing an enterprise wants to do.

In this post I’m going to talk about the process of migrating build tools starts with a repository manager.

Behold the Crazyant Legacy Build

This system is inspired by a real system I worked on for a brief period of time a few years ago. To protect the innocent I’ll call it the “Crazyant Legacy project”. It was awful, and by awful I mean I wanted to run away as fast as I could only an hour after the client granted me access to the source code.

“Wait? Why do these boxes overlap?”, you ask as I lean back in my chair laughing at your fright. “The code modules, they overlap because they share source trees”, I continue, “there is only but one source directory in this project, yet there are many builds. If you’ve done battle with this project, you’ll understand that this diagram is only an approximation. Any one of the hundreds of developers that use this project could define a new project at any moment. Good luck.”

Ok, it wasn’t that dramatic, but it was close. You had Ant projects that were referencing sourcepaths using relative directories like, “../../../../../shared”, and you had a directory called “dependencies/” which contained, you guessed it, all of the binary dependencies in unmarked jars like “SpringProj22.jar”. You pick up a JAR named “SpringPro22″ and there’s no telling what’s in that thing. It could be Spring, maybe? It could be Spring plus someone’s customizations? It could be nothing? You would ask one of the developers, but everyone is too busy because the project can’t stop to migrate to a new build system (also 80% of the developers are proud of this awful build and resent your consultant-tinkering).

A project like this is an unknown minefield, and when someone calls you in to “migrate the build to Maven”, you are being asked to sprint through this minefield as fast as possible. First, you can’t just uproot the whole project and change the structure. Second, you have to take an iterative approach to migration that involves milestones. The alternative, which is never possible in a production environment, is to ask every stakeholder to hold off on requests for two weeks and ask developers to take two weeks of synchronized vacation during that period so you can migrate the build.

That’s never happening, so what’s the first milestone?

Step 1: Download Dependencies from a Repository Manager

Without a doubt, if your goal is to move to a new build tool your first step should have nothing to do with changing the build. Just take a snapshot of the artifacts the build depends on, take a full accounting of the JARs and libraries this behemoth consumes, and adapt the build to download from something like Nexus.

And, you don’t need to be running Maven to do this. You could run Gradle or Ivy or SBT or whatever. The reason you want to move your build to download dependencies from Nexus is that it starts the process of decoupling external dependencies from the contents of your source code repository. It also forces you to make a record of all the builds, how they relate to one another, and what they depend upon. This information will come in useful in a later step.

Step 1 likely took a week or two to complete (longer if your project is huge), but the important thing is that you didn’t have to change toolsets. You didn’t have to ask your developers to stop, drop, and change toolsets, and no one had to tell the boss, “Oh, we’re all just sitting around because the build guy wants to fiddle with the system.” If you carry this process through to the end, you will eventually need to interrupt active development. Minimize this downtime by taking a phased approach.

Step 2a: Publish Artifacts to a Repository

Step 2b: Adapt deployment scripts to consume binaries from a Repository

After the build is consuming artifacts from Nexus, the next step is to start publishing the output of the project to Nexus. While you are making this change you are also concurrently modifying any deployment scripts to grab binaries from Nexus.

Why? Because, quite often, these Crazyant legacy projects have confused development with deployment and there are a series of intricate build scripts that do things like compile the projects and deploy them to production. You’ll want to refactor this out of your project’s source code and make your deployment scripts dependent on binary build output via Nexus. If you do this, it’ll be easier to divide and conquer your builds and get your developers out of the business of production operations.

You will also be defining an interface for your software projects. What is a software project in the context of a repository manager? It is a build that consumes artifacts via Nexus and subsequently publishes artifacts to Nexus. Dependencies in, artifacts out. If you can adapt your existing build to satisfy this contract, the next step will be easier.

Step 3: Carve Out Projects 1 by 1 to a New Build

If you were changing a bunch of awful code what’s the first thing you would do? Would you dive right in and start changing stuff “willy nilly”? No, you’d write a series of unit tests. No one goes near a brittle disaster of technical debt without first making sure that you are not changing the contract.

This is exactly what Steps 1, 2a, and 2b defined for your build. Your build is producing binary output from source code and binary inputs, and your repository manager now stores both the inputs and the outputs. If Crazyant Legacy project had 20 projects and they were all consuming and publishing to Nexus, you have a “testable interface”.

So should you rush everything over to Maven all at once? No. Take your time and carve off projects to a new Maven build one at a time. Greenfield projects can start publishing to Maven by default, and existing projects can move at their own pace.

Decouple with Repositories

If you are moving to a new build system, be it Tesla, Gradle, Maven, or anything else, do yourself a favor and move to Nexus first. Decoupling your projects from direct source dependencies and using tools like Maven or Gradle to consume binary artifacts from a repository will enable you to conduct a lower risk, more piecemeal approach to your build migrations.

Brian Fox sat down with Ken Rimple of Chariot Solutions to talk about Nexus and to put repository management in the context of recent developments with Maven. Ken Rimple and Chariot have been long-term partners with Sonatype supporting our Maven training efforts, so Ken has a lot of background about Maven to ask some interesting questions.

Listen to Chariot TechCast, Episode #71, Brian Fox of Sonatype on Nexus 2

The Legacy of Maven: Binary Reusability

The interview leads off with a discussion about Maven, Maven’s history, and some of the recent developments surrounding the Maven ecosystem. Brian identifies binary reusability and declarative builds as the two important legacies of Maven:

“One of the unique things that Maven brought to the table, and what may in fact be the legacy for Maven years down the road, is that it introduced the concept of having binary reusability and not rebuilding the world as everybody was used to doing. The other aspect was making things more of a declarative model especially the dependencies. That was all unique at the time.”

What it takes to Run Central?

Ken and Brian discuss the introduction and development of Central from the beginning of the repository to the current iteration. From the initial efforts to create a single coordinate system for artifacts in 2001 and 2002. Brian’s very involved in the effort to maintain Central so this is your chance to hear Brian discuss some of the internals of the effort: how much bandwidth does Central consume? how much effort is involved in maintaining Central? and what are the day-to-day operations for running Central?

Key quotes about Central: “We’re approaching half a Terabyte for artifact storage” and “We’re seeing 50 to 60 new projects added every day”.

You will hear about how Nexus is used to enforce standards for artifacts added to Central from forges like Apache, JBoss, java.net, Codehaus, as well as the instance of Nexus that Sonatype provides for independent projects: http://oss.sonatype.org.

New Features in Nexus 2.0

Brian then discusses the important features we’re introducing with Nexus 2.0. Including support for .NET, the Repository Health Check, and our support for distributed proxies.

Again, if you haven’t listened to it, you should. Go over to Chariot Solutions and listen to Chariot Techcast Episode #71 – better yet, why don’t you just open up iTunes and subscribe to Chariot’s podcast? here.

You know who this guy is? Probably not, he’s Rube Goldberg.

I’m surprised by how few engineers know his work. Rube Goldberg was a cartoonist who lived from 1883-1970, he’s famous for drawing cartoons of ridiculous and inconceivably complex machines. His work was important during a time in which the world was becoming increasingly mechanized and automated providing a sort of cultural “steam vent” – a way for people to poke fun at machines and industry. I’d embed his work here, but none of it is public domain, so see for yourself or search Google Images. (Be warned, you can spend hours looking at these cartoons.)

I learned about Rube Goldberg from an Engineering professor who, at the time, said, “Rube Goldberg is the most important thing you’ll learn over the next four years”. Back then, we all thought he was joking, but it turns out that he wasn’t. In fact, I wish more people, especially “build engineers” had some exposure to these cartoons. If they had, they’d take a step back and realize that there has to be a better way.

“accomplishing by complex means what seemingly could be done simply”

The title of this section is the Webster’s New Dictionary’s definition of “Rube Goldberg”. Does this describe your internal software distribution process? I fear it might. I’ve seen some builds and deployment procedures that convince me that people are actively trying to create new Rube Goldberg machines with SCM tools and build scripts. Almost every month I see some new variation of what I eventually characterize as a “Crazy Rube Goldberg” approach to releasing software. Systems that barely work, and that no one dares to change for fear of breaking some crazy contraption.

Goldberg’s cartoons and the build’s inspired by them share common characteristics. The complexity of his cartoons are almost always defined by the complex ways in which some object is transported from one “stage” of the cartoon to the next and the comic “indirection” involved in every process.

A cartoon example, a man slips and falls, stepping on a switch which then activates a mechanical hand that throws a bag of sand on to a scale which then triggers a mechanism to fire an arrow at a balloon. A build example, Jeff runs a build on his laptop because that’s the only place release builds work properly, he then checks the WAR into Subversion and then sends a series of emails to several other people. A deployment engineer then takes the output of Jeff’s build and runs it through a custom, somewhat confusing Perl script to customize a series of configuration variables.

Common Characteristics

The common characteristics between Rube Goldberg cartoons and Rube Goldberg builds are:

• Unnecessary Indirection

  • Cartoon: Instead of just creating a machine to pop a balloon, a cartoon creates a contrived process that involves unnecessary (and often impractical) indirection involving pulleys. Goldberg’s cartons often find inventive ways to convert potential energy to kinetic energy in some fantastic way you wouldn’t have thought of.
  • Awful Build: Instead of using a technology to keep track of binary artifacts, a Rube Goldberg build overloads SCM or (worse) email to send multimegabyte binaries around. Instead of using a technology with built in support for automatic email notification, your build relies on some ad-hoc system of people remember to send just the right individuals a set of process-related emails.

• Reliance on the Unlikely

  • Cartoon: A bag of sand thrown across a room lands exactly on a bulls eye of a target triggering yet another unlikely series of events, continue, repeat…
  • Awful Build: A complex build that takes more than an hour to complete has a random dependency on a web service that provides an series of configuration variables used to configure a binary for production. If the web service happens to be unavailable, or if it returns invalid results, the entire build process must be started from scratch.

• Reliance on Animals (or Humans)

  • Cartoon: One of the comic elements almost always present in a Goldberg cartoon is the presence of an animal of some sort. A bird who, when hit with an arrow squawks loud enough to scare a dog into running and subsequently lifting a weight via a pulley system.
  • Awful Build: Awful build systems frequently require direct human intervention to the point where the build doesn’t work unless the same individual is always “strapped” into the build in just the right way. “After that compilation step completes, Jack usually logs in and modifies the build number before he runs the deployment script”.

• Comedy

  • Cartoons: Rube Goldberg’s cartoons are comedic because they involve the impractical or the impossible. Readers find them amusing because they are plausible, yet inadvisable ways to implement a process.
  • Rube Goldberg-style builds are comedic because they involve the impractical or the impossible. Developers find them amusing because they are plausible, yet inadvisable ways to implement a process.

Build/Deploy systems are Comedy?

If they work, they can be. It’s quite common for the build/deploy lifecycle at a corporation to involve a series of steps that no one fully comprehends. Maybe there’s a crazy 4000 line Ant script or maybe there’s a multi-headed Hydra of bash magic that happens to produce a working system. In the last few months I’ve talked to people with build systems that work, but no one on staff understands how. When you ask about these release procedures people often chuckle a bit – “Oh, right, our release process…. well let’s say it’s a little confusing…”.

“Why does the build do it like this?” is often met by, “I don’t know, but I’m too scared to change this thing right now, the contractor left us with this and it works… for now.” In other words, many of these builds involve a series of unknown pulleys and unexpected indirections that resemble a cartoon.

In the next part of this series, we’re going to talk about how Nexus can be used to untangle a build into something more reasonable. Something that doesn’t involve pulleys, magic, and a series of unlikely events.

Note: Rube Goldberg doesn’t do it for you? Then just watch the OK Go video “This Too Shall Pass” to get a modern idea of what I’m talking about. If you haven’t seen that, you should. In fact, if you haven’t seen this video yet, stop reading our boring Nexus blog and take a break, it’s worth it.

“We will be moving data centers and want to setup another Maven2 repo that is managed with Nexus OSS. We want both repositories to be online and read\writeable until we migrate all our environments to the new site. I have a few questions:

1. What is the best method for copying the repo to the new location?
2. What is the best method for keeping the two repos in sync? We want to minimize network bandwidth usage.”

The video below answers these questions, and offers multiple solutions:

Improving Your Java development with Maven 3 and Hudson

Tuesday, June 7, 2011 @ 10:30AM – 11:00AM EDT (UTC/GMT – 0400)

Presenter: Jason van Zyl, Sonatype CTO & Founder

Attend this webinar to learn about the advantages of upgrading to Apache Maven 3, including improved speed, greater stability and increased compatibility. Jason will also talk about the greatly improved support for Maven 3 within Hudson that is easy to configure and supports complex build scenarios with ease. We will cover the Eclipse IDE integration for both Maven and Hudson that improves developer productivity.

Register now!

About the webinar:

Tycho is a set of Maven plugins and extensions for building Eclipse plugins and OSGI bundles with Maven. Eclipse plugins and OSGI bundles have their own metadata for expressing dependencies, source folder locations, etc. that are normally found in a Maven POM. Tycho uses native metadata for Eclipse plugins and OSGi bundles and uses the POM to configure and drive the build. Tycho supports bundles, fragments, features, update site projects and RCP applications. Tycho also knows how to run JUnit test plugins using OSGi runtime and there is also support for sharing build results using Maven artifact repositories.

Join this webinar to get an overview of the Tycho project and to learn what plans the project has for the future.

• Date: May 3, 2011
• Time: 9:00 am PST / 12:00 pm EST / 4:00 pm UTC / 6:00 pm CET
• Length: 60 minutes
• Enroll here!

Software developer Igor Fedorenko details the new features and changes to m2eclipse 1.0, including pom.xml editor enhancements and reworked build lifecycle mapping.

For more videos from the Sonatype team, visit our Resource Center, or go to our YouTube channel.

To view the replay, click here.

Our next webinar – Hudson Continuous Integration with Sonatype Professional – is taking place on March 29. If your development team is looking for best practices for integrating Maven, Hudson and Nexus Professional this webinar is for you.

Sonatype Professional is the only integrated suite created for Maven-centric builds. The suite includes Sonatype Certified & Integrated Distributions of: Maven, Hudson, Nexus Professional, Eclipse Integration, Developer Onboarding as well as support from the experts. Register to learn how Sonatype Professional empowers development teams to realize the promise of agile through continuous integration, while reducing project risk.

• Date: Tuesday, March 29, 2011
• Time: 9:00 am Eastern Daylight Time (New York, GMT-04:00)
• Duration: 30 mins
• Presenter: Blaine Mincey, Sonatype Senior Systems Engineer
• Register for this webinar