We were talking about creating a guide for this, but someone in the community beat us to it.  Yennick Trevels published an excellent guide in his blog that you can find here.  We highly recommend checking out his post if you want to use Gradle to deploy artifacts to the Central Repository.

At Sonatype, we want to make synchronizing and publishing your artifacts to Central easier and to improve the quality of repository metadata for everyone at the same time. To facilitate this, we offer a dedicated instance of Sonatype Pro for Nexus at http://oss.sonatype.org specifically to host the artifacts of open source projects. In this post, I talk about the process of creating a repository for your open source projects and publishing artifacts so that they will be available from the Central Repository.

This service has been available since 2009 and includes many projects such as Plexus, Jetty, Google Guice, Spring and Ehcache (Greg wrote about his experience with migrating to oss.sonatype.org). We have tooling in place to make it easy for us to process a larger set of requests, so we invite everyone to use this resource. As of October, 2011, we have over 1,500 projects using this repository on a daily basis.

To get the process started, go here. We’ll setup a release and snapshot repository for your project, along with the appropriate configuration to allow you to use the staging features for your releases. If you have an existing repository somewhere, we can migrate that for you too. We’ll even help you add artifacts to Central that you use, but don’t necessarily own — assuming of course that it doesn’t violate the projects license.

The system allows customizable rules to be run during the staging process, which allows us to automatically check things like valid pgp signatures and correct POM parsing. This will ensure that your users have the best experience possible when using your artifacts, and relieve some of the manual validation on your side — a win for everyone.

On the technical details, this instance gets its network connection via Contegix‘s high availability network, the same one running Central, Codehaus.org and Atlassian.com. New Relic has donated monitoring services to help us monitor and tune this instance of Nexus. Since OSSRH is hosted on the same infrastructure as the Central Repository, we are able to frequently synchronize the repositories.

Next time you need to add a project to the Central Repository, you’ll know how.

• Git works better for people with slow or lagging connections
• Branch management is easy
• Submitting patches via a pull request is easier to deal with than a patch attached to a bug report

We started out with SVN mostly because that was what everyone was used to. A year ago or so, one of our remote developers started using a git-svn mirror to remove some of the latency issues they had with SVN. Someone else put together a ten page wiki on how to easily use a git-svn mirror and push back to the canonical SVN repo. As time went on, interest in git grew and new modules were created in git, instead of our SVN repositories.  Finally there was a push to move everything to git.

There were a couple different directory conventions used for our repositories.  Our Nexus SVN repository used the typical project-root/trunk,tags,branches, and git-svn worked nice right out of the box. Our other repositories lumped all modules under the trunk, tags, and branches folder (figure 1).

We decided to give each module its own git repository to better manage tags and branches related to the given module. This made the git import a pain, as you cannot just export tags/* (and tags/module-B-* will not work either), you need to be more selective and configure each tag to pull.

Figure 1.

In a nut shell you need to:

(1) Create a new git repository.

$ git svn init –trunk=trunk/project <svn-root> <repo-name>

(2) Configure the specific tags you want to export (we use the Maven release plugin, so the tagging convention is the same)

$ git config –add svn-remote.svn.tags tags/{module-B-2.1, module-B-2.2}:refs/remotes/tags/*

(3) Fetch everything (for an automated way of generating an authors file see this blog post)

$ git svn fetch –authors-file=<svn-authors-file>

(4) Next you need to clean up the tags and push.

If you script this, you end up with something like:

#!/bin/bash

CURRENT_PATH=`pwd`

# file based URL for reads
SVN_URL=file:///path/to/repositories/sonatype.org/spice
SVN_REMOTE_URL=https://svn.sonatype.org/spice
SVN_TAGS_DIR=tags
SVN_TAG_PREFIX=$1
SVN_TRUNK_DIR=trunk/${SVN_TAG_PREFIX}
SVN_AUTHORS=/path/to/svn-authors.txt

GIT_REPO_NAME=$1
GITHUB_USER=userid
GITHUB_TOKEN=xxx
GITHUB_TEAM_ID=1234
GITHUB_ORG=sonatype

# this is the magic, get the list of tags that match this project the output format looks like:
# tags/{module-1.0,module-1.2,module-2.1}
GIT_TAG_ARG=${SVN_TAGS_DIR}/\{`svn list ${SVN_URL}/${SVN_TAGS_DIR} | grep ${SVN_TAG_PREFIX} | sed 's_/__' | xargs -I {} echo -n ,{} | sed 's_^,__'`\}

#echo ${GIT_TAG_ARG}

# init git repo
git svn init --trunk=${SVN_TRUNK_DIR} ${SVN_URL} ${SVN_TAG_PREFIX}

#copy authors text to repo
cp ${SVN_AUTHORS} ${SVN_TAG_PREFIX}/.svn-authors.txt

# move to directory
cd ${SVN_TAG_PREFIX}

# make sure we have tags for this project
if [ `svn list ${SVN_URL}/${SVN_TAGS_DIR} | grep ${SVN_TAG_PREFIX} | wc -l` -gt 0 ]; then
  #configure the tags, we only want a subset of the tags, because we lumpped all the tags in a single place.
  git config --add svn-remote.svn.tags ${GIT_TAG_ARG}:refs/remotes/tags/*
fi

#fetch
git svn fetch --authors-file=.svn-authors.txt
# make sure there are no errors before we continue
if [ "$?" -ne "0" ]; then
  echo "Error while fetching, see above"
  exit 1
fi

git update-ref -d master

##########################################################
# fix the tags, from: https://github.com/nothingmuch/git-svn-abandon
##########################################################

# create annotated tags out of svn tags
git for-each-ref --format='%(refname)' refs/remotes/tags/* | while read tag_ref; do
    tag=${tag_ref#refs/remotes/tags/}
    tree=$( git rev-parse "$tag_ref": )

    # find the oldest ancestor for which the tree is the same
    parent_ref="$tag_ref";
    while [ $( git rev-parse --quiet --verify "$parent_ref"^: ) = "$tree" ]; do
        parent_ref="$parent_ref"^
    done
    parent=$( git rev-parse "$parent_ref" );

    # if this ancestor is in trunk then we can just tag it
    # otherwise the tag has diverged from trunk and it's actually more like a
    # branch than a tag
    merge=$( git merge-base "refs/remotes/trunk" $parent );
    if [ "$merge" = "$parent" ]; then
        target_ref=$parent
    else
        echo "tag has diverged: $tag"
        target_ref="$tag_ref"
    fi

    # create an annotated tag based on the last commit in the tag, and delete the "branchy" ref for the tag
    git show -s --pretty='format:%s%n%n%b' "$tag_ref" | \
    perl -ne 'next if /^git-svn-id:/; $s++, next if /^\s*r\d+\@.*:.*\|/; s/^ // if $s; print' | \
    env GIT_COMMITTER_NAME="$(  git show -s --pretty='format:%an' "$tag_ref" )" \
        GIT_COMMITTER_EMAIL="$( git show -s --pretty='format:%ae' "$tag_ref" )" \
        GIT_COMMITTER_DATE="$(  git show -s --pretty='format:%ad' "$tag_ref" )" \
        git tag -a -F - "$tag" "$target_ref"

    git update-ref -d "$tag_ref"
done

# create local branches out of svn branches
git for-each-ref --format='%(refname)' refs/remotes/ | while read branch_ref; do
    branch=${branch_ref#refs/remotes/}
    git branch "$branch" "$branch_ref"
    git update-ref -d "$branch_ref"
done

# remove merged branches
git for-each-ref --format='%(refname)' refs/heads | while read branch; do
    git rev-parse --quiet --verify "$branch" || continue # make sure it still exists
    git symbolic-ref HEAD "$branch"
    git branch -d $( git branch --merged | grep -v '^\*' )
done
##########################################################
# done fixing tags
##########################################################

# we should already be on the master, but lets make sure
git checkout master

##
# Use the Github API to create a repo and add it to the team.
##

#create github repo
curl -F "login=${GITHUB_USER}" -F "token=${GITHUB_TOKEN}" https://github.com/api/v2/json/repos/create -F "name=${GITHUB_ORG}/${GIT_REPO_NAME}" -F "has_issues=false" -F "has_downloads=false" -F "has_wiki=false"  --request POST

#add the dev team to the repo
curl -F "login=${GITHUB_USER}" -F "token=${GITHUB_TOKEN}" -F "name=${GITHUB_ORG}/${GIT_REPO_NAME}" http://github.com/api/v2/json/teams/${GITHUB_TEAM_ID}/repositories

# set the origin
git remote add origin git@github.com:${GITHUB_ORG}/${GIT_REPO_NAME}.git
#push it all
git push --tags origin master

#go back to where we started
cd ${CURRENT_PATH}

# some reminders so we do not forget the manual steps
echo "--${GIT_REPO_NAME}--" >> migrate.out
echo "You should now review the git repo at: https://github.com/${GITHUB_ORG}/${GIT_REPO_NAME} then remove the svn repo with:" >> migrate.out
echo "$ svn rm ${SVN_REMOTE_URL}/${SVN_TRUNK_DIR} -m  \"Moved to github: https://github.com/${GITHUB_ORG}/${GIT_REPO_NAME}\"" >> migrate.out
echo "Do not forget to update the grid: https://grid.sonatype.org/ci/\" to use: git://github.com/${GITHUB_ORG}/${GIT_REPO_NAME}.git >> migrate.out
echo "Add redirect from ${SVN_REMOTE_URL}/${SVN_TRUNK_DIR} to https://github.com/${GITHUB_ORG}/${GIT_REPO_NAME}" >> migrate.out
echo "" >> migrate.out

This PDF covers the same ground as the video for those of you who are looking for something simple to print out.

Quickstart Nexus Open Source on Windows

Even though encoding seems to be handled correctly in your environment, with your local language settings or some implicit default value, the best practice is to always explicitly define the character encoding. This will make sure that your build is portable and it will also protect you in the future in case some of your default values change. The bottom line is, just do it!

In the Maven world, you need to specify the character encoding for every plugin that processes source files or creates reports. It would have been great if this were made possible through a shared configuration element in the POM, but that would require an update to the POM model and that won’t happen until Maven 3.x. In the meantime, two special properties have been defined.

project.build.sourceEncoding
project.reporting.outputEncoding

If these properties are set, most plugins will use the character encoding defined through them. So now we don’t have to individually configure all plugins. If needed, the encoding defined through the properties can be overridden by defining some other encoding in the plugin’s configuration. As always, some plugin could work differently and you should always consult the plugin’s documentation.

More information about the thoughts behind these special properties and how to use them can be found in the proposals below. On these pages you can also find out which plugins have been updated to use these properties.

• POM Element for Source File Encoding
• Reporting Encoding Configuration

Dennis, thanks for the tip!

Maven 3.0

With Maven 3.x we have made our best attempt to listen to users, find out what they need and want, and make reasonable preparations and plans to fix the problems, implement useful features, and create the integration points for working properly with third-party systems like Nexus and Hudson. While there have been many new feature requests — and we have many new features done or in the works — the number one concern is backward compatibility. Maven 3.x makes an ardent attempt to be 100% backward-compatible with Maven 2.x. All your Maven 2.x builds should work out of the box using 3.x without change to your projects. We simply can’t cause pain to the existing user base. We are very cognizant that even small changes can have a huge impact and so we have tried achieve complete Maven 2.x fidelity with Maven 3.x.

In Maven 3.x we have reduced the number of modules, we have simplified and extracted the artifact resolution system, we have put a performance framework in place, and we have created an enormous set of integration tests for Maven’s core and for many of the core plugins. We are constantly trying to validate core features and core plugins are working properly. We hope with the simplifications that we have made that more people will get involved in the project. We have done some heavy overhauling of many of the internals to make way for future features and I believe Maven 3.x is really the only path forward for Maven. What’s the technique we used to decide on how to plan and work toward the completion of Maven 3.x? A very practical approach to determining an optimal path called desire lines.

Desire Lines

A desire line usually represents the shortest or most easily navigated route between an origin and destination. The width and amount of erosion of the line represents the amount of demand. The term was coined by Gaston Bachelard in his book The Poetics of Space. Desire lines can usually be found as shortcuts where constructed pathways take a circuitous route. A concrete example: the pathways around the Berkeley campus in California. During the early years of Berkeley no pathways were paved. Instead they let inhabitants walk in optimal paths between the buildings and location over a period time to form clear pathways over the grass. Once these pathways had been established they could be paved to make the pathway more permanent. This is very similar to what happened with Maven 2.x. Consider Maven 2.x the pathways marked in the grass. Consider Maven 3.x taking all that learning from Maven 2.x and adopting the optimal form of use and codifying those forms of use i.e. paving the desire lines.

Over the course of hundreds of thousands of people using Maven we have a very good idea of what these optimal pathways look like now. Here are a few examples of some of the things we’ve seen and the things we have to do in order to make these lasting improvements.

Responding to invitations for improvement

There are many things in Maven 2.x that work reasonably well but have some minor flaws which cause irritation:

• Dependency management. How to effectively manage what we will call a target platform (intentionally using the Eclipse nomenclature). It is easy to manage the versions of an applications runtime or its target platform. But what happens when you start pulling in multiple, separately developed target platforms.
• Version specifications and version ranges. Maven’s notion of this is workable, but when it comes to the version specification and ranges it’s time to just defer to OSGi in this regard. What OSGi specifies and what Maven specifies are not wildly different but different enough to cause some problems. We’ll be working on aligning Maven to the version specification of OSGi.
• The way plugins interact with embedded environments was just wrong. We allowed for no support for incremental changes. The result is that in M2Eclipse when you change an individual resource, that individual resource can be processed efficiently and not fire up all of Maven to run the whole build lifecycle. Sorry, but you won’t have time to get a coffee anymore.
• There are slew more, but we’ll save those for other blog entries.

Responding to … being completely wrong

There are some things in Maven 2.x we just didn’t get right and we have to make reparations:

• Composition versus inheritance in the POM. There are some great debugged toolchains like we have in the Apache Organization POM for releasing, but this toolchain is not easily consumable by outside projects because it makes no sense to inherit from the Apache Organization POM for your projects. So how can we make these chunks of debugged combinations of plugins and their associated configuration? We will introduce mixins to help with this. Essentially it will be a POM consisting of plugins and configurations that can be externally parameterized. These mixins will be deployed to a repository and be referenced with a standard coordinate. Basically it will be an intelligent import with validation which will allow composition in your POMs.
• Checking out the whole source tree versus an individual module and parent element versioning. We have always intentionally made projects specify versions in the parent elements. When you check out individual modules this is necessary. But for projects where the whole source tree is checked out, the version of the parent can be inferred and the requirement for specifying the version element in the parent can be removed.
• Clean separation of operations that need to happen at the beginning and end of the lifecycle versus in the lifecycle itself. To make the OSGi integration we created work we needed to get at the projects before the build lifecycle was executed so we added a clear hook before lifecycle execution. What we have done to make OSGi integration work in Maven 3.x is simply not possible in Maven 2.x. We also had a terrible problem with reporting and aggregation because there was no clear point at which the lifecycle was over. Now once the build lifecycle is complete there is a clear hook to get the projects in the reactor so they can be processed easily. Accurate aggregation is now possible.

• Again, there are slew more, but we’ll also save those for other blog entries.

Responding to under-utilization of what exists

Unfortunately there are still a lot of powerful features and plugins in Maven that a lot of people don’t know about. We have tried to remedy this situation with the four books that we constantly update stream, and a stream of blog posts, and the soon to be enterprise Maven users list which will focus on the holistic and systematic use of Maven, M2Eclipse, Nexus and Hudson. We need to do a better job explaining end-to-end best practices for developing, testing, and provisioning software. We also need to do a better job describing some of the incredibly useful plugins we have like the enforcer plugin and the dependency plugin.

But again, our best attempt to show people what is available through the four books that we have:

• Maven: The Definitive Guide
• Repository Management with Nexus
• Developing with Eclipse and Maven
• The Maven Handbook

Here is a picture I always use to illustrate the point that even though something may be sitting directly in front of you, it’s not always immediately obvious unless someone points it out to you. Th is is very much the case where we see users asking us for help with particular use cases and in frequently it’s very easy to answer the question by pointing to a URL. My example is the Fedex logo which was designed to have an arrow being formed between the last two letters of the logo. Quite ingenious but most people I point this out to haven’t notice it.

We are going to try a lot harder to make sure the value that exists is easier to find. I don’t enjoy reading blog posts from unhappy users at all, but I especially don’t like it when I know there is a solution which has been documented somewhere. Those painful situations can be reduced if not eliminated completely.

Backward Compatibility

I can’t stress enough how important backward compatibility is to us. We have done a non-trivial amount of work to pave the way for new capabilities in Maven 3.x while preserving 100% backward compatibility for:

• Maven itself and the behaviour that users expect from the CLI
• Artifact Resolution API
• Plugin API
• Plugin configuration
• Site generation (even though we’ve extracted site/reporting entirely into the maven-site-plugin)

We must ensure that plugins and reports written against the Maven 2.0.x APIs remain viable in 3.0. We don’t want people rewriting their plugins or having to change their projects at all or it will simply be chaos. We simply have too many users and requiring changes will have an enormous human cost so we’ve been slow in announcing Maven 3.x. The version of Maven 3.x you can pull from the Sonatype grid is probably the best version of Maven that has existed but we are still being careful about releasing it. If you want to try it you can find it here.

We must also ensure that POMs of version 4.0.0 are supported in 3.x along with the behavior currently experienced. We will need to make changes to the POM in order to introduce many of the new features so we’ve made the requisite preparation. We are relying heavily on our integrations tests right now but as we move forward the work that Benjamin is doing on the project/model-builder will help us to accommodate different versions of a POM, and different formats we decide to support. The model-builder code has no limitation with respect to formats. We can support XML, or any source that anyone can dream up. These implementations may find use outside of Maven. For example someone might build something with the Maven, JRuby, and Mercury to create a JRuby-based system. The same could be done for Groovy. We already have some examples of this with the Polyglot Maven project we’ve started which will be the topic of a subsequent post.

We have managed to keep almost everything backward compatible and we will go so far as to provide an isolated execution environment that can contain older versions of the plugin API so that everything from the past can continue to work in Maven 3.x. We will not truly be able to do this until we change the internal Maven runtime over to OSGi but we know what needs to be done.

Integration Testing

An enormous amount of time and energy has gone into improving the the integration tests for Maven. The integration tests are the gatekeeper and let us determine that we have a Maven 3.x that is compatible for Maven 2.x. We now have 506 integration tests that will help protect us as we move forward. We will likely approach 600 integration tests by the time we reach the Maven 3.0 GA.

All core Maven plugins now have ITs and we have started branching out into the Mojo project to create plugin ITs there as well. We have a good pattern so we are encouraging anyone writing a Maven plugin to create ITs so that we can help ensure we don’t break people in the future. We have really taken fixing the problems we see seriously. I couldn’t help but make a little spoof of the SNL skit and adapting it for the Maven perspective.

In the next post I will start talking about some of the technical changes we have made to Maven 3.x in order to achieve our goals. Those will include:

• Refactored model/project builder (the support Polyglot Maven uses)
• Queryable lifecycle
• Lifecycle Extension points
• Error and integrity reporting
• Mercury: Jetty Client & SAT4J
• Embedding
• Incremental build support (used heavily in M2Eclipse for performance improvements, these are changes to the plugin API)

I plan to try and write something about Maven 3.x frequently in preparation for the Maven 3.x talks in Stockholm, Dusseldorf, Cologne, and Devoxx.

Stay tuned!

To illustrate this problem, consider a 1.0 build of your product that depends on foo-1.2.jar. It works great. Then you build 2.0 of your product which still depends on foo-1.2.jar.  Since then, foo-1.2.jar was patched and now breaks your application.  However the application still works for half of your developers because the original foo-1.2.jar is in their local repository and possibly proxied in another Nexus instance.

The solution is to release a new version foo-1.2.1.jar. (ie foo-1.2.2.jar or foo-1.2.1.1-jar), but that alone isn’t enough. You want to make sure that you are using a repository manager that prevents someone from updating a release artifact once it has been published.

It has always been possible to stop people from doing this in Nexus but the solution was difficult to explain.  In the Nexus 1.4 release, we have reworked and simplified the interface to encourage this best-practive.

To disable redeployment, edit your repository and set the ‘Deployment Policy’ field to ‘Disable Redeploy’ then click save.

Upgrading from a previous version of Nexus 1.4.0 will set this field to ‘Allow Redeploy’, so no existing repositories will be changed and the behavior matches previous releases. However the default value when creating a new repository is now ‘Disable Redeploy’.

When you use Nexus, you are not just using a capable repository manager, you are adopting the best-practice.

A good rule of thumb in Maven is to always declare explicit dependencies for classes referenced in your code. If you are going to be importing Commons BeanUtils classes, you should also be declaring a direct dependency on Commons BeanUtils. Fortunately, via bytecode analysis, the Maven Dependency plugin is able to assist you in uncovering direct references to dependencies. Using the updated POMs we previously optimized, let’s look to see if any errors pop up:

$ mvn dependency:analyze
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO]   Chapter 8 Simple Parent Project
[INFO]   Chapter 8 Simple Object Model
[INFO]   Chapter 8 Simple Weather API
[INFO]   Chapter 8 Simple Persistence API
[INFO]   Chapter 8 Simple Command Line Tool
[INFO]   Chapter 8 Simple Web Application
[INFO]   Chapter 8 Parent Project
[INFO] Searching repository for plugin with prefix: 'dependency'.

...

[INFO] ------------------------------------------------------------------------
[INFO] Building Chapter 8 Simple Object Model
[INFO]    task-segment: [dependency:analyze]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing dependency:analyze
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [dependency:analyze]
[WARNING] Used undeclared dependencies found:
[WARNING]    javax.persistence:persistence-api:jar:1.0:compile
[WARNING] Unused declared dependencies found:
[WARNING]    org.hibernate:hibernate-annotations:jar:3.3.0.ga:compile
[WARNING]    org.hibernate:hibernate:jar:3.2.5.ga:compile
[WARNING]    junit:junit:jar:3.8.1:test

...

[INFO] ------------------------------------------------------------------------
[INFO] Building Chapter 8 Simple Web Application
[INFO]    task-segment: [dependency:analyze]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing dependency:analyze
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] No sources to compile
[INFO] [dependency:analyze]
[WARNING] Used undeclared dependencies found:
[WARNING]    org.sonatype.mavenbook.optimize:simple-model:jar:1.0:compile
[WARNING] Unused declared dependencies found:
[WARNING]    org.apache.velocity:velocity:jar:1.5:compile
[WARNING]    javax.servlet:jstl:jar:1.1.2:compile
[WARNING]    taglibs:standard:jar:1.1.2:compile
[WARNING]    junit:junit:jar:3.8.1:test

In the truncated output just shown, you can see the output of the dependency:analyze goal. This goal analyzes the project to see whether there are any indirect dependencies, or dependencies that are being referenced but are not directly declared. In the simple-model project, the Dependency plugin indicates a “used undeclared dependency” on javax.persistence:persistence-api. To investigate further, go to the simple-model directory and run the dependency:tree goal, which will list all of the project’s direct and transitive dependencies:

$ mvn dependency:tree
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'dependency'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Chapter 8 Simple Object Model
[INFO]    task-segment: [dependency:tree]
[INFO] ------------------------------------------------------------------------
[INFO] [dependency:tree]
[INFO] org.sonatype.mavenbook.optimize:simple-model:jar:1.0
[INFO] +- org.hibernate:hibernate-annotations:jar:3.3.0.ga:compile
[INFO] |  \- javax.persistence:persistence-api:jar:1.0:compile
[INFO] +- org.hibernate:hibernate:jar:3.2.5.ga:compile
[INFO] |  +- net.sf.ehcache:ehcache:jar:1.2.3:compile
[INFO] |  +- commons-logging:commons-logging:jar:1.0.4:compile
[INFO] |  +- asm:asm-attrs:jar:1.5.3:compile
[INFO] |  +- dom4j:dom4j:jar:1.6.1:compile
[INFO] |  +- antlr:antlr:jar:2.7.6:compile
[INFO] |  +- cglib:cglib:jar:2.1_3:compile
[INFO] |  +- asm:asm:jar:1.5.3:compile
[INFO] |  \- commons-collections:commons-collections:jar:2.1.1:compile
[INFO] \- junit:junit:jar:3.8.1:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------

From this output, we can see that the persistence-api dependency is coming from hibernate. A cursory scan of the source in this module will reveal many javax.persistence import statements confirming that we are, indeed, directly referencing this dependency. The simple fix is to add a direct reference to the dependency. In this example, we put the dependency version in simple-parent’s dependencyManagement section because the dependency is linked to Hibernate, and the Hibernate version is declared here. Eventually you are going to want to upgrade your project’s version of Hibernate. Listing the persistence-api dependency version near the Hibernate dependency version will make it more obvious later when your team modifies the parent POM to upgrade the Hibernate version.

If you look at the dependency:analyze output from the simple-web module, you will see that we also need to add a direct reference to the simple-model dependency. The code in simple-webapp directly references the model objects in simple-model, and the simple-model is exposed to simple-webapp as a transitive dependency via simple-persist. Since this is a sibling dependency that shares both the version and groupId, the dependency can be defined in simple-webapp’s pom.xml using the ${project.groupId} and ${project.version}.

How did the Maven Dependency plugin uncover these issues? How does dependency:analyze know which classes and dependencies are directly referenced by your project’s bytecode? The Dependency plugin uses the ObjectWeb ASM (http://asm.objectweb.org/) toolkit to analyze the raw bytecode. The Dependency plugin uses ASM to walk through all the classes in the current project, and it builds a list of every other class referenced. It then walks through all the dependencies, direct and transitive, and marks off the classes discovered in the direct dependencies. Any classes not located in the direct dependencies are discovered in the transitive dependencies, and the list of “used, undeclared dependencies” is produced.

In contrast, the list of unused, declared dependencies is a little trickier to validate, and less useful than the “used, undeclared dependencies.” For one, some dependencies are used only at runtime or for tests, and they won’t be found in the bytecode. These are pretty obvious when you see them in the output; for example, JUnit appears in this list, but this is expected because it is used only for unit tests. You’ll also notice that the Velocity and Servlet API dependencies are listed in this list for the simple-web module. This is also expected because, although the project doesn’t have any direct references to the classes of these artifacts, they are still essential during runtime.

Be careful when removing any unused, declared dependencies unless you have very good test coverage, or you might introduce a runtime error. A more sinister issue pops up with bytecode optimization. For example, it is legal for a compiler to substitute the value of a constant and optimize away the reference. Removing this dependency will cause the compile to fail, yet the tool shows it as unused. Future versions of the Maven Dependency plugin will provide better techniques for detecting and/or ignoring these types of issues.

You should use the dependency:analyze tool periodically to detect these common errors in your projects. It can be configured to fail the build if certain conditions are found, and it is also available as a report.

Grouping Dependencies

If you have a set of dependencies which are logically grouped together. You can create a project with pom packaging that groups dependencies together. For example, let’s assume that your application uses Hibernate, a popular Object-Relational mapping framework. Every project which uses Hibernate might also have a dependency on the Spring Framework and a MySQL JDBC driver. Instead of having to include these dependencies in every project that uses Hibernate, Spring, and MySQL you could create a special POM that does nothing more than declare a set of common dependencies. You could create a project called persistence-deps (short for Persistence Dependencies), and have every project that needs to do persistence depend on this convenience project:

<project>
  <groupId>org.sonatype.mavenbook</groupId>
  <artifactId>persistence-deps</artifactId>
  <version>1.0</version>
  <packaging>pom</packaging>
  <dependencies>
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate</artifactId>
      <version>${hibernateVersion}</version>
    </dependency>
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-annotations</artifactId>
      <version>${hibernateAnnotationsVersion}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-hibernate3</artifactId>
      <version>${springVersion}</version>
    </dependency>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>${mysqlVersion}</version>
    </dependency>
  </dependencies>
  <properties>
    <mysqlVersion>(5.1,)</mysqlVersion>
    <springVersion>(2.0.6,)</springVersion>
    <hibernateVersion>3.2.5.ga</hibernateVersion>
    <hibernateAnnotationsVersion>3.3.0.ga</hibernateAnnotationsVersion>
  </properties>
</project>

If you create this project in a directory named persistence-deps, all you need to do is create this pom.xml and run mvn install. Since the packaging type is pom, this POM is installed in your local repository. You can now add this project as a dependency and all of its dependencies will be added to your project. When you declare a dependency on this persistence-deps project, don’t forget to specify the dependency type as pom.

<project>
  <description>This is a project requiring JDBC</description>
  ...
  <dependencies>
    ...
    <dependency>
      <groupId>org.sonatype.mavenbook</groupId>
      <artifactId>persistence-deps</artifactId>
      <version>1.0</version>
      <type>pom</type>
    </dependency>
  </dependencies>
</project>

If you later decide to switch to a different JDBC driver (for example, JTDS), just replace the dependencies in the persistence-deps project to use net.sourceforge.jtds:jtds instead of mysql:mysql-java-connector and update the version number. All projects depending on persistence-deps will use JTDS if they decide to update to the newer version. Consolidating related dependencies is a good way to cut down on the length of pom.xml files that start having to depend on a large number of dependencies. If you need to share a large number of dependencies between projects, you could also just establish parent-child relationships between projects and refactor all common dependencies to the parent project, but the disadvantage of the parent-child approach is that a project can have only one parent. Sometimes it makes more sense to group similar dependencies together and reference a pom dependency. This way, your project can reference as many of these consolidated dependency POMs as it needs.

Maven uses the depth of a dependency in the tree when resolving conflicts using a nearest-wins approach. Using the dependency grouping technique above pushes those dependencies one level down in the tree. Keep this in mind when choosing between grouping in a pom or using dependencyManagement in a parent POM

Introduction

Once you start to use Maven to deploy software to remote repositories and to interact with source control systems directly, you will start to collect a number of passwords in your Maven Settings and without a mechanism for encrypting these passwords, a developer’s ~/.m2/settings.xml will quickly become a security risk as it will contain plain-text passwords to source control and repository managers. Maven 2.1 introduced a facility to encrypt passwords in a user’s Maven Settings (~/.m2/settings.xml). To do this, you must first create a master password and store this master password in a security-settings.xml file in ~/.m2/settings-security.xml. You can then use this master password to encrypt passwords stored in Maven Settings (~/.m2/settings.xml).

To illustrate this feature, consider the process Maven uses to retrieve an unencrypted server password for a user’s Maven Settings as shown in the following figure. A user will reference a named server using an identifier in a project’s POM, Maven looks for a matching server in Maven Settings. When it finds a matching server element in Maven Settings, Maven will then use the password associated with that server element and send this password along to the server. The password is stored as plain-text in ~/.m2/settings.xml and it is readily available to anyone who has read access to this file.

Next, consider the process Maven uses to support encrypted passwords as shown in the following figure.

To configure encrypted passwords, create a master password by running mvn -emp or mvn –encrypt-master-password followed by your master password.

$ mvn -emp mypassword
{rsB56BJcqoEHZqEZ0R1VR4TIspmODx1Ln8/PVvsgaGw=}

Maven prints out an encrypted copy of the password to standard out. Copy this encrypted password and paste it into a ~/.m2/settings-security.xml file.

<settingsSecurity>
  <master>{rsB56BJcqoEHZqEZ0R1VR4TIspmODx1Ln8/PVvsgaGw=}</master>
</settingsSecurity>

After you have created a master password, you can then encrypt passwords for use in your Maven Settings. To encrypt a password with the master password, run mvn -ep or mvn –encrypt-password. Assume that you have a repository manager and you need to send a username of “deployment” and a password of “qualityFIRST”. To encrypt this particular password, you would run the following command:

$ mvn -ep qualityFIRST
{uMrbEOEf/VQHnc0W2X49Qab75j9LSTwiM3mg2LCrOzI=}

At this point, copy the encrypted password printed from the output of mvn -ep and paste it into your Maven Settings.

<settings>
  <servers>
    <server>
      <id>nexus</id>
      <username>deployment</username>
      <password>{uMrbEOEf/VQHnc0W2X49Qab75j9LSTwiM3mg2LCrOzI=}</password>
    </server>
  </servers>
  ...
</settings>

When you run a Maven build that needs to interact with the repository manager, Maven will retrieve the Master password from the ~/.m2/settings-security.xml file and use this master password to decrypt the password stored in your ~/.m2/settings.xml file. Maven will then send the decrypted password to the server.

What does this buy you? It allows you to avoid storing your passwords in ~/.m2/settings.xml as plain-text passwords providing you with the peace of mind that your critical passwords are not being stored, unprotected in a Maven Settings file. Note that this feature does not provide for encryption of the password while it is being sent to the remote server. An enterprising attacker could still capture the password using a network analysis tool.

For an extra level of security, you can encourage your developers to store the encrypted master password on a removable storage device like a USB hard drive. Using this method, a developer would plug a removable drive into a workstation when she wanted to perform a deployment or interact with a remote server. To support this, your ~/.m2/settings-security.xml file would contain a reference to the location of the settings-security.xml file using the relocation element.

<settingsSecurity>
  <relocation>/Volumes/usb-key/settings-security.xml</relocation>
</settingsSecurity>

The developer would then store the settings-security.xml file at /Volumes/usb-key/settings-security.xml which would only be available if the developer were sitting at the workstation.