Maven performance testing


Share it!Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInEmail this to someoneShare on Reddit
Maven_performance

Picture by Roger W, on Creative Commons

I was recently wondering how the number of Maven modules within a multi module project affects the overall build performance. Is there a big difference between having 200 modules with 20 – 50 classes within each and 40 modules having 100 – 250 classes? Or maybe it is worth it to compress them to 10 modules having 400 – 1000 classes to have an outstanding compilation performance? As rebuilding a project is one of the most common developer activities, a slight time improvement may provide great value. If you are not sure, just multiply:

60 developers * 2 minutes per day = 120 minutes per day = 2 hours per day = 64 man days per year!

To test Maven performance I have decided to create three projects: one module, 100 modules and 1000 modules. I wanted to see how much Maven performance degrades with a significant increase in the number of modules. For each project I have executed several runs: building an empty project, building a project with 1100 classes, with 11000 classes and with 26000 classes (classes each time were spread evenly across modules).

Classes used in the test were more or less generated automatically by a bash script. As a base I have used 11 classes composing of a very simple JEE calendar application. They were using Hibernate, Spring core and several Spring modules. Bash script was simply copying all 11 classes, renaming them by adding consecutive numbers to the end of a class name and adjusting class names inside the code.

I have used a typical Linux desktop machine for the test:

  • Intel Core i5-2500K CPU @ 3.30GHz (4 cores),
  • Samsung F3 HD103SJ 1TB 7200,
  • Ubuntu 12.04 LTS.

Let’s move on to the results. I have grabbed my locally installed Maven 3.2.2 version, JDK 1.7 and… I was not believing the results (s – seconds):

 

0 classes 1 100 classes 11 000 classes 26 000 classes
one module project 1 s 4.5 s 15 s 27 s
100 modules project 4 s 10 s 20 s 27 s
1000 modules project 14 s N/A 55 s 66 s

It was fully against my experience and expectations. While increased time on an empty project from 1 second to 14 seconds was quite a usual thing, an increase from 27 seconds to 66 seconds on 26000 classes project was unbelievable. I would expect at least 5 to 10 times decreased performance in the case of the 1000 modules project.

I have searched my memories and realized that the last time I was running a build on a large code base it was required to use Maven version 3.0.5. I was not hesitating long and executed all project setups with this version (and the same JDK 1.7 as before). Time differences were astonishing (s – seconds, m – minutes):

 

0 classes 1 100 classes 11 000 classes 26 000 classes
one module project 1 s 4.5 s 15 s 27 s
100 modules project 4 s 24 s 2 m 3 m
1000 modules project 14 s N/A 10 m 12 m

This time, one module project build for 26000 classes was from 25 to 40 times faster than the 1000 modules project. But why was the performance so different between Maven 3.2.2 and 3.0.5?

I have dug further and found out two highly probable reasons for such major improvement:

  • performance improvements in maven-compiler-plugin 2.4 – it seems that this is the issue: Use plexus-compiler 1.8.6 for performance improvement,
  • performance improvements in Maven 3.1 – I was not able to identify the exact change, but upgrading only the compiler plugin brings partial improvement.

I have two major thoughts after this experiment: skip performance factor while determining the structure and amount of modules in your project. Build time changes are cosmetic rather than significant ones. Second is: upgrade used by you tooling to the newest versions. Check the exact cost and benefits of updating and act accordingly! We are constantly working in an environment where hundreds of critical issues are waiting to be handled. However, sometimes the benefits of upgrades are so huge that skipping them is just losing time and money.

Share it!Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInEmail this to someoneShare on Reddit