Checkstyle vs PMD vs Findbugs


Share it!Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInEmail this to someoneShare on Reddit

Static code analysis is a process of verifying your code without executing it. It can be performed on source code level, on compiled binaries or even on some intermediate states. It can be automated as part of your build process (you can even fail the build in case of major findings) or can be done aside. Results can be stored as simple html pages produced by SCA tools or within dedicated systems (like for example SonarQube).

There are many reasons to use static code analysis within your project. The most common benefits include:

  • detecting potential bugs,
  • detecting introduced vulnerabilities,
  • pointing out duplicated code,
  • detecting design flaws,
  • enforcing coding standards,
  • detecting dead code,
  • measuring technical debt,
  • providing an overall view of the project health.

What is very important is that the checks are very fast due to no application execution. Feedback is usually provided almost instantly.

The most popular SCA tools in Java world are Checkstyle, Findbugs and PMD. What is interesting is that they are not interchangeable. Each of them works in a different way and is able to provide you varied verifications. Check out the below table for a detailed summary.

 

Checkstyle PMD Findbugs
Purpose Verifying compliance with coding rules. Identifying potential problems. Finding bugs.
Analysis check level Analyses pure source code without any major preprocessing (simple AST tree created by Checkstyle). As a consequence it is very fast. Analyses AST (Abstract Syntax Tree) generated by JavaCC. Does not require compilation. More advanced checks can be implemented, but still limited to one class. Analyses byte code. Requires compilation. As a consequence, rules are able to access information not only about the currently analysed class.
Types of verification – naming conventions,

– headers, imports,

– white spaces, formatting,

– javadoc comments,

– good practices, code conventions,

– method parameters,

– cyclomatic complexity,

– any regexp.

– possible bugs,

– dead code,

– duplicated code,

– cyclomatic complexity,

– overcomplicated expressions,

– and in fact everything that Checkstyle is capable of.

– possible bugs,

– design flaws.

– bad practices,

– multithreaded correctness,

– code vulnerabilities.

Writing custom rules – rules are highly customizable through XML files,

– written in Java, traversing through Checkstyle AST,

– pretty simple to write.

– xpath expressions defining incorrect AST state,

– written in Java, traversing through JavaCC AST,

– simple IDE (PMD Rule Designer) available,

– pretty simple to write.

– written in Java, traversing over compiled bytecode,

– requires good bytecode understanding, hard to write and maintain.

 

It is worth it to enable all of them on your project. However, your team has to commit so that they maintain the number of violations at zero. Keeping a small amount of them temporary will result in accepting this state and making such a situation permanent. Then the number of violations would slowly increase and you will finish with developers simply ignoring most of the new violations. However, there are some violations that are false positives or for some reason should be accepted permanently. To handle such situations, maintain an exclusion list and add to it new violations after accepting them by senior staff.

Some rules might make no sense to you. Good examples are the Checkstyle JavadocMethod rule that verifies if each method has Javadoc comment or the PMD OnlyOneReturn rule that verifies if a method has only one exit point, and that it is the last statement in the method. Whenever your team finds out such rule, discuss as a whole whether it is worth keeping it or dropping it. Many rules have some disadvantages and there is no sense in adjusting your code to them if the majority is against. Developers should identify themselves with the project and believe in it to achieve the best productivity. Doing things that they think doesn’t make sense will not motivate them.

There is an additional topic strictly related to SCA tools: how to integrate them with a development process. Generally, there are three major ways: detection during automated build, detection in your Continuous Integration system or in a dedicated system for measuring code quality, like for example SonarQube. All of them have pros and cons but this is a topic for a different post.

SCA rules can greatly benefit your project. There are even advocates of enabling and complying with all of them. They claim that each of them will bring more value than requires effort in a final calculation. Personally, I am not so strongly convinced to such a radical approach but still think that strict following of an agreed set can save you a lot of bugs, maintenance efforts and improve your design. Moreover, you will never be pissed off by a code not following a convention (yep, you can force the whole team to use the same indentation schema: no more tabs mixed with spaces!).

Share it!Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInEmail this to someoneShare on Reddit