There is an increasing drive to use tools, such as Jenkins, to help automate post-build tests in continuous integration (CI) pipelines. The overall idea is to improve software quality through the use of dynamic error detection and diagnosis.    APM tools such as AppDynamics or a software quality tools such as OverOps can act as a gatekeeper, passing or failing tests based upon the runtime errors detected. For example, a null pointer exception could be an immediate fail. Other errors might be allowed within a certain tolerance or budget. However, no amount of tooling can make up for weak or superficial tests that allow builds to pass. The pathological case is where the post-build tests are a no-op. You could imagine that there might be the temptation for a software developer (individual or organisation) to skip tests in order to ship code more quickly. More commonly, there is a tendency for developers to write test cases that are kind to their code, testing the normal flow of execution and avoiding the edge cases. A code coverage tool can help to flag insufficient testing but they don’t tend to diagnose runtime errors. Ultimately, there is no substitute for reasoning about your software. Often it is better to get someone else to reason about it for you: they will be less kind.   At some stage further down the CI/CD pipeline, possibly in production, any lack of testing will reemerge as incidents and bugs. At this stage, it will be more expensive to diagnose, reproduce and fix them, possibly with unhappy customers thrown into the mix. It is important to learn the lessons and not only fix the problem but create test cases that demonstrate they have been fixed and do not reoccur. Being able to use the same tools in production is the best way to ensure that you know all the weird and wonderful ways your application will be used  (or abused) in the wild. This feedback loop is the second important way you can improve the quality of your test cases. The good news is that fixing one specific bug tends to have a beneficial effect on a whole category of bugs. The bad news is that occasionally it has unintended consequences where other modules depend on the incorrect behaviour. Sometimes pragmatism is required.   If you would like to continue the discussion, then just add a comment, or if you prefer, please just contact us hereand we can get in touch.