Build Failures Condition for unit test build runner

Answered

I want to set a build failure condition for unit test cases build runner using NUNIT and DOTCOVER.

My requirement is, If any developer commits the code without test case, build should fail. Even they commit just 2 lines of code.

So I tried to set build failure using STATMEMET Coverage options, but it doesn’t work for some scenarios.

Rules I set, “Fail build based on metrics changes compare to last successful build , if percentage is dropped less than 0.001“  and 0.001 is just added 2 lines of code.

Above rule works perfectly fine for most of the scenarios, but below situation it doesn’t work

Example 1:

Build No:  #5

Status: Success

Statement Coverage: Covered Lines/ no of Lines => 65/268= 24.253 %

Developer 1 committed the code without adding any test cases, and TC automatically triggered the build

Now, Build No: #6

Status: Failed (because Statement coverage ratio is dropped by 0.179)

Statement Coverage Ratio: 65/270=24.074 % (Dropped by 0.179 hence build failed)

Now developer 1 is writing test cases mean time Developer 2 committed the code with more test cases and triggered the build( note : Developer 1 still writing test case not yet committed)

Now, Build No: #7

Status: Passed

Statement Coverage Ratio : 70/272 = 25.735 %

Here is the issue According to my failure condition, TC compared current Statement coverage ratio with last successful build Statement coverage ratio ( BUILD NO  # 5 – 24.253 <  BUILD NO #7 – 25.735) and got passed the current build even BUILD No 6 doesn’t have any test cases.

I don’t know How do I handle such a situation?, Do you know any different ways to achieve my requirement .

This kind of conditions are really more helpful to do some preliminary validation for TDD approach whether team is following or not.

 

Thanks for your patience .

 

 

 

 

 

 

6 comments

Hello Balamurugan,

Sorry for delay. 

You can configure the following failure condition "Fail build if percentage of line coverage compared to constant value is less than 99 default units for this metrics". In this case build will fail if less than 99% of lines are covered. Is it what you are looking for?

0

Thanks for you reply.

I am working in legacy application, So Current covered percentage is 35 %. If I go with constant values, weekly once I may have to revisit the CONSTANT Value which is once again manual process.  

If application is developed from scratch , Yes I can set the failure conditions which you mentioned above(99%)

Here I would like to set condition based on Last successful build(we are improving our TDD) and more over I dont much difference between Statement and Line Coverage percentage. 

I am looking something different.

.

 

0

Hello,

Sorry, I understood the original question incorrectly. Thank you for clarification.

At the moment it's not possible to set failure condition as you want. We have the related request: https://youtrack.jetbrains.com/issue/TW-36030, will it work for you? If yes, please vote for it.

0

Thanks Alina Mishina. I read the YouTrack, issue simillar. However I raised youTrack ticket. Once again thanks you very much for your reply 

0

Please sign in to leave a comment.