Why TC run different number of unit tests?
Hi! In our android project, as a part of CI flow we run unit test JUnit4 with gradle runner.
But always number of launched test is different, why?
Actual number of test is not changed - 490
Please sign in to leave a comment.
Hi,
the tests are usually reported by the runner itself, and in some cases (such as android's UI tests) you might be loading them externally via the XML Report processing tool. So it'd be important to know whether you are using it or not. You can also check the tests themselves by accessing the results page for the build and going to the tests tab. Please ensure that all tests without grouping are being shown.
Next step would be to check whether the missing tests are being run by the gradle runner, for that, you can go into the build log and verify whether some set of logs are missing. You can check two different build logs and see whether there are major differences in what gradle reports through the different builds. We use this feature quite extensively internally, test it and a large amount of users have their tests reported so it's unlikely there is an issue in teamcity itself.
If you find differences, you might need to check why gradle would report different tests (maybe results caching, not running tests that haven't been modified, etc). If you don't, please send the build logs of those 2 builds to us for review using the "Submit a request" button on top of the page.