Previously muted tests got removed from list of muted tests without any reason. Tests still fail.
Dear TeamCity Support,
Could you please let me know if there is a known issue for muted tests to get accidentally unmuted by TeamCity?
I observe this issue from time to time.
Last observation: tests were muted, for particular build only, with option to unmute "Manually".
Tests were not changed, were not renamed and location of them remained the same.
Tests got unmuted without any particular reason.
Is there a log to check what is the reason for unmuting a test?
Thanks a lot!
Please sign in to leave a comment.
Hi Andriy,
Which version are you running? There are several possibilities, such as a test passing and then failing again (a flaky test) might unmute it, the investigations/muting of tests being marked as fixed manually, or set to be removed on a specific date. You can check the Administration - Audit section to see whether some of that happened manually.
The "teamcity-activities.log" does contain some information, prefixed with "test_unmute" about when a test was unmuted. If it was done manually, it will say who was responsible, if it happened automatically it will just say it was unmuted.
Hi Denis!
Thanks a lot for your reply.
We use TeamCity Enterprise 9.1.6 (build 37459).
I checked Administration - Audit section and could not find any record of unmuting my tests.
I looked at test history though and it looks that now those unmuted tests would show no history older than day when they got unmuted.
However, if I open old runs and find those tests they would show full history including execution records before and after day when they got unmuted.
I looked at the code and don't see any duplicate tests (sharing same name).
So, from my understanding, for some reason TeamCity started considering those tests as new ones.
I checked full test path (location), names, nothing was changed.
1. Is there a possibility for this scenario when TeamCity unmutes tests since for some reason it considers those test never existed before?
I would expect that to happen only if test's path/name gets changed.
2. Also, regarding "a flaky test", if I mute a test with option to unmute "Manually" I don't expect test to get automatically unmuted ever (even if it passes, or passes and fails, or just fails). Is my understanding correct?
Thanks a lot for your kind help,
Andriy
Hi Andriy,
Thanks for the answer. First, for some reason I totally missed the bit where you said you already set it to manually, so sorry about that. Yes, marked as unmute manually should wait until it's manually unmuted independently of the result.
Second, it seems that for some reason it's being detected as a new test, even if it shares name and path. If that's the case, it wouldn't be unmuted, it simply would be a new instance of the test that is not muted by default. Could you check whether the old tests are still muted? Or were they unmuted?
Would it be possible for you to check whether the same behavior reproduces for you in a newer version? Several fixes were added to test management in the meantime, so it's not unlikely this was one of them.
Regarding the audit log, it's likely that this feature was introduced in newer releases. 9.1.6 is not supported anymore, so we strongly encourage you to upgrade your installation. If we would find this to be a bug, it won't get fixed in 9.1.6 and we'll need to replicate it in newer releases before we can fix it.
Hi Denis!
Thanks a lot for your response.
I'll talk to our DevOps team to get our version updated.
---
P.S. In the build run tests got unmuted and got reported as failed.
So I muted them again in build 381 (see screenshot below).