Emails don't go out

Created October 29, 2019 04:42

Hi guys,

I have TeamCity email notifier configured to send emails on broken build.

For some reason no emails come through.

I tried:

* Enable notifiation debug logging
* Send test email - works fine

I also attach logs from teamcity-notifications.log, teamcity-server.log and teamcity-activities.log, it looks like it selects 2 users for notification, but no email goes out.

I asked our network engineer and he confirmed that there were no attempts to send email.

Can anyone please give me ideas what else to investigate.

Thanks,

Mike.

-----------

teamcity-notifications.log
[2019-10-29 14:56:42,164] DEBUG - areNotificationListener.jabber - Selected 0 users with matching notification rules for event BUILD_FINISHED_FAILURE in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,165] DEBUG - areNotificationListener.jabber - Selected 0 users with matching notification rules for event FIRST_FAILURE_AFTER_SUCCESS in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,165] DEBUG - areNotificationListener.jabber - Selected 0 users with matching notification rules for event NEW_BUILD_PROBLEM_OCCURRED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,168] DEBUG - cationListener.IDE_Notificator - Selected 0 users with matching notification rules for event BUILD_FINISHED_FAILURE in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,168] DEBUG - cationListener.IDE_Notificator - Selected 0 users with matching notification rules for event FIRST_FAILURE_AFTER_SUCCESS in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,174] DEBUG - cationListener.IDE_Notificator - Selected 1 users with matching notification rules for event NEW_BUILD_PROBLEM_OCCURRED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including); Usernames: kesavakumar.user1
[2019-10-29 14:56:42,208] DEBUG - areNotificationListener.jabber - Selected 0 users with matching notification rules for event BUILD_STARTED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,208] DEBUG - cationListener.IDE_Notificator - Selected 0 users with matching notification rules for event BUILD_STARTED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,209] DEBUG - wareNotificationListener.email - Selected 0 users with matching notification rules for event BUILD_STARTED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,210] DEBUG - tificationListener.WindowsTray - Selected 0 users with matching notification rules for event BUILD_STARTED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,469] DEBUG - wareNotificationListener.email - Selected 2 users with matching notification rules for event BUILD_FINISHED_FAILURE in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including); Usernames: kesavakumar.user1, mike.user2
[2019-10-29 14:56:42,476] DEBUG - wareNotificationListener.email - Selected 0 users with matching notification rules for event FIRST_FAILURE_AFTER_SUCCESS in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,477] DEBUG - wareNotificationListener.email - Selected 0 users with matching notification rules for event NEW_BUILD_PROBLEM_OCCURRED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,478] DEBUG - tificationListener.WindowsTray - Selected 0 users with matching notification rules for event BUILD_FINISHED_FAILURE in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,479] DEBUG - tificationListener.WindowsTray - Selected 0 users with matching notification rules for event FIRST_FAILURE_AFTER_SUCCESS in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)
[2019-10-29 14:56:42,480] DEBUG - tificationListener.WindowsTray - Selected 0 users with matching notification rules for event NEW_BUILD_PROBLEM_OCCURRED in build type CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} and project CI - master {id=Ci, internal id=project12} (including)

teamcity-server.log
[2019-10-29 14:56:42,102] INFO - jetbrains.buildServer.SERVER - Build finished received for build: #336 {build id=22019, buildTypeId=Ci_CompanyName}
[2019-10-29 14:56:42,104] INFO - jetbrains.buildServer.SERVER - Created finish build task for build: #336 {build id=22019, buildTypeId=Ci_CompanyName}, task id: 303
[2019-10-29 14:56:42,105] INFO - jetbrains.buildServer.SERVER - Build scheduled for finishing: #336 {build id=22019, buildTypeId=Ci_CompanyName}
[2019-10-29 14:56:42,110] INFO - ndexer.EventBasedBuildsIndexer - Indexing finishing build: #336 {build id=22019, buildTypeId=Ci_CompanyName}
[2019-10-29 14:56:42,159] INFO - de.impl.history.DBBuildHistory - Created build history entry; build id: 22019, finish date: 2019-10-29 14:56:42.088
[2019-10-29 14:56:42,172] INFO - tbrains.buildServer.ACTIVITIES - Finished build CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} #336 {promotion id=22019, branch=<default>, agent="DEV-X" {id=1}, triggered by "Git" (##vcsName='jetbrains.git' type='vcs' queueMergingEnabled='false'). Started 2019-10-29 13:20:40.933, finished. Status "FAILURE 'Exit code 3 (Step: Command Line) (new)'"}
[2019-10-29 14:56:42,197] INFO - jetbrains.buildServer.SERVER - Running build saved to DB: build id=22020, promotion id=22020
[2019-10-29 14:56:42,198] INFO - tbrains.buildServer.ACTIVITIES - Build started; CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} #337 {promotion id=22020, branch=<default>, agent="DEV-X" {id=1}, triggered by "Git" (##vcsName='jetbrains.git' type='vcs' queueMergingEnabled='false'). Started 2019-10-29 14:56:42.166, running}
[2019-10-29 14:56:42,199] INFO - jetbrains.buildServer.SERVER - Build removed from queue (started): 22020
[2019-10-29 14:56:42,204] INFO - jetbrains.buildServer.SERVER - Flush queue finished in 43ms, number of builds started: 1

teamcity-activities.log
[2019-10-29 14:56:42,172] INFO - tbrains.buildServer.ACTIVITIES - Finished build CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} #336 {promotion id=22019, branch=<default>, agent="DEV-X" {id=1}, triggered by "Git" (##vcsName='jetbrains.git' type='vcs' queueMergingEnabled='false'). Started 2019-10-29 13:20:40.933, finished. Status "FAILURE 'Exit code 3 (Step: Command Line) (new)'"}
[2019-10-29 14:56:42,198] INFO - tbrains.buildServer.ACTIVITIES - Build started; CI - master / CompanyName {id=Ci_CompanyName, internal id=bt83} #337 {promotion id=22020, branch=<default>, agent="DEV-X" {id=1}, triggered by "Git" (##vcsName='jetbrains.git' type='vcs' queueMergingEnabled='false'). Started 2019-10-29 14:56:42.166, running}

6 comments

Anatoly Cherenkov

Created November 01, 2019 09:48

Hi Mike,

Please check all email notification rules configured for an affected user. If there is more than one rule, the notification will only be sent for the first matching rule.

-Anatoly

Mike Kramarenko

Created November 03, 2019 21:58

Anatoly,

No emails get sent whatsoever.
There is only one email notification rule.

I'm more interested in ways to daignose this problem via looking at logs or similar.

Thanks,
Mike.

Mike Kramarenko

Created November 03, 2019 22:11

Another possible solution to my problem: if there was a way to get list of emails that were involved in broken build and pass that list to command line as a parameter, I could take care of emailing myself. So the question is: is there such a way?

Anatoly Cherenkov

Created November 08, 2019 11:59

Mike,

TeamCity cannot return the list of emails. To troubleshoot it further, you can do the following:

First of all, check your spam folder.
Check if any of the groups, to which the affected users belong, redefine the notification rule.
Add an internal property

teamcity.notification.logPerUserFiltering=true

to log more debug messages related to filtering users for sending notifications.
Additionally, make sure you are using the recent version of TeamCity.

Mike Kramarenko

Created November 11, 2019 02:10

Anatoly, email notifier was disabled, as you can see on one of the screenshots. I did not notice that! All good now.

Anatoly Cherenkov

Created November 11, 2019 07:11

Hello Mike,

My bad. I am glad you have spotted it.

Please sign in to leave a comment.