Versioned Settings Becomes Out of Sync Without Changes

Completed
Avatar
Permanently deleted user
Created

I recently migrated a number of projects to export builds as Kotlin DSL into the repo directly. This is on a 2018.1 TC instance.

We have branch protections in place on the repo, so TC is unable to write changes from itself back to the repository. However, I have enabled `teamcity.ui.settings.readOnly` in order to prevent users from making changes outside of source.

Unfortunately, we keep seeing issues where TC seems to want to make a change, can't, and prevents pulling in any further changes from the repo. This also seems to be affecting any running builds, where they do not pull in the settings on their branch.

When I've seen this, the only fix appears to be to disable and re-enable Versioned Settings (and telling it to pull in from the repo). Initially, I believed this to be because we are changing a parameter on the build every run (one of the steps writes back via REST). But I have updated the script to remove this param change.

Comparing the version In-Repo to that from the Kotlin download on the project, the only differences i see (beyond the changes in the skipped commit) is moving around of parameters (or sometimes build steps).

NB: We do still actually have a call remaining that updates the build, but it sets the build counter variable, which as far as I can see does not reside within the Kotlin files.

 

Is there some way to stop this from happening? or at least some way I can see exactly what is causing it to try and write back to the repo?

Votes

1

Share

2 comments
Avatar
Mikhail Efremov
Created

Hi Fergal,

You can try enabling debug-all logging preset in Administration->Diagnostics to enable debug logging. Likely, the best place to look for the info is teamcity-versioned-settings.log. For the sake of examples, I made a couple of changes to a versioned project from the UI. Corresponding log records look like:

[2019-12-11 16:36:46,246] INFO - dCommitPerSettingsRootStrategy - Transaction 7 (new): settings change from UI (patches/projects/_Self.kts), cause: {id: 12, user: 'mikhail.efremov' (Mikhail Efremov) {id=1}, description: project parameters were changed}
[2019-12-11 16:36:46,247] DEBUG - .DirectoryBasedTransactionImpl - Project settings change, transaction: {id: 7}, operation: persist patches/projects/_Self.kts
[2019-12-11 16:36:46,250] INFO - dCommitPerSettingsRootStrategy - Transaction 7: settings change from UI (rm patches/vcsRoots/HttpsGithubComDy1ngTcTestProjectGit.kts), cause: {id: 12, user: 'mikhail.efremov' (Mikhail Efremov) {id=1}, description: project parameters were changed}
[2019-12-11 16:36:46,250] DEBUG - .DirectoryBasedTransactionImpl - Project settings change, transaction: {id: 7}, operation: delete patches/vcsRoots/HttpsGithubComDy1ngTcTestProjectGit.kts
[2019-12-11 16:36:46,253] INFO - dCommitPerSettingsRootStrategy - Transaction 7: settings change from UI (rm patches/buildTypes/TheBuildIsNigh.kts), cause: {id: 12, user: 'mikhail.efremov' (Mikhail Efremov) {id=1}, description: project parameters were changed}
[2019-12-11 16:36:46,253] DEBUG - .DirectoryBasedTransactionImpl - Project settings change, transaction: {id: 7}, operation: delete patches/buildTypes/TheBuildIsNigh.kts
[2019-12-11 16:36:46,256] INFO - dCommitPerSettingsRootStrategy - Transaction 7: settings change from UI (rm patches/buildTypes/TheBuildIsNigh2.kts), cause: {id: 12, user: 'mikhail.efremov' (Mikhail Efremov) {id=1}, description: project parameters were changed}
[2019-12-11 16:36:46,256] DEBUG - .DirectoryBasedTransactionImpl - Project settings change, transaction: {id: 7}, operation: delete patches/buildTypes/TheBuildIsNigh2.kts
[2019-12-11 16:36:56,311] DEBUG - dCommitPerSettingsRootStrategy - Transaction 7: run scheduled commit
[2019-12-11 16:36:56,313] INFO - dCommitPerSettingsRootStrategy - Transaction 7: start committing settings changes to VCS root: "https://github.com/dy1ng/TC-Test-Versioned.git" {instance id=8, parent internal id=11, parent id=VersionedProject_HttpsGithubComDy1ngTcTestVersionedGit, description: "https://github.com/dy1ng/TC-Test-Versioned.git#refs/heads/master"}
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Delete file .teamcity/patches/buildTypes/TheBuildIsNigh2.kts
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Delete file .teamcity/patches/vcsRoots/HttpsGithubComDy1ngTcTestProjectGit.kts
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Delete file .teamcity/patches/buildTypes/TheBuildIsNigh.kts
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Create directory .teamcity
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Create directory .teamcity/patches
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Create directory .teamcity/patches/projects
[2019-12-11 16:36:56,314] DEBUG - ctoryBasedTransactionCommitter - Create file .teamcity/patches/projects/_Self.kts
[2019-12-11 16:37:01,197] INFO - dCommitPerSettingsRootStrategy - Transaction 7: committed successfully, created revision: ed869a4a1e506092cc648697dec7b29a8872cfcc

These logs may hint on what the server is trying to commit. If fruitless, teamcity-server.log will be the next place to search for the info. 

 

Best regards,

Mikhail Efremov

0
Avatar
Permanently deleted user
Created

Hi Mikhail,

 

So, apparently I was too hasty. The problem was that parameter change that I removed. However, there were either builds in-flight with the old template, or the version settings were already out of sync when I applied it.

Since I've toggled the setting off/on again, I don't appear to have had any further issues (yet) with the versioned settings erroring out.

Cheers for the help.

Fergal

0

Please sign in to leave a comment.