How to push changes back to source repo in git

Answered

I curious how one can push changes back to the git source repo.

Basically, the build changes a few source files which then need to be push backed to the source repo in git.

0
23 comments
Official comment
As this question is frequently asked, let me post some clarification on how TeamCity authenticates in Git VCS during checkout, and why just issuing the git push command from the build script may not be enough to push to the VCS.
 
TeamCity doesn't store the credentials from the VCS Roots in credential managers. When TeamCity performs a checkout, it always passes -c core.askpass=<temporary_filename_with_password_or_token> -c credential.helper= parameters. Example from a build log:
[E:\Git\bin\git.exe -c core.askpass=E:\teamcity-agent\temp\buildTmp\pass5505167349342718084.bat -c credential.helper= fetch --progress --recurse-submodules=no origin +refs/pull/163/head:refs/pull/163/head] Receiving objects: 68% (2317/3407)
Before the checkout, TeamCity temporarily stores the token from the VCS Root settings into a file under the buildTmp directory, for example, E:\teamcity-agent\temp\buildTmp\pass5505167349342718084.bat. This file only exists during the checkout stage and is removed as soon as the checkout is finished. The script simply contains the echo <TOKEN> command that is passed to Git's core.askpass parameter with every Git command. The second parameter, -c credential.helper=, ensures that credentials are not saved anywhere.
 
When a git command is invoked on the agent by the user's build script, Git will try using credential managers. The commands requiring credentials would work if the agent has the credentials stored in the credential manager. However, as mentioned earlier, TeamCity doesn't store them in a git credential manager. Although, If someone uses the credentials on the said agent without the -c credential.helper= option (in a build script or by manually running the command), they should be saved on the agent and available in further invocations of Git. Here are some options if you want to run Git commands in the build scripts:
  • Store credentials in a Git credential manager via something like git credential approve. This option is not very secure. If the agent is used by different teams or in other projects, there's a risk that someone from a different project might access your repository.
  • Store the credentials to the VCS Root as 'password' type parameters and use them in your scripts. Best combined with -c credential.helper= option so that Git doesn't store them physically on the machine.
  • Use SSH in the VCS Root settings and add the SSH Agent build feature to your build configuration with the same key that is selected for the corresponding VCS Root. This way, TeamCity will copy the specified key to the agent for the duration of the build and make sure that whenever the SSH Agent is called, e.g., by Git, it will use the pre-loaded key.

I hope this helps someone.

TeamCity does not provide a feature like that.
You can use agent-side checkout and then use command line VCS tools to perform the VCS operations within the build just like you would do in a command prompt on a local machine.
Depending on the changes, it might be good to revert the working copy to the state corresponding to the build start as TeamCity assumes no VCS changes are made during the build.

0

Thanks for the reply. Apologies for getting back so late.

I thought TC's auto merge feature was implemented for this scenario.

Is that true?

0

TeamCity automatic merge feature allows to merge a branch into another branch after successful build. This does not include ability for a build to change files before the merge occurrs.
However, if you rework your process to push the modified files into a branch, TeamCity can then be confiured to build and merge the changes on success.

0

Very good. Thank you for the answer. It helps quite a bit. God bless.

0

One more question, I am getting the following error when use a git push script during a commandline build step.

Any thoughts on this one?

Starting: E:\TeamCity\Temp\agentTmp\custom_script34180244.cmd
in directory: E:\TeamCity\Work\
\Git\bin\git.exe was unexpected at this time.
Process exited with code 255
Step Update Source (Command Line) failed
0

Seems like an error in the script you specified. One of the most common issues is not wrapping paths with space with quotes.

If you need more guidance on this, please include your current script.

0
0
Avatar
Ricardo Rodrigues

Hi. I just saw this, it's exactly what I need to do, I've tried many different ways but I always get errors when trying to push back.

Is this included in the latest teamcity or if not can you help with a workaround?

Thank you.

0

Hi Ricardo,

 

There were several issue in this thread, please describe the one you have in detail.

0
Avatar
Ricardo Rodrigues

Basically I want to use git to push back to origin within a build step. Been having authentication issues or can't find the repo, etc

0

Ricardo,

So as I mentioned in my first reply you should use agent-side checkout and then call git commands from within a build step (just like you would do from a command line). TeamCity does not seem to blame here and you just need to figure out due git command.

Depending on the VS root settings used it might be not possible to reuse repository authentication, so you probably should supply authentication anew to the git command.

0
Avatar
Ricardo Rodrigues

I'm using HTTPS auth and I kept getting these types of errors:

[11:18:49][Step 1/1] bash: /dev/tty: No such device or address
[11:18:49][Step 1/1] error: failed to execute prompt script (exit code 1)

 

When using git directly on the agent in the command line it works, same exact commands, so it's definitely TeamCity related.

1

Ricardo, I still believe this is more related to the environment/generic OS logic and not exactly TeamCity-introduced behavior.

Please check the related topic from our documentation to try to resolve your "script works differently" case.

0
Avatar
Ricardo Rodrigues

Hi.
I'm already using a regular user and everything I execute with that user works - just not under the TeamCity build. So I have tried a few things to try to make it work - nothing except executing the same exact thing while logged into the machine, worked.

I'll have a look at the other suggestions and let you know.

0

Yegor, could you please elaborate on this comment
"Depending on the VS root settings used it might be not possible to reuse repository authentication, so you probably should supply authentication anew to the git command."

What VCS root setting will enable / disable reusing the repo auth?

I always get "fatal: could not read Password for 'https://user@github-enterprise.company.com': No such device or address'

when I try to do git push from teamcity commandline runner

 

thx

Scott

0

Hi Scott,

I guess what you experience is related not to TeamCity per se, but to the headless execution of the process and Git client specifics.

On the Git authentication: "Default private key" is not much related to TeamCity as it depends on the ssh key saved as default for the OS user. Username/password should also be possible to feed to the git client, e.g. via git-credential ( https://git-scm.com/docs/git-credential ).

0

Hi,

I was getting same error as Ricardo and I kept getting this:

[07:31:30][Step 3/3] bash: /dev/tty: No such device or address
[07:31:30][Step 3/3] error: failed to execute prompt script (exit code 1)
[07:31:30][Step 3/3] stty: 'standard input': Inappropriate ioctl for device
 
solution that worked for me was to set agent to logon as specific user instead of LocalService account
0

Hi,

I'm getting an issue when I'm trying to commit changes to the git repository. I have a build configuration that could modify some files depending on certain conditions in the previous build steps. In the last step of this configuration, I have a build step for committing the changes and pushing them to the remote repositories.

The issue is that this last step remains in a running state indefinitely. This last step is a Powershell script that works outside of the TeamCity environment. Could you shed some light on what's going on?

$diff = Invoke-Expression "git diff"
if ($diff.Count -gt 0) {
Invoke-Expression "git config user.name ""TeamCity"""
Invoke-Expression "git config user.email ""w.yepez@example.com"""
Invoke-Expression "git add ."
Invoke-Expression "git commit -m v%Version%"
Invoke-Expression "git push origin master"
}
0

Hi Wyepezy

I am also in process of implementing similar requirement, commit/push changes of git via TC builds

where I got ended up with same situation.

Can I know did you get any clue on this issue with TC build running indefinitely.

Request to you please share info if any... which will save lots of time for me as well :)

Thanks in advance.

 

Thanks & Regards,

Rajkumar Uppala

 

0

Hello together, i had the same Problem..

My solution was for now i've set the push-URL in the settings of the VCS-Root to

https://username:password@myurl/git/myproject/myrepo

The "git push" command in simple command-line works (windows-server).

I'll replace username/password with system variables which i can configure at root-level

Maybe its not the best /securest way, but at my system i'm the only user. 

Since you have more administrator, you should check if this a good solution for you to share this credentials.

maybe you could try this per SSH with creating SSH-Key and giving username to URL. Maybe you have to add the SSH-Key additional to clients .ssh-Folder. Another solution could be to save credentials for git in agent directly per git config credential store (google it).

The last impressions are just untested solution hints. Maybe someone has tested, could post here 

0

Anatoly Cherenkov I'm trying to follow the pattern you describe for how TeamCity handles this scenario in a script but have so far been unsuccessful… Sorry if this is more of a git question than a TeamCity one.
I'm trying to manually recreate the scenario on the build agent by creating a batch file with contents:
echo <Github Personal Access Token>

then from a powershell console running 
git -c core.askpass=<path_to_batch_file> -c credential.helper= <git action here>
This however only gets me an error: remote: Invalid username or password.
if just try to run a normal git command i.e. git ls-remote I get a prompt for credentials as would be expected, since this is GitHub I get an interactive prompt, choose token from the popup then paste the same token I'm using in the batch script, and the command succeeds (then I go and remove the credentials from the credential manager).

EDIT: Came across the following SO article that I was able to successfully reproduce on the build agent… will adapt this to my script
bash - keep git auth as environment variable instead of evaluated value - Stack Overflow
would still be interested in understanding why passing from a file would fail… I tried altering the batch file to echo both a username and password as in the linked article but this also did not work. 

0

Check if you disabled echo in your bat file. Otherwise, not only will the token be echoed, but the command as well. Git binary expects just the password.

This PowerShell step works for me on Windows:

$ErrorActionPreference = "Stop"
$owner = "owner"
$repo = "repo"
$fetchUrl = "https://github.com/$owner/$repo.git"

$ghCredsScriptName = "gh_creds.bat"
Write-Output @"
@echo off
echo %%GH_TOKEN%%
"@ > $ghCredsScriptName

git -c core.askpass=$ghCredsScriptName -c credential.helper= ls-remote $fetchUrl

The build configuration parameter env.GH_TOKEN with the type password must be set.
%%GH_TOKEN%% is replaced with %GH_TOKEN% on the agent.

0

Please sign in to leave a comment.