Force SVN basic authentication method
Hi,
We are using multiple TeamCity server for our various projects. they all access the same svn server and we are starting to see some problem with performance due to the amount of connection/requests from the buildserver.
One thing I noticed is, that everytime the svn server is polled for changes, the NTLM authentication is used first. Unfortunately that doesn't work on our svn server and a "authorization required" message is returned. I tried to disable NTLM by setting
-Dsvnkit.http.methods=Basic
for both the buildagent (in wrapper.conf) and the buildserver itself (using the config util). However it seems to ignore this setting and still uses NTLM first.
is there another way to enforce the authentication method for svn over http?
Cheers,
Jörg
Please sign in to leave a comment.
Actually, TeamCity prefers basic authentication over NTLM. See comments in TW-14381 issue.
What version do you use? Server- or agent-side checkout mode?
We are currently using Teamcity 6.0.1 (it will be upgraded to 6.5 in the next weeks). The checkout method is agent side checkout.
I also have the logs for svn here which clearly state that NTLM is used:
This is already with the authentication method explicitly specified for both server and agent. So it doesn't seem to prefer Basic over NTLM
Hello,
From the log I see that server requests NTLM or Basic authentication:
And TeamCity sends Basic authentication in response:
May be, there is some other problem? For instance, SVN server doesn't accept Basic authentication?
Or I'm missing something?
Regards,
KIR
The log I posted is from the teamcity server not from the svn server.
that means:
1. teamcity send request using NTLM
2. svn server responds "Authorization required"
3. teamcity send request using Basic authentication.
This is atleast how I interpret the log file. Unless the first request from teamcity is simply to request the available authentication methods.
If that's the case, is there way to prevent that? We already know that we want to use basic and there is no need to request it again.
Cheers,
Jörg
Hello Joerg,
The first request comes without any authorization - I've got the confirmation of this from the author of SVNkit library which we use to access subversion servers.
Regarding high load on the subversion servers - did you consider increasing checking for changes interval?
If you have ~100 of VCS roots, we consider setting checking for changes interval to ~120 seconds.
Hope this helps,
KIR
Hi Kir,
Ok. if there is now way around the initial connection without authorization, we will have to live with it.
We don't want to increase the interval since we want to have a quick response to our checkins and trigger a compile. We did however increase the time for some other not so critical projects.
There is one more thing I noticed which might be related to this issues. For each VCS root, I'm seeing more than just one request every 60 seconds.
Here is a log from our svn server showing the request sent for the same VCS root.
builduser [25/Jul/2011:08:16:49 +0200] "PROPFIND /svn/Project/trunk HTTP/1.1" 207 702
the teamcity server is only using on vcs root to point to this svn repository. the vcs root is shared between 26 projects where only 22 of them actually have build triggers (1 vcs trigger and one schedule trigger). Can you tell why I'm seeing more than one request for the same vcs root? I couldn't find a relation between number of project, triggers etc. to the number of requests to the svn server yet. If the requests could be reduced to 1 request instead of 4, that would cut the load quite dramatically.
Cheers,
Jörg
Hello Jörg,
If you use one VCS root and many combinations of VCS root + checkout rules, please consider adding a "fake" build configuration which would collect all changes from the VCS root.
When collecting changes, Teamcity collects changes by all combinations of VCS root + include checkout rule (but if you have include rule for 'dir1' and for 'dir1/dir2', it will make request for 'dir1' only).
So, if you add a fake build configuration with enclosing rule, Teamcity will make request for changes only once for the VCS root.
Another point, if you want faster detection of changes in VCS root, you may consider trying this plugin: http://code.google.com/p/vcsupdate/
Hope, this helps,
KIR
Hi Kir,
The plugin definitely looks very interesting and I'll try it out.
regarding your suggestion of a "fake" build configuration. We are not using checkout rules since we only do incremental updates (using swabra and svn revert to clean the checkout directory). Therefore the scenario you described below is not applicable. We are however using VCS build trigger (different for each configuration), would that cause a similar behaviour?
Cheers,
Jörg
Hi Jörg,
If you're not using checkout rules, do you checkout the same source tree for each of your build configurations?
In this case, Teamcity should collect changes only once for the VCS root, regardless of number of build configurations.
VCS triggers do not affect how Teamcity detects changes internally.
Regards,
KIR
Hi Kir,
yes, we are checking out the same source tree and we are using the same checkout directory (explicitly specified). So all 20 configuration are using the same vcs root and checkout to "C:\ProjectCheckoutDir". The checkout is done on the agent, which is running on the same machine as the teamcity server.
Cheers,
Jörg
Hi Jörg,
Your configuration is rather simple, and actually, should work without any problems.
Multiple requests to subverson server can be caused by externals used by your project, but still, these requests shouldn't affect your server so much.
To research the problem in more detail, we need detailed logs from your TeamCity installation, but first, please consider updating your TeamCity to the latest version (6.5.3 was released just recently).
Each major update of TeamCity includes some enhancements in SVN support, including performance improvements.
Kind regards,
KIR
Hi Kir,
We are planning to update the teamcity server middle of this week. I'll let you know how the performance is afterwards.
We are not using any external in our vcs roots. so this is not the cause of the additional requests.
In case the problem isn't solve with the new version of temacity. which logs do you need exactly?
Cheers,
Jörg
Hello Jörg,
We'll need logs from TeamCity/logs directory, namely teamcity-vcs.log (with enabled debuging).
Regards,
KIR
Hi,
It looks like we may be having a smiliar issue with out TeamCity and Subversion integration. When attempting to click test connection from the 'New VCS Root' in Team City we are getting the following error:
svn: Authentication required for '<http://someserver:80> SVN service'
Here are the contents of teamcity-svn.log
[2011-08-11 11:28:31,461] DEBUG - javasvn.output - NETWORK: Keep-Alive timeout detected
[2011-08-11 11:28:31,586] DEBUG - javasvn.output - NETWORK: SENT
OPTIONS /repository HTTP/1.1
Host: someserver
User-Agent: SVN/1.6.15 SVNKit/1.3.5 (http://svnkit.com/) r7573
Keep-Alive:
Connection: TE, Keep-Alive
TE: trailers
Content-Length: 0
Accept-Encoding: gzip
Content-Type: text/xml; charset="utf-8"
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
[2011-08-11 11:28:31,633] DEBUG - javasvn.output - NETWORK: READ
HTTP/1.1 401 Authorization Required
Date: Thu, 11 Aug 2011 18:28:31 GMT
Server: Apache/2.2.3 (Red Hat)
WWW-Authenticate: Digest realm="SVN service", nonce="some text", algorithm=MD5, domain="/repository/", qop="auth"
Content-Length: 483
Connection: close
Content-Type: text/html; charset=iso-8859-1
[2011-08-11 11:28:31,711] DEBUG - javasvn.output - NETWORK: SENT
OPTIONS /repository HTTP/1.1
Host: someserver
User-Agent: SVN/1.6.15 SVNKit/1.3.5 (http://svnkit.com/) r7573
Keep-Alive:
Connection: TE, Keep-Alive
TE: trailers
Content-Length: 0
Accept-Encoding: gzip
Content-Type: text/xml; charset="utf-8"
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
[2011-08-11 11:28:31,773] DEBUG - javasvn.output - NETWORK: READ
HTTP/1.1 401 Authorization Required
Date: Thu, 11 Aug 2011 18:28:31 GMT
Server: Apache/2.2.3 (Red Hat)
WWW-Authenticate: Digest realm="SVN service", nonce="some text", algorithm=MD5, domain="/repository/", qop="auth"
Content-Length: 483
Connection: close
Content-Type: text/html; charset=iso-8859-1
[2011-08-11 11:28:31,773] DEBUG - javasvn.output - NETWORK: svn: Authentication required for '<http://someserver:80> SVN service'
I attempted to modify the settings as described in this article but with no luck http://confluence.jetbrains.net/display/TCD6/Known+Issues#KnownIssues-SubversionRepositoriesWithNTLMAuthorization
Any further suggestions? I did notice in the teamcity-svn.log file this User-Agent: SVN/1.6.15 SVNKit/1.3.5 (http://svnkit.com/) r7573. We are on Subversion 1.4.2 .. could that be causing any issues?
Hello Jesse,
Your server requires Digest/MD5 authentication, and it is possible that SVNKit is not relyable when working with this type of authentication.
I'd reccomend enabling basic authentication in your server, via mod_auth_sspi Apache module and option "SSPIOfferBasic on".
Hope this helps,
KIR