At my job, we are using TeamCity to run its automated testing on our product. I have a very strange problem I've been unable to resolve so far.
One of the test suites we're running is used to measure performance of our product. We basically run several commands and time how long it takes them to complete (with other things worked in to make sure the times are more or less consistent).
This is working just fine. However the problem is that, for whatever reason, running these tests through TeamCity consistently results in FASTER TIMES than if we launch the tests manually.
Now this would normally be a great thing, however it is a problem in part because it screws up our numbers. We want our numbers to reflect user experience. It also complicates matters for debugging.
We have a dedicated host machine (or "agent" in TC terms) which is set to exclusively run these performance tests (details below). It looks like TeamCity is running the tests under a headless user mode which I suspect might be why there is a performance improvement. I wonder if it ran the tests as a specific user if that would make a difference.
My questions are:
1) Is there any way to have TeamCity run its builds as a given user? If so, how?
2) Is it likely that doing that would resolve my issues with differing performance numbers? If not, then does anyone have any other ideas on what can be done about this?
I appreciate any help you can offer. This one has been driving me crazy for a while now.
TeamCity Enterprise 7.1 (build 23907)
Host Machine / Agent running:
Windows Server 2008 R2 Entrprise w/ SP1 64-bit
Intel i7-3770K (8 threads @ 3.5 GHz)
16 GB RAM