MSTest And Working Copy Directory Follow
Hi,
Some of our MSTest unit tests search files contained within the project. To this end, the unit test walks up the directory tree starting with the current directory, looking for well known files within each directory being inspected. The tests run fine in the IDE and in CruiseControl, but not in TeamCity. TeamCity copies the test assemblies to an entirely unrelated directory, outside of the solution directory, causing the project directory lookup process to fail.
I realize that directory traversal from supposedly self contained assemblies may suggest a "procedure smell", but we did investigate alternatives.
For instance, we've considered the alternative of including the files being searched as assembly resources, but have chosen to avoid this if possible, and for good reason. The file being referenced by the MSTest unit test under consideration is itself a JavaScript unit test and as such it is a HTML file with satellite unit test files, all located within a web component and referencing additional web component files. To move the unit test around and try to make it work from within an assembly would not only prove a challenge, but its setup would require additional work for the developer for each new JavaScript unit test instance. The fewer moving parts in the process of writing JavaScript unit tests, the better.
Among the other alternatives, MSTest could get to the files if the build deploys the web application to IIS (possibly through an installer project); MSTest can then access these files directly from the web applications using a URI. Needless to say, this adds moving parts to the integration build, it increases the build overhead and brings about the problem of avoiding URI clashes for the collection of build configurations (in case some builds are triggered at the same time).
Other alternatives include various schemes for pulling the necessary files into the test environment. As long as these require intervention from the developer writing the unit test, we prefer to avoid them if possible.
To my questions.
1. Is it possible to force MSTest to run within the solution directory or a subdirectory thereof?
2. Does TeamCity provide MSTest with runtime information that includes the path to the working copy of the solution or project being tested?
3. Any other comments that you may find pertinent.
Thanks,
Mihai Danila
Please sign in to leave a comment.
My request was rather unorthodox, so I suspect no TeamCity support was planned, nor will any be planned anytime soon. As I already pointed out, one could deploy the web application that the C# unit test must access, and thus solve the problem.
But I found another solution. Instead of starting the lookup of the solution directory with
System.Environment.CurrentDirectory,
I start it with
new System.IO.File(new System.Diagnostics.StackFrame(true).GetFileName()).Directory.
While this requires the presence of debug information for the unit test assembly, it seems that the unit test does have debug information at hand even when built in the Release configuration. And even if it didn't, the solution's Release configuration can be instructed to build the unit tests in their Debug configuration, so as to forcibly include debug configuration. It would be some maintenance, but at an entirely different level.
Thanks,
Mihai
Hello,
How do you start mstest?
MSTest.exe process should be started in the working directory selected for sln200? build runner.
Could you please dump the location of the assembly from some of your tests my calling:
System.Console.Out.WriteLine(GetType().Assembly.CodeBase
Thanks!
I start MSTest as part of the build process, by configuring The sln2008 runner. I don't see a configuration option allowing me to control the current directory. (Do you have a MSTest command line argument in mind?) I assume TeamCity itself decides the current directory. I'm unsure what the current directory is, but it isn't a descendant of the solution directory (nor is it the solution directory itself). I suspect the current directory is some working folder of the TeamCity agent.
Mihai
To answer your question about the working and assembly directories, here's my test configuration.
Path to MSTest: %system.MSTest.9.0%
MSTest assemblies: a list of paths to test assemblies
MSTest metadata: blank
Testlist from metadata to run: blank
Test: blank
Results file: blank
Additional commandline parameters: blank
Output:
Assembly codebase: file:///C:/Program Files/JetBrains/TeamCity/v4.0/Server/buildAgent/temp/buildTmp/TeamCity_MACHINENAME 2009-03-05 22_23_56/Out/TESTASSEMBLYNAME.DLL
Current directory: C:\Program Files\JetBrains\TeamCity\v4.0\Server\buildAgent\temp\buildTmp\TeamCity_MACHINENAME 2009-03-05 22_23_56\Out
If I change the Results file to: C:\Temporary\results.trx, I get this instead:
Assembly codebase: file:///C:/Temporary/TeamCity_MACHINENAME 2009-03-05 21_33_06/Out/TESTASSEMBLYNAME.DLL
Current directory: C:\Temporary\TeamCity_MACHINENAME 2009-03-05 21_33_06\Out
There was an issue fixed for MSTest
http://www.jetbrains.net/tracker/issue2/TW-7376
Could you please check weather it fixes your issue.
Thanks!
I have a similar problem I believe:
*) We also use MSTest to execute our tests.
*) In our tests we use some files that we load with a relative path.
*) These files are checked out as a part of the build process
On the local machine these test pass.
On the build machine, the files are not found, causing an exception.
I added some output to our tests to diagnose the problem, similar to what you suggested earlier:
Console.WriteLine("CodeBase: " + GetType().Assembly.CodeBase);
Console.WriteLine("Working Directory: " + Directory.GetCurrentDirectory());
This is the output I get during the testrun:
CodeBase: file:///C:/TeamCity/buildAgent/temp/buildTmp/teamcity_DEV-WEB-BUILDER 2010-07-07 14_34_50/Out/Greentube.Gamebase.Web.UnitTest.DLL
Working Directory: C:\TeamCity\buildAgent\temp\buildTmp\teamcity_DEV-WEB-BUILDER 2010-07-07 14_34_50\Out
However the files we need to load are relative to the checkout directory, which is here (currently):
C:/TeamCity/buildAgent/work/4722fb29130b4cf1
Is there anything I can do to make MSTest run in the checkout directory?
Thanks,
Erik
What version of TeamCity do you use?
In TeamCity 5.1.x MSTest process is launched in build working directory.
There are several workaournds possilbe:
- set working directory somewhere in test setup base on assembly location path
- start MSTest runner from msbuild script with properly set directories. You still be able to report tests into TeamCity with use of
Hello Eugene
Thanks for your reply.
We're using TeamCity 5.1.2 (build 13430), which should be a relatively new version.
As you can see from my output in the previous post, it seems as if MSTest is NOT running in the build working directory.
I'm not sure if this is a bug or not. If you need further info form me, please let me know.
Furthermore, I really want MSTest to be started from the Checkout directory.
And finally, I found a workaround in the meantime. As has been stated in one of the previous posts, one seems to be able to influence the working dir of MSTest by changing the location of the results file.
So I set the location of the results file to:
%teamcity.build.workingDir%\UnitTest\results.trx
and now it works for me.
But still this feels like a hack!
Cheers,
Erik
Really strange.
Please try adding 'teamcity.agent.dotnet.debug' environment variable with value 'true' in the build propertyes and environment variables (6) section to make the runner dump generated msbuild script. Could MSTest.exe ignore set working directory?
Hi Eugene,
I've added the env. variable and now have such a dump, but I'm not sure where to look for problems.
I'm attaching the zipped version to this reply. Could you please take a look into it and tell me if things are looking right or wrong?
Thanks,
Erik
Attachment(s):
Gamebase.Web_Gamebase.Web_441.log.zip
This is how TeamCity starts mstest.exe for you (in MSBuild):
Please let me know if you see something strange in this code.
This behaviour (bug?) is still occurring in 6.5. I found a reference in the docs that says the buildTmp directory is used when teamcity launches a new process. My tests are using selenium to launch a browser so that makes sense although it is definitely a problem. Environment.CurrentDirectory should be the checkout directory but it is not.
I use a modified version of the previous poster's code to find the checkout directory for building relative paths. This returns what Environment.CurrentDirectory should return:
Path.GetDirectoryName(new System.Diagnostics.StackFrame(true).GetFileName())
As far as I know MSTest working directory (CurrentDirectory) is the directory where the report file is. To change the directory, specify the path to MSTest report file to point to a report file in the working directory you need.