Data Driven Test Reporting Issue

I have a project that is using MSTest to perform data driven tests. At the conclusion of my build, TeamCity reports that I have successfully run 22 tests. But in the full log MSTest shows that it has run 242 tests. This makes sense because MSTest treats each row of data in a data driven test as a separate test. Is it possible to get TeamCity to report each test run instead of each physical test?

7 comments
Comment actions Permalink

Could you please attach an .trx file. What VS version was it?

0
Comment actions Permalink

Sorry, I should have given you more details.

I am using Visual Studio 2008 SP1. My Team City Project is setup with the sln2008 build runner. I am using the standard path to MSTest and running a single test assembly.

I have attached the trx file.

It appears that TeamCity is counting <UnitTestResult> entries and checking the  outcome attribute. This works for most tests, but data driven tests include a <Counters> element and an <InnerResults> element. If the <Counters> element is present, you should be using the attributes.

    <UnitTestResult executionId="bee66030-168c-43d4-b540-9960eccc9fd7" testId="679e7ba0-fdd2-bca3-f170-f90fc3f8cb11" testName="GetPackageTypeLookup" computerName="BUILD1" duration="00:00:00.0488998" startTime="2009-08-13T14:49:32.1406250-04:00" endTime="2009-08-13T14:49:32.1875000-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" resultType="DataDrivenTest">
      <Output>
      </Output>
      <Counters error="0" failed="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" passed="4" completed="0" inProgress="0" pending="0" />
      <InnerResults>
        <UnitTestResult executionId="2903b231-6a94-4a7c-9b13-6194f839d4b5" parentExecutionId="bee66030-168c-43d4-b540-9960eccc9fd7" testId="679e7ba0-fdd2-bca3-f170-f90fc3f8cb11" testName="GetPackageTypeLookup" computerName="BUILD1" duration="00:00:00.0163723" startTime="2009-08-13T14:49:32.1562500-04:00" endTime="2009-08-13T14:49:32.1718750-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" dataRowInfo="0" resultType="DataDrivenDataRow" />
        <UnitTestResult executionId="440f7cdd-6d0f-4e37-903e-47d6dae79cab" parentExecutionId="bee66030-168c-43d4-b540-9960eccc9fd7" testId="679e7ba0-fdd2-bca3-f170-f90fc3f8cb11" testName="GetPackageTypeLookup" computerName="BUILD1" duration="00:00:00.0127445" startTime="2009-08-13T14:49:32.1718750-04:00" endTime="2009-08-13T14:49:32.1875000-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" dataRowInfo="1" resultType="DataDrivenDataRow" />
        <UnitTestResult executionId="16164642-6190-4640-9364-fd5bb86fa8d2" parentExecutionId="bee66030-168c-43d4-b540-9960eccc9fd7" testId="679e7ba0-fdd2-bca3-f170-f90fc3f8cb11" testName="GetPackageTypeLookup" computerName="BUILD1" duration="00:00:00.0028770" startTime="2009-08-13T14:49:32.1875000-04:00" endTime="2009-08-13T14:49:32.1875000-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" dataRowInfo="2" resultType="DataDrivenDataRow" />
        <UnitTestResult executionId="83e8cff4-3b14-47ce-b7a7-86ed9c773795" parentExecutionId="bee66030-168c-43d4-b540-9960eccc9fd7" testId="679e7ba0-fdd2-bca3-f170-f90fc3f8cb11" testName="GetPackageTypeLookup" computerName="BUILD1" duration="00:00:00.0025480" startTime="2009-08-13T14:49:32.1875000-04:00" endTime="2009-08-13T14:49:32.1875000-04:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" dataRowInfo="3" resultType="DataDrivenDataRow" />
      </InnerResults>
    </UnitTestResult>



Attachment(s):
results.trx
0
Comment actions Permalink

How would you like those tests to be shown? I mean the names of that data driven tests.
I'll create a patch for your issue.

0
Comment actions Permalink

MSTest differentiates each test with "dataRowInfo", which is a number from 0 to the number of data rows.

If we have a data driven test called MyTest with three rows of data, the tests would be MyTest 0, MyTest 1, MyTest 2.

Could we add a column to results with data driven tests called dataRowInfo?

0
Comment actions Permalink

Sorry for the delay. The question is still relevant? If yes please leave a comment or create a separate thread.

Kind regards,
Marina
0
Comment actions Permalink

What was the outcome of this?  I believe I am having the same issue as the original post.

Please let me know when you get a chance.  Thanks! :)

0
Comment actions Permalink

Hello,

Thank you for feedback. Please create an issue at http://youtrack.jetbrains.net and attach sample .trx file

0

Please sign in to leave a comment.