Agra and JUnit 4

I just noticed that the Agra code coverage runner can't handle JUnit4 tests.
It says something like "junit.framework.AssertionFailedError: No tests found in my.package.MyTest" if MyTest uses the annotation style tests of JUnit4 .

I would consider this a bug, is JUnit4 support planned for Agra?

Edit: I noticed it with #3914, I'll give it a try with #3989

21 comments
Comment actions Permalink

Nope, #3989 also can't handle JUnit4 tests

0
Comment actions Permalink

I find it rather strange because the IPR runner is used for code coverage and IDEA works perfectly well with JUnit4.

0
Comment actions Permalink

Tobias,

Please add junit-4 jar to your junit task. This should help.

Kind regards,
KIR

0
Comment actions Permalink

I don't quite understand what you mean by that:
Where can I add the jar to which "junit task"?

0
Comment actions Permalink

Tobias,

You may want to take a look here:
http://www.jetbrains.net/jira/browse/TW-1959

-Dave

0
Comment actions Permalink

Tobias,

TeamCity doesn't have "code coverage" runner. It has code coverage support for Ant runner and for Ipr runner.
Ipr runner of TeamCity uses junit-4 to run project tests.
Ant runner of TeamCity uses by default junit-3.8.1, but if you provide junit-4 jar in classpath for junit task, junit4 tests should work as well.

Could you please clarify which runner do you use, which junit and how to reproduce the problem you've encountered.

With kind regards,
KIR

0
Comment actions Permalink

I use the IPR runner with the option "Code coverage" enabled.

The project uses junit 4.3 for the unit tests.

The tests can be run in IDEA 6.0.5 beta (junit 4.3 doesn't work in 6.0.4, throws a ClassCastException there).

0
Comment actions Permalink

Thanks for the hint, but I use the Ipr runner. At least I try to use it.

0
Comment actions Permalink

Ipr runner of TeamCity uses junit-4 to run project tests.


It seems like the problem is the Ipr runner:
Also with "Code Coverage" disabled, the build fails because of
junit.framework.AssertionFailedError: No tests found in ...

Switching to JUnit 4.1 didn't solve the problem either.

Switching to JUnit 3.8 style tests (extending TestCase instead of using annotations) runs the tests (and leads to the problems I described in thread http://www.intellij.net/forums/thread.jspa?threadID=266427&tstart=0

So I would say the Agra Ipr runner does NOT use junit-4.

Message was edited by:
Tobias Kurzke

0
Comment actions Permalink

Tobias,

Are there chances that classpath of your project contains both junit-3.8 and junit-4?
I'd appreciate if you file a jira request with build agent logs and possibly your project files.

Kind regards,
KIR

0
Comment actions Permalink

I have two projects here where I cannot use JUnit4 style tests:

One of the projects has modules that use JUnit 3.8 and modules that use JUnit 4.
So in this project indeed both versions are in the classpath.
Nevertheless it should work and JUnit4 style test cases should be run regardless of whether there are other modules using other kinds of tests in the project or not.

The second project uses only JUnit4 in all of its modules.
Tests that extend TestCase are run, tests that use annotations lead to the described error.

Next thing I tried is to copy junit-4.3.jar to C:\BuildAgent\plugins\ant\lib, remove the existing junit-3.8.2.jar there and to adjust the junit entry in libraries.properties to 4.3 (no idea if this is necessary, but I thought it couldn't hurt).
When I restart the build agent (manually, I don't have it installed as a service), the agent's command line doesn't include any reference to junit 3.8 anymore.

Still, I cannot get JUnit4 style tests to work.

0
Comment actions Permalink

Another problem with the Ipr runner related to unit tests is, that the runner strictly tries to run every class that ends with "Test" as a unit test even if it as actually just a class that ends with Test (without having anything to do with JUnit).
It also leads to "junit.framework.AssertionFailedError: No tests found in" and a failed build.

The runner should be a little bit more intelligent.

0
Comment actions Permalink

Ok, finally I got it to work, but the reason why it failed is really weird:

I had ant 1.6.5 installed on my machine and had ANT_HOME point to the install location.
I saw in the build agent's log file, that for some strange reason, junit-3.8.1 was included in the class path from "C:\Program Files\apache-ant-1.6.5\lib\".

After I removed the environment variable ANT_HOME and restarted the agent, JUnit 4 tests are run correctly.

It seems like Ant and TeamCity build agent must not be installed on the same machine as teamcity evaluates ANT_HOME if set which leads to the described error.

0
Comment actions Permalink

Tobias,

We have both functional tests for Junit4 tests with annotations and a local project which runs JUnit4 tests from IPR runner (uses annotations to declare tests). And this works just fine. So this is definitely possible.
I'd appreciate if you send me (possibly via email kir jetbrains.com) logs from your build agent, ipr/iml files, and integrationBuild.xml* from buildAgent/work/Server/[projectName] files. Or create Jira issue with all this information.

Kind regards,
KIR

0
Comment actions Permalink

Great news :)
In fact, this is a feature of TeamCity that we allow custom Ant installations for your builds. If you have some custom tasks which require additional jars in Ant's lib directory, you may need a custom Ant installation. That's why we evaluate ANT_HOME environment variable before the build.

With kind regards,
KIR

0
Comment actions Permalink

This is default behaviour for finding tests (just like Ant works, Ipr runner is in fact ant runner with generated ant build file). You can specify test patterns for your tests in the configuration of Ipr runner. You may read FAQ on the syntax of test patterns (reference documentation for that and help links will be available as well).

Regards,
KIR

0
Comment actions Permalink

Now that I got JUnit4 tests to run (by removing ant on my machine), this isn't an issue anymore.

Also the executing of classes ending with "Test" as unit tests regardless of whether they are actually unit tests or not doesn't happen anymore with junit 4.

It might be an issue though for users who still want to use junit 3.8 for some reason.

0
Comment actions Permalink

So what do you then if you have a custom Ant installation on the build server and you want to use JUnit4 tests?
Is it sufficient to copy junit-4.3.jar into the lib directory of the custom Ant installation?
Or is it required to also delete the existing junit-3.8.x there?

Is there something that has to be configured additionally?

0
Comment actions Permalink

Ant 1.6.5 is not compatible with JUnit 4. You just need Ant 1.7.

0
Comment actions Permalink

One (hopefully) last question regarding this topic:
I just checked with our current build server running TC 1.2: there is no custom Ant installed here, nevertheless JUnit4 tests don't work.

Was JUnit4 support for Ipr runner introduced with Agra?
(I hope so, because otherwise something else is still wrong with our TC 1.2 installation).

0
Comment actions Permalink

Junit4 support (and Ant 1.7) was added in Agra. You can mimic this support in TeamCity 1.2 by changing ant165.jar in TeamCity/webapps/ROOT/update/plugins/ to ant1.7 distribution - there were reports that such a trick allows to run JUnit4 tests in TeamCity 1.2.

Kind regards,
KIR

0

Please sign in to leave a comment.