Selenium reporting?

Is there a plugin or plan to support pass/fail reporting for Selenium output?

When using SauceLabs to run Selenium tests, they apparently return the raw Selenium output (which is not the Test::Unit output that TC understands).

Failing that, what would it take to write something to get TC to understand the output?

An example of Selenium test output - this is not being reported as a failing test withing TC's UI...
[Test Output] #<Test::Unit::AssertionFailedError: <false> is not true.> /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:51:in `assert_block' org/jruby/RubyProc.java:258:in `call' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:1370:in `_wrap_assertion' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:49:in `assert_block' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:82:in `assert' org/jruby/RubyProc.java:258:in `call' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:1375:in `_wrap_assertion' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/assertions.rb:64:in `assert' /Users/teamcity/buildAgent/work/9ef3f4d965a09397/test/selenium/ad_selector_selenium_test.rb:42:in `test_Message_specific_to_AdNetwork_appears_when_videos_are_unavailable_in_ie' /Users/teamcity/buildAgent/work/9ef3f4d965a09397/lib//qa/selenium_base.rb:49:in `create_player_driver' /Users/teamcity/buildAgent/work/9ef3f4d965a09397/test/selenium/ad_selector_selenium_test.rb:40:in `test_Message_specific_to_AdNetwork_appears_when_videos_are_unavailable_in_ie' org/jruby/RubyBasicObject.java:1698:in `__send__' /usr/local/rvm/gems/jruby-1.6.7/gems/activesupport-3.0.11/lib/active_support/testing/setup_and_teardown.rb:67:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/activesupport-3.0.11/lib/active_support/callbacks.rb:434:in `_run_setup_callbacks' /usr/local/rvm/gems/jruby-1.6.7/gems/activesupport-3.0.11/lib/active_support/testing/setup_and_teardown.rb:65:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/testsuite.rb:53:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/testsuite.rb:53:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/ui/testrunnermediator.rb:44:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/ui/testrunnermediator.rb:89:in `with_listener' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/ui/testrunnermediator.rb:39:in `run' /Users/teamcity/buildAgent/plugins/rake-runner/rb/patch/testunit/test/unit/ui/testrunnermediator.rb:41:in `run_suite' /Users/teamcity/buildAgent/plugins/rake-runner/rb/patch/testunit/test/unit/ui/teamcity/testrunner.rb:131:in `start_mediator' /Users/teamcity/buildAgent/plugins/rake-runner/rb/patch/testunit/test/unit/ui/teamcity/testrunner.rb:119:in `start' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/ui/testrunnerutilities.rb:24:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/autorunner.rb:378:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/autorunner.rb:434:in `change_work_directory' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/autorunner.rb:377:in `run' /usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit/autorunner.rb:58:in `run'
/usr/local/rvm/gems/jruby-1.6.7/gems/test-unit-2.5.0/lib/test/unit.rb:330:in `(root)'

1 comment
Comment actions Permalink

Andy,

Actually, SourceLabs claims thay support TeamCity, so you can ask them how exactly the support works.

From what I can see on their site, the tests are run from within usual test cases (e.g. JUnit's ones) and should get reported in TeamCity as usual tests.


Do you write in Ruby?
How are your tests run? i.e. do you use a testing framework to execute the test code?

0

Please sign in to leave a comment.