rspec

I'm on OS X.4, running rspec with pre_commit on trunk, at revision 3317. Several failures with test/interop are failing (regarding exit codes). See the following output:

escher: svn st

escher: rake pre_commit_core

(in /Users/smt/src/ruby/rspec/trunk)

(in /Users/smt/src/ruby/rspec/trunk/rspec)

** Invoke pre_commit (first_time)

** Execute pre_commit

** Invoke examples (first_time)

** Execute examples

.....................................PPP...................................................................................

Pending:

pending example (using pending method) should be reported as "PENDING: for some reason" (for some reason)

pending example (with no block) should be reported as "PENDING: Not Yet Implemented" (Not Yet Implemented)

pending example (with block for pending) should have a failing block, passed to pending, reported as "PENDING: for some reason" (for some reason)

Finished in 0.051595 seconds

123 examples, 0 failures, 3 pending

** Invoke verify_rcov (first_time)

** Invoke spec (first_time)

** Invoke clobber_spec (first_time)

** Execute clobber_spec

** Execute spec

Profiling enabled.

....................................................................................................................................................P...........................................................P......P.....................F.F...F.F................................................P............................................................................................................................................................................................................................................................................................P............................................P.................................................................................................................................................................................................................................................................P....................................................................................................................................................................P

Given given.

And no match (PENDING).

When when.

And no match (PENDING).

Then then.

And no match (PENDING)............................................................................

When step

And step................................................................................................................................................................................

Top 10 slowest examples:

1.1209660 SpecParser should find spec name for 'it' at same line

1.1040560 SpecParser should find context and described and example for type with modifications

1.0858160 SpecParser should find spec name for 'specify' at end of spec line

1.0574740 SpecParser should find spec name for 'specify' at same line

1.0542350 SpecParser should find context name for type

1.0491760 SpecParser should find context and description for type

1.0463170 SpecParser should find spec name for 'it' at end of spec line

1.0416590 SpecParser should find example

1.0323070 SpecParser should find context for 'describe' above all specs

1.0265140 SpecParser should find context for 'context' above all specs

Pending:

Enumerable as the parent of nested example groups should be included in examples because it is a module (need to make sure nested groups know the described type)

Object#should should raise error if it receives no argument and it is not used as a left side of an operator (Is it even possible to catch this?)

Object#should_not should raise error if it receives no argument and it is not used as a left side of an operator (Is it even possible to catch this?)

arbitrary predicate with DelegateClass should access methods defined in the delegating class (LH[#48]) (

Looks like DelegateClass is delegating #should to the

delegate. Not sure how to fix this one. Or if we even should."

)

mock failure should tell you when it receives the right message with the wrong args if you stub the method (fix bug 15719)

Spec::Mocks::Mock should fail right away when method defined as never is received (Used to pass (false positive). Which one is wrong, the spec or the actual behavior?)

OptionParser should print help to stdout if no args (A regression since 1.0.8)

Spec::Story::Runner::StoryMediator should create a given scenario step if one matches (need to untangle the dark mysteries of the story runner - something needs to get stubbed here)

1)

'ExampleGroup with test/unit/interop with failing examples should return an exit code of 256' FAILED

expected: 256,

got: # (using ==)

Diff:

@@ -1,2 +1,2 @@

-256

+#

./spec/spec/interop/test/unit/spec_spec.rb:30:

2)

'ExampleGroup with test/unit/interop with example that raises an error should return an exit code of 256' FAILED

expected: 256,

got: # (using ==)

Diff:

@@ -1,2 +1,2 @@

-256

+#

./spec/spec/interop/test/unit/spec_spec.rb:42:

3)

'Test::Unit::TestCase with failing test case should return an exit code of 256' FAILED

expected: 256,

got: # (using ==)

Diff:

@@ -1,2 +1,2 @@

-256

+#

./spec/spec/interop/test/unit/testcase_spec.rb:30:

4)

'Test::Unit::TestCase with test case that raises an error should return an exit code of 256' FAILED

expected: 256,

got: # (using ==)

Diff:

@@ -1,2 +1,2 @@

-256

+#

./spec/spec/interop/test/unit/testcase_spec.rb:42:

Finished in 25.369104 seconds

1305 examples, 4 failures, 8 pending

Running 0 scenarios

Story: foo

Story: foo

0 scenarios: 0 succeeded, 0 failed, 0 pending

Pending Steps:

1) (): no match

2) (): no match

3) (): no match

** Invoke stories (first_time)

** Execute stories

/usr/local/bin/ruby stories/all.rb --colour --format plain --format html:story_server/prototype/rspec_stories.html

Running 15 scenarios

Story: autogenerated docstrings

As an RSpec user

I want examples to generate their own names

So that I can reduce duplication between example names and example code

Scenario: run passing examples with ruby

Given the file ../../examples/pure/autogenerated_docstrings_example.rb

When I run it with the ruby interpreter -fs

Then the stdout should match /should equal 5/

And the stdout should match /should be < 5/

And the stdout should match /should include "a"/

And the stdout should match /should respond to #size/

Scenario: run failing examples with ruby

Given the file ../../failing_examples/failing_autogenerated_docstrings_example.rb

When I run it with the ruby interpreter -fs

Then the stdout should match /should equal 2/

And the stdout should match /should be > 5/

And the stdout should match /should include "b"/

And the stdout should match /should not respond to #size/

Scenario: run passing examples with spec

Given the file ../../examples/pure/autogenerated_docstrings_example.rb

When I run it with the spec script -fs

Then the stdout should match /should equal 5/

And the stdout should match /should be < 5/

And the stdout should match /should include "a"/

And the stdout should match /should respond to #size/

Scenario: run failing examples with spec

Given the file ../../failing_examples/failing_autogenerated_docstrings_example.rb

When I run it with the spec script -fs

Then the stdout should match /should equal 2/

And the stdout should match /should be > 5/

And the stdout should match /should include "b"/

And the stdout should match /should not respond to #size/

Story: Spec::ExampleGroup with should methods

As an RSpec adopter accustomed to classes and methods

I want to use should_* methods in an ExampleGroup

So that I use RSpec with classes and methods that look more like RSpec examples

Scenario: Run with ruby

Given the file spec/example_group_with_should_methods.rb

When I run it with the ruby interpreter

Then the exit code should be 256 (FAILED)

And the stdout should match "2 examples, 1 failure"

Scenario: Run with spec

Given the file spec/example_group_with_should_methods.rb

When I run it with the spec script

Then the exit code should be 256

And the stdout should match "2 examples, 1 failure"

Story: Nested example groups

As an RSpec user

I want to nest examples groups

So that I can better organize my examples

Scenario: Run with ruby

Given the file ../../examples/pure/stack_spec_with_nested_example_groups.rb

When I run it with the ruby interpreter -fs

Then the stdout should match /Stack \(empty\)/

And the stdout should match /Stack \(full\)/

Scenario: Run with ruby

Given the file ../../examples/pure/stack_spec_with_nested_example_groups.rb

When I run it with the spec script -fs

Then the stdout should match /Stack \(empty\)/

And the stdout should match /Stack \(full\)/

Story: Getting correct output

As an RSpec user

I want to see output only once

So that I don't get confused

Scenario: Run with ruby

Given the file spec/simple_spec.rb

When I run it with the ruby interpreter

Then the exit code should be 0

And the stdout should not match /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/m

And the stdout should match "1 example, 0 failures"

Scenario: Run with CommandLine object

Given the file spec/simple_spec.rb

When I run it with the CommandLine object

Then the exit code should be 0

And the stdout should not match "Loaded suite"

And the stdout should not match /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/m

And the stdout should match "1 example, 0 failures"

Scenario: Tweak backtrace

Given the file stories/failing_story.rb

When I run it with the ruby interpreter

Then the stdout should not match /\/lib\/spec\//

Story: Spec and test together

As an RSpec adopter with existing Test::Unit tests

I want to run a few specs alongside my existing Test::Unit tests

So that I can experience a smooth, gradual migration path

Scenario: Run with ruby

Given the file test/spec_and_test_together.rb

When I run it with the ruby interpreter -fs

Then the exit code should be 256 (FAILED)

And the stdout should match "ATest"

And the stdout should match "Test::Unit::AssertionFailedError in 'An Example should fail with assert'"

And the stdout should match "'An Example should fail with should' FAILED"

And the stdout should match "10 examples, 6 failures"

And the stdout should match /expected: 40,\s*got: 4/m

And the stdout should match /expected: 50,\s*got: 5/m

Scenario: Run with spec

Given the file test/spec_and_test_together.rb

When I run it with the spec script -fs

Then the exit code should be 256

And the stdout should match "Test::Unit::AssertionFailedError in 'An Example should fail with assert'"

And the stdout should match "'An Example should fail with should' FAILED"

And the stdout should match "10 examples, 6 failures"

And the stdout should match /expected: 40,\s*got: 4/m

And the stdout should match /expected: 50,\s*got: 5/m

Story: Test::Unit::TestCase extended by rspec with should methods

As an RSpec adopter with existing Test::Unit tests

I want to use should_* methods in a Test::Unit::TestCase

So that I use RSpec with classes and methods that look more like RSpec examples

Scenario: Run with ruby

Given the file test/test_case_with_should_methods.rb

When I run it with the ruby interpreter

Then the exit code should be 256 (FAILED)

And the stdout should match "5 examples, 3 failures"

Scenario: Run with spec

Given the file test/test_case_with_should_methods.rb

When I run it with the spec script

Then the exit code should be 256

And the stdout should match "5 examples, 3 failures"

15 scenarios: 12 succeeded, 3 failed, 0 pending

FAILURES:

1) Spec::ExampleGroup with should methods (Run with ruby) FAILED

RuntimeError: Did not exit with 256, but with 0. Standard error:

./stories/resources/steps/running_rspec.rb:29:in `the exit code should be $exit_code'

stories/all.rb:3

2) Spec and test together (Run with ruby) FAILED

RuntimeError: Did not exit with 256, but with 0. Standard error:

./stories/resources/steps/running_rspec.rb:29:in `the exit code should be $exit_code'

stories/all.rb:3

3) Test::Unit::TestCase extended by rspec with should methods (Run with ruby) FAILED

RuntimeError: Did not exit with 256, but with 0. Standard error:

./stories/resources/steps/running_rspec.rb:29:in `the exit code should be $exit_code'

stories/all.rb:3

** Execute verify_rcov

Coverage: 100.0% (threshold: 100.0%)

** Invoke spec_html (first_time)

** Execute spec_html

Given given

And no match (PENDING)

When when

And no match (PENDING)

Then then

And no match (PENDING)

When step

And steprake aborted!

Command ruby -I"/Users/smt/src/ruby/rspec/trunk/rspec/lib" "/Users/smt/src/ruby/rspec/trunk/rspec/bin/spec" "spec/autotest/discover_spec.rb" "spec/autotest/rspec_spec.rb" "spec/spec/example/configuration_spec.rb" "spec/spec/example/example_group_class_definition_spec.rb" "spec/spec/example/example_group_factory_spec.rb" "spec/spec/example/example_group_methods_spec.rb" "spec/spec/example/example_group_spec.rb" "spec/spec/example/example_matcher_spec.rb" "spec/spec/example/example_methods_spec.rb" "spec/spec/example/example_runner_spec.rb" "spec/spec/example/example_spec.rb" "spec/spec/example/nested_example_group_spec.rb" "spec/spec/example/pending_module_spec.rb" "spec/spec/example/predicate_matcher_spec.rb" "spec/spec/example/shared_example_group_spec.rb" "spec/spec/example/subclassing_example_group_spec.rb" "spec/spec/expectations/differs/default_spec.rb" "spec/spec/expectations/extensions/object_spec.rb" "spec/spec/expectations/fail_with_spec.rb" "spec/spec/extensions/main_spec.rb" "spec/spec/interop/test/unit/spec_spec.rb" "spec/spec/interop/test/unit/testcase_spec.rb" "spec/spec/interop/test/unit/testsuite_adapter_spec.rb" "spec/spec/matchers/be_close_spec.rb" "spec/spec/matchers/be_spec.rb" "spec/spec/matchers/change_spec.rb" "spec/spec/matchers/description_generation_spec.rb" "spec/spec/matchers/eql_spec.rb" "spec/spec/matchers/equal_spec.rb" "spec/spec/matchers/exist_spec.rb" "spec/spec/matchers/handler_spec.rb" "spec/spec/matchers/has_spec.rb" "spec/spec/matchers/have_spec.rb" "spec/spec/matchers/include_spec.rb" "spec/spec/matchers/match_spec.rb" "spec/spec/matchers/matcher_methods_spec.rb" "spec/spec/matchers/mock_constraint_matchers_spec.rb" "spec/spec/matchers/operator_matcher_spec.rb" "spec/spec/matchers/raise_error_spec.rb" "spec/spec/matchers/respond_to_spec.rb" "spec/spec/matchers/satisfy_spec.rb" "spec/spec/matchers/simple_matcher_spec.rb" "spec/spec/matchers/throw_symbol_spec.rb" "spec/spec/mocks/any_number_of_times_spec.rb" "spec/spec/mocks/argument_expectation_spec.rb" "spec/spec/mocks/at_least_spec.rb" "spec/spec/mocks/at_most_spec.rb" "spec/spec/mocks/bug_report_10260_spec.rb" "spec/spec/mocks/bug_report_10263_spec.rb" "spec/spec/mocks/bug_report_11545_spec.rb" "spec/spec/mocks/bug_report_15719_spec.rb" "spec/spec/mocks/bug_report_7611_spec.rb" "spec/spec/mocks/bug_report_7805_spec.rb" "spec/spec/mocks/bug_report_8165_spec.rb" "spec/spec/mocks/bug_report_8302_spec.rb" "spec/spec/mocks/failing_mock_argument_constraints_spec.rb" "spec/spec/mocks/mock_ordering_spec.rb" "spec/spec/mocks/mock_space_spec.rb" "spec/spec/mocks/mock_spec.rb" "spec/spec/mocks/multiple_return_value_spec.rb" "spec/spec/mocks/null_object_mock_spec.rb" "spec/spec/mocks/once_counts_spec.rb" "spec/spec/mocks/options_hash_spec.rb" "spec/spec/mocks/partial_mock_spec.rb" "spec/spec/mocks/partial_mock_using_mocks_directly_spec.rb" "spec/spec/mocks/passing_mock_argument_constraints_spec.rb" "spec/spec/mocks/precise_counts_spec.rb" "spec/spec/mocks/record_messages_spec.rb" "spec/spec/mocks/stub_spec.rb" "spec/spec/mocks/twice_counts_spec.rb" "spec/spec/package/bin_spec_spec.rb" "spec/spec/runner/class_and_argument_parser_spec.rb" "spec/spec/runner/command_line_spec.rb" "spec/spec/runner/drb_command_line_spec.rb" "spec/spec/runner/execution_context_spec.rb" "spec/spec/runner/formatter/failing_example_groups_formatter_spec.rb" "spec/spec/runner/formatter/failing_examples_formatter_spec.rb" "spec/spec/runner/formatter/html_formatter_spec.rb" "spec/spec/runner/formatter/profile_formatter_spec.rb" "spec/spec/runner/formatter/progress_bar_formatter_spec.rb" "spec/spec/runner/formatter/snippet_extractor_spec.rb" "spec/spec/runner/formatter/spec_mate_formatter_spec.rb" "spec/spec/runner/formatter/specdoc_formatter_spec.rb" "spec/spec/runner/formatter/story/html_formatter_spec.rb" "spec/spec/runner/formatter/story/plain_text_formatter_spec.rb" "spec/spec/runner/heckle_runner_spec.rb" "spec/spec/runner/heckler_spec.rb" "spec/spec/runner/noisy_backtrace_tweaker_spec.rb" "spec/spec/runner/option_parser_spec.rb" "spec/spec/runner/options_spec.rb" "spec/spec/runner/output_one_time_spec.rb" "spec/spec/runner/quiet_backtrace_tweaker_spec.rb" "spec/spec/runner/reporter_spec.rb" "spec/spec/runner/resources/a_spec.rb" "spec/spec/runner/spec_parser_spec.rb" "spec/spec/runner_spec.rb" "spec/spec/story/extensions/main_spec.rb" "spec/spec/story/extensions_spec.rb" "spec/spec/story/given_scenario_spec.rb" "spec/spec/story/runner/plain_text_story_runner_spec.rb" "spec/spec/story/runner/scenario_collector_spec.rb" "spec/spec/story/runner/scenario_runner_spec.rb" "spec/spec/story/runner/story_mediator_spec.rb" "spec/spec/story/runner/story_parser_spec.rb" "spec/spec/story/runner/story_runner_spec.rb" "spec/spec/story/runner_spec.rb" "spec/spec/story/scenario_spec.rb" "spec/spec/story/step_group_spec.rb" "spec/spec/story/step_mother_spec.rb" "spec/spec/story/step_spec.rb" "spec/spec/story/story_spec.rb" "spec/spec/story/world_spec.rb" "spec/spec/translator_spec.rb" --format html:../doc/output/report.html --backtrace failed

./lib/spec/rake/spectask.rb:184:in `define'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1003:in `verbose'

./lib/spec/rake/spectask.rb:153:in `define'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in `invoke_with_call_chain'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `invoke_with_call_chain'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke'

/Users/smt/src/ruby/rspec/trunk/rspec/pre_commit/lib/pre_commit/pre_commit.rb:9:in `rake_invoke'

/Users/smt/src/ruby/rspec/trunk/rspec/pre_commit/lib/pre_commit/core.rb:10:in `website'

/Users/smt/src/ruby/rspec/trunk/rspec/pre_commit/lib/pre_commit/core.rb:4:in `pre_commit'

/Users/smt/src/ruby/rspec/trunk/rspec/rakefile:162

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in `invoke_with_call_chain'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `invoke_with_call_chain'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in `invoke_task'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run'

/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31

/usr/local/bin/rake:19:in `load'

/usr/local/bin/rake:19

rake aborted!

RSpec Core pre_commit failed

/Users/smt/src/ruby/rspec/trunk/rakefile:19

(See full trace by running task with --trace)

escher: svn st

escher: svn info

Path: .

URL: svn://rubyforge.org/var/svn/rspec/trunk

Repository Root: svn://rubyforge.org/var/svn/rspec

Repository UUID: 410327ef-2207-0410-a325-f78bbcb22a5a

Revision: 3317

Node Kind: directory

Schedule: normal

Last Changed Author: aslak_hellesoy

Last Changed Rev: 3317

Last Changed Date: 2008-02-21 12:20:06 -0500 (Thu, 21 Feb 2008)

escher: