NSInternalInconsistencyException on failing expectations

[atillman]

Environment
XCode Version 7.3 (7D175)
Nimble, Master
Platform iOS
Unit Testing bundle
Cocoapods 0.39.0

Running a test in a Quick spec my passing specs run fine. The failing tests are throwing an error when reporting the error to XCTest. I can work around it's reported as a failure but the error is always reported as 'caught "NSInternalInconsistencyException", "Attempted to report a test failure to XCTest while no test case was running.' I can dig into the reported error and see the failure that caused the error but that is a pain.

Here is the full error and stacktrace I am getting:

file:///Users/antillman/Projects/PQ/mobile/ios/PrettyQuickTests/Specs/Extensions/CoreLocation/CLLocationCoordinate2D+PQExtensionsSpec.swift: test failure: -[CLLocationCoordinate2D_PQExtensionsSpec Equatable__returns_true_of_both_the_latitude_and_longitude_are_the_same()] failed: failed: caught "NSInternalInconsistencyException", "Attempted to report a test failure to XCTest while no test case was running. The failure was:
"expected to equal <CLLocationCoordinate2D(latitude: 41.889699999999998, longitude: -87.625)>, got <CLLocationCoordinate2D(latitude: 41.889699999999998, longitude: -87.625)>
"
It occurred at: /Users/antillman/Projects/PQ/mobile/ios/PrettyQuickTests/Specs/Extensions/CoreLocation/CLLocationCoordinate2D+PQExtensionsSpec.swift:39"
(
    0   CoreFoundation                      0x00000001122e4d85 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000111d58deb objc_exception_throw + 48
    2   CoreFoundation                      0x00000001122e49c9 -[NSException raise] + 9
    3   Nimble                              0x00000001209a9104 _TTSf4gs_g___TF6NimbleP33_3F48DF7243EE4E52B06D6D1EA023E49113recordFailureFTSS8locationCS_14SourceLocation_T_ + 1252
    4   Nimble                              0x00000001209a9399 _TTSf4n_g_g_d___TFC6Nimble19NimbleXCTestHandler6assertfTSb7messageCS_14FailureMessage8locationCS_14SourceLocation_T_ + 617
    5   Nimble                              0x00000001209a855e _TTWC6Nimble19NimbleXCTestHandlerS_16AssertionHandlerS_FS1_6assertfTSb7messageCS_14FailureMessage8locationCS_14SourceLocation_T_ + 94
    6   Nimble                              0x000000012099ee70 _TFV6Nimble11Expectation6verifyfTSbCS_14FailureMessage_T_ + 128
    7   Nimble                              0x000000012099f03c _TFV6Nimble11Expectation2touRd__S_7Matcherxzwd__9ValueTyperfTqd__11descriptionGSqSS__T_ + 364
    8   Nimble                              0x000000012098e5e1 _TZF6Nimbleoi2eeuRxs9EquatablerFTGVS_11Expectationx_GSqx__T_ + 241
    9   PrettyQuickTests                    0x00000001202ff225 _TFFFC16PrettyQuickTests39CLLocationCoordinate2D_PQExtensionsSpec4specFT_T_U0_FT_T_U2_FT_T_ + 581
    10  Quick                               0x0000000120b23c44 _TPA__TTRXFdCb__dT__XFo__dT__ + 36
    11  Quick                               0x0000000120b16fd3 _TFC5Quick7Example3runfT_T_ + 419
    12  Quick                               0x0000000120b17156 _TToFC5Quick7Example3runfT_T_ + 22
    13  Quick                               0x0000000120b11ad0 __60+[QuickSpec addInstanceMethodForExample:classSelectorNames:]_block_invoke + 80
    14  CoreFoundation                      0x00000001121cd5cc __invoking___ + 140
    15  CoreFoundation                      0x00000001121cd41e -[NSInvocation invoke] + 286
    16  XCTest                              0x000000011f80eef3 __24-[XCTestCase invokeTest]_block_invoke_2 + 362
    17  XCTest                              0x000000011f843613 -[XCTestContext performInScope:] + 190
    18  XCTest                              0x000000011f80ed78 -[XCTestCase invokeTest] + 169
    19  XCTest                              0x000000011f80f3a2 -[XCTestCase performTest:] + 459
    20  XCTest                              0x000000011f80ccf7 -[XCTestSuite performTest:] + 396
    21  XCTest                              0x000000011f80ccf7 -[XCTestSuite performTest:] + 396
    22  XCTest                              0x000000011f80ccf7 -[XCTestSuite performTest:] + 396
    23  XCTest                              0x000000011f7f9b10 __25-[XCTestDriver _runSuite]_block_invoke + 51
    24  XCTest                              0x000000011f81ab4c -[XCTestObservationCenter _observeTestExecutionForBlock:] + 640
    25  XCTest                              0x000000011f7f9a55 -[XCTestDriver _runSuite] + 453
    26  XCTest                              0x000000011f7fa7d1 -[XCTestDriver _checkForTestManager] + 259
    27  XCTest                              0x000000011f844a9a _XCTestMain + 628
    28  CoreFoundation                      0x000000011220a2ec __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    29  CoreFoundation                      0x00000001121fff75 __CFRunLoopDoBlocks + 341
    30  CoreFoundation                      0x00000001121ff6d2 __CFRunLoopRun + 850
    31  CoreFoundation                      0x00000001121ff0f8 CFRunLoopRunSpecific + 488
    32  GraphicsServices                    0x0000000114ca6ad2 GSEventRunModal + 161
    33  UIKit                               0x000000010f382f09 UIApplicationMain + 171
    34  PrettyQuick                         0x000000010c5267bf main + 111
    35  libdyld.dylib                       0x0000000112d5b92d start + 1
)

[atillman]

Also, this issue just started for all expecations in my project after I upgraded to the latest version and did not happen when I was running the xcode 7.2 with the xcode7.1 version of nimble and quick

[phatblat]

This sounds like it could be due to #270 which is fixed in master now but hasn't been included in a release yet.

You can test it out with the following in your Podfile:

pod 'Nimble', :git => 'https://github.com/Quick/Nimble.git'

[atillman]

Oops, I was mistaken. I am already using master. Sorry about that. That is what is already in my Podfile. I will update the description

[phatblat]

OK, good to know. Are any of these performance tests using measureBlock:?

[atillman]

No. They are just basic spec tests. I got it on the new spec I was writing but when back and checked that it happened with the old specs as well.

[kevinmcconnell]

I'm seeing the same problem since #271. Reverting to a version earlier than that seems to solve it, though.

[briancroom]

@atillman @kevinmcconnell Could you try this again with Nimble 4.0.0 or the latest master and report whether you are still seeing this? I have been unable to reproduce it locally and am hoping it may be behind us now.

[kevinmcconnell]

I just updated to 4.0.0 and it's working great for me so far. Test output looks correct, for both passing & failing expectations, and I no longer see any NSInternalInconsistencyException on failure.

I'll let you know if the problem resurfaces for me later. But as far as I can tell for now, it looks fixed.

Thanks!

[gca3020]

I was having a similar issue with 4.0.0, but updating to 4.0.1 seems to have fixed it.

[atillman]

4.0.1 also working for me. Thanks!

[phatblat]

Excellent. Closing as it appears this is resolved now.

[anamariastoica91]

Hei guys, this happens to me too with version 8.0.9

Xcode version 11.6
IOS 13.6
cocoapods 1.9.3

It worked before but now has started to fail.
I've used master too and it's the same.

Any hints of what i am doing wrong ?

: failed: caught "NSInternalInconsistencyException", "Attempted to report a test failure to XCTest while no test case was running. The failure was:
"expected to be true, got <false>
"
It occurred at: /Users/nm87ju/ios-app/dealwiseTests/ModelsTest/Search/FullSearchBuilderTests.swift:121"
(
	0   CoreFoundation                      0x000000011a0d11bb __exceptionPreprocess + 331
	1   libobjc.A.dylib                     0x00000001194f8735 objc_exception_throw + 48
	2   CoreFoundation                      0x000000011a0d0d29 -[NSException raise] + 9
	3   dealwiseTests                       0x0000000136c40e6a $s6Nimble13recordFailure_8locationySS_AA14SourceLocationCtF + 1594
	4   dealwiseTests                       0x0000000136c4080b $s6Nimble0A13XCTestHandlerC6assert_7message8locationySb_AA14FailureMessageCAA14SourceLocationCtF + 523
	5   dealwiseTests                       0x0000000136c40fab $s6Nimble0A13XCTestHandlerCAA09AssertionC0A2aDP6assert_7message8locationySb_AA14FailureMessageCAA14SourceLocationCtFTW + 27
	6   dealwiseTests                       0x0000000136c2c8d3 $s6Nimble11ExpectationV6verifyyySb_AA14FailureMessageCtF + 371
	7   dealwiseTests                       0x0000000136c2d077 $s6Nimble11ExpectationV2to_11descriptionACyxGAA9PredicateVyxG_SSSgtF + 423
	8   dealwiseTests                       0x00000001369cdf89 $s13dealwiseTests017FullSearchBuilderB0C4specyyFyycfU0_ + 2313
	9   dealwiseTests                       0x0000000136c83cbc $s5Quick7ExampleC3runyyF + 2604
	10  dealwiseTests                       0x0000000136c8484b $s5Quick7ExampleC3runyyFTo + 43
	11  dealwiseTests                       0x0000000136c7d545 __60+[QuickSpec addInstanceMethodForExample:classSelectorNames:]_block_invoke + 133
	12  CoreFoundation                      0x000000011a0d803c __invoking___ + 140
	13  CoreFoundation                      0x000000011a0d54d5 -[NSInvocation invoke] + 325
	14  XCTest                              0x0000000135190fa7 __24-[XCTestCase invokeTest]_block_invoke_2 + 52
	15  XCTest                              0x0000000135190f53 __24-[XCTestCase invokeTest]_block_invoke.206 + 320
	16  XCTest                              0x00000001351ebd32 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
	17  XCTest                              0x00000001351ebc44 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
	18  XCTest                              0x0000000135190966 -[XCTestCase invokeTest] + 1183
	19  XCTest                              0x0000000135192299 __26-[XCTestCase performTest:]_block_invoke_2 + 43
	20  XCTest                              0x00000001351ebd32 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
	21  XCTest                              0x00000001351ebc44 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
	22  XCTest                              0x00000001351921d0 __26-[XCTestCase performTest:]_block_invoke.359 + 86
	23  XCTest                              0x00000001351fe97d +[XCTContext runInContextForTestCase:block:] + 211
	24  XCTest                              0x0000000135191a84 -[XCTestCase performTest:] + 566
	25  XCTest                              0x00000001351d82fe -[XCTest runTest] + 57
	26  XCTest                              0x000000013518bcc0 __27-[XCTestSuite performTest:]_block_invoke + 354
	27  XCTest                              0x000000013518b412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
	28  XCTest                              0x00000001351fe97d +[XCTContext runInContextForTestCase:block:] + 211
	29  XCTest                              0x000000013518b3c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
	30  XCTest                              0x000000013518b72e -[XCTestSuite performTest:] + 348
	31  XCTest                              0x00000001351d82fe -[XCTest runTest] + 57
	32  XCTest                              0x000000013518bcc0 __27-[XCTestSuite performTest:]_block_invoke + 354
	33  XCTest                              0x000000013518b412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
	34  XCTest                              0x00000001351fe97d +[XCTContext runInContextForTestCase:block:] + 211
	35  XCTest                              0x000000013518b3c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
	36  XCTest                              0x000000013518b72e -[XCTestSuite performTest:] + 348
	37  XCTest                              0x00000001351d82fe -[XCTest runTest] + 57
	38  XCTest                              0x000000013518bcc0 __27-[XCTestSuite performTest:]_block_invoke + 354
	39  XCTest                              0x000000013518b412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
	40  XCTest                              0x00000001351fe97d +[XCTContext runInContextForTestCase:block:] + 211
	41  XCTest                              0x000000013518b3c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
	42  XCTest                              0x000000013518b72e -[XCTestSuite performTest:] + 348
	43  XCTest                              0x00000001351d82fe -[XCTest runTest] + 57
	44  XCTest                              0x000000013520de84 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
	45  XCTest                              0x000000013520df71 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.100 + 96
	46  XCTest                              0x00000001351a66b6 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 682
	47  XCTest                              0x000000013520dc0f -[XCTTestRunSession runTestsAndReturnError:] + 615
	48  XCTest                              0x000000013516f6b4 -[XCTestDriver runTestsAndReturnError:] + 456
	49  XCTest                              0x00000001351fa5bc _XCTestMain + 2496
	50  libXCTestBundleInject.dylib         0x000000010f7a0bfa __copy_helper_block_e8_32s + 0
	51  CoreFoundation                      0x000000011a03662c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
	52  CoreFoundation                      0x000000011a035de0 __CFRunLoopDoBlocks + 336
	53  CoreFoundation                      0x000000011a030654 __CFRunLoopRun + 1284
	54  CoreFoundation                      0x000000011a02fe11 CFRunLoopRunSpecific + 625
	55  GraphicsServices                    0x000000011c33d1dd GSEventRunModal + 62
	56  UIKitCore                           0x000000012200c81d UIApplicationMain + 140
	57  dealwise                            0x000000010dde333b main + 75
	58  libdyld.dylib                       0x000000011b316575 start + 1
	59  ???                                 0x0000000000000005 0x0 + 5
)