Bug 333041 - Tests fail in armhf build
Summary: Tests fail in armhf build
Status: RESOLVED WORKSFORME
Alias: None
Product: trojita
Classification: Applications
Component: Ubuntu (other bugs)
Version First Reported In: git
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: trojita-ubuntu-bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-04-03 18:18 UTC by Daniel Holbach
Modified: 2014-04-07 14:35 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Entire build log (392.44 KB, text/x-log)
2014-04-03 18:18 UTC, Daniel Holbach
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Holbach 2014-04-03 18:18:34 UTC
Created attachment 85941 [details]
Entire build log

During the build:

...

cd obj*; xvfb-run -a ctest -j 4 --output-on-failure
Test project /tmp/buildd/trojita-0.4.1+1/obj-arm-linux-gnueabihf
      Start  1: test_Composer_Submission
      Start  2: test_Composer_responses
      Start  3: test_Html_formatting
      Start  4: test_Imap_DisappearingMailboxes
 1/26 Test  #2: test_Composer_responses .....................   Passed    0.54 sec
 2/26 Test  #4: test_Imap_DisappearingMailboxes .............   Passed    0.71 sec
      Start  5: test_Imap_Idle
      Start  6: test_Imap_LowLevelParser
 3/26 Test  #1: test_Composer_Submission ....................***Exception: SegFault  0.80 sec
********* Start testing of ComposerSubmissionTest *********
Config: Using QtTest library 5.2.1, Qt 5.2.1
PASS   : ComposerSubmissionTest::initTestCase()
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

 4/26 Test  #3: test_Html_formatting ........................***Exception: SegFault  0.82 sec
libEGL warning: GLX/DRI2 is not supported
libEGL warning: DRI2: failed to create any config
libEGL warning: GLX: failed to load GLX
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

      Start  7: test_Imap_Message
      Start  8: test_Imap_Model
 5/26 Test  #6: test_Imap_LowLevelParser ....................   Passed    0.61 sec
      Start  9: test_Imap_Parser_parse
 6/26 Test  #7: test_Imap_Message ...........................   Passed    0.37 sec
 7/26 Test  #8: test_Imap_Model .............................   Passed    0.39 sec
      Start 10: test_Imap_Responses
      Start 11: test_Imap_SelectedMailboxUpdates
 8/26 Test #10: test_Imap_Responses .........................   Passed    0.53 sec
      Start 12: test_Imap_Tasks_CreateMailbox
 9/26 Test #12: test_Imap_Tasks_CreateMailbox ...............   Passed    0.18 sec
10/26 Test  #5: test_Imap_Idle ..............................***Failed    1.81 sec
********* Start testing of ImapModelIdleTest *********
Config: Using QtTest library 5.2.1, Qt 5.2.1
PASS   : ImapModelIdleTest::initTestCase()
FAIL!  : ImapModelIdleTest::testIdleNo() Compared values are not the same
   Actual   (QString::fromUtf8(SOCK->writtenStuff())): y3 IDLE?

   Expected (QString())                              : 
   Loc: [/tmp/buildd/trojita-0.4.1+1/tests/Utils/LibMailboxSync.cpp(498)]
FAIL!  : ImapModelIdleTest::testIdleNo() Compared values are not the same
   Actual   (written)         : 
   Expected (t.mk("IDLE\r\n")): 79 33 20 49 44 4C 45 0D 0A
   Loc: [/tmp/buildd/trojita-0.4.1+1/tests/Imap/test_Imap_Idle.cpp(61)]
PASS   : ImapModelIdleTest::testIdleImmediateReturn()
PASS   : ImapModelIdleTest::testIdleRenewal()
PASS   : ImapModelIdleTest::testIdleBreakTask()
PASS   : ImapModelIdleTest::testIdleSlowResponses()
PASS   : ImapModelIdleTest::testIdleNoPerpetuateRenewal()
PASS   : ImapModelIdleTest::testIdleMailboxChange()
PASS   : ImapModelIdleTest::cleanupTestCase()
Totals: 8 passed, 2 failed, 0 skipped
********* Finished testing of ImapModelIdleTest *********

11/26 Test  #9: test_Imap_Parser_parse ......................   Passed    1.18 sec
      Start 13: test_Imap_Tasks_DeleteMailbox
      Start 14: test_Imap_Tasks_ListChildMailboxes
      Start 15: test_Imap_Tasks_ObtainSynchronizedMailbox
12/26 Test #11: test_Imap_SelectedMailboxUpdates ............   Passed    1.34 sec
13/26 Test #14: test_Imap_Tasks_ListChildMailboxes ..........   Passed    0.43 sec
14/26 Test #13: test_Imap_Tasks_DeleteMailbox ...............   Passed    0.54 sec
      Start 16: test_Imap_Tasks_OpenConnection
      Start 17: test_Imap_Threading
      Start 18: test_Imap_BodyParts
15/26 Test #16: test_Imap_Tasks_OpenConnection ..............   Passed    0.74 sec
      Start 19: test_Imap_Offline
16/26 Test #19: test_Imap_Offline ...........................   Passed    0.33 sec
      Start 20: test_Imap_CopyAndFlagOperations
17/26 Test #18: test_Imap_BodyParts .........................   Passed    1.67 sec
18/26 Test #20: test_Imap_CopyAndFlagOperations .............   Passed    0.61 sec
      Start 21: test_Rfc5322
      Start 22: test_RingBuffer
19/26 Test #22: test_RingBuffer .............................   Passed    0.22 sec
20/26 Test #21: test_Rfc5322 ................................   Passed    0.23 sec
      Start 23: test_SenderIdentitiesModel
      Start 24: test_SqlCache
21/26 Test #23: test_SenderIdentitiesModel ..................   Passed    0.33 sec
      Start 25: test_algorithms
22/26 Test #24: test_SqlCache ...............................   Passed    0.53 sec
23/26 Test #25: test_algorithms .............................   Passed    0.21 sec
      Start 26: test_rfccodecs
24/26 Test #26: test_rfccodecs ..............................   Passed    0.25 sec
25/26 Test #15: test_Imap_Tasks_ObtainSynchronizedMailbox ...   Passed  148.30 sec
26/26 Test #17: test_Imap_Threading .........................   Passed  269.56 sec

88% tests passed, 3 tests failed out of 26

Total Test time (real) = 272.79 sec

The following tests FAILED:
	  1 - test_Composer_Submission (SEGFAULT)
	  3 - test_Html_formatting (SEGFAULT)
	  5 - test_Imap_Idle (Failed)
Errors while running CTest


...
Comment 1 Jan Kundrát 2014-04-03 20:01:50 UTC
test_Imap_Idle is unfortunately timing-sensitive because it needs to check that the code asks for IDLE within the expected timeframe. This probably happens under some QEMU instance or on native HW, right? If so, please consider disabling parallel execution of ctest and see if it helps. The code is set to ask for IDLE after 30ms and the timeout is 40ms. Do you have some data on what kind of latency we can expect on your build HW?

Regarding test_Composer_Submission and test_Html_formatting, these tests both actually require X. I would suggest further investigation on what makes it die -- perhaps your arm X server doesn't offer all extensions which the ARM build of Qt needs? Cannot help more, sorry.
Comment 2 Daniel Holbach 2014-04-04 09:50:06 UTC
I just disabled parallel tests and ran it in a armhf qemu build locally. I'll try to get answers from people who know more about the machinery we have.

cd obj*; xvfb-run -a ctest --output-on-failure
Test project /tmp/buildd/trojita-0.4.1+1/obj-arm-linux-gnueabihf
      Start  1: test_Composer_Submission
 1/26 Test  #1: test_Composer_Submission ....................***Exception: SegFault  0.31 sec
********* Start testing of ComposerSubmissionTest *********
Config: Using QtTest library 5.2.1, Qt 5.2.1
PASS   : ComposerSubmissionTest::initTestCase()
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

      Start  2: test_Composer_responses
 2/26 Test  #2: test_Composer_responses .....................   Passed    0.19 sec
      Start  3: test_Html_formatting
 3/26 Test  #3: test_Html_formatting ........................***Exception: SegFault  0.49 sec
libEGL warning: GLX/DRI2 is not supported
libEGL warning: DRI2: failed to create any config
libEGL warning: GLX: failed to load GLX
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

      Start  4: test_Imap_DisappearingMailboxes
 4/26 Test  #4: test_Imap_DisappearingMailboxes .............   Passed    0.41 sec
      Start  5: test_Imap_Idle
 5/26 Test  #5: test_Imap_Idle ..............................   Passed    1.10 sec
      Start  6: test_Imap_LowLevelParser
 6/26 Test  #6: test_Imap_LowLevelParser ....................   Passed    0.18 sec
      Start  7: test_Imap_Message
 7/26 Test  #7: test_Imap_Message ...........................   Passed    0.13 sec
      Start  8: test_Imap_Model
 8/26 Test  #8: test_Imap_Model .............................   Passed    0.15 sec
      Start  9: test_Imap_Parser_parse
 9/26 Test  #9: test_Imap_Parser_parse ......................   Passed    0.79 sec
      Start 10: test_Imap_Responses
10/26 Test #10: test_Imap_Responses .........................   Passed    0.11 sec
      Start 11: test_Imap_SelectedMailboxUpdates
11/26 Test #11: test_Imap_SelectedMailboxUpdates ............   Passed    0.54 sec
      Start 12: test_Imap_Tasks_CreateMailbox
12/26 Test #12: test_Imap_Tasks_CreateMailbox ...............   Passed    0.15 sec
      Start 13: test_Imap_Tasks_DeleteMailbox
13/26 Test #13: test_Imap_Tasks_DeleteMailbox ...............   Passed    0.26 sec
      Start 14: test_Imap_Tasks_ListChildMailboxes
14/26 Test #14: test_Imap_Tasks_ListChildMailboxes ..........   Passed    0.21 sec
      Start 15: test_Imap_Tasks_ObtainSynchronizedMailbox
15/26 Test #15: test_Imap_Tasks_ObtainSynchronizedMailbox ...   Passed   94.17 sec
      Start 16: test_Imap_Tasks_OpenConnection
16/26 Test #16: test_Imap_Tasks_OpenConnection ..............   Passed    0.42 sec
      Start 17: test_Imap_Threading
17/26 Test #17: test_Imap_Threading .........................   Passed  284.82 sec
      Start 18: test_Imap_BodyParts
18/26 Test #18: test_Imap_BodyParts .........................   Passed    0.71 sec
      Start 19: test_Imap_Offline
19/26 Test #19: test_Imap_Offline ...........................   Passed    0.22 sec
      Start 20: test_Imap_CopyAndFlagOperations
20/26 Test #20: test_Imap_CopyAndFlagOperations .............   Passed    0.23 sec
      Start 21: test_Rfc5322
21/26 Test #21: test_Rfc5322 ................................   Passed    0.07 sec
      Start 22: test_RingBuffer
22/26 Test #22: test_RingBuffer .............................   Passed    0.06 sec
      Start 23: test_SenderIdentitiesModel
23/26 Test #23: test_SenderIdentitiesModel ..................   Passed    0.20 sec
      Start 24: test_SqlCache
24/26 Test #24: test_SqlCache ...............................   Passed    0.22 sec
      Start 25: test_algorithms
25/26 Test #25: test_algorithms .............................   Passed    0.06 sec
      Start 26: test_rfccodecs
26/26 Test #26: test_rfccodecs ..............................   Passed    0.13 sec

92% tests passed, 2 tests failed out of 26

Total Test time (real) = 386.55 sec

The following tests FAILED:
	  1 - test_Composer_Submission (SEGFAULT)
	  3 - test_Html_formatting (SEGFAULT)
Errors while running CTest
Comment 3 Daniel Holbach 2014-04-07 14:27:00 UTC
We can probably close this bug. The tests pass just fine on real HW. The problem might be missing timer implementation in qemu-armhf(?).

For now an approach might be to just disable the two tests on armhf in the test package and run them on all other architectures.
Comment 4 Jan Kundrát 2014-04-07 14:35:35 UTC
The tests which are failing have, AFAIK, nothing to do with timers; the timing -related failure was in the test_Imap_Idle and it was worked around by not overloading the system by concurrent test execution.

Anyway, I'm fine with closing this one :).