Skip to content
This repository was archived by the owner on Mar 3, 2023. It is now read-only.

[DO NOT MERGE] Fix and re-enable MultiSpoutsMultiTasks test#1687

Open
objmagic wants to merge 12 commits intoapache:masterfrom
objmagic:travis-ci-integration-test-debugging
Open

[DO NOT MERGE] Fix and re-enable MultiSpoutsMultiTasks test#1687
objmagic wants to merge 12 commits intoapache:masterfrom
objmagic:travis-ci-integration-test-debugging

Conversation

@objmagic
Copy link
Contributor

@objmagic objmagic commented Feb 1, 2017

DO NOT MERGE

Recently our Travis CI repeatedly fails on integration test MultiSpoutsMultiTasks.

Can't reproduce Travis CI integration test error locally, so open a PR here to really test Travis CI.

@objmagic objmagic added the test label Feb 1, 2017
@objmagic objmagic self-assigned this Feb 1, 2017
@maosongfu
Copy link
Contributor

Let's also log when an instance receives the tuples from stmgr, which is here:
https://github.com/twitter/heron/blob/master/heron/instance/src/java/com/twitter/heron/network/StreamManagerClient.java#L291

@objmagic
Copy link
Contributor Author

objmagic commented Feb 4, 2017

finally got some useful logs. See attachment.
ci_logs.tar.gz

@objmagic
Copy link
Contributor Author

objmagic commented Feb 4, 2017

Each spout sends out ten tuples exactly.

1691-investigation $ grep "taskid" container_1_ab-spout-1_2.log | wc -l
      10
1691-investigation $ grep "taskid" container_1_ab-spout-1_3.log | wc -l
      10
1691-investigation $ grep "taskid" container_1_ab-spout-2_4.log | wc -l
      10
1691-investigation $ grep "taskid" container_1_ab-spout-2_5.log | wc -l
      10
1691-investigation $ grep "taskid" container_1_ab-spout-2_6.log | wc -l
      10
1691-investigation $ grep "taskid" container_1_ab-spout-1_1.log | wc -l
      10

@objmagic
Copy link
Contributor Author

objmagic commented Feb 4, 2017

grep "taskid" container_1_identity-bolt_8.log | wc -l gives us 64.
grep "taskid" container_1_identity-bolt_8.log | grep 5 | wc -l gives us 14.

container_1_ab-spout-2_5 has some problem?

@objmagic
Copy link
Contributor Author

objmagic commented Feb 4, 2017

ci.log:[2017-02-03 23:03:01 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_79af2046-68eb-401e-99c3-f850dd546d69
ci.log:[2017-02-03 23:03:01 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_9ccb1355-95c0-42bf-a8d8-bffe5a8b60b7
ci.log:[2017-02-03 23:03:31 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_79af2046-68eb-401e-99c3-f850dd546d69
ci.log:[2017-02-03 23:03:31 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_9ccb1355-95c0-42bf-a8d8-bffe5a8b60b7
container_1_ab-spout-2_5.log:[2017-02-03 23:03:01 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_79af2046-68eb-401e-99c3-f850dd546d69
container_1_ab-spout-2_5.log:[2017-02-03 23:03:01 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_9ccb1355-95c0-42bf-a8d8-bffe5a8b60b7
container_1_ab-spout-2_5.log:[2017-02-03 23:03:31 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_79af2046-68eb-401e-99c3-f850dd546d69
container_1_ab-spout-2_5.log:[2017-02-03 23:03:31 +0000] com.twitter.heron.integration_test.core.IntegrationTestSpout INFO:  Received a fail with MessageId: __integration_test_mock_message_id_9ccb1355-95c0-42bf-a8d8-bffe5a8b60b7

container_1_ab-spout-2_5 receives 4 fails and re-emits 4 tuples

@objmagic objmagic changed the title [DO NOT MERGE] MultiSpoutsMultiTasks debugging [DO NOT MERGE] Fix and re-enable MultiSpoutsMultiTasks test Feb 7, 2017
@objmagic objmagic removed their assignment Nov 8, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants