Uploaded image for project: 'ONOS'
  1. ONOS
  2. ONOS-2846

Link Detection Inconsistent

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: None
    • Labels:
      None

      Description

      ONOS sometimes fails to detect all links in a mininet network regularly but not consistently. Detected in HA and FUNC tests. Below is debug information from FUNCintent afflicted with the problem.

      Note: Problem encountered both before and after a topology change. The number and type of links found in each is variable. Additionally, re-runs occasionally yield different results, including a match vs a mismatch.

      015-09-08 13:46:52,949 - FUNCintent - INFO - Command 'devices -j' sent to ONOScli1.
      2015-09-08 13:46:52,950 - FUNCintent - ERROR - [{"id":"of:0000000000000001","type":"SWITCH","available":true,"role":"MASTER","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"1","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42296"}},{"id":"of:0000000000000002","type":"SWITCH","available":true,"role":"MASTER","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"2","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42298"}},{"id":"of:0000000000000003","type":"SWITCH","available":true,"role":"STANDBY","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"3","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42162"}},{"id":"of:0000000000000004","type":"SWITCH","available":true,"role":"STANDBY","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"4","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:35235"}},{"id":"of:0000000000000005","type":"SWITCH","available":true,"role":"STANDBY","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"5","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42308"}},{"id":"of:0000000000000006","type":"SWITCH","available":true,"role":"STANDBY","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"6","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42171"}},{"id":"of:0000000000000007","type":"SWITCH","available":true,"role":"MASTER","mfr":"Nicira, Inc.","hw":"Open vSwitch","sw":"2.3.0","serial":"None","chassisId":"7","annotations":{"protocol":"OF_13","channelId":"10.128.50.10:42315"}}]
      2015-09-08 13:46:52,951 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,951 - FUNCintent - INFO - ONOS1 Switches view is correct
      2015-09-08 13:46:52,952 - FUNCintent - INFO - Assertion Passed
      2015-09-08 13:46:52,952 - FUNCintent - DEBUG - ONOS links
      2015-09-08 13:46:52,952 - FUNCintent - DEBUG - [{u'src':

      {u'device': u'of:0000000000000006', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000007', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'2'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'dst':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000006', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}]
      2015-09-08 13:46:52,952 - FUNCintent - ERROR - Mininet has 10 bidirectional links and ONOS has 10 unidirectional links
      2015-09-08 13:46:52,952 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/1 -> 0000000000000001/1
      2015-09-08 13:46:52,952 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/1 -> 0000000000000001/2
      2015-09-08 13:46:52,953 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/2 -> 0000000000000003/2
      2015-09-08 13:46:52,953 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/2 -> 0000000000000002/2
      2015-09-08 13:46:52,953 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/3 -> 0000000000000005/10
      2015-09-08 13:46:52,953 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/4 -> 0000000000000006/9
      2015-09-08 13:46:52,953 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/3 -> 0000000000000004/2
      2015-09-08 13:46:52,954 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/2 -> 0000000000000003/3
      2015-09-08 13:46:52,954 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/4 -> 0000000000000006/10
      2015-09-08 13:46:52,954 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/3 -> 0000000000000007/9
      2015-09-08 13:46:52,954 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,954 - FUNCintent - ERROR - ONOS1 links view is incorrect

      [REPORT] [08 Sep 2015 13:46:52] ONOS1 links view is incorrect
      2015-09-08 13:46:52,955 - FUNCintent - WARNING - Assertion Failed
      2015-09-08 13:46:52,955 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,955 - FUNCintent - INFO - ONOS1 hosts exist in Mininet
      2015-09-08 13:46:52,955 - FUNCintent - INFO - Assertion Passed
      2015-09-08 13:46:52,956 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,956 - FUNCintent - INFO - ONOS2 Switches view is correct
      2015-09-08 13:46:52,956 - FUNCintent - INFO - Assertion Passed
      2015-09-08 13:46:52,957 - FUNCintent - DEBUG - ONOS links
      2015-09-08 13:46:52,957 - FUNCintent - DEBUG - [{u'src':

      {u'device': u'of:0000000000000006', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000007', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'2'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'dst':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000006', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}]
      2015-09-08 13:46:52,957 - FUNCintent - ERROR - Mininet has 10 bidirectional links and ONOS has 10 unidirectional links
      2015-09-08 13:46:52,957 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/1 -> 0000000000000001/1
      2015-09-08 13:46:52,957 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/1 -> 0000000000000001/2
      2015-09-08 13:46:52,957 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/2 -> 0000000000000003/2
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/2 -> 0000000000000002/2
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/3 -> 0000000000000005/10
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/4 -> 0000000000000006/9
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/3 -> 0000000000000004/2
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/2 -> 0000000000000003/3
      2015-09-08 13:46:52,958 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/4 -> 0000000000000006/10
      2015-09-08 13:46:52,959 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/3 -> 0000000000000007/9
      2015-09-08 13:46:52,959 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,959 - FUNCintent - ERROR - ONOS2 links view is incorrect

      [REPORT] [08 Sep 2015 13:46:52] ONOS2 links view is incorrect
      2015-09-08 13:46:52,959 - FUNCintent - WARNING - Assertion Failed
      2015-09-08 13:46:52,959 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,960 - FUNCintent - INFO - ONOS2 hosts exist in Mininet
      2015-09-08 13:46:52,960 - FUNCintent - INFO - Assertion Passed
      2015-09-08 13:46:52,961 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,961 - FUNCintent - INFO - ONOS3 Switches view is correct
      2015-09-08 13:46:52,961 - FUNCintent - INFO - Assertion Passed
      2015-09-08 13:46:52,961 - FUNCintent - DEBUG - ONOS links
      2015-09-08 13:46:52,961 - FUNCintent - DEBUG - [{u'src':

      {u'device': u'of:0000000000000006', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000007', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'2'}

      , u'dst':

      {u'device': u'of:0000000000000003', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'10'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'dst':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000004', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'3'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'dst':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000006', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000005', u'port': u'9'}

      , u'dst':

      {u'device': u'of:0000000000000001', u'port': u'4'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}, {u'src':

      {u'device': u'of:0000000000000001', u'port': u'1'}

      , u'dst':

      {u'device': u'of:0000000000000002', u'port': u'1'}

      , u'type': u'DIRECT', u'state': u'ACTIVE'}]
      2015-09-08 13:46:52,962 - FUNCintent - ERROR - Mininet has 10 bidirectional links and ONOS has 10 unidirectional links
      2015-09-08 13:46:52,962 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/1 -> 0000000000000001/1
      2015-09-08 13:46:52,962 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/1 -> 0000000000000001/2
      2015-09-08 13:46:52,962 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/2 -> 0000000000000003/2
      2015-09-08 13:46:52,963 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/2 -> 0000000000000002/2
      2015-09-08 13:46:52,963 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/3 -> 0000000000000005/10
      2015-09-08 13:46:52,963 - FUNCintent - ERROR - ONOS does not have the link 0000000000000002/4 -> 0000000000000006/9
      2015-09-08 13:46:52,963 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/3 -> 0000000000000004/2
      2015-09-08 13:46:52,963 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/2 -> 0000000000000003/3
      2015-09-08 13:46:52,964 - FUNCintent - ERROR - ONOS does not have the link 0000000000000003/4 -> 0000000000000006/10
      2015-09-08 13:46:52,964 - FUNCintent - ERROR - ONOS does not have the link 0000000000000004/3 -> 0000000000000007/9
      2015-09-08 13:46:52,964 - FUNCintent - INFO - Verifying the Expected is equal to the actual or not using assert_equal
      2015-09-08 13:46:52,964 - FUNCintent - ERROR - ONOS3 links view is incorrect

      [REPORT] [08 Sep 2015 13:46:52] ONOS3 links view is incorrect
      2015-09-08 13:46:52,965 - FUNCintent - WARNING - Assertion Failed

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            • Assignee:
              austin Austin Schafer
              Reporter:
              austin Austin Schafer
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: