-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: 1.5.0, 1.5.1, 1.6.0, 1.5.2, 1.7.0
-
Component/s: None
-
Labels:
-
Environment:
Single node cluster, onos 1.5, single switch topology
-
Story Points:3
> Created a single switch topology and assigned to a single node onos controller.
> Configured the ovs switch in such a way that, the maximum number flows accepted by ovs switch is limited to 6.
> I tried adding simple point intent to the switch. After the 6th flow, the switch rejected the flow witha flowmodfailed error.
> The flows were not installed in the switch, but still the intent went in to an installed state.
> The flow addition was continously happening in an infinite loop as it is in pending_add state and will never be installed.
Observation:
===================================
Intent Installed stage.
Flow in Pending add state
Getting continous OFFlowModFailedErrorMsgVer13 with error code as TABLE_FULL
flowmodfailed not handled properly
More Observations:
===================================
Issue was happening mainly because of context switching between the threads which executes barrier reply and of-error messages. Sometimes the barrier reply thread executes the process much faster than the thread which handles of-error message. In that case barrier reply thread invalidates the cache much earlier and case for ERROR in switch condition cannot append the failed transactions to failed stage. because of this intent will go in to installed state.
Logs attached.
Exceptions :
===================================
tch org.onosproject.driver.handshaker.NiciraSwitchHandshaker [/192.168.56.107:52020 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE
2016-06-24 23:39:22,038 | WARN | of-event-stats-9 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Received error message OFFlowModFailedErrorMsgVer13(xid=0, code=TABLE_FULL, data=[unparsed: 04 0e 00 58 00 00 00 00 00 4c 00 00 46 37 b7 df 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c8 ff ff ff ff ff ff ff ff ff ff ff ff 00 01 00 00 00 01 00 0a 80 00 0a 02 88 cc 00 00 00 00 00 00]) from 00:00:00:00:00:00:00:01
2016-06-24 23:39:22,041 | ERROR | of-event-stats-9 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Flow installation failed but switch didn't tell us which one.
2016-06-24 23:39:22,038 | WARN | of-event-stats-7 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Received error message OFFlowModFailedErrorMsgVer13(xid=0, code=TABLE_FULL, data=[unparsed: 04 0e 00 68 00 00 00 00 00 25 00 00 46 47 d3 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 41 ff ff ff ff ff ff ff ff ff ff ff ff 00 01 00 00 00 01 00 0c 80 00 00 04 00 00 00 01 00 00 00 00]) from 00:00:00:00:00:00:00:01
2016-06-24 23:39:22,042 | ERROR | of-event-stats-7 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Flow installation failed but switch didn't tell us which one.
2016-06-24 23:39:27,035 | ERROR | ew I/O worker #2 | OFChannelHandler | 167 - org.onosproject.onos-of-ctl - 1.5.2.SNAPSHOT | OFFlowModFailedErrorMsgVer13(xid=0, code=TABLE_FULL, data=[unparsed: 04 0e 00 58 00 00 00 00 00 4c 00 00 46 37 b7 df 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c8 ff ff ff ff ff ff ff ff ff ff ff ff 00 01 00 00 00 01 00 0a 80 00 0a 02 88 cc 00 00 00 00 00 00]) from switch org.onosproject.driver.handshaker.NiciraSwitchHandshaker [/192.168.56.107:52020 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE
2016-06-24 23:39:27,035 | WARN | f-event-stats-18 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Received error message OFFlowModFailedErrorMsgVer13(xid=0, code=TABLE_FULL, data=[unparsed: 04 0e 00 58 00 00 00 00 00 4c 00 00 46 37 b7 df 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c8 ff ff ff ff ff ff ff ff ff ff ff ff 00 01 00 00 00 01 00 0a 80 00 0a 02 88 cc 00 00 00 00 00 00]) from 00:00:00:00:00:00:00:01
2016-06-24 23:39:27,036 | ERROR | f-event-stats-18 | OpenFlowRuleProvider | 170 - org.onosproject.onos-of-provider-flow - 1.5.2.SNAPSHOT | Flow installation failed but switch didn't tell us which one.