-
Type: Bug
-
Status: Closed (View Workflow)
-
Priority: Blocker
-
Resolution: Done
-
Affects Version/s: 1.3.0
-
Fix Version/s: 1.3.0
-
Component/s: None
-
Labels:
-
Environment:
commit 629b99ed41d4d8f3bb273811833b84b3cb0907fe (HEAD, origin/master, origin/HEAD, master)
-
Epic Link:
-
Sprint:Drake Sprint 2 (7/27-8/14) 2
I noticed this in the logs of only one onos node out of a 7 node cluster. It looks like this caused one of the switches to not be reassigned as intended:
2015-07-28 13:04:10,494 - HAsanity - ERROR - Error, controller 10.128.30.17 is not master of device of:0000000000006027. Master is u'10.128.30.14'.
2015-07-28 13:03:49,564 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'device-role of:0000000000006024 10.128.30.17 master' 2015-07-28 13:03:49,645 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006020 ... initializing driver 2015-07-28 13:03:50,014 | INFO | ransfer-executor | ConsistentDeviceMastershipStore | 79 - org.onosproject.onos-core-dist - 1.3.0.SNAPSHOT | Forwarding request to relinquish mastership for device of:0000000000006021 to 10.128.30.12 2015-07-28 13:03:50,015 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'devices -j' 2015-07-28 13:03:50,019 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006021 ... initializing driver 2015-07-28 13:03:50,019 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006021 ... initializing driver 2015-07-28 13:03:50,459 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006021 ... initializing driver 2015-07-28 13:03:50,465 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'device-role of:0000000000006025 10.128.30.17 master' 2015-07-28 13:03:50,465 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006020 ... initializing driver 2015-07-28 13:03:50,620 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006021 ... initializing driver 2015-07-28 13:03:50,916 | INFO | ransfer-executor | ConsistentDeviceMastershipStore | 79 - org.onosproject.onos-core-dist - 1.3.0.SNAPSHOT | Forwarding request to relinquish mastership for device of:0000000000006022 to 10.128.30.14 2015-07-28 13:03:50,917 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'devices -j' 2015-07-28 13:03:50,923 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006022 ... initializing driver 2015-07-28 13:03:50,926 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006022 ... initializing driver 2015-07-28 13:03:51,192 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006022 ... initializing driver 2015-07-28 13:03:51,367 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'device-role of:0000000000006026 10.128.30.17 master' 2015-07-28 13:03:51,818 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'devices -j' 2015-07-28 13:03:51,823 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006023 ... initializing driver 2015-07-28 13:03:51,823 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006023 ... initializing driver 2015-07-28 13:03:51,825 | INFO | ew I/O worker #7 | ntrollerImpl$OpenFlowSwitchAgent | 160 - org.onosproject.onos-of-ctl - 1.3.0.SNAPSHOT | Transitioned switch 00:00:00:00:00:00:60:23 to EQUAL 2015-07-28 13:03:52,000 | INFO | event-dispatch-0 | FlowObjectiveManager | 77 - org.onosproject.onos-core-net - 1.3.0.SNAPSHOT | Driver ovs bound to device of:0000000000006023 ... initializing driver 2015-07-28 13:03:52,269 | INFO | h for user karaf | command | 154 - org.apache.karaf.log.command - 3.0.3 | Sending CLI command: 'device-role of:0000000000006027 10.128.30.17 master' 2015-07-28 13:03:52,428 | ERROR | h for user karaf | ShellUtil | 25 - org.apache.karaf.shell.console - 3.0.3 | Exception caught while executing command org.onosproject.store.service.ConsistentMapException: com.esotericsoftware.kryo.KryoException: Unable to find class: 10.128.30.12 at org.onosproject.store.consistent.impl.DefaultConsistentMap.complete(DefaultConsistentMap.java:178) at org.onosproject.store.consistent.impl.DefaultConsistentMap.computeIf(DefaultConsistentMap.java:103) at org.onosproject.store.consistent.impl.DistributedLeadershipManager.makeTopCandidate(DistributedLeadershipManager.java:342) at org.onosproject.store.mastership.impl.ConsistentDeviceMastershipStore.setMaster(ConsistentDeviceMastershipStore.java:253) at org.onosproject.cluster.impl.MastershipManager.setRole(MastershipManager.java:127) at org.onosproject.cli.net.DeviceRoleCommand.execute(DeviceRoleCommand.java:50) at org.onosproject.cli.AbstractShellCommand.doExecute(AbstractShellCommand.java:129) at org.apache.karaf.shell.console.AbstractAction.execute(AbstractAction.java:33)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.karaf.shell.commands.basic.AbstractCommand.execute(AbstractCommand.java:33)[25:org.apache.karaf.shell.console:3.0.3] at Proxyba514538_1bbc_4b36_aa9e_4ac6331a9c6c.execute(Unknown Source)[:] at Proxyba514538_1bbc_4b36_aa9e_4ac6331a9c6c.execute(Unknown Source)[:] at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:78)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:477)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:403)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)[25:org.apache.karaf.shell.console:3.0.3] at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:92) at org.apache.karaf.shell.console.impl.jline.ConsoleImpl.run(ConsoleImpl.java:208) at org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1.runConsole(ShellFactoryImpl.java:158)[88:org.apache.karaf.shell.ssh:3.0.3] at org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1$1.run(ShellFactoryImpl.java:133) at java.security.AccessController.doPrivileged(Native Method)[:1.8.0_45] at org.apache.karaf.jaas.modules.JaasHelper.doAs(JaasHelper.java:57)[26:org.apache.karaf.jaas.modules:3.0.3] at org.apache.karaf.shell.ssh.ShellFactoryImpl$ShellImpl$1.run(ShellFactoryImpl.java:129)[88:org.apache.karaf.shell.ssh:3.0.3] Caused by: com.esotericsoftware.kryo.KryoException: Unable to find class: 10.128.30.12 at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:666)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:777)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)[58:com.esotericsoftware.kryo:3.0.0] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:786)[58:com.esotericsoftware.kryo:3.0.0] at org.onlab.util.KryoNamespace.deserialize(KryoNamespace.java:307)[73:org.onosproject.onlab-misc:1.3.0.SNAPSHOT] at org.onosproject.store.service.Serializer$1.decode(Serializer.java:70)[75:org.onosproject.onos-api:1.3.0.SNAPSHOT] at org.onosproject.store.consistent.impl.DefaultAsyncConsistentMap$$Lambda$230/64885485.apply(Unknown Source)[79:org.onosproject.onos-core-dist:1.3.0.SNAPSHOT] at org.onosproject.store.service.Versioned.map(Versioned.java:99)[75:org.onosproject.onos-api:1.3.0.SNAPSHOT] at org.onosproject.store.consistent.impl.UpdateResult.map(UpdateResult.java:71)[79:org.onosproject.onos-core-dist:1.3.0.SNAPSHOT] at org.onosproject.store.consistent.impl.DefaultAsyncConsistentMap.lambda$updateAndGet$103(DefaultAsyncConsistentMap.java:426)[79:org.onosproject.onos-core-dist:1.3.0.SNAPSHOT] at org.onosproject.store.consistent.impl.DefaultAsyncConsistentMap$$Lambda$221/285733973.apply(Unknown Source)[79:org.onosproject.onos-core-dist:1.3.0.SNAPSHOT] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)[:1.8.0_45] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)[:1.8.0_45] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)[:1.8.0_45] at java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:561)[:1.8.0_45] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:580)[:1.8.0_45] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)[:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_45] at java.lang.Thread.run(Thread.java:745)[:1.8.0_45] Caused by: java.lang.ClassNotFoundException: 10.128.30.12 not found by com.esotericsoftware.kryo [58] at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1532)[org.apache.felix.framework-4.2.1.jar:] at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)[org.apache.felix.framework-4.2.1.jar:] at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)[:1.8.0_45] at java.lang.Class.forName0(Native Method)[:1.8.0_45] at java.lang.Class.forName(Class.java:348)[:1.8.0_45] at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)[58:com.esotericsoftware.kryo:3.0.0] ... 22 more