[ 252.254749][ T1990] 8021q: 802.1Q VLAN Support v1.8 [ 253.661374][ T2041] br1: port 1(vx10) entered blocking state [ 253.661716][ T2041] br1: port 1(vx10) entered disabled state [ 253.661899][ T2041] vx10: entered allmulticast mode [ 253.663386][ T2041] vx10: entered promiscuous mode [ 253.664642][ T2041] br1: port 1(vx10) entered blocking state [ 253.664863][ T2041] br1: port 1(vx10) entered forwarding state [ 253.794026][ T2046] br1: port 2(vx20) entered blocking state [ 253.794221][ T2046] br1: port 2(vx20) entered disabled state [ 253.794403][ T2046] vx20: entered allmulticast mode [ 253.795899][ T2046] vx20: entered promiscuous mode [ 253.796314][ T2046] br1: port 2(vx20) entered blocking state [ 253.796488][ T2046] br1: port 2(vx20) entered forwarding state [ 253.851964][ T2048] br1: port 3(veth1) entered blocking state [ 253.852262][ T2048] br1: port 3(veth1) entered disabled state [ 253.852449][ T2048] veth1: entered allmulticast mode [ 253.854219][ T2048] veth1: entered promiscuous mode [ 253.881032][ T70] br1: port 3(veth1) entered blocking state [ 253.881270][ T70] br1: port 3(veth1) entered forwarding state [ 253.986200][ T2053] br1: port 4(veth2) entered blocking state [ 253.986419][ T2053] br1: port 4(veth2) entered disabled state [ 253.987228][ T2053] veth2: entered allmulticast mode [ 253.988738][ T2053] veth2: entered promiscuous mode [ 254.025339][ T46] br1: port 4(veth2) entered blocking state [ 254.025524][ T46] br1: port 4(veth2) entered forwarding state [ 255.218135][ T2108] ip (2108) used greatest stack depth: 23856 bytes left [ 255.243507][ T2109] br2: port 1(w1) entered blocking state [ 255.244276][ T2109] br2: port 1(w1) entered disabled state [ 255.249947][ T2109] w1: entered allmulticast mode [ 255.251445][ T2109] w1: entered promiscuous mode [ 255.278171][ T2110] br2: port 1(w1) entered blocking state [ 255.278345][ T2110] br2: port 1(w1) entered forwarding state [ 255.315767][ T46] br2: port 1(w1) entered disabled state [ 255.474972][ T2117] br2: port 2(vx10) entered blocking state [ 255.475168][ T2117] br2: port 2(vx10) entered disabled state [ 255.475355][ T2117] vx10: entered allmulticast mode [ 255.476992][ T2117] vx10: entered promiscuous mode [ 255.477365][ T2117] br2: port 2(vx10) entered blocking state [ 255.477540][ T2117] br2: port 2(vx10) entered forwarding state [ 255.686026][ T2124] br2: port 3(vx20) entered blocking state [ 255.686251][ T2124] br2: port 3(vx20) entered disabled state [ 255.686440][ T2124] vx20: entered allmulticast mode [ 255.688260][ T2124] vx20: entered promiscuous mode [ 255.688612][ T2124] br2: port 3(vx20) entered blocking state [ 255.688800][ T2124] br2: port 3(vx20) entered forwarding state [ 255.910103][ T92] br2: port 1(w1) entered blocking state [ 255.910466][ T92] br2: port 1(w1) entered forwarding state [ 256.710880][ T2165] br2: port 1(w1) entered blocking state [ 256.711057][ T2165] br2: port 1(w1) entered disabled state [ 256.711204][ T2165] w1: entered allmulticast mode [ 256.712757][ T2165] w1: entered promiscuous mode [ 256.743758][ T2166] br2: port 1(w1) entered blocking state [ 256.743922][ T2166] br2: port 1(w1) entered forwarding state [ 256.935518][ T2173] br2: port 2(vx10) entered blocking state [ 256.936295][ T2173] br2: port 2(vx10) entered disabled state [ 256.936489][ T2173] vx10: entered allmulticast mode [ 256.938053][ T2173] vx10: entered promiscuous mode [ 256.938451][ T2173] br2: port 2(vx10) entered blocking state [ 256.938643][ T2173] br2: port 2(vx10) entered forwarding state [ 257.139939][ T2180] br2: port 3(vx20) entered blocking state [ 257.140182][ T2180] br2: port 3(vx20) entered disabled state [ 257.140377][ T2180] vx20: entered allmulticast mode [ 257.142122][ T2180] vx20: entered promiscuous mode [ 257.142495][ T2180] br2: port 3(vx20) entered blocking state [ 257.142692][ T2180] br2: port 3(vx20) entered forwarding state [ 257.331322][ T46] br2: port 1(w1) entered disabled state [ 257.386494][ T92] br2: port 1(w1) entered blocking state [ 257.386675][ T92] br2: port 1(w1) entered forwarding state [ 265.723994][ T2276] GACT probability NOT on [ 352.924351][ T69] vx20: left allmulticast mode [ 352.924631][ T69] vx20: left promiscuous mode [ 352.924927][ T69] br2: port 3(vx20) entered disabled state [ 352.928058][ T69] vx10: left allmulticast mode [ 352.928243][ T69] vx10: left promiscuous mode [ 352.928507][ T69] br2: port 2(vx10) entered disabled state [ 352.931057][ T69] w1: left allmulticast mode [ 352.931240][ T69] w1: left promiscuous mode [ 352.931500][ T69] br2: port 1(w1) entered disabled state [ 352.937024][ T69] [ 352.937093][ T69] ====================================================== [ 352.937210][ T69] WARNING: possible circular locking dependency detected [ 352.937336][ T69] 7.2.0-rc6-virtme #1 Not tainted [ 352.937428][ T69] ------------------------------------------------------ [ 352.937548][ T69] kworker/u16:1/69 is trying to acquire lock: [ 352.937670][ T69] ff1100000d4b3508 ((&n->timer)){+.-.}-{0:0}, at: __timer_delete_sync+0xdb/0x130 [ 352.937844][ T69] [ 352.937844][ T69] but task is already holding lock: [ 352.937984][ T69] ff11000010333b58 (&tbl->lock){+.-.}-{3:3}, at: __neigh_ifdown.isra.0+0xa1/0xea0 [ 352.938151][ T69] [ 352.938151][ T69] which lock already depends on the new lock. [ 352.938151][ T69] [ 352.938339][ T69] [ 352.938339][ T69] the existing dependency chain (in reverse order) is: [ 352.938496][ T69] [ 352.938496][ T69] -> #1 (&tbl->lock){+.-.}-{3:3}: [ 352.938640][ T69] __lock_acquire+0x518/0xc20 [ 352.938736][ T69] lock_acquire.part.0+0xbc/0x260 [ 352.938829][ T69] _raw_spin_lock_bh+0x38/0x50 [ 352.938924][ T69] ___neigh_create+0x2be/0x1070 [ 352.939016][ T69] ip6_finish_output2+0xb64/0x1380 [ 352.939111][ T69] ip6_finish_output+0x646/0xda0 [ 352.939199][ T69] ip6_output+0x23f/0x7f0 [ 352.939294][ T69] ndisc_send_skb+0xba3/0x1520 [ 352.939381][ T69] ndisc_send_ns+0xa9/0x120 [ 352.939463][ T69] ndisc_solicit+0x314/0x480 [ 352.939546][ T69] neigh_probe+0xb3/0x100 [ 352.939631][ T69] neigh_timer_handler+0x191/0x1210 [ 352.939715][ T69] call_timer_fn+0x163/0x4c0 [ 352.939799][ T69] __run_timers+0x68f/0xaa0 [ 352.939886][ T69] run_timer_softirq+0xf0/0x160 [ 352.939970][ T69] handle_softirqs+0x1d8/0x8f0 [ 352.940100][ T69] __irq_exit_rcu+0x103/0x1c0 [ 352.940185][ T69] irq_exit_rcu+0xe/0x30 [ 352.940320][ T69] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 352.940426][ T69] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 352.940528][ T69] pv_native_safe_halt+0xf/0x10 [ 352.940613][ T69] default_idle+0x9/0x10 [ 352.940696][ T69] default_idle_call+0x6e/0xb0 [ 352.940780][ T69] cpuidle_idle_call.constprop.0+0x237/0x410 [ 352.940928][ T69] do_idle+0xd8/0x180 [ 352.941013][ T69] cpu_startup_entry+0x53/0x70 [ 352.941100][ T69] start_secondary+0x204/0x2b0 [ 352.941232][ T69] common_startup_64+0x13e/0x148 [ 352.941320][ T69] [ 352.941320][ T69] -> #0 ((&n->timer)){+.-.}-{0:0}: [ 352.941538][ T69] check_prev_add+0xd8/0xe90 [ 352.941630][ T69] validate_chain+0x6b8/0xc20 [ 352.941713][ T69] __lock_acquire+0x518/0xc20 [ 352.941801][ T69] lock_acquire.part.0+0xbc/0x260 [ 352.941884][ T69] __timer_delete_sync+0xea/0x130 [ 352.941969][ T69] neigh_flush_one+0x4bf/0x650 [ 352.942055][ T69] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 352.942207][ T69] neigh_ifdown+0x10/0x20 [ 352.942300][ T69] rt6_disable_ip+0x183/0x200 [ 352.942384][ T69] addrconf_ifdown.isra.0+0x103/0x1670 [ 352.942491][ T69] addrconf_notify+0x2c8/0xf30 [ 352.942577][ T69] notifier_call_chain+0xae/0x300 [ 352.942665][ T69] netif_close_many+0x366/0x710 [ 352.942747][ T69] unregister_netdevice_many_notify+0x671/0x19a0 [ 352.942853][ T69] ops_undo_list+0x615/0x8f0 [ 352.942945][ T69] cleanup_net+0x431/0x890 [ 352.943080][ T69] process_one_work+0xdf8/0x1410 [ 352.943213][ T69] worker_thread+0x4f1/0xd60 [ 352.943302][ T69] kthread+0x367/0x460 [ 352.943385][ T69] ret_from_fork+0x474/0x6b0 [ 352.943468][ T69] ret_from_fork_asm+0x11/0x20 [ 352.943597][ T69] [ 352.943597][ T69] other info that might help us debug this: [ 352.943597][ T69] [ 352.943761][ T69] Possible unsafe locking scenario: [ 352.943761][ T69] [ 352.943906][ T69] CPU0 CPU1 [ 352.943993][ T69] ---- ---- [ 352.944077][ T69] lock(&tbl->lock); [ 352.944142][ T69] lock((&n->timer)); [ 352.944253][ T69] lock(&tbl->lock); [ 352.944357][ T69] lock((&n->timer)); [ 352.944468][ T69] [ 352.944468][ T69] *** DEADLOCK *** [ 352.944468][ T69] [ 352.944591][ T69] 5 locks held by kworker/u16:1/69: [ 352.944676][ T69] #0: ff110000019d9d40 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0xd55/0x1410 [ 352.944868][ T69] #1: ffa00000004a7d10 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xdb7/0x1410 [ 352.945150][ T69] #2: ffffffff9309e3a8 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbf/0x890 [ 352.945397][ T69] #3: ffffffff930b9bc0 (rtnl_mutex){+.+.}-{4:4}, at: ops_undo_list+0x534/0x8f0 [ 352.945587][ T69] #4: ff11000010333b58 (&tbl->lock){+.-.}-{3:3}, at: __neigh_ifdown.isra.0+0xa1/0xea0 [ 352.945889][ T69] [ 352.945889][ T69] stack backtrace: [ 352.946064][ T69] CPU: 2 UID: 0 PID: 69 Comm: kworker/u16:1 Not tainted 7.2.0-rc6-virtme #1 PREEMPT(full) [ 352.946067][ T69] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 352.946069][ T69] Workqueue: netns cleanup_net [ 352.946073][ T69] Call Trace: [ 352.946074][ T69] [ 352.946076][ T69] dump_stack_lvl+0x6f/0xa0 [ 352.946080][ T69] print_circular_bug.cold+0x38/0x46 [ 352.946083][ T69] check_noncircular+0x171/0x1a0 [ 352.946086][ T69] check_prev_add+0xd8/0xe90 [ 352.946088][ T69] ? look_up_lock_class+0x65/0x150 [ 352.946090][ T69] ? check_prev_add+0xef/0xe90 [ 352.946093][ T69] validate_chain+0x6b8/0xc20 [ 352.946096][ T69] __lock_acquire+0x518/0xc20 [ 352.946098][ T69] lock_acquire.part.0+0xbc/0x260 [ 352.946100][ T69] ? __timer_delete_sync+0xdb/0x130 [ 352.946102][ T69] ? rcu_is_watching+0x15/0xd0 [ 352.946104][ T69] ? lock_acquire+0x13c/0x160 [ 352.946106][ T69] ? __timer_delete_sync+0xdb/0x130 [ 352.946108][ T69] __timer_delete_sync+0xea/0x130 [ 352.946109][ T69] ? __timer_delete_sync+0xdb/0x130 [ 352.946111][ T69] neigh_flush_one+0x4bf/0x650 [ 352.946113][ T69] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 352.946115][ T69] ? fib6_clean_tree+0xf2/0x130 [ 352.946117][ T69] ? fib6_walk+0x3d0/0x3d0 [ 352.946119][ T69] ? neigh_flush_one+0x650/0x650 [ 352.946121][ T69] ? find_held_lock+0x2b/0x80 [ 352.946123][ T69] ? fib6_ifup+0x220/0x220 [ 352.946124][ T69] ? __lock_release.isra.0+0x6b/0x1a0 [ 352.946127][ T69] ? __fib6_clean_all+0x149/0x290 [ 352.946128][ T69] ? __local_bh_enable_ip+0xa5/0x140 [ 352.946131][ T69] ? __fib6_clean_all+0x14e/0x290 [ 352.946133][ T69] neigh_ifdown+0x10/0x20 [ 352.946135][ T69] rt6_disable_ip+0x183/0x200 [ 352.946136][ T69] ? rt6_sync_down_dev+0x120/0x120 [ 352.946138][ T69] ? __neigh_ifdown.isra.0+0x988/0xea0 [ 352.946141][ T69] addrconf_ifdown.isra.0+0x103/0x1670 [ 352.946142][ T69] ? neigh_flush_one+0x650/0x650 [ 352.946145][ T69] ? addrconf_sysctl_addr_gen_mode+0xb10/0xb10 [ 352.946147][ T69] ? mark_held_locks+0x40/0x70 [ 352.946149][ T69] addrconf_notify+0x2c8/0xf30 [ 352.946151][ T69] ? team_port_get_rtnl+0x65/0xc0 [ 352.946154][ T69] notifier_call_chain+0xae/0x300 [ 352.946157][ T69] netif_close_many+0x366/0x710 [ 352.946160][ T69] ? __dev_close_many+0x530/0x530 [ 352.946162][ T69] ? netif_close_many_and_unlock+0x22/0x2c0 [ 352.946164][ T69] unregister_netdevice_many_notify+0x671/0x19a0 [ 352.946166][ T69] ? mark_held_locks+0x40/0x70 [ 352.946168][ T69] ? unregister_netdevice_queued+0x80/0x80 [ 352.946169][ T69] ? lockdep_hardirqs_on+0x8c/0x130 [ 352.946171][ T69] ? __call_rcu_common.constprop.0+0x3d1/0xa10 [ 352.946173][ T69] ? mutex_is_locked+0x1c/0x50 [ 352.946175][ T69] ? rtnl_is_locked+0x15/0x20 [ 352.946177][ T69] ? nexthop_net_exit_rtnl+0x1f/0x270 [ 352.946180][ T69] ops_undo_list+0x615/0x8f0 [ 352.946182][ T69] ? lockdep_hardirqs_on+0x8c/0x130 [ 352.946184][ T69] ? rtnl_net_dumpid_one+0x2d0/0x2d0 [ 352.946185][ T69] ? __lock_release.isra.0+0x6b/0x1a0 [ 352.946187][ T69] ? idr_destroy+0x121/0x250 [ 352.946191][ T69] cleanup_net+0x431/0x890 [ 352.946193][ T69] ? net_passive_dec+0x1c0/0x1c0 [ 352.946194][ T69] ? process_one_work+0xdb7/0x1410 [ 352.946196][ T69] ? lock_acquire+0x13c/0x160 [ 352.946198][ T69] ? rcu_is_watching+0x15/0xd0 [ 352.946199][ T69] process_one_work+0xdf8/0x1410 [ 352.946202][ T69] ? pwq_dec_nr_in_flight+0x710/0x710 [ 352.946203][ T69] ? lock_acquire.part.0+0xbc/0x260 [ 352.946206][ T69] worker_thread+0x4f1/0xd60 [ 352.946208][ T69] ? rescuer_thread+0x1320/0x1320 [ 352.946210][ T69] ? __kthread_parkme+0xbd/0x210 [ 352.946212][ T69] ? rescuer_thread+0x1320/0x1320 [ 352.946213][ T69] ? rescuer_thread+0x1320/0x1320 [ 352.946214][ T69] kthread+0x367/0x460 [ 352.946216][ T69] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 352.946218][ T69] ? kthread_affine_node+0x330/0x330 [ 352.946220][ T69] ret_from_fork+0x474/0x6b0 [ 352.946222][ T69] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 352.946230][ T69] ? __switch_to+0x5a3/0xe00 [ 352.946233][ T69] ? kthread_affine_node+0x330/0x330 [ 352.946234][ T69] ret_from_fork_asm+0x11/0x20 [ 352.946238][ T69] [ 353.024260][ T69] vx20: left allmulticast mode [ 353.024373][ T69] vx20: left promiscuous mode [ 353.024521][ T69] br2: port 3(vx20) entered disabled state [ 353.025983][ T69] vx10: left allmulticast mode [ 353.026097][ T69] vx10: left promiscuous mode [ 353.026259][ T69] br2: port 2(vx10) entered disabled state [ 353.027304][ T69] w1: left allmulticast mode [ 353.027402][ T69] w1: left promiscuous mode [ 353.027538][ T69] br2: port 1(w1) entered disabled state [ 353.493705][ T2504] br1: port 4(veth2) entered disabled state [ 353.509858][ T2506] veth2: left allmulticast mode [ 353.509975][ T2506] veth2: left promiscuous mode [ 353.510127][ T2506] br1: port 4(veth2) entered disabled state [ 353.578818][ T2510] br1: port 3(veth1) entered disabled state [ 353.601846][ T2511] veth1: left allmulticast mode [ 353.601949][ T2511] veth1: left promiscuous mode [ 353.602101][ T2511] br1: port 3(veth1) entered disabled state [ 353.638989][ T2513] vx20: left allmulticast mode [ 353.639109][ T2513] vx20: left promiscuous mode [ 353.639267][ T2513] br1: port 2(vx20) entered disabled state [ 353.718240][ T2517] vx10: left allmulticast mode [ 353.718347][ T2517] vx10: left promiscuous mode [ 353.718501][ T2517] br1: port 1(vx10) entered disabled state