[ 11.535547][ T207] ip (207) used greatest stack depth: 23672 bytes left [ 13.909826][ T290] br0: port 1(gw_l) entered blocking state [ 13.910461][ T290] br0: port 1(gw_l) entered disabled state [ 13.910842][ T290] gw_l: entered allmulticast mode [ 13.912667][ T290] gw_l: entered promiscuous mode [ 13.914333][ T290] br0: port 1(gw_l) entered blocking state [ 13.914761][ T290] br0: port 1(gw_l) entered forwarding state [ 14.198667][ T292] br0: port 2(amtg) entered blocking state [ 14.199138][ T292] br0: port 2(amtg) entered disabled state [ 14.199460][ T292] amtg: entered allmulticast mode [ 14.201287][ T292] amtg: entered promiscuous mode [ 15.107186][ T303] br0: port 2(amtg) entered blocking state [ 15.107524][ T303] br0: port 2(amtg) entered forwarding state [ 15.724007][ T310] amtr: entered allmulticast mode [ 15.724455][ T310] relay_gw: entered allmulticast mode [ 15.725084][ T310] relay_src: entered allmulticast mode [ 913.740796][ T12] [ 913.740891][ T12] ====================================================== [ 913.741009][ T12] WARNING: possible circular locking dependency detected [ 913.741120][ T12] 7.2.0-rc6-virtme #1 Not tainted [ 913.741210][ T12] ------------------------------------------------------ [ 913.741325][ T12] kworker/u16:0/12 is trying to acquire lock: [ 913.741437][ T12] ff1100000d8e2908 ((&n->timer)){+.-.}-{0:0}, at: __timer_delete_sync+0xdb/0x130 [ 913.741604][ T12] [ 913.741604][ T12] but task is already holding lock: [ 913.741732][ T12] ff11000009d22358 (&tbl->lock){+.-.}-{3:3}, at: __neigh_ifdown.isra.0+0xa1/0xea0 [ 913.741893][ T12] [ 913.741893][ T12] which lock already depends on the new lock. [ 913.741893][ T12] [ 913.742070][ T12] [ 913.742070][ T12] the existing dependency chain (in reverse order) is: [ 913.742220][ T12] [ 913.742220][ T12] -> #1 (&tbl->lock){+.-.}-{3:3}: [ 913.742357][ T12] __lock_acquire+0x518/0xc20 [ 913.742446][ T12] lock_acquire.part.0+0xbc/0x260 [ 913.742531][ T12] _raw_spin_lock_bh+0x38/0x50 [ 913.742622][ T12] ___neigh_create+0x2be/0x1070 [ 913.742709][ T12] ip6_finish_output2+0xb9a/0x12f0 [ 913.742800][ T12] ip6_finish_output+0x701/0xe80 [ 913.742886][ T12] ip6_output+0x23f/0x7f0 [ 913.742970][ T12] ip6_send_skb+0xee/0x350 [ 913.743055][ T12] icmp6_send+0x1728/0x1cd0 [ 913.743142][ T12] ip6_link_failure+0x2d/0x460 [ 913.743229][ T12] ndisc_error_report+0x97/0x1b0 [ 913.743320][ T12] neigh_invalidate+0x208/0x560 [ 913.743407][ T12] neigh_timer_handler+0x8e9/0x1210 [ 913.743492][ T12] call_timer_fn+0x163/0x4c0 [ 913.743577][ T12] __run_timers+0x68f/0xaa0 [ 913.743664][ T12] run_timer_softirq+0xf0/0x160 [ 913.743758][ T12] handle_softirqs+0x1d8/0x8f0 [ 913.743847][ T12] __irq_exit_rcu+0x103/0x1c0 [ 913.743932][ T12] irq_exit_rcu+0xe/0x30 [ 913.744017][ T12] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 913.744128][ T12] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 913.744238][ T12] pv_native_safe_halt+0xf/0x10 [ 913.744330][ T12] default_idle+0x9/0x10 [ 913.744459][ T12] default_idle_call+0x6e/0xb0 [ 913.744545][ T12] cpuidle_idle_call.constprop.0+0x237/0x410 [ 913.744654][ T12] do_idle+0xd8/0x180 [ 913.744740][ T12] cpu_startup_entry+0x53/0x70 [ 913.744871][ T12] start_secondary+0x204/0x2b0 [ 913.744959][ T12] common_startup_64+0x13e/0x148 [ 913.745048][ T12] [ 913.745048][ T12] -> #0 ((&n->timer)){+.-.}-{0:0}: [ 913.745222][ T12] check_prev_add+0xd8/0xe90 [ 913.745319][ T12] validate_chain+0x6b8/0xc20 [ 913.745407][ T12] __lock_acquire+0x518/0xc20 [ 913.745500][ T12] lock_acquire.part.0+0xbc/0x260 [ 913.745586][ T12] __timer_delete_sync+0xea/0x130 [ 913.745676][ T12] neigh_flush_one+0x4bf/0x650 [ 913.745772][ T12] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 913.745877][ T12] neigh_ifdown+0x10/0x20 [ 913.746006][ T12] fib_netdev_event+0x1a9/0x5d0 [ 913.746098][ T12] notifier_call_chain+0xae/0x300 [ 913.746185][ T12] netif_close_many+0x366/0x770 [ 913.746278][ T12] unregister_netdevice_many_notify+0x71a/0x1b40 [ 913.746426][ T12] default_device_exit_batch+0x38b/0x600 [ 913.746532][ T12] ops_undo_list+0x2ce/0x8f0 [ 913.746621][ T12] cleanup_net+0x431/0x940 [ 913.746707][ T12] process_one_work+0xdf8/0x1410 [ 913.746841][ T12] worker_thread+0x4f1/0xd60 [ 913.746932][ T12] kthread+0x367/0x460 [ 913.747018][ T12] ret_from_fork+0x474/0x6b0 [ 913.747105][ T12] ret_from_fork_asm+0x11/0x20 [ 913.747234][ T12] [ 913.747234][ T12] other info that might help us debug this: [ 913.747234][ T12] [ 913.747410][ T12] Possible unsafe locking scenario: [ 913.747410][ T12] [ 913.747582][ T12] CPU0 CPU1 [ 913.747668][ T12] ---- ---- [ 913.747755][ T12] lock(&tbl->lock); [ 913.747822][ T12] lock((&n->timer)); [ 913.747973][ T12] lock(&tbl->lock); [ 913.748082][ T12] lock((&n->timer)); [ 913.748150][ T12] [ 913.748150][ T12] *** DEADLOCK *** [ 913.748150][ T12] [ 913.748280][ T12] 5 locks held by kworker/u16:0/12: [ 913.748411][ T12] #0: ff110000019f1d40 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0xd55/0x1410 [ 913.748585][ T12] #1: ffa00000000c7d10 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xdb7/0x1410 [ 913.748784][ T12] #2: ffffffffa12c5528 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbf/0x940 [ 913.748935][ T12] #3: ffffffffa12e0d40 (rtnl_mutex){+.+.}-{4:4}, at: default_device_exit_batch+0x9a/0x600 [ 913.749147][ T12] #4: ff11000009d22358 (&tbl->lock){+.-.}-{3:3}, at: __neigh_ifdown.isra.0+0xa1/0xea0 [ 913.749309][ T12] [ 913.749309][ T12] stack backtrace: [ 913.749417][ T12] CPU: 1 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 7.2.0-rc6-virtme #1 PREEMPT(full) [ 913.749420][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 913.749422][ T12] Workqueue: netns cleanup_net [ 913.749425][ T12] Call Trace: [ 913.749427][ T12] [ 913.749429][ T12] dump_stack_lvl+0x6f/0xa0 [ 913.749432][ T12] print_circular_bug.cold+0x38/0x46 [ 913.749435][ T12] check_noncircular+0x171/0x1a0 [ 913.749438][ T12] check_prev_add+0xd8/0xe90 [ 913.749440][ T12] ? look_up_lock_class+0x65/0x150 [ 913.749442][ T12] ? check_prev_add+0xef/0xe90 [ 913.749444][ T12] validate_chain+0x6b8/0xc20 [ 913.749447][ T12] __lock_acquire+0x518/0xc20 [ 913.749449][ T12] lock_acquire.part.0+0xbc/0x260 [ 913.749451][ T12] ? __timer_delete_sync+0xdb/0x130 [ 913.749453][ T12] ? rcu_is_watching+0x15/0xd0 [ 913.749455][ T12] ? lock_acquire+0x13c/0x160 [ 913.749457][ T12] ? __timer_delete_sync+0xdb/0x130 [ 913.749459][ T12] __timer_delete_sync+0xea/0x130 [ 913.749460][ T12] ? __timer_delete_sync+0xdb/0x130 [ 913.749461][ T12] neigh_flush_one+0x4bf/0x650 [ 913.749463][ T12] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 913.749466][ T12] ? neigh_flush_one+0x650/0x650 [ 913.749467][ T12] ? __lock_release.isra.0+0x6b/0x1a0 [ 913.749469][ T12] ? fib_sync_mtu+0x110/0x110 [ 913.749472][ T12] ? __lock_release.isra.0+0x6b/0x1a0 [ 913.749474][ T12] ? mark_held_locks+0x40/0x70 [ 913.749476][ T12] neigh_ifdown+0x10/0x20 [ 913.749477][ T12] fib_netdev_event+0x1a9/0x5d0 [ 913.749480][ T12] notifier_call_chain+0xae/0x300 [ 913.749482][ T12] netif_close_many+0x366/0x770 [ 913.749484][ T12] ? __dev_close_many+0x680/0x680 [ 913.749485][ T12] ? netif_close_many_and_unlock+0x22/0x2c0 [ 913.749487][ T12] unregister_netdevice_many_notify+0x71a/0x1b40 [ 913.749489][ T12] ? ops_undo_list+0x2ce/0x8f0 [ 913.749491][ T12] ? worker_thread+0x4f1/0xd60 [ 913.749493][ T12] ? kthread+0x367/0x460 [ 913.749494][ T12] ? ret_from_fork+0x474/0x6b0 [ 913.749495][ T12] ? ret_from_fork_asm+0x11/0x20 [ 913.749497][ T12] ? unregister_netdevice_queued+0x80/0x80 [ 913.749500][ T12] default_device_exit_batch+0x38b/0x600 [ 913.749502][ T12] ? unregister_netdev+0x60/0x60 [ 913.749504][ T12] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 913.749506][ T12] ? fou_exit_net+0x30/0x110 [ 913.749509][ T12] ops_undo_list+0x2ce/0x8f0 [ 913.749511][ T12] ? lockdep_hardirqs_on+0x8c/0x130 [ 913.749513][ T12] ? netns_install+0x2a0/0x2a0 [ 913.749515][ T12] ? __lock_release.isra.0+0x6b/0x1a0 [ 913.749517][ T12] ? idr_destroy+0x121/0x250 [ 913.749519][ T12] cleanup_net+0x431/0x940 [ 913.749521][ T12] ? net_passive_dec+0x1c0/0x1c0 [ 913.749522][ T12] ? process_one_work+0xdb7/0x1410 [ 913.749524][ T12] ? lock_acquire+0x13c/0x160 [ 913.749525][ T12] ? rcu_is_watching+0x15/0xd0 [ 913.749527][ T12] process_one_work+0xdf8/0x1410 [ 913.749530][ T12] ? pwq_dec_nr_in_flight+0x710/0x710 [ 913.749531][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 913.749534][ T12] worker_thread+0x4f1/0xd60 [ 913.749536][ T12] ? rescuer_thread+0x1320/0x1320 [ 913.749538][ T12] kthread+0x367/0x460 [ 913.749539][ T12] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 913.749542][ T12] ? kthread_affine_node+0x330/0x330 [ 913.749543][ T12] ret_from_fork+0x474/0x6b0 [ 913.749545][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 913.749547][ T12] ? __switch_to+0x5a3/0xe00 [ 913.749549][ T12] ? kthread_affine_node+0x330/0x330 [ 913.749551][ T12] ret_from_fork_asm+0x11/0x20 [ 913.749554][ T12] [ 913.759121][ T12] br0: port 1(gw_l) entered disabled state [ 913.761829][ T12] gw_l (unregistering): left allmulticast mode [ 913.762007][ T12] gw_l (unregistering): left promiscuous mode [ 913.762114][ T12] br0: port 1(gw_l) entered disabled state [ 913.795046][ T12] amtg: left allmulticast mode [ 913.795157][ T12] amtg: left promiscuous mode [ 913.795472][ T12] br0: port 2(amtg) entered disabled state [ 913.804119][ T310] amtr: left allmulticast mode [ 913.804273][ T310] relay_gw: left allmulticast mode [ 913.804384][ T310] relay_src: left allmulticast mode