[ 92.558401][ T2118] veth-s1: entered promiscuous mode [ 93.611480][ T2123] veth-s2: entered promiscuous mode [ 98.614306][ T2118] veth-s1: left promiscuous mode [ 99.683962][ T2123] veth-s2: left promiscuous mode [ 107.606171][ T2147] veth-s2: entered promiscuous mode [ 111.611632][ T2147] veth-s2: left promiscuous mode [ 120.900862][ T2225] veth-s1: entered promiscuous mode [ 121.925377][ T2230] veth-s2: entered promiscuous mode [ 126.920134][ T2225] veth-s1: left promiscuous mode [ 128.000964][ T2230] veth-s2: left promiscuous mode [ 136.996142][ T2254] veth-s1: entered promiscuous mode [ 139.980837][ C2] [ 139.980928][ C2] ====================================================== [ 139.981044][ C2] WARNING: possible circular locking dependency detected [ 139.981156][ C2] 7.2.0-rc6-virtme #1 Not tainted [ 139.981246][ C2] ------------------------------------------------------ [ 139.981355][ C2] swapper/2/0 is trying to acquire lock: [ 139.981446][ C2] ff1100000f8fd358 (&tbl->lock){+.-.}-{3:3}, at: ___neigh_create+0x2be/0x1070 [ 139.981612][ C2] [ 139.981612][ C2] but task is already holding lock: [ 139.981740][ C2] ffa0000000228c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 139.981895][ C2] [ 139.981895][ C2] which lock already depends on the new lock. [ 139.981895][ C2] [ 139.982071][ C2] [ 139.982071][ C2] the existing dependency chain (in reverse order) is: [ 139.982220][ C2] [ 139.982220][ C2] -> #1 ((&n->timer)){+.-.}-{0:0}: [ 139.982352][ C2] __lock_acquire+0x518/0xc20 [ 139.982444][ C2] lock_acquire.part.0+0xbc/0x260 [ 139.982534][ C2] __timer_delete_sync+0xea/0x130 [ 139.982620][ C2] neigh_flush_one+0x4bf/0x650 [ 139.982705][ C2] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 139.982811][ C2] neigh_ifdown+0x10/0x20 [ 139.982900][ C2] fib_netdev_event+0x1a9/0x5d0 [ 139.982995][ C2] notifier_call_chain+0xae/0x300 [ 139.983084][ C2] netif_close_many+0x366/0x770 [ 139.983169][ C2] unregister_netdevice_many_notify+0x71a/0x1b40 [ 139.983276][ C2] ops_undo_list+0x615/0x8f0 [ 139.983364][ C2] cleanup_net+0x431/0x940 [ 139.983450][ C2] process_one_work+0xdf8/0x1410 [ 139.983538][ C2] worker_thread+0x4f1/0xd60 [ 139.983624][ C2] kthread+0x367/0x460 [ 139.983709][ C2] ret_from_fork+0x474/0x6b0 [ 139.983798][ C2] ret_from_fork_asm+0x11/0x20 [ 139.983886][ C2] [ 139.983886][ C2] -> #0 (&tbl->lock){+.-.}-{3:3}: [ 139.984019][ C2] check_prev_add+0xd8/0xe90 [ 139.984107][ C2] validate_chain+0x6b8/0xc20 [ 139.984196][ C2] __lock_acquire+0x518/0xc20 [ 139.984283][ C2] lock_acquire.part.0+0xbc/0x260 [ 139.984369][ C2] _raw_spin_lock_bh+0x38/0x50 [ 139.984457][ C2] ___neigh_create+0x2be/0x1070 [ 139.984541][ C2] ip6_finish_output2+0xb9a/0x12f0 [ 139.984629][ C2] ip6_finish_output+0x701/0xe80 [ 139.984714][ C2] ip6_output+0x23f/0x7f0 [ 139.984799][ C2] ip6_send_skb+0xee/0x350 [ 139.984887][ C2] icmp6_send+0x1728/0x1cd0 [ 139.984976][ C2] ip6_link_failure+0x2d/0x460 [ 139.985063][ C2] ndisc_error_report+0x97/0x1b0 [ 139.985149][ C2] neigh_invalidate+0x208/0x560 [ 139.985235][ C2] neigh_timer_handler+0x8e9/0x1210 [ 139.985321][ C2] call_timer_fn+0x163/0x4c0 [ 139.985407][ C2] __run_timers+0x68f/0xaa0 [ 139.985492][ C2] run_timer_softirq+0xf0/0x160 [ 139.985581][ C2] handle_softirqs+0x1d8/0x8f0 [ 139.985669][ C2] __irq_exit_rcu+0x103/0x1c0 [ 139.985759][ C2] irq_exit_rcu+0xe/0x30 [ 139.985846][ C2] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 139.985957][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 139.986066][ C2] pv_native_safe_halt+0xf/0x10 [ 139.986151][ C2] default_idle+0x9/0x10 [ 139.986237][ C2] default_idle_call+0x6e/0xb0 [ 139.986322][ C2] cpuidle_idle_call.constprop.0+0x237/0x410 [ 139.986429][ C2] do_idle+0xd8/0x180 [ 139.986515][ C2] cpu_startup_entry+0x53/0x70 [ 139.986601][ C2] start_secondary+0x204/0x2b0 [ 139.986689][ C2] common_startup_64+0x13e/0x148 [ 139.986776][ C2] [ 139.986776][ C2] other info that might help us debug this: [ 139.986776][ C2] [ 139.986950][ C2] Possible unsafe locking scenario: [ 139.986950][ C2] [ 139.987078][ C2] CPU0 CPU1 [ 139.987164][ C2] ---- ---- [ 139.987251][ C2] lock((&n->timer)); [ 139.987317][ C2] lock(&tbl->lock); [ 139.987427][ C2] lock((&n->timer)); [ 139.987534][ C2] lock(&tbl->lock); [ 139.987600][ C2] [ 139.987600][ C2] *** DEADLOCK *** [ 139.987600][ C2] [ 139.987731][ C2] 5 locks held by swapper/2/0: [ 139.987818][ C2] #0: ffa0000000228c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 139.987973][ C2] #1: ffffffff87591d40 (rcu_read_lock){....}-{1:3}, at: icmp6_send+0x283/0x1cd0 [ 139.988125][ C2] #2: ff11000009219318 (k-slock-AF_INET6){+...}-{3:3}, at: icmp6_send+0x9ff/0x1cd0 [ 139.988275][ C2] #3: ffffffff87591d40 (rcu_read_lock){....}-{1:3}, at: ip6_send_skb+0xa2/0x350 [ 139.988429][ C2] #4: ffffffff87591d40 (rcu_read_lock){....}-{1:3}, at: ip6_output+0x11d/0x7f0 [ 139.988607][ C2] [ 139.988607][ C2] stack backtrace: [ 139.988715][ C2] CPU: 2 UID: 0 PID: 0 Comm: swapper/2 Not tainted 7.2.0-rc6-virtme #1 PREEMPT(full) [ 139.988718][ C2] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 139.988720][ C2] Call Trace: [ 139.988721][ C2] [ 139.988723][ C2] dump_stack_lvl+0x6f/0xa0 [ 139.988726][ C2] print_circular_bug.cold+0x38/0x46 [ 139.988729][ C2] check_noncircular+0x171/0x1a0 [ 139.988732][ C2] check_prev_add+0xd8/0xe90 [ 139.988734][ C2] validate_chain+0x6b8/0xc20 [ 139.988736][ C2] __lock_acquire+0x518/0xc20 [ 139.988739][ C2] lock_acquire.part.0+0xbc/0x260 [ 139.988741][ C2] ? ___neigh_create+0x2be/0x1070 [ 139.988742][ C2] ? rcu_is_watching+0x15/0xd0 [ 139.988745][ C2] ? lock_acquire+0x13c/0x160 [ 139.988747][ C2] _raw_spin_lock_bh+0x38/0x50 [ 139.988749][ C2] ? ___neigh_create+0x2be/0x1070 [ 139.988750][ C2] ___neigh_create+0x2be/0x1070 [ 139.988752][ C2] ip6_finish_output2+0xb9a/0x12f0 [ 139.988754][ C2] ? ip6_xmit+0x2000/0x2000 [ 139.988755][ C2] ? find_held_lock+0x2b/0x80 [ 139.988757][ C2] ? __lock_release.isra.0+0x6b/0x1a0 [ 139.988759][ C2] ? ip6_mtu+0x174/0x410 [ 139.988761][ C2] ip6_finish_output+0x701/0xe80 [ 139.988763][ C2] ip6_output+0x23f/0x7f0 [ 139.988765][ C2] ? ip6_finish_output+0xe80/0xe80 [ 139.988766][ C2] ? l3mdev_l3_out.constprop.0+0xe5/0x340 [ 139.988770][ C2] ip6_send_skb+0xee/0x350 [ 139.988772][ C2] icmp6_send+0x1728/0x1cd0 [ 139.988774][ C2] ? icmpv6_push_pending_frames+0x5f0/0x5f0 [ 139.988776][ C2] ? sched_balance_find_src_group+0xda/0x760 [ 139.988778][ C2] ? __lock_release.isra.0+0x6b/0x1a0 [ 139.988779][ C2] ? rcu_read_lock_any_held+0x3c/0x90 [ 139.988783][ C2] ? rcu_read_lock_any_held+0x3c/0x90 [ 139.988786][ C2] ? __lock_acquire+0x518/0xc20 [ 139.988787][ C2] ? find_held_lock+0x2b/0x80 [ 139.988789][ C2] ip6_link_failure+0x2d/0x460 [ 139.988791][ C2] ndisc_error_report+0x97/0x1b0 [ 139.988793][ C2] neigh_invalidate+0x208/0x560 [ 139.988794][ C2] ? neigh_periodic_work+0xa60/0xa60 [ 139.988796][ C2] neigh_timer_handler+0x8e9/0x1210 [ 139.988798][ C2] ? neigh_periodic_work+0xa60/0xa60 [ 139.988799][ C2] ? rcu_is_watching+0x15/0xd0 [ 139.988800][ C2] call_timer_fn+0x163/0x4c0 [ 139.988802][ C2] ? detach_if_pending+0x1d0/0x1d0 [ 139.988803][ C2] ? find_held_lock+0x2b/0x80 [ 139.988805][ C2] ? __lock_release.isra.0+0x6b/0x1a0 [ 139.988807][ C2] ? rcu_is_watching+0x15/0xd0 [ 139.988808][ C2] __run_timers+0x68f/0xaa0 [ 139.988809][ C2] ? neigh_periodic_work+0xa60/0xa60 [ 139.988811][ C2] ? __bpf_trace_itimer_expire+0x10/0x10 [ 139.988812][ C2] ? __lock_acquire+0x518/0xc20 [ 139.988815][ C2] ? __rwlock_init+0x150/0x150 [ 139.988817][ C2] run_timer_softirq+0xf0/0x160 [ 139.988818][ C2] ? clockevents_program_event+0x307/0x7e0 [ 139.988822][ C2] ? __run_timers+0xaa0/0xaa0 [ 139.988823][ C2] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 139.988825][ C2] ? rcu_is_watching+0x15/0xd0 [ 139.988827][ C2] handle_softirqs+0x1d8/0x8f0 [ 139.988829][ C2] ? _local_bh_enable+0xd0/0xd0 [ 139.988831][ C2] __irq_exit_rcu+0x103/0x1c0 [ 139.988833][ C2] irq_exit_rcu+0xe/0x30 [ 139.988835][ C2] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 139.988836][ C2] [ 139.988837][ C2] [ 139.988837][ C2] ? lockdep_hardirqs_on+0x8c/0x130 [ 139.988839][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 139.988840][ C2] RIP: 0010:pv_native_safe_halt+0xf/0x10 [ 139.988843][ C2] Code: 48 8b 3d 44 61 6e 02 e8 1f 00 00 00 48 2b 05 08 22 a7 00 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa eb 07 0f 00 2d a3 dd 19 00 fb f4 0f 1f 40 d6 48 83 ec 20 8b 17 49 89 f8 83 e2 fe 41 89 d2 0f 01 [ 139.988844][ C2] RSP: 0018:ffa0000000157e00 EFLAGS: 00000282 [ 139.988846][ C2] RAX: 000000000066cc85 RBX: ff11000001bac6c0 RCX: ffffffff838f7f27 [ 139.988848][ C2] RDX: ff11000001bac6c0 RSI: ffffffff86ac4933 RDI: ffffffff8647d760 [ 139.988848][ C2] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 139.988849][ C2] R10: 0000000000000002 R11: 0000000000000001 R12: 1ff400000002afc3 [ 139.988850][ C2] R13: 0000000000000000 R14: dffffc0000000000 R15: 0000000000000000 [ 139.988852][ C2] ? cpuidle_idle_call.constprop.0+0x237/0x410 [ 139.988854][ C2] default_idle+0x9/0x10 [ 139.988856][ C2] default_idle_call+0x6e/0xb0 [ 139.988857][ C2] cpuidle_idle_call.constprop.0+0x237/0x410 [ 139.988859][ C2] ? arch_cpu_idle_exit+0x40/0x40 [ 139.988861][ C2] ? mark_tsc_async_resets+0x30/0x30 [ 139.988862][ C2] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 139.988864][ C2] ? rcu_is_watching+0x15/0xd0 [ 139.988865][ C2] do_idle+0xd8/0x180 [ 139.988867][ C2] cpu_startup_entry+0x53/0x70 [ 139.988869][ C2] start_secondary+0x204/0x2b0 [ 139.988871][ C2] ? set_cpu_sibling_map+0x2130/0x2130 [ 139.988873][ C2] common_startup_64+0x13e/0x148 [ 139.988876][ C2] [ 139.995721][ C0] clocksource: Watchdog remote CPU 2 read timed out [ 140.963258][ T2254] veth-s1: left promiscuous mode