[ 8.834131][ T192] ip (192) used greatest stack depth: 24064 bytes left [ 9.508486][ T215] ip (215) used greatest stack depth: 23952 bytes left [ 131.185022][ T4316] awk (4316) used greatest stack depth: 23672 bytes left [ 140.648661][ C0] [ 140.648754][ C0] ====================================================== [ 140.648879][ C0] WARNING: possible circular locking dependency detected [ 140.648999][ C0] 7.2.0-rc6-virtme #1 Not tainted [ 140.649101][ C0] ------------------------------------------------------ [ 140.649217][ C0] swapper/0/0 is trying to acquire lock: [ 140.649319][ C0] ff11000011630b58 (&tbl->lock){+.-.}-{3:3}, at: ___neigh_create+0x2be/0x1070 [ 140.649504][ C0] [ 140.649504][ C0] but task is already holding lock: [ 140.649647][ C0] ffa0000000007c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 140.649816][ C0] [ 140.649816][ C0] which lock already depends on the new lock. [ 140.649816][ C0] [ 140.650002][ C0] [ 140.650002][ C0] the existing dependency chain (in reverse order) is: [ 140.650164][ C0] [ 140.650164][ C0] -> #1 ((&n->timer)){+.-.}-{0:0}: [ 140.650310][ C0] __lock_acquire+0x518/0xc20 [ 140.650409][ C0] lock_acquire.part.0+0xbc/0x260 [ 140.650504][ C0] __timer_delete_sync+0xea/0x130 [ 140.650606][ C0] neigh_flush_one+0x4bf/0x650 [ 140.650700][ C0] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 140.650818][ C0] neigh_ifdown+0x10/0x20 [ 140.650909][ C0] rt6_disable_ip+0x183/0x200 [ 140.651004][ C0] addrconf_ifdown.isra.0+0x103/0x1670 [ 140.651119][ C0] addrconf_notify+0x2c8/0xf30 [ 140.651213][ C0] notifier_call_chain+0xae/0x300 [ 140.651309][ C0] netif_close_many+0x366/0x710 [ 140.651402][ C0] unregister_netdevice_many_notify+0x671/0x19a0 [ 140.651519][ C0] default_device_exit_batch+0x38b/0x600 [ 140.651640][ C0] ops_undo_list+0x2ce/0x8f0 [ 140.651734][ C0] cleanup_net+0x431/0x890 [ 140.651827][ C0] process_one_work+0xdf8/0x1410 [ 140.651924][ C0] worker_thread+0x4f1/0xd60 [ 140.652025][ C0] kthread+0x367/0x460 [ 140.652119][ C0] ret_from_fork+0x474/0x6b0 [ 140.652214][ C0] ret_from_fork_asm+0x11/0x20 [ 140.652309][ C0] [ 140.652309][ C0] -> #0 (&tbl->lock){+.-.}-{3:3}: [ 140.652449][ C0] check_prev_add+0xd8/0xe90 [ 140.652547][ C0] validate_chain+0x6b8/0xc20 [ 140.652641][ C0] __lock_acquire+0x518/0xc20 [ 140.652735][ C0] lock_acquire.part.0+0xbc/0x260 [ 140.652830][ C0] _raw_spin_lock_bh+0x38/0x50 [ 140.652924][ C0] ___neigh_create+0x2be/0x1070 [ 140.653017][ C0] ip_finish_output2+0xe83/0x1f10 [ 140.653112][ C0] ip_output+0x1ca/0x660 [ 140.653204][ C0] ip_push_pending_frames+0xf6/0x250 [ 140.653324][ C0] __icmp_send+0xcb7/0x1ae0 [ 140.653418][ C0] ipv4_send_dest_unreach+0x438/0x9d0 [ 140.653536][ C0] ipv4_link_failure+0x16/0x190 [ 140.653635][ C0] arp_error_report+0x97/0x1b0 [ 140.653731][ C0] neigh_invalidate+0x208/0x560 [ 140.653825][ C0] neigh_timer_handler+0x8e9/0x1210 [ 140.653918][ C0] call_timer_fn+0x163/0x4c0 [ 140.654012][ C0] __run_timers+0x68f/0xaa0 [ 140.654111][ C0] run_timer_softirq+0xf0/0x160 [ 140.654205][ C0] handle_softirqs+0x1d8/0x8f0 [ 140.654300][ C0] __irq_exit_rcu+0x103/0x1c0 [ 140.654392][ C0] irq_exit_rcu+0xe/0x30 [ 140.654486][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 140.654607][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 140.654724][ C0] pv_native_safe_halt+0xf/0x10 [ 140.654818][ C0] default_idle+0x9/0x10 [ 140.654913][ C0] default_idle_call+0x6e/0xb0 [ 140.655008][ C0] cpuidle_idle_call.constprop.0+0x237/0x410 [ 140.655125][ C0] do_idle+0xd8/0x180 [ 140.655217][ C0] cpu_startup_entry+0x53/0x70 [ 140.655311][ C0] rest_init+0x1f7/0x200 [ 140.655405][ C0] start_kernel+0x3af/0x3b0 [ 140.655501][ C0] x86_64_start_reservations+0x24/0x30 [ 140.655622][ C0] x86_64_start_kernel+0x12b/0x130 [ 140.655716][ C0] common_startup_64+0x13e/0x148 [ 140.655810][ C0] [ 140.655810][ C0] other info that might help us debug this: [ 140.655810][ C0] [ 140.655995][ C0] Possible unsafe locking scenario: [ 140.655995][ C0] [ 140.656141][ C0] CPU0 CPU1 [ 140.656236][ C0] ---- ---- [ 140.656330][ C0] lock((&n->timer)); [ 140.656403][ C0] lock(&tbl->lock); [ 140.656521][ C0] lock((&n->timer)); [ 140.656643][ C0] lock(&tbl->lock); [ 140.656714][ C0] [ 140.656714][ C0] *** DEADLOCK *** [ 140.656714][ C0] [ 140.656852][ C0] 5 locks held by swapper/0/0: [ 140.656944][ C0] #0: ffa0000000007c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 140.657113][ C0] #1: ffffffffb01a1cc0 (rcu_read_lock){....}-{1:3}, at: __icmp_send+0x232/0x1ae0 [ 140.657277][ C0] #2: ff1100000585c218 (k-slock-AF_INET){+.-.}-{3:3}, at: __icmp_send+0x75b/0x1ae0 [ 140.657441][ C0] #3: ffffffffb01a1cc0 (rcu_read_lock){....}-{1:3}, at: ip_output+0xa9/0x660 [ 140.657622][ C0] #4: ffffffffb01a1cc0 (rcu_read_lock){....}-{1:3}, at: ip_finish_output2+0x25c/0x1f10 [ 140.657786][ C0] [ 140.657786][ C0] stack backtrace: [ 140.657905][ C0] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.2.0-rc6-virtme #1 PREEMPT(full) [ 140.657908][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 140.657910][ C0] Call Trace: [ 140.657912][ C0] [ 140.657913][ C0] dump_stack_lvl+0x6f/0xa0 [ 140.657916][ C0] print_circular_bug.cold+0x38/0x46 [ 140.657919][ C0] check_noncircular+0x171/0x1a0 [ 140.657922][ C0] check_prev_add+0xd8/0xe90 [ 140.657924][ C0] ? __lock_acquire+0x518/0xc20 [ 140.657926][ C0] validate_chain+0x6b8/0xc20 [ 140.657928][ C0] ? find_held_lock+0x2b/0x80 [ 140.657930][ C0] __lock_acquire+0x518/0xc20 [ 140.657932][ C0] lock_acquire.part.0+0xbc/0x260 [ 140.657934][ C0] ? ___neigh_create+0x2be/0x1070 [ 140.657936][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.657938][ C0] ? lock_acquire+0x13c/0x160 [ 140.657940][ C0] _raw_spin_lock_bh+0x38/0x50 [ 140.657942][ C0] ? ___neigh_create+0x2be/0x1070 [ 140.657943][ C0] ___neigh_create+0x2be/0x1070 [ 140.657944][ C0] ? __rcu_read_lock+0x1/0x70 [ 140.657946][ C0] ip_finish_output2+0xe83/0x1f10 [ 140.657948][ C0] ? ip_dst_mtu_maybe_forward+0x261/0x670 [ 140.657950][ C0] ? ip_fragment.constprop.0+0x2c0/0x2c0 [ 140.657951][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.657953][ C0] ip_output+0x1ca/0x660 [ 140.657955][ C0] ? ip_finish_output+0x460/0x460 [ 140.657957][ C0] ? __ip_make_skb+0x106a/0x2300 [ 140.657960][ C0] ip_push_pending_frames+0xf6/0x250 [ 140.657961][ C0] __icmp_send+0xcb7/0x1ae0 [ 140.657964][ C0] ? unwind_next_frame+0x691/0x1ea0 [ 140.657967][ C0] ? icmpv4_global_allow+0x150/0x150 [ 140.657968][ C0] ? trace_pelt_cfs_tp+0x9b/0x190 [ 140.657970][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.657971][ C0] ? __update_load_avg_cfs_rq+0x36c/0x10c0 [ 140.657974][ C0] ? update_cfs_rq_load_avg+0x4b/0x660 [ 140.657977][ C0] ? update_load_avg+0x825/0xf10 [ 140.657979][ C0] ? rcu_read_lock_any_held+0x3c/0x90 [ 140.657982][ C0] ? __trace_set_current_state+0x160/0x160 [ 140.657984][ C0] ? enqueue_task_fair+0x971/0x1480 [ 140.657986][ C0] ipv4_send_dest_unreach+0x438/0x9d0 [ 140.657988][ C0] ? ip_do_redirect+0x5c0/0x5c0 [ 140.657990][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 140.657992][ C0] ipv4_link_failure+0x16/0x190 [ 140.657994][ C0] arp_error_report+0x97/0x1b0 [ 140.657995][ C0] neigh_invalidate+0x208/0x560 [ 140.657998][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 140.657999][ C0] neigh_timer_handler+0x8e9/0x1210 [ 140.658001][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 140.658002][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.658003][ C0] call_timer_fn+0x163/0x4c0 [ 140.658005][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 140.658007][ C0] ? find_held_lock+0x2b/0x80 [ 140.658009][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 140.658010][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.658012][ C0] __run_timers+0x68f/0xaa0 [ 140.658014][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 140.658016][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 140.658018][ C0] ? __lock_acquire+0x518/0xc20 [ 140.658020][ C0] ? __rwlock_init+0x150/0x150 [ 140.658023][ C0] run_timer_softirq+0xf0/0x160 [ 140.658025][ C0] ? clockevents_program_event+0x307/0x7e0 [ 140.658028][ C0] ? __run_timers+0xaa0/0xaa0 [ 140.658030][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 140.658032][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.658034][ C0] handle_softirqs+0x1d8/0x8f0 [ 140.658036][ C0] ? _local_bh_enable+0xd0/0xd0 [ 140.658038][ C0] __irq_exit_rcu+0x103/0x1c0 [ 140.658039][ C0] irq_exit_rcu+0xe/0x30 [ 140.658041][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 140.658043][ C0] [ 140.658043][ C0] [ 140.658044][ C0] ? lockdep_hardirqs_on_prepare.part.0+0x9a/0x160 [ 140.658046][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 140.658048][ C0] RIP: 0010:pv_native_safe_halt+0xf/0x10 [ 140.658050][ C0] Code: 48 8b 3d 44 21 f6 01 e8 1f 00 00 00 48 2b 05 08 42 91 00 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa eb 07 0f 00 2d a3 4d 0d 00 fb f4 0f 1f 40 d6 48 83 ec 20 8b 17 49 89 f8 83 e2 fe 41 89 d2 0f 01 [ 140.658052][ C0] RSP: 0018:ffffffffafe07e18 EFLAGS: 00000292 [ 140.658054][ C0] RAX: 00000000008a0327 RBX: ffffffffafe40600 RCX: ffffffffacee6de7 [ 140.658055][ C0] RDX: ffffffffafe40600 RSI: ffffffffafc358e3 RDI: ffffffffaf67a8e0 [ 140.658056][ C0] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 140.658057][ C0] R10: 0000000000000000 R11: 0000000000000001 R12: 1ffffffff5fc0fc6 [ 140.658058][ C0] R13: 0000000000000000 R14: dffffc0000000000 R15: 0000000000014770 [ 140.658059][ C0] ? cpuidle_idle_call.constprop.0+0x237/0x410 [ 140.658062][ C0] ? lockdep_hardirqs_on+0x8c/0x130 [ 140.658063][ C0] default_idle+0x9/0x10 [ 140.658065][ C0] default_idle_call+0x6e/0xb0 [ 140.658067][ C0] cpuidle_idle_call.constprop.0+0x237/0x410 [ 140.658068][ C0] ? arch_cpu_idle_exit+0x40/0x40 [ 140.658070][ C0] ? mark_tsc_async_resets+0x30/0x30 [ 140.658073][ C0] ? rcu_is_watching+0x15/0xd0 [ 140.658074][ C0] do_idle+0xd8/0x180 [ 140.658076][ C0] cpu_startup_entry+0x53/0x70 [ 140.658078][ C0] rest_init+0x1f7/0x200 [ 140.658080][ C0] start_kernel+0x3af/0x3b0 [ 140.658081][ C0] x86_64_start_reservations+0x24/0x30 [ 140.658083][ C0] x86_64_start_kernel+0x12b/0x130 [ 140.658085][ C0] common_startup_64+0x13e/0x148 [ 140.658089][ C0] [ 849.383138][T26092] pm_nl_ctl (26092) used greatest stack depth: 23176 bytes left