[ 695.287569][T14124] br0: port 1(veth1) entered blocking state [ 695.287780][T14124] br0: port 1(veth1) entered disabled state [ 695.287952][T14124] veth1: entered allmulticast mode [ 695.289339][T14124] veth1: entered promiscuous mode [ 695.314610][T14126] br0: port 2(veth2) entered blocking state [ 695.314795][T14126] br0: port 2(veth2) entered disabled state [ 695.314971][T14126] veth2: entered allmulticast mode [ 695.316439][T14126] veth2: entered promiscuous mode [ 695.405809][T12010] br0: port 1(veth1) entered blocking state [ 695.405998][T12010] br0: port 1(veth1) entered forwarding state [ 695.453693][ T91] br0: port 2(veth2) entered blocking state [ 695.454011][ T91] br0: port 2(veth2) entered forwarding state [ 737.740514][ C0] [ 737.740605][ C0] ====================================================== [ 737.740734][ C0] WARNING: possible circular locking dependency detected [ 737.740868][ C0] 7.2.0-rc6-virtme #1 Not tainted [ 737.740970][ C0] ------------------------------------------------------ [ 737.741093][ C0] ip/14298 is trying to acquire lock: [ 737.741191][ C0] ff1100000580d358 (&tbl->lock){+.-.}-{3:3}, at: ___neigh_create+0x2be/0x1070 [ 737.741384][ C0] [ 737.741384][ C0] but task is already holding lock: [ 737.741534][ C0] ffa0000000007c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 737.741707][ C0] [ 737.741707][ C0] which lock already depends on the new lock. [ 737.741707][ C0] [ 737.741898][ C0] [ 737.741898][ C0] the existing dependency chain (in reverse order) is: [ 737.742062][ C0] [ 737.742062][ C0] -> #1 ((&n->timer)){+.-.}-{0:0}: [ 737.742210][ C0] __lock_acquire+0x518/0xc20 [ 737.742313][ C0] lock_acquire.part.0+0xbc/0x260 [ 737.742408][ C0] __timer_delete_sync+0xea/0x130 [ 737.742503][ C0] neigh_flush_one+0x4bf/0x650 [ 737.742597][ C0] __neigh_ifdown.isra.0+0x1b5/0xea0 [ 737.742715][ C0] neigh_ifdown+0x10/0x20 [ 737.742811][ C0] rt6_disable_ip+0x183/0x200 [ 737.742908][ C0] addrconf_ifdown.isra.0+0x103/0x1670 [ 737.743032][ C0] addrconf_notify+0x2c8/0xf30 [ 737.743123][ C0] notifier_call_chain+0xae/0x300 [ 737.743218][ C0] netif_close_many+0x366/0x710 [ 737.743313][ C0] unregister_netdevice_many_notify+0x671/0x19a0 [ 737.743429][ C0] ops_undo_list+0x615/0x8f0 [ 737.743521][ C0] cleanup_net+0x431/0x890 [ 737.743615][ C0] process_one_work+0xdf8/0x1410 [ 737.743760][ C0] worker_thread+0x4f1/0xd60 [ 737.743857][ C0] kthread+0x367/0x460 [ 737.743951][ C0] ret_from_fork+0x474/0x6b0 [ 737.744049][ C0] ret_from_fork_asm+0x11/0x20 [ 737.744194][ C0] [ 737.744194][ C0] -> #0 (&tbl->lock){+.-.}-{3:3}: [ 737.744338][ C0] check_prev_add+0xd8/0xe90 [ 737.744432][ C0] validate_chain+0x6b8/0xc20 [ 737.744579][ C0] __lock_acquire+0x518/0xc20 [ 737.744671][ C0] lock_acquire.part.0+0xbc/0x260 [ 737.744763][ C0] _raw_spin_lock_bh+0x38/0x50 [ 737.744863][ C0] ___neigh_create+0x2be/0x1070 [ 737.745008][ C0] ip_finish_output2+0xe83/0x1f10 [ 737.745103][ C0] ip_output+0x1ca/0x660 [ 737.745195][ C0] ip_push_pending_frames+0xf6/0x250 [ 737.745314][ C0] __icmp_send+0xcb7/0x1ae0 [ 737.745460][ C0] ipv4_send_dest_unreach+0x438/0x9d0 [ 737.745574][ C0] ipv4_link_failure+0x16/0x190 [ 737.745665][ C0] arp_error_report+0x97/0x1b0 [ 737.745759][ C0] neigh_invalidate+0x208/0x560 [ 737.745912][ C0] neigh_timer_handler+0x8e9/0x1210 [ 737.746004][ C0] call_timer_fn+0x163/0x4c0 [ 737.746095][ C0] __run_timers+0x68f/0xaa0 [ 737.746187][ C0] run_timer_softirq+0xf0/0x160 [ 737.746280][ C0] handle_softirqs+0x1d8/0x8f0 [ 737.746378][ C0] __irq_exit_rcu+0x103/0x1c0 [ 737.746471][ C0] irq_exit_rcu+0xe/0x30 [ 737.746563][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 737.746733][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 737.746855][ C0] _raw_write_unlock_irq+0x2e/0x50 [ 737.746946][ C0] exit_notify+0x218/0x890 [ 737.747092][ C0] do_exit+0x63d/0xdc0 [ 737.747184][ C0] do_group_exit+0xb8/0x370 [ 737.747278][ C0] __x64_sys_exit_group+0x3c/0x50 [ 737.747375][ C0] x64_sys_call+0x102c/0x1530 [ 737.747521][ C0] do_syscall_64+0x117/0x590 [ 737.747613][ C0] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 737.747726][ C0] [ 737.747726][ C0] other info that might help us debug this: [ 737.747726][ C0] [ 737.747964][ C0] Possible unsafe locking scenario: [ 737.747964][ C0] [ 737.748102][ C0] CPU0 CPU1 [ 737.748194][ C0] ---- ---- [ 737.748287][ C0] lock((&n->timer)); [ 737.748414][ C0] lock(&tbl->lock); [ 737.748528][ C0] lock((&n->timer)); [ 737.748646][ C0] lock(&tbl->lock); [ 737.748716][ C0] [ 737.748716][ C0] *** DEADLOCK *** [ 737.748716][ C0] [ 737.748908][ C0] 5 locks held by ip/14298: [ 737.749002][ C0] #0: ffa0000000007c98 ((&n->timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4c0 [ 737.749220][ C0] #1: ffffffffba37dcc0 (rcu_read_lock){....}-{1:3}, at: __icmp_send+0x232/0x1ae0 [ 737.749389][ C0] #2: ff11000005834218 (k-slock-AF_INET){+...}-{3:3}, at: __icmp_send+0x75b/0x1ae0 [ 737.749556][ C0] #3: ffffffffba37dcc0 (rcu_read_lock){....}-{1:3}, at: ip_output+0xa9/0x660 [ 737.749774][ C0] #4: ffffffffba37dcc0 (rcu_read_lock){....}-{1:3}, at: ip_finish_output2+0x25c/0x1f10 [ 737.749942][ C0] [ 737.749942][ C0] stack backtrace: [ 737.750111][ C0] CPU: 0 UID: 0 PID: 14298 Comm: ip Not tainted 7.2.0-rc6-virtme #1 PREEMPT(full) [ 737.750114][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 737.750116][ C0] Call Trace: [ 737.750118][ C0] [ 737.750119][ C0] dump_stack_lvl+0x6f/0xa0 [ 737.750122][ C0] print_circular_bug.cold+0x38/0x46 [ 737.750125][ C0] check_noncircular+0x171/0x1a0 [ 737.750128][ C0] check_prev_add+0xd8/0xe90 [ 737.750131][ C0] validate_chain+0x6b8/0xc20 [ 737.750133][ C0] ? find_held_lock+0x2b/0x80 [ 737.750135][ C0] __lock_acquire+0x518/0xc20 [ 737.750138][ C0] lock_acquire.part.0+0xbc/0x260 [ 737.750140][ C0] ? ___neigh_create+0x2be/0x1070 [ 737.750142][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750145][ C0] ? lock_acquire+0x13c/0x160 [ 737.750147][ C0] _raw_spin_lock_bh+0x38/0x50 [ 737.750149][ C0] ? ___neigh_create+0x2be/0x1070 [ 737.750151][ C0] ___neigh_create+0x2be/0x1070 [ 737.750152][ C0] ? sync_exp_reset_tree_hotplug+0x3f0/0x3f0 [ 737.750155][ C0] ip_finish_output2+0xe83/0x1f10 [ 737.750157][ C0] ? ip_dst_mtu_maybe_forward+0x261/0x670 [ 737.750159][ C0] ? ip4_dst_hoplimit+0x320/0x320 [ 737.750161][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750163][ C0] ip_output+0x1ca/0x660 [ 737.750165][ C0] ? ip_finish_output+0x460/0x460 [ 737.750167][ C0] ? __ip_make_skb+0x106a/0x2300 [ 737.750170][ C0] ip_push_pending_frames+0xf6/0x250 [ 737.750173][ C0] __icmp_send+0xcb7/0x1ae0 [ 737.750176][ C0] ? icmpv4_global_allow+0x150/0x150 [ 737.750178][ C0] ? mark_usage+0x30/0x170 [ 737.750179][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 737.750181][ C0] ? __lock_acquire+0x4c1/0xc20 [ 737.750183][ C0] ? __lock_acquire+0x518/0xc20 [ 737.750187][ C0] ? sched_balance_find_src_group+0xda/0x760 [ 737.750191][ C0] ipv4_send_dest_unreach+0x438/0x9d0 [ 737.750193][ C0] ? ip_do_redirect+0x5c0/0x5c0 [ 737.750196][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 737.750198][ C0] ipv4_link_failure+0x16/0x190 [ 737.750199][ C0] arp_error_report+0x97/0x1b0 [ 737.750201][ C0] neigh_invalidate+0x208/0x560 [ 737.750204][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 737.750206][ C0] neigh_timer_handler+0x8e9/0x1210 [ 737.750208][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 737.750209][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750211][ C0] call_timer_fn+0x163/0x4c0 [ 737.750213][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 737.750214][ C0] ? find_held_lock+0x2b/0x80 [ 737.750216][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 737.750218][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750220][ C0] __run_timers+0x68f/0xaa0 [ 737.750221][ C0] ? neigh_periodic_work+0xa60/0xa60 [ 737.750224][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 737.750225][ C0] ? __lock_acquire+0x518/0xc20 [ 737.750228][ C0] ? __rwlock_init+0x150/0x150 [ 737.750230][ C0] run_timer_softirq+0xf0/0x160 [ 737.750232][ C0] ? clockevents_program_event+0x307/0x7e0 [ 737.750235][ C0] ? __run_timers+0xaa0/0xaa0 [ 737.750236][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 737.750239][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750240][ C0] handle_softirqs+0x1d8/0x8f0 [ 737.750243][ C0] ? _local_bh_enable+0xd0/0xd0 [ 737.750246][ C0] __irq_exit_rcu+0x103/0x1c0 [ 737.750248][ C0] irq_exit_rcu+0xe/0x30 [ 737.750250][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 737.750252][ C0] [ 737.750252][ C0] [ 737.750253][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 737.750255][ C0] RIP: 0010:_raw_write_unlock_irq+0x2e/0x50 [ 737.750257][ C0] Code: 0f 1f 44 00 00 53 48 8b 74 24 08 48 89 fb 48 83 c7 18 e8 55 5b c1 fd 48 89 df e8 2d b9 c1 fd e8 78 49 e9 fd fb bf 01 00 00 00 bd 93 b3 fd 65 8b 05 66 84 f8 01 85 c0 74 02 5b c3 e8 4b b0 64 [ 737.750259][ C0] RSP: 0018:ffa0000002ddfd60 EFLAGS: 00000282 [ 737.750261][ C0] RAX: 00000000000074dd RBX: ffffffffba00a080 RCX: ffffffffb9530bb8 [ 737.750263][ C0] RDX: ff11000014aa46c0 RSI: ffffffffb9e1781f RDI: 0000000000000001 [ 737.750264][ C0] RBP: ffa0000002ddfdb0 R08: 0000000000000000 R09: 0000000000000000 [ 737.750265][ C0] R10: 0000000000000000 R11: 0000000000000001 R12: 1ff40000005bbfb2 [ 737.750265][ C0] R13: ff11000014aa4fb0 R14: ff11000014aa4c58 R15: ff1100000c9816c0 [ 737.750267][ C0] ? _raw_write_unlock_irq+0x28/0x50 [ 737.750269][ C0] exit_notify+0x218/0x890 [ 737.750271][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750273][ C0] ? forget_original_parent+0xdb0/0xdb0 [ 737.750275][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 737.750276][ C0] ? _raw_spin_unlock_irqrestore+0x53/0x80 [ 737.750278][ C0] ? _raw_spin_unlock_irqrestore+0x40/0x80 [ 737.750279][ C0] ? exit_tasks_rcu_start+0x217/0x350 [ 737.750283][ C0] do_exit+0x63d/0xdc0 [ 737.750284][ C0] ? exit_notify+0x890/0x890 [ 737.750286][ C0] ? __lock_release.isra.0+0x6b/0x1a0 [ 737.750288][ C0] ? rcu_is_watching+0x15/0xd0 [ 737.750290][ C0] do_group_exit+0xb8/0x370 [ 737.750292][ C0] __x64_sys_exit_group+0x3c/0x50 [ 737.750294][ C0] x64_sys_call+0x102c/0x1530 [ 737.750296][ C0] do_syscall_64+0x117/0x590 [ 737.750297][ C0] ? trace_hardirqs_off+0xd/0x30 [ 737.750299][ C0] ? exc_page_fault+0xee/0x100 [ 737.750301][ C0] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 737.750303][ C0] RIP: 0033:0x7f21cc7741b8 [ 737.750306][ C0] Code: Unable to access opcode bytes at 0x7f21cc77418e. [ 737.750308][ C0] RSP: 002b:00007ffc1a8a1ab8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7 [ 737.750310][ C0] RAX: ffffffffffffffda RBX: 00007f21cc8a4f88 RCX: 00007f21cc7741b8 [ 737.750311][ C0] RDX: 00007f21cc4befc8 RSI: fffffffffffffeb8 RDI: 0000000000000000 [ 737.750312][ C0] RBP: 00007ffc1a8a1b10 R08: 0000000000000000 R09: 0000000000000020 [ 737.750313][ C0] R10: 00007ffc1a8a18d0 R11: 0000000000000246 R12: 0000000000000001 [ 737.750313][ C0] R13: 0000000000000000 R14: 00007f21cc8a3680 R15: 00007f21cc8a4fa0 [ 737.750316][ C0] [ 742.332429][T14407] Mirror/redirect action on [ 748.686791][T14447] br0: port 2(veth2) entered disabled state [ 748.708847][T14448] br0: port 1(veth1) entered disabled state [ 748.723664][T14449] veth2: left allmulticast mode [ 748.723766][T14449] veth2: left promiscuous mode [ 748.723915][T14449] br0: port 2(veth2) entered disabled state [ 748.724487][T14449] veth1: left allmulticast mode [ 748.724571][T14449] veth1: left promiscuous mode [ 748.724705][T14449] br0: port 1(veth1) entered disabled state