[ 99.327847][ T2769] 8021q: 802.1Q VLAN Support v1.8 [ 99.921230][ T2799] lag2: Mode changed to "loadbalance" [ 99.981474][ T2803] lag2: Port device veth3 added [ 100.051997][ T2810] lag2: Port device veth5 added [ 100.131274][ T2817] 8021q: adding VLAN 0 to HW filter on device lag2 [ 100.160596][ T2818] 8021q: adding VLAN 0 to HW filter on device lag2 [ 100.304140][ T2823] gre: GRE over IPv4 demultiplexer driver [ 100.312258][ T2823] ip_gre: GRE over IPv4 tunneling driver [ 100.861586][ T2849] lag1: Mode changed to "loadbalance" [ 100.940628][ T2852] lag1: Port device veth2 added [ 101.066680][ T2855] lag1: Port device veth4 added [ 101.117747][ T2857] 8021q: adding VLAN 0 to HW filter on device lag1 [ 105.763395][ C1] [ 105.763530][ C1] ======================================================== [ 105.763676][ C1] WARNING: possible irq lock inversion dependency detected [ 105.763808][ C1] 7.1.0-rc5-virtme #1 Not tainted [ 105.763965][ C1] -------------------------------------------------------- [ 105.764110][ C1] swapper/1/0 just changed the state of lock: [ 105.764229][ C1] ff11000005118d78 (&ndev->lock){++-.}-{3:3}, at: addrconf_rs_timer+0xb0/0x770 [ 105.764403][ C1] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 105.764544][ C1] (acaddr_hash_lock){+.+.}-{3:3} [ 105.764546][ C1] [ 105.764546][ C1] [ 105.764546][ C1] and interrupts could create inverse lock ordering between them. [ 105.764546][ C1] [ 105.764898][ C1] [ 105.764898][ C1] other info that might help us debug this: [ 105.765027][ C1] Possible interrupt unsafe locking scenario: [ 105.765027][ C1] [ 105.765159][ C1] CPU0 CPU1 [ 105.765249][ C1] ---- ---- [ 105.765337][ C1] lock(acaddr_hash_lock); [ 105.765427][ C1] local_irq_disable(); [ 105.765534][ C1] lock(&ndev->lock); [ 105.765647][ C1] lock(acaddr_hash_lock); [ 105.765756][ C1] [ 105.765832][ C1] lock(&ndev->lock); [ 105.765900][ C1] [ 105.765900][ C1] *** DEADLOCK *** [ 105.765900][ C1] [ 105.766031][ C1] 1 lock held by swapper/1/0: [ 105.766119][ C1] #0: ffa00000001d0c90 ((&ndev->rs_timer)){+.-.}-{0:0}, at: call_timer_fn+0x113/0x4f0 [ 105.766270][ C1] [ 105.766270][ C1] the shortest dependencies between 2nd lock and 1st lock: [ 105.766419][ C1] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 105.766508][ C1] HARDIRQ-ON-W at: [ 105.766574][ C1] __lock_acquire+0x378/0xc10 [ 105.766689][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.766804][ C1] _raw_spin_lock+0x33/0x40 [ 105.766920][ C1] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 105.767052][ C1] __ipv6_dev_ac_inc+0x57e/0x930 [ 105.767191][ C1] addrconf_join_anycast+0x1bc/0x260 [ 105.767331][ C1] __ipv6_ifa_notify+0x75b/0xad0 [ 105.767473][ C1] addrconf_dad_completed+0x14b/0xe10 [ 105.767627][ C1] addrconf_dad_work+0x3c1/0x930 [ 105.767737][ C1] process_one_work+0xdf8/0x1410 [ 105.767876][ C1] worker_thread+0x4f1/0xd60 [ 105.768019][ C1] kthread+0x367/0x460 [ 105.768160][ C1] ret_from_fork+0x474/0x6b0 [ 105.768301][ C1] ret_from_fork_asm+0x11/0x20 [ 105.768413][ C1] SOFTIRQ-ON-W at: [ 105.768508][ C1] __lock_acquire+0x378/0xc10 [ 105.768654][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.768803][ C1] _raw_spin_lock+0x33/0x40 [ 105.768942][ C1] __ipv6_dev_ac_dec+0x236/0x5d0 [ 105.769062][ C1] addrconf_leave_anycast+0x1bc/0x260 [ 105.769225][ C1] __ipv6_ifa_notify+0x6b4/0xad0 [ 105.769379][ C1] ipv6_del_addr+0x419/0xa20 [ 105.769497][ C1] inet6_addr_del.isra.0+0x21d/0x6c0 [ 105.769611][ C1] inet6_rtm_deladdr+0x200/0x2e0 [ 105.769750][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 105.769916][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 105.770060][ C1] netlink_unicast+0x486/0x750 [ 105.770175][ C1] netlink_sendmsg+0x735/0xc60 [ 105.770282][ C1] ____sys_sendmsg+0x419/0x850 [ 105.770396][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 105.770523][ C1] __sys_sendmsg+0x145/0x1f0 [ 105.770664][ C1] do_syscall_64+0x117/0x590 [ 105.770808][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 105.770978][ C1] INITIAL USE at: [ 105.771044][ C1] __lock_acquire+0x378/0xc10 [ 105.771178][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.771317][ C1] _raw_spin_lock+0x33/0x40 [ 105.771466][ C1] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 105.771613][ C1] __ipv6_dev_ac_inc+0x57e/0x930 [ 105.771737][ C1] addrconf_join_anycast+0x1bc/0x260 [ 105.771866][ C1] __ipv6_ifa_notify+0x75b/0xad0 [ 105.771981][ C1] addrconf_dad_completed+0x14b/0xe10 [ 105.772101][ C1] addrconf_dad_work+0x3c1/0x930 [ 105.772212][ C1] process_one_work+0xdf8/0x1410 [ 105.772320][ C1] worker_thread+0x4f1/0xd60 [ 105.772428][ C1] kthread+0x367/0x460 [ 105.772566][ C1] ret_from_fork+0x474/0x6b0 [ 105.772676][ C1] ret_from_fork_asm+0x11/0x20 [ 105.772781][ C1] } [ 105.772825][ C1] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 105.772988][ C1] ... acquired at: [ 105.773056][ C1] __lock_acquire+0x508/0xc10 [ 105.773145][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.773228][ C1] _raw_spin_lock+0x33/0x40 [ 105.773315][ C1] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 105.773419][ C1] __ipv6_dev_ac_inc+0x57e/0x930 [ 105.773503][ C1] addrconf_join_anycast+0x1bc/0x260 [ 105.773587][ C1] __ipv6_ifa_notify+0x75b/0xad0 [ 105.773672][ C1] addrconf_dad_completed+0x14b/0xe10 [ 105.773757][ C1] addrconf_dad_work+0x3c1/0x930 [ 105.773851][ C1] process_one_work+0xdf8/0x1410 [ 105.773940][ C1] worker_thread+0x4f1/0xd60 [ 105.774029][ C1] kthread+0x367/0x460 [ 105.774101][ C1] ret_from_fork+0x474/0x6b0 [ 105.774188][ C1] ret_from_fork_asm+0x11/0x20 [ 105.774280][ C1] [ 105.774324][ C1] -> (&ndev->lock){++-.}-{3:3} { [ 105.774416][ C1] HARDIRQ-ON-W at: [ 105.774482][ C1] __lock_acquire+0x378/0xc10 [ 105.774592][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.774702][ C1] _raw_write_lock_bh+0x38/0x50 [ 105.774808][ C1] addrconf_permanent_addr+0x108/0x9c0 [ 105.774920][ C1] addrconf_notify+0x151/0xf30 [ 105.775027][ C1] notifier_call_chain+0xb0/0x320 [ 105.775140][ C1] __dev_notify_flags+0xde/0x280 [ 105.775252][ C1] netif_change_flags+0xfe/0x190 [ 105.775362][ C1] do_setlink.isra.0+0x1cc3/0x2750 [ 105.775469][ C1] rtnl_newlink+0x8d1/0xef0 [ 105.775576][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 105.775689][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 105.775798][ C1] netlink_unicast+0x486/0x750 [ 105.775913][ C1] netlink_sendmsg+0x735/0xc60 [ 105.776024][ C1] ____sys_sendmsg+0x419/0x850 [ 105.776139][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 105.776247][ C1] __sys_sendmsg+0x145/0x1f0 [ 105.776354][ C1] do_syscall_64+0x117/0x590 [ 105.776461][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 105.776593][ C1] HARDIRQ-ON-R at: [ 105.776655][ C1] __lock_acquire+0x378/0xc10 [ 105.776761][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.776872][ C1] _raw_read_lock_bh+0x44/0x80 [ 105.776980][ C1] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 105.777106][ C1] inet6_fill_link_af+0x5c/0xe0 [ 105.777214][ C1] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 105.777392][ C1] rtnl_getlink+0x9c9/0xeb0 [ 105.777536][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 105.777675][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 105.777783][ C1] netlink_unicast+0x486/0x750 [ 105.777946][ C1] netlink_sendmsg+0x735/0xc60 [ 105.778060][ C1] ____sys_sendmsg+0x419/0x850 [ 105.778173][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 105.778277][ C1] __sys_sendmsg+0x145/0x1f0 [ 105.778380][ C1] do_syscall_64+0x117/0x590 [ 105.778493][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 105.778627][ C1] IN-SOFTIRQ-W at: [ 105.778695][ C1] __lock_acquire+0x378/0xc10 [ 105.778804][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.778912][ C1] _raw_write_lock+0x33/0x40 [ 105.779021][ C1] addrconf_rs_timer+0xb0/0x770 [ 105.779135][ C1] call_timer_fn+0x163/0x4f0 [ 105.779245][ C1] __run_timers+0x68f/0xab0 [ 105.779358][ C1] run_timer_softirq+0xf0/0x160 [ 105.779466][ C1] handle_softirqs+0x1d8/0x940 [ 105.779610][ C1] __irq_exit_rcu+0x103/0x1c0 [ 105.779718][ C1] irq_exit_rcu+0xe/0x30 [ 105.779860][ C1] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 105.780017][ C1] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 105.780150][ C1] pv_native_safe_halt+0xf/0x10 [ 105.780260][ C1] default_idle+0x9/0x10 [ 105.780373][ C1] default_idle_call+0x6e/0xb0 [ 105.780483][ C1] cpuidle_idle_call.constprop.0+0x237/0x410 [ 105.780613][ C1] do_idle+0xf5/0x160 [ 105.780701][ C1] cpu_startup_entry+0x53/0x70 [ 105.780812][ C1] start_secondary+0x204/0x2b0 [ 105.780927][ C1] common_startup_64+0x13e/0x148 [ 105.781041][ C1] INITIAL USE at: [ 105.781108][ C1] __lock_acquire+0x378/0xc10 [ 105.781216][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.781326][ C1] _raw_write_lock_bh+0x38/0x50 [ 105.781437][ C1] addrconf_permanent_addr+0x108/0x9c0 [ 105.781543][ C1] addrconf_notify+0x151/0xf30 [ 105.781648][ C1] notifier_call_chain+0xb0/0x320 [ 105.781756][ C1] __dev_notify_flags+0xde/0x280 [ 105.781898][ C1] netif_change_flags+0xfe/0x190 [ 105.782009][ C1] do_setlink.isra.0+0x1cc3/0x2750 [ 105.782137][ C1] rtnl_newlink+0x8d1/0xef0 [ 105.782274][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 105.782420][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 105.782575][ C1] netlink_unicast+0x486/0x750 [ 105.782717][ C1] netlink_sendmsg+0x735/0xc60 [ 105.782858][ C1] ____sys_sendmsg+0x419/0x850 [ 105.783012][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 105.783153][ C1] __sys_sendmsg+0x145/0x1f0 [ 105.783290][ C1] do_syscall_64+0x117/0x590 [ 105.783406][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 105.783559][ C1] INITIAL READ USE at: [ 105.783630][ C1] __lock_acquire+0x378/0xc10 [ 105.783789][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.783913][ C1] _raw_read_lock_bh+0x44/0x80 [ 105.784029][ C1] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 105.784166][ C1] inet6_fill_link_af+0x5c/0xe0 [ 105.784272][ C1] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 105.784402][ C1] rtnl_getlink+0x9c9/0xeb0 [ 105.784507][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 105.784619][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 105.784725][ C1] netlink_unicast+0x486/0x750 [ 105.784832][ C1] netlink_sendmsg+0x735/0xc60 [ 105.784942][ C1] ____sys_sendmsg+0x419/0x850 [ 105.785052][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 105.785165][ C1] __sys_sendmsg+0x145/0x1f0 [ 105.785274][ C1] do_syscall_64+0x117/0x590 [ 105.785379][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 105.785513][ C1] } [ 105.785555][ C1] ... key at: [] __key.37+0x0/0x40 [ 105.785688][ C1] ... acquired at: [ 105.785754][ C1] mark_lock+0x1d7/0xa00 [ 105.785842][ C1] mark_usage+0x42/0x170 [ 105.785930][ C1] __lock_acquire+0x378/0xc10 [ 105.786016][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.786102][ C1] _raw_write_lock+0x33/0x40 [ 105.786187][ C1] addrconf_rs_timer+0xb0/0x770 [ 105.786273][ C1] call_timer_fn+0x163/0x4f0 [ 105.786360][ C1] __run_timers+0x68f/0xab0 [ 105.786445][ C1] run_timer_softirq+0xf0/0x160 [ 105.786529][ C1] handle_softirqs+0x1d8/0x940 [ 105.786620][ C1] __irq_exit_rcu+0x103/0x1c0 [ 105.786705][ C1] irq_exit_rcu+0xe/0x30 [ 105.786790][ C1] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 105.786899][ C1] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 105.787007][ C1] pv_native_safe_halt+0xf/0x10 [ 105.787097][ C1] default_idle+0x9/0x10 [ 105.787180][ C1] default_idle_call+0x6e/0xb0 [ 105.787263][ C1] cpuidle_idle_call.constprop.0+0x237/0x410 [ 105.787368][ C1] do_idle+0xf5/0x160 [ 105.787432][ C1] cpu_startup_entry+0x53/0x70 [ 105.787519][ C1] start_secondary+0x204/0x2b0 [ 105.787609][ C1] common_startup_64+0x13e/0x148 [ 105.787697][ C1] [ 105.787739][ C1] [ 105.787739][ C1] stack backtrace: [ 105.787850][ C1] CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 105.787853][ C1] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 105.787855][ C1] Call Trace: [ 105.787857][ C1] [ 105.787859][ C1] dump_stack_lvl+0x6f/0xa0 [ 105.787863][ C1] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 105.787866][ C1] mark_lock_irq+0x989/0x9c0 [ 105.787869][ C1] mark_lock+0x1d7/0xa00 [ 105.787871][ C1] mark_usage+0x42/0x170 [ 105.787872][ C1] __lock_acquire+0x378/0xc10 [ 105.787874][ C1] ? add_chain_cache+0x12d/0x570 [ 105.787876][ C1] lock_acquire.part.0+0xbc/0x260 [ 105.787877][ C1] ? addrconf_rs_timer+0xb0/0x770 [ 105.787879][ C1] ? rcu_is_watching+0x15/0xd0 [ 105.787882][ C1] ? lock_acquire+0x134/0x160 [ 105.787884][ C1] ? ipv6_get_lladdr+0x3f0/0x3f0 [ 105.787886][ C1] _raw_write_lock+0x33/0x40 [ 105.787887][ C1] ? addrconf_rs_timer+0xb0/0x770 [ 105.787889][ C1] addrconf_rs_timer+0xb0/0x770 [ 105.787891][ C1] ? lock_acquire.part.0+0xbc/0x260 [ 105.787893][ C1] ? ipv6_get_lladdr+0x3f0/0x3f0 [ 105.787895][ C1] ? lock_acquire+0x134/0x160 [ 105.787896][ C1] ? ipv6_get_lladdr+0x3f0/0x3f0 [ 105.787898][ C1] call_timer_fn+0x163/0x4f0 [ 105.787900][ C1] ? detach_if_pending+0x1d0/0x1d0 [ 105.787902][ C1] ? debug_object_active_state+0x430/0x430 [ 105.787905][ C1] ? find_held_lock+0x2b/0x80 [ 105.787908][ C1] ? __lock_release.isra.0+0x6b/0x1a0 [ 105.787910][ C1] ? rcu_is_watching+0x15/0xd0 [ 105.787912][ C1] __run_timers+0x68f/0xab0 [ 105.787913][ C1] ? ipv6_get_lladdr+0x3f0/0x3f0 [ 105.787916][ C1] ? __bpf_trace_itimer_expire+0x10/0x10 [ 105.787917][ C1] ? __lock_acquire+0x508/0xc10 [ 105.787920][ C1] ? __rwlock_init+0x150/0x150 [ 105.787922][ C1] run_timer_softirq+0xf0/0x160 [ 105.787924][ C1] ? __run_timers+0xab0/0xab0 [ 105.787926][ C1] ? trace_irq_enable.constprop.0+0x9b/0x180 [ 105.787929][ C1] ? rcu_is_watching+0x15/0xd0 [ 105.787930][ C1] handle_softirqs+0x1d8/0x940 [ 105.787932][ C1] ? find_held_lock+0x2b/0x80 [ 105.787934][ C1] ? __lock_release.isra.0+0x6b/0x1a0 [ 105.787935][ C1] ? _local_bh_enable+0xd0/0xd0 [ 105.787937][ C1] __irq_exit_rcu+0x103/0x1c0 [ 105.787939][ C1] irq_exit_rcu+0xe/0x30 [ 105.787940][ C1] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 105.787942][ C1] [ 105.787943][ C1] [ 105.787943][ C1] ? rcu_is_watching+0x15/0xd0 [ 105.787945][ C1] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 105.787947][ C1] RIP: 0010:pv_native_safe_halt+0xf/0x10 [ 105.787950][ C1] Code: 48 8b 3d 44 93 00 02 e8 1f 00 00 00 48 2b 05 c8 31 a3 00 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa eb 07 0f 00 2d 43 b4 22 00 fb f4 0f 1f 40 d6 48 83 ec 20 8b 17 49 89 f8 83 e2 fe 41 89 d2 0f 01 [ 105.787951][ C1] RSP: 0018:ffa0000000147de8 EFLAGS: 00000296 [ 105.787954][ C1] RAX: 00000000003008cf RBX: ff11000001bb2340 RCX: ffffffff8aed9667 [ 105.787955][ C1] RDX: ff11000001bb2340 RSI: ffffffff8dc0221d RDI: ffffffff8d66f6c0 [ 105.787956][ C1] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 105.787957][ C1] R10: 0000000000000001 R11: 0000000000000001 R12: 1ff4000000028fc0 [ 105.787957][ C1] R13: 0000000000000000 R14: dffffc0000000000 R15: 0000000000000000 [ 105.787959][ C1] ? cpuidle_idle_call.constprop.0+0x237/0x410 [ 105.787961][ C1] ? lockdep_hardirqs_on+0x8c/0x130 [ 105.787963][ C1] default_idle+0x9/0x10 [ 105.787965][ C1] default_idle_call+0x6e/0xb0 [ 105.787966][ C1] cpuidle_idle_call.constprop.0+0x237/0x410 [ 105.787968][ C1] ? arch_cpu_idle_exit+0x40/0x40 [ 105.787969][ C1] ? mark_tsc_async_resets+0x30/0x30 [ 105.787971][ C1] ? trace_irq_enable.constprop.0+0x9b/0x180 [ 105.787973][ C1] ? rcu_is_watching+0x15/0xd0 [ 105.787974][ C1] do_idle+0xf5/0x160 [ 105.787976][ C1] cpu_startup_entry+0x53/0x70 [ 105.787978][ C1] start_secondary+0x204/0x2b0 [ 105.787979][ C1] ? set_cpu_sibling_map+0x1fb0/0x1fb0 [ 105.787981][ C1] common_startup_64+0x13e/0x148 [ 105.787984][ C1] [ 106.496026][ T2888] Mirror/redirect action on [ 106.533796][ T2890] lag2: Port device veth5 removed [ 110.145984][ T2901] lag2: Port device veth3 removed [ 114.071134][ T2933] lag2: Mode changed to "loadbalance" [ 114.114596][ T2938] lag2: Port device veth3 added [ 114.189915][ T2941] lag2: Port device veth5 added [ 114.227590][ T2943] 8021q: adding VLAN 0 to HW filter on device lag2 [ 114.242967][ T2944] 8021q: adding VLAN 0 to HW filter on device lag2 [ 119.549725][ T2984] lag2: Port device veth3 removed [ 123.161679][ T2994] lag2: Port device veth5 removed [ 127.035655][ T3026] lag2: Mode changed to "loadbalance" [ 127.063900][ T3029] lag2: Port device veth3 added [ 127.139906][ T3032] lag2: Port device veth5 added [ 127.179438][ T3034] 8021q: adding VLAN 0 to HW filter on device lag2 [ 127.199485][ T3035] 8021q: adding VLAN 0 to HW filter on device lag2 [ 132.543767][ T2849] lag1: Port device veth4 removed [ 132.546086][ T2849] lag1: Port device veth2 removed [ 132.952010][ T3026] lag2: Port device veth5 removed [ 132.953057][ T3026] lag2: Port device veth3 removed