[ 32.794987][ T702] 8021q: 802.1Q VLAN Support v1.8 [ 33.312146][ T731] br1: port 1(veth1) entered blocking state [ 33.312367][ T731] br1: port 1(veth1) entered disabled state [ 33.312553][ T731] veth1: entered allmulticast mode [ 33.320484][ T731] veth1: entered promiscuous mode [ 33.350673][ T106] br1: port 1(veth1) entered blocking state [ 33.350900][ T106] br1: port 1(veth1) entered forwarding state [ 42.100075][ T805] ip (805) used greatest stack depth: 23856 bytes left [ 47.428349][ T865] [ 47.428439][ T865] ======================================================== [ 47.428573][ T865] WARNING: possible irq lock inversion dependency detected [ 47.428694][ T865] 7.1.0-rc5-virtme #1 Not tainted [ 47.428775][ T865] -------------------------------------------------------- [ 47.428895][ T865] sysctl/865 just changed the state of lock: [ 47.428993][ T865] ffffffffb897e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.429146][ T865] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 47.429273][ T865] (&ndev->lock){++--}-{3:3} [ 47.429275][ T865] [ 47.429275][ T865] [ 47.429275][ T865] and interrupts could create inverse lock ordering between them. [ 47.429275][ T865] [ 47.429567][ T865] [ 47.429567][ T865] other info that might help us debug this: [ 47.429695][ T865] Possible interrupt unsafe locking scenario: [ 47.429695][ T865] [ 47.429820][ T865] CPU0 CPU1 [ 47.429910][ T865] ---- ---- [ 47.429996][ T865] lock(acaddr_hash_lock); [ 47.430077][ T865] local_irq_disable(); [ 47.430181][ T865] lock(&ndev->lock); [ 47.430281][ T865] lock(acaddr_hash_lock); [ 47.430382][ T865] [ 47.430446][ T865] lock(&ndev->lock); [ 47.430509][ T865] [ 47.430509][ T865] *** DEADLOCK *** [ 47.430509][ T865] [ 47.430637][ T865] 2 locks held by sysctl/865: [ 47.430720][ T865] #0: ff11000009aac3e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 47.430873][ T865] #1: ffffffffb888c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 47.431035][ T865] [ 47.431035][ T865] the shortest dependencies between 2nd lock and 1st lock: [ 47.431195][ T865] -> (&ndev->lock){++--}-{3:3} { [ 47.431281][ T865] HARDIRQ-ON-W at: [ 47.431345][ T865] __lock_acquire+0x378/0xc10 [ 47.431455][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.431562][ T865] _raw_write_lock_bh+0x38/0x50 [ 47.431671][ T865] addrconf_permanent_addr+0x108/0x9c0 [ 47.431798][ T865] addrconf_notify+0x151/0xf30 [ 47.431899][ T865] notifier_call_chain+0xb0/0x320 [ 47.432000][ T865] __dev_notify_flags+0xde/0x280 [ 47.432101][ T865] netif_change_flags+0xfe/0x190 [ 47.432209][ T865] do_setlink.isra.0+0x1cc3/0x2750 [ 47.432310][ T865] rtnl_newlink+0x8d1/0xef0 [ 47.432409][ T865] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 47.432508][ T865] netlink_rcv_skb+0x14e/0x3a0 [ 47.432608][ T865] netlink_unicast+0x486/0x750 [ 47.432710][ T865] netlink_sendmsg+0x735/0xc60 [ 47.432819][ T865] ____sys_sendmsg+0x419/0x850 [ 47.432921][ T865] ___sys_sendmsg+0x14e/0x1d0 [ 47.433027][ T865] __sys_sendmsg+0x145/0x1f0 [ 47.433126][ T865] do_syscall_64+0x117/0x590 [ 47.433234][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.433355][ T865] HARDIRQ-ON-R at: [ 47.433424][ T865] __lock_acquire+0x378/0xc10 [ 47.433525][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.433625][ T865] _raw_read_lock_bh+0x44/0x80 [ 47.433727][ T865] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 47.433849][ T865] inet6_fill_link_af+0x5c/0xe0 [ 47.433959][ T865] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 47.434085][ T865] rtnl_getlink+0x9c9/0xeb0 [ 47.434191][ T865] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 47.434295][ T865] netlink_rcv_skb+0x14e/0x3a0 [ 47.434396][ T865] netlink_unicast+0x486/0x750 [ 47.434497][ T865] netlink_sendmsg+0x735/0xc60 [ 47.434598][ T865] ____sys_sendmsg+0x419/0x850 [ 47.434699][ T865] ___sys_sendmsg+0x14e/0x1d0 [ 47.434800][ T865] __sys_sendmsg+0x145/0x1f0 [ 47.434902][ T865] do_syscall_64+0x117/0x590 [ 47.435004][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.435125][ T865] IN-SOFTIRQ-W at: [ 47.435190][ T865] __lock_acquire+0x378/0xc10 [ 47.435290][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.435391][ T865] _raw_write_lock+0x33/0x40 [ 47.435503][ T865] addrconf_rs_timer+0xb0/0x770 [ 47.435605][ T865] call_timer_fn+0x163/0x4f0 [ 47.435721][ T865] __run_timers+0x68f/0xab0 [ 47.435823][ T865] run_timer_softirq+0xf0/0x160 [ 47.435925][ T865] handle_softirqs+0x1d8/0x940 [ 47.436028][ T865] __irq_exit_rcu+0x103/0x1c0 [ 47.436128][ T865] irq_exit_rcu+0xe/0x30 [ 47.436230][ T865] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 47.436352][ T865] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 47.436472][ T865] pv_native_safe_halt+0xf/0x10 [ 47.436576][ T865] default_idle+0x9/0x10 [ 47.436678][ T865] default_idle_call+0x6e/0xb0 [ 47.436779][ T865] cpuidle_idle_call.constprop.0+0x237/0x410 [ 47.436908][ T865] do_idle+0xf5/0x160 [ 47.437010][ T865] cpu_startup_entry+0x53/0x70 [ 47.437111][ T865] rest_init+0x1f7/0x200 [ 47.437222][ T865] start_kernel+0x3ad/0x3b0 [ 47.437324][ T865] x86_64_start_reservations+0x24/0x30 [ 47.437445][ T865] x86_64_start_kernel+0x12b/0x130 [ 47.437545][ T865] common_startup_64+0x13e/0x148 [ 47.437645][ T865] IN-SOFTIRQ-R at: [ 47.437706][ T865] __lock_acquire+0x378/0xc10 [ 47.437806][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.437950][ T865] _raw_read_lock_bh+0x44/0x80 [ 47.438051][ T865] ipv6_get_lladdr+0xa8/0x3f0 [ 47.438153][ T865] addrconf_rs_timer+0x251/0x770 [ 47.438299][ T865] call_timer_fn+0x163/0x4f0 [ 47.438401][ T865] __run_timers+0x68f/0xab0 [ 47.438502][ T865] run_timer_softirq+0xf0/0x160 [ 47.438645][ T865] handle_softirqs+0x1d8/0x940 [ 47.438746][ T865] __irq_exit_rcu+0x103/0x1c0 [ 47.438847][ T865] irq_exit_rcu+0xe/0x30 [ 47.438989][ T865] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 47.439114][ T865] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 47.439241][ T865] pv_native_safe_halt+0xf/0x10 [ 47.439385][ T865] default_idle+0x9/0x10 [ 47.439485][ T865] default_idle_call+0x6e/0xb0 [ 47.439587][ T865] cpuidle_idle_call.constprop.0+0x237/0x410 [ 47.439749][ T865] do_idle+0xf5/0x160 [ 47.439853][ T865] cpu_startup_entry+0x53/0x70 [ 47.439953][ T865] rest_init+0x1f7/0x200 [ 47.440102][ T865] start_kernel+0x3ad/0x3b0 [ 47.440207][ T865] x86_64_start_reservations+0x24/0x30 [ 47.440329][ T865] x86_64_start_kernel+0x12b/0x130 [ 47.440472][ T865] common_startup_64+0x13e/0x148 [ 47.440572][ T865] INITIAL USE at: [ 47.440632][ T865] __lock_acquire+0x378/0xc10 [ 47.440733][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.440874][ T865] _raw_write_lock_bh+0x38/0x50 [ 47.440976][ T865] addrconf_permanent_addr+0x108/0x9c0 [ 47.441100][ T865] addrconf_notify+0x151/0xf30 [ 47.441247][ T865] notifier_call_chain+0xb0/0x320 [ 47.441347][ T865] __dev_notify_flags+0xde/0x280 [ 47.441451][ T865] netif_change_flags+0xfe/0x190 [ 47.441594][ T865] do_setlink.isra.0+0x1cc3/0x2750 [ 47.441695][ T865] rtnl_newlink+0x8d1/0xef0 [ 47.441798][ T865] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 47.441942][ T865] netlink_rcv_skb+0x14e/0x3a0 [ 47.442045][ T865] netlink_unicast+0x486/0x750 [ 47.442157][ T865] netlink_sendmsg+0x735/0xc60 [ 47.442262][ T865] ____sys_sendmsg+0x419/0x850 [ 47.442410][ T865] ___sys_sendmsg+0x14e/0x1d0 [ 47.442512][ T865] __sys_sendmsg+0x145/0x1f0 [ 47.442614][ T865] do_syscall_64+0x117/0x590 [ 47.442758][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.442882][ T865] INITIAL READ USE at: [ 47.442963][ T865] __lock_acquire+0x378/0xc10 [ 47.443112][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.443237][ T865] _raw_read_lock_bh+0x44/0x80 [ 47.443338][ T865] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 47.443503][ T865] inet6_fill_link_af+0x5c/0xe0 [ 47.443611][ T865] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 47.443734][ T865] rtnl_getlink+0x9c9/0xeb0 [ 47.443836][ T865] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 47.443935][ T865] netlink_rcv_skb+0x14e/0x3a0 [ 47.444037][ T865] netlink_unicast+0x486/0x750 [ 47.444184][ T865] netlink_sendmsg+0x735/0xc60 [ 47.444285][ T865] ____sys_sendmsg+0x419/0x850 [ 47.444389][ T865] ___sys_sendmsg+0x14e/0x1d0 [ 47.444533][ T865] __sys_sendmsg+0x145/0x1f0 [ 47.444643][ T865] do_syscall_64+0x117/0x590 [ 47.444742][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.444905][ T865] } [ 47.444947][ T865] ... key at: [] __key.37+0x0/0x40 [ 47.445073][ T865] ... acquired at: [ 47.445134][ T865] __lock_acquire+0x508/0xc10 [ 47.445260][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.445339][ T865] _raw_spin_lock+0x33/0x40 [ 47.445420][ T865] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 47.445520][ T865] __ipv6_dev_ac_inc+0x57e/0x930 [ 47.445641][ T865] addrconf_join_anycast+0x1bc/0x260 [ 47.445725][ T865] __ipv6_ifa_notify+0x75b/0xad0 [ 47.445806][ T865] addrconf_dad_completed+0x14b/0xe10 [ 47.445891][ T865] addrconf_dad_work+0x3c1/0x930 [ 47.446014][ T865] process_one_work+0xdf8/0x1410 [ 47.446096][ T865] worker_thread+0x4f1/0xd60 [ 47.446180][ T865] kthread+0x367/0x460 [ 47.446241][ T865] ret_from_fork+0x474/0x6b0 [ 47.446363][ T865] ret_from_fork_asm+0x11/0x20 [ 47.446445][ T865] [ 47.446487][ T865] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 47.446569][ T865] HARDIRQ-ON-W at: [ 47.446630][ T865] __lock_acquire+0x378/0xc10 [ 47.446772][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.446872][ T865] _raw_spin_lock+0x33/0x40 [ 47.446973][ T865] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 47.447142][ T865] __ipv6_dev_ac_inc+0x57e/0x930 [ 47.447247][ T865] addrconf_join_anycast+0x1bc/0x260 [ 47.447348][ T865] __ipv6_ifa_notify+0x75b/0xad0 [ 47.447489][ T865] addrconf_dad_completed+0x14b/0xe10 [ 47.447591][ T865] addrconf_dad_work+0x3c1/0x930 [ 47.447692][ T865] process_one_work+0xdf8/0x1410 [ 47.447794][ T865] worker_thread+0x4f1/0xd60 [ 47.447899][ T865] kthread+0x367/0x460 [ 47.447987][ T865] ret_from_fork+0x474/0x6b0 [ 47.448087][ T865] ret_from_fork_asm+0x11/0x20 [ 47.448235][ T865] SOFTIRQ-ON-W at: [ 47.448296][ T865] __lock_acquire+0x378/0xc10 [ 47.448396][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.448544][ T865] _raw_spin_lock+0x33/0x40 [ 47.448650][ T865] __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.448751][ T865] addrconf_leave_anycast+0x1bc/0x260 [ 47.448851][ T865] dev_forward_change+0x3b3/0x850 [ 47.448999][ T865] addrconf_fixup_forwarding+0x2af/0x4d0 [ 47.449123][ T865] addrconf_sysctl_forward+0x209/0x2b0 [ 47.449229][ T865] proc_sys_call_handler+0x31b/0x480 [ 47.449374][ T865] new_sync_write+0x333/0x750 [ 47.449474][ T865] vfs_write+0x6a4/0xc10 [ 47.449580][ T865] ksys_write+0x116/0x250 [ 47.449731][ T865] do_syscall_64+0x117/0x590 [ 47.449832][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.449955][ T865] INITIAL USE at: [ 47.450058][ T865] __lock_acquire+0x378/0xc10 [ 47.450158][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.450267][ T865] _raw_spin_lock+0x33/0x40 [ 47.450411][ T865] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 47.450531][ T865] __ipv6_dev_ac_inc+0x57e/0x930 [ 47.450632][ T865] addrconf_join_anycast+0x1bc/0x260 [ 47.450774][ T865] __ipv6_ifa_notify+0x75b/0xad0 [ 47.450884][ T865] addrconf_dad_completed+0x14b/0xe10 [ 47.450991][ T865] addrconf_dad_work+0x3c1/0x930 [ 47.451095][ T865] process_one_work+0xdf8/0x1410 [ 47.451240][ T865] worker_thread+0x4f1/0xd60 [ 47.451341][ T865] kthread+0x367/0x460 [ 47.451423][ T865] ret_from_fork+0x474/0x6b0 [ 47.451566][ T865] ret_from_fork_asm+0x11/0x20 [ 47.451667][ T865] } [ 47.451707][ T865] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 47.451826][ T865] ... acquired at: [ 47.451932][ T865] mark_lock+0x1d7/0xa00 [ 47.452012][ T865] mark_usage+0x105/0x170 [ 47.452091][ T865] __lock_acquire+0x378/0xc10 [ 47.452175][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.452296][ T865] _raw_spin_lock+0x33/0x40 [ 47.452376][ T865] __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.452456][ T865] addrconf_leave_anycast+0x1bc/0x260 [ 47.452538][ T865] dev_forward_change+0x3b3/0x850 [ 47.452659][ T865] addrconf_fixup_forwarding+0x2af/0x4d0 [ 47.452759][ T865] addrconf_sysctl_forward+0x209/0x2b0 [ 47.452845][ T865] proc_sys_call_handler+0x31b/0x480 [ 47.452925][ T865] new_sync_write+0x333/0x750 [ 47.453047][ T865] vfs_write+0x6a4/0xc10 [ 47.453128][ T865] ksys_write+0x116/0x250 [ 47.453221][ T865] do_syscall_64+0x117/0x590 [ 47.453301][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.453441][ T865] [ 47.453482][ T865] [ 47.453482][ T865] stack backtrace: [ 47.453583][ T865] CPU: 0 UID: 0 PID: 865 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 47.453586][ T865] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 47.453588][ T865] Call Trace: [ 47.453589][ T865] [ 47.453591][ T865] dump_stack_lvl+0x6f/0xa0 [ 47.453595][ T865] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 47.453598][ T865] mark_lock_irq+0x989/0x9c0 [ 47.453601][ T865] mark_lock+0x1d7/0xa00 [ 47.453603][ T865] mark_usage+0x105/0x170 [ 47.453604][ T865] __lock_acquire+0x378/0xc10 [ 47.453606][ T865] lock_acquire.part.0+0xbc/0x260 [ 47.453608][ T865] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.453609][ T865] ? rcu_is_watching+0x15/0xd0 [ 47.453612][ T865] ? do_raw_read_unlock+0x70/0x70 [ 47.453614][ T865] ? lock_acquire+0x134/0x160 [ 47.453616][ T865] _raw_spin_lock+0x33/0x40 [ 47.453617][ T865] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.453619][ T865] __ipv6_dev_ac_dec+0x236/0x5d0 [ 47.453621][ T865] addrconf_leave_anycast+0x1bc/0x260 [ 47.453622][ T865] ? find_held_lock+0x2b/0x80 [ 47.453625][ T865] ? __ipv6_isatap_ifid+0x210/0x210 [ 47.453627][ T865] ? mark_held_locks+0x40/0x70 [ 47.453629][ T865] ? lockdep_hardirqs_on+0x8c/0x130 [ 47.453631][ T865] dev_forward_change+0x3b3/0x850 [ 47.453633][ T865] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 47.453635][ T865] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 47.453638][ T865] addrconf_fixup_forwarding+0x2af/0x4d0 [ 47.453640][ T865] addrconf_sysctl_forward+0x209/0x2b0 [ 47.453642][ T865] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 47.453644][ T865] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 47.453646][ T865] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 47.453650][ T865] ? proc_sys_call_handler+0x25d/0x480 [ 47.453652][ T865] proc_sys_call_handler+0x31b/0x480 [ 47.453654][ T865] ? proc_sys_lookup+0x3d0/0x3d0 [ 47.453656][ T865] ? rcu_read_unlock+0x1b/0x70 [ 47.453659][ T865] ? do_raw_spin_unlock+0x59/0x250 [ 47.453660][ T865] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 47.453662][ T865] ? rcu_read_lock_any_held+0x3c/0x90 [ 47.453664][ T865] ? proc_sys_call_handler+0x480/0x480 [ 47.453666][ T865] new_sync_write+0x333/0x750 [ 47.453668][ T865] ? __lock_acquire+0x508/0xc10 [ 47.453669][ T865] ? new_sync_read+0x740/0x740 [ 47.453671][ T865] ? lock_acquire.part.0+0xbc/0x260 [ 47.453672][ T865] ? ksys_write+0x116/0x250 [ 47.453675][ T865] vfs_write+0x6a4/0xc10 [ 47.453676][ T865] ksys_write+0x116/0x250 [ 47.453678][ T865] ? __ia32_sys_read+0xc0/0xc0 [ 47.453679][ T865] ? rcu_is_watching+0x15/0xd0 [ 47.453681][ T865] ? rcu_is_watching+0x15/0xd0 [ 47.453683][ T865] do_syscall_64+0x117/0x590 [ 47.453684][ T865] ? trace_hardirqs_off+0xd/0x30 [ 47.453687][ T865] ? exc_page_fault+0xee/0x100 [ 47.453689][ T865] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 47.453691][ T865] RIP: 0033:0x7fba5c1a108e [ 47.453694][ T865] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 47.453696][ T865] RSP: 002b:00007ffcd51bcfa0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 47.453699][ T865] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fba5c1a108e [ 47.453700][ T865] RDX: 0000000000000002 RSI: 0000562a244e24b0 RDI: 0000000000000005 [ 47.453701][ T865] RBP: 00007ffcd51bcfb0 R08: 0000000000000000 R09: 0000000000000000 [ 47.453702][ T865] R10: 0000000000000000 R11: 0000000000000202 R12: 0000562a244e45e0 [ 47.453703][ T865] R13: 0000562a244e2470 R14: 0000000000000002 R15: 0000000000000000 [ 47.453705][ T865] [ 47.592351][ T875] br1: port 1(veth1) entered disabled state [ 47.610635][ T876] veth1: left allmulticast mode [ 47.610745][ T876] veth1: left promiscuous mode [ 47.610898][ T876] br1: port 1(veth1) entered disabled state