[ 49.924197][ T1119] [ 49.924283][ T1119] ======================================================== [ 49.924409][ T1119] WARNING: possible irq lock inversion dependency detected [ 49.924524][ T1119] 7.1.0-rc5-virtme #1 Not tainted [ 49.924605][ T1119] -------------------------------------------------------- [ 49.924722][ T1119] sysctl/1119 just changed the state of lock: [ 49.924825][ T1119] ffffffffa557e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.924974][ T1119] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 49.925095][ T1119] (&ndev->lock){++--}-{3:3} [ 49.925098][ T1119] [ 49.925098][ T1119] [ 49.925098][ T1119] and interrupts could create inverse lock ordering between them. [ 49.925098][ T1119] [ 49.925397][ T1119] [ 49.925397][ T1119] other info that might help us debug this: [ 49.925517][ T1119] Possible interrupt unsafe locking scenario: [ 49.925517][ T1119] [ 49.925642][ T1119] CPU0 CPU1 [ 49.925721][ T1119] ---- ---- [ 49.925800][ T1119] lock(acaddr_hash_lock); [ 49.925885][ T1119] local_irq_disable(); [ 49.925980][ T1119] lock(&ndev->lock); [ 49.926079][ T1119] lock(acaddr_hash_lock); [ 49.926181][ T1119] [ 49.926241][ T1119] lock(&ndev->lock); [ 49.926301][ T1119] [ 49.926301][ T1119] *** DEADLOCK *** [ 49.926301][ T1119] [ 49.926418][ T1119] 2 locks held by sysctl/1119: [ 49.926503][ T1119] #0: ff11000008d1c3e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 49.926651][ T1119] #1: ffffffffa548c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 49.926809][ T1119] [ 49.926809][ T1119] the shortest dependencies between 2nd lock and 1st lock: [ 49.926945][ T1119] -> (&ndev->lock){++--}-{3:3} { [ 49.927028][ T1119] HARDIRQ-ON-W at: [ 49.927088][ T1119] __lock_acquire+0x378/0xc10 [ 49.927204][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.927312][ T1119] _raw_write_lock_bh+0x38/0x50 [ 49.927413][ T1119] addrconf_permanent_addr+0x108/0x9c0 [ 49.927536][ T1119] addrconf_notify+0x151/0xf30 [ 49.927642][ T1119] notifier_call_chain+0xb0/0x320 [ 49.927746][ T1119] __dev_notify_flags+0xde/0x280 [ 49.927850][ T1119] netif_change_flags+0xfe/0x190 [ 49.927948][ T1119] do_setlink.isra.0+0x1cc3/0x2750 [ 49.928048][ T1119] rtnl_newlink+0x8d1/0xef0 [ 49.928159][ T1119] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 49.928258][ T1119] netlink_rcv_skb+0x14e/0x3a0 [ 49.928360][ T1119] netlink_unicast+0x486/0x750 [ 49.928458][ T1119] netlink_sendmsg+0x735/0xc60 [ 49.928557][ T1119] ____sys_sendmsg+0x419/0x850 [ 49.928663][ T1119] ___sys_sendmsg+0x14e/0x1d0 [ 49.928761][ T1119] __sys_sendmsg+0x145/0x1f0 [ 49.928864][ T1119] do_syscall_64+0x117/0x590 [ 49.928968][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.929088][ T1119] HARDIRQ-ON-R at: [ 49.929152][ T1119] __lock_acquire+0x378/0xc10 [ 49.929260][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.929363][ T1119] _raw_read_lock_bh+0x44/0x80 [ 49.929462][ T1119] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 49.929585][ T1119] inet6_fill_link_af+0x5c/0xe0 [ 49.929682][ T1119] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 49.929801][ T1119] rtnl_getlink+0x9c9/0xeb0 [ 49.929899][ T1119] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 49.929997][ T1119] netlink_rcv_skb+0x14e/0x3a0 [ 49.930094][ T1119] netlink_unicast+0x486/0x750 [ 49.930204][ T1119] netlink_sendmsg+0x735/0xc60 [ 49.930311][ T1119] ____sys_sendmsg+0x419/0x850 [ 49.930408][ T1119] ___sys_sendmsg+0x14e/0x1d0 [ 49.930505][ T1119] __sys_sendmsg+0x145/0x1f0 [ 49.930609][ T1119] do_syscall_64+0x117/0x590 [ 49.930708][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.930837][ T1119] IN-SOFTIRQ-W at: [ 49.930901][ T1119] __lock_acquire+0x378/0xc10 [ 49.931005][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.931106][ T1119] _raw_write_lock+0x33/0x40 [ 49.931215][ T1119] addrconf_rs_timer+0xb0/0x770 [ 49.931313][ T1119] call_timer_fn+0x163/0x4f0 [ 49.931415][ T1119] __run_timers+0x68f/0xab0 [ 49.931515][ T1119] run_timer_softirq+0xf0/0x160 [ 49.931613][ T1119] handle_softirqs+0x1d8/0x940 [ 49.931712][ T1119] __irq_exit_rcu+0x103/0x1c0 [ 49.931811][ T1119] irq_exit_rcu+0xe/0x30 [ 49.931910][ T1119] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 49.932028][ T1119] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 49.932146][ T1119] pv_native_safe_halt+0xf/0x10 [ 49.932243][ T1119] default_idle+0x9/0x10 [ 49.932346][ T1119] default_idle_call+0x6e/0xb0 [ 49.932444][ T1119] cpuidle_idle_call.constprop.0+0x237/0x410 [ 49.932564][ T1119] do_idle+0xf5/0x160 [ 49.932662][ T1119] cpu_startup_entry+0x53/0x70 [ 49.932760][ T1119] rest_init+0x1f7/0x200 [ 49.932858][ T1119] start_kernel+0x3ad/0x3b0 [ 49.932962][ T1119] x86_64_start_reservations+0x24/0x30 [ 49.933086][ T1119] x86_64_start_kernel+0x12b/0x130 [ 49.933186][ T1119] common_startup_64+0x13e/0x148 [ 49.933285][ T1119] IN-SOFTIRQ-R at: [ 49.933345][ T1119] __lock_acquire+0x378/0xc10 [ 49.933442][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.933542][ T1119] _raw_read_lock_bh+0x44/0x80 [ 49.933640][ T1119] ipv6_get_lladdr+0xa8/0x3f0 [ 49.933738][ T1119] addrconf_rs_timer+0x251/0x770 [ 49.933837][ T1119] call_timer_fn+0x163/0x4f0 [ 49.933935][ T1119] __run_timers+0x68f/0xab0 [ 49.934038][ T1119] run_timer_softirq+0xf0/0x160 [ 49.934141][ T1119] handle_softirqs+0x1d8/0x940 [ 49.934237][ T1119] __irq_exit_rcu+0x103/0x1c0 [ 49.934335][ T1119] irq_exit_rcu+0xe/0x30 [ 49.934435][ T1119] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 49.934559][ T1119] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 49.934682][ T1119] pv_native_safe_halt+0xf/0x10 [ 49.934780][ T1119] default_idle+0x9/0x10 [ 49.934876][ T1119] default_idle_call+0x6e/0xb0 [ 49.934981][ T1119] cpuidle_idle_call.constprop.0+0x237/0x410 [ 49.935102][ T1119] do_idle+0xf5/0x160 [ 49.935206][ T1119] cpu_startup_entry+0x53/0x70 [ 49.935307][ T1119] rest_init+0x1f7/0x200 [ 49.935404][ T1119] start_kernel+0x3ad/0x3b0 [ 49.935506][ T1119] x86_64_start_reservations+0x24/0x30 [ 49.935632][ T1119] x86_64_start_kernel+0x12b/0x130 [ 49.935737][ T1119] common_startup_64+0x13e/0x148 [ 49.935838][ T1119] INITIAL USE at: [ 49.935898][ T1119] __lock_acquire+0x378/0xc10 [ 49.935997][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.936098][ T1119] _raw_write_lock_bh+0x38/0x50 [ 49.936208][ T1119] addrconf_permanent_addr+0x108/0x9c0 [ 49.936329][ T1119] addrconf_notify+0x151/0xf30 [ 49.936429][ T1119] notifier_call_chain+0xb0/0x320 [ 49.936525][ T1119] __dev_notify_flags+0xde/0x280 [ 49.936622][ T1119] netif_change_flags+0xfe/0x190 [ 49.936722][ T1119] do_setlink.isra.0+0x1cc3/0x2750 [ 49.936822][ T1119] rtnl_newlink+0x8d1/0xef0 [ 49.936926][ T1119] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 49.937028][ T1119] netlink_rcv_skb+0x14e/0x3a0 [ 49.937130][ T1119] netlink_unicast+0x486/0x750 [ 49.937236][ T1119] netlink_sendmsg+0x735/0xc60 [ 49.937336][ T1119] ____sys_sendmsg+0x419/0x850 [ 49.937439][ T1119] ___sys_sendmsg+0x14e/0x1d0 [ 49.937542][ T1119] __sys_sendmsg+0x145/0x1f0 [ 49.937643][ T1119] do_syscall_64+0x117/0x590 [ 49.937745][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.937870][ T1119] INITIAL READ USE at: [ 49.937951][ T1119] __lock_acquire+0x378/0xc10 [ 49.938052][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.938174][ T1119] _raw_read_lock_bh+0x44/0x80 [ 49.938277][ T1119] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 49.938401][ T1119] inet6_fill_link_af+0x5c/0xe0 [ 49.938504][ T1119] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 49.938628][ T1119] rtnl_getlink+0x9c9/0xeb0 [ 49.938730][ T1119] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 49.938827][ T1119] netlink_rcv_skb+0x14e/0x3a0 [ 49.938930][ T1119] netlink_unicast+0x486/0x750 [ 49.939028][ T1119] netlink_sendmsg+0x735/0xc60 [ 49.939134][ T1119] ____sys_sendmsg+0x419/0x850 [ 49.939236][ T1119] ___sys_sendmsg+0x14e/0x1d0 [ 49.939338][ T1119] __sys_sendmsg+0x145/0x1f0 [ 49.939439][ T1119] do_syscall_64+0x117/0x590 [ 49.939540][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.939663][ T1119] } [ 49.939708][ T1119] ... key at: [] __key.37+0x0/0x40 [ 49.939827][ T1119] ... acquired at: [ 49.939886][ T1119] __lock_acquire+0x508/0xc10 [ 49.939966][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.940045][ T1119] _raw_spin_lock+0x33/0x40 [ 49.940127][ T1119] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 49.940232][ T1119] __ipv6_dev_ac_inc+0x57e/0x930 [ 49.940314][ T1119] addrconf_join_anycast+0x1bc/0x260 [ 49.940399][ T1119] __ipv6_ifa_notify+0x75b/0xad0 [ 49.940477][ T1119] addrconf_dad_completed+0x14b/0xe10 [ 49.940560][ T1119] addrconf_dad_work+0x3c1/0x930 [ 49.940636][ T1119] process_one_work+0xdf8/0x1410 [ 49.940722][ T1119] worker_thread+0x4f1/0xd60 [ 49.940804][ T1119] kthread+0x367/0x460 [ 49.940864][ T1119] ret_from_fork+0x474/0x6b0 [ 49.940943][ T1119] ret_from_fork_asm+0x11/0x20 [ 49.941029][ T1119] [ 49.941068][ T1119] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 49.941155][ T1119] HARDIRQ-ON-W at: [ 49.941217][ T1119] __lock_acquire+0x378/0xc10 [ 49.941319][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.941420][ T1119] _raw_spin_lock+0x33/0x40 [ 49.941529][ T1119] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 49.941644][ T1119] __ipv6_dev_ac_inc+0x57e/0x930 [ 49.941748][ T1119] addrconf_join_anycast+0x1bc/0x260 [ 49.941854][ T1119] __ipv6_ifa_notify+0x75b/0xad0 [ 49.941951][ T1119] addrconf_dad_completed+0x14b/0xe10 [ 49.942048][ T1119] addrconf_dad_work+0x3c1/0x930 [ 49.942151][ T1119] process_one_work+0xdf8/0x1410 [ 49.942249][ T1119] worker_thread+0x4f1/0xd60 [ 49.942347][ T1119] kthread+0x367/0x460 [ 49.942434][ T1119] ret_from_fork+0x474/0x6b0 [ 49.942533][ T1119] ret_from_fork_asm+0x11/0x20 [ 49.942636][ T1119] SOFTIRQ-ON-W at: [ 49.942695][ T1119] __lock_acquire+0x378/0xc10 [ 49.942792][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.942889][ T1119] _raw_spin_lock+0x33/0x40 [ 49.942994][ T1119] __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.943091][ T1119] addrconf_leave_anycast+0x1bc/0x260 [ 49.943194][ T1119] dev_forward_change+0x3b3/0x850 [ 49.943296][ T1119] addrconf_fixup_forwarding+0x2af/0x4d0 [ 49.943412][ T1119] addrconf_sysctl_forward+0x209/0x2b0 [ 49.943513][ T1119] proc_sys_call_handler+0x31b/0x480 [ 49.943614][ T1119] new_sync_write+0x333/0x750 [ 49.943755][ T1119] vfs_write+0x6a4/0xc10 [ 49.943852][ T1119] ksys_write+0x116/0x250 [ 49.943949][ T1119] do_syscall_64+0x117/0x590 [ 49.944086][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.944209][ T1119] INITIAL USE at: [ 49.944267][ T1119] __lock_acquire+0x378/0xc10 [ 49.944364][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.944505][ T1119] _raw_spin_lock+0x33/0x40 [ 49.944603][ T1119] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 49.944722][ T1119] __ipv6_dev_ac_inc+0x57e/0x930 [ 49.944862][ T1119] addrconf_join_anycast+0x1bc/0x260 [ 49.944964][ T1119] __ipv6_ifa_notify+0x75b/0xad0 [ 49.945061][ T1119] addrconf_dad_completed+0x14b/0xe10 [ 49.945206][ T1119] addrconf_dad_work+0x3c1/0x930 [ 49.945304][ T1119] process_one_work+0xdf8/0x1410 [ 49.945403][ T1119] worker_thread+0x4f1/0xd60 [ 49.945542][ T1119] kthread+0x367/0x460 [ 49.945618][ T1119] ret_from_fork+0x474/0x6b0 [ 49.945724][ T1119] ret_from_fork_asm+0x11/0x20 [ 49.945821][ T1119] } [ 49.945901][ T1119] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 49.946021][ T1119] ... acquired at: [ 49.946081][ T1119] mark_lock+0x1d7/0xa00 [ 49.946163][ T1119] mark_usage+0x105/0x170 [ 49.946280][ T1119] __lock_acquire+0x378/0xc10 [ 49.946356][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.946435][ T1119] _raw_spin_lock+0x33/0x40 [ 49.946512][ T1119] __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.946630][ T1119] addrconf_leave_anycast+0x1bc/0x260 [ 49.946707][ T1119] dev_forward_change+0x3b3/0x850 [ 49.946784][ T1119] addrconf_fixup_forwarding+0x2af/0x4d0 [ 49.946880][ T1119] addrconf_sysctl_forward+0x209/0x2b0 [ 49.947000][ T1119] proc_sys_call_handler+0x31b/0x480 [ 49.947077][ T1119] new_sync_write+0x333/0x750 [ 49.947158][ T1119] vfs_write+0x6a4/0xc10 [ 49.947237][ T1119] ksys_write+0x116/0x250 [ 49.947354][ T1119] do_syscall_64+0x117/0x590 [ 49.947433][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.947528][ T1119] [ 49.947567][ T1119] [ 49.947567][ T1119] stack backtrace: [ 49.947713][ T1119] CPU: 0 UID: 0 PID: 1119 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 49.947716][ T1119] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 49.947718][ T1119] Call Trace: [ 49.947720][ T1119] [ 49.947721][ T1119] dump_stack_lvl+0x6f/0xa0 [ 49.947725][ T1119] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 49.947728][ T1119] mark_lock_irq+0x989/0x9c0 [ 49.947731][ T1119] mark_lock+0x1d7/0xa00 [ 49.947733][ T1119] mark_usage+0x105/0x170 [ 49.947734][ T1119] __lock_acquire+0x378/0xc10 [ 49.947737][ T1119] lock_acquire.part.0+0xbc/0x260 [ 49.947738][ T1119] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.947740][ T1119] ? rcu_is_watching+0x15/0xd0 [ 49.947743][ T1119] ? do_raw_read_unlock+0x70/0x70 [ 49.947745][ T1119] ? lock_acquire+0x134/0x160 [ 49.947747][ T1119] _raw_spin_lock+0x33/0x40 [ 49.947748][ T1119] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.947749][ T1119] __ipv6_dev_ac_dec+0x236/0x5d0 [ 49.947752][ T1119] addrconf_leave_anycast+0x1bc/0x260 [ 49.947753][ T1119] ? find_held_lock+0x2b/0x80 [ 49.947756][ T1119] ? __ipv6_isatap_ifid+0x210/0x210 [ 49.947758][ T1119] ? mark_held_locks+0x40/0x70 [ 49.947759][ T1119] ? lockdep_hardirqs_on+0x8c/0x130 [ 49.947761][ T1119] dev_forward_change+0x3b3/0x850 [ 49.947764][ T1119] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 49.947765][ T1119] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 49.947768][ T1119] addrconf_fixup_forwarding+0x2af/0x4d0 [ 49.947771][ T1119] addrconf_sysctl_forward+0x209/0x2b0 [ 49.947773][ T1119] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 49.947775][ T1119] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 49.947777][ T1119] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 49.947780][ T1119] ? proc_sys_call_handler+0x25d/0x480 [ 49.947783][ T1119] proc_sys_call_handler+0x31b/0x480 [ 49.947785][ T1119] ? proc_sys_lookup+0x3d0/0x3d0 [ 49.947787][ T1119] ? rcu_read_unlock+0x1b/0x70 [ 49.947790][ T1119] ? do_raw_spin_unlock+0x59/0x250 [ 49.947791][ T1119] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 49.947793][ T1119] ? rcu_read_lock_any_held+0x3c/0x90 [ 49.947795][ T1119] ? proc_sys_call_handler+0x480/0x480 [ 49.947797][ T1119] new_sync_write+0x333/0x750 [ 49.947799][ T1119] ? __lock_acquire+0x508/0xc10 [ 49.947800][ T1119] ? new_sync_read+0x740/0x740 [ 49.947802][ T1119] ? lock_acquire.part.0+0xbc/0x260 [ 49.947803][ T1119] ? ksys_write+0x116/0x250 [ 49.947805][ T1119] vfs_write+0x6a4/0xc10 [ 49.947807][ T1119] ksys_write+0x116/0x250 [ 49.947809][ T1119] ? __ia32_sys_read+0xc0/0xc0 [ 49.947810][ T1119] ? rcu_is_watching+0x15/0xd0 [ 49.947812][ T1119] ? rcu_is_watching+0x15/0xd0 [ 49.947814][ T1119] do_syscall_64+0x117/0x590 [ 49.947815][ T1119] ? trace_hardirqs_off+0xd/0x30 [ 49.947818][ T1119] ? exc_page_fault+0xee/0x100 [ 49.947820][ T1119] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 49.947822][ T1119] RIP: 0033:0x7f948247808e [ 49.947825][ T1119] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 49.947826][ T1119] RSP: 002b:00007ffc682bc470 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 49.947829][ T1119] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f948247808e [ 49.947830][ T1119] RDX: 0000000000000002 RSI: 000056205239f4b0 RDI: 0000000000000005 [ 49.947831][ T1119] RBP: 00007ffc682bc480 R08: 0000000000000000 R09: 0000000000000000 [ 49.947832][ T1119] R10: 0000000000000000 R11: 0000000000000202 R12: 00005620523a15e0 [ 49.947833][ T1119] R13: 000056205239f470 R14: 0000000000000002 R15: 0000000000000000 [ 49.947835][ T1119]