[ 158.614544][ T1722] gre: GRE over IPv4 demultiplexer driver [ 158.622321][ T1722] ip_gre: GRE over IPv4 tunneling driver [ 159.141797][ T1744] 8021q: 802.1Q VLAN Support v1.8 [ 232.319597][ T1972] [ 232.319686][ T1972] ======================================================== [ 232.319816][ T1972] WARNING: possible irq lock inversion dependency detected [ 232.319944][ T1972] 7.1.0-rc5-virtme #1 Not tainted [ 232.320035][ T1972] -------------------------------------------------------- [ 232.320168][ T1972] sysctl/1972 just changed the state of lock: [ 232.320277][ T1972] ffffffffad57e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.320432][ T1972] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 232.320562][ T1972] (&ndev->lock){++--}-{3:3} [ 232.320565][ T1972] [ 232.320565][ T1972] [ 232.320565][ T1972] and interrupts could create inverse lock ordering between them. [ 232.320565][ T1972] [ 232.320866][ T1972] [ 232.320866][ T1972] other info that might help us debug this: [ 232.320990][ T1972] Possible interrupt unsafe locking scenario: [ 232.320990][ T1972] [ 232.321114][ T1972] CPU0 CPU1 [ 232.321199][ T1972] ---- ---- [ 232.321282][ T1972] lock(acaddr_hash_lock); [ 232.321367][ T1972] local_irq_disable(); [ 232.321470][ T1972] lock(&ndev->lock); [ 232.321581][ T1972] lock(acaddr_hash_lock); [ 232.321687][ T1972] [ 232.321750][ T1972] lock(&ndev->lock); [ 232.321815][ T1972] [ 232.321815][ T1972] *** DEADLOCK *** [ 232.321815][ T1972] [ 232.321937][ T1972] 2 locks held by sysctl/1972: [ 232.322019][ T1972] #0: ff1100000a0ac3e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 232.322181][ T1972] #1: ffffffffad48c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 232.322349][ T1972] [ 232.322349][ T1972] the shortest dependencies between 2nd lock and 1st lock: [ 232.322493][ T1972] -> (&ndev->lock){++--}-{3:3} { [ 232.322586][ T1972] HARDIRQ-ON-W at: [ 232.322652][ T1972] __lock_acquire+0x378/0xc10 [ 232.322767][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.322872][ T1972] _raw_write_lock_bh+0x38/0x50 [ 232.322984][ T1972] addrconf_permanent_addr+0x108/0x9c0 [ 232.323114][ T1972] addrconf_notify+0x151/0xf30 [ 232.323219][ T1972] notifier_call_chain+0xb0/0x320 [ 232.323324][ T1972] __dev_notify_flags+0xde/0x280 [ 232.323431][ T1972] netif_change_flags+0xfe/0x190 [ 232.323542][ T1972] do_setlink.isra.0+0x1cc3/0x2750 [ 232.323649][ T1972] rtnl_newlink+0x8d1/0xef0 [ 232.323753][ T1972] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 232.323863][ T1972] netlink_rcv_skb+0x14e/0x3a0 [ 232.323972][ T1972] netlink_unicast+0x486/0x750 [ 232.324075][ T1972] netlink_sendmsg+0x735/0xc60 [ 232.324183][ T1972] ____sys_sendmsg+0x419/0x850 [ 232.324327][ T1972] ___sys_sendmsg+0x14e/0x1d0 [ 232.324438][ T1972] __sys_sendmsg+0x145/0x1f0 [ 232.324549][ T1972] do_syscall_64+0x117/0x590 [ 232.324652][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.324779][ T1972] HARDIRQ-ON-R at: [ 232.324846][ T1972] __lock_acquire+0x378/0xc10 [ 232.324953][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.325056][ T1972] _raw_read_lock_bh+0x44/0x80 [ 232.325165][ T1972] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 232.325295][ T1972] inet6_fill_link_af+0x5c/0xe0 [ 232.325406][ T1972] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 232.325537][ T1972] rtnl_getlink+0x9c9/0xeb0 [ 232.325642][ T1972] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 232.325745][ T1972] netlink_rcv_skb+0x14e/0x3a0 [ 232.325848][ T1972] netlink_unicast+0x486/0x750 [ 232.325952][ T1972] netlink_sendmsg+0x735/0xc60 [ 232.326058][ T1972] ____sys_sendmsg+0x419/0x850 [ 232.326169][ T1972] ___sys_sendmsg+0x14e/0x1d0 [ 232.326276][ T1972] __sys_sendmsg+0x145/0x1f0 [ 232.326379][ T1972] do_syscall_64+0x117/0x590 [ 232.326481][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.326606][ T1972] IN-SOFTIRQ-W at: [ 232.326668][ T1972] __lock_acquire+0x378/0xc10 [ 232.326771][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.326873][ T1972] _raw_write_lock+0x33/0x40 [ 232.326975][ T1972] addrconf_rs_timer+0xb0/0x770 [ 232.327077][ T1972] call_timer_fn+0x163/0x4f0 [ 232.327187][ T1972] __run_timers+0x68f/0xab0 [ 232.327291][ T1972] run_timer_softirq+0xf0/0x160 [ 232.327395][ T1972] handle_softirqs+0x1d8/0x940 [ 232.327504][ T1972] __irq_exit_rcu+0x103/0x1c0 [ 232.327606][ T1972] irq_exit_rcu+0xe/0x30 [ 232.327709][ T1972] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 232.327834][ T1972] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 232.327957][ T1972] pv_native_safe_halt+0xf/0x10 [ 232.328060][ T1972] default_idle+0x9/0x10 [ 232.328168][ T1972] default_idle_call+0x6e/0xb0 [ 232.328271][ T1972] cpuidle_idle_call.constprop.0+0x237/0x410 [ 232.328394][ T1972] do_idle+0xf5/0x160 [ 232.328502][ T1972] cpu_startup_entry+0x53/0x70 [ 232.328608][ T1972] start_secondary+0x204/0x2b0 [ 232.328712][ T1972] common_startup_64+0x13e/0x148 [ 232.328822][ T1972] IN-SOFTIRQ-R at: [ 232.328885][ T1972] __lock_acquire+0x378/0xc10 [ 232.328987][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.329091][ T1972] _raw_read_lock_bh+0x44/0x80 [ 232.329198][ T1972] ipv6_get_lladdr+0xa8/0x3f0 [ 232.329302][ T1972] addrconf_rs_timer+0x251/0x770 [ 232.329408][ T1972] call_timer_fn+0x163/0x4f0 [ 232.329525][ T1972] __run_timers+0x68f/0xab0 [ 232.329639][ T1972] run_timer_softirq+0xf0/0x160 [ 232.329742][ T1972] handle_softirqs+0x1d8/0x940 [ 232.329845][ T1972] __irq_exit_rcu+0x103/0x1c0 [ 232.329949][ T1972] irq_exit_rcu+0xe/0x30 [ 232.330051][ T1972] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 232.330176][ T1972] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 232.330298][ T1972] pv_native_safe_halt+0xf/0x10 [ 232.330403][ T1972] default_idle+0x9/0x10 [ 232.330508][ T1972] default_idle_call+0x6e/0xb0 [ 232.330611][ T1972] cpuidle_idle_call.constprop.0+0x237/0x410 [ 232.330733][ T1972] do_idle+0xf5/0x160 [ 232.330841][ T1972] cpu_startup_entry+0x53/0x70 [ 232.330949][ T1972] start_secondary+0x204/0x2b0 [ 232.331053][ T1972] common_startup_64+0x13e/0x148 [ 232.331159][ T1972] INITIAL USE at: [ 232.331221][ T1972] __lock_acquire+0x378/0xc10 [ 232.331324][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.331429][ T1972] _raw_write_lock_bh+0x38/0x50 [ 232.331539][ T1972] addrconf_permanent_addr+0x108/0x9c0 [ 232.331662][ T1972] addrconf_notify+0x151/0xf30 [ 232.331765][ T1972] notifier_call_chain+0xb0/0x320 [ 232.331869][ T1972] __dev_notify_flags+0xde/0x280 [ 232.331976][ T1972] netif_change_flags+0xfe/0x190 [ 232.332079][ T1972] do_setlink.isra.0+0x1cc3/0x2750 [ 232.332186][ T1972] rtnl_newlink+0x8d1/0xef0 [ 232.332289][ T1972] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 232.332392][ T1972] netlink_rcv_skb+0x14e/0x3a0 [ 232.332505][ T1972] netlink_unicast+0x486/0x750 [ 232.332610][ T1972] netlink_sendmsg+0x735/0xc60 [ 232.332713][ T1972] ____sys_sendmsg+0x419/0x850 [ 232.332816][ T1972] ___sys_sendmsg+0x14e/0x1d0 [ 232.332920][ T1972] __sys_sendmsg+0x145/0x1f0 [ 232.333023][ T1972] do_syscall_64+0x117/0x590 [ 232.333127][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.333249][ T1972] INITIAL READ USE at: [ 232.333330][ T1972] __lock_acquire+0x378/0xc10 [ 232.333433][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.333560][ T1972] _raw_read_lock_bh+0x44/0x80 [ 232.333665][ T1972] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 232.333787][ T1972] inet6_fill_link_af+0x5c/0xe0 [ 232.333891][ T1972] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 232.334013][ T1972] rtnl_getlink+0x9c9/0xeb0 [ 232.334117][ T1972] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 232.334222][ T1972] netlink_rcv_skb+0x14e/0x3a0 [ 232.334326][ T1972] netlink_unicast+0x486/0x750 [ 232.334433][ T1972] netlink_sendmsg+0x735/0xc60 [ 232.334541][ T1972] ____sys_sendmsg+0x419/0x850 [ 232.334642][ T1972] ___sys_sendmsg+0x14e/0x1d0 [ 232.334748][ T1972] __sys_sendmsg+0x145/0x1f0 [ 232.334850][ T1972] do_syscall_64+0x117/0x590 [ 232.334952][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.335075][ T1972] } [ 232.335127][ T1972] ... key at: [] __key.37+0x0/0x40 [ 232.335257][ T1972] ... acquired at: [ 232.335320][ T1972] __lock_acquire+0x508/0xc10 [ 232.335404][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.335485][ T1972] _raw_spin_lock+0x33/0x40 [ 232.335574][ T1972] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 232.335675][ T1972] __ipv6_dev_ac_inc+0x57e/0x930 [ 232.335758][ T1972] addrconf_join_anycast+0x1bc/0x260 [ 232.335842][ T1972] dev_forward_change+0x2be/0x850 [ 232.335925][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.336027][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.336111][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.336196][ T1972] new_sync_write+0x333/0x750 [ 232.336278][ T1972] vfs_write+0x6a4/0xc10 [ 232.336359][ T1972] ksys_write+0x116/0x250 [ 232.336444][ T1972] do_syscall_64+0x117/0x590 [ 232.336530][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.336631][ T1972] [ 232.336672][ T1972] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 232.336758][ T1972] HARDIRQ-ON-W at: [ 232.336820][ T1972] __lock_acquire+0x378/0xc10 [ 232.336924][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.337029][ T1972] _raw_spin_lock+0x33/0x40 [ 232.337133][ T1972] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 232.337256][ T1972] __ipv6_dev_ac_inc+0x57e/0x930 [ 232.337359][ T1972] addrconf_join_anycast+0x1bc/0x260 [ 232.337463][ T1972] dev_forward_change+0x2be/0x850 [ 232.337571][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.337695][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.337797][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.337901][ T1972] new_sync_write+0x333/0x750 [ 232.338003][ T1972] vfs_write+0x6a4/0xc10 [ 232.338111][ T1972] ksys_write+0x116/0x250 [ 232.338214][ T1972] do_syscall_64+0x117/0x590 [ 232.338316][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.338438][ T1972] SOFTIRQ-ON-W at: [ 232.338505][ T1972] __lock_acquire+0x378/0xc10 [ 232.338609][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.338710][ T1972] _raw_spin_lock+0x33/0x40 [ 232.338815][ T1972] __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.338917][ T1972] addrconf_leave_anycast+0x1bc/0x260 [ 232.339019][ T1972] dev_forward_change+0x3b3/0x850 [ 232.339125][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.339294][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.339397][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.339503][ T1972] new_sync_write+0x333/0x750 [ 232.339649][ T1972] vfs_write+0x6a4/0xc10 [ 232.339749][ T1972] ksys_write+0x116/0x250 [ 232.339850][ T1972] do_syscall_64+0x117/0x590 [ 232.339994][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.340120][ T1972] INITIAL USE at: [ 232.340182][ T1972] __lock_acquire+0x378/0xc10 [ 232.340283][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.340430][ T1972] _raw_spin_lock+0x33/0x40 [ 232.340537][ T1972] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 232.340659][ T1972] __ipv6_dev_ac_inc+0x57e/0x930 [ 232.340805][ T1972] addrconf_join_anycast+0x1bc/0x260 [ 232.340906][ T1972] dev_forward_change+0x2be/0x850 [ 232.341008][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.341177][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.341279][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.341381][ T1972] new_sync_write+0x333/0x750 [ 232.341530][ T1972] vfs_write+0x6a4/0xc10 [ 232.341632][ T1972] ksys_write+0x116/0x250 [ 232.341738][ T1972] do_syscall_64+0x117/0x590 [ 232.341885][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.342009][ T1972] } [ 232.342050][ T1972] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 232.342173][ T1972] ... acquired at: [ 232.342282][ T1972] mark_lock+0x1d7/0xa00 [ 232.342364][ T1972] mark_usage+0x105/0x170 [ 232.342443][ T1972] __lock_acquire+0x378/0xc10 [ 232.342527][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.342651][ T1972] _raw_spin_lock+0x33/0x40 [ 232.342732][ T1972] __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.342813][ T1972] addrconf_leave_anycast+0x1bc/0x260 [ 232.342894][ T1972] dev_forward_change+0x3b3/0x850 [ 232.343019][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.343123][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.343205][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.343290][ T1972] new_sync_write+0x333/0x750 [ 232.343414][ T1972] vfs_write+0x6a4/0xc10 [ 232.343495][ T1972] ksys_write+0x116/0x250 [ 232.343580][ T1972] do_syscall_64+0x117/0x590 [ 232.343665][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.343807][ T1972] [ 232.343848][ T1972] [ 232.343848][ T1972] stack backtrace: [ 232.343950][ T1972] CPU: 3 UID: 0 PID: 1972 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 232.343953][ T1972] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 232.343955][ T1972] Call Trace: [ 232.343957][ T1972] [ 232.343958][ T1972] dump_stack_lvl+0x6f/0xa0 [ 232.343963][ T1972] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 232.343967][ T1972] mark_lock_irq+0x989/0x9c0 [ 232.343969][ T1972] mark_lock+0x1d7/0xa00 [ 232.343971][ T1972] mark_usage+0x105/0x170 [ 232.343973][ T1972] __lock_acquire+0x378/0xc10 [ 232.343975][ T1972] lock_acquire.part.0+0xbc/0x260 [ 232.343977][ T1972] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.343978][ T1972] ? rcu_is_watching+0x15/0xd0 [ 232.343981][ T1972] ? do_raw_read_unlock+0x70/0x70 [ 232.343984][ T1972] ? lock_acquire+0x134/0x160 [ 232.343985][ T1972] _raw_spin_lock+0x33/0x40 [ 232.343987][ T1972] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.343989][ T1972] __ipv6_dev_ac_dec+0x236/0x5d0 [ 232.343991][ T1972] addrconf_leave_anycast+0x1bc/0x260 [ 232.343993][ T1972] ? find_held_lock+0x2b/0x80 [ 232.343996][ T1972] ? __ipv6_isatap_ifid+0x210/0x210 [ 232.343997][ T1972] ? mark_held_locks+0x40/0x70 [ 232.343999][ T1972] ? lockdep_hardirqs_on+0x8c/0x130 [ 232.344001][ T1972] dev_forward_change+0x3b3/0x850 [ 232.344003][ T1972] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 232.344006][ T1972] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 232.344008][ T1972] addrconf_fixup_forwarding+0x2af/0x4d0 [ 232.344011][ T1972] addrconf_sysctl_forward+0x209/0x2b0 [ 232.344013][ T1972] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 232.344016][ T1972] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 232.344018][ T1972] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 232.344021][ T1972] ? proc_sys_call_handler+0x25d/0x480 [ 232.344024][ T1972] proc_sys_call_handler+0x31b/0x480 [ 232.344026][ T1972] ? proc_sys_lookup+0x3d0/0x3d0 [ 232.344028][ T1972] ? rcu_read_unlock+0x1b/0x70 [ 232.344031][ T1972] ? do_raw_spin_unlock+0x59/0x250 [ 232.344032][ T1972] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 232.344035][ T1972] ? rcu_read_lock_any_held+0x3c/0x90 [ 232.344037][ T1972] ? proc_sys_call_handler+0x480/0x480 [ 232.344039][ T1972] new_sync_write+0x333/0x750 [ 232.344040][ T1972] ? __lock_acquire+0x508/0xc10 [ 232.344042][ T1972] ? new_sync_read+0x740/0x740 [ 232.344044][ T1972] ? lock_acquire.part.0+0xbc/0x260 [ 232.344045][ T1972] ? ksys_write+0x116/0x250 [ 232.344048][ T1972] vfs_write+0x6a4/0xc10 [ 232.344050][ T1972] ksys_write+0x116/0x250 [ 232.344051][ T1972] ? __ia32_sys_read+0xc0/0xc0 [ 232.344053][ T1972] ? rcu_is_watching+0x15/0xd0 [ 232.344054][ T1972] ? rcu_is_watching+0x15/0xd0 [ 232.344057][ T1972] do_syscall_64+0x117/0x590 [ 232.344058][ T1972] ? trace_hardirqs_off+0xd/0x30 [ 232.344061][ T1972] ? exc_page_fault+0xee/0x100 [ 232.344063][ T1972] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 232.344065][ T1972] RIP: 0033:0x7f86c129c08e [ 232.344068][ T1972] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 232.344070][ T1972] RSP: 002b:00007ffdedfce4e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 232.344073][ T1972] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f86c129c08e [ 232.344074][ T1972] RDX: 0000000000000002 RSI: 000055619e18f4b0 RDI: 0000000000000005 [ 232.344075][ T1972] RBP: 00007ffdedfce4f0 R08: 0000000000000000 R09: 0000000000000000 [ 232.344076][ T1972] R10: 0000000000000000 R11: 0000000000000202 R12: 000055619e1915e0 [ 232.344077][ T1972] R13: 000055619e18f470 R14: 0000000000000002 R15: 0000000000000000 [ 232.344079][ T1972]