[ 96.346649][ T970] gre: GRE over IPv4 demultiplexer driver [ 96.361750][ T970] ip_gre: GRE over IPv4 tunneling driver [ 129.479017][ T1303] [ 129.479113][ T1303] ======================================================== [ 129.479249][ T1303] WARNING: possible irq lock inversion dependency detected [ 129.479373][ T1303] 7.1.0-rc5-virtme #1 Not tainted [ 129.479457][ T1303] -------------------------------------------------------- [ 129.479586][ T1303] sysctl/1303 just changed the state of lock: [ 129.479691][ T1303] ffffffffacd7e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.479848][ T1303] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 129.479969][ T1303] (&ndev->lock){++--}-{3:3} [ 129.479971][ T1303] [ 129.479971][ T1303] [ 129.479971][ T1303] and interrupts could create inverse lock ordering between them. [ 129.479971][ T1303] [ 129.480257][ T1303] [ 129.480257][ T1303] other info that might help us debug this: [ 129.480378][ T1303] Possible interrupt unsafe locking scenario: [ 129.480378][ T1303] [ 129.480500][ T1303] CPU0 CPU1 [ 129.480580][ T1303] ---- ---- [ 129.480659][ T1303] lock(acaddr_hash_lock); [ 129.480739][ T1303] local_irq_disable(); [ 129.480844][ T1303] lock(&ndev->lock); [ 129.480947][ T1303] lock(acaddr_hash_lock); [ 129.481048][ T1303] [ 129.481111][ T1303] lock(&ndev->lock); [ 129.481174][ T1303] [ 129.481174][ T1303] *** DEADLOCK *** [ 129.481174][ T1303] [ 129.481293][ T1303] 2 locks held by sysctl/1303: [ 129.481374][ T1303] #0: ff1100000a0ac3e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 129.481529][ T1303] #1: ffffffffacc8c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 129.481690][ T1303] [ 129.481690][ T1303] the shortest dependencies between 2nd lock and 1st lock: [ 129.481834][ T1303] -> (&ndev->lock){++--}-{3:3} { [ 129.481918][ T1303] HARDIRQ-ON-W at: [ 129.481984][ T1303] __lock_acquire+0x378/0xc10 [ 129.482090][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.482192][ T1303] _raw_write_lock_bh+0x38/0x50 [ 129.482297][ T1303] addrconf_permanent_addr+0x108/0x9c0 [ 129.482419][ T1303] addrconf_notify+0x151/0xf30 [ 129.482529][ T1303] notifier_call_chain+0xb0/0x320 [ 129.482630][ T1303] __dev_notify_flags+0xde/0x280 [ 129.482731][ T1303] netif_change_flags+0xfe/0x190 [ 129.482844][ T1303] do_setlink.isra.0+0x1cc3/0x2750 [ 129.482948][ T1303] rtnl_newlink+0x8d1/0xef0 [ 129.483050][ T1303] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 129.483152][ T1303] netlink_rcv_skb+0x14e/0x3a0 [ 129.483252][ T1303] netlink_unicast+0x486/0x750 [ 129.483355][ T1303] netlink_sendmsg+0x735/0xc60 [ 129.483458][ T1303] ____sys_sendmsg+0x419/0x850 [ 129.483567][ T1303] ___sys_sendmsg+0x14e/0x1d0 [ 129.483669][ T1303] __sys_sendmsg+0x145/0x1f0 [ 129.483769][ T1303] do_syscall_64+0x117/0x590 [ 129.483871][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.483992][ T1303] HARDIRQ-ON-R at: [ 129.484052][ T1303] __lock_acquire+0x378/0xc10 [ 129.484152][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.484253][ T1303] _raw_read_lock_bh+0x44/0x80 [ 129.484355][ T1303] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 129.484476][ T1303] inet6_fill_link_af+0x5c/0xe0 [ 129.484577][ T1303] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 129.484697][ T1303] rtnl_getlink+0x9c9/0xeb0 [ 129.484798][ T1303] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 129.484900][ T1303] netlink_rcv_skb+0x14e/0x3a0 [ 129.485002][ T1303] netlink_unicast+0x486/0x750 [ 129.485102][ T1303] netlink_sendmsg+0x735/0xc60 [ 129.485208][ T1303] ____sys_sendmsg+0x419/0x850 [ 129.485317][ T1303] ___sys_sendmsg+0x14e/0x1d0 [ 129.485424][ T1303] __sys_sendmsg+0x145/0x1f0 [ 129.485535][ T1303] do_syscall_64+0x117/0x590 [ 129.485641][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.485762][ T1303] IN-SOFTIRQ-W at: [ 129.485825][ T1303] __lock_acquire+0x378/0xc10 [ 129.485926][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.486026][ T1303] _raw_write_lock+0x33/0x40 [ 129.486126][ T1303] addrconf_rs_timer+0xb0/0x770 [ 129.486228][ T1303] call_timer_fn+0x163/0x4f0 [ 129.486329][ T1303] __run_timers+0x68f/0xab0 [ 129.486427][ T1303] run_timer_softirq+0xf0/0x160 [ 129.486531][ T1303] handle_softirqs+0x1d8/0x940 [ 129.486632][ T1303] __irq_exit_rcu+0x103/0x1c0 [ 129.486731][ T1303] irq_exit_rcu+0xe/0x30 [ 129.486831][ T1303] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 129.486950][ T1303] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 129.487069][ T1303] pv_native_safe_halt+0xf/0x10 [ 129.487169][ T1303] default_idle+0x9/0x10 [ 129.487270][ T1303] default_idle_call+0x6e/0xb0 [ 129.487370][ T1303] cpuidle_idle_call.constprop.0+0x237/0x410 [ 129.487498][ T1303] do_idle+0xf5/0x160 [ 129.487597][ T1303] cpu_startup_entry+0x53/0x70 [ 129.487696][ T1303] start_secondary+0x204/0x2b0 [ 129.487797][ T1303] common_startup_64+0x13e/0x148 [ 129.487898][ T1303] IN-SOFTIRQ-R at: [ 129.487959][ T1303] __lock_acquire+0x378/0xc10 [ 129.488059][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.488158][ T1303] _raw_read_lock_bh+0x44/0x80 [ 129.488257][ T1303] ipv6_get_lladdr+0xa8/0x3f0 [ 129.488357][ T1303] addrconf_rs_timer+0x251/0x770 [ 129.488457][ T1303] call_timer_fn+0x163/0x4f0 [ 129.488560][ T1303] __run_timers+0x68f/0xab0 [ 129.488660][ T1303] run_timer_softirq+0xf0/0x160 [ 129.488759][ T1303] handle_softirqs+0x1d8/0x940 [ 129.488862][ T1303] __irq_exit_rcu+0x103/0x1c0 [ 129.488960][ T1303] irq_exit_rcu+0xe/0x30 [ 129.489059][ T1303] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 129.489182][ T1303] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 129.489299][ T1303] pv_native_safe_halt+0xf/0x10 [ 129.489401][ T1303] default_idle+0x9/0x10 [ 129.489507][ T1303] default_idle_call+0x6e/0xb0 [ 129.489607][ T1303] cpuidle_idle_call.constprop.0+0x237/0x410 [ 129.489726][ T1303] do_idle+0xf5/0x160 [ 129.489827][ T1303] cpu_startup_entry+0x53/0x70 [ 129.489927][ T1303] start_secondary+0x204/0x2b0 [ 129.490026][ T1303] common_startup_64+0x13e/0x148 [ 129.490125][ T1303] INITIAL USE at: [ 129.490185][ T1303] __lock_acquire+0x378/0xc10 [ 129.490289][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.490389][ T1303] _raw_write_lock_bh+0x38/0x50 [ 129.490495][ T1303] addrconf_permanent_addr+0x108/0x9c0 [ 129.490615][ T1303] addrconf_notify+0x151/0xf30 [ 129.490714][ T1303] notifier_call_chain+0xb0/0x320 [ 129.490817][ T1303] __dev_notify_flags+0xde/0x280 [ 129.490915][ T1303] netif_change_flags+0xfe/0x190 [ 129.491013][ T1303] do_setlink.isra.0+0x1cc3/0x2750 [ 129.491113][ T1303] rtnl_newlink+0x8d1/0xef0 [ 129.491213][ T1303] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 129.491313][ T1303] netlink_rcv_skb+0x14e/0x3a0 [ 129.491411][ T1303] netlink_unicast+0x486/0x750 [ 129.491515][ T1303] netlink_sendmsg+0x735/0xc60 [ 129.491615][ T1303] ____sys_sendmsg+0x419/0x850 [ 129.491714][ T1303] ___sys_sendmsg+0x14e/0x1d0 [ 129.491818][ T1303] __sys_sendmsg+0x145/0x1f0 [ 129.491918][ T1303] do_syscall_64+0x117/0x590 [ 129.492016][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.492136][ T1303] INITIAL READ USE at: [ 129.492216][ T1303] __lock_acquire+0x378/0xc10 [ 129.492314][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.492433][ T1303] _raw_read_lock_bh+0x44/0x80 [ 129.492536][ T1303] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 129.492655][ T1303] inet6_fill_link_af+0x5c/0xe0 [ 129.492753][ T1303] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 129.492872][ T1303] rtnl_getlink+0x9c9/0xeb0 [ 129.492972][ T1303] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 129.493070][ T1303] netlink_rcv_skb+0x14e/0x3a0 [ 129.493169][ T1303] netlink_unicast+0x486/0x750 [ 129.493268][ T1303] netlink_sendmsg+0x735/0xc60 [ 129.493369][ T1303] ____sys_sendmsg+0x419/0x850 [ 129.493475][ T1303] ___sys_sendmsg+0x14e/0x1d0 [ 129.493578][ T1303] __sys_sendmsg+0x145/0x1f0 [ 129.493678][ T1303] do_syscall_64+0x117/0x590 [ 129.493780][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.493897][ T1303] } [ 129.493939][ T1303] ... key at: [] __key.37+0x0/0x40 [ 129.494103][ T1303] ... acquired at: [ 129.494163][ T1303] __lock_acquire+0x508/0xc10 [ 129.494242][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.494320][ T1303] _raw_spin_lock+0x33/0x40 [ 129.494443][ T1303] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 129.494547][ T1303] __ipv6_dev_ac_inc+0x57e/0x930 [ 129.494628][ T1303] addrconf_join_anycast+0x1bc/0x260 [ 129.494706][ T1303] dev_forward_change+0x2be/0x850 [ 129.494829][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.494928][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.495007][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.495128][ T1303] new_sync_write+0x333/0x750 [ 129.495209][ T1303] vfs_write+0x6a4/0xc10 [ 129.495289][ T1303] ksys_write+0x116/0x250 [ 129.495367][ T1303] do_syscall_64+0x117/0x590 [ 129.495490][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.495590][ T1303] [ 129.495632][ T1303] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 129.495714][ T1303] HARDIRQ-ON-W at: [ 129.495774][ T1303] __lock_acquire+0x378/0xc10 [ 129.495918][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.496017][ T1303] _raw_spin_lock+0x33/0x40 [ 129.496115][ T1303] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 129.496273][ T1303] __ipv6_dev_ac_inc+0x57e/0x930 [ 129.496370][ T1303] addrconf_join_anycast+0x1bc/0x260 [ 129.496467][ T1303] dev_forward_change+0x2be/0x850 [ 129.496614][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.496730][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.496830][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.496973][ T1303] new_sync_write+0x333/0x750 [ 129.497070][ T1303] vfs_write+0x6a4/0xc10 [ 129.497168][ T1303] ksys_write+0x116/0x250 [ 129.497309][ T1303] do_syscall_64+0x117/0x590 [ 129.497410][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.497532][ T1303] SOFTIRQ-ON-W at: [ 129.497592][ T1303] __lock_acquire+0x378/0xc10 [ 129.497736][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.497839][ T1303] _raw_spin_lock+0x33/0x40 [ 129.497938][ T1303] __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.498082][ T1303] addrconf_leave_anycast+0x1bc/0x260 [ 129.498180][ T1303] dev_forward_change+0x3b3/0x850 [ 129.498278][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.498438][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.498541][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.498639][ T1303] new_sync_write+0x333/0x750 [ 129.498782][ T1303] vfs_write+0x6a4/0xc10 [ 129.498880][ T1303] ksys_write+0x116/0x250 [ 129.498978][ T1303] do_syscall_64+0x117/0x590 [ 129.499120][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.499239][ T1303] INITIAL USE at: [ 129.499298][ T1303] __lock_acquire+0x378/0xc10 [ 129.499397][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.499541][ T1303] _raw_spin_lock+0x33/0x40 [ 129.499640][ T1303] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 129.499760][ T1303] __ipv6_dev_ac_inc+0x57e/0x930 [ 129.499866][ T1303] addrconf_join_anycast+0x1bc/0x260 [ 129.499964][ T1303] dev_forward_change+0x2be/0x850 [ 129.500064][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.500223][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.500322][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.500421][ T1303] new_sync_write+0x333/0x750 [ 129.500564][ T1303] vfs_write+0x6a4/0xc10 [ 129.500662][ T1303] ksys_write+0x116/0x250 [ 129.500762][ T1303] do_syscall_64+0x117/0x590 [ 129.500909][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.501030][ T1303] } [ 129.501069][ T1303] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 129.501227][ T1303] ... acquired at: [ 129.501284][ T1303] mark_lock+0x1d7/0xa00 [ 129.501364][ T1303] mark_usage+0x105/0x170 [ 129.501443][ T1303] __lock_acquire+0x378/0xc10 [ 129.501526][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.501645][ T1303] _raw_spin_lock+0x33/0x40 [ 129.501725][ T1303] __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.501807][ T1303] addrconf_leave_anycast+0x1bc/0x260 [ 129.501884][ T1303] dev_forward_change+0x3b3/0x850 [ 129.502003][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.502101][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.502182][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.502300][ T1303] new_sync_write+0x333/0x750 [ 129.502378][ T1303] vfs_write+0x6a4/0xc10 [ 129.502456][ T1303] ksys_write+0x116/0x250 [ 129.502538][ T1303] do_syscall_64+0x117/0x590 [ 129.502658][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.502759][ T1303] [ 129.502802][ T1303] [ 129.502802][ T1303] stack backtrace: [ 129.502906][ T1303] CPU: 2 UID: 0 PID: 1303 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 129.502909][ T1303] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 129.502911][ T1303] Call Trace: [ 129.502912][ T1303] [ 129.502913][ T1303] dump_stack_lvl+0x6f/0xa0 [ 129.502918][ T1303] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 129.502922][ T1303] mark_lock_irq+0x989/0x9c0 [ 129.502924][ T1303] mark_lock+0x1d7/0xa00 [ 129.502926][ T1303] mark_usage+0x105/0x170 [ 129.502928][ T1303] __lock_acquire+0x378/0xc10 [ 129.502930][ T1303] lock_acquire.part.0+0xbc/0x260 [ 129.502931][ T1303] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.502933][ T1303] ? rcu_is_watching+0x15/0xd0 [ 129.502936][ T1303] ? do_raw_read_unlock+0x70/0x70 [ 129.502938][ T1303] ? lock_acquire+0x134/0x160 [ 129.502939][ T1303] _raw_spin_lock+0x33/0x40 [ 129.502941][ T1303] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.502942][ T1303] __ipv6_dev_ac_dec+0x236/0x5d0 [ 129.502945][ T1303] addrconf_leave_anycast+0x1bc/0x260 [ 129.502946][ T1303] ? find_held_lock+0x2b/0x80 [ 129.502949][ T1303] ? __ipv6_isatap_ifid+0x210/0x210 [ 129.502951][ T1303] ? mark_held_locks+0x40/0x70 [ 129.502952][ T1303] ? lockdep_hardirqs_on+0x8c/0x130 [ 129.502954][ T1303] dev_forward_change+0x3b3/0x850 [ 129.502956][ T1303] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 129.502958][ T1303] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 129.502961][ T1303] addrconf_fixup_forwarding+0x2af/0x4d0 [ 129.502963][ T1303] addrconf_sysctl_forward+0x209/0x2b0 [ 129.502965][ T1303] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 129.502968][ T1303] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 129.502970][ T1303] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 129.502973][ T1303] ? proc_sys_call_handler+0x25d/0x480 [ 129.502975][ T1303] proc_sys_call_handler+0x31b/0x480 [ 129.502977][ T1303] ? proc_sys_lookup+0x3d0/0x3d0 [ 129.502979][ T1303] ? rcu_read_unlock+0x1b/0x70 [ 129.502982][ T1303] ? do_raw_spin_unlock+0x59/0x250 [ 129.502983][ T1303] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 129.502985][ T1303] ? rcu_read_lock_any_held+0x3c/0x90 [ 129.502987][ T1303] ? proc_sys_call_handler+0x480/0x480 [ 129.502989][ T1303] new_sync_write+0x333/0x750 [ 129.502991][ T1303] ? __lock_acquire+0x508/0xc10 [ 129.502992][ T1303] ? new_sync_read+0x740/0x740 [ 129.502994][ T1303] ? lock_acquire.part.0+0xbc/0x260 [ 129.502995][ T1303] ? ksys_write+0x116/0x250 [ 129.502997][ T1303] vfs_write+0x6a4/0xc10 [ 129.502999][ T1303] ksys_write+0x116/0x250 [ 129.503001][ T1303] ? __ia32_sys_read+0xc0/0xc0 [ 129.503002][ T1303] ? rcu_is_watching+0x15/0xd0 [ 129.503003][ T1303] ? rcu_is_watching+0x15/0xd0 [ 129.503005][ T1303] do_syscall_64+0x117/0x590 [ 129.503007][ T1303] ? trace_hardirqs_off+0xd/0x30 [ 129.503010][ T1303] ? exc_page_fault+0xee/0x100 [ 129.503011][ T1303] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 129.503013][ T1303] RIP: 0033:0x7ff9bcbf808e [ 129.503016][ T1303] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 129.503017][ T1303] RSP: 002b:00007ffe9a09ffc0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 129.503020][ T1303] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007ff9bcbf808e [ 129.503021][ T1303] RDX: 0000000000000002 RSI: 000055a93d3284b0 RDI: 0000000000000005 [ 129.503022][ T1303] RBP: 00007ffe9a09ffd0 R08: 0000000000000000 R09: 0000000000000000 [ 129.503023][ T1303] R10: 0000000000000000 R11: 0000000000000202 R12: 000055a93d32a5e0 [ 129.503024][ T1303] R13: 000055a93d328470 R14: 0000000000000002 R15: 0000000000000000 [ 129.503026][ T1303]