[ 8.605935][ T192] ip (192) used greatest stack depth: 24096 bytes left [ 9.739328][ T236] br1: port 1(veth1) entered blocking state [ 9.739619][ T236] br1: port 1(veth1) entered disabled state [ 9.739805][ T236] veth1: entered allmulticast mode [ 9.741515][ T236] veth1: entered promiscuous mode [ 9.770521][ T36] br1: port 1(veth1) entered blocking state [ 9.770810][ T36] br1: port 1(veth1) entered forwarding state [ 17.318080][ T294] veth1: left allmulticast mode [ 17.318307][ T294] veth1: left promiscuous mode [ 17.318581][ T294] br1: port 1(veth1) entered disabled state [ 19.357103][ T296] br1: port 1(veth1) entered blocking state [ 19.357363][ T296] br1: port 1(veth1) entered disabled state [ 19.357608][ T296] veth1: entered allmulticast mode [ 19.359668][ T296] veth1: entered promiscuous mode [ 19.360117][ T296] br1: port 1(veth1) entered blocking state [ 19.360333][ T296] br1: port 1(veth1) entered forwarding state [ 37.988440][ T403] veth1: left allmulticast mode [ 37.988666][ T403] veth1: left promiscuous mode [ 37.988939][ T403] br1: port 1(veth1) entered disabled state [ 40.046587][ T406] br1: port 1(veth1) entered blocking state [ 40.046779][ T406] br1: port 1(veth1) entered disabled state [ 40.046949][ T406] veth1: entered allmulticast mode [ 40.048406][ T406] veth1: entered promiscuous mode [ 40.048664][ T406] br1: port 1(veth1) entered blocking state [ 40.048828][ T406] br1: port 1(veth1) entered forwarding state [ 42.208978][ T440] [ 42.209057][ T440] ======================================================== [ 42.209183][ T440] WARNING: possible irq lock inversion dependency detected [ 42.209294][ T440] 7.1.0-rc5-virtme #1 Not tainted [ 42.209371][ T440] -------------------------------------------------------- [ 42.209484][ T440] sysctl/440 just changed the state of lock: [ 42.209579][ T440] ffffffff8817e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.209724][ T440] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 42.209835][ T440] (&ndev->lock){++.-}-{3:3} [ 42.209838][ T440] [ 42.209838][ T440] [ 42.209838][ T440] and interrupts could create inverse lock ordering between them. [ 42.209838][ T440] [ 42.210112][ T440] [ 42.210112][ T440] other info that might help us debug this: [ 42.210227][ T440] Possible interrupt unsafe locking scenario: [ 42.210227][ T440] [ 42.210341][ T440] CPU0 CPU1 [ 42.210417][ T440] ---- ---- [ 42.210491][ T440] lock(acaddr_hash_lock); [ 42.210569][ T440] local_irq_disable(); [ 42.210667][ T440] lock(&ndev->lock); [ 42.210761][ T440] lock(acaddr_hash_lock); [ 42.210854][ T440] [ 42.210913][ T440] lock(&ndev->lock); [ 42.210972][ T440] [ 42.210972][ T440] *** DEADLOCK *** [ 42.210972][ T440] [ 42.211084][ T440] 2 locks held by sysctl/440: [ 42.211165][ T440] #0: ff11000010c093e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 42.211303][ T440] #1: ffffffff8808c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 42.211456][ T440] [ 42.211456][ T440] the shortest dependencies between 2nd lock and 1st lock: [ 42.211589][ T440] -> (&ndev->lock){++.-}-{3:3} { [ 42.211669][ T440] HARDIRQ-ON-W at: [ 42.211736][ T440] __lock_acquire+0x378/0xc10 [ 42.211843][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.211937][ T440] _raw_write_lock_bh+0x38/0x50 [ 42.212035][ T440] addrconf_permanent_addr+0x108/0x9c0 [ 42.212154][ T440] addrconf_notify+0x151/0xf30 [ 42.212254][ T440] notifier_call_chain+0xb0/0x320 [ 42.212349][ T440] __dev_notify_flags+0xde/0x280 [ 42.212444][ T440] netif_change_flags+0xfe/0x190 [ 42.212537][ T440] do_setlink.isra.0+0x1cc3/0x2750 [ 42.212633][ T440] rtnl_newlink+0x8d1/0xef0 [ 42.212729][ T440] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.212824][ T440] netlink_rcv_skb+0x14e/0x3a0 [ 42.212920][ T440] netlink_unicast+0x486/0x750 [ 42.213014][ T440] netlink_sendmsg+0x735/0xc60 [ 42.213109][ T440] ____sys_sendmsg+0x419/0x850 [ 42.213210][ T440] ___sys_sendmsg+0x14e/0x1d0 [ 42.213306][ T440] __sys_sendmsg+0x145/0x1f0 [ 42.213401][ T440] do_syscall_64+0x117/0x590 [ 42.213498][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.213615][ T440] HARDIRQ-ON-R at: [ 42.213672][ T440] __lock_acquire+0x378/0xc10 [ 42.213771][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.213865][ T440] _raw_read_lock_bh+0x44/0x80 [ 42.213959][ T440] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 42.214073][ T440] inet6_fill_link_af+0x5c/0xe0 [ 42.214171][ T440] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 42.214284][ T440] rtnl_getlink+0x9c9/0xeb0 [ 42.214378][ T440] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.214472][ T440] netlink_rcv_skb+0x14e/0x3a0 [ 42.214567][ T440] netlink_unicast+0x486/0x750 [ 42.214661][ T440] netlink_sendmsg+0x735/0xc60 [ 42.214757][ T440] ____sys_sendmsg+0x419/0x850 [ 42.214851][ T440] ___sys_sendmsg+0x14e/0x1d0 [ 42.214945][ T440] __sys_sendmsg+0x145/0x1f0 [ 42.215040][ T440] do_syscall_64+0x117/0x590 [ 42.215134][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.215251][ T440] IN-SOFTIRQ-R at: [ 42.215308][ T440] __lock_acquire+0x378/0xc10 [ 42.215407][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.215501][ T440] _raw_read_lock_bh+0x44/0x80 [ 42.215596][ T440] ipv6_get_lladdr+0xa8/0x3f0 [ 42.215695][ T440] ndisc_send_ns+0xf3/0x120 [ 42.215793][ T440] ndisc_solicit+0x314/0x480 [ 42.215888][ T440] neigh_probe+0xb3/0x100 [ 42.215983][ T440] __neigh_event_send+0x802/0x1070 [ 42.216078][ T440] neigh_resolve_output+0xbe/0x140 [ 42.216176][ T440] ip6_finish_output2+0x423/0x1300 [ 42.216272][ T440] ip6_finish_output+0x646/0xda0 [ 42.216366][ T440] ip6_output+0x23f/0x7f0 [ 42.216460][ T440] ip6_forward+0x10f4/0x2bd0 [ 42.216555][ T440] ipv6_rcv+0x3c1/0x5a0 [ 42.216649][ T440] __netif_receive_skb_one_core+0xfc/0x180 [ 42.216769][ T440] netif_receive_skb_internal+0xb0/0x330 [ 42.216891][ T440] netif_receive_skb+0x56/0x380 [ 42.216985][ T440] br_handle_frame_finish+0xc3b/0x1fb0 [bridge] [ 42.217109][ T440] br_handle_frame+0x739/0xf40 [bridge] [ 42.217231][ T440] __netif_receive_skb_core.constprop.0+0x536/0x2bc0 [ 42.217345][ T440] __netif_receive_skb_one_core+0xa2/0x180 [ 42.217460][ T440] process_backlog+0x2bc/0x1490 [ 42.217555][ T440] __napi_poll+0xa7/0x3b0 [ 42.217649][ T440] net_rx_action+0x513/0xf50 [ 42.217742][ T440] handle_softirqs+0x1d8/0x940 [ 42.217837][ T440] do_softirq+0xa9/0xe0 [ 42.217933][ T440] __local_bh_enable_ip+0x113/0x140 [ 42.218027][ T440] __neigh_event_send+0x30a/0x1070 [ 42.218123][ T440] neigh_resolve_output+0xbe/0x140 [ 42.218221][ T440] ip6_finish_output2+0x423/0x1300 [ 42.218318][ T440] ip6_finish_output+0x646/0xda0 [ 42.218413][ T440] ip6_output+0x23f/0x7f0 [ 42.218507][ T440] ip6_send_skb+0xee/0x350 [ 42.218601][ T440] rawv6_push_pending_frames+0x4cb/0xa90 [ 42.218712][ T440] rawv6_sendmsg+0x1434/0x1a50 [ 42.218806][ T440] __sys_sendto+0x2c9/0x400 [ 42.218900][ T440] __x64_sys_sendto+0xe4/0x1f0 [ 42.218996][ T440] do_syscall_64+0x117/0x590 [ 42.219093][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.219252][ T440] INITIAL USE at: [ 42.219309][ T440] __lock_acquire+0x378/0xc10 [ 42.219403][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.219496][ T440] _raw_write_lock_bh+0x38/0x50 [ 42.219633][ T440] addrconf_permanent_addr+0x108/0x9c0 [ 42.219752][ T440] addrconf_notify+0x151/0xf30 [ 42.219847][ T440] notifier_call_chain+0xb0/0x320 [ 42.219985][ T440] __dev_notify_flags+0xde/0x280 [ 42.220085][ T440] netif_change_flags+0xfe/0x190 [ 42.220186][ T440] do_setlink.isra.0+0x1cc3/0x2750 [ 42.220320][ T440] rtnl_newlink+0x8d1/0xef0 [ 42.220416][ T440] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.220510][ T440] netlink_rcv_skb+0x14e/0x3a0 [ 42.220648][ T440] netlink_unicast+0x486/0x750 [ 42.220745][ T440] netlink_sendmsg+0x735/0xc60 [ 42.220840][ T440] ____sys_sendmsg+0x419/0x850 [ 42.220943][ T440] ___sys_sendmsg+0x14e/0x1d0 [ 42.221083][ T440] __sys_sendmsg+0x145/0x1f0 [ 42.221182][ T440] do_syscall_64+0x117/0x590 [ 42.221277][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.221429][ T440] INITIAL READ USE at: [ 42.221506][ T440] __lock_acquire+0x378/0xc10 [ 42.221600][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.221753][ T440] _raw_read_lock_bh+0x44/0x80 [ 42.221848][ T440] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 42.221963][ T440] inet6_fill_link_af+0x5c/0xe0 [ 42.222059][ T440] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 42.222179][ T440] rtnl_getlink+0x9c9/0xeb0 [ 42.222273][ T440] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.222407][ T440] netlink_rcv_skb+0x14e/0x3a0 [ 42.222503][ T440] netlink_unicast+0x486/0x750 [ 42.222597][ T440] netlink_sendmsg+0x735/0xc60 [ 42.222732][ T440] ____sys_sendmsg+0x419/0x850 [ 42.222828][ T440] ___sys_sendmsg+0x14e/0x1d0 [ 42.222923][ T440] __sys_sendmsg+0x145/0x1f0 [ 42.223058][ T440] do_syscall_64+0x117/0x590 [ 42.223154][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.223266][ T440] } [ 42.223305][ T440] ... key at: [] __key.37+0x0/0x40 [ 42.223459][ T440] ... acquired at: [ 42.223516][ T440] __lock_acquire+0x508/0xc10 [ 42.223593][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.223667][ T440] _raw_spin_lock+0x33/0x40 [ 42.223782][ T440] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.223876][ T440] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.223950][ T440] addrconf_join_anycast+0x1bc/0x260 [ 42.224025][ T440] __ipv6_ifa_notify+0x75b/0xad0 [ 42.224101][ T440] addrconf_dad_completed+0x14b/0xe10 [ 42.224179][ T440] addrconf_dad_work+0x3c1/0x930 [ 42.224254][ T440] process_one_work+0xdf8/0x1410 [ 42.224369][ T440] worker_thread+0x4f1/0xd60 [ 42.224444][ T440] kthread+0x367/0x460 [ 42.224501][ T440] ret_from_fork+0x474/0x6b0 [ 42.224578][ T440] ret_from_fork_asm+0x11/0x20 [ 42.224654][ T440] [ 42.224733][ T440] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 42.224811][ T440] HARDIRQ-ON-W at: [ 42.224867][ T440] __lock_acquire+0x378/0xc10 [ 42.224962][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.225095][ T440] _raw_spin_lock+0x33/0x40 [ 42.225193][ T440] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.225309][ T440] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.225444][ T440] addrconf_join_anycast+0x1bc/0x260 [ 42.225546][ T440] __ipv6_ifa_notify+0x75b/0xad0 [ 42.225639][ T440] addrconf_dad_completed+0x14b/0xe10 [ 42.225775][ T440] addrconf_dad_work+0x3c1/0x930 [ 42.225873][ T440] process_one_work+0xdf8/0x1410 [ 42.225968][ T440] worker_thread+0x4f1/0xd60 [ 42.226103][ T440] kthread+0x367/0x460 [ 42.226184][ T440] ret_from_fork+0x474/0x6b0 [ 42.226278][ T440] ret_from_fork_asm+0x11/0x20 [ 42.226372][ T440] SOFTIRQ-ON-W at: [ 42.226468][ T440] __lock_acquire+0x378/0xc10 [ 42.226561][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.226654][ T440] _raw_spin_lock+0x33/0x40 [ 42.226792][ T440] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.226885][ T440] addrconf_leave_anycast+0x1bc/0x260 [ 42.226979][ T440] dev_forward_change+0x3b3/0x850 [ 42.227075][ T440] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.227239][ T440] addrconf_sysctl_forward+0x209/0x2b0 [ 42.227334][ T440] proc_sys_call_handler+0x31b/0x480 [ 42.227434][ T440] new_sync_write+0x333/0x750 [ 42.227569][ T440] vfs_write+0x6a4/0xc10 [ 42.227664][ T440] ksys_write+0x116/0x250 [ 42.227757][ T440] do_syscall_64+0x117/0x590 [ 42.227892][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.228004][ T440] INITIAL USE at: [ 42.228059][ T440] __lock_acquire+0x378/0xc10 [ 42.228204][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.228298][ T440] _raw_spin_lock+0x33/0x40 [ 42.228391][ T440] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.228544][ T440] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.228646][ T440] addrconf_join_anycast+0x1bc/0x260 [ 42.228739][ T440] __ipv6_ifa_notify+0x75b/0xad0 [ 42.228874][ T440] addrconf_dad_completed+0x14b/0xe10 [ 42.228969][ T440] addrconf_dad_work+0x3c1/0x930 [ 42.229065][ T440] process_one_work+0xdf8/0x1410 [ 42.229164][ T440] worker_thread+0x4f1/0xd60 [ 42.229298][ T440] kthread+0x367/0x460 [ 42.229373][ T440] ret_from_fork+0x474/0x6b0 [ 42.229467][ T440] ret_from_fork_asm+0x11/0x20 [ 42.229602][ T440] } [ 42.229641][ T440] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 42.229753][ T440] ... acquired at: [ 42.229809][ T440] mark_lock+0x1d7/0xa00 [ 42.229922][ T440] mark_usage+0x105/0x170 [ 42.229996][ T440] __lock_acquire+0x378/0xc10 [ 42.230074][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.230153][ T440] _raw_spin_lock+0x33/0x40 [ 42.230267][ T440] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.230343][ T440] addrconf_leave_anycast+0x1bc/0x260 [ 42.230426][ T440] dev_forward_change+0x3b3/0x850 [ 42.230500][ T440] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.230632][ T440] addrconf_sysctl_forward+0x209/0x2b0 [ 42.230707][ T440] proc_sys_call_handler+0x31b/0x480 [ 42.230782][ T440] new_sync_write+0x333/0x750 [ 42.230857][ T440] vfs_write+0x6a4/0xc10 [ 42.230976][ T440] ksys_write+0x116/0x250 [ 42.231052][ T440] do_syscall_64+0x117/0x590 [ 42.231127][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.231225][ T440] [ 42.231301][ T440] [ 42.231301][ T440] stack backtrace: [ 42.231400][ T440] CPU: 3 UID: 0 PID: 440 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 42.231402][ T440] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 42.231404][ T440] Call Trace: [ 42.231406][ T440] [ 42.231407][ T440] dump_stack_lvl+0x6f/0xa0 [ 42.231411][ T440] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 42.231414][ T440] mark_lock_irq+0x989/0x9c0 [ 42.231417][ T440] mark_lock+0x1d7/0xa00 [ 42.231419][ T440] mark_usage+0x105/0x170 [ 42.231420][ T440] __lock_acquire+0x378/0xc10 [ 42.231422][ T440] lock_acquire.part.0+0xbc/0x260 [ 42.231424][ T440] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.231425][ T440] ? rcu_is_watching+0x15/0xd0 [ 42.231428][ T440] ? do_raw_read_unlock+0x70/0x70 [ 42.231430][ T440] ? lock_acquire+0x134/0x160 [ 42.231432][ T440] _raw_spin_lock+0x33/0x40 [ 42.231434][ T440] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.231435][ T440] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.231437][ T440] addrconf_leave_anycast+0x1bc/0x260 [ 42.231439][ T440] ? find_held_lock+0x2b/0x80 [ 42.231442][ T440] ? __ipv6_isatap_ifid+0x210/0x210 [ 42.231443][ T440] ? mark_held_locks+0x40/0x70 [ 42.231445][ T440] ? lockdep_hardirqs_on+0x8c/0x130 [ 42.231447][ T440] dev_forward_change+0x3b3/0x850 [ 42.231449][ T440] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 42.231451][ T440] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 42.231454][ T440] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.231456][ T440] addrconf_sysctl_forward+0x209/0x2b0 [ 42.231458][ T440] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 42.231460][ T440] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 42.231462][ T440] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 42.231466][ T440] ? proc_sys_call_handler+0x25d/0x480 [ 42.231468][ T440] proc_sys_call_handler+0x31b/0x480 [ 42.231470][ T440] ? proc_sys_lookup+0x3d0/0x3d0 [ 42.231472][ T440] ? rcu_read_unlock+0x1b/0x70 [ 42.231474][ T440] ? do_raw_spin_unlock+0x59/0x250 [ 42.231476][ T440] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 42.231478][ T440] ? rcu_read_lock_any_held+0x3c/0x90 [ 42.231480][ T440] ? proc_sys_call_handler+0x480/0x480 [ 42.231482][ T440] new_sync_write+0x333/0x750 [ 42.231483][ T440] ? __lock_acquire+0x508/0xc10 [ 42.231484][ T440] ? new_sync_read+0x740/0x740 [ 42.231486][ T440] ? lock_acquire.part.0+0xbc/0x260 [ 42.231487][ T440] ? ksys_write+0x116/0x250 [ 42.231489][ T440] vfs_write+0x6a4/0xc10 [ 42.231491][ T440] ksys_write+0x116/0x250 [ 42.231493][ T440] ? __ia32_sys_read+0xc0/0xc0 [ 42.231494][ T440] ? rcu_is_watching+0x15/0xd0 [ 42.231495][ T440] ? rcu_is_watching+0x15/0xd0 [ 42.231497][ T440] do_syscall_64+0x117/0x590 [ 42.231499][ T440] ? trace_hardirqs_off+0xd/0x30 [ 42.231502][ T440] ? exc_page_fault+0xee/0x100 [ 42.231504][ T440] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.231505][ T440] RIP: 0033:0x7f7906aed08e [ 42.231507][ T440] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 42.231509][ T440] RSP: 002b:00007ffd82307810 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 42.231512][ T440] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f7906aed08e [ 42.231513][ T440] RDX: 0000000000000002 RSI: 0000561f2b8cd4b0 RDI: 0000000000000005 [ 42.231514][ T440] RBP: 00007ffd82307820 R08: 0000000000000000 R09: 0000000000000000 [ 42.231515][ T440] R10: 0000000000000000 R11: 0000000000000202 R12: 0000561f2b8cf5e0 [ 42.231515][ T440] R13: 0000561f2b8cd470 R14: 0000000000000002 R15: 0000000000000000 [ 42.231518][ T440] [ 42.329690][ T447] br1: port 1(veth1) entered disabled state [ 42.341102][ T448] veth1: left allmulticast mode [ 42.341215][ T448] veth1: left promiscuous mode [ 42.341349][ T448] br1: port 1(veth1) entered disabled state