[ 10.027573][ T237] br1: port 1(veth1) entered blocking state [ 10.028140][ T237] br1: port 1(veth1) entered disabled state [ 10.028318][ T237] veth1: entered allmulticast mode [ 10.029790][ T237] veth1: entered promiscuous mode [ 10.060574][ T39] br1: port 1(veth1) entered blocking state [ 10.060897][ T39] br1: port 1(veth1) entered forwarding state [ 17.610224][ T297] veth1: left allmulticast mode [ 17.610454][ T297] veth1: left promiscuous mode [ 17.610736][ T297] br1: port 1(veth1) entered disabled state [ 19.665848][ T299] br1: port 1(veth1) entered blocking state [ 19.666060][ T299] br1: port 1(veth1) entered disabled state [ 19.666245][ T299] veth1: entered allmulticast mode [ 19.667649][ T299] veth1: entered promiscuous mode [ 19.668318][ T299] br1: port 1(veth1) entered blocking state [ 19.668493][ T299] br1: port 1(veth1) entered forwarding state [ 38.349418][ T406] veth1: left allmulticast mode [ 38.349644][ T406] veth1: left promiscuous mode [ 38.349925][ T406] br1: port 1(veth1) entered disabled state [ 40.404878][ T409] br1: port 1(veth1) entered blocking state [ 40.405075][ T409] br1: port 1(veth1) entered disabled state [ 40.405247][ T409] veth1: entered allmulticast mode [ 40.406607][ T409] veth1: entered promiscuous mode [ 40.413230][ T409] br1: port 1(veth1) entered blocking state [ 40.413400][ T409] br1: port 1(veth1) entered forwarding state [ 42.570400][ T443] [ 42.570488][ T443] ======================================================== [ 42.570621][ T443] WARNING: possible irq lock inversion dependency detected [ 42.570738][ T443] 7.1.0-rc5-virtme #1 Not tainted [ 42.570819][ T443] -------------------------------------------------------- [ 42.570932][ T443] sysctl/443 just changed the state of lock: [ 42.571032][ T443] ffffffff9257e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.571185][ T443] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 42.571301][ T443] (&ndev->lock){++.-}-{3:3} [ 42.571304][ T443] [ 42.571304][ T443] [ 42.571304][ T443] and interrupts could create inverse lock ordering between them. [ 42.571304][ T443] [ 42.571599][ T443] [ 42.571599][ T443] other info that might help us debug this: [ 42.571714][ T443] Possible interrupt unsafe locking scenario: [ 42.571714][ T443] [ 42.571836][ T443] CPU0 CPU1 [ 42.571914][ T443] ---- ---- [ 42.571995][ T443] lock(acaddr_hash_lock); [ 42.572074][ T443] local_irq_disable(); [ 42.572172][ T443] lock(&ndev->lock); [ 42.572270][ T443] lock(acaddr_hash_lock); [ 42.572365][ T443] [ 42.572424][ T443] lock(&ndev->lock); [ 42.572486][ T443] [ 42.572486][ T443] *** DEADLOCK *** [ 42.572486][ T443] [ 42.572605][ T443] 2 locks held by sysctl/443: [ 42.572690][ T443] #0: ff11000009d243e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 42.572833][ T443] #1: ffffffff9248c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 42.572990][ T443] [ 42.572990][ T443] the shortest dependencies between 2nd lock and 1st lock: [ 42.573121][ T443] -> (&ndev->lock){++.-}-{3:3} { [ 42.573203][ T443] HARDIRQ-ON-W at: [ 42.573268][ T443] __lock_acquire+0x378/0xc10 [ 42.573368][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.573464][ T443] _raw_write_lock_bh+0x38/0x50 [ 42.573568][ T443] addrconf_permanent_addr+0x108/0x9c0 [ 42.573687][ T443] addrconf_notify+0x151/0xf30 [ 42.573785][ T443] notifier_call_chain+0xb0/0x320 [ 42.573882][ T443] __dev_notify_flags+0xde/0x280 [ 42.573980][ T443] netif_change_flags+0xfe/0x190 [ 42.574084][ T443] do_setlink.isra.0+0x1cc3/0x2750 [ 42.574231][ T443] rtnl_newlink+0x8d1/0xef0 [ 42.574328][ T443] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.574428][ T443] netlink_rcv_skb+0x14e/0x3a0 [ 42.574534][ T443] netlink_unicast+0x486/0x750 [ 42.574676][ T443] netlink_sendmsg+0x735/0xc60 [ 42.574779][ T443] ____sys_sendmsg+0x419/0x850 [ 42.574877][ T443] ___sys_sendmsg+0x14e/0x1d0 [ 42.575019][ T443] __sys_sendmsg+0x145/0x1f0 [ 42.575118][ T443] do_syscall_64+0x117/0x590 [ 42.575214][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.575372][ T443] HARDIRQ-ON-R at: [ 42.575431][ T443] __lock_acquire+0x378/0xc10 [ 42.575528][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.575673][ T443] _raw_read_lock_bh+0x44/0x80 [ 42.575769][ T443] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 42.575882][ T443] inet6_fill_link_af+0x5c/0xe0 [ 42.576025][ T443] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 42.576146][ T443] rtnl_getlink+0x9c9/0xeb0 [ 42.576245][ T443] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.576382][ T443] netlink_rcv_skb+0x14e/0x3a0 [ 42.576491][ T443] netlink_unicast+0x486/0x750 [ 42.576594][ T443] netlink_sendmsg+0x735/0xc60 [ 42.576691][ T443] ____sys_sendmsg+0x419/0x850 [ 42.576827][ T443] ___sys_sendmsg+0x14e/0x1d0 [ 42.576922][ T443] __sys_sendmsg+0x145/0x1f0 [ 42.577021][ T443] do_syscall_64+0x117/0x590 [ 42.577162][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.577277][ T443] IN-SOFTIRQ-R at: [ 42.577336][ T443] __lock_acquire+0x378/0xc10 [ 42.577474][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.577575][ T443] _raw_read_lock_bh+0x44/0x80 [ 42.577671][ T443] ipv6_get_lladdr+0xa8/0x3f0 [ 42.577813][ T443] ndisc_send_ns+0xf3/0x120 [ 42.577911][ T443] ndisc_solicit+0x314/0x480 [ 42.578009][ T443] neigh_probe+0xb3/0x100 [ 42.578107][ T443] __neigh_event_send+0x802/0x1070 [ 42.578249][ T443] neigh_resolve_output+0xbe/0x140 [ 42.578345][ T443] ip6_finish_output2+0x423/0x1300 [ 42.578447][ T443] ip6_finish_output+0x646/0xda0 [ 42.578589][ T443] ip6_output+0x23f/0x7f0 [ 42.578686][ T443] ip6_forward+0x10f4/0x2bd0 [ 42.578789][ T443] ipv6_rcv+0x3c1/0x5a0 [ 42.578928][ T443] __netif_receive_skb_one_core+0xfc/0x180 [ 42.579049][ T443] netif_receive_skb_internal+0xb0/0x330 [ 42.579164][ T443] netif_receive_skb+0x56/0x380 [ 42.579301][ T443] br_handle_frame_finish+0xc3b/0x1fb0 [bridge] [ 42.579431][ T443] br_handle_frame+0x739/0xf40 [bridge] [ 42.579556][ T443] __netif_receive_skb_core.constprop.0+0x536/0x2bc0 [ 42.579719][ T443] __netif_receive_skb_one_core+0xa2/0x180 [ 42.579834][ T443] process_backlog+0x2bc/0x1490 [ 42.579974][ T443] __napi_poll+0xa7/0x3b0 [ 42.580070][ T443] net_rx_action+0x513/0xf50 [ 42.580167][ T443] handle_softirqs+0x1d8/0x940 [ 42.580270][ T443] do_softirq+0xa9/0xe0 [ 42.580412][ T443] __local_bh_enable_ip+0x113/0x140 [ 42.580517][ T443] __neigh_event_send+0x30a/0x1070 [ 42.580619][ T443] neigh_resolve_output+0xbe/0x140 [ 42.580756][ T443] ip6_finish_output2+0x423/0x1300 [ 42.580857][ T443] ip6_finish_output+0x646/0xda0 [ 42.580953][ T443] ip6_output+0x23f/0x7f0 [ 42.581100][ T443] ip6_send_skb+0xee/0x350 [ 42.581199][ T443] rawv6_push_pending_frames+0x4cb/0xa90 [ 42.581316][ T443] rawv6_sendmsg+0x1434/0x1a50 [ 42.581452][ T443] __sys_sendto+0x2c9/0x400 [ 42.581550][ T443] __x64_sys_sendto+0xe4/0x1f0 [ 42.581651][ T443] do_syscall_64+0x117/0x590 [ 42.581790][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.581910][ T443] INITIAL USE at: [ 42.581969][ T443] __lock_acquire+0x378/0xc10 [ 42.582064][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.582207][ T443] _raw_write_lock_bh+0x38/0x50 [ 42.582307][ T443] addrconf_permanent_addr+0x108/0x9c0 [ 42.582422][ T443] addrconf_notify+0x151/0xf30 [ 42.582562][ T443] notifier_call_chain+0xb0/0x320 [ 42.582663][ T443] __dev_notify_flags+0xde/0x280 [ 42.582763][ T443] netif_change_flags+0xfe/0x190 [ 42.582864][ T443] do_setlink.isra.0+0x1cc3/0x2750 [ 42.582960][ T443] rtnl_newlink+0x8d1/0xef0 [ 42.583057][ T443] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.583199][ T443] netlink_rcv_skb+0x14e/0x3a0 [ 42.583300][ T443] netlink_unicast+0x486/0x750 [ 42.583396][ T443] netlink_sendmsg+0x735/0xc60 [ 42.583534][ T443] ____sys_sendmsg+0x419/0x850 [ 42.583643][ T443] ___sys_sendmsg+0x14e/0x1d0 [ 42.583742][ T443] __sys_sendmsg+0x145/0x1f0 [ 42.583880][ T443] do_syscall_64+0x117/0x590 [ 42.583976][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.584090][ T443] INITIAL READ USE at: [ 42.584167][ T443] __lock_acquire+0x378/0xc10 [ 42.584308][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.584428][ T443] _raw_read_lock_bh+0x44/0x80 [ 42.584534][ T443] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 42.584694][ T443] inet6_fill_link_af+0x5c/0xe0 [ 42.584793][ T443] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 42.584951][ T443] rtnl_getlink+0x9c9/0xeb0 [ 42.585050][ T443] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 42.585154][ T443] netlink_rcv_skb+0x14e/0x3a0 [ 42.585254][ T443] netlink_unicast+0x486/0x750 [ 42.585396][ T443] netlink_sendmsg+0x735/0xc60 [ 42.585498][ T443] ____sys_sendmsg+0x419/0x850 [ 42.585605][ T443] ___sys_sendmsg+0x14e/0x1d0 [ 42.585750][ T443] __sys_sendmsg+0x145/0x1f0 [ 42.585847][ T443] do_syscall_64+0x117/0x590 [ 42.585945][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.586107][ T443] } [ 42.586152][ T443] ... key at: [] __key.37+0x0/0x40 [ 42.586272][ T443] ... acquired at: [ 42.586331][ T443] __lock_acquire+0x508/0xc10 [ 42.586449][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.586528][ T443] _raw_spin_lock+0x33/0x40 [ 42.586613][ T443] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.586718][ T443] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.586840][ T443] addrconf_join_anycast+0x1bc/0x260 [ 42.586923][ T443] __ipv6_ifa_notify+0x75b/0xad0 [ 42.587001][ T443] addrconf_dad_completed+0x14b/0xe10 [ 42.587079][ T443] addrconf_dad_work+0x3c1/0x930 [ 42.587195][ T443] process_one_work+0xdf8/0x1410 [ 42.587277][ T443] worker_thread+0x4f1/0xd60 [ 42.587354][ T443] kthread+0x367/0x460 [ 42.587413][ T443] ret_from_fork+0x474/0x6b0 [ 42.587537][ T443] ret_from_fork_asm+0x11/0x20 [ 42.587620][ T443] [ 42.587659][ T443] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 42.587738][ T443] HARDIRQ-ON-W at: [ 42.587797][ T443] __lock_acquire+0x378/0xc10 [ 42.587937][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.588033][ T443] _raw_spin_lock+0x33/0x40 [ 42.588131][ T443] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.588286][ T443] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.588383][ T443] addrconf_join_anycast+0x1bc/0x260 [ 42.588485][ T443] __ipv6_ifa_notify+0x75b/0xad0 [ 42.588629][ T443] addrconf_dad_completed+0x14b/0xe10 [ 42.588736][ T443] addrconf_dad_work+0x3c1/0x930 [ 42.588837][ T443] process_one_work+0xdf8/0x1410 [ 42.588973][ T443] worker_thread+0x4f1/0xd60 [ 42.589069][ T443] kthread+0x367/0x460 [ 42.589146][ T443] ret_from_fork+0x474/0x6b0 [ 42.589292][ T443] ret_from_fork_asm+0x11/0x20 [ 42.589392][ T443] SOFTIRQ-ON-W at: [ 42.589452][ T443] __lock_acquire+0x378/0xc10 [ 42.589548][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.589690][ T443] _raw_spin_lock+0x33/0x40 [ 42.589789][ T443] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.589885][ T443] addrconf_leave_anycast+0x1bc/0x260 [ 42.590025][ T443] dev_forward_change+0x3b3/0x850 [ 42.590122][ T443] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.590238][ T443] addrconf_sysctl_forward+0x209/0x2b0 [ 42.590375][ T443] proc_sys_call_handler+0x31b/0x480 [ 42.590474][ T443] new_sync_write+0x333/0x750 [ 42.590581][ T443] vfs_write+0x6a4/0xc10 [ 42.590719][ T443] ksys_write+0x116/0x250 [ 42.590815][ T443] do_syscall_64+0x117/0x590 [ 42.590911][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.591072][ T443] INITIAL USE at: [ 42.591129][ T443] __lock_acquire+0x378/0xc10 [ 42.591234][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.591330][ T443] _raw_spin_lock+0x33/0x40 [ 42.591474][ T443] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 42.591593][ T443] __ipv6_dev_ac_inc+0x57e/0x930 [ 42.591688][ T443] addrconf_join_anycast+0x1bc/0x260 [ 42.591828][ T443] __ipv6_ifa_notify+0x75b/0xad0 [ 42.591926][ T443] addrconf_dad_completed+0x14b/0xe10 [ 42.592024][ T443] addrconf_dad_work+0x3c1/0x930 [ 42.592160][ T443] process_one_work+0xdf8/0x1410 [ 42.592258][ T443] worker_thread+0x4f1/0xd60 [ 42.592354][ T443] kthread+0x367/0x460 [ 42.592434][ T443] ret_from_fork+0x474/0x6b0 [ 42.592576][ T443] ret_from_fork_asm+0x11/0x20 [ 42.592672][ T443] } [ 42.592712][ T443] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 42.592866][ T443] ... acquired at: [ 42.592923][ T443] mark_lock+0x1d7/0xa00 [ 42.592998][ T443] mark_usage+0x105/0x170 [ 42.593075][ T443] __lock_acquire+0x378/0xc10 [ 42.593192][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.593268][ T443] _raw_spin_lock+0x33/0x40 [ 42.593345][ T443] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.593421][ T443] addrconf_leave_anycast+0x1bc/0x260 [ 42.593537][ T443] dev_forward_change+0x3b3/0x850 [ 42.593619][ T443] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.593718][ T443] addrconf_sysctl_forward+0x209/0x2b0 [ 42.593795][ T443] proc_sys_call_handler+0x31b/0x480 [ 42.593910][ T443] new_sync_write+0x333/0x750 [ 42.593989][ T443] vfs_write+0x6a4/0xc10 [ 42.594066][ T443] ksys_write+0x116/0x250 [ 42.594143][ T443] do_syscall_64+0x117/0x590 [ 42.594261][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.594361][ T443] [ 42.594399][ T443] [ 42.594399][ T443] stack backtrace: [ 42.594496][ T443] CPU: 3 UID: 0 PID: 443 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 42.594498][ T443] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 42.594500][ T443] Call Trace: [ 42.594502][ T443] [ 42.594503][ T443] dump_stack_lvl+0x6f/0xa0 [ 42.594507][ T443] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 42.594510][ T443] mark_lock_irq+0x989/0x9c0 [ 42.594513][ T443] mark_lock+0x1d7/0xa00 [ 42.594514][ T443] mark_usage+0x105/0x170 [ 42.594516][ T443] __lock_acquire+0x378/0xc10 [ 42.594518][ T443] lock_acquire.part.0+0xbc/0x260 [ 42.594519][ T443] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.594521][ T443] ? rcu_is_watching+0x15/0xd0 [ 42.594524][ T443] ? do_raw_read_unlock+0x70/0x70 [ 42.594525][ T443] ? lock_acquire+0x134/0x160 [ 42.594527][ T443] _raw_spin_lock+0x33/0x40 [ 42.594529][ T443] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.594530][ T443] __ipv6_dev_ac_dec+0x236/0x5d0 [ 42.594532][ T443] addrconf_leave_anycast+0x1bc/0x260 [ 42.594533][ T443] ? find_held_lock+0x2b/0x80 [ 42.594536][ T443] ? __ipv6_isatap_ifid+0x210/0x210 [ 42.594538][ T443] ? mark_held_locks+0x40/0x70 [ 42.594540][ T443] ? lockdep_hardirqs_on+0x8c/0x130 [ 42.594542][ T443] dev_forward_change+0x3b3/0x850 [ 42.594544][ T443] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 42.594546][ T443] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 42.594549][ T443] addrconf_fixup_forwarding+0x2af/0x4d0 [ 42.594551][ T443] addrconf_sysctl_forward+0x209/0x2b0 [ 42.594553][ T443] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 42.594555][ T443] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 42.594557][ T443] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 42.594560][ T443] ? proc_sys_call_handler+0x25d/0x480 [ 42.594563][ T443] proc_sys_call_handler+0x31b/0x480 [ 42.594565][ T443] ? proc_sys_lookup+0x3d0/0x3d0 [ 42.594570][ T443] ? rcu_read_unlock+0x1b/0x70 [ 42.594572][ T443] ? do_raw_spin_unlock+0x59/0x250 [ 42.594574][ T443] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 42.594576][ T443] ? rcu_read_lock_any_held+0x3c/0x90 [ 42.594577][ T443] ? proc_sys_call_handler+0x480/0x480 [ 42.594579][ T443] new_sync_write+0x333/0x750 [ 42.594581][ T443] ? __lock_acquire+0x508/0xc10 [ 42.594582][ T443] ? new_sync_read+0x740/0x740 [ 42.594584][ T443] ? lock_acquire.part.0+0xbc/0x260 [ 42.594585][ T443] ? ksys_write+0x116/0x250 [ 42.594587][ T443] vfs_write+0x6a4/0xc10 [ 42.594589][ T443] ksys_write+0x116/0x250 [ 42.594590][ T443] ? __ia32_sys_read+0xc0/0xc0 [ 42.594592][ T443] ? rcu_is_watching+0x15/0xd0 [ 42.594593][ T443] ? rcu_is_watching+0x15/0xd0 [ 42.594595][ T443] do_syscall_64+0x117/0x590 [ 42.594596][ T443] ? trace_hardirqs_off+0xd/0x30 [ 42.594599][ T443] ? exc_page_fault+0xee/0x100 [ 42.594601][ T443] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 42.594603][ T443] RIP: 0033:0x7eff62f6708e [ 42.594605][ T443] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 42.594607][ T443] RSP: 002b:00007ffc9bc52740 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 42.594610][ T443] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007eff62f6708e [ 42.594611][ T443] RDX: 0000000000000002 RSI: 000056489a28a4b0 RDI: 0000000000000005 [ 42.594612][ T443] RBP: 00007ffc9bc52750 R08: 0000000000000000 R09: 0000000000000000 [ 42.594613][ T443] R10: 0000000000000000 R11: 0000000000000202 R12: 000056489a28c5e0 [ 42.594613][ T443] R13: 000056489a28a470 R14: 0000000000000002 R15: 0000000000000000 [ 42.594616][ T443] [ 42.708626][ T450] br1: port 1(veth1) entered disabled state [ 42.731687][ T451] veth1: left allmulticast mode [ 42.731845][ T451] veth1: left promiscuous mode [ 42.732055][ T451] br1: port 1(veth1) entered disabled state