[ 9.766793][ T214] 8021q: 802.1Q VLAN Support v1.8 [ 10.760297][ T253] br1: port 1(veth1) entered blocking state [ 10.760602][ T253] br1: port 1(veth1) entered disabled state [ 10.760835][ T253] veth1: entered allmulticast mode [ 10.762290][ T253] veth1: entered promiscuous mode [ 10.787340][ T38] br1: port 1(veth1) entered blocking state [ 10.787656][ T38] br1: port 1(veth1) entered forwarding state [ 37.253932][ T426] [ 37.254016][ T426] ======================================================== [ 37.254135][ T426] WARNING: possible irq lock inversion dependency detected [ 37.254257][ T426] 7.1.0-rc5-virtme #1 Not tainted [ 37.254339][ T426] -------------------------------------------------------- [ 37.254459][ T426] sysctl/426 just changed the state of lock: [ 37.254556][ T426] ffffffff9457e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.254701][ T426] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 37.254814][ T426] (&ndev->lock){++.-}-{3:3} [ 37.254816][ T426] [ 37.254816][ T426] [ 37.254816][ T426] and interrupts could create inverse lock ordering between them. [ 37.254816][ T426] [ 37.255095][ T426] [ 37.255095][ T426] other info that might help us debug this: [ 37.255216][ T426] Possible interrupt unsafe locking scenario: [ 37.255216][ T426] [ 37.255330][ T426] CPU0 CPU1 [ 37.255415][ T426] ---- ---- [ 37.255495][ T426] lock(acaddr_hash_lock); [ 37.255577][ T426] local_irq_disable(); [ 37.255678][ T426] lock(&ndev->lock); [ 37.255781][ T426] lock(acaddr_hash_lock); [ 37.255876][ T426] [ 37.255935][ T426] lock(&ndev->lock); [ 37.255995][ T426] [ 37.255995][ T426] *** DEADLOCK *** [ 37.255995][ T426] [ 37.256108][ T426] 2 locks held by sysctl/426: [ 37.256186][ T426] #0: ff1100000a13c3e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 37.256327][ T426] #1: ffffffff9448c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 37.256484][ T426] [ 37.256484][ T426] the shortest dependencies between 2nd lock and 1st lock: [ 37.256621][ T426] -> (&ndev->lock){++.-}-{3:3} { [ 37.256703][ T426] HARDIRQ-ON-W at: [ 37.256771][ T426] __lock_acquire+0x378/0xc10 [ 37.256876][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.256981][ T426] _raw_write_lock_bh+0x38/0x50 [ 37.257084][ T426] addrconf_permanent_addr+0x108/0x9c0 [ 37.257215][ T426] addrconf_notify+0x151/0xf30 [ 37.257312][ T426] notifier_call_chain+0xb0/0x320 [ 37.257407][ T426] __dev_notify_flags+0xde/0x280 [ 37.257503][ T426] netif_change_flags+0xfe/0x190 [ 37.257601][ T426] do_setlink.isra.0+0x1cc3/0x2750 [ 37.257700][ T426] rtnl_newlink+0x8d1/0xef0 [ 37.257801][ T426] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 37.257896][ T426] netlink_rcv_skb+0x14e/0x3a0 [ 37.257991][ T426] netlink_unicast+0x486/0x750 [ 37.258085][ T426] netlink_sendmsg+0x735/0xc60 [ 37.258186][ T426] ____sys_sendmsg+0x419/0x850 [ 37.258296][ T426] ___sys_sendmsg+0x14e/0x1d0 [ 37.258392][ T426] __sys_sendmsg+0x145/0x1f0 [ 37.258487][ T426] do_syscall_64+0x117/0x590 [ 37.258583][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.258704][ T426] HARDIRQ-ON-R at: [ 37.258762][ T426] __lock_acquire+0x378/0xc10 [ 37.258862][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.258958][ T426] _raw_read_lock_bh+0x44/0x80 [ 37.259053][ T426] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 37.259177][ T426] inet6_fill_link_af+0x5c/0xe0 [ 37.259275][ T426] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 37.259390][ T426] rtnl_getlink+0x9c9/0xeb0 [ 37.259486][ T426] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 37.259582][ T426] netlink_rcv_skb+0x14e/0x3a0 [ 37.259676][ T426] netlink_unicast+0x486/0x750 [ 37.259771][ T426] netlink_sendmsg+0x735/0xc60 [ 37.259873][ T426] ____sys_sendmsg+0x419/0x850 [ 37.259968][ T426] ___sys_sendmsg+0x14e/0x1d0 [ 37.260063][ T426] __sys_sendmsg+0x145/0x1f0 [ 37.260159][ T426] do_syscall_64+0x117/0x590 [ 37.260260][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.260374][ T426] IN-SOFTIRQ-R at: [ 37.260432][ T426] __lock_acquire+0x378/0xc10 [ 37.260528][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.260622][ T426] _raw_read_lock_bh+0x44/0x80 [ 37.260723][ T426] ipv6_get_lladdr+0xa8/0x3f0 [ 37.260819][ T426] ndisc_send_ns+0xf3/0x120 [ 37.260916][ T426] ndisc_solicit+0x314/0x480 [ 37.261016][ T426] neigh_probe+0xb3/0x100 [ 37.261114][ T426] __neigh_event_send+0x802/0x1070 [ 37.261213][ T426] neigh_resolve_output+0xbe/0x140 [ 37.261311][ T426] ip6_finish_output2+0x423/0x1300 [ 37.261405][ T426] ip6_finish_output+0x646/0xda0 [ 37.261500][ T426] ip6_output+0x23f/0x7f0 [ 37.261595][ T426] ip6_forward+0x10f4/0x2bd0 [ 37.261689][ T426] ipv6_rcv+0x3c1/0x5a0 [ 37.261789][ T426] __netif_receive_skb_one_core+0xfc/0x180 [ 37.261904][ T426] netif_receive_skb_internal+0xb0/0x330 [ 37.262016][ T426] netif_receive_skb+0x56/0x380 [ 37.262120][ T426] br_handle_frame_finish+0xc3b/0x1fb0 [bridge] [ 37.262258][ T426] br_handle_frame+0x739/0xf40 [bridge] [ 37.262377][ T426] __netif_receive_skb_core.constprop.0+0x536/0x2bc0 [ 37.262496][ T426] __netif_receive_skb_one_core+0xa2/0x180 [ 37.262609][ T426] process_backlog+0x2bc/0x1490 [ 37.262702][ T426] __napi_poll+0xa7/0x3b0 [ 37.262797][ T426] net_rx_action+0x513/0xf50 [ 37.262890][ T426] handle_softirqs+0x1d8/0x940 [ 37.262987][ T426] do_softirq+0xa9/0xe0 [ 37.263084][ T426] __local_bh_enable_ip+0x113/0x140 [ 37.263182][ T426] __neigh_event_send+0x30a/0x1070 [ 37.263280][ T426] neigh_resolve_output+0xbe/0x140 [ 37.263377][ T426] ip6_finish_output2+0x423/0x1300 [ 37.263481][ T426] ip6_finish_output+0x646/0xda0 [ 37.263580][ T426] ip6_output+0x23f/0x7f0 [ 37.263688][ T426] ip6_send_skb+0xee/0x350 [ 37.263785][ T426] rawv6_push_pending_frames+0x4cb/0xa90 [ 37.263901][ T426] rawv6_sendmsg+0x1434/0x1a50 [ 37.263998][ T426] __sys_sendto+0x2c9/0x400 [ 37.264096][ T426] __x64_sys_sendto+0xe4/0x1f0 [ 37.264201][ T426] do_syscall_64+0x117/0x590 [ 37.264300][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.264425][ T426] INITIAL USE at: [ 37.264487][ T426] __lock_acquire+0x378/0xc10 [ 37.264582][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.264690][ T426] _raw_write_lock_bh+0x38/0x50 [ 37.264786][ T426] addrconf_permanent_addr+0x108/0x9c0 [ 37.264901][ T426] addrconf_notify+0x151/0xf30 [ 37.264996][ T426] notifier_call_chain+0xb0/0x320 [ 37.265092][ T426] __dev_notify_flags+0xde/0x280 [ 37.265188][ T426] netif_change_flags+0xfe/0x190 [ 37.265286][ T426] do_setlink.isra.0+0x1cc3/0x2750 [ 37.265381][ T426] rtnl_newlink+0x8d1/0xef0 [ 37.265478][ T426] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 37.265573][ T426] netlink_rcv_skb+0x14e/0x3a0 [ 37.265667][ T426] netlink_unicast+0x486/0x750 [ 37.265761][ T426] netlink_sendmsg+0x735/0xc60 [ 37.265856][ T426] ____sys_sendmsg+0x419/0x850 [ 37.265951][ T426] ___sys_sendmsg+0x14e/0x1d0 [ 37.266046][ T426] __sys_sendmsg+0x145/0x1f0 [ 37.266145][ T426] do_syscall_64+0x117/0x590 [ 37.266243][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.266358][ T426] INITIAL READ USE at: [ 37.266434][ T426] __lock_acquire+0x378/0xc10 [ 37.266530][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.266643][ T426] _raw_read_lock_bh+0x44/0x80 [ 37.266741][ T426] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 37.266861][ T426] inet6_fill_link_af+0x5c/0xe0 [ 37.266957][ T426] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 37.267072][ T426] rtnl_getlink+0x9c9/0xeb0 [ 37.267173][ T426] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 37.267275][ T426] netlink_rcv_skb+0x14e/0x3a0 [ 37.267370][ T426] netlink_unicast+0x486/0x750 [ 37.267466][ T426] netlink_sendmsg+0x735/0xc60 [ 37.267559][ T426] ____sys_sendmsg+0x419/0x850 [ 37.267654][ T426] ___sys_sendmsg+0x14e/0x1d0 [ 37.267750][ T426] __sys_sendmsg+0x145/0x1f0 [ 37.267844][ T426] do_syscall_64+0x117/0x590 [ 37.267942][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.268054][ T426] } [ 37.268094][ T426] ... key at: [] __key.37+0x0/0x40 [ 37.268216][ T426] ... acquired at: [ 37.268275][ T426] __lock_acquire+0x508/0xc10 [ 37.268353][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.268430][ T426] _raw_spin_lock+0x33/0x40 [ 37.268507][ T426] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 37.268603][ T426] __ipv6_dev_ac_inc+0x57e/0x930 [ 37.268679][ T426] addrconf_join_anycast+0x1bc/0x260 [ 37.268761][ T426] __ipv6_ifa_notify+0x75b/0xad0 [ 37.268837][ T426] addrconf_dad_completed+0x14b/0xe10 [ 37.268912][ T426] addrconf_dad_work+0x3c1/0x930 [ 37.268989][ T426] process_one_work+0xdf8/0x1410 [ 37.269067][ T426] worker_thread+0x4f1/0xd60 [ 37.269148][ T426] kthread+0x367/0x460 [ 37.269205][ T426] ret_from_fork+0x474/0x6b0 [ 37.269290][ T426] ret_from_fork_asm+0x11/0x20 [ 37.269368][ T426] [ 37.269408][ T426] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 37.269486][ T426] HARDIRQ-ON-W at: [ 37.269544][ T426] __lock_acquire+0x378/0xc10 [ 37.269639][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.269733][ T426] _raw_spin_lock+0x33/0x40 [ 37.269830][ T426] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 37.269946][ T426] __ipv6_dev_ac_inc+0x57e/0x930 [ 37.270041][ T426] addrconf_join_anycast+0x1bc/0x260 [ 37.270145][ T426] __ipv6_ifa_notify+0x75b/0xad0 [ 37.270244][ T426] addrconf_dad_completed+0x14b/0xe10 [ 37.270343][ T426] addrconf_dad_work+0x3c1/0x930 [ 37.270443][ T426] process_one_work+0xdf8/0x1410 [ 37.270539][ T426] worker_thread+0x4f1/0xd60 [ 37.270643][ T426] kthread+0x367/0x460 [ 37.270719][ T426] ret_from_fork+0x474/0x6b0 [ 37.270814][ T426] ret_from_fork_asm+0x11/0x20 [ 37.270915][ T426] SOFTIRQ-ON-W at: [ 37.270972][ T426] __lock_acquire+0x378/0xc10 [ 37.271067][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.271162][ T426] _raw_spin_lock+0x33/0x40 [ 37.271260][ T426] __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.271356][ T426] addrconf_leave_anycast+0x1bc/0x260 [ 37.271452][ T426] dev_forward_change+0x3b3/0x850 [ 37.271548][ T426] addrconf_fixup_forwarding+0x2af/0x4d0 [ 37.271662][ T426] addrconf_sysctl_forward+0x209/0x2b0 [ 37.271758][ T426] proc_sys_call_handler+0x31b/0x480 [ 37.271855][ T426] new_sync_write+0x333/0x750 [ 37.271953][ T426] vfs_write+0x6a4/0xc10 [ 37.272048][ T426] ksys_write+0x116/0x250 [ 37.272143][ T426] do_syscall_64+0x117/0x590 [ 37.272243][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.272356][ T426] INITIAL USE at: [ 37.272413][ T426] __lock_acquire+0x378/0xc10 [ 37.272516][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.272611][ T426] _raw_spin_lock+0x33/0x40 [ 37.272712][ T426] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 37.272832][ T426] __ipv6_dev_ac_inc+0x57e/0x930 [ 37.272932][ T426] addrconf_join_anycast+0x1bc/0x260 [ 37.273032][ T426] __ipv6_ifa_notify+0x75b/0xad0 [ 37.273129][ T426] addrconf_dad_completed+0x14b/0xe10 [ 37.273230][ T426] addrconf_dad_work+0x3c1/0x930 [ 37.273327][ T426] process_one_work+0xdf8/0x1410 [ 37.273422][ T426] worker_thread+0x4f1/0xd60 [ 37.273518][ T426] kthread+0x367/0x460 [ 37.273597][ T426] ret_from_fork+0x474/0x6b0 [ 37.273698][ T426] ret_from_fork_asm+0x11/0x20 [ 37.273840][ T426] } [ 37.273879][ T426] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 37.273994][ T426] ... acquired at: [ 37.274054][ T426] mark_lock+0x1d7/0xa00 [ 37.274133][ T426] mark_usage+0x105/0x170 [ 37.274251][ T426] __lock_acquire+0x378/0xc10 [ 37.274326][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.274401][ T426] _raw_spin_lock+0x33/0x40 [ 37.274477][ T426] __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.274601][ T426] addrconf_leave_anycast+0x1bc/0x260 [ 37.274677][ T426] dev_forward_change+0x3b3/0x850 [ 37.274755][ T426] addrconf_fixup_forwarding+0x2af/0x4d0 [ 37.274888][ T426] addrconf_sysctl_forward+0x209/0x2b0 [ 37.274964][ T426] proc_sys_call_handler+0x31b/0x480 [ 37.275040][ T426] new_sync_write+0x333/0x750 [ 37.275116][ T426] vfs_write+0x6a4/0xc10 [ 37.275236][ T426] ksys_write+0x116/0x250 [ 37.275315][ T426] do_syscall_64+0x117/0x590 [ 37.275391][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.275488][ T426] [ 37.275527][ T426] [ 37.275527][ T426] stack backtrace: [ 37.275661][ T426] CPU: 0 UID: 0 PID: 426 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 37.275663][ T426] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 37.275665][ T426] Call Trace: [ 37.275667][ T426] [ 37.275668][ T426] dump_stack_lvl+0x6f/0xa0 [ 37.275672][ T426] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 37.275675][ T426] mark_lock_irq+0x989/0x9c0 [ 37.275678][ T426] mark_lock+0x1d7/0xa00 [ 37.275679][ T426] mark_usage+0x105/0x170 [ 37.275681][ T426] __lock_acquire+0x378/0xc10 [ 37.275683][ T426] lock_acquire.part.0+0xbc/0x260 [ 37.275684][ T426] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.275686][ T426] ? rcu_is_watching+0x15/0xd0 [ 37.275689][ T426] ? do_raw_read_unlock+0x70/0x70 [ 37.275690][ T426] ? lock_acquire+0x134/0x160 [ 37.275692][ T426] _raw_spin_lock+0x33/0x40 [ 37.275694][ T426] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.275695][ T426] __ipv6_dev_ac_dec+0x236/0x5d0 [ 37.275697][ T426] addrconf_leave_anycast+0x1bc/0x260 [ 37.275698][ T426] ? find_held_lock+0x2b/0x80 [ 37.275701][ T426] ? __ipv6_isatap_ifid+0x210/0x210 [ 37.275703][ T426] ? mark_held_locks+0x40/0x70 [ 37.275704][ T426] ? lockdep_hardirqs_on+0x8c/0x130 [ 37.275707][ T426] dev_forward_change+0x3b3/0x850 [ 37.275709][ T426] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 37.275711][ T426] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 37.275713][ T426] addrconf_fixup_forwarding+0x2af/0x4d0 [ 37.275716][ T426] addrconf_sysctl_forward+0x209/0x2b0 [ 37.275718][ T426] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 37.275720][ T426] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 37.275722][ T426] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 37.275725][ T426] ? proc_sys_call_handler+0x25d/0x480 [ 37.275727][ T426] proc_sys_call_handler+0x31b/0x480 [ 37.275729][ T426] ? proc_sys_lookup+0x3d0/0x3d0 [ 37.275731][ T426] ? rcu_read_unlock+0x1b/0x70 [ 37.275733][ T426] ? do_raw_spin_unlock+0x59/0x250 [ 37.275735][ T426] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 37.275737][ T426] ? rcu_read_lock_any_held+0x3c/0x90 [ 37.275738][ T426] ? proc_sys_call_handler+0x480/0x480 [ 37.275740][ T426] new_sync_write+0x333/0x750 [ 37.275742][ T426] ? __lock_acquire+0x508/0xc10 [ 37.275743][ T426] ? new_sync_read+0x740/0x740 [ 37.275745][ T426] ? lock_acquire.part.0+0xbc/0x260 [ 37.275746][ T426] ? ksys_write+0x116/0x250 [ 37.275748][ T426] vfs_write+0x6a4/0xc10 [ 37.275750][ T426] ksys_write+0x116/0x250 [ 37.275751][ T426] ? __ia32_sys_read+0xc0/0xc0 [ 37.275753][ T426] ? rcu_is_watching+0x15/0xd0 [ 37.275754][ T426] ? rcu_is_watching+0x15/0xd0 [ 37.275756][ T426] do_syscall_64+0x117/0x590 [ 37.275757][ T426] ? trace_hardirqs_off+0xd/0x30 [ 37.275760][ T426] ? exc_page_fault+0xee/0x100 [ 37.275762][ T426] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 37.275763][ T426] RIP: 0033:0x7f3ffc92608e [ 37.275766][ T426] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 37.275767][ T426] RSP: 002b:00007ffd08e295e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 37.275770][ T426] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f3ffc92608e [ 37.275771][ T426] RDX: 0000000000000002 RSI: 000055bdfb4ba4b0 RDI: 0000000000000005 [ 37.275772][ T426] RBP: 00007ffd08e295f0 R08: 0000000000000000 R09: 0000000000000000 [ 37.275773][ T426] R10: 0000000000000000 R11: 0000000000000202 R12: 000055bdfb4bc5e0 [ 37.275774][ T426] R13: 000055bdfb4ba470 R14: 0000000000000002 R15: 0000000000000000 [ 37.275776][ T426] [ 37.433530][ T438] br1: port 1(veth1) entered disabled state [ 37.448078][ T439] veth1: left allmulticast mode [ 37.448202][ T439] veth1: left promiscuous mode [ 37.448363][ T439] br1: port 1(veth1) entered disabled state