[ 9.211917][ T208] veth0: entered allmulticast mode [ 9.212896][ T208] veth1: entered allmulticast mode [ 9.213748][ T208] veth2: entered allmulticast mode [ 9.214459][ T208] veth3: entered allmulticast mode [ 17.706252][ T299] GACT probability NOT on [ 19.506710][ T432] [ 19.506800][ T432] ======================================================== [ 19.506924][ T432] WARNING: possible irq lock inversion dependency detected [ 19.507036][ T432] 7.1.0-rc5-virtme #1 Not tainted [ 19.507119][ T432] -------------------------------------------------------- [ 19.507234][ T432] sysctl/432 just changed the state of lock: [ 19.507342][ T432] ffffffffa8b7e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.507491][ T432] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 19.507611][ T432] (&ndev->lock){++.-}-{3:3} [ 19.507613][ T432] [ 19.507613][ T432] [ 19.507613][ T432] and interrupts could create inverse lock ordering between them. [ 19.507613][ T432] [ 19.507902][ T432] [ 19.507902][ T432] other info that might help us debug this: [ 19.508019][ T432] Possible interrupt unsafe locking scenario: [ 19.508019][ T432] [ 19.508144][ T432] CPU0 CPU1 [ 19.508226][ T432] ---- ---- [ 19.508315][ T432] lock(acaddr_hash_lock); [ 19.508394][ T432] local_irq_disable(); [ 19.508502][ T432] lock(&ndev->lock); [ 19.508603][ T432] lock(acaddr_hash_lock); [ 19.508702][ T432] [ 19.508764][ T432] lock(&ndev->lock); [ 19.508827][ T432] [ 19.508827][ T432] *** DEADLOCK *** [ 19.508827][ T432] [ 19.508944][ T432] 2 locks held by sysctl/432: [ 19.509023][ T432] #0: ff1100000a0d43e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 19.509171][ T432] #1: ffffffffa8a8c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 19.509332][ T432] [ 19.509332][ T432] the shortest dependencies between 2nd lock and 1st lock: [ 19.509467][ T432] -> (&ndev->lock){++.-}-{3:3} { [ 19.509553][ T432] HARDIRQ-ON-W at: [ 19.509615][ T432] __lock_acquire+0x378/0xc10 [ 19.509719][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.509822][ T432] _raw_write_lock_bh+0x38/0x50 [ 19.509928][ T432] addrconf_permanent_addr+0x108/0x9c0 [ 19.510049][ T432] addrconf_notify+0x151/0xf30 [ 19.510154][ T432] notifier_call_chain+0xb0/0x320 [ 19.510259][ T432] __dev_notify_flags+0xde/0x280 [ 19.510362][ T432] netif_change_flags+0xfe/0x190 [ 19.510462][ T432] do_setlink.isra.0+0x1cc3/0x2750 [ 19.510565][ T432] rtnl_newlink+0x8d1/0xef0 [ 19.510665][ T432] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.510767][ T432] netlink_rcv_skb+0x14e/0x3a0 [ 19.510874][ T432] netlink_unicast+0x486/0x750 [ 19.510979][ T432] netlink_sendmsg+0x735/0xc60 [ 19.511079][ T432] ____sys_sendmsg+0x419/0x850 [ 19.511185][ T432] ___sys_sendmsg+0x14e/0x1d0 [ 19.511292][ T432] __sys_sendmsg+0x145/0x1f0 [ 19.511395][ T432] do_syscall_64+0x117/0x590 [ 19.511498][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.511615][ T432] HARDIRQ-ON-R at: [ 19.511673][ T432] __lock_acquire+0x378/0xc10 [ 19.511772][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.511875][ T432] _raw_read_lock_bh+0x44/0x80 [ 19.511977][ T432] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 19.512098][ T432] inet6_fill_link_af+0x5c/0xe0 [ 19.512200][ T432] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 19.512330][ T432] rtnl_getlink+0x9c9/0xeb0 [ 19.512433][ T432] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.512535][ T432] netlink_rcv_skb+0x14e/0x3a0 [ 19.512635][ T432] netlink_unicast+0x486/0x750 [ 19.512738][ T432] netlink_sendmsg+0x735/0xc60 [ 19.512839][ T432] ____sys_sendmsg+0x419/0x850 [ 19.512937][ T432] ___sys_sendmsg+0x14e/0x1d0 [ 19.513036][ T432] __sys_sendmsg+0x145/0x1f0 [ 19.513137][ T432] do_syscall_64+0x117/0x590 [ 19.513240][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.513412][ T432] IN-SOFTIRQ-R at: [ 19.513478][ T432] __lock_acquire+0x378/0xc10 [ 19.513581][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.513677][ T432] _raw_read_lock_bh+0x44/0x80 [ 19.513781][ T432] ipv6_get_lladdr+0xa8/0x3f0 [ 19.513885][ T432] ndisc_send_ns+0xf3/0x120 [ 19.513983][ T432] ndisc_solicit+0x314/0x480 [ 19.514081][ T432] neigh_probe+0xb3/0x100 [ 19.514181][ T432] __neigh_event_send+0x802/0x1070 [ 19.514291][ T432] neigh_resolve_output+0xbe/0x140 [ 19.514428][ T432] ip6_finish_output2+0x423/0x1300 [ 19.514533][ T432] ip6_finish_output+0x646/0xda0 [ 19.514630][ T432] ip6_output+0x23f/0x7f0 [ 19.514735][ T432] ip6_forward+0x10f4/0x2bd0 [ 19.514835][ T432] ipv6_rcv+0x3c1/0x5a0 [ 19.514939][ T432] __netif_receive_skb_one_core+0xfc/0x180 [ 19.515062][ T432] process_backlog+0x2bc/0x1490 [ 19.515205][ T432] __napi_poll+0xa7/0x3b0 [ 19.515307][ T432] net_rx_action+0x513/0xf50 [ 19.515404][ T432] handle_softirqs+0x1d8/0x940 [ 19.515547][ T432] do_softirq+0xa9/0xe0 [ 19.515643][ T432] __local_bh_enable_ip+0x113/0x140 [ 19.515739][ T432] __neigh_event_send+0x30a/0x1070 [ 19.515882][ T432] neigh_resolve_output+0xbe/0x140 [ 19.515978][ T432] ip6_finish_output2+0x423/0x1300 [ 19.516078][ T432] ip6_finish_output+0x646/0xda0 [ 19.516215][ T432] ip6_output+0x23f/0x7f0 [ 19.516315][ T432] ip6_send_skb+0xee/0x350 [ 19.516412][ T432] rawv6_push_pending_frames+0x4cb/0xa90 [ 19.516574][ T432] rawv6_sendmsg+0x1434/0x1a50 [ 19.516676][ T432] __sys_sendto+0x2c9/0x400 [ 19.516781][ T432] __x64_sys_sendto+0xe4/0x1f0 [ 19.516921][ T432] do_syscall_64+0x117/0x590 [ 19.517019][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.517140][ T432] INITIAL USE at: [ 19.517200][ T432] __lock_acquire+0x378/0xc10 [ 19.517352][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.517460][ T432] _raw_write_lock_bh+0x38/0x50 [ 19.517559][ T432] addrconf_permanent_addr+0x108/0x9c0 [ 19.517682][ T432] addrconf_notify+0x151/0xf30 [ 19.517780][ T432] notifier_call_chain+0xb0/0x320 [ 19.517876][ T432] __dev_notify_flags+0xde/0x280 [ 19.518017][ T432] netif_change_flags+0xfe/0x190 [ 19.518116][ T432] do_setlink.isra.0+0x1cc3/0x2750 [ 19.518218][ T432] rtnl_newlink+0x8d1/0xef0 [ 19.518362][ T432] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.518460][ T432] netlink_rcv_skb+0x14e/0x3a0 [ 19.518563][ T432] netlink_unicast+0x486/0x750 [ 19.518701][ T432] netlink_sendmsg+0x735/0xc60 [ 19.518803][ T432] ____sys_sendmsg+0x419/0x850 [ 19.518909][ T432] ___sys_sendmsg+0x14e/0x1d0 [ 19.519007][ T432] __sys_sendmsg+0x145/0x1f0 [ 19.519151][ T432] do_syscall_64+0x117/0x590 [ 19.519248][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.519375][ T432] INITIAL READ USE at: [ 19.519497][ T432] __lock_acquire+0x378/0xc10 [ 19.519599][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.519719][ T432] _raw_read_lock_bh+0x44/0x80 [ 19.519870][ T432] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 19.519990][ T432] inet6_fill_link_af+0x5c/0xe0 [ 19.520087][ T432] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 19.520250][ T432] rtnl_getlink+0x9c9/0xeb0 [ 19.520349][ T432] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.520446][ T432] netlink_rcv_skb+0x14e/0x3a0 [ 19.520583][ T432] netlink_unicast+0x486/0x750 [ 19.520680][ T432] netlink_sendmsg+0x735/0xc60 [ 19.520785][ T432] ____sys_sendmsg+0x419/0x850 [ 19.520925][ T432] ___sys_sendmsg+0x14e/0x1d0 [ 19.521028][ T432] __sys_sendmsg+0x145/0x1f0 [ 19.521126][ T432] do_syscall_64+0x117/0x590 [ 19.521266][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.521383][ T432] } [ 19.521424][ T432] ... key at: [] __key.37+0x0/0x40 [ 19.521551][ T432] ... acquired at: [ 19.521652][ T432] __lock_acquire+0x508/0xc10 [ 19.521734][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.521822][ T432] _raw_spin_lock+0x33/0x40 [ 19.521901][ T432] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.522045][ T432] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.522122][ T432] addrconf_join_anycast+0x1bc/0x260 [ 19.522206][ T432] __ipv6_ifa_notify+0x75b/0xad0 [ 19.522286][ T432] addrconf_dad_completed+0x14b/0xe10 [ 19.522405][ T432] addrconf_dad_work+0x3c1/0x930 [ 19.522483][ T432] process_one_work+0xdf8/0x1410 [ 19.522563][ T432] worker_thread+0x4f1/0xd60 [ 19.522642][ T432] kthread+0x367/0x460 [ 19.522746][ T432] ret_from_fork+0x474/0x6b0 [ 19.522831][ T432] ret_from_fork_asm+0x11/0x20 [ 19.522916][ T432] [ 19.522956][ T432] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 19.523082][ T432] HARDIRQ-ON-W at: [ 19.523146][ T432] __lock_acquire+0x378/0xc10 [ 19.523254][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.523353][ T432] _raw_spin_lock+0x33/0x40 [ 19.523495][ T432] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.523613][ T432] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.523714][ T432] addrconf_join_anycast+0x1bc/0x260 [ 19.523858][ T432] __ipv6_ifa_notify+0x75b/0xad0 [ 19.523956][ T432] addrconf_dad_completed+0x14b/0xe10 [ 19.524064][ T432] addrconf_dad_work+0x3c1/0x930 [ 19.524207][ T432] process_one_work+0xdf8/0x1410 [ 19.524307][ T432] worker_thread+0x4f1/0xd60 [ 19.524407][ T432] kthread+0x367/0x460 [ 19.524486][ T432] ret_from_fork+0x474/0x6b0 [ 19.524631][ T432] ret_from_fork_asm+0x11/0x20 [ 19.524729][ T432] SOFTIRQ-ON-W at: [ 19.524788][ T432] __lock_acquire+0x378/0xc10 [ 19.524926][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.525023][ T432] _raw_spin_lock+0x33/0x40 [ 19.525121][ T432] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.525259][ T432] addrconf_leave_anycast+0x1bc/0x260 [ 19.525372][ T432] dev_forward_change+0x3b3/0x850 [ 19.525473][ T432] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.525632][ T432] addrconf_sysctl_forward+0x209/0x2b0 [ 19.525730][ T432] proc_sys_call_handler+0x31b/0x480 [ 19.525828][ T432] new_sync_write+0x333/0x750 [ 19.525931][ T432] vfs_write+0x6a4/0xc10 [ 19.526071][ T432] ksys_write+0x116/0x250 [ 19.526168][ T432] do_syscall_64+0x117/0x590 [ 19.526267][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.526431][ T432] INITIAL USE at: [ 19.526493][ T432] __lock_acquire+0x378/0xc10 [ 19.526595][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.526737][ T432] _raw_spin_lock+0x33/0x40 [ 19.526835][ T432] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.526953][ T432] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.527092][ T432] addrconf_join_anycast+0x1bc/0x260 [ 19.527190][ T432] __ipv6_ifa_notify+0x75b/0xad0 [ 19.527292][ T432] addrconf_dad_completed+0x14b/0xe10 [ 19.527435][ T432] addrconf_dad_work+0x3c1/0x930 [ 19.527537][ T432] process_one_work+0xdf8/0x1410 [ 19.527641][ T432] worker_thread+0x4f1/0xd60 [ 19.527751][ T432] kthread+0x367/0x460 [ 19.527879][ T432] ret_from_fork+0x474/0x6b0 [ 19.527977][ T432] ret_from_fork_asm+0x11/0x20 [ 19.528073][ T432] } [ 19.528114][ T432] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 19.528277][ T432] ... acquired at: [ 19.528335][ T432] mark_lock+0x1d7/0xa00 [ 19.528419][ T432] mark_usage+0x105/0x170 [ 19.528546][ T432] __lock_acquire+0x378/0xc10 [ 19.528627][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.528704][ T432] _raw_spin_lock+0x33/0x40 [ 19.528781][ T432] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.528897][ T432] addrconf_leave_anycast+0x1bc/0x260 [ 19.528980][ T432] dev_forward_change+0x3b3/0x850 [ 19.529065][ T432] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.529169][ T432] addrconf_sysctl_forward+0x209/0x2b0 [ 19.529293][ T432] proc_sys_call_handler+0x31b/0x480 [ 19.529372][ T432] new_sync_write+0x333/0x750 [ 19.529450][ T432] vfs_write+0x6a4/0xc10 [ 19.529527][ T432] ksys_write+0x116/0x250 [ 19.529645][ T432] do_syscall_64+0x117/0x590 [ 19.529722][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.529826][ T432] [ 19.529865][ T432] [ 19.529865][ T432] stack backtrace: [ 19.530004][ T432] CPU: 0 UID: 0 PID: 432 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 19.530007][ T432] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 19.530010][ T432] Call Trace: [ 19.530011][ T432] [ 19.530012][ T432] dump_stack_lvl+0x6f/0xa0 [ 19.530017][ T432] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 19.530020][ T432] mark_lock_irq+0x989/0x9c0 [ 19.530023][ T432] mark_lock+0x1d7/0xa00 [ 19.530024][ T432] mark_usage+0x105/0x170 [ 19.530026][ T432] __lock_acquire+0x378/0xc10 [ 19.530028][ T432] lock_acquire.part.0+0xbc/0x260 [ 19.530029][ T432] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.530031][ T432] ? rcu_is_watching+0x15/0xd0 [ 19.530034][ T432] ? do_raw_read_unlock+0x70/0x70 [ 19.530036][ T432] ? lock_acquire+0x134/0x160 [ 19.530038][ T432] _raw_spin_lock+0x33/0x40 [ 19.530039][ T432] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.530040][ T432] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.530042][ T432] addrconf_leave_anycast+0x1bc/0x260 [ 19.530044][ T432] ? find_held_lock+0x2b/0x80 [ 19.530047][ T432] ? __ipv6_isatap_ifid+0x210/0x210 [ 19.530049][ T432] ? mark_held_locks+0x40/0x70 [ 19.530050][ T432] ? lockdep_hardirqs_on+0x8c/0x130 [ 19.530053][ T432] dev_forward_change+0x3b3/0x850 [ 19.530055][ T432] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 19.530057][ T432] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 19.530059][ T432] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.530062][ T432] addrconf_sysctl_forward+0x209/0x2b0 [ 19.530064][ T432] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 19.530066][ T432] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 19.530068][ T432] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 19.530071][ T432] ? proc_sys_call_handler+0x25d/0x480 [ 19.530073][ T432] proc_sys_call_handler+0x31b/0x480 [ 19.530075][ T432] ? proc_sys_lookup+0x3d0/0x3d0 [ 19.530077][ T432] ? rcu_read_unlock+0x1b/0x70 [ 19.530080][ T432] ? do_raw_spin_unlock+0x59/0x250 [ 19.530081][ T432] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 19.530083][ T432] ? rcu_read_lock_any_held+0x3c/0x90 [ 19.530085][ T432] ? proc_sys_call_handler+0x480/0x480 [ 19.530087][ T432] new_sync_write+0x333/0x750 [ 19.530088][ T432] ? __lock_acquire+0x508/0xc10 [ 19.530090][ T432] ? new_sync_read+0x740/0x740 [ 19.530091][ T432] ? lock_acquire.part.0+0xbc/0x260 [ 19.530093][ T432] ? ksys_write+0x116/0x250 [ 19.530095][ T432] vfs_write+0x6a4/0xc10 [ 19.530097][ T432] ksys_write+0x116/0x250 [ 19.530098][ T432] ? __ia32_sys_read+0xc0/0xc0 [ 19.530100][ T432] ? rcu_is_watching+0x15/0xd0 [ 19.530101][ T432] ? rcu_is_watching+0x15/0xd0 [ 19.530103][ T432] do_syscall_64+0x117/0x590 [ 19.530104][ T432] ? trace_hardirqs_off+0xd/0x30 [ 19.530107][ T432] ? exc_page_fault+0xee/0x100 [ 19.530109][ T432] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.530111][ T432] RIP: 0033:0x7f3ed3d3f08e [ 19.530114][ T432] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 19.530116][ T432] RSP: 002b:00007fff7ed955d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 19.530119][ T432] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f3ed3d3f08e [ 19.530120][ T432] RDX: 0000000000000002 RSI: 000055ed404244b0 RDI: 0000000000000005 [ 19.530121][ T432] RBP: 00007fff7ed955e0 R08: 0000000000000000 R09: 0000000000000000 [ 19.530122][ T432] R10: 0000000000000000 R11: 0000000000000202 R12: 000055ed404265e0 [ 19.530122][ T432] R13: 000055ed40424470 R14: 0000000000000002 R15: 0000000000000000 [ 19.530125][ T432] [ 20.069843][ T208] veth0: left allmulticast mode [ 20.070001][ T208] veth1: left allmulticast mode [ 20.070100][ T208] veth2: left allmulticast mode [ 20.070213][ T208] veth3: left allmulticast mode