[ 9.220696][ T192] ip (192) used greatest stack depth: 24128 bytes left [ 9.540271][ T219] veth0: entered allmulticast mode [ 9.540676][ T219] veth1: entered allmulticast mode [ 9.540939][ T219] veth2: entered allmulticast mode [ 9.541150][ T219] veth3: entered allmulticast mode [ 17.973946][ T312] GACT probability NOT on [ 19.735321][ T445] [ 19.735409][ T445] ======================================================== [ 19.735537][ T445] WARNING: possible irq lock inversion dependency detected [ 19.735660][ T445] 7.1.0-rc5-virtme #1 Not tainted [ 19.735741][ T445] -------------------------------------------------------- [ 19.735857][ T445] sysctl/445 just changed the state of lock: [ 19.735967][ T445] ffffffff8ef7e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.736117][ T445] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 19.736234][ T445] (&ndev->lock){++.-}-{3:3} [ 19.736236][ T445] [ 19.736236][ T445] [ 19.736236][ T445] and interrupts could create inverse lock ordering between them. [ 19.736236][ T445] [ 19.736528][ T445] [ 19.736528][ T445] other info that might help us debug this: [ 19.736649][ T445] Possible interrupt unsafe locking scenario: [ 19.736649][ T445] [ 19.736780][ T445] CPU0 CPU1 [ 19.736866][ T445] ---- ---- [ 19.736950][ T445] lock(acaddr_hash_lock); [ 19.737030][ T445] local_irq_disable(); [ 19.737131][ T445] lock(&ndev->lock); [ 19.737231][ T445] lock(acaddr_hash_lock); [ 19.737341][ T445] [ 19.737402][ T445] lock(&ndev->lock); [ 19.737464][ T445] [ 19.737464][ T445] *** DEADLOCK *** [ 19.737464][ T445] [ 19.737583][ T445] 2 locks held by sysctl/445: [ 19.737666][ T445] #0: ff1100000a0243e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 19.737813][ T445] #1: ffffffff8ee8c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 19.737979][ T445] [ 19.737979][ T445] the shortest dependencies between 2nd lock and 1st lock: [ 19.738127][ T445] -> (&ndev->lock){++.-}-{3:3} { [ 19.738215][ T445] HARDIRQ-ON-W at: [ 19.738283][ T445] __lock_acquire+0x378/0xc10 [ 19.738394][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.738496][ T445] _raw_write_lock_bh+0x38/0x50 [ 19.738601][ T445] addrconf_permanent_addr+0x108/0x9c0 [ 19.738727][ T445] addrconf_notify+0x151/0xf30 [ 19.738832][ T445] notifier_call_chain+0xb0/0x320 [ 19.738941][ T445] __dev_notify_flags+0xde/0x280 [ 19.739043][ T445] netif_change_flags+0xfe/0x190 [ 19.739145][ T445] do_setlink.isra.0+0x1cc3/0x2750 [ 19.739257][ T445] rtnl_newlink+0x8d1/0xef0 [ 19.739360][ T445] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.739466][ T445] netlink_rcv_skb+0x14e/0x3a0 [ 19.739574][ T445] netlink_unicast+0x486/0x750 [ 19.739681][ T445] netlink_sendmsg+0x735/0xc60 [ 19.739787][ T445] ____sys_sendmsg+0x419/0x850 [ 19.739901][ T445] ___sys_sendmsg+0x14e/0x1d0 [ 19.740002][ T445] __sys_sendmsg+0x145/0x1f0 [ 19.740102][ T445] do_syscall_64+0x117/0x590 [ 19.740201][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.740324][ T445] HARDIRQ-ON-R at: [ 19.740388][ T445] __lock_acquire+0x378/0xc10 [ 19.740489][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.740592][ T445] _raw_read_lock_bh+0x44/0x80 [ 19.740691][ T445] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 19.740814][ T445] inet6_fill_link_af+0x5c/0xe0 [ 19.740917][ T445] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 19.741035][ T445] rtnl_getlink+0x9c9/0xeb0 [ 19.741135][ T445] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.741240][ T445] netlink_rcv_skb+0x14e/0x3a0 [ 19.741343][ T445] netlink_unicast+0x486/0x750 [ 19.741445][ T445] netlink_sendmsg+0x735/0xc60 [ 19.741546][ T445] ____sys_sendmsg+0x419/0x850 [ 19.741658][ T445] ___sys_sendmsg+0x14e/0x1d0 [ 19.741766][ T445] __sys_sendmsg+0x145/0x1f0 [ 19.741869][ T445] do_syscall_64+0x117/0x590 [ 19.741979][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.742101][ T445] IN-SOFTIRQ-R at: [ 19.742161][ T445] __lock_acquire+0x378/0xc10 [ 19.742268][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.742367][ T445] _raw_read_lock_bh+0x44/0x80 [ 19.742468][ T445] ipv6_get_lladdr+0xa8/0x3f0 [ 19.742573][ T445] ndisc_send_ns+0xf3/0x120 [ 19.742673][ T445] ndisc_solicit+0x314/0x480 [ 19.742774][ T445] neigh_probe+0xb3/0x100 [ 19.742876][ T445] __neigh_event_send+0x802/0x1070 [ 19.742976][ T445] neigh_resolve_output+0xbe/0x140 [ 19.743077][ T445] ip6_finish_output2+0x423/0x1300 [ 19.743175][ T445] ip6_finish_output+0x646/0xda0 [ 19.743284][ T445] ip6_output+0x23f/0x7f0 [ 19.743385][ T445] ip6_forward+0x10f4/0x2bd0 [ 19.743487][ T445] ipv6_rcv+0x3c1/0x5a0 [ 19.743593][ T445] __netif_receive_skb_one_core+0xfc/0x180 [ 19.743717][ T445] process_backlog+0x2bc/0x1490 [ 19.743818][ T445] __napi_poll+0xa7/0x3b0 [ 19.743921][ T445] net_rx_action+0x513/0xf50 [ 19.744021][ T445] handle_softirqs+0x1d8/0x940 [ 19.744122][ T445] do_softirq+0xa9/0xe0 [ 19.744228][ T445] __local_bh_enable_ip+0x113/0x140 [ 19.744329][ T445] __neigh_event_send+0x30a/0x1070 [ 19.744430][ T445] neigh_resolve_output+0xbe/0x140 [ 19.744532][ T445] ip6_finish_output2+0x423/0x1300 [ 19.744638][ T445] ip6_finish_output+0x646/0xda0 [ 19.744739][ T445] ip6_output+0x23f/0x7f0 [ 19.744840][ T445] ip6_send_skb+0xee/0x350 [ 19.744948][ T445] rawv6_push_pending_frames+0x4cb/0xa90 [ 19.745072][ T445] rawv6_sendmsg+0x1434/0x1a50 [ 19.745172][ T445] __sys_sendto+0x2c9/0x400 [ 19.745274][ T445] __x64_sys_sendto+0xe4/0x1f0 [ 19.745380][ T445] do_syscall_64+0x117/0x590 [ 19.745480][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.745614][ T445] INITIAL USE at: [ 19.745674][ T445] __lock_acquire+0x378/0xc10 [ 19.745776][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.745881][ T445] _raw_write_lock_bh+0x38/0x50 [ 19.745981][ T445] addrconf_permanent_addr+0x108/0x9c0 [ 19.746104][ T445] addrconf_notify+0x151/0xf30 [ 19.746212][ T445] notifier_call_chain+0xb0/0x320 [ 19.746312][ T445] __dev_notify_flags+0xde/0x280 [ 19.746418][ T445] netif_change_flags+0xfe/0x190 [ 19.746523][ T445] do_setlink.isra.0+0x1cc3/0x2750 [ 19.746628][ T445] rtnl_newlink+0x8d1/0xef0 [ 19.746738][ T445] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.746837][ T445] netlink_rcv_skb+0x14e/0x3a0 [ 19.746938][ T445] netlink_unicast+0x486/0x750 [ 19.747039][ T445] netlink_sendmsg+0x735/0xc60 [ 19.747139][ T445] ____sys_sendmsg+0x419/0x850 [ 19.747242][ T445] ___sys_sendmsg+0x14e/0x1d0 [ 19.747342][ T445] __sys_sendmsg+0x145/0x1f0 [ 19.747449][ T445] do_syscall_64+0x117/0x590 [ 19.747567][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.747687][ T445] INITIAL READ USE at: [ 19.747775][ T445] __lock_acquire+0x378/0xc10 [ 19.747875][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.747999][ T445] _raw_read_lock_bh+0x44/0x80 [ 19.748103][ T445] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 19.748225][ T445] inet6_fill_link_af+0x5c/0xe0 [ 19.748332][ T445] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 19.748459][ T445] rtnl_getlink+0x9c9/0xeb0 [ 19.748569][ T445] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 19.748669][ T445] netlink_rcv_skb+0x14e/0x3a0 [ 19.748770][ T445] netlink_unicast+0x486/0x750 [ 19.748872][ T445] netlink_sendmsg+0x735/0xc60 [ 19.748974][ T445] ____sys_sendmsg+0x419/0x850 [ 19.749091][ T445] ___sys_sendmsg+0x14e/0x1d0 [ 19.749201][ T445] __sys_sendmsg+0x145/0x1f0 [ 19.749306][ T445] do_syscall_64+0x117/0x590 [ 19.749408][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.749532][ T445] } [ 19.749577][ T445] ... key at: [] __key.37+0x0/0x40 [ 19.749748][ T445] ... acquired at: [ 19.749809][ T445] __lock_acquire+0x508/0xc10 [ 19.749899][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.749982][ T445] _raw_spin_lock+0x33/0x40 [ 19.750111][ T445] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.750212][ T445] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.750293][ T445] addrconf_join_anycast+0x1bc/0x260 [ 19.750373][ T445] __ipv6_ifa_notify+0x75b/0xad0 [ 19.750494][ T445] addrconf_dad_completed+0x14b/0xe10 [ 19.750580][ T445] addrconf_dad_work+0x3c1/0x930 [ 19.750660][ T445] process_one_work+0xdf8/0x1410 [ 19.750743][ T445] worker_thread+0x4f1/0xd60 [ 19.750865][ T445] kthread+0x367/0x460 [ 19.750925][ T445] ret_from_fork+0x474/0x6b0 [ 19.751011][ T445] ret_from_fork_asm+0x11/0x20 [ 19.751097][ T445] [ 19.751142][ T445] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 19.751266][ T445] HARDIRQ-ON-W at: [ 19.751327][ T445] __lock_acquire+0x378/0xc10 [ 19.751435][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.751540][ T445] _raw_spin_lock+0x33/0x40 [ 19.751693][ T445] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.751816][ T445] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.751921][ T445] addrconf_join_anycast+0x1bc/0x260 [ 19.752070][ T445] __ipv6_ifa_notify+0x75b/0xad0 [ 19.752173][ T445] addrconf_dad_completed+0x14b/0xe10 [ 19.752272][ T445] addrconf_dad_work+0x3c1/0x930 [ 19.752416][ T445] process_one_work+0xdf8/0x1410 [ 19.752516][ T445] worker_thread+0x4f1/0xd60 [ 19.752632][ T445] kthread+0x367/0x460 [ 19.752754][ T445] ret_from_fork+0x474/0x6b0 [ 19.752856][ T445] ret_from_fork_asm+0x11/0x20 [ 19.752960][ T445] SOFTIRQ-ON-W at: [ 19.753022][ T445] __lock_acquire+0x378/0xc10 [ 19.753174][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.753279][ T445] _raw_spin_lock+0x33/0x40 [ 19.753381][ T445] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.753523][ T445] addrconf_leave_anycast+0x1bc/0x260 [ 19.753631][ T445] dev_forward_change+0x3b3/0x850 [ 19.753730][ T445] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.753897][ T445] addrconf_sysctl_forward+0x209/0x2b0 [ 19.754004][ T445] proc_sys_call_handler+0x31b/0x480 [ 19.754110][ T445] new_sync_write+0x333/0x750 [ 19.754275][ T445] vfs_write+0x6a4/0xc10 [ 19.754378][ T445] ksys_write+0x116/0x250 [ 19.754483][ T445] do_syscall_64+0x117/0x590 [ 19.754631][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.754759][ T445] INITIAL USE at: [ 19.754819][ T445] __lock_acquire+0x378/0xc10 [ 19.754919][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.755065][ T445] _raw_spin_lock+0x33/0x40 [ 19.755165][ T445] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 19.755285][ T445] __ipv6_dev_ac_inc+0x57e/0x930 [ 19.755438][ T445] addrconf_join_anycast+0x1bc/0x260 [ 19.755539][ T445] __ipv6_ifa_notify+0x75b/0xad0 [ 19.755642][ T445] addrconf_dad_completed+0x14b/0xe10 [ 19.755784][ T445] addrconf_dad_work+0x3c1/0x930 [ 19.755885][ T445] process_one_work+0xdf8/0x1410 [ 19.755985][ T445] worker_thread+0x4f1/0xd60 [ 19.756128][ T445] kthread+0x367/0x460 [ 19.756209][ T445] ret_from_fork+0x474/0x6b0 [ 19.756309][ T445] ret_from_fork_asm+0x11/0x20 [ 19.756409][ T445] } [ 19.756492][ T445] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 19.756626][ T445] ... acquired at: [ 19.756694][ T445] mark_lock+0x1d7/0xa00 [ 19.756776][ T445] mark_usage+0x105/0x170 [ 19.756898][ T445] __lock_acquire+0x378/0xc10 [ 19.756977][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.757056][ T445] _raw_spin_lock+0x33/0x40 [ 19.757137][ T445] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.757260][ T445] addrconf_leave_anycast+0x1bc/0x260 [ 19.757339][ T445] dev_forward_change+0x3b3/0x850 [ 19.757420][ T445] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.757520][ T445] addrconf_sysctl_forward+0x209/0x2b0 [ 19.757645][ T445] proc_sys_call_handler+0x31b/0x480 [ 19.757726][ T445] new_sync_write+0x333/0x750 [ 19.757806][ T445] vfs_write+0x6a4/0xc10 [ 19.757887][ T445] ksys_write+0x116/0x250 [ 19.758009][ T445] do_syscall_64+0x117/0x590 [ 19.758092][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.758191][ T445] [ 19.758231][ T445] [ 19.758231][ T445] stack backtrace: [ 19.758372][ T445] CPU: 3 UID: 0 PID: 445 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 19.758375][ T445] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 19.758377][ T445] Call Trace: [ 19.758379][ T445] [ 19.758380][ T445] dump_stack_lvl+0x6f/0xa0 [ 19.758385][ T445] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 19.758388][ T445] mark_lock_irq+0x989/0x9c0 [ 19.758390][ T445] mark_lock+0x1d7/0xa00 [ 19.758392][ T445] mark_usage+0x105/0x170 [ 19.758394][ T445] __lock_acquire+0x378/0xc10 [ 19.758396][ T445] lock_acquire.part.0+0xbc/0x260 [ 19.758397][ T445] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.758399][ T445] ? rcu_is_watching+0x15/0xd0 [ 19.758402][ T445] ? do_raw_read_unlock+0x70/0x70 [ 19.758404][ T445] ? lock_acquire+0x134/0x160 [ 19.758406][ T445] _raw_spin_lock+0x33/0x40 [ 19.758407][ T445] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.758408][ T445] __ipv6_dev_ac_dec+0x236/0x5d0 [ 19.758411][ T445] addrconf_leave_anycast+0x1bc/0x260 [ 19.758412][ T445] ? find_held_lock+0x2b/0x80 [ 19.758415][ T445] ? __ipv6_isatap_ifid+0x210/0x210 [ 19.758417][ T445] ? mark_held_locks+0x40/0x70 [ 19.758419][ T445] ? lockdep_hardirqs_on+0x8c/0x130 [ 19.758421][ T445] dev_forward_change+0x3b3/0x850 [ 19.758423][ T445] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 19.758425][ T445] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 19.758428][ T445] addrconf_fixup_forwarding+0x2af/0x4d0 [ 19.758430][ T445] addrconf_sysctl_forward+0x209/0x2b0 [ 19.758432][ T445] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 19.758435][ T445] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 19.758437][ T445] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 19.758440][ T445] ? proc_sys_call_handler+0x25d/0x480 [ 19.758442][ T445] proc_sys_call_handler+0x31b/0x480 [ 19.758444][ T445] ? proc_sys_lookup+0x3d0/0x3d0 [ 19.758446][ T445] ? rcu_read_unlock+0x1b/0x70 [ 19.758449][ T445] ? do_raw_spin_unlock+0x59/0x250 [ 19.758450][ T445] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 19.758453][ T445] ? rcu_read_lock_any_held+0x3c/0x90 [ 19.758454][ T445] ? proc_sys_call_handler+0x480/0x480 [ 19.758456][ T445] new_sync_write+0x333/0x750 [ 19.758458][ T445] ? __lock_acquire+0x508/0xc10 [ 19.758459][ T445] ? new_sync_read+0x740/0x740 [ 19.758461][ T445] ? lock_acquire.part.0+0xbc/0x260 [ 19.758462][ T445] ? ksys_write+0x116/0x250 [ 19.758464][ T445] vfs_write+0x6a4/0xc10 [ 19.758466][ T445] ksys_write+0x116/0x250 [ 19.758468][ T445] ? __ia32_sys_read+0xc0/0xc0 [ 19.758469][ T445] ? rcu_is_watching+0x15/0xd0 [ 19.758471][ T445] ? rcu_is_watching+0x15/0xd0 [ 19.758473][ T445] do_syscall_64+0x117/0x590 [ 19.758474][ T445] ? trace_hardirqs_off+0xd/0x30 [ 19.758477][ T445] ? exc_page_fault+0xee/0x100 [ 19.758479][ T445] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 19.758481][ T445] RIP: 0033:0x7fb974db208e [ 19.758484][ T445] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 19.758485][ T445] RSP: 002b:00007ffdd88d3620 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 19.758488][ T445] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fb974db208e [ 19.758490][ T445] RDX: 0000000000000002 RSI: 000056387645c4b0 RDI: 0000000000000005 [ 19.758491][ T445] RBP: 00007ffdd88d3630 R08: 0000000000000000 R09: 0000000000000000 [ 19.758491][ T445] R10: 0000000000000000 R11: 0000000000000202 R12: 000056387645e5e0 [ 19.758492][ T445] R13: 000056387645c470 R14: 0000000000000002 R15: 0000000000000000 [ 19.758495][ T445] [ 20.303207][ T219] veth0: left allmulticast mode [ 20.303445][ T219] veth1: left allmulticast mode [ 20.303624][ T219] veth2: left allmulticast mode [ 20.303777][ T219] veth3: left allmulticast mode