[ 9.087349][ T193] systemd-sysctl (193) used greatest stack depth: 24480 bytes left [ 9.123718][ T191] ip (191) used greatest stack depth: 24032 bytes left [ 9.702344][ T216] 8021q: 802.1Q VLAN Support v1.8 [ 10.412265][ T247] br1: port 1(veth1) entered blocking state [ 10.412580][ T247] br1: port 1(veth1) entered disabled state [ 10.412769][ T247] veth1: entered allmulticast mode [ 10.414886][ T247] veth1: entered promiscuous mode [ 10.448529][ T59] br1: port 1(veth1) entered blocking state [ 10.448847][ T59] br1: port 1(veth1) entered forwarding state [ 21.277368][ T343] [ 21.277455][ T343] ======================================================== [ 21.277589][ T343] WARNING: possible irq lock inversion dependency detected [ 21.277711][ T343] 7.1.0-rc5-virtme #1 Not tainted [ 21.277819][ T343] -------------------------------------------------------- [ 21.277947][ T343] sysctl/343 just changed the state of lock: [ 21.278049][ T343] ffffffff9af7e8b8 (acaddr_hash_lock){+.+.}-{3:3}, at: __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.278194][ T343] but this lock was taken by another, SOFTIRQ-READ-safe lock in the past: [ 21.278310][ T343] (&ndev->lock){++.-}-{3:3} [ 21.278312][ T343] [ 21.278312][ T343] [ 21.278312][ T343] and interrupts could create inverse lock ordering between them. [ 21.278312][ T343] [ 21.278606][ T343] [ 21.278606][ T343] other info that might help us debug this: [ 21.278725][ T343] Possible interrupt unsafe locking scenario: [ 21.278725][ T343] [ 21.278847][ T343] CPU0 CPU1 [ 21.278925][ T343] ---- ---- [ 21.279011][ T343] lock(acaddr_hash_lock); [ 21.279092][ T343] local_irq_disable(); [ 21.279191][ T343] lock(&ndev->lock); [ 21.279293][ T343] lock(acaddr_hash_lock); [ 21.279390][ T343] [ 21.279450][ T343] lock(&ndev->lock); [ 21.279520][ T343] [ 21.279520][ T343] *** DEADLOCK *** [ 21.279520][ T343] [ 21.279646][ T343] 2 locks held by sysctl/343: [ 21.279724][ T343] #0: ff110000097813e0 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 21.279872][ T343] #1: ffffffff9ae8c9c0 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_fixup_forwarding+0x4d/0x4d0 [ 21.280036][ T343] [ 21.280036][ T343] the shortest dependencies between 2nd lock and 1st lock: [ 21.280177][ T343] -> (&ndev->lock){++.-}-{3:3} { [ 21.280261][ T343] HARDIRQ-ON-W at: [ 21.280331][ T343] __lock_acquire+0x378/0xc10 [ 21.280436][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.280534][ T343] _raw_write_lock_bh+0x38/0x50 [ 21.280642][ T343] addrconf_permanent_addr+0x108/0x9c0 [ 21.280763][ T343] addrconf_notify+0x151/0xf30 [ 21.280865][ T343] notifier_call_chain+0xb0/0x320 [ 21.280969][ T343] __dev_notify_flags+0xde/0x280 [ 21.281073][ T343] netif_change_flags+0xfe/0x190 [ 21.281178][ T343] do_setlink.isra.0+0x1cc3/0x2750 [ 21.281287][ T343] rtnl_newlink+0x8d1/0xef0 [ 21.281386][ T343] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 21.281484][ T343] netlink_rcv_skb+0x14e/0x3a0 [ 21.281586][ T343] netlink_unicast+0x486/0x750 [ 21.281682][ T343] netlink_sendmsg+0x735/0xc60 [ 21.281782][ T343] ____sys_sendmsg+0x419/0x850 [ 21.281881][ T343] ___sys_sendmsg+0x14e/0x1d0 [ 21.281990][ T343] __sys_sendmsg+0x145/0x1f0 [ 21.282092][ T343] do_syscall_64+0x117/0x590 [ 21.282190][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.282309][ T343] HARDIRQ-ON-R at: [ 21.282371][ T343] __lock_acquire+0x378/0xc10 [ 21.282468][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.282571][ T343] _raw_read_lock_bh+0x44/0x80 [ 21.282668][ T343] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 21.282789][ T343] inet6_fill_link_af+0x5c/0xe0 [ 21.282889][ T343] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 21.283007][ T343] rtnl_getlink+0x9c9/0xeb0 [ 21.283104][ T343] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 21.283201][ T343] netlink_rcv_skb+0x14e/0x3a0 [ 21.283298][ T343] netlink_unicast+0x486/0x750 [ 21.283396][ T343] netlink_sendmsg+0x735/0xc60 [ 21.283493][ T343] ____sys_sendmsg+0x419/0x850 [ 21.283594][ T343] ___sys_sendmsg+0x14e/0x1d0 [ 21.283691][ T343] __sys_sendmsg+0x145/0x1f0 [ 21.283788][ T343] do_syscall_64+0x117/0x590 [ 21.283892][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.284011][ T343] IN-SOFTIRQ-R at: [ 21.284074][ T343] __lock_acquire+0x378/0xc10 [ 21.284172][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.284269][ T343] _raw_read_lock_bh+0x44/0x80 [ 21.284366][ T343] ipv6_get_lladdr+0xa8/0x3f0 [ 21.284465][ T343] ndisc_send_ns+0xf3/0x120 [ 21.284568][ T343] ndisc_solicit+0x314/0x480 [ 21.284665][ T343] neigh_probe+0xb3/0x100 [ 21.284762][ T343] __neigh_event_send+0x802/0x1070 [ 21.284860][ T343] neigh_resolve_output+0xbe/0x140 [ 21.284957][ T343] ip6_finish_output2+0x423/0x1300 [ 21.285055][ T343] ip6_finish_output+0x646/0xda0 [ 21.285158][ T343] ip6_output+0x23f/0x7f0 [ 21.285263][ T343] ip6_forward+0x10f4/0x2bd0 [ 21.285364][ T343] ipv6_rcv+0x3c1/0x5a0 [ 21.285464][ T343] __netif_receive_skb_one_core+0xfc/0x180 [ 21.285590][ T343] netif_receive_skb_internal+0xb0/0x330 [ 21.285706][ T343] netif_receive_skb+0x56/0x380 [ 21.285808][ T343] br_handle_frame_finish+0xc3b/0x1fb0 [bridge] [ 21.285936][ T343] br_handle_frame+0x739/0xf40 [bridge] [ 21.286057][ T343] __netif_receive_skb_core.constprop.0+0x536/0x2bc0 [ 21.286182][ T343] __netif_receive_skb_one_core+0xa2/0x180 [ 21.286309][ T343] process_backlog+0x2bc/0x1490 [ 21.286411][ T343] __napi_poll+0xa7/0x3b0 [ 21.286513][ T343] net_rx_action+0x513/0xf50 [ 21.286619][ T343] handle_softirqs+0x1d8/0x940 [ 21.286722][ T343] do_softirq+0xa9/0xe0 [ 21.286823][ T343] __local_bh_enable_ip+0x113/0x140 [ 21.286926][ T343] __neigh_event_send+0x30a/0x1070 [ 21.287021][ T343] neigh_resolve_output+0xbe/0x140 [ 21.287118][ T343] ip6_finish_output2+0x423/0x1300 [ 21.287215][ T343] ip6_finish_output+0x646/0xda0 [ 21.287315][ T343] ip6_output+0x23f/0x7f0 [ 21.287419][ T343] ip6_send_skb+0xee/0x350 [ 21.287524][ T343] rawv6_push_pending_frames+0x4cb/0xa90 [ 21.287652][ T343] rawv6_sendmsg+0x1434/0x1a50 [ 21.287752][ T343] __sys_sendto+0x2c9/0x400 [ 21.287851][ T343] __x64_sys_sendto+0xe4/0x1f0 [ 21.287952][ T343] do_syscall_64+0x117/0x590 [ 21.288057][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.288172][ T343] INITIAL USE at: [ 21.288232][ T343] __lock_acquire+0x378/0xc10 [ 21.288337][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.288438][ T343] _raw_write_lock_bh+0x38/0x50 [ 21.288539][ T343] addrconf_permanent_addr+0x108/0x9c0 [ 21.288660][ T343] addrconf_notify+0x151/0xf30 [ 21.288769][ T343] notifier_call_chain+0xb0/0x320 [ 21.288869][ T343] __dev_notify_flags+0xde/0x280 [ 21.288967][ T343] netif_change_flags+0xfe/0x190 [ 21.289065][ T343] do_setlink.isra.0+0x1cc3/0x2750 [ 21.289164][ T343] rtnl_newlink+0x8d1/0xef0 [ 21.289262][ T343] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 21.289360][ T343] netlink_rcv_skb+0x14e/0x3a0 [ 21.289460][ T343] netlink_unicast+0x486/0x750 [ 21.289568][ T343] netlink_sendmsg+0x735/0xc60 [ 21.289669][ T343] ____sys_sendmsg+0x419/0x850 [ 21.289767][ T343] ___sys_sendmsg+0x14e/0x1d0 [ 21.289867][ T343] __sys_sendmsg+0x145/0x1f0 [ 21.289964][ T343] do_syscall_64+0x117/0x590 [ 21.290066][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.290191][ T343] INITIAL READ USE at: [ 21.290274][ T343] __lock_acquire+0x378/0xc10 [ 21.290375][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.290493][ T343] _raw_read_lock_bh+0x44/0x80 [ 21.290597][ T343] inet6_fill_ifla6_attrs+0x1aee/0x21e0 [ 21.290721][ T343] inet6_fill_link_af+0x5c/0xe0 [ 21.290821][ T343] rtnl_fill_ifinfo.isra.0+0x1a45/0x2c20 [ 21.290950][ T343] rtnl_getlink+0x9c9/0xeb0 [ 21.291056][ T343] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 21.291158][ T343] netlink_rcv_skb+0x14e/0x3a0 [ 21.291256][ T343] netlink_unicast+0x486/0x750 [ 21.291354][ T343] netlink_sendmsg+0x735/0xc60 [ 21.291460][ T343] ____sys_sendmsg+0x419/0x850 [ 21.291604][ T343] ___sys_sendmsg+0x14e/0x1d0 [ 21.291703][ T343] __sys_sendmsg+0x145/0x1f0 [ 21.291800][ T343] do_syscall_64+0x117/0x590 [ 21.291898][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.292014][ T343] } [ 21.292055][ T343] ... key at: [] __key.37+0x0/0x40 [ 21.292215][ T343] ... acquired at: [ 21.292279][ T343] __lock_acquire+0x508/0xc10 [ 21.292358][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.292437][ T343] _raw_spin_lock+0x33/0x40 [ 21.292515][ T343] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 21.292660][ T343] __ipv6_dev_ac_inc+0x57e/0x930 [ 21.292735][ T343] addrconf_join_anycast+0x1bc/0x260 [ 21.292813][ T343] __ipv6_ifa_notify+0x75b/0xad0 [ 21.292936][ T343] addrconf_dad_completed+0x14b/0xe10 [ 21.293017][ T343] addrconf_dad_work+0x3c1/0x930 [ 21.293100][ T343] process_one_work+0xdf8/0x1410 [ 21.293184][ T343] worker_thread+0x4f1/0xd60 [ 21.293311][ T343] kthread+0x367/0x460 [ 21.293371][ T343] ret_from_fork+0x474/0x6b0 [ 21.293451][ T343] ret_from_fork_asm+0x11/0x20 [ 21.293535][ T343] [ 21.293580][ T343] -> (acaddr_hash_lock){+.+.}-{3:3} { [ 21.293701][ T343] HARDIRQ-ON-W at: [ 21.293759][ T343] __lock_acquire+0x378/0xc10 [ 21.293859][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.293960][ T343] _raw_spin_lock+0x33/0x40 [ 21.294103][ T343] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 21.294229][ T343] __ipv6_dev_ac_inc+0x57e/0x930 [ 21.294325][ T343] addrconf_join_anycast+0x1bc/0x260 [ 21.294470][ T343] __ipv6_ifa_notify+0x75b/0xad0 [ 21.294576][ T343] addrconf_dad_completed+0x14b/0xe10 [ 21.294673][ T343] addrconf_dad_work+0x3c1/0x930 [ 21.294815][ T343] process_one_work+0xdf8/0x1410 [ 21.294911][ T343] worker_thread+0x4f1/0xd60 [ 21.295012][ T343] kthread+0x367/0x460 [ 21.295134][ T343] ret_from_fork+0x474/0x6b0 [ 21.295231][ T343] ret_from_fork_asm+0x11/0x20 [ 21.295329][ T343] SOFTIRQ-ON-W at: [ 21.295396][ T343] __lock_acquire+0x378/0xc10 [ 21.295539][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.295644][ T343] _raw_spin_lock+0x33/0x40 [ 21.295746][ T343] __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.295843][ T343] addrconf_leave_anycast+0x1bc/0x260 [ 21.295942][ T343] dev_forward_change+0x3b3/0x850 [ 21.296039][ T343] addrconf_fixup_forwarding+0x2af/0x4d0 [ 21.296199][ T343] addrconf_sysctl_forward+0x209/0x2b0 [ 21.296297][ T343] proc_sys_call_handler+0x31b/0x480 [ 21.296398][ T343] new_sync_write+0x333/0x750 [ 21.296539][ T343] vfs_write+0x6a4/0xc10 [ 21.296643][ T343] ksys_write+0x116/0x250 [ 21.296740][ T343] do_syscall_64+0x117/0x590 [ 21.296838][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.296995][ T343] INITIAL USE at: [ 21.297054][ T343] __lock_acquire+0x378/0xc10 [ 21.297158][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.297306][ T343] _raw_spin_lock+0x33/0x40 [ 21.297408][ T343] ipv6_add_acaddr_hash.isra.0+0x16d/0x350 [ 21.297530][ T343] __ipv6_dev_ac_inc+0x57e/0x930 [ 21.297678][ T343] addrconf_join_anycast+0x1bc/0x260 [ 21.297776][ T343] __ipv6_ifa_notify+0x75b/0xad0 [ 21.297876][ T343] addrconf_dad_completed+0x14b/0xe10 [ 21.298015][ T343] addrconf_dad_work+0x3c1/0x930 [ 21.298114][ T343] process_one_work+0xdf8/0x1410 [ 21.298215][ T343] worker_thread+0x4f1/0xd60 [ 21.298361][ T343] kthread+0x367/0x460 [ 21.298440][ T343] ret_from_fork+0x474/0x6b0 [ 21.298537][ T343] ret_from_fork_asm+0x11/0x20 [ 21.298638][ T343] } [ 21.298719][ T343] ... key at: [] acaddr_hash_lock+0x18/0x45a0 [ 21.298836][ T343] ... acquired at: [ 21.298893][ T343] mark_lock+0x1d7/0xa00 [ 21.298974][ T343] mark_usage+0x105/0x170 [ 21.299093][ T343] __lock_acquire+0x378/0xc10 [ 21.299171][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.299248][ T343] _raw_spin_lock+0x33/0x40 [ 21.299326][ T343] __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.299451][ T343] addrconf_leave_anycast+0x1bc/0x260 [ 21.299530][ T343] dev_forward_change+0x3b3/0x850 [ 21.299612][ T343] addrconf_fixup_forwarding+0x2af/0x4d0 [ 21.299710][ T343] addrconf_sysctl_forward+0x209/0x2b0 [ 21.299833][ T343] proc_sys_call_handler+0x31b/0x480 [ 21.299910][ T343] new_sync_write+0x333/0x750 [ 21.299994][ T343] vfs_write+0x6a4/0xc10 [ 21.300070][ T343] ksys_write+0x116/0x250 [ 21.300192][ T343] do_syscall_64+0x117/0x590 [ 21.300274][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.300372][ T343] [ 21.300411][ T343] [ 21.300411][ T343] stack backtrace: [ 21.300548][ T343] CPU: 3 UID: 0 PID: 343 Comm: sysctl Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 21.300551][ T343] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 21.300557][ T343] Call Trace: [ 21.300558][ T343] [ 21.300559][ T343] dump_stack_lvl+0x6f/0xa0 [ 21.300564][ T343] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 21.300567][ T343] mark_lock_irq+0x989/0x9c0 [ 21.300569][ T343] mark_lock+0x1d7/0xa00 [ 21.300571][ T343] mark_usage+0x105/0x170 [ 21.300572][ T343] __lock_acquire+0x378/0xc10 [ 21.300574][ T343] lock_acquire.part.0+0xbc/0x260 [ 21.300576][ T343] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.300577][ T343] ? rcu_is_watching+0x15/0xd0 [ 21.300580][ T343] ? do_raw_read_unlock+0x70/0x70 [ 21.300582][ T343] ? lock_acquire+0x134/0x160 [ 21.300584][ T343] _raw_spin_lock+0x33/0x40 [ 21.300586][ T343] ? __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.300587][ T343] __ipv6_dev_ac_dec+0x236/0x5d0 [ 21.300589][ T343] addrconf_leave_anycast+0x1bc/0x260 [ 21.300591][ T343] ? find_held_lock+0x2b/0x80 [ 21.300593][ T343] ? __ipv6_isatap_ifid+0x210/0x210 [ 21.300595][ T343] ? mark_held_locks+0x40/0x70 [ 21.300597][ T343] ? lockdep_hardirqs_on+0x8c/0x130 [ 21.300599][ T343] dev_forward_change+0x3b3/0x850 [ 21.300601][ T343] ? addrconf_fixup_forwarding+0x4d/0x4d0 [ 21.300603][ T343] ? addrconf_sysctl_proxy_ndp+0x2f0/0x2f0 [ 21.300606][ T343] addrconf_fixup_forwarding+0x2af/0x4d0 [ 21.300608][ T343] addrconf_sysctl_forward+0x209/0x2b0 [ 21.300610][ T343] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 21.300612][ T343] ? addrconf_fixup_forwarding+0x4d0/0x4d0 [ 21.300614][ T343] ? __kvmalloc_node_noprof+0x305/0x8a0 [ 21.300617][ T343] ? proc_sys_call_handler+0x25d/0x480 [ 21.300620][ T343] proc_sys_call_handler+0x31b/0x480 [ 21.300622][ T343] ? proc_sys_lookup+0x3d0/0x3d0 [ 21.300624][ T343] ? rcu_read_unlock+0x1b/0x70 [ 21.300626][ T343] ? do_raw_spin_unlock+0x59/0x250 [ 21.300628][ T343] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 21.300630][ T343] ? rcu_read_lock_any_held+0x3c/0x90 [ 21.300631][ T343] ? proc_sys_call_handler+0x480/0x480 [ 21.300634][ T343] new_sync_write+0x333/0x750 [ 21.300635][ T343] ? __lock_acquire+0x508/0xc10 [ 21.300636][ T343] ? new_sync_read+0x740/0x740 [ 21.300638][ T343] ? lock_acquire.part.0+0xbc/0x260 [ 21.300639][ T343] ? ksys_write+0x116/0x250 [ 21.300641][ T343] vfs_write+0x6a4/0xc10 [ 21.300643][ T343] ksys_write+0x116/0x250 [ 21.300645][ T343] ? __ia32_sys_read+0xc0/0xc0 [ 21.300646][ T343] ? rcu_is_watching+0x15/0xd0 [ 21.300647][ T343] ? rcu_is_watching+0x15/0xd0 [ 21.300649][ T343] do_syscall_64+0x117/0x590 [ 21.300650][ T343] ? trace_hardirqs_off+0xd/0x30 [ 21.300653][ T343] ? exc_page_fault+0xee/0x100 [ 21.300655][ T343] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 21.300657][ T343] RIP: 0033:0x7f7c6df8108e [ 21.300659][ T343] Code: 4d 89 d8 e8 94 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 21.300661][ T343] RSP: 002b:00007ffec75d4ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 21.300663][ T343] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f7c6df8108e [ 21.300665][ T343] RDX: 0000000000000002 RSI: 0000558b031434b0 RDI: 0000000000000005 [ 21.300666][ T343] RBP: 00007ffec75d4ae0 R08: 0000000000000000 R09: 0000000000000000 [ 21.300667][ T343] R10: 0000000000000000 R11: 0000000000000202 R12: 0000558b031455e0 [ 21.300667][ T343] R13: 0000558b03143470 R14: 0000000000000002 R15: 0000000000000000 [ 21.300670][ T343] [ 21.439861][ T351] br1: port 1(veth1) entered disabled state [ 21.453569][ T352] veth1: left allmulticast mode [ 21.453677][ T352] veth1: left promiscuous mode [ 21.454486][ T352] br1: port 1(veth1) entered disabled state