[ 76.854060][ T1502] Mirror/redirect action on [ 76.892422][ C1] tc mirred to Houston: device veth2 is down [ 77.088491][ C1] tc mirred to Houston: device veth3 is down [ 84.637359][ C1] [ 84.637455][ C1] ================================ [ 84.637555][ C1] WARNING: inconsistent lock state [ 84.637647][ C1] 7.1.0-rc5-virtme #1 Not tainted [ 84.637743][ C1] -------------------------------- [ 84.637833][ C1] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. [ 84.637948][ C1] kworker/1:1/51 [HC0[0]:SC1[1]:HE1:SE0] takes: [ 84.638062][ C1] ffffffffc08297b8 (mirred_list_lock){+.?.}-{3:3}, at: tcf_mirred_release+0x1d/0x2e0 [act_mirred] [ 84.638266][ C1] {SOFTIRQ-ON-W} state was registered at: [ 84.638360][ C1] __lock_acquire+0x378/0xc10 [ 84.638456][ C1] lock_acquire.part.0+0xbc/0x260 [ 84.638549][ C1] _raw_spin_lock+0x33/0x40 [ 84.638641][ C1] tcf_mirred_init+0x844/0xd70 [act_mirred] [ 84.638758][ C1] tcf_action_init_1+0x428/0x6b0 [ 84.638851][ C1] tcf_action_init+0x3cf/0x890 [ 84.638942][ C1] tcf_exts_validate_ex+0x411/0x670 [ 84.639036][ C1] mall_change+0x3cd/0xe9b [cls_matchall] [ 84.639155][ C1] tc_new_tfilter+0x875/0x21e0 [ 84.639247][ C1] rtnetlink_rcv_msg+0x6fd/0xbd0 [ 84.639341][ C1] netlink_rcv_skb+0x14e/0x3a0 [ 84.639431][ C1] netlink_unicast+0x486/0x750 [ 84.639523][ C1] netlink_sendmsg+0x735/0xc60 [ 84.639612][ C1] ____sys_sendmsg+0x419/0x850 [ 84.639704][ C1] ___sys_sendmsg+0x14e/0x1d0 [ 84.639794][ C1] __sys_sendmsg+0x145/0x1f0 [ 84.639884][ C1] do_syscall_64+0x117/0x590 [ 84.639977][ C1] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 84.640097][ C1] irq event stamp: 2865172 [ 84.640188][ C1] hardirqs last enabled at (2865172): [] _raw_spin_unlock_irqrestore+0x53/0x80 [ 84.640372][ C1] hardirqs last disabled at (2865171): [] _raw_spin_lock_irqsave+0x50/0x60 [ 84.640555][ C1] softirqs last enabled at (2864398): [] handle_softirqs+0x611/0x940 [ 84.640715][ C1] softirqs last disabled at (2864937): [] __irq_exit_rcu+0x103/0x1c0 [ 84.640876][ C1] [ 84.640876][ C1] other info that might help us debug this: [ 84.641011][ C1] Possible unsafe locking scenario: [ 84.641011][ C1] [ 84.641153][ C1] CPU0 [ 84.641222][ C1] ---- [ 84.641288][ C1] lock(mirred_list_lock); [ 84.641380][ C1] [ 84.641449][ C1] lock(mirred_list_lock); [ 84.641540][ C1] [ 84.641540][ C1] *** DEADLOCK *** [ 84.641540][ C1] [ 84.641671][ C1] 4 locks held by kworker/1:1/51: [ 84.641762][ C1] #0: ff11000001139b40 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xd55/0x1410 [ 84.641948][ C1] #1: ffa000000037fd10 ((work_completion)(&fsvq->done_work)){+.+.}-{0:0}, at: process_one_work+0xdb7/0x1410 [ 84.642156][ C1] #2: ff11000005d305d8 (&fsvq->lock){+.+.}-{3:3}, at: virtio_fs_requests_done_work+0xf5/0x8f0 [ 84.642341][ C1] #3: ffffffffb3577c00 (rcu_callback){....}-{0:0}, at: rcu_do_batch+0x389/0x1020 [ 84.642501][ C1] [ 84.642501][ C1] stack backtrace: [ 84.642613][ C1] CPU: 1 UID: 0 PID: 51 Comm: kworker/1:1 Not tainted 7.1.0-rc5-virtme #1 PREEMPT(full) [ 84.642615][ C1] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 84.642617][ C1] Workqueue: events virtio_fs_requests_done_work [ 84.642621][ C1] Call Trace: [ 84.642622][ C1] [ 84.642624][ C1] dump_stack_lvl+0x6f/0xa0 [ 84.642628][ C1] print_usage_bug.part.0.cold+0xa9/0xb7 [ 84.642631][ C1] ? filter_irq_stacks+0xd0/0xd0 [ 84.642635][ C1] mark_lock_irq+0x4e3/0x9c0 [ 84.642636][ C1] ? sysvec_apic_timer_interrupt+0x9d/0xe0 [ 84.642639][ C1] ? asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 84.642641][ C1] ? save_trace+0x68/0x280 [ 84.642642][ C1] ? virtqueue_enable_cb_prepare_packed+0x2a0/0x410 [ 84.642646][ C1] mark_lock+0x1d7/0xa00 [ 84.642648][ C1] mark_usage+0x42/0x170 [ 84.642649][ C1] __lock_acquire+0x378/0xc10 [ 84.642651][ C1] ? mark_held_locks+0x40/0x70 [ 84.642652][ C1] ? rcu_do_batch+0x2b4/0x1020 [ 84.642655][ C1] lock_acquire.part.0+0xbc/0x260 [ 84.642657][ C1] ? tcf_mirred_release+0x1d/0x2e0 [act_mirred] [ 84.642658][ C1] ? rcu_is_watching+0x15/0xd0 [ 84.642660][ C1] ? lock_acquire+0x134/0x160 [ 84.642662][ C1] _raw_spin_lock+0x33/0x40 [ 84.642664][ C1] ? tcf_mirred_release+0x1d/0x2e0 [act_mirred] [ 84.642666][ C1] tcf_mirred_release+0x1d/0x2e0 [act_mirred] [ 84.642667][ C1] tcf_action_rcu_free+0x74/0x1a0 [ 84.642669][ C1] ? rcu_do_batch+0x2b4/0x1020 [ 84.642671][ C1] rcu_do_batch+0x2b6/0x1020 [ 84.642673][ C1] ? trace_rcu_batch_end+0x330/0x330 [ 84.642676][ C1] ? mark_held_locks+0x40/0x70 [ 84.642677][ C1] ? lockdep_hardirqs_on_prepare.part.0+0x9a/0x160 [ 84.642679][ C1] ? lockdep_hardirqs_on+0x8c/0x130 [ 84.642681][ C1] rcu_core+0x2bf/0x640 [ 84.642683][ C1] handle_softirqs+0x1d8/0x940 [ 84.642684][ C1] ? find_held_lock+0x2b/0x80 [ 84.642687][ C1] ? __lock_release.isra.0+0x6b/0x1a0 [ 84.642689][ C1] ? _local_bh_enable+0xd0/0xd0 [ 84.642691][ C1] __irq_exit_rcu+0x103/0x1c0 [ 84.642692][ C1] irq_exit_rcu+0xe/0x30 [ 84.642694][ C1] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 84.642695][ C1] [ 84.642696][ C1] [ 84.642697][ C1] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 84.642698][ C1] RIP: 0010:__unwind_start+0x42d/0x910 [ 84.642702][ C1] Code: 4c 24 08 85 d2 0f 84 8f 00 00 00 48 bd 00 00 00 00 00 fc ff df 48 c1 e8 03 48 c1 e9 03 4c 8d 2c 28 4c 8d 24 29 eb 26 4c 89 ff 2e dd ff ff 4c 89 f8 48 c1 e8 03 0f b6 04 28 84 c0 74 08 3c 03 [ 84.642704][ C1] RSP: 0018:ffa000000037f658 EFLAGS: 00000216 [ 84.642706][ C1] RAX: 1ff400000006fede RBX: ffa000000037f740 RCX: ffa000000037f748 [ 84.642707][ C1] RDX: ffa0000000380000 RSI: ff110000024ac640 RDI: ffa000000037f6a8 [ 84.642708][ C1] RBP: dffffc0000000000 R08: ffa000000037f6d0 R09: ffa000000037f6a8 [ 84.642708][ C1] R10: fff3fc000006fee3 R11: fff3fc000006fee3 R12: fff3fc000006fed7 [ 84.642709][ C1] R13: fff3fc000006fed6 R14: ffa000000037f6f0 R15: ffa000000037f6a8 [ 84.642711][ C1] ? __unwind_start+0x3c5/0x910 [ 84.642713][ C1] ? __unwind_start+0x432/0x910 [ 84.642714][ C1] ? write_profile+0xf0/0xf0 [ 84.642716][ C1] arch_stack_walk+0xb3/0x130 [ 84.642720][ C1] ? __unwind_start+0x35b/0x910 [ 84.642722][ C1] ? detach_buf_split_in_order+0x438/0x670 [ 84.642724][ C1] stack_trace_save+0x93/0xc0 [ 84.642726][ C1] ? filter_irq_stacks+0xd0/0xd0 [ 84.642728][ C1] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 84.642730][ C1] ? validate_chain+0x38b/0xc20 [ 84.642732][ C1] ? sched_ttwu_pending+0x5e0/0x5e0 [ 84.642735][ C1] kasan_save_stack+0x2f/0x50 [ 84.642743][ C1] kasan_record_aux_stack+0x9b/0xc0 [ 84.642745][ C1] ? __remove_object+0x90/0x90 [ 84.642747][ C1] __call_rcu_common.constprop.0+0xb2/0xa10 [ 84.642748][ C1] ? lockdep_hardirqs_on+0x8c/0x130 [ 84.642750][ C1] ? _raw_spin_unlock_irqrestore+0x53/0x80 [ 84.642752][ C1] ? _raw_spin_unlock_irqrestore+0x40/0x80 [ 84.642753][ C1] kfree+0x36c/0x5a0 [ 84.642757][ C1] detach_buf_split_in_order+0x438/0x670 [ 84.642759][ C1] ? do_raw_spin_lock+0x131/0x280 [ 84.642761][ C1] virtqueue_get_buf_ctx_split+0x2d1/0x820 [ 84.642764][ C1] virtio_fs_requests_done_work+0x296/0x8f0 [ 84.642766][ C1] ? virtio_fs_complete_req_work+0x90/0x90 [ 84.642768][ C1] ? process_one_work+0xdb7/0x1410 [ 84.642770][ C1] ? lock_acquire+0x134/0x160 [ 84.642771][ C1] ? rcu_is_watching+0x15/0xd0 [ 84.642773][ C1] process_one_work+0xdf8/0x1410 [ 84.642776][ C1] ? pwq_dec_nr_in_flight+0x710/0x710 [ 84.642777][ C1] ? lock_acquire.part.0+0xbc/0x260 [ 84.642780][ C1] worker_thread+0x4f1/0xd60 [ 84.642782][ C1] ? rescuer_thread+0x1320/0x1320 [ 84.642784][ C1] ? __kthread_parkme+0xbd/0x210 [ 84.642786][ C1] ? rescuer_thread+0x1320/0x1320 [ 84.642788][ C1] kthread+0x367/0x460 [ 84.642790][ C1] ? trace_irq_enable.constprop.0+0x9b/0x180 [ 84.642793][ C1] ? kthread_affine_node+0x330/0x330 [ 84.642795][ C1] ret_from_fork+0x474/0x6b0 [ 84.642797][ C1] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 84.642799][ C1] ? __switch_to+0x5a3/0xe00 [ 84.642801][ C1] ? kthread_affine_node+0x330/0x330 [ 84.642803][ C1] ret_from_fork_asm+0x11/0x20 [ 84.642806][ C1]