[ 9.522131][ C0] clocksource: Watchdog remote CPU 1 read timed out [ 9.522166][ C0] [ 9.522168][ C0] ======================================================== [ 9.522169][ C0] WARNING: possible irq lock inversion dependency detected [ 9.522171][ C0] 7.2.0-virtme #1 Tainted: G W [ 9.522172][ C0] -------------------------------------------------------- [ 9.522173][ C0] kworker/0:1/10 just changed the state of lock: [ 9.522174][ C0] ffffffffa6c9d760 (console_owner){..-.}-{0:0}, at: console_lock_spinning_enable+0x40/0x60 [ 9.522186][ C0] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 9.522187][ C0] (fs_reclaim){+.+.}-{0:0} [ 9.522189][ C0] [ 9.522189][ C0] [ 9.522189][ C0] and interrupts could create inverse lock ordering between them. [ 9.522189][ C0] [ 9.522189][ C0] [ 9.522189][ C0] other info that might help us debug this: [ 9.522190][ C0] Possible interrupt unsafe locking scenario: [ 9.522190][ C0] [ 9.522191][ C0] CPU0 CPU1 [ 9.522191][ C0] ---- ---- [ 9.522191][ C0] lock(fs_reclaim); [ 9.522192][ C0] local_irq_disable(); [ 9.522193][ C0] lock(console_owner); [ 9.522194][ C0] lock(fs_reclaim); [ 9.522194][ C0] [ 9.522195][ C0] lock(console_owner); [ 9.522196][ C0] [ 9.522196][ C0] *** DEADLOCK *** [ 9.522196][ C0] [ 9.522196][ C0] locks held by kworker/0:1/10: 9, last CPU#0: [ 9.522197][ C0] #0: ff11000001151b40 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xd9a/0x1560 [ 9.522203][ C0] #1: ffa00000000a7d10 ((shepherd).work){+.+.}-{0:0}, at: process_one_work+0xdfc/0x1560 [ 9.522206][ C0] #2: ffffffffa6c0f8f0 (cpu_hotplug_lock){++++}-{0:0}, at: vmstat_shepherd+0x1e/0x300 [ 9.522211][ C0] #3: ffffffffa6d94c00 (rcu_read_lock){....}-{1:3}, at: vmstat_shepherd+0xc0/0x300 [ 9.522214][ C0] #4: ffffffffa6d94c00 (rcu_read_lock){....}-{1:3}, at: work_busy+0x4e/0x380 [ 9.522217][ C0] #5: ffa0000000007c90 ((&watchdog_timer)){+.-.}-{0:0}, at: call_timer_fn+0x110/0x4d0 [ 9.522222][ C0] #6: ffffffffa6de29f8 (watchdog_lock){+.-.}-{3:3}, at: clocksource_watchdog+0x15/0x30 [ 9.522226][ C0] #7: ffffffffa6d7ddc0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x320/0x3e0 [ 9.522228][ C0] #8: ffffffffa6d7de38 (console_srcu){....}-{0:0}, at: console_flush_one_record+0x106/0x710 [ 9.522232][ C0] [ 9.522232][ C0] the shortest dependencies between 2nd lock and 1st lock: [ 9.522235][ C0] -> (fs_reclaim){+.+.}-{0:0} { [ 9.522237][ C0] HARDIRQ-ON-W at: [ 9.522239][ C0] __lock_acquire+0x388/0xc20 [ 9.522242][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522243][ C0] fs_reclaim_acquire+0xd5/0x120 [ 9.522246][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 9.522248][ C0] kthread_create_worker_on_node+0xea/0x210 [ 9.522250][ C0] workqueue_init+0x2a/0x680 [ 9.522254][ C0] kernel_init_freeable+0x2fe/0x630 [ 9.522257][ C0] kernel_init+0x21/0x150 [ 9.522260][ C0] ret_from_fork+0x474/0x6b0 [ 9.522263][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522266][ C0] SOFTIRQ-ON-W at: [ 9.522267][ C0] __lock_acquire+0x388/0xc20 [ 9.522269][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522270][ C0] fs_reclaim_acquire+0xd5/0x120 [ 9.522271][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 9.522272][ C0] kthread_create_worker_on_node+0xea/0x210 [ 9.522274][ C0] workqueue_init+0x2a/0x680 [ 9.522275][ C0] kernel_init_freeable+0x2fe/0x630 [ 9.522277][ C0] kernel_init+0x21/0x150 [ 9.522278][ C0] ret_from_fork+0x474/0x6b0 [ 9.522279][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522280][ C0] INITIAL USE at: [ 9.522281][ C0] __lock_acquire+0x388/0xc20 [ 9.522283][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522284][ C0] fs_reclaim_acquire+0xd5/0x120 [ 9.522286][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 9.522287][ C0] kthread_create_worker_on_node+0xea/0x210 [ 9.522288][ C0] workqueue_init+0x2a/0x680 [ 9.522290][ C0] kernel_init_freeable+0x2fe/0x630 [ 9.522291][ C0] kernel_init+0x21/0x150 [ 9.522292][ C0] ret_from_fork+0x474/0x6b0 [ 9.522294][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522295][ C0] } [ 9.522295][ C0] ... key at: [] __fs_reclaim_map+0x0/0xbe0 [ 9.522299][ C0] ... acquired at: [ 9.522300][ C0] __lock_acquire+0x518/0xc20 [ 9.522301][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522302][ C0] fs_reclaim_acquire+0xd5/0x120 [ 9.522304][ C0] __kmalloc_noprof+0xd3/0x760 [ 9.522305][ C0] alloc_buf.isra.0+0x4b/0x260 [ 9.522308][ C0] put_chars+0x1e1/0x2f0 [ 9.522309][ C0] hvc_console_print+0x292/0x780 [ 9.522312][ C0] console_emit_next_record+0x22f/0x4c0 [ 9.522314][ C0] console_flush_one_record+0x46f/0x710 [ 9.522315][ C0] console_unlock+0xee/0x1f0 [ 9.522317][ C0] vprintk_emit+0x37c/0x3e0 [ 9.522318][ C0] _printk+0xc7/0x100 [ 9.522321][ C0] tun_init+0x24/0xb0 [ 9.522323][ C0] do_one_initcall+0x124/0x4f0 [ 9.522324][ C0] kernel_init_freeable+0x596/0x630 [ 9.522326][ C0] kernel_init+0x21/0x150 [ 9.522327][ C0] ret_from_fork+0x474/0x6b0 [ 9.522328][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522329][ C0] [ 9.522330][ C0] -> (console_owner){..-.}-{0:0} { [ 9.522331][ C0] IN-SOFTIRQ-W at: [ 9.522332][ C0] __lock_acquire+0x388/0xc20 [ 9.522334][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522335][ C0] console_lock_spinning_enable+0x5c/0x60 [ 9.522337][ C0] console_emit_next_record+0x1d1/0x4c0 [ 9.522338][ C0] console_flush_one_record+0x46f/0x710 [ 9.522340][ C0] console_unlock+0xee/0x1f0 [ 9.522341][ C0] vprintk_emit+0x37c/0x3e0 [ 9.522342][ C0] _printk+0xc7/0x100 [ 9.522343][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 9.522346][ C0] call_timer_fn+0x160/0x4d0 [ 9.522347][ C0] __run_timers+0x68f/0xaa0 [ 9.522349][ C0] run_timer_softirq+0xf0/0x160 [ 9.522350][ C0] handle_softirqs+0x1d3/0x900 [ 9.522353][ C0] __irq_exit_rcu+0x145/0x1c0 [ 9.522354][ C0] irq_exit_rcu+0xe/0x30 [ 9.522356][ C0] common_interrupt+0xb5/0xf0 [ 9.522357][ C0] asm_common_interrupt+0x26/0x40 [ 9.522358][ C0] lock_acquire.part.0+0x112/0x280 [ 9.522360][ C0] work_busy+0x65/0x380 [ 9.522361][ C0] vmstat_shepherd+0x1ab/0x300 [ 9.522363][ C0] process_one_work+0xe3e/0x1560 [ 9.522364][ C0] worker_thread+0x4f1/0xd60 [ 9.522366][ C0] kthread+0x367/0x460 [ 9.522368][ C0] ret_from_fork+0x474/0x6b0 [ 9.522369][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522370][ C0] INITIAL USE at: [ 9.522371][ C0] } [ 9.522371][ C0] ... key at: [] console_owner_dep_map+0x0/0x60 [ 9.522373][ C0] ... acquired at: [ 9.522374][ C0] mark_lock+0x1d7/0xa00 [ 9.522375][ C0] mark_usage+0x42/0x170 [ 9.522376][ C0] __lock_acquire+0x388/0xc20 [ 9.522378][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522379][ C0] console_lock_spinning_enable+0x5c/0x60 [ 9.522381][ C0] console_emit_next_record+0x1d1/0x4c0 [ 9.522382][ C0] console_flush_one_record+0x46f/0x710 [ 9.522384][ C0] console_unlock+0xee/0x1f0 [ 9.522385][ C0] vprintk_emit+0x37c/0x3e0 [ 9.522386][ C0] _printk+0xc7/0x100 [ 9.522387][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 9.522388][ C0] call_timer_fn+0x160/0x4d0 [ 9.522390][ C0] __run_timers+0x68f/0xaa0 [ 9.522391][ C0] run_timer_softirq+0xf0/0x160 [ 9.522393][ C0] handle_softirqs+0x1d3/0x900 [ 9.522394][ C0] __irq_exit_rcu+0x145/0x1c0 [ 9.522395][ C0] irq_exit_rcu+0xe/0x30 [ 9.522396][ C0] common_interrupt+0xb5/0xf0 [ 9.522397][ C0] asm_common_interrupt+0x26/0x40 [ 9.522398][ C0] lock_acquire.part.0+0x112/0x280 [ 9.522399][ C0] work_busy+0x65/0x380 [ 9.522401][ C0] vmstat_shepherd+0x1ab/0x300 [ 9.522402][ C0] process_one_work+0xe3e/0x1560 [ 9.522404][ C0] worker_thread+0x4f1/0xd60 [ 9.522405][ C0] kthread+0x367/0x460 [ 9.522407][ C0] ret_from_fork+0x474/0x6b0 [ 9.522408][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522409][ C0] [ 9.522409][ C0] [ 9.522409][ C0] stack backtrace: [ 9.522412][ C0] CPU: 0 UID: 0 PID: 10 Comm: kworker/0:1 Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 9.522415][ C0] Tainted: [W]=WARN [ 9.522416][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 9.522417][ C0] Workqueue: events vmstat_shepherd [ 9.522421][ C0] Call Trace: [ 9.522422][ C0] [ 9.522423][ C0] dump_stack_lvl+0x6f/0xa0 [ 9.522428][ C0] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 9.522430][ C0] mark_lock_irq+0x989/0x9c0 [ 9.522432][ C0] ? add_lock_to_list+0x2c/0x1a0 [ 9.522435][ C0] mark_lock+0x1d7/0xa00 [ 9.522436][ C0] mark_usage+0x42/0x170 [ 9.522438][ C0] __lock_acquire+0x388/0xc20 [ 9.522440][ C0] lock_acquire.part.0+0xd4/0x280 [ 9.522442][ C0] ? console_lock_spinning_enable+0x40/0x60 [ 9.522444][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522446][ C0] ? lock_acquire+0x13c/0x160 [ 9.522448][ C0] console_lock_spinning_enable+0x5c/0x60 [ 9.522450][ C0] ? console_lock_spinning_enable+0x40/0x60 [ 9.522451][ C0] console_emit_next_record+0x1d1/0x4c0 [ 9.522454][ C0] ? devkmsg_read+0x4b0/0x4b0 [ 9.522455][ C0] ? console_flush_one_record+0x106/0x710 [ 9.522457][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522458][ C0] ? lock_acquire+0x13c/0x160 [ 9.522460][ C0] console_flush_one_record+0x46f/0x710 [ 9.522462][ C0] ? console_emit_next_record+0x4c0/0x4c0 [ 9.522464][ C0] ? __lock_acquire+0x518/0xc20 [ 9.522466][ C0] console_unlock+0xee/0x1f0 [ 9.522468][ C0] ? console_flush_one_record+0x710/0x710 [ 9.522470][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522471][ C0] ? lock_acquire+0xe0/0x160 [ 9.522473][ C0] ? __down_trylock_console_sem+0x5e/0xa0 [ 9.522475][ C0] ? vprintk_emit+0x320/0x3e0 [ 9.522476][ C0] vprintk_emit+0x37c/0x3e0 [ 9.522478][ C0] ? wake_up_klogd_work_func+0x90/0x90 [ 9.522479][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 9.522481][ C0] _printk+0xc7/0x100 [ 9.522483][ C0] ? snapshot_read.cold+0x21/0x21 [ 9.522484][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 9.522486][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 9.522488][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 9.522490][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522492][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522493][ C0] call_timer_fn+0x160/0x4d0 [ 9.522495][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 9.522497][ C0] ? debug_object_active_state+0x430/0x430 [ 9.522500][ C0] ? find_held_lock+0x2b/0x80 [ 9.522501][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 9.522503][ C0] ? mark_held_locks+0x40/0x70 [ 9.522505][ C0] __run_timers+0x68f/0xaa0 [ 9.522507][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522509][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 9.522511][ C0] ? __lock_acquire+0x518/0xc20 [ 9.522513][ C0] ? __rwlock_init+0x150/0x150 [ 9.522516][ C0] run_timer_softirq+0xf0/0x160 [ 9.522518][ C0] ? __run_timers+0xaa0/0xaa0 [ 9.522519][ C0] ? handle_softirqs+0x19c/0x900 [ 9.522521][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522522][ C0] handle_softirqs+0x1d3/0x900 [ 9.522524][ C0] ? _local_bh_enable+0xc0/0xc0 [ 9.522525][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 9.522528][ C0] __irq_exit_rcu+0x145/0x1c0 [ 9.522529][ C0] irq_exit_rcu+0xe/0x30 [ 9.522531][ C0] common_interrupt+0xb5/0xf0 [ 9.522532][ C0] [ 9.522532][ C0] [ 9.522533][ C0] asm_common_interrupt+0x26/0x40 [ 9.522534][ C0] RIP: 0010:lock_acquire.part.0+0x112/0x280 [ 9.522537][ C0] Code: ff 65 0f c1 05 7f 77 ed 04 83 f8 01 4c 8b 14 24 0f 85 35 01 00 00 9c 58 f6 c4 02 0f 85 52 01 00 00 4d 85 d2 0f 85 1b 01 00 00 <48> 8b 44 24 30 65 48 2b 05 51 32 ed 04 0f 85 48 01 00 00 48 83 c4 [ 9.522538][ C0] RSP: 0018:ffa00000000a7b48 EFLAGS: 00000206 [ 9.522540][ C0] RAX: 0000000000000046 RBX: 0000000000000000 RCX: e07592054855beb8 [ 9.522541][ C0] RDX: 0000000000000000 RSI: ffffffffa62f9f03 RDI: ffffffffa5c949e0 [ 9.522542][ C0] RBP: 0000000000000000 R08: 0000000000000001 R09: ff11000001b60bf0 [ 9.522543][ C0] R10: 0000000000000200 R11: 0000000000000001 R12: ffffffffa6d94c00 [ 9.522544][ C0] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002 [ 9.522546][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522547][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 9.522549][ C0] ? lock_acquire+0x13c/0x160 [ 9.522550][ C0] work_busy+0x65/0x380 [ 9.522552][ C0] ? work_busy+0x4e/0x380 [ 9.522554][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522555][ C0] vmstat_shepherd+0x1ab/0x300 [ 9.522557][ C0] ? process_one_work+0xe16/0x1560 [ 9.522559][ C0] process_one_work+0xe3e/0x1560 [ 9.522561][ C0] ? __queue_delayed_work+0x470/0x470 [ 9.522563][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 9.522566][ C0] worker_thread+0x4f1/0xd60 [ 9.522568][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522570][ C0] ? __kthread_parkme+0xbd/0x210 [ 9.522572][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522573][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522575][ C0] kthread+0x367/0x460 [ 9.522577][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 9.522579][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 9.522581][ C0] ret_from_fork+0x474/0x6b0 [ 9.522583][ C0] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 9.522585][ C0] ? __switch_to+0x5a3/0xe00 [ 9.522587][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 9.522589][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522592][ C0] [ 9.522596][ C0] BUG: sleeping function called from invalid context at ./include/linux/sched/mm.h:322 [ 9.522598][ C0] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 10, name: kworker/0:1 [ 9.522599][ C0] preempt_count: 102, expected: 0 [ 9.522600][ C0] RCU nest depth: 2, expected: 0 [ 9.522601][ C0] INFO: lockdep is turned off. [ 9.522601][ C0] irq event stamp: 19801 [ 9.522602][ C0] hardirqs last enabled at (19800): [] __down_trylock_console_sem+0x86/0xa0 [ 9.522604][ C0] hardirqs last disabled at (19801): [] console_emit_next_record+0x3d4/0x4c0 [ 9.522606][ C0] softirqs last enabled at (18712): [] handle_softirqs+0x67c/0x900 [ 9.522607][ C0] softirqs last disabled at (18719): [] __irq_exit_rcu+0x145/0x1c0 [ 9.522609][ C0] Preemption disabled at: [ 9.522609][ C0] [] handle_softirqs+0xf6/0x900 [ 9.522612][ C0] CPU: 0 UID: 0 PID: 10 Comm: kworker/0:1 Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 9.522614][ C0] Tainted: [W]=WARN [ 9.522614][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 9.522615][ C0] Workqueue: events vmstat_shepherd [ 9.522617][ C0] Call Trace: [ 9.522617][ C0] [ 9.522618][ C0] dump_stack_lvl+0x6f/0xa0 [ 9.522620][ C0] ? handle_softirqs+0xf6/0x900 [ 9.522622][ C0] __might_resched.cold+0x1fe/0x2c1 [ 9.522624][ C0] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 9.522627][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522628][ C0] __kmalloc_noprof+0x443/0x760 [ 9.522630][ C0] ? __rwlock_init+0x150/0x150 [ 9.522631][ C0] ? alloc_buf.isra.0+0x4b/0x260 [ 9.522633][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 9.522635][ C0] alloc_buf.isra.0+0x4b/0x260 [ 9.522637][ C0] put_chars+0x1e1/0x2f0 [ 9.522639][ C0] ? __send_to_port+0x420/0x420 [ 9.522640][ C0] ? mark_lock+0x1d7/0xa00 [ 9.522642][ C0] hvc_console_print+0x292/0x780 [ 9.522645][ C0] ? hvc_write+0x3a0/0x3a0 [ 9.522647][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522648][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522650][ C0] console_emit_next_record+0x22f/0x4c0 [ 9.522652][ C0] ? devkmsg_read+0x4b0/0x4b0 [ 9.522654][ C0] ? console_flush_one_record+0x106/0x710 [ 9.522656][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522657][ C0] ? lock_acquire+0x13c/0x160 [ 9.522658][ C0] console_flush_one_record+0x46f/0x710 [ 9.522661][ C0] ? console_emit_next_record+0x4c0/0x4c0 [ 9.522662][ C0] ? __lock_acquire+0x518/0xc20 [ 9.522665][ C0] console_unlock+0xee/0x1f0 [ 9.522666][ C0] ? console_flush_one_record+0x710/0x710 [ 9.522668][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522669][ C0] ? lock_acquire+0xe0/0x160 [ 9.522671][ C0] ? __down_trylock_console_sem+0x5e/0xa0 [ 9.522673][ C0] ? vprintk_emit+0x320/0x3e0 [ 9.522674][ C0] vprintk_emit+0x37c/0x3e0 [ 9.522675][ C0] ? wake_up_klogd_work_func+0x90/0x90 [ 9.522677][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 9.522679][ C0] _printk+0xc7/0x100 [ 9.522680][ C0] ? snapshot_read.cold+0x21/0x21 [ 9.522682][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 9.522684][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 9.522686][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 9.522687][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522689][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522691][ C0] call_timer_fn+0x160/0x4d0 [ 9.522693][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 9.522694][ C0] ? debug_object_active_state+0x430/0x430 [ 9.522696][ C0] ? find_held_lock+0x2b/0x80 [ 9.522697][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 9.522699][ C0] ? mark_held_locks+0x40/0x70 [ 9.522701][ C0] __run_timers+0x68f/0xaa0 [ 9.522703][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 9.522705][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 9.522707][ C0] ? __lock_acquire+0x518/0xc20 [ 9.522709][ C0] ? __rwlock_init+0x150/0x150 [ 9.522711][ C0] run_timer_softirq+0xf0/0x160 [ 9.522713][ C0] ? __run_timers+0xaa0/0xaa0 [ 9.522715][ C0] ? handle_softirqs+0x19c/0x900 [ 9.522717][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522718][ C0] handle_softirqs+0x1d3/0x900 [ 9.522720][ C0] ? _local_bh_enable+0xc0/0xc0 [ 9.522721][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 9.522723][ C0] __irq_exit_rcu+0x145/0x1c0 [ 9.522725][ C0] irq_exit_rcu+0xe/0x30 [ 9.522726][ C0] common_interrupt+0xb5/0xf0 [ 9.522727][ C0] [ 9.522728][ C0] [ 9.522728][ C0] asm_common_interrupt+0x26/0x40 [ 9.522730][ C0] RIP: 0010:lock_acquire.part.0+0x112/0x280 [ 9.522731][ C0] Code: ff 65 0f c1 05 7f 77 ed 04 83 f8 01 4c 8b 14 24 0f 85 35 01 00 00 9c 58 f6 c4 02 0f 85 52 01 00 00 4d 85 d2 0f 85 1b 01 00 00 <48> 8b 44 24 30 65 48 2b 05 51 32 ed 04 0f 85 48 01 00 00 48 83 c4 [ 9.522732][ C0] RSP: 0018:ffa00000000a7b48 EFLAGS: 00000206 [ 9.522733][ C0] RAX: 0000000000000046 RBX: 0000000000000000 RCX: e07592054855beb8 [ 9.522734][ C0] RDX: 0000000000000000 RSI: ffffffffa62f9f03 RDI: ffffffffa5c949e0 [ 9.522735][ C0] RBP: 0000000000000000 R08: 0000000000000001 R09: ff11000001b60bf0 [ 9.522736][ C0] R10: 0000000000000200 R11: 0000000000000001 R12: ffffffffa6d94c00 [ 9.522736][ C0] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002 [ 9.522738][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522740][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 9.522741][ C0] ? lock_acquire+0x13c/0x160 [ 9.522743][ C0] work_busy+0x65/0x380 [ 9.522745][ C0] ? work_busy+0x4e/0x380 [ 9.522746][ C0] ? rcu_is_watching+0x16/0xd0 [ 9.522748][ C0] vmstat_shepherd+0x1ab/0x300 [ 9.522750][ C0] ? process_one_work+0xe16/0x1560 [ 9.522751][ C0] process_one_work+0xe3e/0x1560 [ 9.522754][ C0] ? __queue_delayed_work+0x470/0x470 [ 9.522756][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 9.522759][ C0] worker_thread+0x4f1/0xd60 [ 9.522761][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522762][ C0] ? __kthread_parkme+0xbd/0x210 [ 9.522764][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522766][ C0] ? rescuer_thread+0x1340/0x1340 [ 9.522767][ C0] kthread+0x367/0x460 [ 9.522769][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 9.522770][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 9.522772][ C0] ret_from_fork+0x474/0x6b0 [ 9.522774][ C0] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 9.522776][ C0] ? __switch_to+0x5a3/0xe00 [ 9.522777][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 9.522779][ C0] ret_from_fork_asm+0x11/0x20 [ 9.522782][ C0] [ 10.522077][ C0] BUG: sleeping function called from invalid context at ./include/linux/sched/mm.h:322 [ 10.522081][ C0] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 10, name: kworker/0:1 [ 10.522083][ C0] preempt_count: 102, expected: 0 [ 10.522083][ C0] RCU nest depth: 2, expected: 0 [ 10.522084][ C0] INFO: lockdep is turned off. [ 10.522085][ C0] irq event stamp: 19801 [ 10.522086][ C0] hardirqs last enabled at (19800): [] __down_trylock_console_sem+0x86/0xa0 [ 10.522094][ C0] hardirqs last disabled at (19801): [] console_emit_next_record+0x3d4/0x4c0 [ 10.522097][ C0] softirqs last enabled at (18712): [] handle_softirqs+0x67c/0x900 [ 10.522100][ C0] softirqs last disabled at (18719): [] __irq_exit_rcu+0x145/0x1c0 [ 10.522102][ C0] Preemption disabled at: [ 10.522103][ C0] [] handle_softirqs+0xf6/0x900 [ 10.522108][ C0] CPU: 0 UID: 0 PID: 10 Comm: kworker/0:1 Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 10.522112][ C0] Tainted: [W]=WARN [ 10.522113][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 10.522114][ C0] Workqueue: events vmstat_shepherd [ 10.522120][ C0] Call Trace: [ 10.522121][ C0] [ 10.522123][ C0] dump_stack_lvl+0x6f/0xa0 [ 10.522128][ C0] ? handle_softirqs+0xf6/0x900 [ 10.522130][ C0] __might_resched.cold+0x1fe/0x2c1 [ 10.522133][ C0] ? prb_next_reserve_seq+0x2d0/0x2d0 [ 10.522135][ C0] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 10.522138][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522141][ C0] __kmalloc_noprof+0x443/0x760 [ 10.522143][ C0] ? __rwlock_init+0x150/0x150 [ 10.522145][ C0] ? alloc_buf.isra.0+0x4b/0x260 [ 10.522150][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 10.522152][ C0] alloc_buf.isra.0+0x4b/0x260 [ 10.522154][ C0] put_chars+0x1e1/0x2f0 [ 10.522155][ C0] ? __send_to_port+0x420/0x420 [ 10.522156][ C0] ? console_prepend_replay+0x20/0x20 [ 10.522159][ C0] hvc_console_print+0x292/0x780 [ 10.522163][ C0] ? hvc_write+0x3a0/0x3a0 [ 10.522165][ C0] ? __rwlock_init+0x150/0x150 [ 10.522166][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522167][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522169][ C0] console_emit_next_record+0x22f/0x4c0 [ 10.522171][ C0] ? devkmsg_read+0x4b0/0x4b0 [ 10.522173][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522174][ C0] ? lock_acquire+0x13c/0x160 [ 10.522176][ C0] ? console_flush_one_record+0x111/0x710 [ 10.522178][ C0] console_flush_one_record+0x46f/0x710 [ 10.522180][ C0] ? console_emit_next_record+0x4c0/0x4c0 [ 10.522183][ C0] console_unlock+0xee/0x1f0 [ 10.522185][ C0] ? console_flush_one_record+0x710/0x710 [ 10.522186][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522187][ C0] ? lock_acquire+0xe0/0x160 [ 10.522189][ C0] ? __down_trylock_console_sem+0x5e/0xa0 [ 10.522191][ C0] ? vprintk_emit+0x320/0x3e0 [ 10.522192][ C0] vprintk_emit+0x37c/0x3e0 [ 10.522194][ C0] ? wake_up_klogd_work_func+0x90/0x90 [ 10.522196][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 10.522199][ C0] _printk+0xc7/0x100 [ 10.522201][ C0] ? snapshot_read.cold+0x21/0x21 [ 10.522203][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 10.522204][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 10.522207][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 10.522210][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 10.522211][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 10.522213][ C0] call_timer_fn+0x160/0x4d0 [ 10.522216][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 10.522217][ C0] ? debug_object_active_state+0x430/0x430 [ 10.522220][ C0] ? find_held_lock+0x2b/0x80 [ 10.522222][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 10.522224][ C0] ? mark_held_locks+0x40/0x70 [ 10.522226][ C0] __run_timers+0x68f/0xaa0 [ 10.522227][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 10.522230][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 10.522231][ C0] ? __lock_acquire+0x518/0xc20 [ 10.522234][ C0] ? __rwlock_init+0x150/0x150 [ 10.522236][ C0] run_timer_softirq+0xf0/0x160 [ 10.522238][ C0] ? __run_timers+0xaa0/0xaa0 [ 10.522240][ C0] ? handle_softirqs+0x19c/0x900 [ 10.522241][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522243][ C0] handle_softirqs+0x1d3/0x900 [ 10.522244][ C0] ? _local_bh_enable+0xc0/0xc0 [ 10.522246][ C0] ? do_raw_spin_unlock+0x59/0x250 [ 10.522248][ C0] __irq_exit_rcu+0x145/0x1c0 [ 10.522250][ C0] irq_exit_rcu+0xe/0x30 [ 10.522251][ C0] common_interrupt+0xb5/0xf0 [ 10.522254][ C0] [ 10.522255][ C0] [ 10.522255][ C0] asm_common_interrupt+0x26/0x40 [ 10.522257][ C0] RIP: 0010:lock_acquire.part.0+0x112/0x280 [ 10.522260][ C0] Code: ff 65 0f c1 05 7f 77 ed 04 83 f8 01 4c 8b 14 24 0f 85 35 01 00 00 9c 58 f6 c4 02 0f 85 52 01 00 00 4d 85 d2 0f 85 1b 01 00 00 <48> 8b 44 24 30 65 48 2b 05 51 32 ed 04 0f 85 48 01 00 00 48 83 c4 [ 10.522261][ C0] RSP: 0018:ffa00000000a7b48 EFLAGS: 00000206 [ 10.522263][ C0] RAX: 0000000000000046 RBX: 0000000000000000 RCX: e07592054855beb8 [ 10.522264][ C0] RDX: 0000000000000000 RSI: ffffffffa62f9f03 RDI: ffffffffa5c949e0 [ 10.522265][ C0] RBP: 0000000000000000 R08: 0000000000000001 R09: ff11000001b60bf0 [ 10.522266][ C0] R10: 0000000000000200 R11: 0000000000000001 R12: ffffffffa6d94c00 [ 10.522267][ C0] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002 [ 10.522269][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522270][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 10.522272][ C0] ? lock_acquire+0x13c/0x160 [ 10.522273][ C0] work_busy+0x65/0x380 [ 10.522276][ C0] ? work_busy+0x4e/0x380 [ 10.522278][ C0] ? rcu_is_watching+0x16/0xd0 [ 10.522279][ C0] vmstat_shepherd+0x1ab/0x300 [ 10.522281][ C0] ? process_one_work+0xe16/0x1560 [ 10.522283][ C0] process_one_work+0xe3e/0x1560 [ 10.522286][ C0] ? __queue_delayed_work+0x470/0x470 [ 10.522288][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 10.522291][ C0] worker_thread+0x4f1/0xd60 [ 10.522293][ C0] ? rescuer_thread+0x1340/0x1340 [ 10.522294][ C0] ? __kthread_parkme+0xbd/0x210 [ 10.522296][ C0] ? rescuer_thread+0x1340/0x1340 [ 10.522298][ C0] ? rescuer_thread+0x1340/0x1340 [ 10.522300][ C0] kthread+0x367/0x460 [ 10.522302][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 10.522304][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 10.522306][ C0] ret_from_fork+0x474/0x6b0 [ 10.522309][ C0] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 10.522311][ C0] ? __switch_to+0x5a3/0xe00 [ 10.522314][ C0] ? kthread_affine_preferred+0x4c0/0x4c0 [ 10.522316][ C0] ret_from_fork_asm+0x11/0x20 [ 10.522319][ C0]