virtme: waiting for virtiofsd to start virtme: use 'microvm' QEMU architecture [ 1.187091][ T1] loop: module loaded [ 1.187128][ T1] BUG: sleeping function called from invalid context at ./include/linux/sched/mm.h:322 [ 1.187130][ T1] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 1, name: swapper/0 [ 1.187132][ T1] preempt_count: 1, expected: 0 [ 1.187133][ T1] RCU nest depth: 0, expected: 0 [ 1.187135][ T1] locks held by swapper/0/1: 4, last CPU#3: [ 1.187137][ T1] #0: ffffffff86b69cc0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x320/0x3e0 [ 1.187152][ T1] #1: ffffffff86b69d38 (console_srcu){....}-{0:0}, at: console_flush_one_record+0x106/0x710 [ 1.187158][ T1] #2: ffffffff86a89660 (console_owner){....}-{0:0}, at: console_lock_spinning_enable+0x40/0x60 [ 1.187163][ T1] #3: ffffffff86a89560 (printk_legacy_map-wait-type-override){....}-{3:3}, at: console_emit_next_record+0x1df/0x4c0 [ 1.187169][ T1] irq event stamp: 289758 [ 1.187170][ T1] hardirqs last enabled at (289757): [] __down_trylock_console_sem+0x86/0xa0 [ 1.187174][ T1] hardirqs last disabled at (289758): [] console_emit_next_record+0x3d4/0x4c0 [ 1.187177][ T1] softirqs last enabled at (289704): [] bdi_register_va+0x491/0x780 [ 1.187182][ T1] softirqs last disabled at (289702): [] bdi_register_va+0x2ef/0x780 [ 1.187186][ T1] Preemption disabled at: [ 1.187187][ T1] [] vprintk_emit+0x31b/0x3e0 [ 1.187193][ T1] CPU: 3 UID: 0 PID: 1 Comm: swapper/0 Not tainted 7.2.0-virtme #1 PREEMPT(full) [ 1.187196][ T1] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1.187199][ T1] Call Trace: [ 1.187201][ T1] [ 1.187205][ T1] dump_stack_lvl+0x6f/0xa0 [ 1.187211][ T1] ? vprintk_emit+0x31b/0x3e0 [ 1.187214][ T1] __might_resched.cold+0x1fe/0x2c1 [ 1.187220][ T1] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 1.187224][ T1] ? __kmalloc_noprof+0xdb/0x760 [ 1.187232][ T1] __kmalloc_noprof+0x443/0x760 [ 1.187236][ T1] ? alloc_buf.isra.0+0x4b/0x260 [ 1.187245][ T1] ? do_raw_spin_unlock+0x59/0x250 [ 1.187248][ T1] alloc_buf.isra.0+0x4b/0x260 [ 1.187253][ T1] put_chars+0x1e1/0x2f0 [ 1.187256][ T1] ? desc_read_finalized_seq+0x79/0x120 [ 1.187260][ T1] ? __send_to_port+0x420/0x420 [ 1.187265][ T1] ? rcu_read_lock_any_held+0x3c/0x90 [ 1.187270][ T1] ? validate_chain+0x38b/0xc20 [ 1.187278][ T1] hvc_console_print+0x292/0x780 [ 1.187282][ T1] ? __lock_acquire+0x518/0xc20 [ 1.187284][ T1] ? __lock_acquire+0x518/0xc20 [ 1.187292][ T1] ? hvc_write+0x3a0/0x3a0 [ 1.187295][ T1] ? console_emit_next_record+0x1df/0x4c0 [ 1.187299][ T1] ? rcu_is_watching+0x16/0xd0 [ 1.187303][ T1] ? lock_acquire+0x13c/0x160 [ 1.187309][ T1] console_emit_next_record+0x22f/0x4c0 [ 1.187314][ T1] ? devkmsg_read+0x4b0/0x4b0 [ 1.187316][ T1] ? console_flush_one_record+0x106/0x710 [ 1.187321][ T1] ? rcu_is_watching+0x16/0xd0 [ 1.187324][ T1] ? lock_acquire+0x13c/0x160 [ 1.187331][ T1] console_flush_one_record+0x46f/0x710 [ 1.187337][ T1] ? console_emit_next_record+0x4c0/0x4c0 [ 1.187339][ T1] ? __lock_acquire+0x518/0xc20 [ 1.187348][ T1] console_unlock+0xee/0x1f0 [ 1.187352][ T1] ? console_flush_one_record+0x710/0x710 [ 1.187354][ T1] ? rcu_is_watching+0x16/0xd0 [ 1.187357][ T1] ? lock_acquire+0x60/0x160 [ 1.187365][ T1] ? __down_trylock_console_sem+0x5e/0xa0 [ 1.187366][ T1] ? vprintk_emit+0x320/0x3e0 [ 1.187366][ T1] vprintk_emit+0x37c/0x3e0 [ 1.187366][ T1] ? wake_up_klogd_work_func+0x90/0x90 [ 1.187366][ T1] ? max_loop_setup+0x30/0x30 [ 1.187366][ T1] _printk+0xc7/0x100 [ 1.187366][ T1] ? snapshot_read.cold+0x21/0x21 [ 1.187366][ T1] ? __mutex_unlock_slowpath+0x14d/0x740 [ 1.187366][ T1] loop_init+0x12a/0x130 [ 1.187366][ T1] do_one_initcall+0x124/0x4f0 [ 1.187366][ T1] ? trace_event_raw_event_initcall_level+0x280/0x280 [ 1.187366][ T1] ? parameq+0x110/0x110 [ 1.187366][ T1] ? kernel_init_freeable+0x3f1/0x630 [ 1.187366][ T1] ? rcu_is_watching+0x16/0xd0 [ 1.187366][ T1] kernel_init_freeable+0x596/0x630 [ 1.187366][ T1] ? rest_init+0x280/0x280 [ 1.187366][ T1] kernel_init+0x21/0x150 [ 1.187366][ T1] ? rest_init+0x280/0x280 [ 1.187366][ T1] ret_from_fork+0x474/0x6b0 [ 1.187366][ T1] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 1.187366][ T1] ? __switch_to+0x5a3/0xe00 [ 1.187366][ T1] ? rest_init+0x280/0x280 [ 1.187366][ T1] ret_from_fork_asm+0x11/0x20 [ 1.187366][ T1] [ 1.228240][ T1] tun: Universal TUN/TAP device driver, 1.6 [ 1.230427][ T1] i8042: PNP: No PS/2 controller found. [ 1.241544][ T1] rtc_cmos PNP0B00:00: registered as rtc0 [ 1.242098][ T1] rtc_cmos PNP0B00:00: setting system clock to 2026-08-28T12:56:47 UTC (1787921807) [ 1.243539][ T1] rtc_cmos PNP0B00:00: alarms up to one day, 242 bytes nvram [ 1.248010][ T1] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev [ 1.258858][ T1] NET: Registered PF_INET6 protocol family [ 1.265013][ T1] Segment Routing with IPv6 [ 1.265647][ T1] In-situ OAM (IOAM) with IPv6 [ 1.266214][ T1] NET: Registered PF_PACKET protocol family [ 1.266721][ T1] 9pnet: Installing 9P2000 support [ 1.267268][ T1] Key type dns_resolver registered [ 1.268278][ T1] NET: Registered PF_VSOCK protocol family [ 1.273706][ T1] IPI shorthand broadcast: enabled [ 1.387967][ T1] sched_clock: Marking stable (1350001818, 37363129)->(1485644943, -98279996) [ 1.390298][ T1] registered taskstats version 1 [ 1.392682][ T1] Loading compiled-in X.509 certificates [ 1.463968][ T72] kwatchdog (72) used greatest stack depth: 29688 bytes left [ 1.491054][ T1] Demotion targets for Node 0: null [ 1.491330][ T1] kmemleak: Kernel memory leak detector initialized (mem pool available: 11733) [ 1.491636][ T1] page_owner is disabled [ 1.492233][ T1] PM: Magic number: 6:281:937 [ 1.492657][ T1] netconsole: network logging started [ 1.494007][ T1] ALSA device list: [ 1.494681][ T1] No soundcards found. [ 1.495757][ T1] md: Skipping autodetection of RAID arrays. (raid=autodetect will force) [ 1.498246][ T1] VFS: Mounted root (virtiofs filesystem) readonly on device 0:24. [ 1.499060][ T1] devtmpfs: mounted [ 1.499360][ T1] VFS: Pivoted into new rootfs [ 1.525187][ T1] Freeing unused kernel image (initmem) memory: 2536K [ 1.527965][ T1] Write protecting the kernel read-only data: 55296k [ 1.528811][ T1] Freeing unused kernel image (text/rodata gap) memory: 660K [ 1.529243][ T1] Freeing unused kernel image (rodata/data gap) memory: 624K [ 1.529661][ T1] Run /usr/lib/python3.14/site-packages/virtme/guest/bin/virtme-ng-init as init process [ 1.529928][ T1] with arguments: [ 1.530050][ T1] /usr/lib/python3.14/site-packages/virtme/guest/bin/virtme-ng-init [ 1.530291][ T1] with environment: [ 1.531785][ T1] HOME=/ [ 1.531907][ T1] TERM=dumb [ 1.532032][ T1] virtme_hostname=vmksft-packetdrill-dbg,debug-threads=on [ 1.532261][ T1] nr_open=2147483584 [ 1.532382][ T1] virtme_link_mods=/srv/vmksft/testing/wt-8/.virtme_mods/lib/modules/0.0.0 [ 1.532651][ T1] virtme_rw_overlay0=/etc [ 1.532800][ T1] virtme_rw_overlay1=/lib [ 1.532959][ T1] virtme_rw_overlay2=/home [ 1.533112][ T1] virtme_rw_overlay3=/opt [ 1.533264][ T1] virtme_rw_overlay4=/srv [ 1.533422][ T1] virtme_rw_overlay5=/usr [ 1.533576][ T1] virtme_rw_overlay6=/var [ 1.533725][ T1] virtme_rw_overlay7=/tmp [ 1.533884][ T1] virtme_console=ttyS0 [ 1.534032][ T1] virtme_chdir=srv/vmksft/testing/wt-8 [ 1.549502][ T1] virtme-ng-init: mount devtmpfs -> /dev: EBUSY: Device or resource busy [ 1.550708][ T1] virtme-ng-init: Setting hostname to vmksft-packetdrill-dbg,debug-threads=on... [ 1.561815][ T1] overlayfs: failed to set xattr on upper [ 1.562070][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.562344][ T1] overlayfs: ...falling back to uuid=null. [ 1.564587][ T1] overlayfs: failed to set xattr on upper [ 1.564793][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.565030][ T1] overlayfs: ...falling back to uuid=null. [ 1.567096][ T1] overlayfs: failed to set xattr on upper [ 1.567304][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.567552][ T1] overlayfs: ...falling back to uuid=null. [ 1.569350][ T1] overlayfs: failed to set xattr on upper [ 1.569628][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.569872][ T1] overlayfs: ...falling back to uuid=null. [ 1.571666][ T1] overlayfs: failed to set xattr on upper [ 1.571874][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.572118][ T1] overlayfs: ...falling back to uuid=null. [ 1.573727][ T1] overlayfs: failed to set xattr on upper [ 1.573932][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.574176][ T1] overlayfs: ...falling back to uuid=null. [ 1.575945][ T1] overlayfs: failed to set xattr on upper [ 1.576150][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.576397][ T1] overlayfs: ...falling back to uuid=null. [ 1.578224][ T1] overlayfs: failed to set xattr on upper [ 1.578871][ T1] overlayfs: ...falling back to redirect_dir=nofollow. [ 1.579118][ T1] overlayfs: ...falling back to uuid=null. [ 1.590449][ T1] virtme-ng-init: running systemd-tmpfiles [ 4.011701][ T74] systemd-tmpfile (74) used greatest stack depth: 24384 bytes left [ 4.011721][ T74] BUG: sleeping function called from invalid context at ./include/linux/sched/mm.h:322 [ 4.011723][ T74] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 74, name: systemd-tmpfile [ 4.011726][ T74] preempt_count: 2, expected: 0 [ 4.011727][ T74] RCU nest depth: 0, expected: 0 [ 4.011728][ T74] locks held by systemd-tmpfile/74: 5, last CPU#2: [ 4.011731][ T74] #0: ffffffff86a027b8 (low_water_lock){+.+.}-{3:3}, at: do_exit+0x887/0xdc0 [ 4.011745][ T74] #1: ffffffff86b69cc0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x320/0x3e0 [ 4.011752][ T74] #2: ffffffff86b69d38 (console_srcu){....}-{0:0}, at: console_flush_one_record+0x106/0x710 [ 4.011757][ T74] #3: ffffffff86a89660 (console_owner){....}-{0:0}, at: console_lock_spinning_enable+0x40/0x60 [ 4.011762][ T74] #4: ffffffff86a89560 (printk_legacy_map-wait-type-override){....}-{3:3}, at: console_emit_next_record+0x1df/0x4c0 [ 4.011767][ T74] irq event stamp: 2353104 [ 4.011768][ T74] hardirqs last enabled at (2353103): [] __down_trylock_console_sem+0x86/0xa0 [ 4.011773][ T74] hardirqs last disabled at (2353104): [] console_emit_next_record+0x3d4/0x4c0 [ 4.011775][ T74] softirqs last enabled at (2352258): [] handle_softirqs+0x67c/0x900 [ 4.011778][ T74] softirqs last disabled at (2351059): [] __irq_exit_rcu+0x145/0x1c0 [ 4.011782][ T74] Preemption disabled at: [ 4.011782][ T74] [<0000000000000000>] 0x0 [ 4.011791][ T74] CPU: 2 UID: 0 PID: 74 Comm: systemd-tmpfile Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 4.011795][ T74] Tainted: [W]=WARN [ 4.011796][ T74] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 4.011798][ T74] Call Trace: [ 4.011801][ T74] [ 4.011803][ T74] dump_stack_lvl+0x6f/0xa0 [ 4.011810][ T74] __might_resched.cold+0x1fe/0x2c1 [ 4.011816][ T74] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 4.011820][ T74] ? __kmalloc_noprof+0xdb/0x760 [ 4.011827][ T74] __kmalloc_noprof+0x443/0x760 [ 4.011831][ T74] ? alloc_buf.isra.0+0x4b/0x260 [ 4.011840][ T74] ? do_raw_spin_unlock+0x59/0x250 [ 4.011844][ T74] alloc_buf.isra.0+0x4b/0x260 [ 4.011849][ T74] put_chars+0x1e1/0x2f0 [ 4.011854][ T74] ? __send_to_port+0x420/0x420 [ 4.011856][ T74] ? rcu_lockdep_current_cpu_online+0x3f/0x1b0 [ 4.011862][ T74] ? validate_chain+0x38b/0xc20 [ 4.011872][ T74] hvc_console_print+0x292/0x780 [ 4.011883][ T74] ? hvc_write+0x3a0/0x3a0 [ 4.011887][ T74] ? rcu_is_watching+0x16/0xd0 [ 4.011890][ T74] ? lock_acquire+0x13c/0x160 [ 4.011897][ T74] console_emit_next_record+0x22f/0x4c0 [ 4.011903][ T74] ? devkmsg_read+0x4b0/0x4b0 [ 4.011905][ T74] ? console_flush_one_record+0x106/0x710 [ 4.011909][ T74] ? rcu_is_watching+0x16/0xd0 [ 4.011912][ T74] ? lock_acquire+0x13c/0x160 [ 4.011919][ T74] console_flush_one_record+0x46f/0x710 [ 4.011925][ T74] ? console_emit_next_record+0x4c0/0x4c0 [ 4.011927][ T74] ? __lock_acquire+0x518/0xc20 [ 4.011936][ T74] console_unlock+0xee/0x1f0 [ 4.011939][ T1] virtme-ng-init: Failed to read '/usr/lib/tmpfiles.d/audit.conf': Permission denied [ 4.011939][ T1] Failed to create directory or subvolume "/var/spool/at/spool": Permission denied [ 4.011939][ T1] Failed to create file /var/spool/at/.SEQ: Permission denied [ 4.011939][ T1] Failed to opendir() '/proc/self/fd/6': Permission denied [ 4.011940][ T74] ? console_flush_one_record+0x710/0x710 [ 4.011942][ T74] ? rcu_is_watching+0x16/0xd0 [ 4.011945][ T74] ? lock_acquire+0x60/0x160 [ 4.011951][ T74] ? __down_trylock_console_sem+0x5e/0xa0 [ 4.011954][ T74] ? vprintk_emit+0x320/0x3e0 [ 4.011958][ T74] vprintk_emit+0x37c/0x3e0 [ 4.011963][ T74] ? wake_up_klogd_work_func+0x90/0x90 [ 4.011968][ T74] ? __lock_acquire+0x518/0xc20 [ 4.011974][ T74] _printk+0xc7/0x100 [ 4.011979][ T74] ? snapshot_read.cold+0x21/0x21 [ 4.011983][ T74] ? do_raw_spin_lock+0x131/0x280 [ 4.011987][ T74] ? __rwlock_init+0x150/0x150 [ 4.011994][ T74] ? do_raw_spin_lock+0x131/0x280 [ 4.011998][ T74] do_exit.cold+0x82/0x9c [ 4.012003][ T74] ? exit_notify+0x890/0x890 [ 4.012005][ T74] ? __lock_release.isra.0+0x69/0x1a0 [ 4.012010][ T74] ? rcu_is_watching+0x16/0xd0 [ 4.012016][ T74] do_group_exit+0xb8/0x370 [ 4.012021][ T74] __x64_sys_exit_group+0x3c/0x50 [ 4.012024][ T74] x64_sys_call+0x1567/0x1570 [ 4.012027][ T74] do_syscall_64+0xff/0x530 [ 4.012032][ T74] ? exc_page_fault+0xee/0x100 [ 4.012036][ T74] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4.012040][ T74] RIP: 0033:0x7fb90a6311b8 [ 4.012043][ T74] Code: Unable to access opcode bytes at 0x7fb90a63118e. [ 4.012044][ T74] RSP: 002b:00007ffccbd7ba88 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7 [ 4.012048][ T74] RAX: ffffffffffffffda RBX: 00007fb90a761f88 RCX: 00007fb90a6311b8 [ 4.012049][ T74] RDX: 00007fb909e8f4c8 RSI: fffffffffffffe90 RDI: 0000000000000049 [ 4.012051][ T74] RBP: 00007ffccbd7bae0 R08: 0000000000000000 R09: 0000000000001000 [ 4.012052][ T74] R10: 00007ffccbd7b8a0 R11: 0000000000000246 R12: 0000000000000001 [ 4.012054][ T74] R13: 0000000000000049 R14: 00007fb90a760680 R15: 00007fb90a761fa0 [ 4.012066][ T74] [ 4.029498][ T1] virtme-ng-init: basic initialization done [ 4.104133][ T75] virtme-ng-init: Starting systemd-udevd version 259.8-1.fc44 [ 4.104537][ T75] virtme-ng-init: triggering udev coldplug [ 6.457591][ T75] virtme-ng-init: waiting for udev to settle [ 6.457605][ T75] BUG: sleeping function called from invalid context at ./include/linux/sched/mm.h:322 [ 6.457608][ T75] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 75, name: virtme-ng-init [ 6.457609][ T75] preempt_count: 1, expected: 0 [ 6.457610][ T75] RCU nest depth: 0, expected: 0 [ 6.457611][ T75] locks held by virtme-ng-init/75: 4, last CPU#3: [ 6.457614][ T75] #0: ffffffff86b69cc0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x320/0x3e0 [ 6.457626][ T75] #1: ffffffff86b69d38 (console_srcu){....}-{0:0}, at: console_flush_one_record+0x106/0x710 [ 6.457629][ T75] #2: ffffffff86a89660 (console_owner){....}-{0:0}, at: console_lock_spinning_enable+0x40/0x60 [ 6.457633][ T75] #3: ffffffff86a89560 (printk_legacy_map-wait-type-override){....}-{3:3}, at: console_emit_next_record+0x1df/0x4c0 [ 6.457637][ T75] irq event stamp: 3162 [ 6.457638][ T75] hardirqs last enabled at (3161): [] __down_trylock_console_sem+0x86/0xa0 [ 6.457641][ T75] hardirqs last disabled at (3162): [] console_emit_next_record+0x3d4/0x4c0 [ 6.457643][ T75] softirqs last enabled at (2568): [] fpu_clone+0xd9/0x410 [ 6.457647][ T75] softirqs last disabled at (2566): [] fpu_clone+0x82/0x410 [ 6.457649][ T75] Preemption disabled at: [ 6.457650][ T75] [] vprintk_emit+0x31b/0x3e0 [ 6.457656][ T75] CPU: 3 UID: 0 PID: 75 Comm: virtme-ng-init Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 6.457659][ T75] Tainted: [W]=WARN [ 6.457660][ T75] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 6.457661][ T75] Call Trace: [ 6.457663][ T75] [ 6.457665][ T75] dump_stack_lvl+0x6f/0xa0 [ 6.457669][ T75] ? vprintk_emit+0x31b/0x3e0 [ 6.457671][ T75] __might_resched.cold+0x1fe/0x2c1 [ 6.457675][ T75] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 6.457679][ T75] ? __kmalloc_noprof+0xdb/0x760 [ 6.457684][ T75] __kmalloc_noprof+0x443/0x760 [ 6.457686][ T75] ? alloc_buf.isra.0+0x4b/0x260 [ 6.457692][ T75] ? do_raw_spin_unlock+0x59/0x250 [ 6.457695][ T75] alloc_buf.isra.0+0x4b/0x260 [ 6.457699][ T75] put_chars+0x1e1/0x2f0 [ 6.457702][ T75] ? __send_to_port+0x420/0x420 [ 6.457703][ T75] ? rcu_lockdep_current_cpu_online+0x3f/0x1b0 [ 6.457707][ T75] ? validate_chain+0x38b/0xc20 [ 6.457713][ T75] hvc_console_print+0x292/0x780 [ 6.457719][ T75] ? hvc_write+0x3a0/0x3a0 [ 6.457722][ T75] ? rcu_is_watching+0x16/0xd0 [ 6.457724][ T75] ? lock_acquire+0x13c/0x160 [ 6.457728][ T75] console_emit_next_record+0x22f/0x4c0 [ 6.457731][ T75] ? devkmsg_read+0x4b0/0x4b0 [ 6.457732][ T75] ? console_flush_one_record+0x106/0x710 [ 6.457735][ T75] ? rcu_is_watching+0x16/0xd0 [ 6.457737][ T75] ? lock_acquire+0x13c/0x160 [ 6.457741][ T75] console_flush_one_record+0x46f/0x710 [ 6.457745][ T75] ? console_emit_next_record+0x4c0/0x4c0 [ 6.457746][ T75] ? __lock_acquire+0x518/0xc20 [ 6.457751][ T75] console_unlock+0xee/0x1f0 [ 6.457753][ T75] ? console_flush_one_record+0x710/0x710 [ 6.457755][ T75] ? rcu_is_watching+0x16/0xd0 [ 6.457757][ T75] ? lock_acquire+0x60/0x160 [ 6.457760][ T75] ? __down_trylock_console_sem+0x5e/0xa0 [ 6.457762][ T75] ? vprintk_emit+0x320/0x3e0 [ 6.457765][ T75] vprintk_emit+0x37c/0x3e0 [ 6.457768][ T75] ? wake_up_klogd_work_func+0x90/0x90 [ 6.457771][ T75] ? _copy_from_iter+0x1bb/0x1810 [ 6.457777][ T75] devkmsg_emit.constprop.0+0xbc/0xf1 [ 6.457781][ T75] ? vprintk_emit.cold+0x107/0x107 [ 6.457783][ T75] ? simple_strntoull+0x10f/0x140 [ 6.457788][ T75] ? date_str+0x1e0/0x1e0 [ 6.457791][ T75] ? devkmsg_write+0xd1/0x2c0 [ 6.457795][ T75] devkmsg_write.cold+0x5a/0x8b [ 6.457798][ T75] ? vprintk_default+0x20/0x20 [ 6.457802][ T75] ? vprintk_default+0x20/0x20 [ 6.457804][ T75] new_sync_write+0x33e/0x760 [ 6.457807][ T75] ? kasan_quarantine_put+0x42/0x2b0 [ 6.457811][ T75] ? new_sync_read+0x750/0x750 [ 6.457816][ T75] ? __lock_release.isra.0+0x69/0x1a0 [ 6.457820][ T75] ? __fget_files+0x1e3/0x460 [ 6.457824][ T75] vfs_write+0x6a2/0xbd0 [ 6.457828][ T75] ksys_write+0x116/0x250 [ 6.457831][ T75] ? __ia32_sys_read+0xc0/0xc0 [ 6.457833][ T75] ? rcu_is_watching+0x16/0xd0 [ 6.457837][ T75] do_syscall_64+0xff/0x530 [ 6.457839][ T75] ? exc_page_fault+0xee/0x100 [ 6.457842][ T75] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 6.457845][ T75] RIP: 0033:0x7f465bf37ed2 [ 6.457848][ T75] Code: 08 0f 85 51 ec ff ff 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa 55 48 89 e5 [ 6.457850][ T75] RSP: 002b:00007f465be7ec68 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 6.457853][ T75] RAX: ffffffffffffffda RBX: 000000000000002e RCX: 00007f465bf37ed2 [ 6.457854][ T75] RDX: 000000000000002e RSI: 00007f4654001090 RDI: 0000000000000003 [ 6.457855][ T75] RBP: 00007f465be7ec90 R08: 0000000000000000 R09: 0000000000000000 [ 6.457856][ T75] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f465bfc20a0 [ 6.457857][ T75] R13: 00007f465bec03e0 R14: 00007f4654001090 R15: 00007f465be7eda8 [ 6.457864][ T75] [ 6.463499][ C0] clocksource: Watchdog remote CPU 3 read timed out [ 6.463527][ C0] [ 6.463528][ C0] ======================================================== [ 6.463529][ C0] WARNING: possible irq lock inversion dependency detected [ 6.463530][ C0] 7.2.0-virtme #1 Tainted: G W [ 6.463532][ C0] -------------------------------------------------------- [ 6.463532][ C0] (udev-worker)/92 just changed the state of lock: [ 6.463533][ C0] ffffffff86a89660 (console_owner){..-.}-{0:0}, at: console_trylock_spinning+0xa4/0x1e0 [ 6.463543][ C0] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 6.463544][ C0] (fs_reclaim){+.+.}-{0:0} [ 6.463545][ C0] [ 6.463545][ C0] [ 6.463545][ C0] and interrupts could create inverse lock ordering between them. [ 6.463545][ C0] [ 6.463546][ C0] [ 6.463546][ C0] other info that might help us debug this: [ 6.463546][ C0] Possible interrupt unsafe locking scenario: [ 6.463546][ C0] [ 6.463547][ C0] CPU0 CPU1 [ 6.463548][ C0] ---- ---- [ 6.463548][ C0] lock(fs_reclaim); [ 6.463549][ C0] local_irq_disable(); [ 6.463550][ C0] lock(console_owner); [ 6.463551][ C0] lock(fs_reclaim); [ 6.463552][ C0] [ 6.463552][ C0] lock(console_owner); [ 6.463553][ C0] [ 6.463553][ C0] *** DEADLOCK *** [ 6.463553][ C0] [ 6.463553][ C0] locks held by (udev-worker)/92: 2, last CPU#0: [ 6.463555][ C0] #0: ffa0000000007c90 ((&watchdog_timer)){+.-.}-{0:0}, at: call_timer_fn+0x110/0x4d0 [ 6.463559][ C0] #1: ffffffff86bce8b8 (watchdog_lock){+.-.}-{3:3}, at: clocksource_watchdog+0x15/0x30 [ 6.463564][ C0] [ 6.463564][ C0] the shortest dependencies between 2nd lock and 1st lock: [ 6.463568][ C0] -> (fs_reclaim){+.+.}-{0:0} { [ 6.463570][ C0] HARDIRQ-ON-W at: [ 6.463572][ C0] __lock_acquire+0x388/0xc20 [ 6.463575][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463577][ C0] fs_reclaim_acquire+0xd5/0x120 [ 6.463580][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 6.463582][ C0] kthread_create_worker_on_node+0xea/0x210 [ 6.463585][ C0] workqueue_init+0x2a/0x680 [ 6.463588][ C0] kernel_init_freeable+0x2fe/0x630 [ 6.463591][ C0] kernel_init+0x21/0x150 [ 6.463594][ C0] ret_from_fork+0x474/0x6b0 [ 6.463597][ C0] ret_from_fork_asm+0x11/0x20 [ 6.463600][ C0] SOFTIRQ-ON-W at: [ 6.463601][ C0] __lock_acquire+0x388/0xc20 [ 6.463603][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463604][ C0] fs_reclaim_acquire+0xd5/0x120 [ 6.463605][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 6.463607][ C0] kthread_create_worker_on_node+0xea/0x210 [ 6.463609][ C0] workqueue_init+0x2a/0x680 [ 6.463610][ C0] kernel_init_freeable+0x2fe/0x630 [ 6.463611][ C0] kernel_init+0x21/0x150 [ 6.463613][ C0] ret_from_fork+0x474/0x6b0 [ 6.463614][ C0] ret_from_fork_asm+0x11/0x20 [ 6.463616][ C0] INITIAL USE at: [ 6.463617][ C0] __lock_acquire+0x388/0xc20 [ 6.463618][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463620][ C0] fs_reclaim_acquire+0xd5/0x120 [ 6.463621][ C0] __kmalloc_cache_noprof+0x6e/0x620 [ 6.463622][ C0] kthread_create_worker_on_node+0xea/0x210 [ 6.463624][ C0] workqueue_init+0x2a/0x680 [ 6.463626][ C0] kernel_init_freeable+0x2fe/0x630 [ 6.463627][ C0] kernel_init+0x21/0x150 [ 6.463628][ C0] ret_from_fork+0x474/0x6b0 [ 6.463630][ C0] ret_from_fork_asm+0x11/0x20 [ 6.463631][ C0] } [ 6.463632][ C0] ... key at: [] __fs_reclaim_map+0x0/0xbe0 [ 6.463636][ C0] ... acquired at: [ 6.463637][ C0] __lock_acquire+0x518/0xc20 [ 6.463639][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463640][ C0] fs_reclaim_acquire+0xd5/0x120 [ 6.463641][ C0] __kmalloc_noprof+0xd3/0x760 [ 6.463643][ C0] alloc_buf.isra.0+0x4b/0x260 [ 6.463646][ C0] put_chars+0x1e1/0x2f0 [ 6.463647][ C0] hvc_console_print+0x292/0x780 [ 6.463650][ C0] console_emit_next_record+0x22f/0x4c0 [ 6.463651][ C0] console_flush_one_record+0x46f/0x710 [ 6.463652][ C0] console_unlock+0xee/0x1f0 [ 6.463653][ C0] vprintk_emit+0x37c/0x3e0 [ 6.463654][ C0] _printk+0xc7/0x100 [ 6.463656][ C0] loop_init+0x12a/0x130 [ 6.463659][ C0] do_one_initcall+0x124/0x4f0 [ 6.463660][ C0] kernel_init_freeable+0x596/0x630 [ 6.463662][ C0] kernel_init+0x21/0x150 [ 6.463663][ C0] ret_from_fork+0x474/0x6b0 [ 6.463664][ C0] ret_from_fork_asm+0x11/0x20 [ 6.463666][ C0] [ 6.463666][ C0] -> (console_owner){..-.}-{0:0} { [ 6.463668][ C0] IN-SOFTIRQ-W at: [ 6.463669][ C0] __lock_acquire+0x388/0xc20 [ 6.463670][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463672][ C0] console_trylock_spinning+0xb5/0x1e0 [ 6.463674][ C0] vprintk_emit+0x320/0x3e0 [ 6.463675][ C0] _printk+0xc7/0x100 [ 6.463676][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 6.463678][ C0] call_timer_fn+0x160/0x4d0 [ 6.463679][ C0] __run_timers+0x68f/0xaa0 [ 6.463680][ C0] run_timer_softirq+0xf0/0x160 [ 6.463681][ C0] handle_softirqs+0x1d3/0x900 [ 6.463684][ C0] __irq_exit_rcu+0x145/0x1c0 [ 6.463686][ C0] irq_exit_rcu+0xe/0x30 [ 6.463687][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 6.463689][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 6.463691][ C0] __rcu_read_lock+0x4f/0x70 [ 6.463693][ C0] unwind_next_frame+0x16f/0x1ea0 [ 6.463695][ C0] __unwind_start+0x42e/0x900 [ 6.463697][ C0] arch_stack_walk+0xb3/0x130 [ 6.463698][ C0] stack_trace_save+0x93/0xc0 [ 6.463700][ C0] kasan_save_stack+0x2f/0x50 [ 6.463703][ C0] kasan_save_track+0x14/0x30 [ 6.463705][ C0] __kasan_slab_alloc+0x60/0x70 [ 6.463707][ C0] kmem_cache_alloc_noprof+0x21f/0x5c0 [ 6.463709][ C0] __alloc_object+0x30/0x260 [ 6.463711][ C0] __create_object+0x30/0x110 [ 6.463712][ C0] kmem_cache_alloc_noprof+0x467/0x5c0 [ 6.463713][ C0] do_getname.isra.0+0x32/0x240 [ 6.463716][ C0] do_sys_openat2+0xac/0x170 [ 6.463718][ C0] __x64_sys_openat+0x10e/0x210 [ 6.463719][ C0] do_syscall_64+0xff/0x530 [ 6.463721][ C0] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 6.463723][ C0] INITIAL USE at: [ 6.463723][ C0] } [ 6.463724][ C0] ... key at: [] console_owner_dep_map+0x0/0x60 [ 6.463727][ C0] ... acquired at: [ 6.463727][ C0] mark_lock+0x1d7/0xa00 [ 6.463729][ C0] mark_usage+0x42/0x170 [ 6.463730][ C0] __lock_acquire+0x388/0xc20 [ 6.463732][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463734][ C0] console_trylock_spinning+0xb5/0x1e0 [ 6.463735][ C0] vprintk_emit+0x320/0x3e0 [ 6.463736][ C0] _printk+0xc7/0x100 [ 6.463738][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 6.463739][ C0] call_timer_fn+0x160/0x4d0 [ 6.463740][ C0] __run_timers+0x68f/0xaa0 [ 6.463741][ C0] run_timer_softirq+0xf0/0x160 [ 6.463742][ C0] handle_softirqs+0x1d3/0x900 [ 6.463743][ C0] __irq_exit_rcu+0x145/0x1c0 [ 6.463745][ C0] irq_exit_rcu+0xe/0x30 [ 6.463746][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 6.463747][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 6.463749][ C0] __rcu_read_lock+0x4f/0x70 [ 6.463750][ C0] unwind_next_frame+0x16f/0x1ea0 [ 6.463751][ C0] __unwind_start+0x42e/0x900 [ 6.463752][ C0] arch_stack_walk+0xb3/0x130 [ 6.463753][ C0] stack_trace_save+0x93/0xc0 [ 6.463755][ C0] kasan_save_stack+0x2f/0x50 [ 6.463756][ C0] kasan_save_track+0x14/0x30 [ 6.463758][ C0] __kasan_slab_alloc+0x60/0x70 [ 6.463759][ C0] kmem_cache_alloc_noprof+0x21f/0x5c0 [ 6.463761][ C0] __alloc_object+0x30/0x260 [ 6.463763][ C0] __create_object+0x30/0x110 [ 6.463764][ C0] kmem_cache_alloc_noprof+0x467/0x5c0 [ 6.463765][ C0] do_getname.isra.0+0x32/0x240 [ 6.463767][ C0] do_sys_openat2+0xac/0x170 [ 6.463768][ C0] __x64_sys_openat+0x10e/0x210 [ 6.463770][ C0] do_syscall_64+0xff/0x530 [ 6.463771][ C0] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 6.463772][ C0] [ 6.463773][ C0] [ 6.463773][ C0] stack backtrace: [ 6.463775][ C0] CPU: 0 UID: 0 PID: 92 Comm: (udev-worker) Tainted: G W 7.2.0-virtme #1 PREEMPT(full) [ 6.463778][ C0] Tainted: [W]=WARN [ 6.463778][ C0] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 6.463780][ C0] Call Trace: [ 6.463781][ C0] [ 6.463782][ C0] dump_stack_lvl+0x6f/0xa0 [ 6.463785][ C0] print_irq_inversion_bug.part.0.cold+0xe6/0x143 [ 6.463788][ C0] mark_lock_irq+0x989/0x9c0 [ 6.463789][ C0] ? add_lock_to_list+0x2c/0x1a0 [ 6.463793][ C0] mark_lock+0x1d7/0xa00 [ 6.463795][ C0] mark_usage+0x42/0x170 [ 6.463796][ C0] __lock_acquire+0x388/0xc20 [ 6.463799][ C0] lock_acquire.part.0+0xd4/0x280 [ 6.463801][ C0] ? console_trylock_spinning+0xa4/0x1e0 [ 6.463803][ C0] ? rcu_is_watching+0x16/0xd0 [ 6.463804][ C0] ? lock_acquire+0x13c/0x160 [ 6.463807][ C0] console_trylock_spinning+0xb5/0x1e0 [ 6.463808][ C0] ? console_trylock_spinning+0xa4/0x1e0 [ 6.463811][ C0] vprintk_emit+0x320/0x3e0 [ 6.463812][ C0] ? wake_up_klogd_work_func+0x90/0x90 [ 6.463814][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 6.463816][ C0] _printk+0xc7/0x100 [ 6.463818][ C0] ? snapshot_read.cold+0x21/0x21 [ 6.463820][ C0] ? clocksource_watchdog.part.0+0x450/0x540 [ 6.463822][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 6.463824][ C0] clocksource_watchdog.part.0.cold+0x105/0x1dd [ 6.463826][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 6.463828][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 6.463829][ C0] call_timer_fn+0x160/0x4d0 [ 6.463831][ C0] ? detach_if_pending+0x1d0/0x1d0 [ 6.463832][ C0] ? debug_object_active_state+0x430/0x430 [ 6.463835][ C0] ? find_held_lock+0x2b/0x80 [ 6.463837][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 6.463839][ C0] ? rcu_is_watching+0x16/0xd0 [ 6.463841][ C0] __run_timers+0x68f/0xaa0 [ 6.463842][ C0] ? clocksource_watchdog.part.0+0x540/0x540 [ 6.463844][ C0] ? __bpf_trace_itimer_expire+0x10/0x10 [ 6.463846][ C0] ? __lock_acquire+0x518/0xc20 [ 6.463849][ C0] ? __rwlock_init+0x150/0x150 [ 6.463851][ C0] run_timer_softirq+0xf0/0x160 [ 6.463853][ C0] ? __run_timers+0xaa0/0xaa0 [ 6.463854][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 6.463857][ C0] ? rcu_is_watching+0x16/0xd0 [ 6.463858][ C0] handle_softirqs+0x1d3/0x900 [ 6.463860][ C0] ? __lock_release.isra.0+0x69/0x1a0 [ 6.463862][ C0] ? _local_bh_enable+0xc0/0xc0 [ 6.463864][ C0] __irq_exit_rcu+0x145/0x1c0 [ 6.463867][ C0] irq_exit_rcu+0xe/0x30 [ 6.463869][ C0] sysvec_apic_timer_interrupt+0x9d/0xe0 [ 6.463871][ C0] [ 6.463872][ C0] [ 6.463873][ C0] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 6.463875][ C0] RIP: 0010:__rcu_read_lock+0x4f/0x70 [ 6.463877][ C0] Code: 48 c1 e9 03 0f b6 0c 11 48 89 fa 83 e2 07 83 c2 03 38 ca 7c 04 84 c9 75 19 83 80 84 04 00 00 01 81 b8 84 04 00 00 ff ff ff 3f <7f> 01 c3 90 0f 0b 90 c3 e8 34 e8 7c 00 65 48 8b 05 ec ba 2d 04 eb [ 6.463879][ C0] RSP: 0018:ffa00000005e7770 EFLAGS: 00000293 [ 6.463881][ C0] RAX: ff1100000c38a380 RBX: 1ff40000000bcef8 RCX: 0000000000000000 [ 6.463882][ C0] RDX: 0000000000000007 RSI: ff1100000c38a380 RDI: ff1100000c38a804 [ 6.463883][ C0] RBP: 0000000000000001 R08: ffa00000005e78f0 R09: ffa00000005e78c8 [ 6.463884][ C0] R10: fff3fc00000bcf27 R11: fff3fc00000bcf27 R12: ffa00000005e7920 [ 6.463884][ C0] R13: ffa00000005e78c8 R14: ffa00000005e7910 R15: ffa00000005e78c8 [ 6.463887][ C0] unwind_next_frame+0x16f/0x1ea0 [ 6.463888][ C0] ? __unwind_start+0x357/0x900 [ 6.463890][ C0] ? find_held_lock+0x2/0x80 [ 6.463892][ C0] ? stack_access_ok+0x1e0/0x1e0 [ 6.463894][ C0] ? find_held_lock+0x2b/0x80 [ 6.463895][ C0] ? mark_usage+0x61/0x170 [ 6.463897][ C0] ? get_stack_info_noinstr+0x1a/0x120 [ 6.463898][ C0] ? __lock_acquire+0x518/0xc20 [ 6.463900][ C0] __unwind_start+0x42e/0x900 [ 6.463902][ C0] ? lock_acquire.part.0+0xd4/0x280 [ 6.463904][ C0] ? write_profile+0xf0/0xf0 [ 6.463905][ C0] arch_stack_walk+0xb3/0x130 [ 6.463907][ C0] ? __unwind_start+0x357/0x900 [ 6.463909][ C0] stack_trace_save+0x93/0xc0 [ 6.463911][ C0] ? filter_irq_stacks+0xd0/0xd0 [ 6.463912][ C0] ? arch_stack_walk+0xed/0x130 [ 6.463915][ C0] kasan_save_stack+0x2f/0x50 [ 6.463921][ C0] kasan_save_track+0x14/0x30 [ 6.463923][ C0] __kasan_slab_alloc+0x60/0x70 [ 6.463925][ C0] kmem_cache_alloc_noprof+0x21f/0x5c0 [ 6.463927][ C0] ? __alloc_object+0x30/0x260 [ 6.463929][ C0] __alloc_object+0x30/0x260 [ 6.463931][ C0] __create_object+0x30/0x110 [ 6.463932][ C0] ? kasan_save_track+0x14/0x30 [ 6.463935][ C0] kmem_cache_alloc_noprof+0x467/0x5c0 [ 6.463936][ C0] ? do_getname.isra.0+0x32/0x240 [ 6.463938][ C0] do_getname.isra.0+0x32/0x240 [ 6.463940][ C0] do_sys_openat2+0xac/0x170 [ 6.463942][ C0] ? build_open_flags+0x6e0/0x6e0 [ 6.463944][ C0] ? rcu_is_watching+0x16/0xd0 [ 6.463946][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 6.463947][ C0] ? lockdep_hardirqs_on+0x91/0x130 [ 6.463949][ C0] __x64_sys_openat+0x10e/0x210 [ 6.463951][ C0] ? fput_close_sync+0xde/0x1b0 [ 6.463953][ C0] ? __ia32_sys_open+0x1b0/0x1b0 [ 6.463955][ C0] ? rcu_is_watching+0x16/0xd0 [ 6.463956][ C0] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 6.463958][ C0] do_syscall_64+0xff/0x530 [ 6.463960][ C0] ? irq_exit_rcu+0x1a/0x30 [ 6.463961][ C0] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 6.463963][ C0] RIP: 0033:0x7f793ae9554e [ 6.463965][ C0] Code: 4d 89 d8 e8 b4 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa [ 6.463966][ C0] RSP: 002b:00007ffea42c1e00 EFLAGS: 00000202 ORIG_RAX: 0000000000000101 [ 6.463967][ C0] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f793ae9554e [ 6.463968][ C0] RDX: 00000000002a0000 RSI: 00005596e8db7a80 RDI: 0000000000000025 [ 6.463969][ C0] RBP: 00007ffea42c1e10 R08: 0000000000000000 R09: 0000000000000000 [ 6.463970][ C0] R10: 0000000000000000 R11: 0000000000000202 R12: 00005596e8c060b1 [ 6.463971][ C0] R13: 0000000000000025 R14: 00005596e8db7a80 R15: 0000000000000026 [ 6.463973][ C0] [ 6.668092][ T75] virtme-ng-init: udev is done [ 6.670146][ T1] virtme-ng-init: initialization done