[ 792.576884] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 792.579308] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py [ 792.733970] kselftest: Running tests in drivers/net [ 960.991086] kworker/u130:2 (319) used greatest stack depth: 22176 bytes left [ 960.999829] kworker/u130:0 (178) used greatest stack depth: 21936 bytes left [ 979.702473] block nvme0n1: No UUID available providing old NGUID SUBSYSTEM=block DEVICE=b259:0 [ 989.664354] INFO: task python3:5868 blocked for more than 122 seconds. [ 989.672574] Not tainted 7.1.0-rc2-marr-g20d76249b45a #1 [ 989.680364] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.689954] task:python3 state:D stack:25672 pid:5868 tgid:5868 ppid:5867 task_flags:0x400100 flags:0x00080000 [ 989.703161] Call Trace: [ 989.706700] [ 989.709852] __schedule+0xcba/0x1940 [ 989.714671] ? io_schedule_timeout+0x140/0x140 [ 989.720454] ? find_held_lock+0x2b/0x80 [ 989.725551] ? __lock_release.isra.0+0x6b/0x1a0 [ 989.731434] schedule+0xd4/0x260 [ 989.735850] schedule_preempt_disabled+0x18/0x30 [ 989.741829] __mutex_lock+0x6a3/0x2000 [ 989.746835] ? threaded_store+0x132/0x220 [ 989.752131] ? ww_mutex_lock+0x160/0x160 [ 989.757329] ? selinux_inode_free_security+0x330/0x330 [ 989.763898] ? cap_capable+0x1d6/0x3e0 [ 989.768906] ? threaded_store+0x132/0x220 [ 989.774200] threaded_store+0x132/0x220 [ 989.779300] ? xps_rxqs_store+0x2e0/0x2e0 [ 989.784598] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.789897] ? sysfs_kf_write+0x65/0x160 [ 989.795095] ? sysfs_file_ops+0x140/0x140 [ 989.800391] kernfs_fop_write_iter+0x335/0x510 [ 989.806182] ? kernfs_ops+0xf0/0xf0 [ 989.810892] new_sync_write+0x333/0x750 [ 989.815994] ? __lock_acquire+0x508/0xc10 [ 989.821289] ? new_sync_read+0x740/0x740 [ 989.826490] ? lock_acquire.part.0+0xbc/0x260 [ 989.832175] ? ksys_write+0x116/0x250 [ 989.837088] vfs_write+0x657/0x9b0 [ 989.841702] ksys_write+0x116/0x250 [ 989.846414] ? __ia32_sys_read+0xc0/0xc0 [ 989.851610] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.857200] ? rcu_is_watching+0x15/0xd0 [ 989.862399] do_syscall_64+0xf3/0x650 [ 989.867302] ? trace_hardirqs_off+0xd/0x30 [ 989.872694] ? exc_page_fault+0xda/0xf0 [ 989.877795] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.884269] RIP: 0033:0x7f0185d5022e [ 989.889079] RSP: 002b:00007fff07a3b9f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.898376] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f0185d5022e [ 989.907186] RDX: 0000000000000001 RSI: 00005560432c1fe0 RDI: 0000000000000007 [ 989.915994] RBP: 00007fff07a3ba00 R08: 0000000000000000 R09: 0000000000000000 [ 989.924803] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f0185bc8390 [ 989.933610] R13: 00005560432c1fe0 R14: 0000000000000007 R15: 0000000000000003 [ 989.942425] [ 989.945720] INFO: task python3:5868 is blocked on a mutex likely owned by task python3:3819. [ 989.955990] task:python3 state:D stack:25336 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802 [ 989.969194] Call Trace: [ 989.972731] [ 989.975882] __schedule+0xcba/0x1940 [ 989.980683] ? io_schedule_timeout+0x140/0x140 [ 989.986468] ? find_held_lock+0x2b/0x80 [ 989.991567] ? __try_to_del_timer_sync+0x155/0x1f0 [ 989.997743] schedule+0xd4/0x260 [ 990.002161] schedule_timeout+0x13f/0x290 [ 990.007457] ? hrtimer_nanosleep_restart+0x270/0x270 [ 990.013826] ? trace_contention_end+0xb3/0x1a0 [ 990.019611] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 990.025289] ? ww_mutex_lock+0x160/0x160 [ 990.030488] msleep+0x1b/0x30 [ 990.034613] napi_set_threaded+0x3a5/0x440 [ 990.040006] ? napi_resume_irqs+0x1a0/0x1a0 [ 990.045498] netif_set_threaded+0x257/0x400 [ 990.050992] threaded_store+0x1b8/0x220 [ 990.056091] ? xps_rxqs_store+0x2e0/0x2e0 [ 990.061387] ? sysfs_file_kobj+0xbd/0x1e0 [ 990.066682] ? sysfs_kf_write+0x65/0x160 [ 990.071883] ? sysfs_file_ops+0x140/0x140 [ 990.077175] kernfs_fop_write_iter+0x335/0x510 [ 990.082958] ? kernfs_ops+0xf0/0xf0 [ 990.087669] new_sync_write+0x333/0x750 [ 990.092771] ? __lock_acquire+0x508/0xc10 [ 990.098066] ? new_sync_read+0x740/0x740 [ 990.103265] ? lock_acquire.part.0+0xbc/0x260 [ 990.108943] ? ksys_write+0x116/0x250 [ 990.113850] vfs_write+0x657/0x9b0 [ 990.118466] ksys_write+0x116/0x250 [ 990.123174] ? __ia32_sys_read+0xc0/0xc0 [ 990.128371] ? kernfs_fop_llseek+0x160/0x1f0 [ 990.133959] ? rcu_is_watching+0x15/0xd0 [ 990.139157] do_syscall_64+0xf3/0x650 [ 990.144060] ? trace_hardirqs_off+0xd/0x30 [ 990.149455] ? exc_page_fault+0xda/0xf0 [ 990.154548] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 990.161012] RIP: 0033:0x7f7d3f81722e [ 990.165820] RSP: 002b:00007ffd38cb95c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 990.175116] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7d3f81722e [ 990.183917] RDX: 0000000000000001 RSI: 000056310fd29fe0 RDI: 0000000000000007 [ 990.192725] RBP: 00007ffd38cb95d0 R08: 0000000000000000 R09: 0000000000000000 [ 990.201521] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7d3f68f390 [ 990.210326] R13: 000056310fd29fe0 R14: 0000000000000007 R15: 0000000000000003 [ 990.219137] [ 990.222398] \x0aShowing all locks held in the system: [ 990.230933] 1 lock held by khungtaskd/185: [ 990.236325] #0: ffffffffafafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 990.247808] 4 locks held by python3/3819: [ 990.253100] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 990.262986] #1: ff1100013f5d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 990.273948] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.285104] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.295206] 4 locks held by python3/5868: [ 990.300500] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 990.310394] #1: ff110001f2869480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 990.321368] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.332531] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.342617] [ 990.345082] =============================================\x0a [ 1112.544491] INFO: task python3:5868 blocked for more than 245 seconds. [ 1112.552638] Not tainted 7.1.0-rc2-marr-g20d76249b45a #1 [ 1112.559879] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.569466] task:python3 state:D stack:25672 pid:5868 tgid:5868 ppid:5867 task_flags:0x400100 flags:0x00080000 [ 1112.582670] Call Trace: [ 1112.586209] [ 1112.589357] __schedule+0xcba/0x1940 [ 1112.594171] ? io_schedule_timeout+0x140/0x140 [ 1112.599958] ? find_held_lock+0x2b/0x80 [ 1112.605060] ? __lock_release.isra.0+0x6b/0x1a0 [ 1112.610945] schedule+0xd4/0x260 [ 1112.615362] schedule_preempt_disabled+0x18/0x30 [ 1112.621338] __mutex_lock+0x6a3/0x2000 [ 1112.626340] ? threaded_store+0x132/0x220 [ 1112.631639] ? ww_mutex_lock+0x160/0x160 [ 1112.636837] ? selinux_inode_free_security+0x330/0x330 [ 1112.643406] ? cap_capable+0x1d6/0x3e0 [ 1112.648412] ? threaded_store+0x132/0x220 [ 1112.653708] threaded_store+0x132/0x220 [ 1112.658809] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.664105] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.669404] ? sysfs_kf_write+0x65/0x160 [ 1112.674603] ? sysfs_file_ops+0x140/0x140 [ 1112.679896] kernfs_fop_write_iter+0x335/0x510 [ 1112.685679] ? kernfs_ops+0xf0/0xf0 [ 1112.690391] new_sync_write+0x333/0x750 [ 1112.695494] ? __lock_acquire+0x508/0xc10 [ 1112.700790] ? new_sync_read+0x740/0x740 [ 1112.705992] ? lock_acquire.part.0+0xbc/0x260 [ 1112.711679] ? ksys_write+0x116/0x250 [ 1112.716579] vfs_write+0x657/0x9b0 [ 1112.721192] ksys_write+0x116/0x250 [ 1112.725904] ? __ia32_sys_read+0xc0/0xc0 [ 1112.731101] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.736688] ? rcu_is_watching+0x15/0xd0 [ 1112.741890] do_syscall_64+0xf3/0x650 [ 1112.746794] ? trace_hardirqs_off+0xd/0x30 [ 1112.752186] ? exc_page_fault+0xda/0xf0 [ 1112.757289] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.763756] RIP: 0033:0x7f0185d5022e [ 1112.768566] RSP: 002b:00007fff07a3b9f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.777866] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f0185d5022e [ 1112.786671] RDX: 0000000000000001 RSI: 00005560432c1fe0 RDI: 0000000000000007 [ 1112.795478] RBP: 00007fff07a3ba00 R08: 0000000000000000 R09: 0000000000000000 [ 1112.804284] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f0185bc8390 [ 1112.813091] R13: 00005560432c1fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1112.821900] [ 1112.825201] INFO: task python3:5868 is blocked on a mutex likely owned by task python3:3819. [ 1112.835472] task:python3 state:D stack:25336 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802 [ 1112.848676] Call Trace: [ 1112.852214] [ 1112.855361] __schedule+0xcba/0x1940 [ 1112.860164] ? io_schedule_timeout+0x140/0x140 [ 1112.865950] ? find_held_lock+0x2b/0x80 [ 1112.871053] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1112.877232] schedule+0xd4/0x260 [ 1112.881651] schedule_timeout+0x13f/0x290 [ 1112.886948] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1112.893309] ? trace_contention_end+0xb3/0x1a0 [ 1112.899092] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1112.904781] ? ww_mutex_lock+0x160/0x160 [ 1112.909968] msleep+0x1b/0x30 [ 1112.914094] napi_set_threaded+0x3a5/0x440 [ 1112.919480] ? napi_resume_irqs+0x1a0/0x1a0 [ 1112.924975] netif_set_threaded+0x257/0x400 [ 1112.930467] threaded_store+0x1b8/0x220 [ 1112.935560] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.940856] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.946153] ? sysfs_kf_write+0x65/0x160 [ 1112.951350] ? sysfs_file_ops+0x140/0x140 [ 1112.956644] kernfs_fop_write_iter+0x335/0x510 [ 1112.962428] ? kernfs_ops+0xf0/0xf0 [ 1112.967137] new_sync_write+0x333/0x750 [ 1112.972237] ? __lock_acquire+0x508/0xc10 [ 1112.977534] ? new_sync_read+0x740/0x740 [ 1112.982732] ? lock_acquire.part.0+0xbc/0x260 [ 1112.988415] ? ksys_write+0x116/0x250 [ 1112.993330] vfs_write+0x657/0x9b0 [ 1112.997944] ksys_write+0x116/0x250 [ 1113.002655] ? __ia32_sys_read+0xc0/0xc0 [ 1113.007855] ? kernfs_fop_llseek+0x160/0x1f0 [ 1113.013434] ? rcu_is_watching+0x15/0xd0 [ 1113.018636] do_syscall_64+0xf3/0x650 [ 1113.023541] ? trace_hardirqs_off+0xd/0x30 [ 1113.028934] ? exc_page_fault+0xda/0xf0 [ 1113.034033] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1113.040500] RIP: 0033:0x7f7d3f81722e [ 1113.045308] RSP: 002b:00007ffd38cb95c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1113.054603] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7d3f81722e [ 1113.063409] RDX: 0000000000000001 RSI: 000056310fd29fe0 RDI: 0000000000000007 [ 1113.072217] RBP: 00007ffd38cb95d0 R08: 0000000000000000 R09: 0000000000000000 [ 1113.081025] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7d3f68f390 [ 1113.089831] R13: 000056310fd29fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1113.098643] [ 1113.101908] \x0aShowing all locks held in the system: [ 1113.110451] 1 lock held by khungtaskd/185: [ 1113.115841] #0: ffffffffafafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1113.127335] 4 locks held by python3/3819: [ 1113.132621] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1113.142513] #1: ff1100013f5d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1113.153473] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.164628] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.174732] 4 locks held by python3/5868: [ 1113.180026] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1113.189913] #1: ff110001f2869480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1113.200873] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.212029] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.222118] [ 1113.224584] =============================================\x0a [ 1235.425643] INFO: task python3:5868 blocked for more than 368 seconds. [ 1235.433790] Not tainted 7.1.0-rc2-marr-g20d76249b45a #1 [ 1235.441040] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.450629] task:python3 state:D stack:25672 pid:5868 tgid:5868 ppid:5867 task_flags:0x400100 flags:0x00080002 [ 1235.463836] Call Trace: [ 1235.467377] [ 1235.470527] __schedule+0xcba/0x1940 [ 1235.475344] ? io_schedule_timeout+0x140/0x140 [ 1235.481132] ? find_held_lock+0x2b/0x80 [ 1235.486235] ? __lock_release.isra.0+0x6b/0x1a0 [ 1235.492113] schedule+0xd4/0x260 [ 1235.496531] schedule_preempt_disabled+0x18/0x30 [ 1235.502509] __mutex_lock+0x6a3/0x2000 [ 1235.507514] ? threaded_store+0x132/0x220 [ 1235.512814] ? ww_mutex_lock+0x160/0x160 [ 1235.518005] ? selinux_inode_free_security+0x330/0x330 [ 1235.524576] ? cap_capable+0x1d6/0x3e0 [ 1235.529586] ? threaded_store+0x132/0x220 [ 1235.534887] threaded_store+0x132/0x220 [ 1235.539990] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.545288] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.550585] ? sysfs_kf_write+0x65/0x160 [ 1235.555788] ? sysfs_file_ops+0x140/0x140 [ 1235.561087] kernfs_fop_write_iter+0x335/0x510 [ 1235.566874] ? kernfs_ops+0xf0/0xf0 [ 1235.571587] new_sync_write+0x333/0x750 [ 1235.576691] ? __lock_acquire+0x508/0xc10 [ 1235.581988] ? new_sync_read+0x740/0x740 [ 1235.587191] ? lock_acquire.part.0+0xbc/0x260 [ 1235.592878] ? ksys_write+0x116/0x250 [ 1235.597790] vfs_write+0x657/0x9b0 [ 1235.602407] ksys_write+0x116/0x250 [ 1235.607120] ? __ia32_sys_read+0xc0/0xc0 [ 1235.612319] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.617909] ? rcu_is_watching+0x15/0xd0 [ 1235.623112] do_syscall_64+0xf3/0x650 [ 1235.628017] ? trace_hardirqs_off+0xd/0x30 [ 1235.633413] ? exc_page_fault+0xda/0xf0 [ 1235.638518] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.644985] RIP: 0033:0x7f0185d5022e [ 1235.649797] RSP: 002b:00007fff07a3b9f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.659096] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f0185d5022e [ 1235.667907] RDX: 0000000000000001 RSI: 00005560432c1fe0 RDI: 0000000000000007 [ 1235.676716] RBP: 00007fff07a3ba00 R08: 0000000000000000 R09: 0000000000000000 [ 1235.685526] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f0185bc8390 [ 1235.694336] R13: 00005560432c1fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1235.703150] [ 1235.706454] INFO: task python3:5868 is blocked on a mutex likely owned by task python3:3819. [ 1235.716727] task:python3 state:D stack:25336 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802 [ 1235.729931] Call Trace: [ 1235.733475] [ 1235.736627] __schedule+0xcba/0x1940 [ 1235.741440] ? io_schedule_timeout+0x140/0x140 [ 1235.747226] ? find_held_lock+0x2b/0x80 [ 1235.752329] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1235.758510] schedule+0xd4/0x260 [ 1235.762928] schedule_timeout+0x13f/0x290 [ 1235.768224] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1235.774593] ? trace_contention_end+0xb3/0x1a0 [ 1235.780379] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1235.786069] ? ww_mutex_lock+0x160/0x160 [ 1235.791268] msleep+0x1b/0x30 [ 1235.795393] napi_set_threaded+0x3a5/0x440 [ 1235.800792] ? napi_resume_irqs+0x1a0/0x1a0 [ 1235.806289] netif_set_threaded+0x257/0x400 [ 1235.811783] threaded_store+0x1b8/0x220 [ 1235.816888] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.822185] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.827479] ? sysfs_kf_write+0x65/0x160 [ 1235.832680] ? sysfs_file_ops+0x140/0x140 [ 1235.837976] kernfs_fop_write_iter+0x335/0x510 [ 1235.843763] ? kernfs_ops+0xf0/0xf0 [ 1235.848479] new_sync_write+0x333/0x750 [ 1235.853581] ? __lock_acquire+0x508/0xc10 [ 1235.858880] ? new_sync_read+0x740/0x740 [ 1235.864081] ? lock_acquire.part.0+0xbc/0x260 [ 1235.869766] ? ksys_write+0x116/0x250 [ 1235.874678] vfs_write+0x657/0x9b0 [ 1235.879292] ksys_write+0x116/0x250 [ 1235.883996] ? __ia32_sys_read+0xc0/0xc0 [ 1235.889186] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.894776] ? rcu_is_watching+0x15/0xd0 [ 1235.899979] do_syscall_64+0xf3/0x650 [ 1235.904888] ? trace_hardirqs_off+0xd/0x30 [ 1235.910281] ? exc_page_fault+0xda/0xf0 [ 1235.915382] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.921850] RIP: 0033:0x7f7d3f81722e [ 1235.926658] RSP: 002b:00007ffd38cb95c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.935956] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7d3f81722e [ 1235.944765] RDX: 0000000000000001 RSI: 000056310fd29fe0 RDI: 0000000000000007 [ 1235.953572] RBP: 00007ffd38cb95d0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.962380] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7d3f68f390 [ 1235.971187] R13: 000056310fd29fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1235.979999] [ 1235.983271] \x0aShowing all locks held in the system: [ 1235.991811] 1 lock held by khungtaskd/185: [ 1235.997207] #0: ffffffffafafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1236.008683] 4 locks held by python3/3819: [ 1236.013980] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1236.023871] #1: ff1100013f5d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1236.034834] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.045994] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.056094] 4 locks held by python3/5868: [ 1236.061388] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1236.071277] #1: ff110001f2869480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1236.082228] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.093389] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.103487] [ 1236.105954] =============================================\x0a [ 1358.306809] INFO: task python3:5868 blocked for more than 491 seconds. [ 1358.314957] Not tainted 7.1.0-rc2-marr-g20d76249b45a #1 [ 1358.322198] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.331787] task:python3 state:D stack:25672 pid:5868 tgid:5868 ppid:5867 task_flags:0x400100 flags:0x00080002 [ 1358.344991] Call Trace: [ 1358.348532] [ 1358.351683] __schedule+0xcba/0x1940 [ 1358.356500] ? io_schedule_timeout+0x140/0x140 [ 1358.362288] ? find_held_lock+0x2b/0x80 [ 1358.367393] ? __lock_release.isra.0+0x6b/0x1a0 [ 1358.373277] schedule+0xd4/0x260 [ 1358.377695] schedule_preempt_disabled+0x18/0x30 [ 1358.383673] __mutex_lock+0x6a3/0x2000 [ 1358.388676] ? threaded_store+0x132/0x220 [ 1358.393966] ? ww_mutex_lock+0x160/0x160 [ 1358.399167] ? selinux_inode_free_security+0x330/0x330 [ 1358.405737] ? cap_capable+0x1d6/0x3e0 [ 1358.410748] ? threaded_store+0x132/0x220 [ 1358.416046] threaded_store+0x132/0x220 [ 1358.421147] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.426447] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.431747] ? sysfs_kf_write+0x65/0x160 [ 1358.436950] ? sysfs_file_ops+0x140/0x140 [ 1358.442249] kernfs_fop_write_iter+0x335/0x510 [ 1358.448036] ? kernfs_ops+0xf0/0xf0 [ 1358.452749] new_sync_write+0x333/0x750 [ 1358.457845] ? __lock_acquire+0x508/0xc10 [ 1358.463142] ? new_sync_read+0x740/0x740 [ 1358.468346] ? lock_acquire.part.0+0xbc/0x260 [ 1358.474034] ? ksys_write+0x116/0x250 [ 1358.478943] vfs_write+0x657/0x9b0 [ 1358.483557] ksys_write+0x116/0x250 [ 1358.488270] ? __ia32_sys_read+0xc0/0xc0 [ 1358.493462] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.499055] ? rcu_is_watching+0x15/0xd0 [ 1358.504256] do_syscall_64+0xf3/0x650 [ 1358.509156] ? trace_hardirqs_off+0xd/0x30 [ 1358.514553] ? exc_page_fault+0xda/0xf0 [ 1358.519656] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.526124] RIP: 0033:0x7f0185d5022e [ 1358.530936] RSP: 002b:00007fff07a3b9f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.540235] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f0185d5022e [ 1358.549044] RDX: 0000000000000001 RSI: 00005560432c1fe0 RDI: 0000000000000007 [ 1358.557854] RBP: 00007fff07a3ba00 R08: 0000000000000000 R09: 0000000000000000 [ 1358.566664] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f0185bc8390 [ 1358.575471] R13: 00005560432c1fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1358.584278] [ 1358.587564] INFO: task python3:5868 is blocked on a mutex likely owned by task python3:3819. [ 1358.597840] task:python3 state:D stack:25336 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802 [ 1358.611046] Call Trace: [ 1358.614583] [ 1358.617737] __schedule+0xcba/0x1940 [ 1358.622551] ? io_schedule_timeout+0x140/0x140 [ 1358.628336] ? find_held_lock+0x2b/0x80 [ 1358.633432] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1358.639610] schedule+0xd4/0x260 [ 1358.644030] schedule_timeout+0x13f/0x290 [ 1358.649328] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1358.655689] ? trace_contention_end+0xb3/0x1a0 [ 1358.661472] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1358.667161] ? ww_mutex_lock+0x160/0x160 [ 1358.672362] msleep+0x1b/0x30 [ 1358.676488] napi_set_threaded+0x3a5/0x440 [ 1358.681883] ? napi_resume_irqs+0x1a0/0x1a0 [ 1358.687377] netif_set_threaded+0x257/0x400 [ 1358.692873] threaded_store+0x1b8/0x220 [ 1358.697974] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.703273] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.708570] ? sysfs_kf_write+0x65/0x160 [ 1358.713772] ? sysfs_file_ops+0x140/0x140 [ 1358.719067] kernfs_fop_write_iter+0x335/0x510 [ 1358.724850] ? kernfs_ops+0xf0/0xf0 [ 1358.729562] new_sync_write+0x333/0x750 [ 1358.734666] ? __lock_acquire+0x508/0xc10 [ 1358.739962] ? new_sync_read+0x740/0x740 [ 1358.745164] ? lock_acquire.part.0+0xbc/0x260 [ 1358.750850] ? ksys_write+0x116/0x250 [ 1358.755759] vfs_write+0x657/0x9b0 [ 1358.760376] ksys_write+0x116/0x250 [ 1358.765090] ? __ia32_sys_read+0xc0/0xc0 [ 1358.770287] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.775878] ? rcu_is_watching+0x15/0xd0 [ 1358.781078] do_syscall_64+0xf3/0x650 [ 1358.785987] ? trace_hardirqs_off+0xd/0x30 [ 1358.791381] ? exc_page_fault+0xda/0xf0 [ 1358.796483] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.802951] RIP: 0033:0x7f7d3f81722e [ 1358.807762] RSP: 002b:00007ffd38cb95c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.817060] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7d3f81722e [ 1358.825869] RDX: 0000000000000001 RSI: 000056310fd29fe0 RDI: 0000000000000007 [ 1358.834680] RBP: 00007ffd38cb95d0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.843489] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7d3f68f390 [ 1358.852298] R13: 000056310fd29fe0 R14: 0000000000000007 R15: 0000000000000003 [ 1358.861111] [ 1358.864382] \x0aShowing all locks held in the system: [ 1358.872923] 1 lock held by khungtaskd/185: [ 1358.878315] #0: ffffffffafafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1358.889801] 4 locks held by python3/3819: [ 1358.895097] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1358.904990] #1: ff1100013f5d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1358.915950] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1358.927108] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1358.937196] 4 locks held by python3/5868: [ 1358.942491] #0: ff110001308d13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1358.952381] #1: ff110001f2869480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1358.963342] #2: ff11000123d2cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1358.974499] #3: ff11000123d99d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1358.984594] [ 1358.987062] =============================================\x0a