[ 792.939470] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 792.943715] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 793.073897] kselftest: Running tests in drivers/net
[ 989.698984] INFO: task python3:5887 blocked for more than 122 seconds.
[ 989.707204] Not tainted 7.1.0-rc4-olrp-g5759c1b1736e #1
[ 989.715051] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.724639] task:python3 state:D stack:25672 pid:5887 tgid:5887 ppid:5886 task_flags:0x400100 flags:0x00080000
[ 989.737848] Call Trace:
[ 989.741386]
[ 989.744538] __schedule+0xcc3/0x19c0
[ 989.749356] ? io_schedule_timeout+0x140/0x140
[ 989.755140] ? find_held_lock+0x2b/0x80
[ 989.760245] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.766133] schedule+0xd8/0x260
[ 989.770552] schedule_preempt_disabled+0x18/0x30
[ 989.776531] __mutex_lock+0x6af/0x2010
[ 989.781535] ? threaded_store+0x132/0x220
[ 989.786835] ? ww_mutex_lock+0x160/0x160
[ 989.792027] ? selinux_inode_free_security+0x330/0x330
[ 989.798600] ? cap_capable+0x1d6/0x3e0
[ 989.803607] ? threaded_store+0x132/0x220
[ 989.808904] threaded_store+0x132/0x220
[ 989.814002] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.819298] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.824597] ? sysfs_kf_write+0x65/0x160
[ 989.829792] ? sysfs_file_ops+0x140/0x140
[ 989.835087] kernfs_fop_write_iter+0x335/0x510
[ 989.840872] ? kernfs_ops+0xf0/0xf0
[ 989.845577] new_sync_write+0x333/0x750
[ 989.850679] ? __lock_acquire+0x508/0xc10
[ 989.855978] ? new_sync_read+0x740/0x740
[ 989.861179] ? lock_acquire.part.0+0xbc/0x260
[ 989.866863] ? ksys_write+0x116/0x250
[ 989.871776] vfs_write+0x657/0x9b0
[ 989.876391] ksys_write+0x116/0x250
[ 989.881095] ? __ia32_sys_read+0xc0/0xc0
[ 989.886294] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.891885] ? rcu_is_watching+0x15/0xd0
[ 989.897087] do_syscall_64+0xf3/0x650
[ 989.901996] ? trace_hardirqs_off+0xd/0x30
[ 989.907391] ? exc_page_fault+0xda/0xf0
[ 989.912491] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.918966] RIP: 0033:0x7f7fc476822e
[ 989.923766] RSP: 002b:00007ffc3fca7490 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.933067] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fc476822e
[ 989.941875] RDX: 0000000000000001 RSI: 000055bae50706b0 RDI: 0000000000000007
[ 989.950673] RBP: 00007ffc3fca74a0 R08: 0000000000000000 R09: 0000000000000000
[ 989.959481] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fc45e0390
[ 989.968288] R13: 000055bae50706b0 R14: 0000000000000007 R15: 0000000000000003
[ 989.977101]
[ 989.980396] INFO: task python3:5887 is blocked on a mutex likely owned by task python3:3873.
[ 989.990668] task:python3 state:D stack:25672 pid:3873 tgid:3873 ppid:3872 task_flags:0x400100 flags:0x00080802
[ 990.003871] Call Trace:
[ 990.007413]
[ 990.010563] __schedule+0xcc3/0x19c0
[ 990.015374] ? io_schedule_timeout+0x140/0x140
[ 990.021156] ? find_held_lock+0x2b/0x80
[ 990.026260] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.032439] schedule+0xd8/0x260
[ 990.036860] schedule_timeout+0x13f/0x290
[ 990.042159] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.048531] ? trace_contention_end+0xb3/0x1a0
[ 990.054316] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.060007] ? ww_mutex_lock+0x160/0x160
[ 990.065206] msleep+0x1b/0x30
[ 990.069331] napi_set_threaded+0x3a5/0x440
[ 990.074726] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.080222] netif_set_threaded+0x257/0x400
[ 990.085715] threaded_store+0x1b8/0x220
[ 990.090822] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.096122] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.101419] ? sysfs_kf_write+0x65/0x160
[ 990.106621] ? sysfs_file_ops+0x140/0x140
[ 990.111919] kernfs_fop_write_iter+0x335/0x510
[ 990.117705] ? kernfs_ops+0xf0/0xf0
[ 990.122417] new_sync_write+0x333/0x750
[ 990.127512] ? __lock_acquire+0x508/0xc10
[ 990.132816] ? new_sync_read+0x740/0x740
[ 990.138013] ? lock_acquire.part.0+0xbc/0x260
[ 990.143697] ? ksys_write+0x116/0x250
[ 990.148608] vfs_write+0x657/0x9b0
[ 990.153222] ksys_write+0x116/0x250
[ 990.157934] ? __ia32_sys_read+0xc0/0xc0
[ 990.163133] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.168722] ? rcu_is_watching+0x15/0xd0
[ 990.173923] do_syscall_64+0xf3/0x650
[ 990.178829] ? trace_hardirqs_off+0xd/0x30
[ 990.184221] ? exc_page_fault+0xda/0xf0
[ 990.189327] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.195792] RIP: 0033:0x7f2a8937b22e
[ 990.200599] RSP: 002b:00007ffeddfcabd0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.209897] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2a8937b22e
[ 990.218705] RDX: 0000000000000001 RSI: 000055b9adeb66b0 RDI: 0000000000000007
[ 990.227513] RBP: 00007ffeddfcabe0 R08: 0000000000000000 R09: 0000000000000000
[ 990.236320] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2a891f3390
[ 990.245130] R13: 000055b9adeb66b0 R14: 0000000000000007 R15: 0000000000000003
[ 990.253946]
[ 990.257205] \x0aShowing all locks held in the system:
[ 990.265745] 1 lock held by khungtaskd/185:
[ 990.271139] #0: ffffffff940fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.282622] 4 locks held by python3/3873:
[ 990.287916] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.297805] #1: ff11000161c74a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.308774] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.319922] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.330014] 4 locks held by python3/5887:
[ 990.335308] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.345199] #1: ff110001ece78280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.356162] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.367320] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.377412]
[ 990.379878] =============================================\x0a
[ 1112.580098] INFO: task python3:5887 blocked for more than 245 seconds.
[ 1112.588246] Not tainted 7.1.0-rc4-olrp-g5759c1b1736e #1
[ 1112.595489] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.605079] task:python3 state:D stack:25672 pid:5887 tgid:5887 ppid:5886 task_flags:0x400100 flags:0x00080000
[ 1112.618275] Call Trace:
[ 1112.621813]
[ 1112.624964] __schedule+0xcc3/0x19c0
[ 1112.629780] ? io_schedule_timeout+0x140/0x140
[ 1112.635565] ? find_held_lock+0x2b/0x80
[ 1112.640670] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.646544] schedule+0xd8/0x260
[ 1112.650959] schedule_preempt_disabled+0x18/0x30
[ 1112.656943] __mutex_lock+0x6af/0x2010
[ 1112.661949] ? threaded_store+0x132/0x220
[ 1112.667250] ? ww_mutex_lock+0x160/0x160
[ 1112.672447] ? selinux_inode_free_security+0x330/0x330
[ 1112.679006] ? cap_capable+0x1d6/0x3e0
[ 1112.684012] ? threaded_store+0x132/0x220
[ 1112.689309] threaded_store+0x132/0x220
[ 1112.694408] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.699705] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.705002] ? sysfs_kf_write+0x65/0x160
[ 1112.710204] ? sysfs_file_ops+0x140/0x140
[ 1112.715499] kernfs_fop_write_iter+0x335/0x510
[ 1112.721281] ? kernfs_ops+0xf0/0xf0
[ 1112.725991] new_sync_write+0x333/0x750
[ 1112.731094] ? __lock_acquire+0x508/0xc10
[ 1112.736388] ? new_sync_read+0x740/0x740
[ 1112.741589] ? lock_acquire.part.0+0xbc/0x260
[ 1112.747275] ? ksys_write+0x116/0x250
[ 1112.752183] vfs_write+0x657/0x9b0
[ 1112.756790] ksys_write+0x116/0x250
[ 1112.761502] ? __ia32_sys_read+0xc0/0xc0
[ 1112.766698] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.772288] ? rcu_is_watching+0x15/0xd0
[ 1112.777488] do_syscall_64+0xf3/0x650
[ 1112.782392] ? trace_hardirqs_off+0xd/0x30
[ 1112.787786] ? exc_page_fault+0xda/0xf0
[ 1112.792889] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.799348] RIP: 0033:0x7f7fc476822e
[ 1112.804159] RSP: 002b:00007ffc3fca7490 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.813456] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fc476822e
[ 1112.822263] RDX: 0000000000000001 RSI: 000055bae50706b0 RDI: 0000000000000007
[ 1112.831071] RBP: 00007ffc3fca74a0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.839879] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fc45e0390
[ 1112.848679] R13: 000055bae50706b0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.857493]
[ 1112.860773] INFO: task python3:5887 is blocked on a mutex likely owned by task python3:3873.
[ 1112.871043] task:python3 state:D stack:25672 pid:3873 tgid:3873 ppid:3872 task_flags:0x400100 flags:0x00080802
[ 1112.884245] Call Trace:
[ 1112.887783]
[ 1112.890930] __schedule+0xcc3/0x19c0
[ 1112.895746] ? io_schedule_timeout+0x140/0x140
[ 1112.901531] ? find_held_lock+0x2b/0x80
[ 1112.906622] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.912800] schedule+0xd8/0x260
[ 1112.917218] schedule_timeout+0x13f/0x290
[ 1112.922515] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.928884] ? trace_contention_end+0xb3/0x1a0
[ 1112.934657] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.940345] ? ww_mutex_lock+0x160/0x160
[ 1112.945541] msleep+0x1b/0x30
[ 1112.949667] napi_set_threaded+0x3a5/0x440
[ 1112.955062] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.960556] netif_set_threaded+0x257/0x400
[ 1112.966051] threaded_store+0x1b8/0x220
[ 1112.971152] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.976448] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.981743] ? sysfs_kf_write+0x65/0x160
[ 1112.986944] ? sysfs_file_ops+0x140/0x140
[ 1112.992227] kernfs_fop_write_iter+0x335/0x510
[ 1112.998010] ? kernfs_ops+0xf0/0xf0
[ 1113.002722] new_sync_write+0x333/0x750
[ 1113.007821] ? __lock_acquire+0x508/0xc10
[ 1113.013117] ? new_sync_read+0x740/0x740
[ 1113.018315] ? lock_acquire.part.0+0xbc/0x260
[ 1113.024001] ? ksys_write+0x116/0x250
[ 1113.028903] vfs_write+0x657/0x9b0
[ 1113.033517] ksys_write+0x116/0x250
[ 1113.038226] ? __ia32_sys_read+0xc0/0xc0
[ 1113.043425] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.049012] ? rcu_is_watching+0x15/0xd0
[ 1113.054212] do_syscall_64+0xf3/0x650
[ 1113.059117] ? trace_hardirqs_off+0xd/0x30
[ 1113.064509] ? exc_page_fault+0xda/0xf0
[ 1113.069608] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.076074] RIP: 0033:0x7f2a8937b22e
[ 1113.080880] RSP: 002b:00007ffeddfcabd0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.090174] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2a8937b22e
[ 1113.098979] RDX: 0000000000000001 RSI: 000055b9adeb66b0 RDI: 0000000000000007
[ 1113.107786] RBP: 00007ffeddfcabe0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.116592] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2a891f3390
[ 1113.125398] R13: 000055b9adeb66b0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.134209]
[ 1113.137471] \x0aShowing all locks held in the system:
[ 1113.146006] 1 lock held by khungtaskd/185:
[ 1113.151400] #0: ffffffff940fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.162872] 2 locks held by systemd-journal/806:
[ 1113.168849] #0: ff1100180dfb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.178738] #1: ffffffff943029a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.191563] 4 locks held by python3/3873:
[ 1113.196858] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.206745] #1: ff11000161c74a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.217705] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.228862] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.238967] 4 locks held by python3/5887:
[ 1113.244265] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.254155] #1: ff110001ece78280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.265116] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.276270] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.286361]
[ 1113.288828] =============================================\x0a
[ 1235.460221] INFO: task python3:5887 blocked for more than 368 seconds.
[ 1235.468367] Not tainted 7.1.0-rc4-olrp-g5759c1b1736e #1
[ 1235.475616] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.485204] task:python3 state:D stack:25672 pid:5887 tgid:5887 ppid:5886 task_flags:0x400100 flags:0x00080002
[ 1235.498402] Call Trace:
[ 1235.501940]
[ 1235.505092] __schedule+0xcc3/0x19c0
[ 1235.509908] ? io_schedule_timeout+0x140/0x140
[ 1235.515691] ? find_held_lock+0x2b/0x80
[ 1235.520787] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.526661] schedule+0xd8/0x260
[ 1235.531077] schedule_preempt_disabled+0x18/0x30
[ 1235.537061] __mutex_lock+0x6af/0x2010
[ 1235.542066] ? threaded_store+0x132/0x220
[ 1235.547367] ? ww_mutex_lock+0x160/0x160
[ 1235.552565] ? selinux_inode_free_security+0x330/0x330
[ 1235.559133] ? cap_capable+0x1d6/0x3e0
[ 1235.564139] ? threaded_store+0x132/0x220
[ 1235.569436] threaded_store+0x132/0x220
[ 1235.574534] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.579833] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.585130] ? sysfs_kf_write+0x65/0x160
[ 1235.590331] ? sysfs_file_ops+0x140/0x140
[ 1235.595627] kernfs_fop_write_iter+0x335/0x510
[ 1235.601409] ? kernfs_ops+0xf0/0xf0
[ 1235.606118] new_sync_write+0x333/0x750
[ 1235.611222] ? __lock_acquire+0x508/0xc10
[ 1235.616516] ? new_sync_read+0x740/0x740
[ 1235.621717] ? lock_acquire.part.0+0xbc/0x260
[ 1235.627401] ? ksys_write+0x116/0x250
[ 1235.632310] vfs_write+0x657/0x9b0
[ 1235.636926] ksys_write+0x116/0x250
[ 1235.641638] ? __ia32_sys_read+0xc0/0xc0
[ 1235.646834] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.652424] ? rcu_is_watching+0x15/0xd0
[ 1235.657624] do_syscall_64+0xf3/0x650
[ 1235.662519] ? trace_hardirqs_off+0xd/0x30
[ 1235.667914] ? exc_page_fault+0xda/0xf0
[ 1235.673016] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.679481] RIP: 0033:0x7f7fc476822e
[ 1235.684290] RSP: 002b:00007ffc3fca7490 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.693587] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fc476822e
[ 1235.702396] RDX: 0000000000000001 RSI: 000055bae50706b0 RDI: 0000000000000007
[ 1235.711205] RBP: 00007ffc3fca74a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.720012] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fc45e0390
[ 1235.728821] R13: 000055bae50706b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.737635]
[ 1235.740930] INFO: task python3:5887 is blocked on a mutex likely owned by task python3:3873.
[ 1235.751203] task:python3 state:D stack:25672 pid:3873 tgid:3873 ppid:3872 task_flags:0x400100 flags:0x00080802
[ 1235.764407] Call Trace:
[ 1235.767944]
[ 1235.771095] __schedule+0xcc3/0x19c0
[ 1235.775907] ? io_schedule_timeout+0x140/0x140
[ 1235.781689] ? find_held_lock+0x2b/0x80
[ 1235.786792] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.792969] schedule+0xd8/0x260
[ 1235.797386] schedule_timeout+0x13f/0x290
[ 1235.802682] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.809051] ? trace_contention_end+0xb3/0x1a0
[ 1235.814833] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.820524] ? ww_mutex_lock+0x160/0x160
[ 1235.825723] msleep+0x1b/0x30
[ 1235.829840] napi_set_threaded+0x3a5/0x440
[ 1235.835233] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.840726] netif_set_threaded+0x257/0x400
[ 1235.846207] threaded_store+0x1b8/0x220
[ 1235.851308] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.856605] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.861890] ? sysfs_kf_write+0x65/0x160
[ 1235.867091] ? sysfs_file_ops+0x140/0x140
[ 1235.872383] kernfs_fop_write_iter+0x335/0x510
[ 1235.878165] ? kernfs_ops+0xf0/0xf0
[ 1235.882876] new_sync_write+0x333/0x750
[ 1235.887978] ? __lock_acquire+0x508/0xc10
[ 1235.893272] ? new_sync_read+0x740/0x740
[ 1235.898473] ? lock_acquire.part.0+0xbc/0x260
[ 1235.904160] ? ksys_write+0x116/0x250
[ 1235.909066] vfs_write+0x657/0x9b0
[ 1235.913678] ksys_write+0x116/0x250
[ 1235.918388] ? __ia32_sys_read+0xc0/0xc0
[ 1235.923586] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.929173] ? rcu_is_watching+0x15/0xd0
[ 1235.934373] do_syscall_64+0xf3/0x650
[ 1235.939269] ? trace_hardirqs_off+0xd/0x30
[ 1235.944659] ? exc_page_fault+0xda/0xf0
[ 1235.949758] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.956223] RIP: 0033:0x7f2a8937b22e
[ 1235.961028] RSP: 002b:00007ffeddfcabd0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.970326] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2a8937b22e
[ 1235.979132] RDX: 0000000000000001 RSI: 000055b9adeb66b0 RDI: 0000000000000007
[ 1235.987937] RBP: 00007ffeddfcabe0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.996745] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2a891f3390
[ 1236.005550] R13: 000055b9adeb66b0 R14: 0000000000000007 R15: 0000000000000003
[ 1236.014360]
[ 1236.017625] \x0aShowing all locks held in the system:
[ 1236.026158] 1 lock held by khungtaskd/185:
[ 1236.031552] #0: ffffffff940fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.043024] 3 locks held by systemd-journal/806:
[ 1236.049000] #0: ff1100180e0b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.058889] #1: ffffffff943029a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.071702] #2: ff1100015acf38e0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.083260] 4 locks held by python3/3873:
[ 1236.088554] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.098440] #1: ff11000161c74a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.109401] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.120550] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.130638] 4 locks held by python3/5887:
[ 1236.135931] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.145821] #1: ff110001ece78280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.156781] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.167936] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.178027]
[ 1236.180493] =============================================\x0a
[ 1297.732331] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0
[ 1358.342346] INFO: task python3:5887 blocked for more than 491 seconds.
[ 1358.350494] Not tainted 7.1.0-rc4-olrp-g5759c1b1736e #1
[ 1358.357745] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.367334] task:python3 state:D stack:25672 pid:5887 tgid:5887 ppid:5886 task_flags:0x400100 flags:0x00080002
[ 1358.380540] Call Trace:
[ 1358.384080]
[ 1358.387230] __schedule+0xcc3/0x19c0
[ 1358.392046] ? io_schedule_timeout+0x140/0x140
[ 1358.397835] ? find_held_lock+0x2b/0x80
[ 1358.402942] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.408840] schedule+0xd8/0x260
[ 1358.413266] schedule_preempt_disabled+0x18/0x30
[ 1358.419247] __mutex_lock+0x6af/0x2010
[ 1358.424254] ? threaded_store+0x132/0x220
[ 1358.429555] ? ww_mutex_lock+0x160/0x160
[ 1358.434753] ? selinux_inode_free_security+0x330/0x330
[ 1358.441323] ? cap_capable+0x1d6/0x3e0
[ 1358.446332] ? threaded_store+0x132/0x220
[ 1358.451630] threaded_store+0x132/0x220
[ 1358.456733] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.462031] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.467331] ? sysfs_kf_write+0x65/0x160
[ 1358.472529] ? sysfs_file_ops+0x140/0x140
[ 1358.477828] kernfs_fop_write_iter+0x335/0x510
[ 1358.483615] ? kernfs_ops+0xf0/0xf0
[ 1358.488328] new_sync_write+0x333/0x750
[ 1358.493434] ? __lock_acquire+0x508/0xc10
[ 1358.498730] ? new_sync_read+0x740/0x740
[ 1358.504015] ? lock_acquire.part.0+0xbc/0x260
[ 1358.509702] ? ksys_write+0x116/0x250
[ 1358.514614] vfs_write+0x657/0x9b0
[ 1358.519228] ksys_write+0x116/0x250
[ 1358.523931] ? __ia32_sys_read+0xc0/0xc0
[ 1358.529128] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.534716] ? rcu_is_watching+0x15/0xd0
[ 1358.539917] do_syscall_64+0xf3/0x650
[ 1358.544814] ? trace_hardirqs_off+0xd/0x30
[ 1358.550209] ? exc_page_fault+0xda/0xf0
[ 1358.555313] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.561782] RIP: 0033:0x7f7fc476822e
[ 1358.566594] RSP: 002b:00007ffc3fca7490 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.575891] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fc476822e
[ 1358.584700] RDX: 0000000000000001 RSI: 000055bae50706b0 RDI: 0000000000000007
[ 1358.593507] RBP: 00007ffc3fca74a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.602314] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fc45e0390
[ 1358.611114] R13: 000055bae50706b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.619932]
[ 1358.623244] INFO: task python3:5887 is blocked on a mutex likely owned by task python3:3873.
[ 1358.633517] task:python3 state:D stack:25672 pid:3873 tgid:3873 ppid:3872 task_flags:0x400100 flags:0x00080802
[ 1358.646722] Call Trace:
[ 1358.650267]
[ 1358.653418] __schedule+0xcc3/0x19c0
[ 1358.658224] ? io_schedule_timeout+0x140/0x140
[ 1358.664011] ? find_held_lock+0x2b/0x80
[ 1358.669115] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.675297] schedule+0xd8/0x260
[ 1358.679716] schedule_timeout+0x13f/0x290
[ 1358.685014] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.691385] ? trace_contention_end+0xb3/0x1a0
[ 1358.697173] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.702864] ? ww_mutex_lock+0x160/0x160
[ 1358.708067] msleep+0x1b/0x30
[ 1358.712201] napi_set_threaded+0x3a5/0x440
[ 1358.717599] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.723095] netif_set_threaded+0x257/0x400
[ 1358.728590] threaded_store+0x1b8/0x220
[ 1358.733696] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.738994] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.744293] ? sysfs_kf_write+0x65/0x160
[ 1358.749491] ? sysfs_file_ops+0x140/0x140
[ 1358.754790] kernfs_fop_write_iter+0x335/0x510
[ 1358.760575] ? kernfs_ops+0xf0/0xf0
[ 1358.765288] new_sync_write+0x333/0x750
[ 1358.770391] ? __lock_acquire+0x508/0xc10
[ 1358.775686] ? new_sync_read+0x740/0x740
[ 1358.780889] ? lock_acquire.part.0+0xbc/0x260
[ 1358.786575] ? ksys_write+0x116/0x250
[ 1358.791488] vfs_write+0x657/0x9b0
[ 1358.796105] ksys_write+0x116/0x250
[ 1358.800883] ? __ia32_sys_read+0xc0/0xc0
[ 1358.806084] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.811675] ? rcu_is_watching+0x15/0xd0
[ 1358.816878] do_syscall_64+0xf3/0x650
[ 1358.821788] ? trace_hardirqs_off+0xd/0x30
[ 1358.827185] ? exc_page_fault+0xda/0xf0
[ 1358.832287] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.838753] RIP: 0033:0x7f2a8937b22e
[ 1358.843563] RSP: 002b:00007ffeddfcabd0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.852863] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2a8937b22e
[ 1358.861672] RDX: 0000000000000001 RSI: 000055b9adeb66b0 RDI: 0000000000000007
[ 1358.870479] RBP: 00007ffeddfcabe0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.879280] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2a891f3390
[ 1358.888092] R13: 000055b9adeb66b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.896905]
[ 1358.900178] \x0aShowing all locks held in the system:
[ 1358.908726] 1 lock held by khungtaskd/185:
[ 1358.914121] #0: ffffffff940fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.925610] 4 locks held by python3/3873:
[ 1358.930907] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.940797] #1: ff11000161c74a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.951760] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.962917] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.973018] 4 locks held by python3/5887:
[ 1358.978316] #0: ff1100011db613e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.988210] #1: ff110001ece78280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.999175] #2: ff110001224f6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.010333] #3: ff11000122539d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.020422] 1 lock held by systemd/7772:
[ 1359.025622]
[ 1359.028088] =============================================\x0a