[ 788.338722] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 788.341889] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 788.472355] kselftest: Running tests in drivers/net
[ 989.660287] INFO: task python3:5829 blocked for more than 122 seconds.
[ 989.668519] Not tainted 7.1.0-rc3-howg-g93f407eee4a2 #1
[ 989.676370] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.685957] task:python3 state:D stack:25672 pid:5829 tgid:5829 ppid:5828 task_flags:0x400100 flags:0x00080000
[ 989.699160] Call Trace:
[ 989.702688]
[ 989.705838] __schedule+0xcc3/0x19c0
[ 989.710652] ? io_schedule_timeout+0x140/0x140
[ 989.716435] ? find_held_lock+0x2b/0x80
[ 989.721539] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.727424] schedule+0xd4/0x260
[ 989.731841] schedule_preempt_disabled+0x18/0x30
[ 989.737818] __mutex_lock+0x6a3/0x2000
[ 989.742822] ? threaded_store+0x132/0x220
[ 989.748121] ? ww_mutex_lock+0x160/0x160
[ 989.753317] ? selinux_inode_free_security+0x330/0x330
[ 989.759884] ? cap_capable+0x1d6/0x3e0
[ 989.764880] ? threaded_store+0x132/0x220
[ 989.770177] threaded_store+0x132/0x220
[ 989.775272] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.780568] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.785867] ? sysfs_kf_write+0x65/0x160
[ 989.791054] ? sysfs_file_ops+0x140/0x140
[ 989.796349] kernfs_fop_write_iter+0x335/0x510
[ 989.802134] ? kernfs_ops+0xf0/0xf0
[ 989.806834] new_sync_write+0x333/0x750
[ 989.811937] ? __lock_acquire+0x508/0xc10
[ 989.817234] ? new_sync_read+0x740/0x740
[ 989.822434] ? lock_acquire.part.0+0xbc/0x260
[ 989.828118] ? ksys_write+0x116/0x250
[ 989.833029] vfs_write+0x657/0x9b0
[ 989.837644] ksys_write+0x116/0x250
[ 989.842356] ? __ia32_sys_read+0xc0/0xc0
[ 989.847547] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.853137] ? rcu_is_watching+0x15/0xd0
[ 989.858336] do_syscall_64+0xf3/0x650
[ 989.863238] ? trace_hardirqs_off+0xd/0x30
[ 989.868634] ? exc_page_fault+0xda/0xf0
[ 989.873735] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.880203] RIP: 0033:0x7f7fa3d5222e
[ 989.885012] RSP: 002b:00007ffeab211890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.894307] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa3d5222e
[ 989.903115] RDX: 0000000000000001 RSI: 00005629c4cb3cd0 RDI: 0000000000000007
[ 989.911919] RBP: 00007ffeab2118a0 R08: 0000000000000000 R09: 0000000000000000
[ 989.920727] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa3bca390
[ 989.929524] R13: 00005629c4cb3cd0 R14: 0000000000000007 R15: 0000000000000003
[ 989.938343]
[ 989.941611] INFO: task python3:5829 is blocked on a mutex likely owned by task python3:3780.
[ 989.951884] task:python3 state:D stack:25672 pid:3780 tgid:3780 ppid:3779 task_flags:0x400100 flags:0x00080802
[ 989.965082] Call Trace:
[ 989.968619]
[ 989.971771] __schedule+0xcc3/0x19c0
[ 989.976582] ? io_schedule_timeout+0x140/0x140
[ 989.982365] ? find_held_lock+0x2b/0x80
[ 989.987470] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.993645] schedule+0xd4/0x260
[ 989.998062] schedule_timeout+0x13f/0x290
[ 990.003357] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.009727] ? trace_contention_end+0xb3/0x1a0
[ 990.015512] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.021199] ? ww_mutex_lock+0x160/0x160
[ 990.026399] msleep+0x1b/0x30
[ 990.030525] napi_set_threaded+0x3a5/0x440
[ 990.035917] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.041409] netif_set_threaded+0x257/0x400
[ 990.046901] threaded_store+0x1b8/0x220
[ 990.052000] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.057296] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.062590] ? sysfs_kf_write+0x65/0x160
[ 990.067790] ? sysfs_file_ops+0x140/0x140
[ 990.073085] kernfs_fop_write_iter+0x335/0x510
[ 990.078869] ? kernfs_ops+0xf0/0xf0
[ 990.083578] new_sync_write+0x333/0x750
[ 990.088680] ? __lock_acquire+0x508/0xc10
[ 990.093975] ? new_sync_read+0x740/0x740
[ 990.099175] ? lock_acquire.part.0+0xbc/0x260
[ 990.104861] ? ksys_write+0x116/0x250
[ 990.109768] vfs_write+0x657/0x9b0
[ 990.114381] ksys_write+0x116/0x250
[ 990.119089] ? __ia32_sys_read+0xc0/0xc0
[ 990.124286] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.129873] ? rcu_is_watching+0x15/0xd0
[ 990.135071] do_syscall_64+0xf3/0x650
[ 990.139974] ? trace_hardirqs_off+0xd/0x30
[ 990.145368] ? exc_page_fault+0xda/0xf0
[ 990.150470] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.156937] RIP: 0033:0x7f2deab7e22e
[ 990.161745] RSP: 002b:00007ffca516ed20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.171041] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2deab7e22e
[ 990.179841] RDX: 0000000000000001 RSI: 00005601fe596cd0 RDI: 0000000000000007
[ 990.188650] RBP: 00007ffca516ed30 R08: 0000000000000000 R09: 0000000000000000
[ 990.197454] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2dea9f6390
[ 990.206262] R13: 00005601fe596cd0 R14: 0000000000000007 R15: 0000000000000003
[ 990.215072]
[ 990.218330] \x0aShowing all locks held in the system:
[ 990.226867] 1 lock held by khungtaskd/184:
[ 990.232258] #0: ffffffff944fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.243731] 4 locks held by systemd-journal/803:
[ 990.249706] #0: ff1100180e7372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.259592] #1: ffffffff947033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: lockref_get_not_dead+0xe/0x60
[ 990.272404] #2: ff110001898804f0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.283941] #3: ff1100019d594f58 (&xa->xa_lock#5){-.-.}-{3:3}, at: __folio_mark_dirty+0x9d/0x2c0
[ 990.294716] 4 locks held by python3/3780:
[ 990.299999] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.309889] #1: ff110001755ddc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.320848] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.332005] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.342090] 4 locks held by python3/5829:
[ 990.347384] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.357271] #1: ff110001f0cc9a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.368232] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.379389] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.389474]
[ 990.391939] =============================================\x0a
[ 1112.542394] INFO: task python3:5829 blocked for more than 245 seconds.
[ 1112.550540] Not tainted 7.1.0-rc3-howg-g93f407eee4a2 #1
[ 1112.557790] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.567380] task:python3 state:D stack:25672 pid:5829 tgid:5829 ppid:5828 task_flags:0x400100 flags:0x00080000
[ 1112.580584] Call Trace:
[ 1112.584123]
[ 1112.587275] __schedule+0xcc3/0x19c0
[ 1112.592091] ? io_schedule_timeout+0x140/0x140
[ 1112.597875] ? find_held_lock+0x2b/0x80
[ 1112.602982] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.608865] schedule+0xd4/0x260
[ 1112.613271] schedule_preempt_disabled+0x18/0x30
[ 1112.619250] __mutex_lock+0x6a3/0x2000
[ 1112.624252] ? threaded_store+0x132/0x220
[ 1112.629549] ? ww_mutex_lock+0x160/0x160
[ 1112.634746] ? selinux_inode_free_security+0x330/0x330
[ 1112.641316] ? cap_capable+0x1d6/0x3e0
[ 1112.646324] ? threaded_store+0x132/0x220
[ 1112.651621] threaded_store+0x132/0x220
[ 1112.656721] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.662018] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.667317] ? sysfs_kf_write+0x65/0x160
[ 1112.672520] ? sysfs_file_ops+0x140/0x140
[ 1112.677815] kernfs_fop_write_iter+0x335/0x510
[ 1112.683597] ? kernfs_ops+0xf0/0xf0
[ 1112.688308] new_sync_write+0x333/0x750
[ 1112.693402] ? __lock_acquire+0x508/0xc10
[ 1112.698697] ? new_sync_read+0x740/0x740
[ 1112.703899] ? lock_acquire.part.0+0xbc/0x260
[ 1112.709585] ? ksys_write+0x116/0x250
[ 1112.714493] vfs_write+0x657/0x9b0
[ 1112.719105] ksys_write+0x116/0x250
[ 1112.723817] ? __ia32_sys_read+0xc0/0xc0
[ 1112.729014] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.734604] ? rcu_is_watching+0x15/0xd0
[ 1112.739804] do_syscall_64+0xf3/0x650
[ 1112.744699] ? trace_hardirqs_off+0xd/0x30
[ 1112.750096] ? exc_page_fault+0xda/0xf0
[ 1112.755190] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.761655] RIP: 0033:0x7f7fa3d5222e
[ 1112.766464] RSP: 002b:00007ffeab211890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.775762] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa3d5222e
[ 1112.784573] RDX: 0000000000000001 RSI: 00005629c4cb3cd0 RDI: 0000000000000007
[ 1112.793380] RBP: 00007ffeab2118a0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.802186] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa3bca390
[ 1112.810995] R13: 00005629c4cb3cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.819811]
[ 1112.823083] INFO: task python3:5829 is blocked on a mutex likely owned by task python3:3780.
[ 1112.833352] task:python3 state:D stack:25672 pid:3780 tgid:3780 ppid:3779 task_flags:0x400100 flags:0x00080802
[ 1112.846556] Call Trace:
[ 1112.850094]
[ 1112.853241] __schedule+0xcc3/0x19c0
[ 1112.858050] ? io_schedule_timeout+0x140/0x140
[ 1112.863831] ? find_held_lock+0x2b/0x80
[ 1112.868930] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.875110] schedule+0xd4/0x260
[ 1112.879529] schedule_timeout+0x13f/0x290
[ 1112.884825] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.891198] ? trace_contention_end+0xb3/0x1a0
[ 1112.896983] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.902673] ? ww_mutex_lock+0x160/0x160
[ 1112.907877] msleep+0x1b/0x30
[ 1112.907882] napi_set_threaded+0x3a5/0x440
[ 1112.907886] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.907893] netif_set_threaded+0x257/0x400
[ 1112.928419] threaded_store+0x1b8/0x220
[ 1112.933521] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.938822] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.944117] ? sysfs_kf_write+0x65/0x160
[ 1112.949318] ? sysfs_file_ops+0x140/0x140
[ 1112.954615] kernfs_fop_write_iter+0x335/0x510
[ 1112.960400] ? kernfs_ops+0xf0/0xf0
[ 1112.965103] new_sync_write+0x333/0x750
[ 1112.970206] ? __lock_acquire+0x508/0xc10
[ 1112.975503] ? new_sync_read+0x740/0x740
[ 1112.980705] ? lock_acquire.part.0+0xbc/0x260
[ 1112.986393] ? ksys_write+0x116/0x250
[ 1112.991303] vfs_write+0x657/0x9b0
[ 1112.995920] ksys_write+0x116/0x250
[ 1113.000634] ? __ia32_sys_read+0xc0/0xc0
[ 1113.005833] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.011423] ? rcu_is_watching+0x15/0xd0
[ 1113.016622] do_syscall_64+0xf3/0x650
[ 1113.021528] ? trace_hardirqs_off+0xd/0x30
[ 1113.026924] ? exc_page_fault+0xda/0xf0
[ 1113.032026] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.038486] RIP: 0033:0x7f2deab7e22e
[ 1113.043295] RSP: 002b:00007ffca516ed20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.052585] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2deab7e22e
[ 1113.061391] RDX: 0000000000000001 RSI: 00005601fe596cd0 RDI: 0000000000000007
[ 1113.070203] RBP: 00007ffca516ed30 R08: 0000000000000000 R09: 0000000000000000
[ 1113.079010] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2dea9f6390
[ 1113.087819] R13: 00005601fe596cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.096633]
[ 1113.099896] \x0aShowing all locks held in the system:
[ 1113.108434] 1 lock held by khungtaskd/184:
[ 1113.113828] #0: ffffffff944fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.125309] 4 locks held by python3/3780:
[ 1113.130607] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.140491] #1: ff110001755ddc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.151453] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.162609] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.172702] 4 locks held by python3/5829:
[ 1113.178000] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.187890] #1: ff110001f0cc9a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.198850] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.210011] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.220101] 5 locks held by (systemd-user-)/6880:
[ 1113.226169] #0: ff110001121d43e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.236059] #1: ff11000149ad5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.247021] #2: ffffffff94569de0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1113.257691] #3: ffffffff94368970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1113.269334] #4: ffffffff94569b50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1113.281763]
[ 1113.284234] =============================================\x0a
[ 1235.423509] INFO: task python3:5829 blocked for more than 368 seconds.
[ 1235.431657] Not tainted 7.1.0-rc3-howg-g93f407eee4a2 #1
[ 1235.438897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.448478] task:python3 state:D stack:25672 pid:5829 tgid:5829 ppid:5828 task_flags:0x400100 flags:0x00080002
[ 1235.461681] Call Trace:
[ 1235.465211]
[ 1235.468359] __schedule+0xcc3/0x19c0
[ 1235.473173] ? io_schedule_timeout+0x140/0x140
[ 1235.478958] ? find_held_lock+0x2b/0x80
[ 1235.484065] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.489948] schedule+0xd4/0x260
[ 1235.494357] schedule_preempt_disabled+0x18/0x30
[ 1235.500337] __mutex_lock+0x6a3/0x2000
[ 1235.505329] ? threaded_store+0x132/0x220
[ 1235.510628] ? ww_mutex_lock+0x160/0x160
[ 1235.515827] ? selinux_inode_free_security+0x330/0x330
[ 1235.522396] ? cap_capable+0x1d6/0x3e0
[ 1235.527404] ? threaded_store+0x132/0x220
[ 1235.532699] threaded_store+0x132/0x220
[ 1235.537799] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.543095] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.548385] ? sysfs_kf_write+0x65/0x160
[ 1235.553586] ? sysfs_file_ops+0x140/0x140
[ 1235.558869] kernfs_fop_write_iter+0x335/0x510
[ 1235.564653] ? kernfs_ops+0xf0/0xf0
[ 1235.569364] new_sync_write+0x333/0x750
[ 1235.574465] ? __lock_acquire+0x508/0xc10
[ 1235.579761] ? new_sync_read+0x740/0x740
[ 1235.584960] ? lock_acquire.part.0+0xbc/0x260
[ 1235.590643] ? ksys_write+0x116/0x250
[ 1235.595552] vfs_write+0x657/0x9b0
[ 1235.600166] ksys_write+0x116/0x250
[ 1235.604875] ? __ia32_sys_read+0xc0/0xc0
[ 1235.610072] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.615661] ? rcu_is_watching+0x15/0xd0
[ 1235.620862] do_syscall_64+0xf3/0x650
[ 1235.625767] ? trace_hardirqs_off+0xd/0x30
[ 1235.631160] ? exc_page_fault+0xda/0xf0
[ 1235.636260] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.642726] RIP: 0033:0x7f7fa3d5222e
[ 1235.647533] RSP: 002b:00007ffeab211890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.656827] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa3d5222e
[ 1235.665634] RDX: 0000000000000001 RSI: 00005629c4cb3cd0 RDI: 0000000000000007
[ 1235.674438] RBP: 00007ffeab2118a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.683245] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa3bca390
[ 1235.692050] R13: 00005629c4cb3cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.700864]
[ 1235.704136] INFO: task python3:5829 is blocked on a mutex likely owned by task python3:3780.
[ 1235.714408] task:python3 state:D stack:25672 pid:3780 tgid:3780 ppid:3779 task_flags:0x400100 flags:0x00080802
[ 1235.727606] Call Trace:
[ 1235.731141]
[ 1235.734290] __schedule+0xcc3/0x19c0
[ 1235.739098] ? io_schedule_timeout+0x140/0x140
[ 1235.744881] ? find_held_lock+0x2b/0x80
[ 1235.749983] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.756149] schedule+0xd4/0x260
[ 1235.760564] schedule_timeout+0x13f/0x290
[ 1235.765858] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.772226] ? trace_contention_end+0xb3/0x1a0
[ 1235.778010] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.783699] ? ww_mutex_lock+0x160/0x160
[ 1235.788898] msleep+0x1b/0x30
[ 1235.793021] napi_set_threaded+0x3a5/0x440
[ 1235.798417] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.803913] netif_set_threaded+0x257/0x400
[ 1235.809404] threaded_store+0x1b8/0x220
[ 1235.814504] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.819800] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.825097] ? sysfs_kf_write+0x65/0x160
[ 1235.830295] ? sysfs_file_ops+0x140/0x140
[ 1235.835589] kernfs_fop_write_iter+0x335/0x510
[ 1235.841373] ? kernfs_ops+0xf0/0xf0
[ 1235.846082] new_sync_write+0x333/0x750
[ 1235.851183] ? __lock_acquire+0x508/0xc10
[ 1235.856476] ? new_sync_read+0x740/0x740
[ 1235.861675] ? lock_acquire.part.0+0xbc/0x260
[ 1235.867359] ? ksys_write+0x116/0x250
[ 1235.872266] vfs_write+0x657/0x9b0
[ 1235.876880] ksys_write+0x116/0x250
[ 1235.881592] ? __ia32_sys_read+0xc0/0xc0
[ 1235.886789] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.892377] ? rcu_is_watching+0x15/0xd0
[ 1235.897575] do_syscall_64+0xf3/0x650
[ 1235.902478] ? trace_hardirqs_off+0xd/0x30
[ 1235.907871] ? exc_page_fault+0xda/0xf0
[ 1235.912971] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.919438] RIP: 0033:0x7f2deab7e22e
[ 1235.924246] RSP: 002b:00007ffca516ed20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.933539] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2deab7e22e
[ 1235.942343] RDX: 0000000000000001 RSI: 00005601fe596cd0 RDI: 0000000000000007
[ 1235.951153] RBP: 00007ffca516ed30 R08: 0000000000000000 R09: 0000000000000000
[ 1235.959962] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2dea9f6390
[ 1235.968769] R13: 00005601fe596cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.977581]
[ 1235.980842] \x0aShowing all locks held in the system:
[ 1235.989381] 1 lock held by khungtaskd/184:
[ 1235.994771] #0: ffffffff944fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.006250] 4 locks held by python3/3780:
[ 1236.011545] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.021435] #1: ff110001755ddc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.032385] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.043540] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.053632] 4 locks held by python3/5829:
[ 1236.058923] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.068812] #1: ff110001f0cc9a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.079762] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.090918] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.101009]
[ 1236.103473] =============================================\x0a
[ 1358.304653] INFO: task python3:5829 blocked for more than 491 seconds.
[ 1358.312810] Not tainted 7.1.0-rc3-howg-g93f407eee4a2 #1
[ 1358.320070] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.329662] task:python3 state:D stack:25672 pid:5829 tgid:5829 ppid:5828 task_flags:0x400100 flags:0x00080002
[ 1358.342867] Call Trace:
[ 1358.346404]
[ 1358.349555] __schedule+0xcc3/0x19c0
[ 1358.354370] ? io_schedule_timeout+0x140/0x140
[ 1358.360156] ? find_held_lock+0x2b/0x80
[ 1358.365263] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.371160] schedule+0xd4/0x260
[ 1358.375583] schedule_preempt_disabled+0x18/0x30
[ 1358.381561] __mutex_lock+0x6a3/0x2000
[ 1358.386565] ? threaded_store+0x132/0x220
[ 1358.391866] ? ww_mutex_lock+0x160/0x160
[ 1358.397062] ? selinux_inode_free_security+0x330/0x330
[ 1358.403632] ? cap_capable+0x1d6/0x3e0
[ 1358.408637] ? threaded_store+0x132/0x220
[ 1358.413923] threaded_store+0x132/0x220
[ 1358.419025] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.424312] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.429608] ? sysfs_kf_write+0x65/0x160
[ 1358.434806] ? sysfs_file_ops+0x140/0x140
[ 1358.440101] kernfs_fop_write_iter+0x335/0x510
[ 1358.445885] ? kernfs_ops+0xf0/0xf0
[ 1358.450594] new_sync_write+0x333/0x750
[ 1358.455697] ? __lock_acquire+0x508/0xc10
[ 1358.460991] ? new_sync_read+0x740/0x740
[ 1358.466191] ? lock_acquire.part.0+0xbc/0x260
[ 1358.471876] ? ksys_write+0x116/0x250
[ 1358.476784] vfs_write+0x657/0x9b0
[ 1358.481399] ksys_write+0x116/0x250
[ 1358.486101] ? __ia32_sys_read+0xc0/0xc0
[ 1358.491300] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.496891] ? rcu_is_watching+0x15/0xd0
[ 1358.502093] do_syscall_64+0xf3/0x650
[ 1358.506999] ? trace_hardirqs_off+0xd/0x30
[ 1358.512392] ? exc_page_fault+0xda/0xf0
[ 1358.517494] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.523961] RIP: 0033:0x7f7fa3d5222e
[ 1358.528771] RSP: 002b:00007ffeab211890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.538066] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa3d5222e
[ 1358.546874] RDX: 0000000000000001 RSI: 00005629c4cb3cd0 RDI: 0000000000000007
[ 1358.555672] RBP: 00007ffeab2118a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.564485] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa3bca390
[ 1358.573294] R13: 00005629c4cb3cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.582105]
[ 1358.585418] INFO: task python3:5829 is blocked on a mutex likely owned by task python3:3780.
[ 1358.595689] task:python3 state:D stack:25672 pid:3780 tgid:3780 ppid:3779 task_flags:0x400100 flags:0x00080802
[ 1358.608888] Call Trace:
[ 1358.612424]
[ 1358.615573] __schedule+0xcc3/0x19c0
[ 1358.620383] ? io_schedule_timeout+0x140/0x140
[ 1358.626166] ? find_held_lock+0x2b/0x80
[ 1358.631269] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.637449] schedule+0xd4/0x260
[ 1358.641866] schedule_timeout+0x13f/0x290
[ 1358.647160] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.653528] ? trace_contention_end+0xb3/0x1a0
[ 1358.659312] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.664992] ? ww_mutex_lock+0x160/0x160
[ 1358.670192] msleep+0x1b/0x30
[ 1358.674315] napi_set_threaded+0x3a5/0x440
[ 1358.679712] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.685195] netif_set_threaded+0x257/0x400
[ 1358.690688] threaded_store+0x1b8/0x220
[ 1358.695788] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.701083] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.706376] ? sysfs_kf_write+0x65/0x160
[ 1358.711576] ? sysfs_file_ops+0x140/0x140
[ 1358.716863] kernfs_fop_write_iter+0x335/0x510
[ 1358.722648] ? kernfs_ops+0xf0/0xf0
[ 1358.727358] new_sync_write+0x333/0x750
[ 1358.732459] ? __lock_acquire+0x508/0xc10
[ 1358.737754] ? new_sync_read+0x740/0x740
[ 1358.742954] ? lock_acquire.part.0+0xbc/0x260
[ 1358.748639] ? ksys_write+0x116/0x250
[ 1358.753547] vfs_write+0x657/0x9b0
[ 1358.758163] ksys_write+0x116/0x250
[ 1358.762876] ? __ia32_sys_read+0xc0/0xc0
[ 1358.768063] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.773648] ? rcu_is_watching+0x15/0xd0
[ 1358.778846] do_syscall_64+0xf3/0x650
[ 1358.783750] ? trace_hardirqs_off+0xd/0x30
[ 1358.789143] ? exc_page_fault+0xda/0xf0
[ 1358.794244] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.800712] RIP: 0033:0x7f2deab7e22e
[ 1358.805508] RSP: 002b:00007ffca516ed20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.814804] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f2deab7e22e
[ 1358.823608] RDX: 0000000000000001 RSI: 00005601fe596cd0 RDI: 0000000000000007
[ 1358.832416] RBP: 00007ffca516ed30 R08: 0000000000000000 R09: 0000000000000000
[ 1358.841225] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2dea9f6390
[ 1358.850033] R13: 00005601fe596cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.858837]
[ 1358.862105] \x0aShowing all locks held in the system:
[ 1358.870654] 1 lock held by khungtaskd/184:
[ 1358.876037] #0: ffffffff944fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.887513] 3 locks held by systemd-journal/803:
[ 1358.893490] #0: ff1100180e6372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.903377] #1: ffffffff947033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.916191] #2: ff11000189880de0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.927749] 4 locks held by python3/3780:
[ 1358.933045] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.942933] #1: ff110001755ddc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.953894] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.965051] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.975152] 4 locks held by python3/5829:
[ 1358.980448] #0: ff1100012eecc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.990336] #1: ff110001f0cc9a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.001294] #2: ff1100012189eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.012448] #3: ff110001218d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.022534] 5 locks held by systemd/7672:
[ 1359.027825] #0: ff110001121d43e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.037711] #1: ff11000141b58e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.048672] #2: ffffffff94569de0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1359.059340] #3: ffffffff94368970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1359.070985] #4: ffffffff94569b50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1359.083409]
[ 1359.085877] =============================================\x0a