[ 793.180574] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 793.182988] nipa-hw-worker: [5/44] Retrying drivers/net:napi_threaded.py
[ 793.336267] kselftest: Running tests in drivers/net
[ 969.213926] kworker/u130:6 (852) used greatest stack depth: 21456 bytes left
[ 989.693189] INFO: task python3:5948 blocked for more than 122 seconds.
[ 989.701380] Not tainted 7.1.0-rc1-kcuo-ga15578502849 #1
[ 989.708842] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.718433] task:python3 state:D stack:25032 pid:5948 tgid:5948 ppid:5947 task_flags:0x400100 flags:0x00080000
[ 989.731638] Call Trace:
[ 989.735175]
[ 989.738324] __schedule+0xcba/0x1940
[ 989.743141] ? io_schedule_timeout+0x140/0x140
[ 989.748924] ? find_held_lock+0x2b/0x80
[ 989.754028] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.759912] schedule+0xd4/0x260
[ 989.764328] schedule_preempt_disabled+0x18/0x30
[ 989.770306] __mutex_lock+0x6a3/0x2000
[ 989.775311] ? threaded_store+0x132/0x220
[ 989.780613] ? ww_mutex_lock+0x160/0x160
[ 989.785810] ? selinux_inode_free_security+0x330/0x330
[ 989.792380] ? cap_capable+0x1d6/0x3e0
[ 989.797386] ? threaded_store+0x132/0x220
[ 989.802681] threaded_store+0x132/0x220
[ 989.807784] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.813082] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.818380] ? sysfs_kf_write+0x65/0x160
[ 989.823578] ? sysfs_file_ops+0x140/0x140
[ 989.828872] kernfs_fop_write_iter+0x335/0x510
[ 989.834655] ? kernfs_ops+0xf0/0xf0
[ 989.839366] new_sync_write+0x333/0x750
[ 989.844469] ? __lock_acquire+0x508/0xc10
[ 989.849767] ? new_sync_read+0x740/0x740
[ 989.854966] ? lock_acquire.part.0+0xbc/0x260
[ 989.860650] ? ksys_write+0x116/0x250
[ 989.865559] vfs_write+0x657/0x9b0
[ 989.870172] ksys_write+0x116/0x250
[ 989.874882] ? __ia32_sys_read+0xc0/0xc0
[ 989.880082] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.885671] ? rcu_is_watching+0x15/0xd0
[ 989.890874] do_syscall_64+0xf3/0x650
[ 989.895779] ? trace_hardirqs_off+0xd/0x30
[ 989.901172] ? exc_page_fault+0xda/0xf0
[ 989.906271] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.912739] RIP: 0033:0x7f6197d7122e
[ 989.917547] RSP: 002b:00007ffcac5389f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.926845] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6197d7122e
[ 989.935651] RDX: 0000000000000001 RSI: 000055a0b2e4da10 RDI: 0000000000000007
[ 989.944459] RBP: 00007ffcac538a00 R08: 0000000000000000 R09: 0000000000000000
[ 989.953258] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6197be9390
[ 989.962066] R13: 000055a0b2e4da10 R14: 0000000000000007 R15: 0000000000000003
[ 989.970871]
[ 989.974175] INFO: task python3:5948 is blocked on a mutex likely owned by task python3:3915.
[ 989.984446] task:python3 state:D stack:25472 pid:3915 tgid:3915 ppid:3914 task_flags:0x400100 flags:0x00080802
[ 989.997647] Call Trace:
[ 990.001185]
[ 990.004336] __schedule+0xcba/0x1940
[ 990.009136] ? io_schedule_timeout+0x140/0x140
[ 990.014919] ? find_held_lock+0x2b/0x80
[ 990.020022] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.026198] schedule+0xd4/0x260
[ 990.030615] schedule_timeout+0x13f/0x290
[ 990.035910] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.042278] ? trace_contention_end+0xb3/0x1a0
[ 990.048063] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.053762] ? ww_mutex_lock+0x160/0x160
[ 990.058964] msleep+0x1b/0x30
[ 990.063086] napi_set_threaded+0x3a5/0x440
[ 990.068482] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.073967] netif_set_threaded+0x257/0x400
[ 990.079459] threaded_store+0x1b8/0x220
[ 990.084559] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.089855] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.095151] ? sysfs_kf_write+0x65/0x160
[ 990.100349] ? sysfs_file_ops+0x140/0x140
[ 990.105645] kernfs_fop_write_iter+0x335/0x510
[ 990.111431] ? kernfs_ops+0xf0/0xf0
[ 990.116139] new_sync_write+0x333/0x750
[ 990.121240] ? __lock_acquire+0x508/0xc10
[ 990.126533] ? new_sync_read+0x740/0x740
[ 990.131732] ? lock_acquire.part.0+0xbc/0x260
[ 990.137416] ? ksys_write+0x116/0x250
[ 990.142323] vfs_write+0x657/0x9b0
[ 990.146937] ksys_write+0x116/0x250
[ 990.151649] ? __ia32_sys_read+0xc0/0xc0
[ 990.156845] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.162433] ? rcu_is_watching+0x15/0xd0
[ 990.167631] do_syscall_64+0xf3/0x650
[ 990.172533] ? irq_exit_rcu+0x1a/0x30
[ 990.177439] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.183906] RIP: 0033:0x7fa88734c22e
[ 990.188711] RSP: 002b:00007ffcd8027df0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.198005] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa88734c22e
[ 990.206810] RDX: 0000000000000001 RSI: 000055b645d4ea10 RDI: 0000000000000007
[ 990.215618] RBP: 00007ffcd8027e00 R08: 0000000000000000 R09: 0000000000000000
[ 990.224425] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8871c4390
[ 990.233233] R13: 000055b645d4ea10 R14: 0000000000000007 R15: 0000000000000003
[ 990.242043]
[ 990.245301] \x0aShowing all locks held in the system:
[ 990.253842] 1 lock held by khungtaskd/185:
[ 990.259235] #0: ffffffffa6cfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.270719] 4 locks held by python3/3915:
[ 990.276018] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.285898] #1: ff11000163183280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.296851] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.308004] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.318262] 4 locks held by python3/5948:
[ 990.323559] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.333446] #1: ff110001f1b15080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.344405] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.355563] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.365655]
[ 990.368120] =============================================\x0a
[ 1112.574361] INFO: task python3:5948 blocked for more than 245 seconds.
[ 1112.582510] Not tainted 7.1.0-rc1-kcuo-ga15578502849 #1
[ 1112.589761] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.599350] task:python3 state:D stack:25032 pid:5948 tgid:5948 ppid:5947 task_flags:0x400100 flags:0x00080000
[ 1112.612554] Call Trace:
[ 1112.616096]
[ 1112.619250] __schedule+0xcba/0x1940
[ 1112.624068] ? io_schedule_timeout+0x140/0x140
[ 1112.629854] ? find_held_lock+0x2b/0x80
[ 1112.634961] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.640847] schedule+0xd4/0x260
[ 1112.645264] schedule_preempt_disabled+0x18/0x30
[ 1112.651246] __mutex_lock+0x6a3/0x2000
[ 1112.656251] ? threaded_store+0x132/0x220
[ 1112.661550] ? ww_mutex_lock+0x160/0x160
[ 1112.666741] ? selinux_inode_free_security+0x330/0x330
[ 1112.673312] ? cap_capable+0x1d6/0x3e0
[ 1112.678328] ? threaded_store+0x132/0x220
[ 1112.683634] threaded_store+0x132/0x220
[ 1112.688733] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.694029] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.699322] ? sysfs_kf_write+0x65/0x160
[ 1112.704524] ? sysfs_file_ops+0x140/0x140
[ 1112.709820] kernfs_fop_write_iter+0x335/0x510
[ 1112.715607] ? kernfs_ops+0xf0/0xf0
[ 1112.720318] new_sync_write+0x333/0x750
[ 1112.725424] ? __lock_acquire+0x508/0xc10
[ 1112.730720] ? new_sync_read+0x740/0x740
[ 1112.735922] ? lock_acquire.part.0+0xbc/0x260
[ 1112.741609] ? ksys_write+0x116/0x250
[ 1112.746512] vfs_write+0x657/0x9b0
[ 1112.751129] ksys_write+0x116/0x250
[ 1112.755842] ? __ia32_sys_read+0xc0/0xc0
[ 1112.761041] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.766632] ? rcu_is_watching+0x15/0xd0
[ 1112.771834] do_syscall_64+0xf3/0x650
[ 1112.776742] ? trace_hardirqs_off+0xd/0x30
[ 1112.782138] ? exc_page_fault+0xda/0xf0
[ 1112.787242] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.793712] RIP: 0033:0x7f6197d7122e
[ 1112.798521] RSP: 002b:00007ffcac5389f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.807820] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6197d7122e
[ 1112.816629] RDX: 0000000000000001 RSI: 000055a0b2e4da10 RDI: 0000000000000007
[ 1112.825439] RBP: 00007ffcac538a00 R08: 0000000000000000 R09: 0000000000000000
[ 1112.834251] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6197be9390
[ 1112.843058] R13: 000055a0b2e4da10 R14: 0000000000000007 R15: 0000000000000003
[ 1112.851872]
[ 1112.855150] INFO: task python3:5948 is blocked on a mutex likely owned by task python3:3915.
[ 1112.865425] task:python3 state:D stack:25472 pid:3915 tgid:3915 ppid:3914 task_flags:0x400100 flags:0x00080802
[ 1112.878628] Call Trace:
[ 1112.882167]
[ 1112.885317] __schedule+0xcba/0x1940
[ 1112.890129] ? io_schedule_timeout+0x140/0x140
[ 1112.895915] ? find_held_lock+0x2b/0x80
[ 1112.901018] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.907198] schedule+0xd4/0x260
[ 1112.911616] schedule_timeout+0x13f/0x290
[ 1112.916912] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.923282] ? trace_contention_end+0xb3/0x1a0
[ 1112.929060] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.934751] ? ww_mutex_lock+0x160/0x160
[ 1112.939951] msleep+0x1b/0x30
[ 1112.944076] napi_set_threaded+0x3a5/0x440
[ 1112.949472] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.954966] netif_set_threaded+0x257/0x400
[ 1112.960459] threaded_store+0x1b8/0x220
[ 1112.965563] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.970860] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.976158] ? sysfs_kf_write+0x65/0x160
[ 1112.981349] ? sysfs_file_ops+0x140/0x140
[ 1112.986637] kernfs_fop_write_iter+0x335/0x510
[ 1112.992423] ? kernfs_ops+0xf0/0xf0
[ 1112.997133] new_sync_write+0x333/0x750
[ 1113.002235] ? __lock_acquire+0x508/0xc10
[ 1113.007531] ? new_sync_read+0x740/0x740
[ 1113.012731] ? lock_acquire.part.0+0xbc/0x260
[ 1113.018416] ? ksys_write+0x116/0x250
[ 1113.023329] vfs_write+0x657/0x9b0
[ 1113.027945] ksys_write+0x116/0x250
[ 1113.032659] ? __ia32_sys_read+0xc0/0xc0
[ 1113.037859] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.043450] ? rcu_is_watching+0x15/0xd0
[ 1113.048651] do_syscall_64+0xf3/0x650
[ 1113.053561] ? irq_exit_rcu+0x1a/0x30
[ 1113.058469] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.064936] RIP: 0033:0x7fa88734c22e
[ 1113.069744] RSP: 002b:00007ffcd8027df0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.079040] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa88734c22e
[ 1113.087839] RDX: 0000000000000001 RSI: 000055b645d4ea10 RDI: 0000000000000007
[ 1113.096641] RBP: 00007ffcd8027e00 R08: 0000000000000000 R09: 0000000000000000
[ 1113.105451] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8871c4390
[ 1113.114261] R13: 000055b645d4ea10 R14: 0000000000000007 R15: 0000000000000003
[ 1113.123074]
[ 1113.126335] \x0aShowing all locks held in the system:
[ 1113.134874] 1 lock held by khungtaskd/185:
[ 1113.140269] #0: ffffffffa6cfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.151743] 2 locks held by systemd-journal/804:
[ 1113.157713] #0: ff1100180e3b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.167602] #1: ff110001a2d8ef78 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.178382] 4 locks held by python3/3915:
[ 1113.183675] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.193567] #1: ff11000163183280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.204529] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.215688] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.225783] 4 locks held by python3/5948:
[ 1113.231079] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.240967] #1: ff110001f1b15080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.251930] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.263085] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.273173]
[ 1113.275641] =============================================\x0a
[ 1235.456524] INFO: task python3:5948 blocked for more than 368 seconds.
[ 1235.464666] Not tainted 7.1.0-rc1-kcuo-ga15578502849 #1
[ 1235.471914] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.481500] task:python3 state:D stack:25032 pid:5948 tgid:5948 ppid:5947 task_flags:0x400100 flags:0x00080002
[ 1235.494704] Call Trace:
[ 1235.498243]
[ 1235.501393] __schedule+0xcba/0x1940
[ 1235.506210] ? io_schedule_timeout+0x140/0x140
[ 1235.511996] ? find_held_lock+0x2b/0x80
[ 1235.517104] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.522988] schedule+0xd4/0x260
[ 1235.527408] schedule_preempt_disabled+0x18/0x30
[ 1235.533387] __mutex_lock+0x6a3/0x2000
[ 1235.538380] ? threaded_store+0x132/0x220
[ 1235.543678] ? ww_mutex_lock+0x160/0x160
[ 1235.548876] ? selinux_inode_free_security+0x330/0x330
[ 1235.555445] ? cap_capable+0x1d6/0x3e0
[ 1235.560454] ? threaded_store+0x132/0x220
[ 1235.565753] threaded_store+0x132/0x220
[ 1235.570853] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.576148] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.581444] ? sysfs_kf_write+0x65/0x160
[ 1235.586643] ? sysfs_file_ops+0x140/0x140
[ 1235.591938] kernfs_fop_write_iter+0x335/0x510
[ 1235.597720] ? kernfs_ops+0xf0/0xf0
[ 1235.602431] new_sync_write+0x333/0x750
[ 1235.607526] ? __lock_acquire+0x508/0xc10
[ 1235.612820] ? new_sync_read+0x740/0x740
[ 1235.618021] ? lock_acquire.part.0+0xbc/0x260
[ 1235.623706] ? ksys_write+0x116/0x250
[ 1235.628613] vfs_write+0x657/0x9b0
[ 1235.633228] ksys_write+0x116/0x250
[ 1235.637932] ? __ia32_sys_read+0xc0/0xc0
[ 1235.643131] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.648718] ? rcu_is_watching+0x15/0xd0
[ 1235.653918] do_syscall_64+0xf3/0x650
[ 1235.658817] ? trace_hardirqs_off+0xd/0x30
[ 1235.664213] ? exc_page_fault+0xda/0xf0
[ 1235.669314] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.675780] RIP: 0033:0x7f6197d7122e
[ 1235.680589] RSP: 002b:00007ffcac5389f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.689879] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6197d7122e
[ 1235.698685] RDX: 0000000000000001 RSI: 000055a0b2e4da10 RDI: 0000000000000007
[ 1235.707491] RBP: 00007ffcac538a00 R08: 0000000000000000 R09: 0000000000000000
[ 1235.716297] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6197be9390
[ 1235.725102] R13: 000055a0b2e4da10 R14: 0000000000000007 R15: 0000000000000003
[ 1235.733916]
[ 1235.737203] INFO: task python3:5948 is blocked on a mutex likely owned by task python3:3915.
[ 1235.747475] task:python3 state:D stack:25472 pid:3915 tgid:3915 ppid:3914 task_flags:0x400100 flags:0x00080802
[ 1235.760676] Call Trace:
[ 1235.764215]
[ 1235.767363] __schedule+0xcba/0x1940
[ 1235.772175] ? io_schedule_timeout+0x140/0x140
[ 1235.777961] ? find_held_lock+0x2b/0x80
[ 1235.783064] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.789240] schedule+0xd4/0x260
[ 1235.793659] schedule_timeout+0x13f/0x290
[ 1235.798954] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.805324] ? trace_contention_end+0xb3/0x1a0
[ 1235.811107] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.816794] ? ww_mutex_lock+0x160/0x160
[ 1235.821991] msleep+0x1b/0x30
[ 1235.826113] napi_set_threaded+0x3a5/0x440
[ 1235.831507] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.837001] netif_set_threaded+0x257/0x400
[ 1235.842492] threaded_store+0x1b8/0x220
[ 1235.847592] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.852890] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.858175] ? sysfs_kf_write+0x65/0x160
[ 1235.863374] ? sysfs_file_ops+0x140/0x140
[ 1235.868668] kernfs_fop_write_iter+0x335/0x510
[ 1235.874451] ? kernfs_ops+0xf0/0xf0
[ 1235.879162] new_sync_write+0x333/0x750
[ 1235.884264] ? __lock_acquire+0x508/0xc10
[ 1235.889558] ? new_sync_read+0x740/0x740
[ 1235.894756] ? lock_acquire.part.0+0xbc/0x260
[ 1235.900440] ? ksys_write+0x116/0x250
[ 1235.905355] vfs_write+0x657/0x9b0
[ 1235.909969] ksys_write+0x116/0x250
[ 1235.914682] ? __ia32_sys_read+0xc0/0xc0
[ 1235.919880] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.925466] ? rcu_is_watching+0x15/0xd0
[ 1235.930665] do_syscall_64+0xf3/0x650
[ 1235.935571] ? irq_exit_rcu+0x1a/0x30
[ 1235.940467] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.946933] RIP: 0033:0x7fa88734c22e
[ 1235.951738] RSP: 002b:00007ffcd8027df0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.961035] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa88734c22e
[ 1235.969841] RDX: 0000000000000001 RSI: 000055b645d4ea10 RDI: 0000000000000007
[ 1235.978649] RBP: 00007ffcd8027e00 R08: 0000000000000000 R09: 0000000000000000
[ 1235.987455] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8871c4390
[ 1235.996263] R13: 000055b645d4ea10 R14: 0000000000000007 R15: 0000000000000003
[ 1236.005075]
[ 1236.008340] \x0aShowing all locks held in the system:
[ 1236.016869] 1 lock held by khungtaskd/185:
[ 1236.022263] #0: ffffffffa6cfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.033743] 4 locks held by python3/3915:
[ 1236.039037] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.048927] #1: ff11000163183280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.059879] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.071033] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.081126] 4 locks held by python3/5948:
[ 1236.086418] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.096305] #1: ff110001f1b15080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.107265] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.118420] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.128510]
[ 1236.130975] =============================================\x0a
[ 1358.337672] INFO: task python3:5948 blocked for more than 491 seconds.
[ 1358.345819] Not tainted 7.1.0-rc1-kcuo-ga15578502849 #1
[ 1358.353068] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.362659] task:python3 state:D stack:25032 pid:5948 tgid:5948 ppid:5947 task_flags:0x400100 flags:0x00080002
[ 1358.375864] Call Trace:
[ 1358.379402]
[ 1358.382554] __schedule+0xcba/0x1940
[ 1358.387371] ? io_schedule_timeout+0x140/0x140
[ 1358.393156] ? find_held_lock+0x2b/0x80
[ 1358.398258] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.404142] schedule+0xd4/0x260
[ 1358.408562] schedule_preempt_disabled+0x18/0x30
[ 1358.414538] __mutex_lock+0x6a3/0x2000
[ 1358.419541] ? threaded_store+0x132/0x220
[ 1358.424842] ? ww_mutex_lock+0x160/0x160
[ 1358.430040] ? selinux_inode_free_security+0x330/0x330
[ 1358.436610] ? cap_capable+0x1d6/0x3e0
[ 1358.441617] ? threaded_store+0x132/0x220
[ 1358.446914] threaded_store+0x132/0x220
[ 1358.452017] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.457304] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.462604] ? sysfs_kf_write+0x65/0x160
[ 1358.467807] ? sysfs_file_ops+0x140/0x140
[ 1358.473102] kernfs_fop_write_iter+0x335/0x510
[ 1358.478887] ? kernfs_ops+0xf0/0xf0
[ 1358.483588] new_sync_write+0x333/0x750
[ 1358.488693] ? __lock_acquire+0x508/0xc10
[ 1358.493977] ? new_sync_read+0x740/0x740
[ 1358.499179] ? lock_acquire.part.0+0xbc/0x260
[ 1358.504865] ? ksys_write+0x116/0x250
[ 1358.509773] vfs_write+0x657/0x9b0
[ 1358.514381] ksys_write+0x116/0x250
[ 1358.519093] ? __ia32_sys_read+0xc0/0xc0
[ 1358.524281] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.529871] ? rcu_is_watching+0x15/0xd0
[ 1358.535073] do_syscall_64+0xf3/0x650
[ 1358.539981] ? trace_hardirqs_off+0xd/0x30
[ 1358.545376] ? exc_page_fault+0xda/0xf0
[ 1358.550478] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.556944] RIP: 0033:0x7f6197d7122e
[ 1358.561754] RSP: 002b:00007ffcac5389f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.571052] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6197d7122e
[ 1358.579858] RDX: 0000000000000001 RSI: 000055a0b2e4da10 RDI: 0000000000000007
[ 1358.588664] RBP: 00007ffcac538a00 R08: 0000000000000000 R09: 0000000000000000
[ 1358.597469] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6197be9390
[ 1358.606276] R13: 000055a0b2e4da10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.615090]
[ 1358.618363] INFO: task python3:5948 is blocked on a mutex likely owned by task python3:3915.
[ 1358.628636] task:python3 state:D stack:25472 pid:3915 tgid:3915 ppid:3914 task_flags:0x400100 flags:0x00080802
[ 1358.641838] Call Trace:
[ 1358.645378]
[ 1358.648525] __schedule+0xcba/0x1940
[ 1358.653333] ? io_schedule_timeout+0x140/0x140
[ 1358.659117] ? find_held_lock+0x2b/0x80
[ 1358.664220] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.670397] schedule+0xd4/0x260
[ 1358.674816] schedule_timeout+0x13f/0x290
[ 1358.680112] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.686482] ? trace_contention_end+0xb3/0x1a0
[ 1358.692264] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.697953] ? ww_mutex_lock+0x160/0x160
[ 1358.703144] msleep+0x1b/0x30
[ 1358.707269] napi_set_threaded+0x3a5/0x440
[ 1358.712662] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.718156] netif_set_threaded+0x257/0x400
[ 1358.723647] threaded_store+0x1b8/0x220
[ 1358.728748] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.734047] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.739341] ? sysfs_kf_write+0x65/0x160
[ 1358.744541] ? sysfs_file_ops+0x140/0x140
[ 1358.749828] kernfs_fop_write_iter+0x335/0x510
[ 1358.755611] ? kernfs_ops+0xf0/0xf0
[ 1358.760324] new_sync_write+0x333/0x750
[ 1358.765428] ? __lock_acquire+0x508/0xc10
[ 1358.770722] ? new_sync_read+0x740/0x740
[ 1358.775920] ? lock_acquire.part.0+0xbc/0x260
[ 1358.781603] ? ksys_write+0x116/0x250
[ 1358.786519] vfs_write+0x657/0x9b0
[ 1358.791126] ksys_write+0x116/0x250
[ 1358.795836] ? __ia32_sys_read+0xc0/0xc0
[ 1358.801034] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.806622] ? rcu_is_watching+0x15/0xd0
[ 1358.811822] do_syscall_64+0xf3/0x650
[ 1358.816729] ? irq_exit_rcu+0x1a/0x30
[ 1358.821637] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.828102] RIP: 0033:0x7fa88734c22e
[ 1358.832911] RSP: 002b:00007ffcd8027df0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.842205] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa88734c22e
[ 1358.851012] RDX: 0000000000000001 RSI: 000055b645d4ea10 RDI: 0000000000000007
[ 1358.859816] RBP: 00007ffcd8027e00 R08: 0000000000000000 R09: 0000000000000000
[ 1358.868624] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8871c4390
[ 1358.877430] R13: 000055b645d4ea10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.886240]
[ 1358.889515] \x0aShowing all locks held in the system:
[ 1358.898049] 1 lock held by khungtaskd/185:
[ 1358.903442] #0: ffffffffa6cfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.914922] 4 locks held by python3/3915:
[ 1358.920216] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.930102] #1: ff11000163183280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.941065] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.952221] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.962311] 4 locks held by python3/5948:
[ 1358.967604] #0: ff110017dab243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.977493] #1: ff110001f1b15080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.988443] #2: ff11000122ea4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.999598] #3: ff11000122f09d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.009692]
[ 1359.012157] =============================================\x0a