[ 790.753898] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 790.756484] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 790.912853] kselftest: Running tests in drivers/net
[ 989.701344] INFO: task python3:5900 blocked for more than 122 seconds.
[ 989.709584] Not tainted 7.1.0-rc5-grjd-g62a5b6f2702a #1
[ 989.717405] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.726985] task:python3 state:D stack:25672 pid:5900 tgid:5900 ppid:5899 task_flags:0x400100 flags:0x00080000
[ 989.740194] Call Trace:
[ 989.743733]
[ 989.746884] __schedule+0xcc3/0x19c0
[ 989.751702] ? io_schedule_timeout+0x140/0x140
[ 989.757486] ? find_held_lock+0x2b/0x80
[ 989.762591] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.768469] schedule+0xd8/0x260
[ 989.772888] schedule_preempt_disabled+0x18/0x30
[ 989.778868] __mutex_lock+0x6af/0x2010
[ 989.783864] ? threaded_store+0x132/0x220
[ 989.789168] ? ww_mutex_lock+0x160/0x160
[ 989.794366] ? selinux_inode_free_security+0x330/0x330
[ 989.800937] ? cap_capable+0x1d6/0x3e0
[ 989.805944] ? threaded_store+0x132/0x220
[ 989.811243] threaded_store+0x132/0x220
[ 989.816349] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.821645] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.826946] ? sysfs_kf_write+0x65/0x160
[ 989.832144] ? sysfs_file_ops+0x140/0x140
[ 989.837439] kernfs_fop_write_iter+0x335/0x510
[ 989.843224] ? kernfs_ops+0xf0/0xf0
[ 989.847928] new_sync_write+0x333/0x750
[ 989.853030] ? __lock_acquire+0x508/0xc10
[ 989.858329] ? new_sync_read+0x740/0x740
[ 989.863534] ? lock_acquire.part.0+0xbc/0x260
[ 989.869222] ? ksys_write+0x116/0x250
[ 989.874134] vfs_write+0x657/0x9b0
[ 989.878750] ksys_write+0x116/0x250
[ 989.883462] ? __ia32_sys_read+0xc0/0xc0
[ 989.888652] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.894242] ? rcu_is_watching+0x15/0xd0
[ 989.899444] do_syscall_64+0xf3/0x650
[ 989.904352] ? trace_hardirqs_off+0xd/0x30
[ 989.909746] ? exc_page_fault+0xda/0xf0
[ 989.914846] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.921325] RIP: 0033:0x7f90d518f22e
[ 989.926135] RSP: 002b:00007ffc7d2d3310 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.935432] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f90d518f22e
[ 989.944230] RDX: 0000000000000001 RSI: 00005565dd9e56b0 RDI: 0000000000000007
[ 989.953041] RBP: 00007ffc7d2d3320 R08: 0000000000000000 R09: 0000000000000000
[ 989.961849] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f90d5007390
[ 989.970655] R13: 00005565dd9e56b0 R14: 0000000000000007 R15: 0000000000000003
[ 989.979467]
[ 989.982775] INFO: task python3:5900 is blocked on a mutex likely owned by task python3:3814.
[ 989.993048] task:python3 state:D stack:25040 pid:3814 tgid:3814 ppid:3813 task_flags:0x400100 flags:0x00080802
[ 990.006251] Call Trace:
[ 990.009792]
[ 990.012941] __schedule+0xcc3/0x19c0
[ 990.017751] ? io_schedule_timeout+0x140/0x140
[ 990.023528] ? find_held_lock+0x2b/0x80
[ 990.028634] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.034813] schedule+0xd8/0x260
[ 990.039234] schedule_timeout+0x13f/0x290
[ 990.044533] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.050906] ? trace_contention_end+0xb3/0x1a0
[ 990.056693] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.062382] ? ww_mutex_lock+0x160/0x160
[ 990.067585] msleep+0x1b/0x30
[ 990.071713] napi_set_threaded+0x3a5/0x440
[ 990.077108] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.082603] netif_set_threaded+0x257/0x400
[ 990.088097] threaded_store+0x1b8/0x220
[ 990.093200] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.098495] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.103792] ? sysfs_kf_write+0x65/0x160
[ 990.108993] ? sysfs_file_ops+0x140/0x140
[ 990.114291] kernfs_fop_write_iter+0x335/0x510
[ 990.120065] ? kernfs_ops+0xf0/0xf0
[ 990.124778] new_sync_write+0x333/0x750
[ 990.129883] ? __lock_acquire+0x508/0xc10
[ 990.135179] ? new_sync_read+0x740/0x740
[ 990.140378] ? lock_acquire.part.0+0xbc/0x260
[ 990.146066] ? ksys_write+0x116/0x250
[ 990.150975] vfs_write+0x657/0x9b0
[ 990.155590] ksys_write+0x116/0x250
[ 990.160293] ? __ia32_sys_read+0xc0/0xc0
[ 990.165492] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.171084] ? rcu_is_watching+0x15/0xd0
[ 990.176285] do_syscall_64+0xf3/0x650
[ 990.181190] ? trace_hardirqs_off+0xd/0x30
[ 990.186587] ? exc_page_fault+0xda/0xf0
[ 990.191691] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.198157] RIP: 0033:0x7f010bfe222e
[ 990.202972] RSP: 002b:00007fff29c71660 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.212270] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f010bfe222e
[ 990.221077] RDX: 0000000000000001 RSI: 0000560ab41846b0 RDI: 0000000000000007
[ 990.229888] RBP: 00007fff29c71670 R08: 0000000000000000 R09: 0000000000000000
[ 990.238697] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f010be5a390
[ 990.247503] R13: 0000560ab41846b0 R14: 0000000000000007 R15: 0000000000000003
[ 990.256317]
[ 990.259578] \x0aShowing all locks held in the system:
[ 990.268120] 1 lock held by khungtaskd/185:
[ 990.273515] #0: ffffffffae8fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.285001] 4 locks held by python3/3814:
[ 990.290298] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.300182] #1: ff110001749d2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.311144] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.322303] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.332409] 4 locks held by python3/5900:
[ 990.337706] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.347588] #1: ff110001084a6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.358550] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.369709] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.379802]
[ 990.382266] =============================================\x0a
[ 1112.582617] INFO: task python3:5900 blocked for more than 245 seconds.
[ 1112.590764] Not tainted 7.1.0-rc5-grjd-g62a5b6f2702a #1
[ 1112.598012] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.607602] task:python3 state:D stack:25672 pid:5900 tgid:5900 ppid:5899 task_flags:0x400100 flags:0x00080000
[ 1112.620807] Call Trace:
[ 1112.624344]
[ 1112.627495] __schedule+0xcc3/0x19c0
[ 1112.632311] ? io_schedule_timeout+0x140/0x140
[ 1112.638096] ? find_held_lock+0x2b/0x80
[ 1112.643201] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.649084] schedule+0xd8/0x260
[ 1112.653503] schedule_preempt_disabled+0x18/0x30
[ 1112.659482] __mutex_lock+0x6af/0x2010
[ 1112.664487] ? threaded_store+0x132/0x220
[ 1112.669787] ? ww_mutex_lock+0x160/0x160
[ 1112.674984] ? selinux_inode_free_security+0x330/0x330
[ 1112.681556] ? cap_capable+0x1d6/0x3e0
[ 1112.686562] ? threaded_store+0x132/0x220
[ 1112.691860] threaded_store+0x132/0x220
[ 1112.696955] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.702251] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.707548] ? sysfs_kf_write+0x65/0x160
[ 1112.712750] ? sysfs_file_ops+0x140/0x140
[ 1112.718045] kernfs_fop_write_iter+0x335/0x510
[ 1112.723830] ? kernfs_ops+0xf0/0xf0
[ 1112.728540] new_sync_write+0x333/0x750
[ 1112.733643] ? __lock_acquire+0x508/0xc10
[ 1112.738936] ? new_sync_read+0x740/0x740
[ 1112.744137] ? lock_acquire.part.0+0xbc/0x260
[ 1112.749821] ? ksys_write+0x116/0x250
[ 1112.754729] vfs_write+0x657/0x9b0
[ 1112.759336] ksys_write+0x116/0x250
[ 1112.764048] ? __ia32_sys_read+0xc0/0xc0
[ 1112.769245] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.774834] ? rcu_is_watching+0x15/0xd0
[ 1112.780033] do_syscall_64+0xf3/0x650
[ 1112.784936] ? trace_hardirqs_off+0xd/0x30
[ 1112.790331] ? exc_page_fault+0xda/0xf0
[ 1112.795434] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.801893] RIP: 0033:0x7f90d518f22e
[ 1112.806704] RSP: 002b:00007ffc7d2d3310 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.816000] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f90d518f22e
[ 1112.824809] RDX: 0000000000000001 RSI: 00005565dd9e56b0 RDI: 0000000000000007
[ 1112.833616] RBP: 00007ffc7d2d3320 R08: 0000000000000000 R09: 0000000000000000
[ 1112.842415] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f90d5007390
[ 1112.851224] R13: 00005565dd9e56b0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.860037]
[ 1112.863317] INFO: task python3:5900 is blocked on a mutex likely owned by task python3:3814.
[ 1112.873586] task:python3 state:D stack:25040 pid:3814 tgid:3814 ppid:3813 task_flags:0x400100 flags:0x00080802
[ 1112.886789] Call Trace:
[ 1112.890327]
[ 1112.893474] __schedule+0xcc3/0x19c0
[ 1112.898286] ? io_schedule_timeout+0x140/0x140
[ 1112.904071] ? find_held_lock+0x2b/0x80
[ 1112.909170] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.915347] schedule+0xd8/0x260
[ 1112.919764] schedule_timeout+0x13f/0x290
[ 1112.925062] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.931422] ? trace_contention_end+0xb3/0x1a0
[ 1112.937204] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.942891] ? ww_mutex_lock+0x160/0x160
[ 1112.948091] msleep+0x1b/0x30
[ 1112.952209] napi_set_threaded+0x3a5/0x440
[ 1112.957595] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.963087] netif_set_threaded+0x257/0x400
[ 1112.968572] threaded_store+0x1b8/0x220
[ 1112.973672] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.978967] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.984261] ? sysfs_kf_write+0x65/0x160
[ 1112.989467] ? sysfs_file_ops+0x140/0x140
[ 1112.994761] kernfs_fop_write_iter+0x335/0x510
[ 1113.000544] ? kernfs_ops+0xf0/0xf0
[ 1113.005256] new_sync_write+0x333/0x750
[ 1113.010359] ? __lock_acquire+0x508/0xc10
[ 1113.015655] ? new_sync_read+0x740/0x740
[ 1113.020853] ? lock_acquire.part.0+0xbc/0x260
[ 1113.026539] ? ksys_write+0x116/0x250
[ 1113.031449] vfs_write+0x657/0x9b0
[ 1113.036062] ksys_write+0x116/0x250
[ 1113.040771] ? __ia32_sys_read+0xc0/0xc0
[ 1113.045970] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.051557] ? rcu_is_watching+0x15/0xd0
[ 1113.056756] do_syscall_64+0xf3/0x650
[ 1113.061652] ? trace_hardirqs_off+0xd/0x30
[ 1113.067044] ? exc_page_fault+0xda/0xf0
[ 1113.072147] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.078611] RIP: 0033:0x7f010bfe222e
[ 1113.083417] RSP: 002b:00007fff29c71660 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.092711] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f010bfe222e
[ 1113.101516] RDX: 0000000000000001 RSI: 0000560ab41846b0 RDI: 0000000000000007
[ 1113.110323] RBP: 00007fff29c71670 R08: 0000000000000000 R09: 0000000000000000
[ 1113.119132] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f010be5a390
[ 1113.127935] R13: 0000560ab41846b0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.136746]
[ 1113.140008] \x0aShowing all locks held in the system:
[ 1113.148546] 1 lock held by khungtaskd/185:
[ 1113.153940] #0: ffffffffae8fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.165422] 4 locks held by python3/3814:
[ 1113.170718] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.180606] #1: ff110001749d2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.191567] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.202723] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.212808] 4 locks held by python3/5900:
[ 1113.218101] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.227989] #1: ff110001084a6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.238950] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.250108] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.260197]
[ 1113.262663] =============================================\x0a
[ 1124.871004] kworker/u130:2 (351) used greatest stack depth: 21456 bytes left
[ 1235.467153] INFO: task python3:5900 blocked for more than 368 seconds.
[ 1235.475292] Not tainted 7.1.0-rc5-grjd-g62a5b6f2702a #1
[ 1235.482532] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.492124] task:python3 state:D stack:25672 pid:5900 tgid:5900 ppid:5899 task_flags:0x400100 flags:0x00080002
[ 1235.505331] Call Trace:
[ 1235.508871]
[ 1235.512024] __schedule+0xcc3/0x19c0
[ 1235.516842] ? io_schedule_timeout+0x140/0x140
[ 1235.522627] ? find_held_lock+0x2b/0x80
[ 1235.527735] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.533619] schedule+0xd8/0x260
[ 1235.538037] schedule_preempt_disabled+0x18/0x30
[ 1235.544018] __mutex_lock+0x6af/0x2010
[ 1235.549025] ? threaded_store+0x132/0x220
[ 1235.554324] ? ww_mutex_lock+0x160/0x160
[ 1235.559515] ? selinux_inode_free_security+0x330/0x330
[ 1235.566085] ? cap_capable+0x1d6/0x3e0
[ 1235.571094] ? threaded_store+0x132/0x220
[ 1235.576391] threaded_store+0x132/0x220
[ 1235.581492] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.586792] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.592092] ? sysfs_kf_write+0x65/0x160
[ 1235.597291] ? sysfs_file_ops+0x140/0x140
[ 1235.602588] kernfs_fop_write_iter+0x335/0x510
[ 1235.608375] ? kernfs_ops+0xf0/0xf0
[ 1235.613087] new_sync_write+0x333/0x750
[ 1235.618192] ? __lock_acquire+0x508/0xc10
[ 1235.623482] ? new_sync_read+0x740/0x740
[ 1235.628683] ? lock_acquire.part.0+0xbc/0x260
[ 1235.634370] ? ksys_write+0x116/0x250
[ 1235.639270] vfs_write+0x657/0x9b0
[ 1235.643887] ksys_write+0x116/0x250
[ 1235.648601] ? __ia32_sys_read+0xc0/0xc0
[ 1235.653798] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.659390] ? rcu_is_watching+0x15/0xd0
[ 1235.664592] do_syscall_64+0xf3/0x650
[ 1235.669501] ? trace_hardirqs_off+0xd/0x30
[ 1235.674896] ? exc_page_fault+0xda/0xf0
[ 1235.680004] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.686473] RIP: 0033:0x7f90d518f22e
[ 1235.691284] RSP: 002b:00007ffc7d2d3310 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.700582] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f90d518f22e
[ 1235.709391] RDX: 0000000000000001 RSI: 00005565dd9e56b0 RDI: 0000000000000007
[ 1235.718201] RBP: 00007ffc7d2d3320 R08: 0000000000000000 R09: 0000000000000000
[ 1235.727011] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f90d5007390
[ 1235.735819] R13: 00005565dd9e56b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.744624]
[ 1235.747900] INFO: task python3:5900 is blocked on a mutex likely owned by task python3:3814.
[ 1235.758175] task:python3 state:D stack:25040 pid:3814 tgid:3814 ppid:3813 task_flags:0x400100 flags:0x00080802
[ 1235.771370] Call Trace:
[ 1235.774910]
[ 1235.778063] __schedule+0xcc3/0x19c0
[ 1235.782876] ? io_schedule_timeout+0x140/0x140
[ 1235.788661] ? find_held_lock+0x2b/0x80
[ 1235.793765] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.799943] schedule+0xd8/0x260
[ 1235.804361] schedule_timeout+0x13f/0x290
[ 1235.809659] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.816030] ? trace_contention_end+0xb3/0x1a0
[ 1235.821817] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.827509] ? ww_mutex_lock+0x160/0x160
[ 1235.832710] msleep+0x1b/0x30
[ 1235.836835] napi_set_threaded+0x3a5/0x440
[ 1235.842233] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.847726] netif_set_threaded+0x257/0x400
[ 1235.853220] threaded_store+0x1b8/0x220
[ 1235.858325] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.863622] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.868917] ? sysfs_kf_write+0x65/0x160
[ 1235.874118] ? sysfs_file_ops+0x140/0x140
[ 1235.879413] kernfs_fop_write_iter+0x335/0x510
[ 1235.885196] ? kernfs_ops+0xf0/0xf0
[ 1235.889907] new_sync_write+0x333/0x750
[ 1235.895011] ? __lock_acquire+0x508/0xc10
[ 1235.900306] ? new_sync_read+0x740/0x740
[ 1235.905507] ? lock_acquire.part.0+0xbc/0x260
[ 1235.911193] ? ksys_write+0x116/0x250
[ 1235.916103] vfs_write+0x657/0x9b0
[ 1235.920719] ksys_write+0x116/0x250
[ 1235.925433] ? __ia32_sys_read+0xc0/0xc0
[ 1235.930635] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.936226] ? rcu_is_watching+0x15/0xd0
[ 1235.941424] do_syscall_64+0xf3/0x650
[ 1235.946329] ? trace_hardirqs_off+0xd/0x30
[ 1235.951723] ? exc_page_fault+0xda/0xf0
[ 1235.956826] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.963285] RIP: 0033:0x7f010bfe222e
[ 1235.968094] RSP: 002b:00007fff29c71660 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.977382] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f010bfe222e
[ 1235.986192] RDX: 0000000000000001 RSI: 0000560ab41846b0 RDI: 0000000000000007
[ 1235.995005] RBP: 00007fff29c71670 R08: 0000000000000000 R09: 0000000000000000
[ 1236.003815] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f010be5a390
[ 1236.012623] R13: 0000560ab41846b0 R14: 0000000000000007 R15: 0000000000000003
[ 1236.021436]
[ 1236.024706] \x0aShowing all locks held in the system:
[ 1236.033243] 1 lock held by khungtaskd/185:
[ 1236.038635] #0: ffffffffae8fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.050114] 4 locks held by python3/3814:
[ 1236.055409] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.065301] #1: ff110001749d2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.076261] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.087420] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.097517] 4 locks held by python3/5900:
[ 1236.102813] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.112701] #1: ff110001084a6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.123661] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.134821] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.144913]
[ 1236.147382] =============================================\x0a
[ 1358.347581] INFO: task python3:5900 blocked for more than 491 seconds.
[ 1358.355728] Not tainted 7.1.0-rc5-grjd-g62a5b6f2702a #1
[ 1358.362976] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.372565] task:python3 state:D stack:25672 pid:5900 tgid:5900 ppid:5899 task_flags:0x400100 flags:0x00080002
[ 1358.385769] Call Trace:
[ 1358.389308]
[ 1358.392461] __schedule+0xcc3/0x19c0
[ 1358.397277] ? io_schedule_timeout+0x140/0x140
[ 1358.403062] ? find_held_lock+0x2b/0x80
[ 1358.408167] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.414047] schedule+0xd8/0x260
[ 1358.418471] schedule_preempt_disabled+0x18/0x30
[ 1358.424452] __mutex_lock+0x6af/0x2010
[ 1358.429458] ? threaded_store+0x132/0x220
[ 1358.434759] ? ww_mutex_lock+0x160/0x160
[ 1358.439956] ? selinux_inode_free_security+0x330/0x330
[ 1358.446514] ? cap_capable+0x1d6/0x3e0
[ 1358.451511] ? threaded_store+0x132/0x220
[ 1358.456809] threaded_store+0x132/0x220
[ 1358.461909] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.467207] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.472502] ? sysfs_kf_write+0x65/0x160
[ 1358.477703] ? sysfs_file_ops+0x140/0x140
[ 1358.482998] kernfs_fop_write_iter+0x335/0x510
[ 1358.488779] ? kernfs_ops+0xf0/0xf0
[ 1358.493489] new_sync_write+0x333/0x750
[ 1358.498592] ? __lock_acquire+0x508/0xc10
[ 1358.503885] ? new_sync_read+0x740/0x740
[ 1358.509087] ? lock_acquire.part.0+0xbc/0x260
[ 1358.514771] ? ksys_write+0x116/0x250
[ 1358.519679] vfs_write+0x657/0x9b0
[ 1358.524296] ksys_write+0x116/0x250
[ 1358.529008] ? __ia32_sys_read+0xc0/0xc0
[ 1358.534199] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.539788] ? rcu_is_watching+0x15/0xd0
[ 1358.544988] do_syscall_64+0xf3/0x650
[ 1358.549893] ? trace_hardirqs_off+0xd/0x30
[ 1358.555288] ? exc_page_fault+0xda/0xf0
[ 1358.560390] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.566854] RIP: 0033:0x7f90d518f22e
[ 1358.571665] RSP: 002b:00007ffc7d2d3310 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.580961] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f90d518f22e
[ 1358.589761] RDX: 0000000000000001 RSI: 00005565dd9e56b0 RDI: 0000000000000007
[ 1358.598570] RBP: 00007ffc7d2d3320 R08: 0000000000000000 R09: 0000000000000000
[ 1358.607377] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f90d5007390
[ 1358.616177] R13: 00005565dd9e56b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.624990]
[ 1358.628267] INFO: task python3:5900 is blocked on a mutex likely owned by task python3:3814.
[ 1358.638540] task:python3 state:D stack:25040 pid:3814 tgid:3814 ppid:3813 task_flags:0x400100 flags:0x00080802
[ 1358.651742] Call Trace:
[ 1358.655279]
[ 1358.658430] __schedule+0xcc3/0x19c0
[ 1358.663233] ? io_schedule_timeout+0x140/0x140
[ 1358.669017] ? find_held_lock+0x2b/0x80
[ 1358.674112] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.680291] schedule+0xd8/0x260
[ 1358.684709] schedule_timeout+0x13f/0x290
[ 1358.690006] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.696375] ? trace_contention_end+0xb3/0x1a0
[ 1358.702158] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.707845] ? ww_mutex_lock+0x160/0x160
[ 1358.713045] msleep+0x1b/0x30
[ 1358.717172] napi_set_threaded+0x3a5/0x440
[ 1358.722566] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.728059] netif_set_threaded+0x257/0x400
[ 1358.733552] threaded_store+0x1b8/0x220
[ 1358.738652] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.743947] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.749243] ? sysfs_kf_write+0x65/0x160
[ 1358.754441] ? sysfs_file_ops+0x140/0x140
[ 1358.759734] kernfs_fop_write_iter+0x335/0x510
[ 1358.765516] ? kernfs_ops+0xf0/0xf0
[ 1358.770227] new_sync_write+0x333/0x750
[ 1358.775327] ? __lock_acquire+0x508/0xc10
[ 1358.780622] ? new_sync_read+0x740/0x740
[ 1358.785820] ? lock_acquire.part.0+0xbc/0x260
[ 1358.791506] ? ksys_write+0x116/0x250
[ 1358.796417] vfs_write+0x657/0x9b0
[ 1358.801031] ksys_write+0x116/0x250
[ 1358.805740] ? __ia32_sys_read+0xc0/0xc0
[ 1358.810939] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.816526] ? rcu_is_watching+0x15/0xd0
[ 1358.821726] do_syscall_64+0xf3/0x650
[ 1358.826621] ? trace_hardirqs_off+0xd/0x30
[ 1358.832012] ? exc_page_fault+0xda/0xf0
[ 1358.837113] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.843577] RIP: 0033:0x7f010bfe222e
[ 1358.848382] RSP: 002b:00007fff29c71660 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.857676] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f010bfe222e
[ 1358.866481] RDX: 0000000000000001 RSI: 0000560ab41846b0 RDI: 0000000000000007
[ 1358.875288] RBP: 00007fff29c71670 R08: 0000000000000000 R09: 0000000000000000
[ 1358.884093] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f010be5a390
[ 1358.892898] R13: 0000560ab41846b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.901709]
[ 1358.904978] \x0aShowing all locks held in the system:
[ 1358.913515] 1 lock held by khungtaskd/185:
[ 1358.918909] #0: ffffffffae8fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.930388] 4 locks held by python3/3814:
[ 1358.935682] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.945570] #1: ff110001749d2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.956532] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.967688] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.977784] 4 locks held by python3/5900:
[ 1358.983077] #0: ff110017da23c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.992963] #1: ff110001084a6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.003923] #2: ff11000122f07450 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.015082] #3: ff11000122f31d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.025171]
[ 1359.027635] =============================================\x0a