[ 784.902099] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 784.905091] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 785.059698] kselftest: Running tests in drivers/net
[ 989.657919] INFO: task python3:5708 blocked for more than 122 seconds.
[ 989.666152] Not tainted 7.1.0-rc2-ltws-g5bf6b77b793f #1
[ 989.674035] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.683625] task:python3 state:D stack:24800 pid:5708 tgid:5708 ppid:5707 task_flags:0x400100 flags:0x00080000
[ 989.696823] Call Trace:
[ 989.700365]
[ 989.703518] __schedule+0xcba/0x1940
[ 989.708335] ? io_schedule_timeout+0x140/0x140
[ 989.714119] ? find_held_lock+0x2b/0x80
[ 989.719214] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.725100] schedule+0xd4/0x260
[ 989.729518] schedule_preempt_disabled+0x18/0x30
[ 989.735496] __mutex_lock+0x6a3/0x2000
[ 989.740501] ? threaded_store+0x132/0x220
[ 989.745801] ? ww_mutex_lock+0x160/0x160
[ 989.751002] ? selinux_inode_free_security+0x330/0x330
[ 989.757573] ? cap_capable+0x1d6/0x3e0
[ 989.762581] ? threaded_store+0x132/0x220
[ 989.767878] threaded_store+0x132/0x220
[ 989.772982] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.778281] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.783579] ? sysfs_kf_write+0x65/0x160
[ 989.788778] ? sysfs_file_ops+0x140/0x140
[ 989.794073] kernfs_fop_write_iter+0x335/0x510
[ 989.799859] ? kernfs_ops+0xf0/0xf0
[ 989.804571] new_sync_write+0x333/0x750
[ 989.809677] ? __lock_acquire+0x508/0xc10
[ 989.814976] ? new_sync_read+0x740/0x740
[ 989.820176] ? lock_acquire.part.0+0xbc/0x260
[ 989.825863] ? ksys_write+0x116/0x250
[ 989.830773] vfs_write+0x657/0x9b0
[ 989.835387] ksys_write+0x116/0x250
[ 989.840098] ? __ia32_sys_read+0xc0/0xc0
[ 989.845297] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.850885] ? rcu_is_watching+0x15/0xd0
[ 989.856077] do_syscall_64+0xf3/0x650
[ 989.860975] ? trace_hardirqs_off+0xd/0x30
[ 989.866361] ? exc_page_fault+0xda/0xf0
[ 989.871461] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.877927] RIP: 0033:0x7f241152222e
[ 989.882740] RSP: 002b:00007ffef0b0ee40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.892045] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f241152222e
[ 989.900854] RDX: 0000000000000001 RSI: 00005573c0e56fe0 RDI: 0000000000000007
[ 989.909662] RBP: 00007ffef0b0ee50 R08: 0000000000000000 R09: 0000000000000000
[ 989.918463] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f241139a390
[ 989.927272] R13: 00005573c0e56fe0 R14: 0000000000000007 R15: 0000000000000003
[ 989.936084]
[ 989.939384] INFO: task python3:5708 is blocked on a mutex likely owned by task python3:3651.
[ 989.949656] task:python3 state:D stack:25672 pid:3651 tgid:3651 ppid:3650 task_flags:0x400100 flags:0x00080802
[ 989.962860] Call Trace:
[ 989.966400]
[ 989.969551] __schedule+0xcba/0x1940
[ 989.974363] ? io_schedule_timeout+0x140/0x140
[ 989.980148] ? find_held_lock+0x2b/0x80
[ 989.985241] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.991419] schedule+0xd4/0x260
[ 989.995838] schedule_timeout+0x13f/0x290
[ 990.001137] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.007508] ? trace_contention_end+0xb3/0x1a0
[ 990.013292] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.018984] ? ww_mutex_lock+0x160/0x160
[ 990.024186] msleep+0x1b/0x30
[ 990.028310] napi_set_threaded+0x3a5/0x440
[ 990.033704] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.039199] netif_set_threaded+0x257/0x400
[ 990.044694] threaded_store+0x1b8/0x220
[ 990.049795] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.055091] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.060387] ? sysfs_kf_write+0x65/0x160
[ 990.065587] ? sysfs_file_ops+0x140/0x140
[ 990.070885] kernfs_fop_write_iter+0x335/0x510
[ 990.076669] ? kernfs_ops+0xf0/0xf0
[ 990.081378] new_sync_write+0x333/0x750
[ 990.086480] ? __lock_acquire+0x508/0xc10
[ 990.091767] ? new_sync_read+0x740/0x740
[ 990.096966] ? lock_acquire.part.0+0xbc/0x260
[ 990.102655] ? ksys_write+0x116/0x250
[ 990.107565] vfs_write+0x657/0x9b0
[ 990.112182] ksys_write+0x116/0x250
[ 990.116893] ? __ia32_sys_read+0xc0/0xc0
[ 990.122092] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.127681] ? rcu_is_watching+0x15/0xd0
[ 990.132880] do_syscall_64+0xf3/0x650
[ 990.137787] ? trace_hardirqs_off+0xd/0x30
[ 990.143182] ? exc_page_fault+0xda/0xf0
[ 990.148285] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.154755] RIP: 0033:0x7f592334622e
[ 990.159565] RSP: 002b:00007ffcd09470d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.168853] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f592334622e
[ 990.177663] RDX: 0000000000000001 RSI: 0000558f9ae57fe0 RDI: 0000000000000007
[ 990.186470] RBP: 00007ffcd09470e0 R08: 0000000000000000 R09: 0000000000000000
[ 990.195268] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f59231be390
[ 990.204074] R13: 0000558f9ae57fe0 R14: 0000000000000007 R15: 0000000000000003
[ 990.212886]
[ 990.216143] \x0aShowing all locks held in the system:
[ 990.224681] 1 lock held by khungtaskd/186:
[ 990.230074] #0: ffffffffa7cfdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.241559] 4 locks held by python3/3651:
[ 990.246853] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.256743] #1: ff1100016a22d080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.267706] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.278863] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.288959] 4 locks held by python3/5708:
[ 990.294254] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.304144] #1: ff110001eb850e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.315106] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.326254] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.336342]
[ 990.338807] =============================================\x0a
[ 1112.540142] INFO: task python3:5708 blocked for more than 245 seconds.
[ 1112.548289] Not tainted 7.1.0-rc2-ltws-g5bf6b77b793f #1
[ 1112.555539] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.565131] task:python3 state:D stack:24800 pid:5708 tgid:5708 ppid:5707 task_flags:0x400100 flags:0x00080000
[ 1112.578337] Call Trace:
[ 1112.581878]
[ 1112.585031] __schedule+0xcba/0x1940
[ 1112.589848] ? io_schedule_timeout+0x140/0x140
[ 1112.595634] ? find_held_lock+0x2b/0x80
[ 1112.600739] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.606628] schedule+0xd4/0x260
[ 1112.611047] schedule_preempt_disabled+0x18/0x30
[ 1112.617020] __mutex_lock+0x6a3/0x2000
[ 1112.622024] ? threaded_store+0x132/0x220
[ 1112.627317] ? ww_mutex_lock+0x160/0x160
[ 1112.632516] ? selinux_inode_free_security+0x330/0x330
[ 1112.639089] ? cap_capable+0x1d6/0x3e0
[ 1112.644096] ? threaded_store+0x132/0x220
[ 1112.649392] threaded_store+0x132/0x220
[ 1112.654497] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.659795] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.665096] ? sysfs_kf_write+0x65/0x160
[ 1112.670297] ? sysfs_file_ops+0x140/0x140
[ 1112.675584] kernfs_fop_write_iter+0x335/0x510
[ 1112.681370] ? kernfs_ops+0xf0/0xf0
[ 1112.686082] new_sync_write+0x333/0x750
[ 1112.691185] ? __lock_acquire+0x508/0xc10
[ 1112.696484] ? new_sync_read+0x740/0x740
[ 1112.701684] ? lock_acquire.part.0+0xbc/0x260
[ 1112.707372] ? ksys_write+0x116/0x250
[ 1112.712283] vfs_write+0x657/0x9b0
[ 1112.716898] ksys_write+0x116/0x250
[ 1112.721601] ? __ia32_sys_read+0xc0/0xc0
[ 1112.726798] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.732387] ? rcu_is_watching+0x15/0xd0
[ 1112.737582] do_syscall_64+0xf3/0x650
[ 1112.742487] ? trace_hardirqs_off+0xd/0x30
[ 1112.747883] ? exc_page_fault+0xda/0xf0
[ 1112.752986] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.759456] RIP: 0033:0x7f241152222e
[ 1112.764267] RSP: 002b:00007ffef0b0ee40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.773565] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f241152222e
[ 1112.782375] RDX: 0000000000000001 RSI: 00005573c0e56fe0 RDI: 0000000000000007
[ 1112.791185] RBP: 00007ffef0b0ee50 R08: 0000000000000000 R09: 0000000000000000
[ 1112.799996] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f241139a390
[ 1112.808806] R13: 00005573c0e56fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.817621]
[ 1112.820908] INFO: task python3:5708 is blocked on a mutex likely owned by task python3:3651.
[ 1112.831181] task:python3 state:D stack:25672 pid:3651 tgid:3651 ppid:3650 task_flags:0x400100 flags:0x00080802
[ 1112.844381] Call Trace:
[ 1112.847919]
[ 1112.851068] __schedule+0xcba/0x1940
[ 1112.855879] ? io_schedule_timeout+0x140/0x140
[ 1112.861654] ? find_held_lock+0x2b/0x80
[ 1112.866759] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.872936] schedule+0xd4/0x260
[ 1112.877352] schedule_timeout+0x13f/0x290
[ 1112.882649] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.889018] ? trace_contention_end+0xb3/0x1a0
[ 1112.894805] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.900495] ? ww_mutex_lock+0x160/0x160
[ 1112.905695] msleep+0x1b/0x30
[ 1112.909821] napi_set_threaded+0x3a5/0x440
[ 1112.915210] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.920707] netif_set_threaded+0x257/0x400
[ 1112.926201] threaded_store+0x1b8/0x220
[ 1112.931295] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.936592] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.941889] ? sysfs_kf_write+0x65/0x160
[ 1112.947091] ? sysfs_file_ops+0x140/0x140
[ 1112.952386] kernfs_fop_write_iter+0x335/0x510
[ 1112.958171] ? kernfs_ops+0xf0/0xf0
[ 1112.962882] new_sync_write+0x333/0x750
[ 1112.967985] ? __lock_acquire+0x508/0xc10
[ 1112.973272] ? new_sync_read+0x740/0x740
[ 1112.978474] ? lock_acquire.part.0+0xbc/0x260
[ 1112.984162] ? ksys_write+0x116/0x250
[ 1112.989072] vfs_write+0x657/0x9b0
[ 1112.993689] ksys_write+0x116/0x250
[ 1112.998402] ? __ia32_sys_read+0xc0/0xc0
[ 1113.003602] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.009192] ? rcu_is_watching+0x15/0xd0
[ 1113.014391] do_syscall_64+0xf3/0x650
[ 1113.019296] ? trace_hardirqs_off+0xd/0x30
[ 1113.024690] ? exc_page_fault+0xda/0xf0
[ 1113.029792] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.036261] RIP: 0033:0x7f592334622e
[ 1113.041070] RSP: 002b:00007ffcd09470d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.050358] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f592334622e
[ 1113.059157] RDX: 0000000000000001 RSI: 0000558f9ae57fe0 RDI: 0000000000000007
[ 1113.067965] RBP: 00007ffcd09470e0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.076776] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f59231be390
[ 1113.085585] R13: 0000558f9ae57fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.094399]
[ 1113.097655] \x0aShowing all locks held in the system:
[ 1113.106197] 1 lock held by khungtaskd/186:
[ 1113.111583] #0: ffffffffa7cfdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.123067] 4 locks held by python3/3651:
[ 1113.128364] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.138248] #1: ff1100016a22d080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.149211] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.160369] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.170463] 4 locks held by python3/5708:
[ 1113.175759] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.185651] #1: ff110001eb850e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.196614] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.207771] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.217851]
[ 1113.220315] =============================================\x0a
[ 1231.324279] kworker/u129:1 (187) used greatest stack depth: 21936 bytes left
[ 1235.420367] INFO: task python3:5708 blocked for more than 368 seconds.
[ 1235.428514] Not tainted 7.1.0-rc2-ltws-g5bf6b77b793f #1
[ 1235.435764] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.445351] task:python3 state:D stack:24800 pid:5708 tgid:5708 ppid:5707 task_flags:0x400100 flags:0x00080002
[ 1235.458557] Call Trace:
[ 1235.462096]
[ 1235.465245] __schedule+0xcba/0x1940
[ 1235.470063] ? io_schedule_timeout+0x140/0x140
[ 1235.475847] ? find_held_lock+0x2b/0x80
[ 1235.480951] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.486839] schedule+0xd4/0x260
[ 1235.491257] schedule_preempt_disabled+0x18/0x30
[ 1235.497237] __mutex_lock+0x6a3/0x2000
[ 1235.502241] ? threaded_store+0x132/0x220
[ 1235.507538] ? ww_mutex_lock+0x160/0x160
[ 1235.512737] ? selinux_inode_free_security+0x330/0x330
[ 1235.519305] ? cap_capable+0x1d6/0x3e0
[ 1235.524311] ? threaded_store+0x132/0x220
[ 1235.529597] threaded_store+0x132/0x220
[ 1235.534699] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.539996] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.545297] ? sysfs_kf_write+0x65/0x160
[ 1235.550497] ? sysfs_file_ops+0x140/0x140
[ 1235.555782] kernfs_fop_write_iter+0x335/0x510
[ 1235.561567] ? kernfs_ops+0xf0/0xf0
[ 1235.566278] new_sync_write+0x333/0x750
[ 1235.571371] ? __lock_acquire+0x508/0xc10
[ 1235.576666] ? new_sync_read+0x740/0x740
[ 1235.581867] ? lock_acquire.part.0+0xbc/0x260
[ 1235.587552] ? ksys_write+0x116/0x250
[ 1235.592459] vfs_write+0x657/0x9b0
[ 1235.597071] ksys_write+0x116/0x250
[ 1235.601783] ? __ia32_sys_read+0xc0/0xc0
[ 1235.606982] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.612570] ? rcu_is_watching+0x15/0xd0
[ 1235.617762] do_syscall_64+0xf3/0x650
[ 1235.622661] ? trace_hardirqs_off+0xd/0x30
[ 1235.628046] ? exc_page_fault+0xda/0xf0
[ 1235.633147] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.639616] RIP: 0033:0x7f241152222e
[ 1235.644425] RSP: 002b:00007ffef0b0ee40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.653723] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f241152222e
[ 1235.662532] RDX: 0000000000000001 RSI: 00005573c0e56fe0 RDI: 0000000000000007
[ 1235.671339] RBP: 00007ffef0b0ee50 R08: 0000000000000000 R09: 0000000000000000
[ 1235.680142] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f241139a390
[ 1235.688948] R13: 00005573c0e56fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.697758]
[ 1235.701037] INFO: task python3:5708 is blocked on a mutex likely owned by task python3:3651.
[ 1235.711308] task:python3 state:D stack:25672 pid:3651 tgid:3651 ppid:3650 task_flags:0x400100 flags:0x00080802
[ 1235.724510] Call Trace:
[ 1235.728049]
[ 1235.731198] __schedule+0xcba/0x1940
[ 1235.736007] ? io_schedule_timeout+0x140/0x140
[ 1235.741791] ? find_held_lock+0x2b/0x80
[ 1235.746894] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.753072] schedule+0xd4/0x260
[ 1235.757491] schedule_timeout+0x13f/0x290
[ 1235.762787] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.769158] ? trace_contention_end+0xb3/0x1a0
[ 1235.774932] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.780620] ? ww_mutex_lock+0x160/0x160
[ 1235.785810] msleep+0x1b/0x30
[ 1235.789926] napi_set_threaded+0x3a5/0x440
[ 1235.795320] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.800813] netif_set_threaded+0x257/0x400
[ 1235.806307] threaded_store+0x1b8/0x220
[ 1235.811406] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.816701] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.821998] ? sysfs_kf_write+0x65/0x160
[ 1235.827198] ? sysfs_file_ops+0x140/0x140
[ 1235.832492] kernfs_fop_write_iter+0x335/0x510
[ 1235.838275] ? kernfs_ops+0xf0/0xf0
[ 1235.842983] new_sync_write+0x333/0x750
[ 1235.848085] ? __lock_acquire+0x508/0xc10
[ 1235.853381] ? new_sync_read+0x740/0x740
[ 1235.858578] ? lock_acquire.part.0+0xbc/0x260
[ 1235.864262] ? ksys_write+0x116/0x250
[ 1235.869172] vfs_write+0x657/0x9b0
[ 1235.873784] ksys_write+0x116/0x250
[ 1235.878495] ? __ia32_sys_read+0xc0/0xc0
[ 1235.883694] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.889282] ? rcu_is_watching+0x15/0xd0
[ 1235.894481] do_syscall_64+0xf3/0x650
[ 1235.899386] ? trace_hardirqs_off+0xd/0x30
[ 1235.904777] ? exc_page_fault+0xda/0xf0
[ 1235.909879] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.916345] RIP: 0033:0x7f592334622e
[ 1235.921151] RSP: 002b:00007ffcd09470d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.930447] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f592334622e
[ 1235.939255] RDX: 0000000000000001 RSI: 0000558f9ae57fe0 RDI: 0000000000000007
[ 1235.948054] RBP: 00007ffcd09470e0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.956860] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f59231be390
[ 1235.965667] R13: 0000558f9ae57fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.974478]
[ 1235.977743] \x0aShowing all locks held in the system:
[ 1235.986280] 1 lock held by khungtaskd/186:
[ 1235.991675] #0: ffffffffa7cfdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.003155] 4 locks held by python3/3651:
[ 1236.008450] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.018338] #1: ff1100016a22d080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.029300] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.040457] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.050550] 4 locks held by python3/5708:
[ 1236.055843] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.065730] #1: ff110001eb850e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.076690] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.087847] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.097936]
[ 1236.100394] =============================================\x0a
[ 1358.301618] INFO: task python3:5708 blocked for more than 491 seconds.
[ 1358.309764] Not tainted 7.1.0-rc2-ltws-g5bf6b77b793f #1
[ 1358.317014] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.326606] task:python3 state:D stack:24800 pid:5708 tgid:5708 ppid:5707 task_flags:0x400100 flags:0x00080002
[ 1358.339812] Call Trace:
[ 1358.343351]
[ 1358.346502] __schedule+0xcba/0x1940
[ 1358.351319] ? io_schedule_timeout+0x140/0x140
[ 1358.357105] ? find_held_lock+0x2b/0x80
[ 1358.362210] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.368095] schedule+0xd4/0x260
[ 1358.372515] schedule_preempt_disabled+0x18/0x30
[ 1358.378496] __mutex_lock+0x6a3/0x2000
[ 1358.383503] ? threaded_store+0x132/0x220
[ 1358.388804] ? ww_mutex_lock+0x160/0x160
[ 1358.394006] ? selinux_inode_free_security+0x330/0x330
[ 1358.400575] ? cap_capable+0x1d6/0x3e0
[ 1358.405582] ? threaded_store+0x132/0x220
[ 1358.410878] threaded_store+0x132/0x220
[ 1358.415983] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.421280] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.426581] ? sysfs_kf_write+0x65/0x160
[ 1358.431782] ? sysfs_file_ops+0x140/0x140
[ 1358.437077] kernfs_fop_write_iter+0x335/0x510
[ 1358.442862] ? kernfs_ops+0xf0/0xf0
[ 1358.447567] new_sync_write+0x333/0x750
[ 1358.452671] ? __lock_acquire+0x508/0xc10
[ 1358.457966] ? new_sync_read+0x740/0x740
[ 1358.463169] ? lock_acquire.part.0+0xbc/0x260
[ 1358.468858] ? ksys_write+0x116/0x250
[ 1358.473769] vfs_write+0x657/0x9b0
[ 1358.478385] ksys_write+0x116/0x250
[ 1358.483095] ? __ia32_sys_read+0xc0/0xc0
[ 1358.488293] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.493882] ? rcu_is_watching+0x15/0xd0
[ 1358.499087] do_syscall_64+0xf3/0x650
[ 1358.503995] ? trace_hardirqs_off+0xd/0x30
[ 1358.509389] ? exc_page_fault+0xda/0xf0
[ 1358.514492] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.520961] RIP: 0033:0x7f241152222e
[ 1358.525769] RSP: 002b:00007ffef0b0ee40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.535066] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f241152222e
[ 1358.543866] RDX: 0000000000000001 RSI: 00005573c0e56fe0 RDI: 0000000000000007
[ 1358.552676] RBP: 00007ffef0b0ee50 R08: 0000000000000000 R09: 0000000000000000
[ 1358.561486] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f241139a390
[ 1358.570296] R13: 00005573c0e56fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.579108]
[ 1358.582390] INFO: task python3:5708 is blocked on a mutex likely owned by task python3:3651.
[ 1358.592665] task:python3 state:D stack:25672 pid:3651 tgid:3651 ppid:3650 task_flags:0x400100 flags:0x00080802
[ 1358.605858] Call Trace:
[ 1358.609398]
[ 1358.612551] __schedule+0xcba/0x1940
[ 1358.617362] ? io_schedule_timeout+0x140/0x140
[ 1358.623146] ? find_held_lock+0x2b/0x80
[ 1358.628250] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.634430] schedule+0xd4/0x260
[ 1358.638850] schedule_timeout+0x13f/0x290
[ 1358.644137] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.650505] ? trace_contention_end+0xb3/0x1a0
[ 1358.656292] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.661982] ? ww_mutex_lock+0x160/0x160
[ 1358.667181] msleep+0x1b/0x30
[ 1358.671306] napi_set_threaded+0x3a5/0x440
[ 1358.676701] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.682194] netif_set_threaded+0x257/0x400
[ 1358.687687] threaded_store+0x1b8/0x220
[ 1358.692779] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.698076] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.703375] ? sysfs_kf_write+0x65/0x160
[ 1358.708575] ? sysfs_file_ops+0x140/0x140
[ 1358.713861] kernfs_fop_write_iter+0x335/0x510
[ 1358.719644] ? kernfs_ops+0xf0/0xf0
[ 1358.724354] new_sync_write+0x333/0x750
[ 1358.729457] ? __lock_acquire+0x508/0xc10
[ 1358.734753] ? new_sync_read+0x740/0x740
[ 1358.739955] ? lock_acquire.part.0+0xbc/0x260
[ 1358.745642] ? ksys_write+0x116/0x250
[ 1358.750553] vfs_write+0x657/0x9b0
[ 1358.755169] ksys_write+0x116/0x250
[ 1358.759883] ? __ia32_sys_read+0xc0/0xc0
[ 1358.765084] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.770673] ? rcu_is_watching+0x15/0xd0
[ 1358.775874] do_syscall_64+0xf3/0x650
[ 1358.780783] ? trace_hardirqs_off+0xd/0x30
[ 1358.786178] ? exc_page_fault+0xda/0xf0
[ 1358.791279] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.797746] RIP: 0033:0x7f592334622e
[ 1358.802555] RSP: 002b:00007ffcd09470d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.811846] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f592334622e
[ 1358.820655] RDX: 0000000000000001 RSI: 0000558f9ae57fe0 RDI: 0000000000000007
[ 1358.829465] RBP: 00007ffcd09470e0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.838274] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f59231be390
[ 1358.847083] R13: 0000558f9ae57fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.855897]
[ 1358.859162] \x0aShowing all locks held in the system:
[ 1358.867702] 1 lock held by khungtaskd/186:
[ 1358.873097] #0: ffffffffa7cfdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.884574] 4 locks held by python3/3651:
[ 1358.889869] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.899758] #1: ff1100016a22d080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.910719] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.921876] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.931961] 4 locks held by python3/5708:
[ 1358.937256] #0: ff110017d99243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.947145] #1: ff110001eb850e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.958108] #2: ff11000121a28b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.969266] #3: ff11000121a01d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.979360]
[ 1358.981826] =============================================\x0a