[ 789.185252] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 789.187811] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 789.314811] kselftest: Running tests in drivers/net
[ 961.004519] kworker/u130:3 (527) used greatest stack depth: 21456 bytes left
[ 989.677736] INFO: task python3:5803 blocked for more than 122 seconds.
[ 989.685927] Not tainted 7.1.0-rc5-tvoo-g22155531bf5e #1
[ 989.693575] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.703163] task:python3 state:D stack:25672 pid:5803 tgid:5803 ppid:5802 task_flags:0x400100 flags:0x00080000
[ 989.716366] Call Trace:
[ 989.719905]
[ 989.723055] __schedule+0xcc3/0x19c0
[ 989.727873] ? io_schedule_timeout+0x140/0x140
[ 989.733658] ? find_held_lock+0x2b/0x80
[ 989.738764] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.744649] schedule+0xd8/0x260
[ 989.749067] schedule_preempt_disabled+0x18/0x30
[ 989.755045] __mutex_lock+0x6af/0x2010
[ 989.760051] ? threaded_store+0x132/0x220
[ 989.765350] ? ww_mutex_lock+0x160/0x160
[ 989.770547] ? selinux_inode_free_security+0x330/0x330
[ 989.777117] ? cap_capable+0x1d6/0x3e0
[ 989.782129] ? threaded_store+0x132/0x220
[ 989.787432] threaded_store+0x132/0x220
[ 989.792535] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.797831] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.803119] ? sysfs_kf_write+0x65/0x160
[ 989.808316] ? sysfs_file_ops+0x140/0x140
[ 989.813609] kernfs_fop_write_iter+0x335/0x510
[ 989.819392] ? kernfs_ops+0xf0/0xf0
[ 989.824101] new_sync_write+0x333/0x750
[ 989.829202] ? __lock_acquire+0x508/0xc10
[ 989.834500] ? new_sync_read+0x740/0x740
[ 989.839701] ? lock_acquire.part.0+0xbc/0x260
[ 989.845387] ? ksys_write+0x116/0x250
[ 989.850295] vfs_write+0x657/0x9b0
[ 989.854910] ksys_write+0x116/0x250
[ 989.859613] ? __ia32_sys_read+0xc0/0xc0
[ 989.864811] ? rcu_is_watching+0x15/0xd0
[ 989.870008] ? rcu_is_watching+0x15/0xd0
[ 989.875205] do_syscall_64+0xf3/0x650
[ 989.880112] ? irq_exit_rcu+0x1a/0x30
[ 989.885019] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.891486] RIP: 0033:0x7f27879a522e
[ 989.896285] RSP: 002b:00007fff226cbe80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.905584] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f27879a522e
[ 989.914391] RDX: 0000000000000001 RSI: 00005645e82896b0 RDI: 0000000000000007
[ 989.923196] RBP: 00007fff226cbe90 R08: 0000000000000000 R09: 0000000000000000
[ 989.932001] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f278781d390
[ 989.940810] R13: 00005645e82896b0 R14: 0000000000000007 R15: 0000000000000003
[ 989.949628]
[ 989.952908] INFO: task python3:5803 is blocked on a mutex likely owned by task python3:3777.
[ 989.963180] task:python3 state:D stack:25672 pid:3777 tgid:3777 ppid:3776 task_flags:0x400100 flags:0x00080802
[ 989.976384] Call Trace:
[ 989.979920]
[ 989.983070] __schedule+0xcc3/0x19c0
[ 989.987879] ? io_schedule_timeout+0x140/0x140
[ 989.993655] ? find_held_lock+0x2b/0x80
[ 989.998757] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.004934] schedule+0xd8/0x260
[ 990.009349] schedule_timeout+0x13f/0x290
[ 990.014643] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.021011] ? trace_contention_end+0xb3/0x1a0
[ 990.026793] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.032480] ? ww_mutex_lock+0x160/0x160
[ 990.037679] msleep+0x1b/0x30
[ 990.041796] napi_set_threaded+0x3a5/0x440
[ 990.047191] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.052683] netif_set_threaded+0x257/0x400
[ 990.058177] threaded_store+0x1b8/0x220
[ 990.063277] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.068575] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.073861] ? sysfs_kf_write+0x65/0x160
[ 990.079059] ? sysfs_file_ops+0x140/0x140
[ 990.084352] kernfs_fop_write_iter+0x335/0x510
[ 990.090135] ? kernfs_ops+0xf0/0xf0
[ 990.094847] new_sync_write+0x333/0x750
[ 990.099949] ? __lock_acquire+0x508/0xc10
[ 990.105242] ? new_sync_read+0x740/0x740
[ 990.110441] ? lock_acquire.part.0+0xbc/0x260
[ 990.116127] ? ksys_write+0x116/0x250
[ 990.121034] vfs_write+0x657/0x9b0
[ 990.125639] ksys_write+0x116/0x250
[ 990.130348] ? __ia32_sys_read+0xc0/0xc0
[ 990.135545] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.141131] ? rcu_is_watching+0x15/0xd0
[ 990.146328] do_syscall_64+0xf3/0x650
[ 990.151231] ? trace_hardirqs_off+0xd/0x30
[ 990.156623] ? exc_page_fault+0xda/0xf0
[ 990.161722] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.168186] RIP: 0033:0x7fc2aa30122e
[ 990.172992] RSP: 002b:00007ffd563f91f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.182288] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc2aa30122e
[ 990.191096] RDX: 0000000000000001 RSI: 0000559dc408d6b0 RDI: 0000000000000007
[ 990.199905] RBP: 00007ffd563f9200 R08: 0000000000000000 R09: 0000000000000000
[ 990.208711] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc2aa179390
[ 990.217517] R13: 0000559dc408d6b0 R14: 0000000000000007 R15: 0000000000000003
[ 990.226330]
[ 990.229585] \x0aShowing all locks held in the system:
[ 990.238123] 1 lock held by khungtaskd/184:
[ 990.243514] #0: ffffffff94afdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.254992] 4 locks held by python3/3777:
[ 990.260287] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.270177] #1: ff110001614bca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.281137] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.292291] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.302382] 4 locks held by python3/5803:
[ 990.307666] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.317574] #1: ff110001e88af480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.328542] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.339697] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.349781]
[ 990.352246] =============================================\x0a
[ 1112.558768] INFO: task python3:5803 blocked for more than 245 seconds.
[ 1112.566915] Not tainted 7.1.0-rc5-tvoo-g22155531bf5e #1
[ 1112.574165] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.583752] task:python3 state:D stack:25672 pid:5803 tgid:5803 ppid:5802 task_flags:0x400100 flags:0x00080000
[ 1112.596955] Call Trace:
[ 1112.600494]
[ 1112.603645] __schedule+0xcc3/0x19c0
[ 1112.608458] ? io_schedule_timeout+0x140/0x140
[ 1112.614241] ? find_held_lock+0x2b/0x80
[ 1112.619337] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.625220] schedule+0xd8/0x260
[ 1112.629638] schedule_preempt_disabled+0x18/0x30
[ 1112.635618] __mutex_lock+0x6af/0x2010
[ 1112.640613] ? threaded_store+0x132/0x220
[ 1112.645903] ? ww_mutex_lock+0x160/0x160
[ 1112.651101] ? selinux_inode_free_security+0x330/0x330
[ 1112.657670] ? cap_capable+0x1d6/0x3e0
[ 1112.662679] ? threaded_store+0x132/0x220
[ 1112.667977] threaded_store+0x132/0x220
[ 1112.673077] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.678374] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.683670] ? sysfs_kf_write+0x65/0x160
[ 1112.688868] ? sysfs_file_ops+0x140/0x140
[ 1112.694164] kernfs_fop_write_iter+0x335/0x510
[ 1112.699945] ? kernfs_ops+0xf0/0xf0
[ 1112.704658] new_sync_write+0x333/0x750
[ 1112.709758] ? __lock_acquire+0x508/0xc10
[ 1112.715054] ? new_sync_read+0x740/0x740
[ 1112.720255] ? lock_acquire.part.0+0xbc/0x260
[ 1112.725938] ? ksys_write+0x116/0x250
[ 1112.730849] vfs_write+0x657/0x9b0
[ 1112.735461] ksys_write+0x116/0x250
[ 1112.740170] ? __ia32_sys_read+0xc0/0xc0
[ 1112.745368] ? rcu_is_watching+0x15/0xd0
[ 1112.750566] ? rcu_is_watching+0x15/0xd0
[ 1112.755765] do_syscall_64+0xf3/0x650
[ 1112.760660] ? irq_exit_rcu+0x1a/0x30
[ 1112.765565] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.772031] RIP: 0033:0x7f27879a522e
[ 1112.776839] RSP: 002b:00007fff226cbe80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.786135] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f27879a522e
[ 1112.794943] RDX: 0000000000000001 RSI: 00005645e82896b0 RDI: 0000000000000007
[ 1112.803740] RBP: 00007fff226cbe90 R08: 0000000000000000 R09: 0000000000000000
[ 1112.812547] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f278781d390
[ 1112.821344] R13: 00005645e82896b0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.830157]
[ 1112.833423] INFO: task python3:5803 is blocked on a mutex likely owned by task python3:3777.
[ 1112.843688] task:python3 state:D stack:25672 pid:3777 tgid:3777 ppid:3776 task_flags:0x400100 flags:0x00080802
[ 1112.856891] Call Trace:
[ 1112.860431]
[ 1112.863578] __schedule+0xcc3/0x19c0
[ 1112.868388] ? io_schedule_timeout+0x140/0x140
[ 1112.874169] ? find_held_lock+0x2b/0x80
[ 1112.879268] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.885446] schedule+0xd8/0x260
[ 1112.889864] schedule_timeout+0x13f/0x290
[ 1112.895154] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.901513] ? trace_contention_end+0xb3/0x1a0
[ 1112.907296] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.912986] ? ww_mutex_lock+0x160/0x160
[ 1112.918182] msleep+0x1b/0x30
[ 1112.922304] napi_set_threaded+0x3a5/0x440
[ 1112.927696] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.933188] netif_set_threaded+0x257/0x400
[ 1112.938680] threaded_store+0x1b8/0x220
[ 1112.943781] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.949079] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.954375] ? sysfs_kf_write+0x65/0x160
[ 1112.959573] ? sysfs_file_ops+0x140/0x140
[ 1112.964867] kernfs_fop_write_iter+0x335/0x510
[ 1112.970648] ? kernfs_ops+0xf0/0xf0
[ 1112.975356] new_sync_write+0x333/0x750
[ 1112.980457] ? __lock_acquire+0x508/0xc10
[ 1112.985750] ? new_sync_read+0x740/0x740
[ 1112.990949] ? lock_acquire.part.0+0xbc/0x260
[ 1112.996632] ? ksys_write+0x116/0x250
[ 1113.001540] vfs_write+0x657/0x9b0
[ 1113.006156] ksys_write+0x116/0x250
[ 1113.010868] ? __ia32_sys_read+0xc0/0xc0
[ 1113.016068] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.021656] ? rcu_is_watching+0x15/0xd0
[ 1113.026856] do_syscall_64+0xf3/0x650
[ 1113.031753] ? trace_hardirqs_off+0xd/0x30
[ 1113.037146] ? exc_page_fault+0xda/0xf0
[ 1113.042246] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.048713] RIP: 0033:0x7fc2aa30122e
[ 1113.053521] RSP: 002b:00007ffd563f91f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.062814] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc2aa30122e
[ 1113.071620] RDX: 0000000000000001 RSI: 0000559dc408d6b0 RDI: 0000000000000007
[ 1113.080427] RBP: 00007ffd563f9200 R08: 0000000000000000 R09: 0000000000000000
[ 1113.089233] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc2aa179390
[ 1113.098037] R13: 0000559dc408d6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.106850]
[ 1113.110113] \x0aShowing all locks held in the system:
[ 1113.118649] 1 lock held by khungtaskd/184:
[ 1113.124042] #0: ffffffff94afdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.135509] 2 locks held by systemd-journal/802:
[ 1113.141485] #0: ff1100180e0372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.151373] #1: ffffffff94d02c20 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1113.163615] 4 locks held by python3/3777:
[ 1113.168909] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.178795] #1: ff110001614bca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.189757] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.200903] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.210994] 4 locks held by python3/5803:
[ 1113.216289] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.226179] #1: ff110001e88af480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.237138] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.248292] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.258377]
[ 1113.260842] =============================================\x0a
[ 1235.439873] INFO: task python3:5803 blocked for more than 368 seconds.
[ 1235.448027] Not tainted 7.1.0-rc5-tvoo-g22155531bf5e #1
[ 1235.455276] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.464866] task:python3 state:D stack:25672 pid:5803 tgid:5803 ppid:5802 task_flags:0x400100 flags:0x00080002
[ 1235.478072] Call Trace:
[ 1235.481614]
[ 1235.484763] __schedule+0xcc3/0x19c0
[ 1235.489581] ? io_schedule_timeout+0x140/0x140
[ 1235.495370] ? find_held_lock+0x2b/0x80
[ 1235.500476] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.506363] schedule+0xd8/0x260
[ 1235.510782] schedule_preempt_disabled+0x18/0x30
[ 1235.516763] __mutex_lock+0x6af/0x2010
[ 1235.521772] ? threaded_store+0x132/0x220
[ 1235.527071] ? ww_mutex_lock+0x160/0x160
[ 1235.532270] ? selinux_inode_free_security+0x330/0x330
[ 1235.538839] ? cap_capable+0x1d6/0x3e0
[ 1235.543850] ? threaded_store+0x132/0x220
[ 1235.549150] threaded_store+0x132/0x220
[ 1235.554254] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.559552] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.564849] ? sysfs_kf_write+0x65/0x160
[ 1235.570049] ? sysfs_file_ops+0x140/0x140
[ 1235.575337] kernfs_fop_write_iter+0x335/0x510
[ 1235.581123] ? kernfs_ops+0xf0/0xf0
[ 1235.585834] new_sync_write+0x333/0x750
[ 1235.590929] ? __lock_acquire+0x508/0xc10
[ 1235.596226] ? new_sync_read+0x740/0x740
[ 1235.601420] ? lock_acquire.part.0+0xbc/0x260
[ 1235.607107] ? ksys_write+0x116/0x250
[ 1235.612019] vfs_write+0x657/0x9b0
[ 1235.616636] ksys_write+0x116/0x250
[ 1235.621350] ? __ia32_sys_read+0xc0/0xc0
[ 1235.626551] ? rcu_is_watching+0x15/0xd0
[ 1235.631753] ? rcu_is_watching+0x15/0xd0
[ 1235.636946] do_syscall_64+0xf3/0x650
[ 1235.641854] ? irq_exit_rcu+0x1a/0x30
[ 1235.646763] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.653234] RIP: 0033:0x7f27879a522e
[ 1235.658044] RSP: 002b:00007fff226cbe80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.667336] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f27879a522e
[ 1235.676144] RDX: 0000000000000001 RSI: 00005645e82896b0 RDI: 0000000000000007
[ 1235.684942] RBP: 00007fff226cbe90 R08: 0000000000000000 R09: 0000000000000000
[ 1235.693749] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f278781d390
[ 1235.702558] R13: 00005645e82896b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.711372]
[ 1235.714642] INFO: task python3:5803 is blocked on a mutex likely owned by task python3:3777.
[ 1235.724917] task:python3 state:D stack:25672 pid:3777 tgid:3777 ppid:3776 task_flags:0x400100 flags:0x00080802
[ 1235.738121] Call Trace:
[ 1235.741660]
[ 1235.744811] __schedule+0xcc3/0x19c0
[ 1235.749625] ? io_schedule_timeout+0x140/0x140
[ 1235.755410] ? find_held_lock+0x2b/0x80
[ 1235.760505] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.766686] schedule+0xd8/0x260
[ 1235.771107] schedule_timeout+0x13f/0x290
[ 1235.776405] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.782766] ? trace_contention_end+0xb3/0x1a0
[ 1235.788545] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.794237] ? ww_mutex_lock+0x160/0x160
[ 1235.799437] msleep+0x1b/0x30
[ 1235.803563] napi_set_threaded+0x3a5/0x440
[ 1235.808958] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.814453] netif_set_threaded+0x257/0x400
[ 1235.819948] threaded_store+0x1b8/0x220
[ 1235.825051] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.830349] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.835638] ? sysfs_kf_write+0x65/0x160
[ 1235.840839] ? sysfs_file_ops+0x140/0x140
[ 1235.846136] kernfs_fop_write_iter+0x335/0x510
[ 1235.851922] ? kernfs_ops+0xf0/0xf0
[ 1235.856635] new_sync_write+0x333/0x750
[ 1235.861736] ? __lock_acquire+0x508/0xc10
[ 1235.867031] ? new_sync_read+0x740/0x740
[ 1235.872233] ? lock_acquire.part.0+0xbc/0x260
[ 1235.877920] ? ksys_write+0x116/0x250
[ 1235.882828] vfs_write+0x657/0x9b0
[ 1235.887445] ksys_write+0x116/0x250
[ 1235.892159] ? __ia32_sys_read+0xc0/0xc0
[ 1235.897358] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.902948] ? rcu_is_watching+0x15/0xd0
[ 1235.908143] do_syscall_64+0xf3/0x650
[ 1235.913051] ? trace_hardirqs_off+0xd/0x30
[ 1235.918447] ? exc_page_fault+0xda/0xf0
[ 1235.923551] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.930018] RIP: 0033:0x7fc2aa30122e
[ 1235.934826] RSP: 002b:00007ffd563f91f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.944123] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc2aa30122e
[ 1235.952931] RDX: 0000000000000001 RSI: 0000559dc408d6b0 RDI: 0000000000000007
[ 1235.961731] RBP: 00007ffd563f9200 R08: 0000000000000000 R09: 0000000000000000
[ 1235.970538] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc2aa179390
[ 1235.979339] R13: 0000559dc408d6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.988155]
[ 1235.991420] \x0aShowing all locks held in the system:
[ 1235.999959] 1 lock held by khungtaskd/184:
[ 1236.005353] #0: ffffffff94afdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.016824] 2 locks held by systemd-journal/802:
[ 1236.022805] #0: ff1100180e3372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.032696] #1: ffffffff94d02c20 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1236.044935] 4 locks held by python3/3777:
[ 1236.050231] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.060122] #1: ff110001614bca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.071084] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.082240] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.092337] 4 locks held by python3/5803:
[ 1236.097638] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.107529] #1: ff110001e88af480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.118489] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.129644] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.139737]
[ 1236.142202] =============================================\x0a
[ 1358.321049] INFO: task python3:5803 blocked for more than 491 seconds.
[ 1358.329198] Not tainted 7.1.0-rc5-tvoo-g22155531bf5e #1
[ 1358.336447] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.346037] task:python3 state:D stack:25672 pid:5803 tgid:5803 ppid:5802 task_flags:0x400100 flags:0x00080002
[ 1358.359242] Call Trace:
[ 1358.362784]
[ 1358.365938] __schedule+0xcc3/0x19c0
[ 1358.370757] ? io_schedule_timeout+0x140/0x140
[ 1358.376543] ? find_held_lock+0x2b/0x80
[ 1358.381649] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.387533] schedule+0xd8/0x260
[ 1358.391952] schedule_preempt_disabled+0x18/0x30
[ 1358.397929] __mutex_lock+0x6af/0x2010
[ 1358.402937] ? threaded_store+0x132/0x220
[ 1358.408235] ? ww_mutex_lock+0x160/0x160
[ 1358.413435] ? selinux_inode_free_security+0x330/0x330
[ 1358.420005] ? cap_capable+0x1d6/0x3e0
[ 1358.425017] ? threaded_store+0x132/0x220
[ 1358.430314] threaded_store+0x132/0x220
[ 1358.435408] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.440703] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.446000] ? sysfs_kf_write+0x65/0x160
[ 1358.451201] ? sysfs_file_ops+0x140/0x140
[ 1358.456495] kernfs_fop_write_iter+0x335/0x510
[ 1358.462281] ? kernfs_ops+0xf0/0xf0
[ 1358.466983] new_sync_write+0x333/0x750
[ 1358.472080] ? __lock_acquire+0x508/0xc10
[ 1358.477378] ? new_sync_read+0x740/0x740
[ 1358.482582] ? lock_acquire.part.0+0xbc/0x260
[ 1358.488268] ? ksys_write+0x116/0x250
[ 1358.493173] vfs_write+0x657/0x9b0
[ 1358.497792] ksys_write+0x116/0x250
[ 1358.502505] ? __ia32_sys_read+0xc0/0xc0
[ 1358.507698] ? rcu_is_watching+0x15/0xd0
[ 1358.512902] ? rcu_is_watching+0x15/0xd0
[ 1358.518103] do_syscall_64+0xf3/0x650
[ 1358.523009] ? irq_exit_rcu+0x1a/0x30
[ 1358.527917] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.534385] RIP: 0033:0x7f27879a522e
[ 1358.539196] RSP: 002b:00007fff226cbe80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.548494] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f27879a522e
[ 1358.557303] RDX: 0000000000000001 RSI: 00005645e82896b0 RDI: 0000000000000007
[ 1358.566109] RBP: 00007fff226cbe90 R08: 0000000000000000 R09: 0000000000000000
[ 1358.574916] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f278781d390
[ 1358.583725] R13: 00005645e82896b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.592538]
[ 1358.595802] INFO: task python3:5803 is blocked on a mutex likely owned by task python3:3777.
[ 1358.606078] task:python3 state:D stack:25672 pid:3777 tgid:3777 ppid:3776 task_flags:0x400100 flags:0x00080802
[ 1358.619271] Call Trace:
[ 1358.622809]
[ 1358.625961] __schedule+0xcc3/0x19c0
[ 1358.630774] ? io_schedule_timeout+0x140/0x140
[ 1358.636559] ? find_held_lock+0x2b/0x80
[ 1358.641664] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.647842] schedule+0xd8/0x260
[ 1358.652262] schedule_timeout+0x13f/0x290
[ 1358.657559] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.663931] ? trace_contention_end+0xb3/0x1a0
[ 1358.669714] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.675406] ? ww_mutex_lock+0x160/0x160
[ 1358.680606] msleep+0x1b/0x30
[ 1358.684734] napi_set_threaded+0x3a5/0x440
[ 1358.690130] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.695628] netif_set_threaded+0x257/0x400
[ 1358.701122] threaded_store+0x1b8/0x220
[ 1358.706226] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.711525] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.716822] ? sysfs_kf_write+0x65/0x160
[ 1358.722023] ? sysfs_file_ops+0x140/0x140
[ 1358.727320] kernfs_fop_write_iter+0x335/0x510
[ 1358.733107] ? kernfs_ops+0xf0/0xf0
[ 1358.737820] new_sync_write+0x333/0x750
[ 1358.742921] ? __lock_acquire+0x508/0xc10
[ 1358.748219] ? new_sync_read+0x740/0x740
[ 1358.753410] ? lock_acquire.part.0+0xbc/0x260
[ 1358.759089] ? ksys_write+0x116/0x250
[ 1358.763998] vfs_write+0x657/0x9b0
[ 1358.768613] ksys_write+0x116/0x250
[ 1358.773325] ? __ia32_sys_read+0xc0/0xc0
[ 1358.778525] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.784115] ? rcu_is_watching+0x15/0xd0
[ 1358.789305] do_syscall_64+0xf3/0x650
[ 1358.794210] ? trace_hardirqs_off+0xd/0x30
[ 1358.799607] ? exc_page_fault+0xda/0xf0
[ 1358.804710] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.811172] RIP: 0033:0x7fc2aa30122e
[ 1358.815982] RSP: 002b:00007ffd563f91f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.825281] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc2aa30122e
[ 1358.834090] RDX: 0000000000000001 RSI: 0000559dc408d6b0 RDI: 0000000000000007
[ 1358.842902] RBP: 00007ffd563f9200 R08: 0000000000000000 R09: 0000000000000000
[ 1358.851709] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc2aa179390
[ 1358.860508] R13: 0000559dc408d6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.869323]
[ 1358.872590] \x0aShowing all locks held in the system:
[ 1358.881129] 1 lock held by khungtaskd/184:
[ 1358.886525] #0: ffffffff94afdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.898007] 4 locks held by python3/3777:
[ 1358.903301] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.913193] #1: ff110001614bca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.924156] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.935313] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.945395] 4 locks held by python3/5803:
[ 1358.950684] #0: ff1100012ff5c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.960573] #1: ff110001e88af480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.971536] #2: ff110001232d4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.982695] #3: ff110001232e1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.992790]
[ 1358.995258] =============================================\x0a