[ 790.310965] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 790.313490] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 790.446430] kselftest: Running tests in drivers/net
[ 813.540631] kworker/u129:14 (311) used greatest stack depth: 21456 bytes left
[ 989.670508] INFO: task python3:5850 blocked for more than 122 seconds.
[ 989.678743] Not tainted 7.1.0-rc3-rnfr-g25a6e3c1d2bf #1
[ 989.686745] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.696340] task:python3 state:D stack:25672 pid:5850 tgid:5850 ppid:5849 task_flags:0x400100 flags:0x00080000
[ 989.709545] Call Trace:
[ 989.713084]
[ 989.716235] __schedule+0xcc3/0x19c0
[ 989.721053] ? io_schedule_timeout+0x140/0x140
[ 989.726837] ? find_held_lock+0x2b/0x80
[ 989.731943] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.737826] schedule+0xd4/0x260
[ 989.742234] schedule_preempt_disabled+0x18/0x30
[ 989.748211] __mutex_lock+0x6a3/0x2000
[ 989.753217] ? threaded_store+0x132/0x220
[ 989.758518] ? ww_mutex_lock+0x160/0x160
[ 989.763716] ? selinux_inode_free_security+0x330/0x330
[ 989.770285] ? cap_capable+0x1d6/0x3e0
[ 989.775292] ? threaded_store+0x132/0x220
[ 989.780588] threaded_store+0x132/0x220
[ 989.785690] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.790980] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.796281] ? sysfs_kf_write+0x65/0x160
[ 989.801479] ? sysfs_file_ops+0x140/0x140
[ 989.806774] kernfs_fop_write_iter+0x335/0x510
[ 989.812555] ? kernfs_ops+0xf0/0xf0
[ 989.817263] new_sync_write+0x333/0x750
[ 989.822365] ? __lock_acquire+0x508/0xc10
[ 989.827660] ? new_sync_read+0x740/0x740
[ 989.832861] ? lock_acquire.part.0+0xbc/0x260
[ 989.838547] ? ksys_write+0x116/0x250
[ 989.843456] vfs_write+0x657/0x9b0
[ 989.848072] ksys_write+0x116/0x250
[ 989.852784] ? __ia32_sys_read+0xc0/0xc0
[ 989.857973] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.863562] ? rcu_is_watching+0x15/0xd0
[ 989.868760] do_syscall_64+0xf3/0x650
[ 989.873667] ? trace_hardirqs_off+0xd/0x30
[ 989.879062] ? exc_page_fault+0xda/0xf0
[ 989.884165] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.890638] RIP: 0033:0x7f9b5089922e
[ 989.895447] RSP: 002b:00007fff0a8f3840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.904746] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9b5089922e
[ 989.913551] RDX: 0000000000000001 RSI: 000055d28973fcd0 RDI: 0000000000000007
[ 989.922356] RBP: 00007fff0a8f3850 R08: 0000000000000000 R09: 0000000000000000
[ 989.931163] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9b50711390
[ 989.939961] R13: 000055d28973fcd0 R14: 0000000000000007 R15: 0000000000000003
[ 989.948773]
[ 989.952053] INFO: task python3:5850 is blocked on a mutex likely owned by task python3:3839.
[ 989.962324] task:python3 state:D stack:25672 pid:3839 tgid:3839 ppid:3838 task_flags:0x400100 flags:0x00080802
[ 989.975525] Call Trace:
[ 989.979063]
[ 989.982213] __schedule+0xcc3/0x19c0
[ 989.987023] ? io_schedule_timeout+0x140/0x140
[ 989.992805] ? find_held_lock+0x2b/0x80
[ 989.997906] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.004074] schedule+0xd4/0x260
[ 990.008492] schedule_timeout+0x13f/0x290
[ 990.013788] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.020154] ? trace_contention_end+0xb3/0x1a0
[ 990.025937] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.031624] ? ww_mutex_lock+0x160/0x160
[ 990.036825] msleep+0x1b/0x30
[ 990.040947] napi_set_threaded+0x3a5/0x440
[ 990.046346] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.051839] netif_set_threaded+0x257/0x400
[ 990.057329] threaded_store+0x1b8/0x220
[ 990.062432] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.067717] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.073010] ? sysfs_kf_write+0x65/0x160
[ 990.078206] ? sysfs_file_ops+0x140/0x140
[ 990.083500] kernfs_fop_write_iter+0x335/0x510
[ 990.089285] ? kernfs_ops+0xf0/0xf0
[ 990.093995] new_sync_write+0x333/0x750
[ 990.099096] ? __lock_acquire+0x508/0xc10
[ 990.104393] ? new_sync_read+0x740/0x740
[ 990.109593] ? lock_acquire.part.0+0xbc/0x260
[ 990.115270] ? ksys_write+0x116/0x250
[ 990.120178] vfs_write+0x657/0x9b0
[ 990.124794] ksys_write+0x116/0x250
[ 990.129505] ? __ia32_sys_read+0xc0/0xc0
[ 990.134703] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.140289] ? rcu_is_watching+0x15/0xd0
[ 990.145489] do_syscall_64+0xf3/0x650
[ 990.150384] ? trace_hardirqs_off+0xd/0x30
[ 990.155779] ? exc_page_fault+0xda/0xf0
[ 990.160881] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.167349] RIP: 0033:0x7f87f1a1622e
[ 990.172158] RSP: 002b:00007ffceb00eef0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.181452] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f87f1a1622e
[ 990.190260] RDX: 0000000000000001 RSI: 0000562c45354cd0 RDI: 0000000000000007
[ 990.199060] RBP: 00007ffceb00ef00 R08: 0000000000000000 R09: 0000000000000000
[ 990.207868] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f87f188e390
[ 990.216668] R13: 0000562c45354cd0 R14: 0000000000000007 R15: 0000000000000003
[ 990.225479]
[ 990.228738] \x0aShowing all locks held in the system:
[ 990.237279] 1 lock held by khungtaskd/185:
[ 990.242670] #0: ffffffffbb2fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.254140] 3 locks held by systemd-journal/799:
[ 990.260118] #0: ff1100180e0372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.270008] #1: ffffffffbb5033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 990.282237] #2: ff110001a2d66a70 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.293796] 4 locks held by python3/3839:
[ 990.299089] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.308977] #1: ff1100018af4ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.319936] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.331091] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.341188] 4 locks held by python3/5850:
[ 990.346481] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.356370] #1: ff11000128de6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.367330] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.378484] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.388572]
[ 990.391035] =============================================\x0a
[ 1112.550791] INFO: task python3:5850 blocked for more than 245 seconds.
[ 1112.558937] Not tainted 7.1.0-rc3-rnfr-g25a6e3c1d2bf #1
[ 1112.566179] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.575772] task:python3 state:D stack:25672 pid:5850 tgid:5850 ppid:5849 task_flags:0x400100 flags:0x00080000
[ 1112.588977] Call Trace:
[ 1112.592517]
[ 1112.595666] __schedule+0xcc3/0x19c0
[ 1112.600485] ? io_schedule_timeout+0x140/0x140
[ 1112.606271] ? find_held_lock+0x2b/0x80
[ 1112.611376] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.617262] schedule+0xd4/0x260
[ 1112.621680] schedule_preempt_disabled+0x18/0x30
[ 1112.627659] __mutex_lock+0x6a3/0x2000
[ 1112.632663] ? threaded_store+0x132/0x220
[ 1112.637965] ? ww_mutex_lock+0x160/0x160
[ 1112.643165] ? selinux_inode_free_security+0x330/0x330
[ 1112.649736] ? cap_capable+0x1d6/0x3e0
[ 1112.654744] ? threaded_store+0x132/0x220
[ 1112.660041] threaded_store+0x132/0x220
[ 1112.665143] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.670444] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.675745] ? sysfs_kf_write+0x65/0x160
[ 1112.680948] ? sysfs_file_ops+0x140/0x140
[ 1112.686246] kernfs_fop_write_iter+0x335/0x510
[ 1112.692023] ? kernfs_ops+0xf0/0xf0
[ 1112.696735] new_sync_write+0x333/0x750
[ 1112.701839] ? __lock_acquire+0x508/0xc10
[ 1112.707127] ? new_sync_read+0x740/0x740
[ 1112.712329] ? lock_acquire.part.0+0xbc/0x260
[ 1112.718016] ? ksys_write+0x116/0x250
[ 1112.722918] vfs_write+0x657/0x9b0
[ 1112.727535] ksys_write+0x116/0x250
[ 1112.732249] ? __ia32_sys_read+0xc0/0xc0
[ 1112.737449] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.743041] ? rcu_is_watching+0x15/0xd0
[ 1112.748242] do_syscall_64+0xf3/0x650
[ 1112.753147] ? trace_hardirqs_off+0xd/0x30
[ 1112.758544] ? exc_page_fault+0xda/0xf0
[ 1112.763642] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.770109] RIP: 0033:0x7f9b5089922e
[ 1112.774910] RSP: 002b:00007fff0a8f3840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.784210] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9b5089922e
[ 1112.793021] RDX: 0000000000000001 RSI: 000055d28973fcd0 RDI: 0000000000000007
[ 1112.801827] RBP: 00007fff0a8f3850 R08: 0000000000000000 R09: 0000000000000000
[ 1112.810641] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9b50711390
[ 1112.819451] R13: 000055d28973fcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.828267]
[ 1112.831552] INFO: task python3:5850 is blocked on a mutex likely owned by task python3:3839.
[ 1112.841827] task:python3 state:D stack:25672 pid:3839 tgid:3839 ppid:3838 task_flags:0x400100 flags:0x00080802
[ 1112.855031] Call Trace:
[ 1112.858570]
[ 1112.861720] __schedule+0xcc3/0x19c0
[ 1112.866533] ? io_schedule_timeout+0x140/0x140
[ 1112.872319] ? find_held_lock+0x2b/0x80
[ 1112.877425] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.883602] schedule+0xd4/0x260
[ 1112.888021] schedule_timeout+0x13f/0x290
[ 1112.893318] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.899688] ? trace_contention_end+0xb3/0x1a0
[ 1112.905475] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.911166] ? ww_mutex_lock+0x160/0x160
[ 1112.916366] msleep+0x1b/0x30
[ 1112.920483] napi_set_threaded+0x3a5/0x440
[ 1112.925880] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.931375] netif_set_threaded+0x257/0x400
[ 1112.936869] threaded_store+0x1b8/0x220
[ 1112.941974] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.947272] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.952572] ? sysfs_kf_write+0x65/0x160
[ 1112.957773] ? sysfs_file_ops+0x140/0x140
[ 1112.963069] kernfs_fop_write_iter+0x335/0x510
[ 1112.968856] ? kernfs_ops+0xf0/0xf0
[ 1112.973567] new_sync_write+0x333/0x750
[ 1112.978670] ? __lock_acquire+0x508/0xc10
[ 1112.983965] ? new_sync_read+0x740/0x740
[ 1112.989167] ? lock_acquire.part.0+0xbc/0x260
[ 1112.994852] ? ksys_write+0x116/0x250
[ 1112.999762] vfs_write+0x657/0x9b0
[ 1113.004379] ksys_write+0x116/0x250
[ 1113.009093] ? __ia32_sys_read+0xc0/0xc0
[ 1113.014291] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.019879] ? rcu_is_watching+0x15/0xd0
[ 1113.025081] do_syscall_64+0xf3/0x650
[ 1113.029985] ? trace_hardirqs_off+0xd/0x30
[ 1113.035379] ? exc_page_fault+0xda/0xf0
[ 1113.040482] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.046940] RIP: 0033:0x7f87f1a1622e
[ 1113.051749] RSP: 002b:00007ffceb00eef0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.061038] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f87f1a1622e
[ 1113.069848] RDX: 0000000000000001 RSI: 0000562c45354cd0 RDI: 0000000000000007
[ 1113.078657] RBP: 00007ffceb00ef00 R08: 0000000000000000 R09: 0000000000000000
[ 1113.087467] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f87f188e390
[ 1113.096278] R13: 0000562c45354cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.105093]
[ 1113.108357] \x0aShowing all locks held in the system:
[ 1113.116897] 1 lock held by khungtaskd/185:
[ 1113.122289] #0: ffffffffbb2fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.133777] 4 locks held by python3/3839:
[ 1113.139074] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.148966] #1: ff1100018af4ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.159929] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.171087] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.181183] 4 locks held by python3/5850:
[ 1113.186480] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.196370] #1: ff11000128de6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.207332] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.218492] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.228582]
[ 1113.231040] =============================================\x0a
[ 1235.433105] INFO: task python3:5850 blocked for more than 368 seconds.
[ 1235.441258] Not tainted 7.1.0-rc3-rnfr-g25a6e3c1d2bf #1
[ 1235.448510] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.458104] task:python3 state:D stack:25672 pid:5850 tgid:5850 ppid:5849 task_flags:0x400100 flags:0x00080002
[ 1235.471310] Call Trace:
[ 1235.474850]
[ 1235.478000] __schedule+0xcc3/0x19c0
[ 1235.482819] ? io_schedule_timeout+0x140/0x140
[ 1235.488607] ? find_held_lock+0x2b/0x80
[ 1235.493713] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.499601] schedule+0xd4/0x260
[ 1235.504022] schedule_preempt_disabled+0x18/0x30
[ 1235.510002] __mutex_lock+0x6a3/0x2000
[ 1235.514999] ? threaded_store+0x132/0x220
[ 1235.520302] ? ww_mutex_lock+0x160/0x160
[ 1235.525502] ? selinux_inode_free_security+0x330/0x330
[ 1235.532074] ? cap_capable+0x1d6/0x3e0
[ 1235.537082] ? threaded_store+0x132/0x220
[ 1235.542381] threaded_store+0x132/0x220
[ 1235.547487] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.552785] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.558085] ? sysfs_kf_write+0x65/0x160
[ 1235.563285] ? sysfs_file_ops+0x140/0x140
[ 1235.568582] kernfs_fop_write_iter+0x335/0x510
[ 1235.574367] ? kernfs_ops+0xf0/0xf0
[ 1235.579077] new_sync_write+0x333/0x750
[ 1235.584181] ? __lock_acquire+0x508/0xc10
[ 1235.589477] ? new_sync_read+0x740/0x740
[ 1235.594680] ? lock_acquire.part.0+0xbc/0x260
[ 1235.600365] ? ksys_write+0x116/0x250
[ 1235.605277] vfs_write+0x657/0x9b0
[ 1235.609893] ksys_write+0x116/0x250
[ 1235.614606] ? __ia32_sys_read+0xc0/0xc0
[ 1235.619807] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.625396] ? rcu_is_watching+0x15/0xd0
[ 1235.630601] do_syscall_64+0xf3/0x650
[ 1235.635509] ? trace_hardirqs_off+0xd/0x30
[ 1235.640904] ? exc_page_fault+0xda/0xf0
[ 1235.646006] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.652467] RIP: 0033:0x7f9b5089922e
[ 1235.657276] RSP: 002b:00007fff0a8f3840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.666577] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9b5089922e
[ 1235.675383] RDX: 0000000000000001 RSI: 000055d28973fcd0 RDI: 0000000000000007
[ 1235.684190] RBP: 00007fff0a8f3850 R08: 0000000000000000 R09: 0000000000000000
[ 1235.692999] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9b50711390
[ 1235.701808] R13: 000055d28973fcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.710623]
[ 1235.713931] INFO: task python3:5850 is blocked on a mutex likely owned by task python3:3839.
[ 1235.724210] task:python3 state:D stack:25672 pid:3839 tgid:3839 ppid:3838 task_flags:0x400100 flags:0x00080802
[ 1235.737412] Call Trace:
[ 1235.740949]
[ 1235.744098] __schedule+0xcc3/0x19c0
[ 1235.748911] ? io_schedule_timeout+0x140/0x140
[ 1235.754695] ? find_held_lock+0x2b/0x80
[ 1235.759801] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.765980] schedule+0xd4/0x260
[ 1235.770401] schedule_timeout+0x13f/0x290
[ 1235.775697] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.782069] ? trace_contention_end+0xb3/0x1a0
[ 1235.787854] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.793545] ? ww_mutex_lock+0x160/0x160
[ 1235.798744] msleep+0x1b/0x30
[ 1235.802870] napi_set_threaded+0x3a5/0x440
[ 1235.808268] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.813765] netif_set_threaded+0x257/0x400
[ 1235.819260] threaded_store+0x1b8/0x220
[ 1235.824364] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.829660] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.834957] ? sysfs_kf_write+0x65/0x160
[ 1235.840157] ? sysfs_file_ops+0x140/0x140
[ 1235.845454] kernfs_fop_write_iter+0x335/0x510
[ 1235.851238] ? kernfs_ops+0xf0/0xf0
[ 1235.855950] new_sync_write+0x333/0x750
[ 1235.861044] ? __lock_acquire+0x508/0xc10
[ 1235.866334] ? new_sync_read+0x740/0x740
[ 1235.871535] ? lock_acquire.part.0+0xbc/0x260
[ 1235.877221] ? ksys_write+0x116/0x250
[ 1235.882130] vfs_write+0x657/0x9b0
[ 1235.886746] ksys_write+0x116/0x250
[ 1235.891459] ? __ia32_sys_read+0xc0/0xc0
[ 1235.896658] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.902249] ? rcu_is_watching+0x15/0xd0
[ 1235.907452] do_syscall_64+0xf3/0x650
[ 1235.912358] ? trace_hardirqs_off+0xd/0x30
[ 1235.917752] ? exc_page_fault+0xda/0xf0
[ 1235.922856] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.929323] RIP: 0033:0x7f87f1a1622e
[ 1235.934123] RSP: 002b:00007ffceb00eef0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.943422] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f87f1a1622e
[ 1235.952229] RDX: 0000000000000001 RSI: 0000562c45354cd0 RDI: 0000000000000007
[ 1235.961039] RBP: 00007ffceb00ef00 R08: 0000000000000000 R09: 0000000000000000
[ 1235.969847] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f87f188e390
[ 1235.978655] R13: 0000562c45354cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.987468]
[ 1235.990734] \x0aShowing all locks held in the system:
[ 1235.999268] 1 lock held by khungtaskd/185:
[ 1236.004664] #0: ffffffffbb2fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.016132] 2 locks held by systemd-journal/799:
[ 1236.022112] #0: ff1100180ddb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.032001] #1: ffffffffbb5033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.044830] 4 locks held by python3/3839:
[ 1236.050126] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.060017] #1: ff1100018af4ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.070981] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.082138] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.092238] 4 locks held by python3/5850:
[ 1236.097526] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.107414] #1: ff11000128de6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.118378] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.129536] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.139629]
[ 1236.142097] =============================================\x0a
[ 1358.314395] INFO: task python3:5850 blocked for more than 491 seconds.
[ 1358.322544] Not tainted 7.1.0-rc3-rnfr-g25a6e3c1d2bf #1
[ 1358.329794] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.339385] task:python3 state:D stack:25672 pid:5850 tgid:5850 ppid:5849 task_flags:0x400100 flags:0x00080002
[ 1358.352591] Call Trace:
[ 1358.356130]
[ 1358.359284] __schedule+0xcc3/0x19c0
[ 1358.364102] ? io_schedule_timeout+0x140/0x140
[ 1358.369889] ? find_held_lock+0x2b/0x80
[ 1358.374997] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.380886] schedule+0xd4/0x260
[ 1358.385319] schedule_preempt_disabled+0x18/0x30
[ 1358.391303] __mutex_lock+0x6a3/0x2000
[ 1358.396310] ? threaded_store+0x132/0x220
[ 1358.401612] ? ww_mutex_lock+0x160/0x160
[ 1358.406810] ? selinux_inode_free_security+0x330/0x330
[ 1358.413383] ? cap_capable+0x1d6/0x3e0
[ 1358.418393] ? threaded_store+0x132/0x220
[ 1358.423689] threaded_store+0x132/0x220
[ 1358.428793] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.434089] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.439380] ? sysfs_kf_write+0x65/0x160
[ 1358.444580] ? sysfs_file_ops+0x140/0x140
[ 1358.449881] kernfs_fop_write_iter+0x335/0x510
[ 1358.455674] ? kernfs_ops+0xf0/0xf0
[ 1358.460387] new_sync_write+0x333/0x750
[ 1358.465491] ? __lock_acquire+0x508/0xc10
[ 1358.470787] ? new_sync_read+0x740/0x740
[ 1358.475988] ? lock_acquire.part.0+0xbc/0x260
[ 1358.481673] ? ksys_write+0x116/0x250
[ 1358.486583] vfs_write+0x657/0x9b0
[ 1358.491199] ksys_write+0x116/0x250
[ 1358.495911] ? __ia32_sys_read+0xc0/0xc0
[ 1358.501103] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.506691] ? rcu_is_watching+0x15/0xd0
[ 1358.511883] do_syscall_64+0xf3/0x650
[ 1358.516782] ? trace_hardirqs_off+0xd/0x30
[ 1358.522177] ? exc_page_fault+0xda/0xf0
[ 1358.527280] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.533750] RIP: 0033:0x7f9b5089922e
[ 1358.538559] RSP: 002b:00007fff0a8f3840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.547858] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9b5089922e
[ 1358.556666] RDX: 0000000000000001 RSI: 000055d28973fcd0 RDI: 0000000000000007
[ 1358.565468] RBP: 00007fff0a8f3850 R08: 0000000000000000 R09: 0000000000000000
[ 1358.574277] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9b50711390
[ 1358.583085] R13: 000055d28973fcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.591898]
[ 1358.595187] INFO: task python3:5850 is blocked on a mutex likely owned by task python3:3839.
[ 1358.605457] task:python3 state:D stack:25672 pid:3839 tgid:3839 ppid:3838 task_flags:0x400100 flags:0x00080802
[ 1358.618661] Call Trace:
[ 1358.622198]
[ 1358.625351] __schedule+0xcc3/0x19c0
[ 1358.630163] ? io_schedule_timeout+0x140/0x140
[ 1358.635948] ? find_held_lock+0x2b/0x80
[ 1358.641042] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.647219] schedule+0xd4/0x260
[ 1358.651637] schedule_timeout+0x13f/0x290
[ 1358.656937] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.663304] ? trace_contention_end+0xb3/0x1a0
[ 1358.669090] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.674782] ? ww_mutex_lock+0x160/0x160
[ 1358.679982] msleep+0x1b/0x30
[ 1358.684109] napi_set_threaded+0x3a5/0x440
[ 1358.689497] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.694981] netif_set_threaded+0x257/0x400
[ 1358.700475] threaded_store+0x1b8/0x220
[ 1358.705579] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.710877] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.716167] ? sysfs_kf_write+0x65/0x160
[ 1358.721368] ? sysfs_file_ops+0x140/0x140
[ 1358.726666] kernfs_fop_write_iter+0x335/0x510
[ 1358.732452] ? kernfs_ops+0xf0/0xf0
[ 1358.737164] new_sync_write+0x333/0x750
[ 1358.742266] ? __lock_acquire+0x508/0xc10
[ 1358.747562] ? new_sync_read+0x740/0x740
[ 1358.752762] ? lock_acquire.part.0+0xbc/0x260
[ 1358.758438] ? ksys_write+0x116/0x250
[ 1358.763348] vfs_write+0x657/0x9b0
[ 1358.767966] ksys_write+0x116/0x250
[ 1358.772679] ? __ia32_sys_read+0xc0/0xc0
[ 1358.777876] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.783466] ? rcu_is_watching+0x15/0xd0
[ 1358.788665] do_syscall_64+0xf3/0x650
[ 1358.793573] ? trace_hardirqs_off+0xd/0x30
[ 1358.798967] ? exc_page_fault+0xda/0xf0
[ 1358.804070] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.810530] RIP: 0033:0x7f87f1a1622e
[ 1358.815340] RSP: 002b:00007ffceb00eef0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.824630] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f87f1a1622e
[ 1358.833439] RDX: 0000000000000001 RSI: 0000562c45354cd0 RDI: 0000000000000007
[ 1358.842248] RBP: 00007ffceb00ef00 R08: 0000000000000000 R09: 0000000000000000
[ 1358.851058] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f87f188e390
[ 1358.859864] R13: 0000562c45354cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.868679]
[ 1358.871951] \x0aShowing all locks held in the system:
[ 1358.880492] 1 lock held by khungtaskd/185:
[ 1358.885884] #0: ffffffffbb2fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.897367] 4 locks held by python3/3839:
[ 1358.902662] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.912552] #1: ff1100018af4ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.923512] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.934670] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.944764] 4 locks held by python3/5850:
[ 1358.950061] #0: ff1100012fccc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.959948] #1: ff11000128de6880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.970912] #2: ff110001235b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.982071] #3: ff1100011c5b1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.992164]
[ 1358.994631] =============================================\x0a