[ 785.766908] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 785.769867] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 785.900195] kselftest: Running tests in drivers/net
[ 989.668311] INFO: task python3:5736 blocked for more than 122 seconds.
[ 989.676544] Not tainted 7.1.0-rc2-rhfv-ge446480fb140 #1
[ 989.683794] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.694008] task:python3 state:D stack:25672 pid:5736 tgid:5736 ppid:5735 task_flags:0x400100 flags:0x00080000
[ 989.707217] Call Trace:
[ 989.710758]
[ 989.713909] __schedule+0xcba/0x1940
[ 989.718727] ? io_schedule_timeout+0x140/0x140
[ 989.724513] ? find_held_lock+0x2b/0x80
[ 989.729617] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.735505] schedule+0xd4/0x260
[ 989.739922] schedule_preempt_disabled+0x18/0x30
[ 989.745904] __mutex_lock+0x6a3/0x2000
[ 989.750911] ? threaded_store+0x132/0x220
[ 989.756210] ? ww_mutex_lock+0x160/0x160
[ 989.761403] ? selinux_inode_free_security+0x330/0x330
[ 989.767975] ? cap_capable+0x1d6/0x3e0
[ 989.772985] ? threaded_store+0x132/0x220
[ 989.778283] threaded_store+0x132/0x220
[ 989.783379] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.788680] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.793977] ? sysfs_kf_write+0x65/0x160
[ 989.799179] ? sysfs_file_ops+0x140/0x140
[ 989.804474] kernfs_fop_write_iter+0x335/0x510
[ 989.810259] ? kernfs_ops+0xf0/0xf0
[ 989.814971] new_sync_write+0x333/0x750
[ 989.820076] ? __lock_acquire+0x508/0xc10
[ 989.825373] ? new_sync_read+0x740/0x740
[ 989.830575] ? lock_acquire.part.0+0xbc/0x260
[ 989.836264] ? ksys_write+0x116/0x250
[ 989.841173] vfs_write+0x657/0x9b0
[ 989.845788] ksys_write+0x116/0x250
[ 989.850492] ? __ia32_sys_read+0xc0/0xc0
[ 989.855692] ? rcu_is_watching+0x15/0xd0
[ 989.860890] ? rcu_is_watching+0x15/0xd0
[ 989.866091] do_syscall_64+0xf3/0x650
[ 989.870998] ? irq_exit_rcu+0x1a/0x30
[ 989.875898] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.882365] RIP: 0033:0x7fd654dd622e
[ 989.887166] RSP: 002b:00007ffd8bf43990 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.896469] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd654dd622e
[ 989.905276] RDX: 0000000000000001 RSI: 0000564a90f1afe0 RDI: 0000000000000007
[ 989.914082] RBP: 00007ffd8bf439a0 R08: 0000000000000000 R09: 0000000000000000
[ 989.922892] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd654c4e390
[ 989.931703] R13: 0000564a90f1afe0 R14: 0000000000000007 R15: 0000000000000003
[ 989.940520]
[ 989.943819] INFO: task python3:5736 is blocked on a mutex likely owned by task python3:3674.
[ 989.954089] task:python3 state:D stack:25672 pid:3674 tgid:3674 ppid:3673 task_flags:0x400100 flags:0x00080802
[ 989.967291] Call Trace:
[ 989.970832]
[ 989.973982] __schedule+0xcba/0x1940
[ 989.978794] ? io_schedule_timeout+0x140/0x140
[ 989.984580] ? find_held_lock+0x2b/0x80
[ 989.989684] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.995863] schedule+0xd4/0x260
[ 990.000283] schedule_timeout+0x13f/0x290
[ 990.005581] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.011943] ? trace_contention_end+0xb3/0x1a0
[ 990.017729] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.023417] ? ww_mutex_lock+0x160/0x160
[ 990.028618] msleep+0x1b/0x30
[ 990.032745] napi_set_threaded+0x3a5/0x440
[ 990.038132] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.043627] netif_set_threaded+0x257/0x400
[ 990.049122] threaded_store+0x1b8/0x220
[ 990.054223] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.059521] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.064816] ? sysfs_kf_write+0x65/0x160
[ 990.070018] ? sysfs_file_ops+0x140/0x140
[ 990.075314] kernfs_fop_write_iter+0x335/0x510
[ 990.081098] ? kernfs_ops+0xf0/0xf0
[ 990.085811] new_sync_write+0x333/0x750
[ 990.090915] ? __lock_acquire+0x508/0xc10
[ 990.096215] ? new_sync_read+0x740/0x740
[ 990.101417] ? lock_acquire.part.0+0xbc/0x260
[ 990.107104] ? ksys_write+0x116/0x250
[ 990.112013] vfs_write+0x657/0x9b0
[ 990.116630] ksys_write+0x116/0x250
[ 990.121342] ? __ia32_sys_read+0xc0/0xc0
[ 990.126542] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.132133] ? rcu_is_watching+0x15/0xd0
[ 990.137334] do_syscall_64+0xf3/0x650
[ 990.142240] ? irq_exit_rcu+0x1a/0x30
[ 990.147148] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.153607] RIP: 0033:0x7f9941c7222e
[ 990.158415] RSP: 002b:00007ffd63ae0ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.167712] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9941c7222e
[ 990.176522] RDX: 0000000000000001 RSI: 0000556bb060bfe0 RDI: 0000000000000007
[ 990.185332] RBP: 00007ffd63ae0ae0 R08: 0000000000000000 R09: 0000000000000000
[ 990.194148] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9941aea390
[ 990.202949] R13: 0000556bb060bfe0 R14: 0000000000000007 R15: 0000000000000003
[ 990.211763]
[ 990.215025] \x0aShowing all locks held in the system:
[ 990.223565] 1 lock held by khungtaskd/185:
[ 990.228957] #0: ffffffff8c0fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.240441] 4 locks held by python3/3674:
[ 990.245737] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.255630] #1: ff11000168795680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.266591] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.277741] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.287842] 4 locks held by python3/5736:
[ 990.293142] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.303030] #1: ff110001e85abe80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.314000] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.325169] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.335262] 5 locks held by systemd/6375:
[ 990.340556] #0: ff1100013050c3e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.350444] #1: ff110001c5fd2080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.361408] #2: ffffffff8c169ce0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_lock_and_drain_offline+0xa6/0x640
[ 990.373052] #3: ffffffff8bf68970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2a3/0x5c0
[ 990.384599] #4: ffffffff8c169a50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2af/0x5c0
[ 990.396929]
[ 990.399395] =============================================\x0a
[ 1112.549452] INFO: task python3:5736 blocked for more than 245 seconds.
[ 1112.557601] Not tainted 7.1.0-rc2-rhfv-ge446480fb140 #1
[ 1112.564852] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.574443] task:python3 state:D stack:25672 pid:5736 tgid:5736 ppid:5735 task_flags:0x400100 flags:0x00080000
[ 1112.587651] Call Trace:
[ 1112.591193]
[ 1112.594346] __schedule+0xcba/0x1940
[ 1112.599166] ? io_schedule_timeout+0x140/0x140
[ 1112.604952] ? find_held_lock+0x2b/0x80
[ 1112.610058] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.615944] schedule+0xd4/0x260
[ 1112.620364] schedule_preempt_disabled+0x18/0x30
[ 1112.626346] __mutex_lock+0x6a3/0x2000
[ 1112.631350] ? threaded_store+0x132/0x220
[ 1112.636644] ? ww_mutex_lock+0x160/0x160
[ 1112.641844] ? selinux_inode_free_security+0x330/0x330
[ 1112.648413] ? cap_capable+0x1d6/0x3e0
[ 1112.653422] ? threaded_store+0x132/0x220
[ 1112.658718] threaded_store+0x132/0x220
[ 1112.663819] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.669119] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.674421] ? sysfs_kf_write+0x65/0x160
[ 1112.679621] ? sysfs_file_ops+0x140/0x140
[ 1112.684917] kernfs_fop_write_iter+0x335/0x510
[ 1112.690704] ? kernfs_ops+0xf0/0xf0
[ 1112.695414] new_sync_write+0x333/0x750
[ 1112.700518] ? __lock_acquire+0x508/0xc10
[ 1112.705809] ? new_sync_read+0x740/0x740
[ 1112.711011] ? lock_acquire.part.0+0xbc/0x260
[ 1112.716696] ? ksys_write+0x116/0x250
[ 1112.721605] vfs_write+0x657/0x9b0
[ 1112.726222] ksys_write+0x116/0x250
[ 1112.730934] ? __ia32_sys_read+0xc0/0xc0
[ 1112.736136] ? rcu_is_watching+0x15/0xd0
[ 1112.741337] ? rcu_is_watching+0x15/0xd0
[ 1112.746538] do_syscall_64+0xf3/0x650
[ 1112.751446] ? irq_exit_rcu+0x1a/0x30
[ 1112.756354] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.762824] RIP: 0033:0x7fd654dd622e
[ 1112.767634] RSP: 002b:00007ffd8bf43990 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.776933] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd654dd622e
[ 1112.785742] RDX: 0000000000000001 RSI: 0000564a90f1afe0 RDI: 0000000000000007
[ 1112.794550] RBP: 00007ffd8bf439a0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.803350] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd654c4e390
[ 1112.812161] R13: 0000564a90f1afe0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.820976]
[ 1112.824241] INFO: task python3:5736 is blocked on a mutex likely owned by task python3:3674.
[ 1112.834516] task:python3 state:D stack:25672 pid:3674 tgid:3674 ppid:3673 task_flags:0x400100 flags:0x00080802
[ 1112.847719] Call Trace:
[ 1112.851258]
[ 1112.854408] __schedule+0xcba/0x1940
[ 1112.859220] ? io_schedule_timeout+0x140/0x140
[ 1112.865002] ? find_held_lock+0x2b/0x80
[ 1112.870106] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.876290] schedule+0xd4/0x260
[ 1112.880707] schedule_timeout+0x13f/0x290
[ 1112.886008] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.892377] ? trace_contention_end+0xb3/0x1a0
[ 1112.898163] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.903853] ? ww_mutex_lock+0x160/0x160
[ 1112.909052] msleep+0x1b/0x30
[ 1112.913173] napi_set_threaded+0x3a5/0x440
[ 1112.918569] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.924064] netif_set_threaded+0x257/0x400
[ 1112.929557] threaded_store+0x1b8/0x220
[ 1112.934660] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.939958] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.945248] ? sysfs_kf_write+0x65/0x160
[ 1112.950447] ? sysfs_file_ops+0x140/0x140
[ 1112.955736] kernfs_fop_write_iter+0x335/0x510
[ 1112.961523] ? kernfs_ops+0xf0/0xf0
[ 1112.966233] new_sync_write+0x333/0x750
[ 1112.971334] ? __lock_acquire+0x508/0xc10
[ 1112.976635] ? new_sync_read+0x740/0x740
[ 1112.981836] ? lock_acquire.part.0+0xbc/0x260
[ 1112.987522] ? ksys_write+0x116/0x250
[ 1112.992435] vfs_write+0x657/0x9b0
[ 1112.997052] ksys_write+0x116/0x250
[ 1113.001763] ? __ia32_sys_read+0xc0/0xc0
[ 1113.006964] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.012553] ? rcu_is_watching+0x15/0xd0
[ 1113.017755] do_syscall_64+0xf3/0x650
[ 1113.022663] ? irq_exit_rcu+0x1a/0x30
[ 1113.027570] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.034039] RIP: 0033:0x7f9941c7222e
[ 1113.038846] RSP: 002b:00007ffd63ae0ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.048142] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9941c7222e
[ 1113.056943] RDX: 0000000000000001 RSI: 0000556bb060bfe0 RDI: 0000000000000007
[ 1113.065769] RBP: 00007ffd63ae0ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.074600] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9941aea390
[ 1113.083411] R13: 0000556bb060bfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.092225]
[ 1113.095487] \x0aShowing all locks held in the system:
[ 1113.104026] 1 lock held by khungtaskd/185:
[ 1113.109419] #0: ffffffff8c0fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.120904] 4 locks held by python3/3674:
[ 1113.126192] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.136074] #1: ff11000168795680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.147029] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.158188] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.168288] 4 locks held by python3/5736:
[ 1113.173580] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.183468] #1: ff110001e85abe80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.194432] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.205591] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.215684]
[ 1113.218151] =============================================\x0a
[ 1235.429743] INFO: task python3:5736 blocked for more than 368 seconds.
[ 1235.437888] Not tainted 7.1.0-rc2-rhfv-ge446480fb140 #1
[ 1235.445132] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.454724] task:python3 state:D stack:25672 pid:5736 tgid:5736 ppid:5735 task_flags:0x400100 flags:0x00080002
[ 1235.467932] Call Trace:
[ 1235.471471]
[ 1235.474626] __schedule+0xcba/0x1940
[ 1235.479444] ? io_schedule_timeout+0x140/0x140
[ 1235.485231] ? find_held_lock+0x2b/0x80
[ 1235.490338] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.496226] schedule+0xd4/0x260
[ 1235.500647] schedule_preempt_disabled+0x18/0x30
[ 1235.506625] __mutex_lock+0x6a3/0x2000
[ 1235.511633] ? threaded_store+0x132/0x220
[ 1235.516934] ? ww_mutex_lock+0x160/0x160
[ 1235.522133] ? selinux_inode_free_security+0x330/0x330
[ 1235.528702] ? cap_capable+0x1d6/0x3e0
[ 1235.533714] ? threaded_store+0x132/0x220
[ 1235.539012] threaded_store+0x132/0x220
[ 1235.544113] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.549411] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.554707] ? sysfs_kf_write+0x65/0x160
[ 1235.559908] ? sysfs_file_ops+0x140/0x140
[ 1235.565194] kernfs_fop_write_iter+0x335/0x510
[ 1235.570981] ? kernfs_ops+0xf0/0xf0
[ 1235.575692] new_sync_write+0x333/0x750
[ 1235.580787] ? __lock_acquire+0x508/0xc10
[ 1235.586073] ? new_sync_read+0x740/0x740
[ 1235.591275] ? lock_acquire.part.0+0xbc/0x260
[ 1235.596963] ? ksys_write+0x116/0x250
[ 1235.601872] vfs_write+0x657/0x9b0
[ 1235.606489] ksys_write+0x116/0x250
[ 1235.611200] ? __ia32_sys_read+0xc0/0xc0
[ 1235.616403] ? rcu_is_watching+0x15/0xd0
[ 1235.621603] ? rcu_is_watching+0x15/0xd0
[ 1235.626802] do_syscall_64+0xf3/0x650
[ 1235.631702] ? irq_exit_rcu+0x1a/0x30
[ 1235.636612] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.643079] RIP: 0033:0x7fd654dd622e
[ 1235.647955] RSP: 002b:00007ffd8bf43990 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.657320] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd654dd622e
[ 1235.666128] RDX: 0000000000000001 RSI: 0000564a90f1afe0 RDI: 0000000000000007
[ 1235.674937] RBP: 00007ffd8bf439a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.683743] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd654c4e390
[ 1235.692550] R13: 0000564a90f1afe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.701363]
[ 1235.704670] INFO: task python3:5736 is blocked on a mutex likely owned by task python3:3674.
[ 1235.714943] task:python3 state:D stack:25672 pid:3674 tgid:3674 ppid:3673 task_flags:0x400100 flags:0x00080802
[ 1235.728147] Call Trace:
[ 1235.731688]
[ 1235.734837] __schedule+0xcba/0x1940
[ 1235.739650] ? io_schedule_timeout+0x140/0x140
[ 1235.745435] ? find_held_lock+0x2b/0x80
[ 1235.750530] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.756708] schedule+0xd4/0x260
[ 1235.761126] schedule_timeout+0x13f/0x290
[ 1235.766422] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.772793] ? trace_contention_end+0xb3/0x1a0
[ 1235.778577] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.784266] ? ww_mutex_lock+0x160/0x160
[ 1235.789466] msleep+0x1b/0x30
[ 1235.793594] napi_set_threaded+0x3a5/0x440
[ 1235.798990] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.804483] netif_set_threaded+0x257/0x400
[ 1235.809978] threaded_store+0x1b8/0x220
[ 1235.815082] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.820378] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.825673] ? sysfs_kf_write+0x65/0x160
[ 1235.830875] ? sysfs_file_ops+0x140/0x140
[ 1235.836171] kernfs_fop_write_iter+0x335/0x510
[ 1235.841955] ? kernfs_ops+0xf0/0xf0
[ 1235.846666] new_sync_write+0x333/0x750
[ 1235.851769] ? __lock_acquire+0x508/0xc10
[ 1235.857067] ? new_sync_read+0x740/0x740
[ 1235.862270] ? lock_acquire.part.0+0xbc/0x260
[ 1235.867956] ? ksys_write+0x116/0x250
[ 1235.872865] vfs_write+0x657/0x9b0
[ 1235.877480] ksys_write+0x116/0x250
[ 1235.882192] ? __ia32_sys_read+0xc0/0xc0
[ 1235.887394] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.892983] ? rcu_is_watching+0x15/0xd0
[ 1235.898182] do_syscall_64+0xf3/0x650
[ 1235.903090] ? irq_exit_rcu+0x1a/0x30
[ 1235.907989] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.914456] RIP: 0033:0x7f9941c7222e
[ 1235.919264] RSP: 002b:00007ffd63ae0ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.928562] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9941c7222e
[ 1235.937370] RDX: 0000000000000001 RSI: 0000556bb060bfe0 RDI: 0000000000000007
[ 1235.946179] RBP: 00007ffd63ae0ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.954988] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9941aea390
[ 1235.963799] R13: 0000556bb060bfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.972615]
[ 1235.975881] \x0aShowing all locks held in the system:
[ 1235.984425] 1 lock held by khungtaskd/185:
[ 1235.989819] #0: ffffffff8c0fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.001293] 3 locks held by systemd-journal/801:
[ 1236.007271] #0: ff1100180dd372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1236.017158] #1: ffffffff8c3032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.029975] #2: ff110001a4533db0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.041534] 4 locks held by python3/3674:
[ 1236.046829] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.056719] #1: ff11000168795680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.067683] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.078839] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.088938] 4 locks held by python3/5736:
[ 1236.094234] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.104123] #1: ff110001e85abe80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.115076] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.126231] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.136321]
[ 1236.138781] =============================================\x0a
[ 1358.311949] INFO: task python3:5736 blocked for more than 491 seconds.
[ 1358.320099] Not tainted 7.1.0-rc2-rhfv-ge446480fb140 #1
[ 1358.327350] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.336941] task:python3 state:D stack:25672 pid:5736 tgid:5736 ppid:5735 task_flags:0x400100 flags:0x00080002
[ 1358.350144] Call Trace:
[ 1358.353684]
[ 1358.356836] __schedule+0xcba/0x1940
[ 1358.361650] ? io_schedule_timeout+0x140/0x140
[ 1358.367436] ? find_held_lock+0x2b/0x80
[ 1358.372542] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.378425] schedule+0xd4/0x260
[ 1358.382835] schedule_preempt_disabled+0x18/0x30
[ 1358.388815] __mutex_lock+0x6a3/0x2000
[ 1358.393811] ? threaded_store+0x132/0x220
[ 1358.399112] ? ww_mutex_lock+0x160/0x160
[ 1358.404311] ? selinux_inode_free_security+0x330/0x330
[ 1358.410880] ? cap_capable+0x1d6/0x3e0
[ 1358.415890] ? threaded_store+0x132/0x220
[ 1358.421187] threaded_store+0x132/0x220
[ 1358.426287] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.431582] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.436881] ? sysfs_kf_write+0x65/0x160
[ 1358.442073] ? sysfs_file_ops+0x140/0x140
[ 1358.447361] kernfs_fop_write_iter+0x335/0x510
[ 1358.453147] ? kernfs_ops+0xf0/0xf0
[ 1358.457857] new_sync_write+0x333/0x750
[ 1358.462962] ? __lock_acquire+0x508/0xc10
[ 1358.468257] ? new_sync_read+0x740/0x740
[ 1358.473459] ? lock_acquire.part.0+0xbc/0x260
[ 1358.479146] ? ksys_write+0x116/0x250
[ 1358.484057] vfs_write+0x657/0x9b0
[ 1358.488670] ksys_write+0x116/0x250
[ 1358.493382] ? __ia32_sys_read+0xc0/0xc0
[ 1358.498578] ? rcu_is_watching+0x15/0xd0
[ 1358.503779] ? rcu_is_watching+0x15/0xd0
[ 1358.508983] do_syscall_64+0xf3/0x650
[ 1358.513888] ? irq_exit_rcu+0x1a/0x30
[ 1358.518796] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.525262] RIP: 0033:0x7fd654dd622e
[ 1358.530071] RSP: 002b:00007ffd8bf43990 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.539369] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd654dd622e
[ 1358.548176] RDX: 0000000000000001 RSI: 0000564a90f1afe0 RDI: 0000000000000007
[ 1358.556985] RBP: 00007ffd8bf439a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.565792] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd654c4e390
[ 1358.574589] R13: 0000564a90f1afe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.583400]
[ 1358.586663] INFO: task python3:5736 is blocked on a mutex likely owned by task python3:3674.
[ 1358.596935] task:python3 state:D stack:25672 pid:3674 tgid:3674 ppid:3673 task_flags:0x400100 flags:0x00080802
[ 1358.610133] Call Trace:
[ 1358.613670]
[ 1358.616821] __schedule+0xcba/0x1940
[ 1358.621629] ? io_schedule_timeout+0x140/0x140
[ 1358.627414] ? find_held_lock+0x2b/0x80
[ 1358.632516] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.638694] schedule+0xd4/0x260
[ 1358.643110] schedule_timeout+0x13f/0x290
[ 1358.648407] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.654776] ? trace_contention_end+0xb3/0x1a0
[ 1358.660559] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.666250] ? ww_mutex_lock+0x160/0x160
[ 1358.671450] msleep+0x1b/0x30
[ 1358.675565] napi_set_threaded+0x3a5/0x440
[ 1358.680958] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.686451] netif_set_threaded+0x257/0x400
[ 1358.691945] threaded_store+0x1b8/0x220
[ 1358.697047] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.702345] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.707640] ? sysfs_kf_write+0x65/0x160
[ 1358.712837] ? sysfs_file_ops+0x140/0x140
[ 1358.718128] kernfs_fop_write_iter+0x335/0x510
[ 1358.723912] ? kernfs_ops+0xf0/0xf0
[ 1358.728624] new_sync_write+0x333/0x750
[ 1358.733799] ? __lock_acquire+0x508/0xc10
[ 1358.739094] ? new_sync_read+0x740/0x740
[ 1358.744296] ? lock_acquire.part.0+0xbc/0x260
[ 1358.749981] ? ksys_write+0x116/0x250
[ 1358.754891] vfs_write+0x657/0x9b0
[ 1358.759505] ksys_write+0x116/0x250
[ 1358.764218] ? __ia32_sys_read+0xc0/0xc0
[ 1358.769406] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.774996] ? rcu_is_watching+0x15/0xd0
[ 1358.780195] do_syscall_64+0xf3/0x650
[ 1358.785101] ? irq_exit_rcu+0x1a/0x30
[ 1358.790010] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.796477] RIP: 0033:0x7f9941c7222e
[ 1358.801284] RSP: 002b:00007ffd63ae0ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.810583] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9941c7222e
[ 1358.819390] RDX: 0000000000000001 RSI: 0000556bb060bfe0 RDI: 0000000000000007
[ 1358.828196] RBP: 00007ffd63ae0ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.837006] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9941aea390
[ 1358.845817] R13: 0000556bb060bfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.854629]
[ 1358.857901] \x0aShowing all locks held in the system:
[ 1358.866421] 2 locks held by systemd/1:
[ 1358.871422] #0: ff1100013050c3e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.881314] #1: ff11000142f08280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.892297] 1 lock held by khungtaskd/185:
[ 1358.897691] #0: ffffffff8c0fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.909173] 4 locks held by python3/3674:
[ 1358.914467] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.924357] #1: ff11000168795680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.935320] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.946477] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.956570] 4 locks held by python3/5736:
[ 1358.961866] #0: ff110001302ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.971754] #1: ff110001e85abe80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.982711] #2: ff11000111b20b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.993869] #3: ff11000111b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.003962]
[ 1359.006429] =============================================\x0a