[ 782.590000] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 782.594630] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 782.748810] kselftest: Running tests in drivers/net
[ 989.658989] INFO: task python3:5704 blocked for more than 122 seconds.
[ 989.667215] Not tainted 7.1.0-rc2-ebdt-gd0662d73439f #1
[ 989.675095] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.684683] task:python3 state:D stack:25672 pid:5704 tgid:5704 ppid:5703 task_flags:0x400100 flags:0x00080000
[ 989.697892] Call Trace:
[ 989.701432]
[ 989.704584] __schedule+0xcba/0x1940
[ 989.709402] ? io_schedule_timeout+0x140/0x140
[ 989.715189] ? find_held_lock+0x2b/0x80
[ 989.720293] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.726181] schedule+0xd4/0x260
[ 989.730600] schedule_preempt_disabled+0x18/0x30
[ 989.736578] __mutex_lock+0x6a3/0x2000
[ 989.741585] ? threaded_store+0x132/0x220
[ 989.746885] ? ww_mutex_lock+0x160/0x160
[ 989.752086] ? selinux_inode_free_security+0x330/0x330
[ 989.758659] ? cap_capable+0x1d6/0x3e0
[ 989.763670] ? threaded_store+0x132/0x220
[ 989.768969] threaded_store+0x132/0x220
[ 989.774071] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.779362] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.784659] ? sysfs_kf_write+0x65/0x160
[ 989.789863] ? sysfs_file_ops+0x140/0x140
[ 989.795157] kernfs_fop_write_iter+0x335/0x510
[ 989.800942] ? kernfs_ops+0xf0/0xf0
[ 989.805646] new_sync_write+0x333/0x750
[ 989.810752] ? __lock_acquire+0x508/0xc10
[ 989.816050] ? new_sync_read+0x740/0x740
[ 989.821249] ? lock_acquire.part.0+0xbc/0x260
[ 989.826929] ? ksys_write+0x116/0x250
[ 989.831840] vfs_write+0x657/0x9b0
[ 989.836454] ksys_write+0x116/0x250
[ 989.841168] ? __ia32_sys_read+0xc0/0xc0
[ 989.846368] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.851955] ? rcu_is_watching+0x15/0xd0
[ 989.857159] do_syscall_64+0xf3/0x650
[ 989.862065] ? trace_hardirqs_off+0xd/0x30
[ 989.867462] ? exc_page_fault+0xda/0xf0
[ 989.872565] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.879031] RIP: 0033:0x7f7fa6e1322e
[ 989.883841] RSP: 002b:00007ffda6cf8a60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.893138] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa6e1322e
[ 989.901937] RDX: 0000000000000001 RSI: 0000564c26aa6fe0 RDI: 0000000000000007
[ 989.910735] RBP: 00007ffda6cf8a70 R08: 0000000000000000 R09: 0000000000000000
[ 989.919546] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa6c8b390
[ 989.928353] R13: 0000564c26aa6fe0 R14: 0000000000000007 R15: 0000000000000003
[ 989.937177]
[ 989.940476] INFO: task python3:5704 is blocked on a mutex likely owned by task python3:3642.
[ 989.950748] task:python3 state:D stack:25672 pid:3642 tgid:3642 ppid:3641 task_flags:0x400100 flags:0x00080802
[ 989.963951] Call Trace:
[ 989.967492]
[ 989.970643] __schedule+0xcba/0x1940
[ 989.975448] ? io_schedule_timeout+0x140/0x140
[ 989.981233] ? find_held_lock+0x2b/0x80
[ 989.986335] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.992514] schedule+0xd4/0x260
[ 989.996934] schedule_timeout+0x13f/0x290
[ 990.002232] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.008603] ? trace_contention_end+0xb3/0x1a0
[ 990.014391] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.020080] ? ww_mutex_lock+0x160/0x160
[ 990.025282] msleep+0x1b/0x30
[ 990.029408] napi_set_threaded+0x3a5/0x440
[ 990.034797] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.040291] netif_set_threaded+0x257/0x400
[ 990.045785] threaded_store+0x1b8/0x220
[ 990.050887] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.056186] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.061480] ? sysfs_kf_write+0x65/0x160
[ 990.066681] ? sysfs_file_ops+0x140/0x140
[ 990.071978] kernfs_fop_write_iter+0x335/0x510
[ 990.077754] ? kernfs_ops+0xf0/0xf0
[ 990.082465] new_sync_write+0x333/0x750
[ 990.087570] ? __lock_acquire+0x508/0xc10
[ 990.092867] ? new_sync_read+0x740/0x740
[ 990.098066] ? lock_acquire.part.0+0xbc/0x260
[ 990.103753] ? ksys_write+0x116/0x250
[ 990.108654] vfs_write+0x657/0x9b0
[ 990.113270] ksys_write+0x116/0x250
[ 990.117985] ? __ia32_sys_read+0xc0/0xc0
[ 990.123185] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.128773] ? rcu_is_watching+0x15/0xd0
[ 990.133974] do_syscall_64+0xf3/0x650
[ 990.138882] ? trace_hardirqs_off+0xd/0x30
[ 990.144270] ? exc_page_fault+0xda/0xf0
[ 990.149373] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.155835] RIP: 0033:0x7fcb694b422e
[ 990.160644] RSP: 002b:00007ffcb9e80190 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.169941] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcb694b422e
[ 990.178748] RDX: 0000000000000001 RSI: 00005577232bdfe0 RDI: 0000000000000007
[ 990.187557] RBP: 00007ffcb9e801a0 R08: 0000000000000000 R09: 0000000000000000
[ 990.196364] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcb6932c390
[ 990.205170] R13: 00005577232bdfe0 R14: 0000000000000007 R15: 0000000000000003
[ 990.213984]
[ 990.217244] \x0aShowing all locks held in the system:
[ 990.225781] 1 lock held by khungtaskd/185:
[ 990.231175] #0: ffffffffa1afdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.242650] 2 locks held by systemd-journal/803:
[ 990.248630] #0: ff1100180e2372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 990.258521] #1: ffffffffa1d032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.271349] 4 locks held by python3/3642:
[ 990.276646] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.286537] #1: ff11000169559480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.297500] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.308657] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.318747] 4 locks held by python3/5704:
[ 990.324044] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.333933] #1: ff110001e5e4a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.344893] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.356049] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.366136]
[ 990.368603] =============================================\x0a
[ 1112.540530] INFO: task python3:5704 blocked for more than 245 seconds.
[ 1112.548675] Not tainted 7.1.0-rc2-ebdt-gd0662d73439f #1
[ 1112.555926] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.565515] task:python3 state:D stack:25672 pid:5704 tgid:5704 ppid:5703 task_flags:0x400100 flags:0x00080000
[ 1112.578712] Call Trace:
[ 1112.582252]
[ 1112.585402] __schedule+0xcba/0x1940
[ 1112.590219] ? io_schedule_timeout+0x140/0x140
[ 1112.596005] ? find_held_lock+0x2b/0x80
[ 1112.601109] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.606984] schedule+0xd4/0x260
[ 1112.611403] schedule_preempt_disabled+0x18/0x30
[ 1112.617379] __mutex_lock+0x6a3/0x2000
[ 1112.622381] ? threaded_store+0x132/0x220
[ 1112.627682] ? ww_mutex_lock+0x160/0x160
[ 1112.632881] ? selinux_inode_free_security+0x330/0x330
[ 1112.639449] ? cap_capable+0x1d6/0x3e0
[ 1112.644456] ? threaded_store+0x132/0x220
[ 1112.649744] threaded_store+0x132/0x220
[ 1112.654843] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.660142] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.665438] ? sysfs_kf_write+0x65/0x160
[ 1112.670638] ? sysfs_file_ops+0x140/0x140
[ 1112.675922] kernfs_fop_write_iter+0x335/0x510
[ 1112.681703] ? kernfs_ops+0xf0/0xf0
[ 1112.686415] new_sync_write+0x333/0x750
[ 1112.691506] ? __lock_acquire+0x508/0xc10
[ 1112.696802] ? new_sync_read+0x740/0x740
[ 1112.702003] ? lock_acquire.part.0+0xbc/0x260
[ 1112.707693] ? ksys_write+0x116/0x250
[ 1112.712600] vfs_write+0x657/0x9b0
[ 1112.717217] ksys_write+0x116/0x250
[ 1112.721930] ? __ia32_sys_read+0xc0/0xc0
[ 1112.727126] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.732716] ? rcu_is_watching+0x15/0xd0
[ 1112.737917] do_syscall_64+0xf3/0x650
[ 1112.742821] ? trace_hardirqs_off+0xd/0x30
[ 1112.748215] ? exc_page_fault+0xda/0xf0
[ 1112.753319] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.759786] RIP: 0033:0x7f7fa6e1322e
[ 1112.764597] RSP: 002b:00007ffda6cf8a60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.773898] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa6e1322e
[ 1112.782707] RDX: 0000000000000001 RSI: 0000564c26aa6fe0 RDI: 0000000000000007
[ 1112.791506] RBP: 00007ffda6cf8a70 R08: 0000000000000000 R09: 0000000000000000
[ 1112.800307] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa6c8b390
[ 1112.809119] R13: 0000564c26aa6fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.817934]
[ 1112.821216] INFO: task python3:5704 is blocked on a mutex likely owned by task python3:3642.
[ 1112.831492] task:python3 state:D stack:25672 pid:3642 tgid:3642 ppid:3641 task_flags:0x400100 flags:0x00080802
[ 1112.844697] Call Trace:
[ 1112.848235]
[ 1112.851386] __schedule+0xcba/0x1940
[ 1112.856196] ? io_schedule_timeout+0x140/0x140
[ 1112.861973] ? find_held_lock+0x2b/0x80
[ 1112.867078] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.873258] schedule+0xd4/0x260
[ 1112.877677] schedule_timeout+0x13f/0x290
[ 1112.882977] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.889350] ? trace_contention_end+0xb3/0x1a0
[ 1112.895134] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.900823] ? ww_mutex_lock+0x160/0x160
[ 1112.906026] msleep+0x1b/0x30
[ 1112.910152] napi_set_threaded+0x3a5/0x440
[ 1112.915548] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.921043] netif_set_threaded+0x257/0x400
[ 1112.926607] threaded_store+0x1b8/0x220
[ 1112.931709] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.937009] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.942307] ? sysfs_kf_write+0x65/0x160
[ 1112.947498] ? sysfs_file_ops+0x140/0x140
[ 1112.952796] kernfs_fop_write_iter+0x335/0x510
[ 1112.958644] ? kernfs_ops+0xf0/0xf0
[ 1112.963358] new_sync_write+0x333/0x750
[ 1112.968461] ? __lock_acquire+0x508/0xc10
[ 1112.973761] ? new_sync_read+0x740/0x740
[ 1112.973767] ? lock_acquire.part.0+0xbc/0x260
[ 1112.973770] ? ksys_write+0x116/0x250
[ 1112.973778] vfs_write+0x657/0x9b0
[ 1112.973783] ksys_write+0x116/0x250
[ 1112.998923] ? __ia32_sys_read+0xc0/0xc0
[ 1113.004124] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.009716] ? rcu_is_watching+0x15/0xd0
[ 1113.014915] do_syscall_64+0xf3/0x650
[ 1113.019825] ? trace_hardirqs_off+0xd/0x30
[ 1113.025220] ? exc_page_fault+0xda/0xf0
[ 1113.030322] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.036788] RIP: 0033:0x7fcb694b422e
[ 1113.041599] RSP: 002b:00007ffcb9e80190 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.050895] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcb694b422e
[ 1113.059702] RDX: 0000000000000001 RSI: 00005577232bdfe0 RDI: 0000000000000007
[ 1113.068511] RBP: 00007ffcb9e801a0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.077321] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcb6932c390
[ 1113.086127] R13: 00005577232bdfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.094940]
[ 1113.098204] \x0aShowing all locks held in the system:
[ 1113.106744] 1 lock held by khungtaskd/185:
[ 1113.112138] #0: ffffffffa1afdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.123622] 4 locks held by python3/3642:
[ 1113.128908] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.138800] #1: ff11000169559480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.149760] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.160916] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.171011] 4 locks held by python3/5704:
[ 1113.176307] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.186197] #1: ff110001e5e4a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.197159] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.208308] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.218396] 5 locks held by (systemd-user-)/6763:
[ 1113.224470] #0: ff1100012f9343e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.234357] #1: ff1100013e550280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.245326] #2: ffffffffa1b69ce0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1113.255994] #3: ffffffffa1968970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1113.267639] #4: ffffffffa1b69a50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1113.280069]
[ 1113.282537] =============================================\x0a
[ 1235.422088] INFO: task python3:5704 blocked for more than 368 seconds.
[ 1235.430237] Not tainted 7.1.0-rc2-ebdt-gd0662d73439f #1
[ 1235.437487] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.447077] task:python3 state:D stack:25672 pid:5704 tgid:5704 ppid:5703 task_flags:0x400100 flags:0x00080002
[ 1235.460284] Call Trace:
[ 1235.463826]
[ 1235.466979] __schedule+0xcba/0x1940
[ 1235.471798] ? io_schedule_timeout+0x140/0x140
[ 1235.477585] ? find_held_lock+0x2b/0x80
[ 1235.482691] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.488576] schedule+0xd4/0x260
[ 1235.492993] schedule_preempt_disabled+0x18/0x30
[ 1235.498974] __mutex_lock+0x6a3/0x2000
[ 1235.503981] ? threaded_store+0x132/0x220
[ 1235.509282] ? ww_mutex_lock+0x160/0x160
[ 1235.514481] ? selinux_inode_free_security+0x330/0x330
[ 1235.521050] ? cap_capable+0x1d6/0x3e0
[ 1235.526061] ? threaded_store+0x132/0x220
[ 1235.531360] threaded_store+0x132/0x220
[ 1235.536463] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.541759] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.547059] ? sysfs_kf_write+0x65/0x160
[ 1235.552250] ? sysfs_file_ops+0x140/0x140
[ 1235.557538] kernfs_fop_write_iter+0x335/0x510
[ 1235.563321] ? kernfs_ops+0xf0/0xf0
[ 1235.568031] new_sync_write+0x333/0x750
[ 1235.573132] ? __lock_acquire+0x508/0xc10
[ 1235.578430] ? new_sync_read+0x740/0x740
[ 1235.583631] ? lock_acquire.part.0+0xbc/0x260
[ 1235.589317] ? ksys_write+0x116/0x250
[ 1235.594227] vfs_write+0x657/0x9b0
[ 1235.598843] ksys_write+0x116/0x250
[ 1235.603554] ? __ia32_sys_read+0xc0/0xc0
[ 1235.608755] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.614344] ? rcu_is_watching+0x15/0xd0
[ 1235.619545] do_syscall_64+0xf3/0x650
[ 1235.624450] ? trace_hardirqs_off+0xd/0x30
[ 1235.629846] ? exc_page_fault+0xda/0xf0
[ 1235.634948] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.641418] RIP: 0033:0x7f7fa6e1322e
[ 1235.646228] RSP: 002b:00007ffda6cf8a60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.655527] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa6e1322e
[ 1235.664337] RDX: 0000000000000001 RSI: 0000564c26aa6fe0 RDI: 0000000000000007
[ 1235.673144] RBP: 00007ffda6cf8a70 R08: 0000000000000000 R09: 0000000000000000
[ 1235.681951] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa6c8b390
[ 1235.690761] R13: 0000564c26aa6fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.699575]
[ 1235.702849] INFO: task python3:5704 is blocked on a mutex likely owned by task python3:3642.
[ 1235.713115] task:python3 state:D stack:25672 pid:3642 tgid:3642 ppid:3641 task_flags:0x400100 flags:0x00080802
[ 1235.726317] Call Trace:
[ 1235.729857]
[ 1235.733007] __schedule+0xcba/0x1940
[ 1235.737820] ? io_schedule_timeout+0x140/0x140
[ 1235.743606] ? find_held_lock+0x2b/0x80
[ 1235.748711] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.754889] schedule+0xd4/0x260
[ 1235.759306] schedule_timeout+0x13f/0x290
[ 1235.764604] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.770974] ? trace_contention_end+0xb3/0x1a0
[ 1235.776761] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.782451] ? ww_mutex_lock+0x160/0x160
[ 1235.787652] msleep+0x1b/0x30
[ 1235.791778] napi_set_threaded+0x3a5/0x440
[ 1235.797176] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.802673] netif_set_threaded+0x257/0x400
[ 1235.808166] threaded_store+0x1b8/0x220
[ 1235.813267] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.818564] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.823861] ? sysfs_kf_write+0x65/0x160
[ 1235.829059] ? sysfs_file_ops+0x140/0x140
[ 1235.834356] kernfs_fop_write_iter+0x335/0x510
[ 1235.840142] ? kernfs_ops+0xf0/0xf0
[ 1235.844853] new_sync_write+0x333/0x750
[ 1235.849955] ? __lock_acquire+0x508/0xc10
[ 1235.855251] ? new_sync_read+0x740/0x740
[ 1235.860452] ? lock_acquire.part.0+0xbc/0x260
[ 1235.866138] ? ksys_write+0x116/0x250
[ 1235.871047] vfs_write+0x657/0x9b0
[ 1235.875664] ksys_write+0x116/0x250
[ 1235.880377] ? __ia32_sys_read+0xc0/0xc0
[ 1235.885578] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.891168] ? rcu_is_watching+0x15/0xd0
[ 1235.896360] do_syscall_64+0xf3/0x650
[ 1235.901268] ? trace_hardirqs_off+0xd/0x30
[ 1235.906663] ? exc_page_fault+0xda/0xf0
[ 1235.911768] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.918236] RIP: 0033:0x7fcb694b422e
[ 1235.923043] RSP: 002b:00007ffcb9e80190 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.932339] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcb694b422e
[ 1235.941150] RDX: 0000000000000001 RSI: 00005577232bdfe0 RDI: 0000000000000007
[ 1235.949960] RBP: 00007ffcb9e801a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.958766] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcb6932c390
[ 1235.967575] R13: 00005577232bdfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.976389]
[ 1235.979651] \x0aShowing all locks held in the system:
[ 1235.988190] 1 lock held by khungtaskd/185:
[ 1235.993581] #0: ffffffffa1afdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.005062] 4 locks held by python3/3642:
[ 1236.010360] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.020241] #1: ff11000169559480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.031202] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.042357] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.052451] 4 locks held by python3/5704:
[ 1236.057746] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.067637] #1: ff110001e5e4a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.078599] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.089757] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.099850]
[ 1236.102318] =============================================\x0a
[ 1358.303525] INFO: task python3:5704 blocked for more than 491 seconds.
[ 1358.311674] Not tainted 7.1.0-rc2-ebdt-gd0662d73439f #1
[ 1358.318925] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.328515] task:python3 state:D stack:25672 pid:5704 tgid:5704 ppid:5703 task_flags:0x400100 flags:0x00080002
[ 1358.341720] Call Trace:
[ 1358.345327]
[ 1358.348479] __schedule+0xcba/0x1940
[ 1358.353297] ? io_schedule_timeout+0x140/0x140
[ 1358.359080] ? find_held_lock+0x2b/0x80
[ 1358.364178] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.370067] schedule+0xd4/0x260
[ 1358.374486] schedule_preempt_disabled+0x18/0x30
[ 1358.380467] __mutex_lock+0x6a3/0x2000
[ 1358.385472] ? threaded_store+0x132/0x220
[ 1358.390772] ? ww_mutex_lock+0x160/0x160
[ 1358.395971] ? selinux_inode_free_security+0x330/0x330
[ 1358.402543] ? cap_capable+0x1d6/0x3e0
[ 1358.407556] ? threaded_store+0x132/0x220
[ 1358.412853] threaded_store+0x132/0x220
[ 1358.417947] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.423237] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.428537] ? sysfs_kf_write+0x65/0x160
[ 1358.433737] ? sysfs_file_ops+0x140/0x140
[ 1358.439024] kernfs_fop_write_iter+0x335/0x510
[ 1358.444810] ? kernfs_ops+0xf0/0xf0
[ 1358.449520] new_sync_write+0x333/0x750
[ 1358.454616] ? __lock_acquire+0x508/0xc10
[ 1358.459903] ? new_sync_read+0x740/0x740
[ 1358.465103] ? lock_acquire.part.0+0xbc/0x260
[ 1358.470790] ? ksys_write+0x116/0x250
[ 1358.475702] vfs_write+0x657/0x9b0
[ 1358.480317] ksys_write+0x116/0x250
[ 1358.485029] ? __ia32_sys_read+0xc0/0xc0
[ 1358.490220] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.495810] ? rcu_is_watching+0x15/0xd0
[ 1358.501016] do_syscall_64+0xf3/0x650
[ 1358.505924] ? trace_hardirqs_off+0xd/0x30
[ 1358.511317] ? exc_page_fault+0xda/0xf0
[ 1358.516420] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.522890] RIP: 0033:0x7f7fa6e1322e
[ 1358.527698] RSP: 002b:00007ffda6cf8a60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.536997] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7fa6e1322e
[ 1358.545807] RDX: 0000000000000001 RSI: 0000564c26aa6fe0 RDI: 0000000000000007
[ 1358.554617] RBP: 00007ffda6cf8a70 R08: 0000000000000000 R09: 0000000000000000
[ 1358.563427] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7fa6c8b390
[ 1358.572236] R13: 0000564c26aa6fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.581048]
[ 1358.584390] INFO: task python3:5704 is blocked on a mutex likely owned by task python3:3642.
[ 1358.594663] task:python3 state:D stack:25672 pid:3642 tgid:3642 ppid:3641 task_flags:0x400100 flags:0x00080802
[ 1358.607864] Call Trace:
[ 1358.611402]
[ 1358.614553] __schedule+0xcba/0x1940
[ 1358.619370] ? io_schedule_timeout+0x140/0x140
[ 1358.625153] ? find_held_lock+0x2b/0x80
[ 1358.630256] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.636435] schedule+0xd4/0x260
[ 1358.640854] schedule_timeout+0x13f/0x290
[ 1358.646155] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.652527] ? trace_contention_end+0xb3/0x1a0
[ 1358.658311] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.664000] ? ww_mutex_lock+0x160/0x160
[ 1358.669201] msleep+0x1b/0x30
[ 1358.673329] napi_set_threaded+0x3a5/0x440
[ 1358.678725] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.684220] netif_set_threaded+0x257/0x400
[ 1358.689717] threaded_store+0x1b8/0x220
[ 1358.694818] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.700116] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.705413] ? sysfs_kf_write+0x65/0x160
[ 1358.710611] ? sysfs_file_ops+0x140/0x140
[ 1358.715910] kernfs_fop_write_iter+0x335/0x510
[ 1358.721692] ? kernfs_ops+0xf0/0xf0
[ 1358.726405] new_sync_write+0x333/0x750
[ 1358.731506] ? __lock_acquire+0x508/0xc10
[ 1358.736803] ? new_sync_read+0x740/0x740
[ 1358.742005] ? lock_acquire.part.0+0xbc/0x260
[ 1358.747692] ? ksys_write+0x116/0x250
[ 1358.752601] vfs_write+0x657/0x9b0
[ 1358.757220] ksys_write+0x116/0x250
[ 1358.761934] ? __ia32_sys_read+0xc0/0xc0
[ 1358.767132] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.772722] ? rcu_is_watching+0x15/0xd0
[ 1358.777923] do_syscall_64+0xf3/0x650
[ 1358.782829] ? trace_hardirqs_off+0xd/0x30
[ 1358.788221] ? exc_page_fault+0xda/0xf0
[ 1358.793326] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.799792] RIP: 0033:0x7fcb694b422e
[ 1358.804601] RSP: 002b:00007ffcb9e80190 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.813901] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcb694b422e
[ 1358.822707] RDX: 0000000000000001 RSI: 00005577232bdfe0 RDI: 0000000000000007
[ 1358.831515] RBP: 00007ffcb9e801a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.840322] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcb6932c390
[ 1358.849128] R13: 00005577232bdfe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.857940]
[ 1358.861210] \x0aShowing all locks held in the system:
[ 1358.869753] 1 lock held by khungtaskd/185:
[ 1358.875145] #0: ffffffffa1afdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.886619] 3 locks held by systemd-journal/803:
[ 1358.892600] #0: ff1100180da372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1358.902489] #1: ffffffffa1d032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.915306] #2: ff1100010e2c5db0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.926867] 4 locks held by python3/3642:
[ 1358.932163] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.942054] #1: ff11000169559480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.953019] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.964176] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.974273] 4 locks held by python3/5704:
[ 1358.979568] #0: ff1100012f5a43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.989455] #1: ff110001e5e4a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.000417] #2: ff11000111742b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.011574] #3: ff110001117c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.021662] 5 locks held by systemd/7558:
[ 1359.026958] #0: ff1100012f9343e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.036845] #1: ff11000131544480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.047808] #2: ffffffffa1b69ce0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1359.058475] #3: ffffffffa1968970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1359.070118] #4: ffffffffa1b69a50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1359.082544]
[ 1359.085010] =============================================\x0a