[ 789.047336] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 789.050157] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 789.199222] kselftest: Running tests in drivers/net
[ 989.679865] INFO: task python3:5833 blocked for more than 122 seconds.
[ 989.688096] Not tainted 7.1.0-rc3-xdox-gbab1383a0340 #1
[ 989.695847] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.705437] task:python3 state:D stack:25616 pid:5833 tgid:5833 ppid:5832 task_flags:0x400100 flags:0x00080000
[ 989.718644] Call Trace:
[ 989.722186]
[ 989.725339] __schedule+0xcc3/0x19c0
[ 989.730159] ? io_schedule_timeout+0x140/0x140
[ 989.735945] ? find_held_lock+0x2b/0x80
[ 989.741053] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.746937] schedule+0xd4/0x260
[ 989.751357] schedule_preempt_disabled+0x18/0x30
[ 989.757337] __mutex_lock+0x6a3/0x2000
[ 989.762343] ? threaded_store+0x132/0x220
[ 989.767644] ? ww_mutex_lock+0x160/0x160
[ 989.772843] ? selinux_inode_free_security+0x330/0x330
[ 989.779412] ? cap_capable+0x1d6/0x3e0
[ 989.784420] ? threaded_store+0x132/0x220
[ 989.789719] threaded_store+0x132/0x220
[ 989.794826] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.800124] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.805425] ? sysfs_kf_write+0x65/0x160
[ 989.810624] ? sysfs_file_ops+0x140/0x140
[ 989.815920] kernfs_fop_write_iter+0x335/0x510
[ 989.821707] ? kernfs_ops+0xf0/0xf0
[ 989.826417] new_sync_write+0x333/0x750
[ 989.831511] ? __lock_acquire+0x508/0xc10
[ 989.836810] ? new_sync_read+0x740/0x740
[ 989.842012] ? lock_acquire.part.0+0xbc/0x260
[ 989.847697] ? ksys_write+0x116/0x250
[ 989.852611] vfs_write+0x657/0x9b0
[ 989.857227] ksys_write+0x116/0x250
[ 989.861940] ? __ia32_sys_read+0xc0/0xc0
[ 989.867131] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.872714] ? rcu_is_watching+0x15/0xd0
[ 989.877917] do_syscall_64+0xf3/0x650
[ 989.882823] ? trace_hardirqs_off+0xd/0x30
[ 989.888219] ? exc_page_fault+0xda/0xf0
[ 989.893321] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.899782] RIP: 0033:0x7fa75e2c722e
[ 989.904593] RSP: 002b:00007fff570fdd90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.913889] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa75e2c722e
[ 989.922699] RDX: 0000000000000001 RSI: 000055b5a2c58cd0 RDI: 0000000000000007
[ 989.931507] RBP: 00007fff570fdda0 R08: 0000000000000000 R09: 0000000000000000
[ 989.940307] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa75e13f390
[ 989.949116] R13: 000055b5a2c58cd0 R14: 0000000000000007 R15: 0000000000000003
[ 989.957937]
[ 989.961223] INFO: task python3:5833 is blocked on a mutex likely owned by task python3:3813.
[ 989.971496] task:python3 state:D stack:25672 pid:3813 tgid:3813 ppid:3810 task_flags:0x400100 flags:0x00080802
[ 989.984698] Call Trace:
[ 989.988236]
[ 989.991388] __schedule+0xcc3/0x19c0
[ 989.996200] ? io_schedule_timeout+0x140/0x140
[ 990.001983] ? find_held_lock+0x2b/0x80
[ 990.007089] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.013266] schedule+0xd4/0x260
[ 990.017685] schedule_timeout+0x13f/0x290
[ 990.022981] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.029351] ? trace_contention_end+0xb3/0x1a0
[ 990.035127] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.040810] ? ww_mutex_lock+0x160/0x160
[ 990.046011] msleep+0x1b/0x30
[ 990.050138] napi_set_threaded+0x3a5/0x440
[ 990.055536] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.061033] netif_set_threaded+0x257/0x400
[ 990.066527] threaded_store+0x1b8/0x220
[ 990.071622] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.076920] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.082215] ? sysfs_kf_write+0x65/0x160
[ 990.087415] ? sysfs_file_ops+0x140/0x140
[ 990.092702] kernfs_fop_write_iter+0x335/0x510
[ 990.098490] ? kernfs_ops+0xf0/0xf0
[ 990.103200] new_sync_write+0x333/0x750
[ 990.108302] ? __lock_acquire+0x508/0xc10
[ 990.113601] ? new_sync_read+0x740/0x740
[ 990.118801] ? lock_acquire.part.0+0xbc/0x260
[ 990.124487] ? ksys_write+0x116/0x250
[ 990.129400] vfs_write+0x657/0x9b0
[ 990.134017] ksys_write+0x116/0x250
[ 990.138730] ? __ia32_sys_read+0xc0/0xc0
[ 990.143922] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.149503] ? rcu_is_watching+0x15/0xd0
[ 990.154706] do_syscall_64+0xf3/0x650
[ 990.159614] ? irq_exit_rcu+0x1a/0x30
[ 990.164521] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.170988] RIP: 0033:0x7fa081e1e22e
[ 990.175797] RSP: 002b:00007ffc1573c2c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.185093] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa081e1e22e
[ 990.193901] RDX: 0000000000000001 RSI: 000055d8eba55cd0 RDI: 0000000000000007
[ 990.202711] RBP: 00007ffc1573c2d0 R08: 0000000000000000 R09: 0000000000000000
[ 990.211520] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa081c96390
[ 990.220326] R13: 000055d8eba55cd0 R14: 0000000000000007 R15: 0000000000000003
[ 990.229138]
[ 990.232399] \x0aShowing all locks held in the system:
[ 990.240939] 1 lock held by khungtaskd/185:
[ 990.246330] #0: ffffffffa4afe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.257817] 4 locks held by python3/3813:
[ 990.263114] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.273006] #1: ff11000198481a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.283967] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.295126] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.305221] 4 locks held by python3/5833:
[ 990.310518] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.320407] #1: ff1100011bda4480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.331371] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.342531] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.352620]
[ 990.355087] =============================================\x0a
[ 1112.562104] INFO: task python3:5833 blocked for more than 245 seconds.
[ 1112.570255] Not tainted 7.1.0-rc3-xdox-gbab1383a0340 #1
[ 1112.577506] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.587095] task:python3 state:D stack:25616 pid:5833 tgid:5833 ppid:5832 task_flags:0x400100 flags:0x00080000
[ 1112.600296] Call Trace:
[ 1112.603835]
[ 1112.606989] __schedule+0xcc3/0x19c0
[ 1112.607001] ? io_schedule_timeout+0x140/0x140
[ 1112.617590] ? find_held_lock+0x2b/0x80
[ 1112.622699] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.628580] schedule+0xd4/0x260
[ 1112.633000] schedule_preempt_disabled+0x18/0x30
[ 1112.638980] __mutex_lock+0x6a3/0x2000
[ 1112.643983] ? threaded_store+0x132/0x220
[ 1112.649284] ? ww_mutex_lock+0x160/0x160
[ 1112.654483] ? selinux_inode_free_security+0x330/0x330
[ 1112.661050] ? cap_capable+0x1d6/0x3e0
[ 1112.666058] ? threaded_store+0x132/0x220
[ 1112.671354] threaded_store+0x132/0x220
[ 1112.676458] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.681756] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.687052] ? sysfs_kf_write+0x65/0x160
[ 1112.692251] ? sysfs_file_ops+0x140/0x140
[ 1112.697546] kernfs_fop_write_iter+0x335/0x510
[ 1112.703330] ? kernfs_ops+0xf0/0xf0
[ 1112.708040] new_sync_write+0x333/0x750
[ 1112.713143] ? __lock_acquire+0x508/0xc10
[ 1112.718437] ? new_sync_read+0x740/0x740
[ 1112.723637] ? lock_acquire.part.0+0xbc/0x260
[ 1112.729322] ? ksys_write+0x116/0x250
[ 1112.734229] vfs_write+0x657/0x9b0
[ 1112.738845] ksys_write+0x116/0x250
[ 1112.743556] ? __ia32_sys_read+0xc0/0xc0
[ 1112.748755] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.754346] ? rcu_is_watching+0x15/0xd0
[ 1112.759545] do_syscall_64+0xf3/0x650
[ 1112.764448] ? trace_hardirqs_off+0xd/0x30
[ 1112.769843] ? exc_page_fault+0xda/0xf0
[ 1112.774949] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.781414] RIP: 0033:0x7fa75e2c722e
[ 1112.786223] RSP: 002b:00007fff570fdd90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.795510] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa75e2c722e
[ 1112.804318] RDX: 0000000000000001 RSI: 000055b5a2c58cd0 RDI: 0000000000000007
[ 1112.813126] RBP: 00007fff570fdda0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.821932] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa75e13f390
[ 1112.830740] R13: 000055b5a2c58cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.839552]
[ 1112.842877] INFO: task python3:5833 is blocked on a mutex likely owned by task python3:3813.
[ 1112.853150] task:python3 state:D stack:25672 pid:3813 tgid:3813 ppid:3810 task_flags:0x400100 flags:0x00080802
[ 1112.866353] Call Trace:
[ 1112.869889]
[ 1112.873040] __schedule+0xcc3/0x19c0
[ 1112.877851] ? io_schedule_timeout+0x140/0x140
[ 1112.883633] ? find_held_lock+0x2b/0x80
[ 1112.888737] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.894915] schedule+0xd4/0x260
[ 1112.899333] schedule_timeout+0x13f/0x290
[ 1112.904629] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.910998] ? trace_contention_end+0xb3/0x1a0
[ 1112.916783] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.922464] ? ww_mutex_lock+0x160/0x160
[ 1112.927662] msleep+0x1b/0x30
[ 1112.931786] napi_set_threaded+0x3a5/0x440
[ 1112.937182] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.942669] netif_set_threaded+0x257/0x400
[ 1112.948163] threaded_store+0x1b8/0x220
[ 1112.953264] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.958560] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.963857] ? sysfs_kf_write+0x65/0x160
[ 1112.969055] ? sysfs_file_ops+0x140/0x140
[ 1112.974351] kernfs_fop_write_iter+0x335/0x510
[ 1112.980133] ? kernfs_ops+0xf0/0xf0
[ 1112.984842] new_sync_write+0x333/0x750
[ 1112.989950] ? __lock_acquire+0x508/0xc10
[ 1112.995243] ? new_sync_read+0x740/0x740
[ 1113.000442] ? lock_acquire.part.0+0xbc/0x260
[ 1113.006127] ? ksys_write+0x116/0x250
[ 1113.011035] vfs_write+0x657/0x9b0
[ 1113.015652] ksys_write+0x116/0x250
[ 1113.020366] ? __ia32_sys_read+0xc0/0xc0
[ 1113.025566] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.031154] ? rcu_is_watching+0x15/0xd0
[ 1113.036352] do_syscall_64+0xf3/0x650
[ 1113.041255] ? irq_exit_rcu+0x1a/0x30
[ 1113.046162] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.052629] RIP: 0033:0x7fa081e1e22e
[ 1113.057436] RSP: 002b:00007ffc1573c2c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.066731] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa081e1e22e
[ 1113.075540] RDX: 0000000000000001 RSI: 000055d8eba55cd0 RDI: 0000000000000007
[ 1113.084349] RBP: 00007ffc1573c2d0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.093155] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa081c96390
[ 1113.101961] R13: 000055d8eba55cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.110773]
[ 1113.114036] \x0aShowing all locks held in the system:
[ 1113.122767] 1 lock held by khungtaskd/185:
[ 1113.128158] #0: ffffffffa4afe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.139644] 4 locks held by python3/3813:
[ 1113.144938] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.154834] #1: ff11000198481a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.165793] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.176955] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.187056] 4 locks held by python3/5833:
[ 1113.192350] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.202238] #1: ff1100011bda4480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.213197] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.224353] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.234441] 5 locks held by (systemd-user-)/6876:
[ 1113.240517] #0: ff1100012f8743e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.250405] #1: ff110001649e8280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.261364] #2: ffffffffa4b69de0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1113.272031] #3: ffffffffa4968970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1113.283672] #4: ffffffffa4b69b50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1113.296100]
[ 1113.298565] =============================================\x0a
[ 1174.001737] kworker/u130:4 (528) used greatest stack depth: 21808 bytes left
[ 1235.443373] INFO: task python3:5833 blocked for more than 368 seconds.
[ 1235.451522] Not tainted 7.1.0-rc3-xdox-gbab1383a0340 #1
[ 1235.458770] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.468360] task:python3 state:D stack:25616 pid:5833 tgid:5833 ppid:5832 task_flags:0x400100 flags:0x00080002
[ 1235.481557] Call Trace:
[ 1235.485095]
[ 1235.488248] __schedule+0xcc3/0x19c0
[ 1235.493064] ? io_schedule_timeout+0x140/0x140
[ 1235.498849] ? find_held_lock+0x2b/0x80
[ 1235.503954] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.509838] schedule+0xd4/0x260
[ 1235.514248] schedule_preempt_disabled+0x18/0x30
[ 1235.520222] __mutex_lock+0x6a3/0x2000
[ 1235.525226] ? threaded_store+0x132/0x220
[ 1235.530524] ? ww_mutex_lock+0x160/0x160
[ 1235.535722] ? selinux_inode_free_security+0x330/0x330
[ 1235.542282] ? cap_capable+0x1d6/0x3e0
[ 1235.547290] ? threaded_store+0x132/0x220
[ 1235.552587] threaded_store+0x132/0x220
[ 1235.557687] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.562985] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.568282] ? sysfs_kf_write+0x65/0x160
[ 1235.573483] ? sysfs_file_ops+0x140/0x140
[ 1235.578770] kernfs_fop_write_iter+0x335/0x510
[ 1235.584552] ? kernfs_ops+0xf0/0xf0
[ 1235.589262] new_sync_write+0x333/0x750
[ 1235.594356] ? __lock_acquire+0x508/0xc10
[ 1235.599653] ? new_sync_read+0x740/0x740
[ 1235.604855] ? lock_acquire.part.0+0xbc/0x260
[ 1235.610540] ? ksys_write+0x116/0x250
[ 1235.615449] vfs_write+0x657/0x9b0
[ 1235.620062] ksys_write+0x116/0x250
[ 1235.624770] ? __ia32_sys_read+0xc0/0xc0
[ 1235.629966] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.635552] ? rcu_is_watching+0x15/0xd0
[ 1235.640754] do_syscall_64+0xf3/0x650
[ 1235.645658] ? trace_hardirqs_off+0xd/0x30
[ 1235.651053] ? exc_page_fault+0xda/0xf0
[ 1235.656156] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.662611] RIP: 0033:0x7fa75e2c722e
[ 1235.667421] RSP: 002b:00007fff570fdd90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.676710] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa75e2c722e
[ 1235.685515] RDX: 0000000000000001 RSI: 000055b5a2c58cd0 RDI: 0000000000000007
[ 1235.694324] RBP: 00007fff570fdda0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.703132] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa75e13f390
[ 1235.711943] R13: 000055b5a2c58cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.720756]
[ 1235.724033] INFO: task python3:5833 is blocked on a mutex likely owned by task python3:3813.
[ 1235.734303] task:python3 state:D stack:25672 pid:3813 tgid:3813 ppid:3810 task_flags:0x400100 flags:0x00080802
[ 1235.747506] Call Trace:
[ 1235.751045]
[ 1235.754192] __schedule+0xcc3/0x19c0
[ 1235.759002] ? io_schedule_timeout+0x140/0x140
[ 1235.764787] ? find_held_lock+0x2b/0x80
[ 1235.769887] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.776066] schedule+0xd4/0x260
[ 1235.780484] schedule_timeout+0x13f/0x290
[ 1235.785778] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.792148] ? trace_contention_end+0xb3/0x1a0
[ 1235.797922] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.803612] ? ww_mutex_lock+0x160/0x160
[ 1235.808809] msleep+0x1b/0x30
[ 1235.812931] napi_set_threaded+0x3a5/0x440
[ 1235.818326] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.823821] netif_set_threaded+0x257/0x400
[ 1235.829313] threaded_store+0x1b8/0x220
[ 1235.834413] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.839710] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.845006] ? sysfs_kf_write+0x65/0x160
[ 1235.850202] ? sysfs_file_ops+0x140/0x140
[ 1235.855497] kernfs_fop_write_iter+0x335/0x510
[ 1235.861281] ? kernfs_ops+0xf0/0xf0
[ 1235.865989] new_sync_write+0x333/0x750
[ 1235.871091] ? __lock_acquire+0x508/0xc10
[ 1235.876386] ? new_sync_read+0x740/0x740
[ 1235.881585] ? lock_acquire.part.0+0xbc/0x260
[ 1235.887269] ? ksys_write+0x116/0x250
[ 1235.892179] vfs_write+0x657/0x9b0
[ 1235.896794] ksys_write+0x116/0x250
[ 1235.901505] ? __ia32_sys_read+0xc0/0xc0
[ 1235.906705] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.912285] ? rcu_is_watching+0x15/0xd0
[ 1235.917481] do_syscall_64+0xf3/0x650
[ 1235.922388] ? irq_exit_rcu+0x1a/0x30
[ 1235.927285] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.933752] RIP: 0033:0x7fa081e1e22e
[ 1235.938557] RSP: 002b:00007ffc1573c2c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.947855] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa081e1e22e
[ 1235.956660] RDX: 0000000000000001 RSI: 000055d8eba55cd0 RDI: 0000000000000007
[ 1235.965467] RBP: 00007ffc1573c2d0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.974274] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa081c96390
[ 1235.983080] R13: 000055d8eba55cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.991891]
[ 1235.995154] \x0aShowing all locks held in the system:
[ 1236.003694] 1 lock held by khungtaskd/185:
[ 1236.009088] #0: ffffffffa4afe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.020561] 2 locks held by systemd-journal/802:
[ 1236.026538] #0: ff1100180e9b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.036427] #1: ff110001a199e378 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.047205] 4 locks held by python3/3813:
[ 1236.052500] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.062387] #1: ff11000198481a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.073346] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.084503] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.094595] 4 locks held by python3/5833:
[ 1236.099889] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.109777] #1: ff1100011bda4480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.120739] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.131894] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.141982]
[ 1236.144447] =============================================\x0a
[ 1358.324676] INFO: task python3:5833 blocked for more than 491 seconds.
[ 1358.332827] Not tainted 7.1.0-rc3-xdox-gbab1383a0340 #1
[ 1358.340077] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.349667] task:python3 state:D stack:25616 pid:5833 tgid:5833 ppid:5832 task_flags:0x400100 flags:0x00080002
[ 1358.362874] Call Trace:
[ 1358.366413]
[ 1358.369565] __schedule+0xcc3/0x19c0
[ 1358.374384] ? io_schedule_timeout+0x140/0x140
[ 1358.380169] ? find_held_lock+0x2b/0x80
[ 1358.385275] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.391164] schedule+0xd4/0x260
[ 1358.395585] schedule_preempt_disabled+0x18/0x30
[ 1358.401564] __mutex_lock+0x6a3/0x2000
[ 1358.406570] ? threaded_store+0x132/0x220
[ 1358.411871] ? ww_mutex_lock+0x160/0x160
[ 1358.417069] ? selinux_inode_free_security+0x330/0x330
[ 1358.423642] ? cap_capable+0x1d6/0x3e0
[ 1358.428649] ? threaded_store+0x132/0x220
[ 1358.433948] threaded_store+0x132/0x220
[ 1358.439054] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.444351] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.449650] ? sysfs_kf_write+0x65/0x160
[ 1358.454841] ? sysfs_file_ops+0x140/0x140
[ 1358.460128] kernfs_fop_write_iter+0x335/0x510
[ 1358.465915] ? kernfs_ops+0xf0/0xf0
[ 1358.470625] new_sync_write+0x333/0x750
[ 1358.475729] ? __lock_acquire+0x508/0xc10
[ 1358.481028] ? new_sync_read+0x740/0x740
[ 1358.486229] ? lock_acquire.part.0+0xbc/0x260
[ 1358.491915] ? ksys_write+0x116/0x250
[ 1358.496824] vfs_write+0x657/0x9b0
[ 1358.501440] ksys_write+0x116/0x250
[ 1358.506153] ? __ia32_sys_read+0xc0/0xc0
[ 1358.511353] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.516934] ? rcu_is_watching+0x15/0xd0
[ 1358.522126] do_syscall_64+0xf3/0x650
[ 1358.527034] ? trace_hardirqs_off+0xd/0x30
[ 1358.532430] ? exc_page_fault+0xda/0xf0
[ 1358.537533] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.544002] RIP: 0033:0x7fa75e2c722e
[ 1358.548812] RSP: 002b:00007fff570fdd90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.558105] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa75e2c722e
[ 1358.566913] RDX: 0000000000000001 RSI: 000055b5a2c58cd0 RDI: 0000000000000007
[ 1358.575722] RBP: 00007fff570fdda0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.584531] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa75e13f390
[ 1358.593339] R13: 000055b5a2c58cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.602152]
[ 1358.605474] INFO: task python3:5833 is blocked on a mutex likely owned by task python3:3813.
[ 1358.615747] task:python3 state:D stack:25672 pid:3813 tgid:3813 ppid:3810 task_flags:0x400100 flags:0x00080802
[ 1358.628948] Call Trace:
[ 1358.632493]
[ 1358.635643] __schedule+0xcc3/0x19c0
[ 1358.640454] ? io_schedule_timeout+0x140/0x140
[ 1358.646237] ? find_held_lock+0x2b/0x80
[ 1358.651340] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.657520] schedule+0xd4/0x260
[ 1358.661938] schedule_timeout+0x13f/0x290
[ 1358.667238] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.673605] ? trace_contention_end+0xb3/0x1a0
[ 1358.679391] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.685080] ? ww_mutex_lock+0x160/0x160
[ 1358.690279] msleep+0x1b/0x30
[ 1358.694403] napi_set_threaded+0x3a5/0x440
[ 1358.699800] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.705297] netif_set_threaded+0x257/0x400
[ 1358.710789] threaded_store+0x1b8/0x220
[ 1358.715893] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.721191] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.726478] ? sysfs_kf_write+0x65/0x160
[ 1358.731677] ? sysfs_file_ops+0x140/0x140
[ 1358.736973] kernfs_fop_write_iter+0x335/0x510
[ 1358.742760] ? kernfs_ops+0xf0/0xf0
[ 1358.747469] new_sync_write+0x333/0x750
[ 1358.752572] ? __lock_acquire+0x508/0xc10
[ 1358.757870] ? new_sync_read+0x740/0x740
[ 1358.763071] ? lock_acquire.part.0+0xbc/0x260
[ 1358.768760] ? ksys_write+0x116/0x250
[ 1358.773672] vfs_write+0x657/0x9b0
[ 1358.778287] ksys_write+0x116/0x250
[ 1358.782990] ? __ia32_sys_read+0xc0/0xc0
[ 1358.788182] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.793762] ? rcu_is_watching+0x15/0xd0
[ 1358.798963] do_syscall_64+0xf3/0x650
[ 1358.803872] ? irq_exit_rcu+0x1a/0x30
[ 1358.808778] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.815246] RIP: 0033:0x7fa081e1e22e
[ 1358.820044] RSP: 002b:00007ffc1573c2c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.829343] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa081e1e22e
[ 1358.838151] RDX: 0000000000000001 RSI: 000055d8eba55cd0 RDI: 0000000000000007
[ 1358.846959] RBP: 00007ffc1573c2d0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.855766] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa081c96390
[ 1358.864565] R13: 000055d8eba55cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.873368]
[ 1358.876638] \x0aShowing all locks held in the system:
[ 1358.885176] 1 lock held by khungtaskd/185:
[ 1358.890572] #0: ffffffffa4afe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.902046] 2 locks held by systemd-journal/802:
[ 1358.908025] #0: ff1100180e4372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.917915] #1: ffffffffa4d033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.930743] 4 locks held by python3/3813:
[ 1358.936040] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.945930] #1: ff11000198481a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.956892] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.968050] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.978147] 4 locks held by python3/5833:
[ 1358.983445] #0: ff1100012f7443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.993327] #1: ff1100011bda4480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.004288] #2: ff11000122bc0b50 (kn->active#159){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.015443] #3: ff11000122be9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.025534] 5 locks held by systemd/7676:
[ 1359.030832] #0: ff1100012f8743e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.040722] #1: ff1100015f447480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.051683] #2: ffffffffa4b69de0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1359.062354] #3: ffffffffa4968970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1359.073997] #4: ffffffffa4b69b50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1359.086425]
[ 1359.088893] =============================================\x0a