[ 790.404866] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 790.407389] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 790.558491] kselftest: Running tests in drivers/net
[ 915.946650] kworker/u129:2 (187) used greatest stack depth: 21936 bytes left
[ 989.676074] INFO: task python3:5865 blocked for more than 122 seconds.
[ 989.684289] Not tainted 7.1.0-rc3-odag-g1991050ad993 #1
[ 989.692188] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.701777] task:python3 state:D stack:25672 pid:5865 tgid:5865 ppid:5864 task_flags:0x400100 flags:0x00080000
[ 989.714983] Call Trace:
[ 989.718520]
[ 989.721668] __schedule+0xcc3/0x19c0
[ 989.726483] ? io_schedule_timeout+0x140/0x140
[ 989.732265] ? find_held_lock+0x2b/0x80
[ 989.737370] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.743254] schedule+0xd4/0x260
[ 989.747672] schedule_preempt_disabled+0x18/0x30
[ 989.753648] __mutex_lock+0x6a3/0x2000
[ 989.758650] ? threaded_store+0x132/0x220
[ 989.763949] ? ww_mutex_lock+0x160/0x160
[ 989.769145] ? selinux_inode_free_security+0x330/0x330
[ 989.775712] ? cap_capable+0x1d6/0x3e0
[ 989.780721] ? threaded_store+0x132/0x220
[ 989.786018] threaded_store+0x132/0x220
[ 989.791121] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.796418] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.801718] ? sysfs_kf_write+0x65/0x160
[ 989.806914] ? sysfs_file_ops+0x140/0x140
[ 989.812208] kernfs_fop_write_iter+0x335/0x510
[ 989.817992] ? kernfs_ops+0xf0/0xf0
[ 989.822700] new_sync_write+0x333/0x750
[ 989.827805] ? __lock_acquire+0x508/0xc10
[ 989.833093] ? new_sync_read+0x740/0x740
[ 989.838291] ? lock_acquire.part.0+0xbc/0x260
[ 989.843975] ? ksys_write+0x116/0x250
[ 989.848886] vfs_write+0x657/0x9b0
[ 989.853501] ksys_write+0x116/0x250
[ 989.858210] ? __ia32_sys_read+0xc0/0xc0
[ 989.863408] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.868996] ? rcu_is_watching+0x15/0xd0
[ 989.874198] do_syscall_64+0xf3/0x650
[ 989.879105] ? trace_hardirqs_off+0xd/0x30
[ 989.884489] ? exc_page_fault+0xda/0xf0
[ 989.889590] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.896067] RIP: 0033:0x7f254801822e
[ 989.900875] RSP: 002b:00007fff28889f90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.910170] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f254801822e
[ 989.918977] RDX: 0000000000000001 RSI: 0000558f63da6cd0 RDI: 0000000000000007
[ 989.927783] RBP: 00007fff28889fa0 R08: 0000000000000000 R09: 0000000000000000
[ 989.936591] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2547e90390
[ 989.945391] R13: 0000558f63da6cd0 R14: 0000000000000007 R15: 0000000000000003
[ 989.954203]
[ 989.957518] INFO: task python3:5865 is blocked on a mutex likely owned by task python3:3800.
[ 989.967791] task:python3 state:D stack:25672 pid:3800 tgid:3800 ppid:3799 task_flags:0x400100 flags:0x00080802
[ 989.980989] Call Trace:
[ 989.984524]
[ 989.987673] __schedule+0xcc3/0x19c0
[ 989.992483] ? io_schedule_timeout+0x140/0x140
[ 989.998263] ? find_held_lock+0x2b/0x80
[ 990.003366] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.009544] schedule+0xd4/0x260
[ 990.013959] schedule_timeout+0x13f/0x290
[ 990.019254] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.025620] ? sysvec_apic_timer_interrupt+0xaa/0xe0
[ 990.031988] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.037678] ? __free_object+0x2ad/0x490
[ 990.042877] msleep+0x1b/0x30
[ 990.047002] napi_set_threaded+0x3a5/0x440
[ 990.052397] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.057892] netif_set_threaded+0x257/0x400
[ 990.063383] threaded_store+0x1b8/0x220
[ 990.068487] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.073781] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.079078] ? sysfs_kf_write+0x65/0x160
[ 990.084276] ? sysfs_file_ops+0x140/0x140
[ 990.089571] kernfs_fop_write_iter+0x335/0x510
[ 990.095356] ? kernfs_ops+0xf0/0xf0
[ 990.100064] new_sync_write+0x333/0x750
[ 990.105166] ? __lock_acquire+0x508/0xc10
[ 990.110462] ? new_sync_read+0x740/0x740
[ 990.115661] ? lock_acquire.part.0+0xbc/0x260
[ 990.121344] ? ksys_write+0x116/0x250
[ 990.126256] vfs_write+0x657/0x9b0
[ 990.130869] ksys_write+0x116/0x250
[ 990.135582] ? __ia32_sys_read+0xc0/0xc0
[ 990.140772] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.146360] ? rcu_is_watching+0x15/0xd0
[ 990.151561] do_syscall_64+0xf3/0x650
[ 990.156467] ? irq_exit_rcu+0x1a/0x30
[ 990.161364] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.167831] RIP: 0033:0x7f91c2e8f22e
[ 990.172638] RSP: 002b:00007ffc3650f9c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.181932] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91c2e8f22e
[ 990.190738] RDX: 0000000000000001 RSI: 00005568f62fccd0 RDI: 0000000000000007
[ 990.199547] RBP: 00007ffc3650f9d0 R08: 0000000000000000 R09: 0000000000000000
[ 990.208354] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91c2d07390
[ 990.217160] R13: 00005568f62fccd0 R14: 0000000000000007 R15: 0000000000000003
[ 990.225973]
[ 990.229231] \x0aShowing all locks held in the system:
[ 990.237758] 1 lock held by khungtaskd/185:
[ 990.243149] #0: ffffffff922fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.254620] 3 locks held by systemd-journal/800:
[ 990.260597] #0: ff1100180e9b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.270483] #1: ffffffff925033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.283296] #2: ff110001a4e6aff0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.294855] 4 locks held by python3/3800:
[ 990.300148] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.310037] #1: ff110001252aa680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.320997] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.332162] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.342272] 4 locks held by python3/5865:
[ 990.347565] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.357452] #1: ff110001f2315c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.368412] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.379565] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.389650]
[ 990.392116] =============================================\x0a
[ 1112.556838] INFO: task python3:5865 blocked for more than 245 seconds.
[ 1112.564982] Not tainted 7.1.0-rc3-odag-g1991050ad993 #1
[ 1112.572234] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.581821] task:python3 state:D stack:25672 pid:5865 tgid:5865 ppid:5864 task_flags:0x400100 flags:0x00080000
[ 1112.595027] Call Trace:
[ 1112.598566]
[ 1112.601714] __schedule+0xcc3/0x19c0
[ 1112.606532] ? io_schedule_timeout+0x140/0x140
[ 1112.612317] ? find_held_lock+0x2b/0x80
[ 1112.617425] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.623307] schedule+0xd4/0x260
[ 1112.627722] schedule_preempt_disabled+0x18/0x30
[ 1112.633698] __mutex_lock+0x6a3/0x2000
[ 1112.638700] ? threaded_store+0x132/0x220
[ 1112.643998] ? ww_mutex_lock+0x160/0x160
[ 1112.649197] ? selinux_inode_free_security+0x330/0x330
[ 1112.655765] ? cap_capable+0x1d6/0x3e0
[ 1112.660773] ? threaded_store+0x132/0x220
[ 1112.666067] threaded_store+0x132/0x220
[ 1112.671167] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.676467] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.681766] ? sysfs_kf_write+0x65/0x160
[ 1112.686965] ? sysfs_file_ops+0x140/0x140
[ 1112.692258] kernfs_fop_write_iter+0x335/0x510
[ 1112.698043] ? kernfs_ops+0xf0/0xf0
[ 1112.702745] new_sync_write+0x333/0x750
[ 1112.707846] ? __lock_acquire+0x508/0xc10
[ 1112.713141] ? new_sync_read+0x740/0x740
[ 1112.718339] ? lock_acquire.part.0+0xbc/0x260
[ 1112.724025] ? ksys_write+0x116/0x250
[ 1112.728933] vfs_write+0x657/0x9b0
[ 1112.733546] ksys_write+0x116/0x250
[ 1112.738254] ? __ia32_sys_read+0xc0/0xc0
[ 1112.743451] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.749038] ? rcu_is_watching+0x15/0xd0
[ 1112.754239] do_syscall_64+0xf3/0x650
[ 1112.759143] ? trace_hardirqs_off+0xd/0x30
[ 1112.764539] ? exc_page_fault+0xda/0xf0
[ 1112.769642] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.776107] RIP: 0033:0x7f254801822e
[ 1112.780914] RSP: 002b:00007fff28889f90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.790209] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f254801822e
[ 1112.799014] RDX: 0000000000000001 RSI: 0000558f63da6cd0 RDI: 0000000000000007
[ 1112.807820] RBP: 00007fff28889fa0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.816625] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2547e90390
[ 1112.825433] R13: 0000558f63da6cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.834243]
[ 1112.837531] INFO: task python3:5865 is blocked on a mutex likely owned by task python3:3800.
[ 1112.847800] task:python3 state:D stack:25672 pid:3800 tgid:3800 ppid:3799 task_flags:0x400100 flags:0x00080802
[ 1112.860998] Call Trace:
[ 1112.864537]
[ 1112.867686] __schedule+0xcc3/0x19c0
[ 1112.872494] ? io_schedule_timeout+0x140/0x140
[ 1112.878279] ? find_held_lock+0x2b/0x80
[ 1112.883380] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.889557] schedule+0xd4/0x260
[ 1112.893976] schedule_timeout+0x13f/0x290
[ 1112.899270] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.905640] ? sysvec_apic_timer_interrupt+0xaa/0xe0
[ 1112.911999] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.917689] ? __free_object+0x2ad/0x490
[ 1112.922889] msleep+0x1b/0x30
[ 1112.927013] napi_set_threaded+0x3a5/0x440
[ 1112.932408] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.937903] netif_set_threaded+0x257/0x400
[ 1112.943394] threaded_store+0x1b8/0x220
[ 1112.948495] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.953789] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.959083] ? sysfs_kf_write+0x65/0x160
[ 1112.964281] ? sysfs_file_ops+0x140/0x140
[ 1112.969575] kernfs_fop_write_iter+0x335/0x510
[ 1112.975357] ? kernfs_ops+0xf0/0xf0
[ 1112.980065] new_sync_write+0x333/0x750
[ 1112.985165] ? __lock_acquire+0x508/0xc10
[ 1112.990461] ? new_sync_read+0x740/0x740
[ 1112.995660] ? lock_acquire.part.0+0xbc/0x260
[ 1113.001343] ? ksys_write+0x116/0x250
[ 1113.006250] vfs_write+0x657/0x9b0
[ 1113.010865] ksys_write+0x116/0x250
[ 1113.015576] ? __ia32_sys_read+0xc0/0xc0
[ 1113.020775] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.026362] ? rcu_is_watching+0x15/0xd0
[ 1113.031562] do_syscall_64+0xf3/0x650
[ 1113.036459] ? irq_exit_rcu+0x1a/0x30
[ 1113.041363] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.047830] RIP: 0033:0x7f91c2e8f22e
[ 1113.052636] RSP: 002b:00007ffc3650f9c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.061930] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91c2e8f22e
[ 1113.070738] RDX: 0000000000000001 RSI: 00005568f62fccd0 RDI: 0000000000000007
[ 1113.079537] RBP: 00007ffc3650f9d0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.088335] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91c2d07390
[ 1113.097142] R13: 00005568f62fccd0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.105955]
[ 1113.109212] \x0aShowing all locks held in the system:
[ 1113.117748] 1 lock held by khungtaskd/185:
[ 1113.123143] #0: ffffffff922fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.134612] 1 lock held by systemd-journal/800:
[ 1113.140492] #0: ff1100180e1b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.150390] 4 locks held by python3/3800:
[ 1113.155687] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.165577] #1: ff110001252aa680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.176536] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.187691] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.197786] 4 locks held by python3/5865:
[ 1113.203081] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.212969] #1: ff110001f2315c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.223930] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.235077] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.245166]
[ 1113.247629] =============================================\x0a
[ 1235.436988] INFO: task python3:5865 blocked for more than 368 seconds.
[ 1235.445132] Not tainted 7.1.0-rc3-odag-g1991050ad993 #1
[ 1235.452381] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.461972] task:python3 state:D stack:25672 pid:5865 tgid:5865 ppid:5864 task_flags:0x400100 flags:0x00080002
[ 1235.475180] Call Trace:
[ 1235.478719]
[ 1235.481867] __schedule+0xcc3/0x19c0
[ 1235.486685] ? io_schedule_timeout+0x140/0x140
[ 1235.492469] ? find_held_lock+0x2b/0x80
[ 1235.497571] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.503455] schedule+0xd4/0x260
[ 1235.507861] schedule_preempt_disabled+0x18/0x30
[ 1235.513837] __mutex_lock+0x6a3/0x2000
[ 1235.518841] ? threaded_store+0x132/0x220
[ 1235.524130] ? ww_mutex_lock+0x160/0x160
[ 1235.529326] ? selinux_inode_free_security+0x330/0x330
[ 1235.535895] ? cap_capable+0x1d6/0x3e0
[ 1235.540901] ? threaded_store+0x132/0x220
[ 1235.546199] threaded_store+0x132/0x220
[ 1235.551300] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.556597] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.561893] ? sysfs_kf_write+0x65/0x160
[ 1235.567090] ? sysfs_file_ops+0x140/0x140
[ 1235.572383] kernfs_fop_write_iter+0x335/0x510
[ 1235.578165] ? kernfs_ops+0xf0/0xf0
[ 1235.582877] new_sync_write+0x333/0x750
[ 1235.587978] ? __lock_acquire+0x508/0xc10
[ 1235.593273] ? new_sync_read+0x740/0x740
[ 1235.598474] ? lock_acquire.part.0+0xbc/0x260
[ 1235.604161] ? ksys_write+0x116/0x250
[ 1235.609071] vfs_write+0x657/0x9b0
[ 1235.613684] ksys_write+0x116/0x250
[ 1235.618393] ? __ia32_sys_read+0xc0/0xc0
[ 1235.623591] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.629176] ? rcu_is_watching+0x15/0xd0
[ 1235.634378] do_syscall_64+0xf3/0x650
[ 1235.639282] ? trace_hardirqs_off+0xd/0x30
[ 1235.644674] ? exc_page_fault+0xda/0xf0
[ 1235.649777] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.656243] RIP: 0033:0x7f254801822e
[ 1235.661050] RSP: 002b:00007fff28889f90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.670349] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f254801822e
[ 1235.679154] RDX: 0000000000000001 RSI: 0000558f63da6cd0 RDI: 0000000000000007
[ 1235.687960] RBP: 00007fff28889fa0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.696766] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2547e90390
[ 1235.705575] R13: 0000558f63da6cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.714388]
[ 1235.717681] INFO: task python3:5865 is blocked on a mutex likely owned by task python3:3800.
[ 1235.727951] task:python3 state:D stack:25672 pid:3800 tgid:3800 ppid:3799 task_flags:0x400100 flags:0x00080802
[ 1235.741151] Call Trace:
[ 1235.744690]
[ 1235.747837] __schedule+0xcc3/0x19c0
[ 1235.752645] ? io_schedule_timeout+0x140/0x140
[ 1235.758430] ? find_held_lock+0x2b/0x80
[ 1235.763521] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.769699] schedule+0xd4/0x260
[ 1235.774114] schedule_timeout+0x13f/0x290
[ 1235.779412] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.785782] ? sysvec_apic_timer_interrupt+0xaa/0xe0
[ 1235.792152] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.797833] ? __free_object+0x2ad/0x490
[ 1235.803033] msleep+0x1b/0x30
[ 1235.807157] napi_set_threaded+0x3a5/0x440
[ 1235.812552] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.818047] netif_set_threaded+0x257/0x400
[ 1235.823538] threaded_store+0x1b8/0x220
[ 1235.828639] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.833937] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.839234] ? sysfs_kf_write+0x65/0x160
[ 1235.844431] ? sysfs_file_ops+0x140/0x140
[ 1235.849724] kernfs_fop_write_iter+0x335/0x510
[ 1235.855507] ? kernfs_ops+0xf0/0xf0
[ 1235.860214] new_sync_write+0x333/0x750
[ 1235.865314] ? __lock_acquire+0x508/0xc10
[ 1235.870611] ? new_sync_read+0x740/0x740
[ 1235.875809] ? lock_acquire.part.0+0xbc/0x260
[ 1235.881492] ? ksys_write+0x116/0x250
[ 1235.886403] vfs_write+0x657/0x9b0
[ 1235.891017] ksys_write+0x116/0x250
[ 1235.895729] ? __ia32_sys_read+0xc0/0xc0
[ 1235.900929] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.906517] ? rcu_is_watching+0x15/0xd0
[ 1235.911716] do_syscall_64+0xf3/0x650
[ 1235.916612] ? irq_exit_rcu+0x1a/0x30
[ 1235.921517] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.927983] RIP: 0033:0x7f91c2e8f22e
[ 1235.932791] RSP: 002b:00007ffc3650f9c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.942085] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91c2e8f22e
[ 1235.950894] RDX: 0000000000000001 RSI: 00005568f62fccd0 RDI: 0000000000000007
[ 1235.959700] RBP: 00007ffc3650f9d0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.968506] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91c2d07390
[ 1235.977313] R13: 00005568f62fccd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.986125]
[ 1235.989390] \x0aShowing all locks held in the system:
[ 1235.997926] 1 lock held by khungtaskd/185:
[ 1236.003316] #0: ffffffff922fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.014786] 2 locks held by systemd-journal/800:
[ 1236.020763] #0: ff1100180e1b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.030651] #1: ffffffff925033a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.043475] 4 locks held by python3/3800:
[ 1236.048771] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.058657] #1: ff110001252aa680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.069618] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.080772] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.090865] 4 locks held by python3/5865:
[ 1236.096157] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.106045] #1: ff110001f2315c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.117004] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.128160] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.138248]
[ 1236.140712] =============================================\x0a
[ 1358.319185] INFO: task python3:5865 blocked for more than 491 seconds.
[ 1358.327342] Not tainted 7.1.0-rc3-odag-g1991050ad993 #1
[ 1358.334586] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.344175] task:python3 state:D stack:25672 pid:5865 tgid:5865 ppid:5864 task_flags:0x400100 flags:0x00080002
[ 1358.357383] Call Trace:
[ 1358.360924]
[ 1358.364076] __schedule+0xcc3/0x19c0
[ 1358.368895] ? io_schedule_timeout+0x140/0x140
[ 1358.374681] ? find_held_lock+0x2b/0x80
[ 1358.379788] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.385677] schedule+0xd4/0x260
[ 1358.390095] schedule_preempt_disabled+0x18/0x30
[ 1358.396076] __mutex_lock+0x6a3/0x2000
[ 1358.401083] ? threaded_store+0x132/0x220
[ 1358.406382] ? ww_mutex_lock+0x160/0x160
[ 1358.411582] ? selinux_inode_free_security+0x330/0x330
[ 1358.418151] ? cap_capable+0x1d6/0x3e0
[ 1358.423159] ? threaded_store+0x132/0x220
[ 1358.428456] threaded_store+0x132/0x220
[ 1358.433557] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.438855] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.444156] ? sysfs_kf_write+0x65/0x160
[ 1358.449358] ? sysfs_file_ops+0x140/0x140
[ 1358.454655] kernfs_fop_write_iter+0x335/0x510
[ 1358.460437] ? kernfs_ops+0xf0/0xf0
[ 1358.465148] new_sync_write+0x333/0x750
[ 1358.470243] ? __lock_acquire+0x508/0xc10
[ 1358.475539] ? new_sync_read+0x740/0x740
[ 1358.480741] ? lock_acquire.part.0+0xbc/0x260
[ 1358.486428] ? ksys_write+0x116/0x250
[ 1358.491341] vfs_write+0x657/0x9b0
[ 1358.495958] ksys_write+0x116/0x250
[ 1358.500671] ? __ia32_sys_read+0xc0/0xc0
[ 1358.505864] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.511455] ? rcu_is_watching+0x15/0xd0
[ 1358.516656] do_syscall_64+0xf3/0x650
[ 1358.521564] ? trace_hardirqs_off+0xd/0x30
[ 1358.526960] ? exc_page_fault+0xda/0xf0
[ 1358.532064] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.538531] RIP: 0033:0x7f254801822e
[ 1358.543341] RSP: 002b:00007fff28889f90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.552639] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f254801822e
[ 1358.561448] RDX: 0000000000000001 RSI: 0000558f63da6cd0 RDI: 0000000000000007
[ 1358.570257] RBP: 00007fff28889fa0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.579067] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f2547e90390
[ 1358.587875] R13: 0000558f63da6cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.596688]
[ 1358.599965] INFO: task python3:5865 is blocked on a mutex likely owned by task python3:3800.
[ 1358.610235] task:python3 state:D stack:25672 pid:3800 tgid:3800 ppid:3799 task_flags:0x400100 flags:0x00080802
[ 1358.623439] Call Trace:
[ 1358.626975]
[ 1358.630127] __schedule+0xcc3/0x19c0
[ 1358.634930] ? io_schedule_timeout+0x140/0x140
[ 1358.640716] ? find_held_lock+0x2b/0x80
[ 1358.645820] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.651997] schedule+0xd4/0x260
[ 1358.656416] schedule_timeout+0x13f/0x290
[ 1358.661713] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.668082] ? sysvec_apic_timer_interrupt+0xaa/0xe0
[ 1358.674454] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.680142] ? __free_object+0x2ad/0x490
[ 1358.685345] msleep+0x1b/0x30
[ 1358.689471] napi_set_threaded+0x3a5/0x440
[ 1358.694866] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.700359] netif_set_threaded+0x257/0x400
[ 1358.705843] threaded_store+0x1b8/0x220
[ 1358.710948] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.716247] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.721542] ? sysfs_kf_write+0x65/0x160
[ 1358.726742] ? sysfs_file_ops+0x140/0x140
[ 1358.732036] kernfs_fop_write_iter+0x335/0x510
[ 1358.737820] ? kernfs_ops+0xf0/0xf0
[ 1358.742530] new_sync_write+0x333/0x750
[ 1358.747635] ? __lock_acquire+0x508/0xc10
[ 1358.752929] ? new_sync_read+0x740/0x740
[ 1358.758132] ? lock_acquire.part.0+0xbc/0x260
[ 1358.763818] ? ksys_write+0x116/0x250
[ 1358.768727] vfs_write+0x657/0x9b0
[ 1358.773345] ksys_write+0x116/0x250
[ 1358.778055] ? __ia32_sys_read+0xc0/0xc0
[ 1358.783253] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.788844] ? rcu_is_watching+0x15/0xd0
[ 1358.794046] do_syscall_64+0xf3/0x650
[ 1358.798952] ? irq_exit_rcu+0x1a/0x30
[ 1358.803860] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.810328] RIP: 0033:0x7f91c2e8f22e
[ 1358.815137] RSP: 002b:00007ffc3650f9c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.824434] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91c2e8f22e
[ 1358.833244] RDX: 0000000000000001 RSI: 00005568f62fccd0 RDI: 0000000000000007
[ 1358.842054] RBP: 00007ffc3650f9d0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.850862] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91c2d07390
[ 1358.859670] R13: 00005568f62fccd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.868475]
[ 1358.871740] \x0aShowing all locks held in the system:
[ 1358.880276] 1 lock held by khungtaskd/185:
[ 1358.885672] #0: ffffffff922fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.897152] 4 locks held by python3/3800:
[ 1358.902449] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.912341] #1: ff110001252aa680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.923301] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.934458] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.944544] 4 locks held by python3/5865:
[ 1358.949840] #0: ff1100012ff143e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.959728] #1: ff110001f2315c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.970689] #2: ff11000123686b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.981841] #3: ff11000123661d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.991934]
[ 1358.994402] =============================================\x0a