[ 778.140431] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 778.143418] nipa-hw-worker: [5/44] Retrying drivers/net:napi_threaded.py
[ 778.293291] kselftest: Running tests in drivers/net
[ 989.660993] INFO: task python3:5621 blocked for more than 122 seconds.
[ 989.669197] Not tainted 7.1.0-rc1-nzzr-gda504e9ad9b4 #1
[ 989.676879] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.686461] task:python3 state:D stack:25496 pid:5621 tgid:5621 ppid:5620 task_flags:0x400100 flags:0x00080000
[ 989.699669] Call Trace:
[ 989.703209]
[ 989.706362] __schedule+0xcba/0x1940
[ 989.711180] ? io_schedule_timeout+0x140/0x140
[ 989.716966] ? find_held_lock+0x2b/0x80
[ 989.722074] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.727958] schedule+0xd4/0x260
[ 989.732377] schedule_preempt_disabled+0x18/0x30
[ 989.738349] __mutex_lock+0x6a3/0x2000
[ 989.743355] ? threaded_store+0x132/0x220
[ 989.748658] ? ww_mutex_lock+0x160/0x160
[ 989.753857] ? selinux_inode_free_security+0x330/0x330
[ 989.760427] ? cap_capable+0x1d6/0x3e0
[ 989.765435] ? threaded_store+0x132/0x220
[ 989.770732] threaded_store+0x132/0x220
[ 989.775838] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.781139] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.786441] ? sysfs_kf_write+0x65/0x160
[ 989.791640] ? sysfs_file_ops+0x140/0x140
[ 989.796935] kernfs_fop_write_iter+0x335/0x510
[ 989.802722] ? kernfs_ops+0xf0/0xf0
[ 989.807436] new_sync_write+0x333/0x750
[ 989.812540] ? __lock_acquire+0x508/0xc10
[ 989.817834] ? new_sync_read+0x740/0x740
[ 989.823036] ? lock_acquire.part.0+0xbc/0x260
[ 989.828721] ? ksys_write+0x116/0x250
[ 989.833634] vfs_write+0x657/0x9b0
[ 989.838249] ksys_write+0x116/0x250
[ 989.842953] ? __ia32_sys_read+0xc0/0xc0
[ 989.848154] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.853746] ? rcu_is_watching+0x15/0xd0
[ 989.858948] do_syscall_64+0xf3/0x650
[ 989.863857] ? trace_hardirqs_off+0xd/0x30
[ 989.869254] ? exc_page_fault+0xda/0xf0
[ 989.874355] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.880823] RIP: 0033:0x7fcfe523122e
[ 989.885636] RSP: 002b:00007fff61437360 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.894937] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfe523122e
[ 989.903745] RDX: 0000000000000001 RSI: 00005557d4d553d0 RDI: 0000000000000007
[ 989.912542] RBP: 00007fff61437370 R08: 0000000000000000 R09: 0000000000000000
[ 989.921352] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfe50a9390
[ 989.930160] R13: 00005557d4d553d0 R14: 0000000000000007 R15: 0000000000000003
[ 989.938975]
[ 989.942266] INFO: task python3:5621 is blocked on a mutex likely owned by task python3:3591.
[ 989.952539] task:python3 state:D stack:25672 pid:3591 tgid:3591 ppid:3590 task_flags:0x400100 flags:0x00080802
[ 989.965741] Call Trace:
[ 989.969281]
[ 989.972432] __schedule+0xcba/0x1940
[ 989.977243] ? io_schedule_timeout+0x140/0x140
[ 989.983039] ? find_held_lock+0x2b/0x80
[ 989.988143] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.994322] schedule+0xd4/0x260
[ 989.998739] schedule_timeout+0x13f/0x290
[ 990.004036] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.010406] ? trace_contention_end+0xb3/0x1a0
[ 990.016189] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.021889] ? ww_mutex_lock+0x160/0x160
[ 990.027090] msleep+0x1b/0x30
[ 990.031214] napi_set_threaded+0x3a5/0x440
[ 990.036609] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.042095] netif_set_threaded+0x257/0x400
[ 990.047594] threaded_store+0x1b8/0x220
[ 990.052696] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.057987] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.063284] ? sysfs_kf_write+0x65/0x160
[ 990.068482] ? sysfs_file_ops+0x140/0x140
[ 990.073781] kernfs_fop_write_iter+0x335/0x510
[ 990.079568] ? kernfs_ops+0xf0/0xf0
[ 990.084281] new_sync_write+0x333/0x750
[ 990.089372] ? __lock_acquire+0x508/0xc10
[ 990.094669] ? new_sync_read+0x740/0x740
[ 990.099868] ? lock_acquire.part.0+0xbc/0x260
[ 990.105554] ? ksys_write+0x116/0x250
[ 990.110464] vfs_write+0x657/0x9b0
[ 990.115080] ksys_write+0x116/0x250
[ 990.119790] ? __ia32_sys_read+0xc0/0xc0
[ 990.124989] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.130579] ? rcu_is_watching+0x15/0xd0
[ 990.135779] do_syscall_64+0xf3/0x650
[ 990.140685] ? irq_exit_rcu+0x1a/0x30
[ 990.145586] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.152052] RIP: 0033:0x7fd95fe8422e
[ 990.156860] RSP: 002b:00007ffe50a2ba80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.166155] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd95fe8422e
[ 990.174964] RDX: 0000000000000001 RSI: 0000557b18b8b3d0 RDI: 0000000000000007
[ 990.183773] RBP: 00007ffe50a2ba90 R08: 0000000000000000 R09: 0000000000000000
[ 990.192582] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd95fcfc390
[ 990.201388] R13: 0000557b18b8b3d0 R14: 0000000000000007 R15: 0000000000000003
[ 990.210201]
[ 990.213459] \x0aShowing all locks held in the system:
[ 990.221995] 1 lock held by khungtaskd/184:
[ 990.227389] #0: ffffffffaaefdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.238872] 4 locks held by python3/3591:
[ 990.244166] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.254057] #1: ff1100015910ee80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.265018] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.276169] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.286267] 4 locks held by python3/5621:
[ 990.291562] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.301451] #1: ff11000210b08880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.312404] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.323565] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.333655]
[ 990.336121] =============================================\x0a
[ 1112.542198] INFO: task python3:5621 blocked for more than 245 seconds.
[ 1112.550344] Not tainted 7.1.0-rc1-nzzr-gda504e9ad9b4 #1
[ 1112.557596] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.567183] task:python3 state:D stack:25496 pid:5621 tgid:5621 ppid:5620 task_flags:0x400100 flags:0x00080000
[ 1112.580387] Call Trace:
[ 1112.583926]
[ 1112.587075] __schedule+0xcba/0x1940
[ 1112.591892] ? io_schedule_timeout+0x140/0x140
[ 1112.597677] ? find_held_lock+0x2b/0x80
[ 1112.602779] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.608664] schedule+0xd4/0x260
[ 1112.613082] schedule_preempt_disabled+0x18/0x30
[ 1112.619057] __mutex_lock+0x6a3/0x2000
[ 1112.624060] ? threaded_store+0x132/0x220
[ 1112.629350] ? ww_mutex_lock+0x160/0x160
[ 1112.634549] ? selinux_inode_free_security+0x330/0x330
[ 1112.641119] ? cap_capable+0x1d6/0x3e0
[ 1112.646124] ? threaded_store+0x132/0x220
[ 1112.651420] threaded_store+0x132/0x220
[ 1112.656523] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.661821] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.667117] ? sysfs_kf_write+0x65/0x160
[ 1112.672314] ? sysfs_file_ops+0x140/0x140
[ 1112.677610] kernfs_fop_write_iter+0x335/0x510
[ 1112.683395] ? kernfs_ops+0xf0/0xf0
[ 1112.688106] new_sync_write+0x333/0x750
[ 1112.693207] ? __lock_acquire+0x508/0xc10
[ 1112.698503] ? new_sync_read+0x740/0x740
[ 1112.703704] ? lock_acquire.part.0+0xbc/0x260
[ 1112.709390] ? ksys_write+0x116/0x250
[ 1112.714300] vfs_write+0x657/0x9b0
[ 1112.718911] ksys_write+0x116/0x250
[ 1112.723620] ? __ia32_sys_read+0xc0/0xc0
[ 1112.728817] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.734404] ? rcu_is_watching+0x15/0xd0
[ 1112.739606] do_syscall_64+0xf3/0x650
[ 1112.744510] ? trace_hardirqs_off+0xd/0x30
[ 1112.749905] ? exc_page_fault+0xda/0xf0
[ 1112.755008] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.761473] RIP: 0033:0x7fcfe523122e
[ 1112.766282] RSP: 002b:00007fff61437360 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.775580] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfe523122e
[ 1112.784386] RDX: 0000000000000001 RSI: 00005557d4d553d0 RDI: 0000000000000007
[ 1112.793192] RBP: 00007fff61437370 R08: 0000000000000000 R09: 0000000000000000
[ 1112.801997] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfe50a9390
[ 1112.810804] R13: 00005557d4d553d0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.819618]
[ 1112.822917] INFO: task python3:5621 is blocked on a mutex likely owned by task python3:3591.
[ 1112.833188] task:python3 state:D stack:25672 pid:3591 tgid:3591 ppid:3590 task_flags:0x400100 flags:0x00080802
[ 1112.846389] Call Trace:
[ 1112.849926]
[ 1112.853073] __schedule+0xcba/0x1940
[ 1112.857881] ? io_schedule_timeout+0x140/0x140
[ 1112.863666] ? find_held_lock+0x2b/0x80
[ 1112.868757] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.874934] schedule+0xd4/0x260
[ 1112.879353] schedule_timeout+0x13f/0x290
[ 1112.884642] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.891008] ? trace_contention_end+0xb3/0x1a0
[ 1112.896791] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.902479] ? ww_mutex_lock+0x160/0x160
[ 1112.907676] msleep+0x1b/0x30
[ 1112.911799] napi_set_threaded+0x3a5/0x440
[ 1112.917195] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.922679] netif_set_threaded+0x257/0x400
[ 1112.928169] threaded_store+0x1b8/0x220
[ 1112.933271] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.938567] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.943862] ? sysfs_kf_write+0x65/0x160
[ 1112.949058] ? sysfs_file_ops+0x140/0x140
[ 1112.954351] kernfs_fop_write_iter+0x335/0x510
[ 1112.960136] ? kernfs_ops+0xf0/0xf0
[ 1112.964848] new_sync_write+0x333/0x750
[ 1112.969948] ? __lock_acquire+0x508/0xc10
[ 1112.975245] ? new_sync_read+0x740/0x740
[ 1112.980443] ? lock_acquire.part.0+0xbc/0x260
[ 1112.986126] ? ksys_write+0x116/0x250
[ 1112.991040] vfs_write+0x657/0x9b0
[ 1112.995654] ksys_write+0x116/0x250
[ 1113.000364] ? __ia32_sys_read+0xc0/0xc0
[ 1113.005563] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.011153] ? rcu_is_watching+0x15/0xd0
[ 1113.016353] do_syscall_64+0xf3/0x650
[ 1113.021250] ? irq_exit_rcu+0x1a/0x30
[ 1113.026155] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.032620] RIP: 0033:0x7fd95fe8422e
[ 1113.037426] RSP: 002b:00007ffe50a2ba80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.046720] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd95fe8422e
[ 1113.055527] RDX: 0000000000000001 RSI: 0000557b18b8b3d0 RDI: 0000000000000007
[ 1113.064334] RBP: 00007ffe50a2ba90 R08: 0000000000000000 R09: 0000000000000000
[ 1113.073142] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd95fcfc390
[ 1113.081947] R13: 0000557b18b8b3d0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.090759]
[ 1113.094022] \x0aShowing all locks held in the system:
[ 1113.102563] 1 lock held by khungtaskd/184:
[ 1113.107957] #0: ffffffffaaefdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.119431] 2 locks held by systemd-journal/804:
[ 1113.125407] #0: ff1100180e6b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.135293] #1: ffffffffab1031a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.148118] 4 locks held by python3/3591:
[ 1113.153410] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.163296] #1: ff1100015910ee80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.174255] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.185412] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.195507] 4 locks held by python3/5621:
[ 1113.200804] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.210690] #1: ff11000210b08880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.221651] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.232806] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.242897]
[ 1113.245360] =============================================\x0a
[ 1235.422523] INFO: task python3:5621 blocked for more than 368 seconds.
[ 1235.430673] Not tainted 7.1.0-rc1-nzzr-gda504e9ad9b4 #1
[ 1235.437925] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.447517] task:python3 state:D stack:25496 pid:5621 tgid:5621 ppid:5620 task_flags:0x400100 flags:0x00080002
[ 1235.447527] Call Trace:
[ 1235.447529]
[ 1235.447533] __schedule+0xcba/0x1940
[ 1235.472209] ? io_schedule_timeout+0x140/0x140
[ 1235.477996] ? find_held_lock+0x2b/0x80
[ 1235.483106] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.488991] schedule+0xd4/0x260
[ 1235.493409] schedule_preempt_disabled+0x18/0x30
[ 1235.499386] __mutex_lock+0x6a3/0x2000
[ 1235.504390] ? threaded_store+0x132/0x220
[ 1235.509690] ? ww_mutex_lock+0x160/0x160
[ 1235.514882] ? selinux_inode_free_security+0x330/0x330
[ 1235.521454] ? cap_capable+0x1d6/0x3e0
[ 1235.526464] ? threaded_store+0x132/0x220
[ 1235.531766] threaded_store+0x132/0x220
[ 1235.536870] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.542169] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.547466] ? sysfs_kf_write+0x65/0x160
[ 1235.552668] ? sysfs_file_ops+0x140/0x140
[ 1235.557966] kernfs_fop_write_iter+0x335/0x510
[ 1235.563754] ? kernfs_ops+0xf0/0xf0
[ 1235.568466] new_sync_write+0x333/0x750
[ 1235.573571] ? __lock_acquire+0x508/0xc10
[ 1235.578868] ? new_sync_read+0x740/0x740
[ 1235.584070] ? lock_acquire.part.0+0xbc/0x260
[ 1235.589757] ? ksys_write+0x116/0x250
[ 1235.594658] vfs_write+0x657/0x9b0
[ 1235.599276] ksys_write+0x116/0x250
[ 1235.603989] ? __ia32_sys_read+0xc0/0xc0
[ 1235.609190] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.614783] ? rcu_is_watching+0x15/0xd0
[ 1235.619983] do_syscall_64+0xf3/0x650
[ 1235.624883] ? trace_hardirqs_off+0xd/0x30
[ 1235.630279] ? exc_page_fault+0xda/0xf0
[ 1235.635373] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.641840] RIP: 0033:0x7fcfe523122e
[ 1235.646651] RSP: 002b:00007fff61437360 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.655941] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfe523122e
[ 1235.664748] RDX: 0000000000000001 RSI: 00005557d4d553d0 RDI: 0000000000000007
[ 1235.673559] RBP: 00007fff61437370 R08: 0000000000000000 R09: 0000000000000000
[ 1235.682370] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfe50a9390
[ 1235.691179] R13: 00005557d4d553d0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.699994]
[ 1235.703277] INFO: task python3:5621 is blocked on a mutex likely owned by task python3:3591.
[ 1235.713549] task:python3 state:D stack:25672 pid:3591 tgid:3591 ppid:3590 task_flags:0x400100 flags:0x00080802
[ 1235.726753] Call Trace:
[ 1235.730294]
[ 1235.733439] __schedule+0xcba/0x1940
[ 1235.738249] ? io_schedule_timeout+0x140/0x140
[ 1235.744034] ? find_held_lock+0x2b/0x80
[ 1235.749140] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.755318] schedule+0xd4/0x260
[ 1235.759737] schedule_timeout+0x13f/0x290
[ 1235.765027] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.771388] ? trace_contention_end+0xb3/0x1a0
[ 1235.777166] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.782855] ? ww_mutex_lock+0x160/0x160
[ 1235.788057] msleep+0x1b/0x30
[ 1235.792184] napi_set_threaded+0x3a5/0x440
[ 1235.797570] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.803065] netif_set_threaded+0x257/0x400
[ 1235.808560] threaded_store+0x1b8/0x220
[ 1235.813663] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.818961] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.824257] ? sysfs_kf_write+0x65/0x160
[ 1235.829458] ? sysfs_file_ops+0x140/0x140
[ 1235.834755] kernfs_fop_write_iter+0x335/0x510
[ 1235.840541] ? kernfs_ops+0xf0/0xf0
[ 1235.845253] new_sync_write+0x333/0x750
[ 1235.850363] ? __lock_acquire+0x508/0xc10
[ 1235.855659] ? new_sync_read+0x740/0x740
[ 1235.860862] ? lock_acquire.part.0+0xbc/0x260
[ 1235.866550] ? ksys_write+0x116/0x250
[ 1235.871450] vfs_write+0x657/0x9b0
[ 1235.876064] ksys_write+0x116/0x250
[ 1235.880779] ? __ia32_sys_read+0xc0/0xc0
[ 1235.885981] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.891569] ? rcu_is_watching+0x15/0xd0
[ 1235.896767] do_syscall_64+0xf3/0x650
[ 1235.901673] ? irq_exit_rcu+0x1a/0x30
[ 1235.906581] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.913051] RIP: 0033:0x7fd95fe8422e
[ 1235.917861] RSP: 002b:00007ffe50a2ba80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.927158] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd95fe8422e
[ 1235.935969] RDX: 0000000000000001 RSI: 0000557b18b8b3d0 RDI: 0000000000000007
[ 1235.944779] RBP: 00007ffe50a2ba90 R08: 0000000000000000 R09: 0000000000000000
[ 1235.953586] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd95fcfc390
[ 1235.962392] R13: 0000557b18b8b3d0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.971205]
[ 1235.974469] \x0aShowing all locks held in the system:
[ 1235.983009] 1 lock held by khungtaskd/184:
[ 1235.988402] #0: ffffffffaaefdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1235.999876] 3 locks held by systemd-journal/804:
[ 1236.005848] #0: ff1100180e3b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1236.015737] #1: ffffffffab1031a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.028555] #2: ff110001a5a98de0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.040116] 4 locks held by python3/3591:
[ 1236.045412] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.055301] #1: ff1100015910ee80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.066267] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.077422] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.087521] 4 locks held by python3/5621:
[ 1236.092809] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.102700] #1: ff11000210b08880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.113661] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.124817] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.134906] 5 locks held by (systemd-user-)/7084:
[ 1236.140983] #0: ff1100012eef43e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.150873] #1: ff11000186a03880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.161834] #2: ffffffffaaf69ca0 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1236.172502] #3: ffffffffaad68970 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x6b0
[ 1236.184146] #4: ffffffffaaf69a10 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x6b0
[ 1236.196565]
[ 1236.199032] =============================================\x0a
[ 1358.304854] INFO: task python3:5621 blocked for more than 491 seconds.
[ 1358.313005] Not tainted 7.1.0-rc1-nzzr-gda504e9ad9b4 #1
[ 1358.320258] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.329845] task:python3 state:D stack:25496 pid:5621 tgid:5621 ppid:5620 task_flags:0x400100 flags:0x00080002
[ 1358.343052] Call Trace:
[ 1358.346592]
[ 1358.349744] __schedule+0xcba/0x1940
[ 1358.354558] ? io_schedule_timeout+0x140/0x140
[ 1358.360343] ? find_held_lock+0x2b/0x80
[ 1358.365445] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.371329] schedule+0xd4/0x260
[ 1358.375744] schedule_preempt_disabled+0x18/0x30
[ 1358.381720] __mutex_lock+0x6a3/0x2000
[ 1358.386723] ? threaded_store+0x132/0x220
[ 1358.392013] ? ww_mutex_lock+0x160/0x160
[ 1358.397212] ? selinux_inode_free_security+0x330/0x330
[ 1358.403780] ? cap_capable+0x1d6/0x3e0
[ 1358.408785] ? threaded_store+0x132/0x220
[ 1358.414078] threaded_store+0x132/0x220
[ 1358.419179] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.424480] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.429778] ? sysfs_kf_write+0x65/0x160
[ 1358.434974] ? sysfs_file_ops+0x140/0x140
[ 1358.440267] kernfs_fop_write_iter+0x335/0x510
[ 1358.446051] ? kernfs_ops+0xf0/0xf0
[ 1358.450763] new_sync_write+0x333/0x750
[ 1358.455866] ? __lock_acquire+0x508/0xc10
[ 1358.461162] ? new_sync_read+0x740/0x740
[ 1358.466360] ? lock_acquire.part.0+0xbc/0x260
[ 1358.472048] ? ksys_write+0x116/0x250
[ 1358.476949] vfs_write+0x657/0x9b0
[ 1358.481562] ksys_write+0x116/0x250
[ 1358.486275] ? __ia32_sys_read+0xc0/0xc0
[ 1358.491473] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.497061] ? rcu_is_watching+0x15/0xd0
[ 1358.502261] do_syscall_64+0xf3/0x650
[ 1358.507167] ? trace_hardirqs_off+0xd/0x30
[ 1358.512558] ? exc_page_fault+0xda/0xf0
[ 1358.517661] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.524127] RIP: 0033:0x7fcfe523122e
[ 1358.528933] RSP: 002b:00007fff61437360 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.538229] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfe523122e
[ 1358.547035] RDX: 0000000000000001 RSI: 00005557d4d553d0 RDI: 0000000000000007
[ 1358.555840] RBP: 00007fff61437370 R08: 0000000000000000 R09: 0000000000000000
[ 1358.564649] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfe50a9390
[ 1358.573458] R13: 00005557d4d553d0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.582271]
[ 1358.585559] INFO: task python3:5621 is blocked on a mutex likely owned by task python3:3591.
[ 1358.595829] task:python3 state:D stack:25672 pid:3591 tgid:3591 ppid:3590 task_flags:0x400100 flags:0x00080802
[ 1358.609029] Call Trace:
[ 1358.612568]
[ 1358.615715] __schedule+0xcba/0x1940
[ 1358.620523] ? io_schedule_timeout+0x140/0x140
[ 1358.626303] ? find_held_lock+0x2b/0x80
[ 1358.631404] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.637581] schedule+0xd4/0x260
[ 1358.641999] schedule_timeout+0x13f/0x290
[ 1358.647297] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.653662] ? trace_contention_end+0xb3/0x1a0
[ 1358.659445] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.665133] ? ww_mutex_lock+0x160/0x160
[ 1358.670330] msleep+0x1b/0x30
[ 1358.674454] napi_set_threaded+0x3a5/0x440
[ 1358.679849] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.685344] netif_set_threaded+0x257/0x400
[ 1358.690833] threaded_store+0x1b8/0x220
[ 1358.695936] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.701221] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.706517] ? sysfs_kf_write+0x65/0x160
[ 1358.711713] ? sysfs_file_ops+0x140/0x140
[ 1358.717007] kernfs_fop_write_iter+0x335/0x510
[ 1358.722791] ? kernfs_ops+0xf0/0xf0
[ 1358.727500] new_sync_write+0x333/0x750
[ 1358.732601] ? __lock_acquire+0x508/0xc10
[ 1358.737897] ? new_sync_read+0x740/0x740
[ 1358.743096] ? lock_acquire.part.0+0xbc/0x260
[ 1358.748781] ? ksys_write+0x116/0x250
[ 1358.753691] vfs_write+0x657/0x9b0
[ 1358.758310] ksys_write+0x116/0x250
[ 1358.763022] ? __ia32_sys_read+0xc0/0xc0
[ 1358.768211] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.773800] ? rcu_is_watching+0x15/0xd0
[ 1358.778996] do_syscall_64+0xf3/0x650
[ 1358.783902] ? irq_exit_rcu+0x1a/0x30
[ 1358.788808] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.795274] RIP: 0033:0x7fd95fe8422e
[ 1358.800080] RSP: 002b:00007ffe50a2ba80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.809376] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd95fe8422e
[ 1358.818181] RDX: 0000000000000001 RSI: 0000557b18b8b3d0 RDI: 0000000000000007
[ 1358.826987] RBP: 00007ffe50a2ba90 R08: 0000000000000000 R09: 0000000000000000
[ 1358.835794] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd95fcfc390
[ 1358.844591] R13: 0000557b18b8b3d0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.853402]
[ 1358.856668] \x0aShowing all locks held in the system:
[ 1358.865205] 1 lock held by khungtaskd/184:
[ 1358.870599] #0: ffffffffaaefdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.882079] 4 locks held by python3/3591:
[ 1358.887374] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.897261] #1: ff1100015910ee80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.908222] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.919375] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.929467] 4 locks held by python3/5621:
[ 1358.934761] #0: ff1100012ee043e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.944649] #1: ff11000210b08880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.955609] #2: ff110001218a8b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.966764] #3: ff11000121941d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.976857]
[ 1358.979322] =============================================\x0a