[ 790.915448] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 790.918289] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 791.067767] kselftest: Running tests in drivers/net
[ 989.669152] INFO: task python3:5853 blocked for more than 122 seconds.
[ 989.677381] Not tainted 7.1.0-rc3-nfvp-g0c01f800133a #1
[ 989.685267] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.694856] task:python3 state:D stack:25672 pid:5853 tgid:5853 ppid:5852 task_flags:0x400100 flags:0x00080000
[ 989.708060] Call Trace:
[ 989.711598]
[ 989.714749] __schedule+0xcc3/0x19c0
[ 989.719565] ? io_schedule_timeout+0x140/0x140
[ 989.725348] ? find_held_lock+0x2b/0x80
[ 989.730454] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.736340] schedule+0xd4/0x260
[ 989.740758] schedule_preempt_disabled+0x18/0x30
[ 989.746737] __mutex_lock+0x6a3/0x2000
[ 989.751740] ? threaded_store+0x132/0x220
[ 989.757040] ? ww_mutex_lock+0x160/0x160
[ 989.762236] ? selinux_inode_free_security+0x330/0x330
[ 989.768807] ? cap_capable+0x1d6/0x3e0
[ 989.773819] ? threaded_store+0x132/0x220
[ 989.779116] threaded_store+0x132/0x220
[ 989.784219] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.789515] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.794815] ? sysfs_kf_write+0x65/0x160
[ 989.800011] ? sysfs_file_ops+0x140/0x140
[ 989.805305] kernfs_fop_write_iter+0x335/0x510
[ 989.811091] ? kernfs_ops+0xf0/0xf0
[ 989.815790] new_sync_write+0x333/0x750
[ 989.820892] ? __lock_acquire+0x508/0xc10
[ 989.826189] ? new_sync_read+0x740/0x740
[ 989.831379] ? lock_acquire.part.0+0xbc/0x260
[ 989.837065] ? ksys_write+0x116/0x250
[ 989.841973] vfs_write+0x657/0x9b0
[ 989.846593] ksys_write+0x116/0x250
[ 989.851294] ? __ia32_sys_read+0xc0/0xc0
[ 989.856494] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.862084] ? rcu_is_watching+0x15/0xd0
[ 989.867281] do_syscall_64+0xf3/0x650
[ 989.872188] ? trace_hardirqs_off+0xd/0x30
[ 989.877575] ? exc_page_fault+0xda/0xf0
[ 989.882676] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.889143] RIP: 0033:0x7f9a9f66f22e
[ 989.893953] RSP: 002b:00007ffffde13b50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.903251] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9a9f66f22e
[ 989.912060] RDX: 0000000000000001 RSI: 000056054e18dcd0 RDI: 0000000000000007
[ 989.920868] RBP: 00007ffffde13b60 R08: 0000000000000000 R09: 0000000000000000
[ 989.929679] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9a9f4e7390
[ 989.938485] R13: 000056054e18dcd0 R14: 0000000000000007 R15: 0000000000000003
[ 989.947306]
[ 989.950579] INFO: task python3:5853 is blocked on a mutex likely owned by task python3:3821.
[ 989.960849] task:python3 state:D stack:25672 pid:3821 tgid:3821 ppid:3820 task_flags:0x400100 flags:0x00080802
[ 989.974051] Call Trace:
[ 989.977588]
[ 989.980738] __schedule+0xcc3/0x19c0
[ 989.985548] ? io_schedule_timeout+0x140/0x140
[ 989.991322] ? find_held_lock+0x2b/0x80
[ 989.996427] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.002602] schedule+0xd4/0x260
[ 990.007019] schedule_timeout+0x13f/0x290
[ 990.012315] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.018683] ? trace_contention_end+0xb3/0x1a0
[ 990.024469] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.030159] ? ww_mutex_lock+0x160/0x160
[ 990.035358] msleep+0x1b/0x30
[ 990.039483] napi_set_threaded+0x3a5/0x440
[ 990.044879] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.050366] netif_set_threaded+0x257/0x400
[ 990.055858] threaded_store+0x1b8/0x220
[ 990.060948] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.066246] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.071540] ? sysfs_kf_write+0x65/0x160
[ 990.076739] ? sysfs_file_ops+0x140/0x140
[ 990.082033] kernfs_fop_write_iter+0x335/0x510
[ 990.087815] ? kernfs_ops+0xf0/0xf0
[ 990.092525] new_sync_write+0x333/0x750
[ 990.097627] ? __lock_acquire+0x508/0xc10
[ 990.102920] ? new_sync_read+0x740/0x740
[ 990.108120] ? lock_acquire.part.0+0xbc/0x260
[ 990.113807] ? ksys_write+0x116/0x250
[ 990.118709] vfs_write+0x657/0x9b0
[ 990.123322] ksys_write+0x116/0x250
[ 990.128031] ? __ia32_sys_read+0xc0/0xc0
[ 990.133228] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.138814] ? rcu_is_watching+0x15/0xd0
[ 990.144013] do_syscall_64+0xf3/0x650
[ 990.148916] ? trace_hardirqs_off+0xd/0x30
[ 990.154309] ? exc_page_fault+0xda/0xf0
[ 990.159413] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.165880] RIP: 0033:0x7fa272e9422e
[ 990.170688] RSP: 002b:00007ffe1ebe1bf0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.179989] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa272e9422e
[ 990.188796] RDX: 0000000000000001 RSI: 0000562a54b26cd0 RDI: 0000000000000007
[ 990.197596] RBP: 00007ffe1ebe1c00 R08: 0000000000000000 R09: 0000000000000000
[ 990.206401] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa272d0c390
[ 990.215206] R13: 0000562a54b26cd0 R14: 0000000000000007 R15: 0000000000000003
[ 990.224016]
[ 990.227273] \x0aShowing all locks held in the system:
[ 990.235800] 1 lock held by khungtaskd/185:
[ 990.241192] #0: ffffffffb16fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.252673] 4 locks held by python3/3821:
[ 990.257966] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.267854] #1: ff11000172b9fa80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.278816] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.289962] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.300054] 4 locks held by python3/5853:
[ 990.305348] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.315235] #1: ff1100020d7c2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.326197] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.337355] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.347442]
[ 990.349905] =============================================\x0a
[ 1112.551344] INFO: task python3:5853 blocked for more than 245 seconds.
[ 1112.559492] Not tainted 7.1.0-rc3-nfvp-g0c01f800133a #1
[ 1112.566748] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.576343] task:python3 state:D stack:25672 pid:5853 tgid:5853 ppid:5852 task_flags:0x400100 flags:0x00080000
[ 1112.589558] Call Trace:
[ 1112.593101]
[ 1112.596258] __schedule+0xcc3/0x19c0
[ 1112.601080] ? io_schedule_timeout+0x140/0x140
[ 1112.606870] ? find_held_lock+0x2b/0x80
[ 1112.611980] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.617872] schedule+0xd4/0x260
[ 1112.622296] schedule_preempt_disabled+0x18/0x30
[ 1112.628281] __mutex_lock+0x6a3/0x2000
[ 1112.633290] ? threaded_store+0x132/0x220
[ 1112.638597] ? ww_mutex_lock+0x160/0x160
[ 1112.643790] ? selinux_inode_free_security+0x330/0x330
[ 1112.650367] ? cap_capable+0x1d6/0x3e0
[ 1112.655377] ? threaded_store+0x132/0x220
[ 1112.660678] threaded_store+0x132/0x220
[ 1112.665785] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.671089] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.676391] ? sysfs_kf_write+0x65/0x160
[ 1112.681597] ? sysfs_file_ops+0x140/0x140
[ 1112.686898] kernfs_fop_write_iter+0x335/0x510
[ 1112.692688] ? kernfs_ops+0xf0/0xf0
[ 1112.697405] new_sync_write+0x333/0x750
[ 1112.702513] ? __lock_acquire+0x508/0xc10
[ 1112.707815] ? new_sync_read+0x740/0x740
[ 1112.713025] ? lock_acquire.part.0+0xbc/0x260
[ 1112.718716] ? ksys_write+0x116/0x250
[ 1112.723634] vfs_write+0x657/0x9b0
[ 1112.728252] ksys_write+0x116/0x250
[ 1112.732968] ? __ia32_sys_read+0xc0/0xc0
[ 1112.738173] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.743766] ? rcu_is_watching+0x15/0xd0
[ 1112.748972] do_syscall_64+0xf3/0x650
[ 1112.753884] ? trace_hardirqs_off+0xd/0x30
[ 1112.759283] ? exc_page_fault+0xda/0xf0
[ 1112.764393] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.770855] RIP: 0033:0x7f9a9f66f22e
[ 1112.775669] RSP: 002b:00007ffffde13b50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.784973] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9a9f66f22e
[ 1112.793785] RDX: 0000000000000001 RSI: 000056054e18dcd0 RDI: 0000000000000007
[ 1112.802595] RBP: 00007ffffde13b60 R08: 0000000000000000 R09: 0000000000000000
[ 1112.811408] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9a9f4e7390
[ 1112.820222] R13: 000056054e18dcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.829048]
[ 1112.832348] INFO: task python3:5853 is blocked on a mutex likely owned by task python3:3821.
[ 1112.842626] task:python3 state:D stack:25672 pid:3821 tgid:3821 ppid:3820 task_flags:0x400100 flags:0x00080802
[ 1112.855834] Call Trace:
[ 1112.859373]
[ 1112.862528] __schedule+0xcc3/0x19c0
[ 1112.867347] ? io_schedule_timeout+0x140/0x140
[ 1112.873135] ? find_held_lock+0x2b/0x80
[ 1112.878247] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.884428] schedule+0xd4/0x260
[ 1112.888849] schedule_timeout+0x13f/0x290
[ 1112.894152] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.900526] ? trace_contention_end+0xb3/0x1a0
[ 1112.906315] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.912014] ? ww_mutex_lock+0x160/0x160
[ 1112.917216] msleep+0x1b/0x30
[ 1112.921343] napi_set_threaded+0x3a5/0x440
[ 1112.926743] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.932245] netif_set_threaded+0x257/0x400
[ 1112.937740] threaded_store+0x1b8/0x220
[ 1112.942849] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.948151] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.953454] ? sysfs_kf_write+0x65/0x160
[ 1112.958658] ? sysfs_file_ops+0x140/0x140
[ 1112.963957] kernfs_fop_write_iter+0x335/0x510
[ 1112.969744] ? kernfs_ops+0xf0/0xf0
[ 1112.974461] new_sync_write+0x333/0x750
[ 1112.979565] ? __lock_acquire+0x508/0xc10
[ 1112.984867] ? new_sync_read+0x740/0x740
[ 1112.990072] ? lock_acquire.part.0+0xbc/0x260
[ 1112.995764] ? ksys_write+0x116/0x250
[ 1113.000677] vfs_write+0x657/0x9b0
[ 1113.005298] ksys_write+0x116/0x250
[ 1113.010012] ? __ia32_sys_read+0xc0/0xc0
[ 1113.015205] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.020796] ? rcu_is_watching+0x15/0xd0
[ 1113.025993] do_syscall_64+0xf3/0x650
[ 1113.030901] ? trace_hardirqs_off+0xd/0x30
[ 1113.036300] ? exc_page_fault+0xda/0xf0
[ 1113.041407] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.047878] RIP: 0033:0x7fa272e9422e
[ 1113.052691] RSP: 002b:00007ffe1ebe1bf0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.061996] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa272e9422e
[ 1113.070809] RDX: 0000000000000001 RSI: 0000562a54b26cd0 RDI: 0000000000000007
[ 1113.079624] RBP: 00007ffe1ebe1c00 R08: 0000000000000000 R09: 0000000000000000
[ 1113.088428] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa272d0c390
[ 1113.097242] R13: 0000562a54b26cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.106060]
[ 1113.109327] \x0aShowing all locks held in the system:
[ 1113.117870] 1 lock held by khungtaskd/185:
[ 1113.123269] #0: ffffffffb16fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.134768] 4 locks held by python3/3821:
[ 1113.140069] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.149968] #1: ff11000172b9fa80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.160940] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.172108] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.182215] 4 locks held by python3/5853:
[ 1113.187504] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.197402] #1: ff1100020d7c2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.208370] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.219536] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.229634]
[ 1113.232104] =============================================\x0a
[ 1235.432544] INFO: task python3:5853 blocked for more than 368 seconds.
[ 1235.440693] Not tainted 7.1.0-rc3-nfvp-g0c01f800133a #1
[ 1235.447943] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.457530] task:python3 state:D stack:25672 pid:5853 tgid:5853 ppid:5852 task_flags:0x400100 flags:0x00080002
[ 1235.470736] Call Trace:
[ 1235.474278]
[ 1235.477429] __schedule+0xcc3/0x19c0
[ 1235.482245] ? io_schedule_timeout+0x140/0x140
[ 1235.488032] ? find_held_lock+0x2b/0x80
[ 1235.493141] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.499025] schedule+0xd4/0x260
[ 1235.503443] schedule_preempt_disabled+0x18/0x30
[ 1235.509421] __mutex_lock+0x6a3/0x2000
[ 1235.514424] ? threaded_store+0x132/0x220
[ 1235.519724] ? ww_mutex_lock+0x160/0x160
[ 1235.524925] ? selinux_inode_free_security+0x330/0x330
[ 1235.531495] ? cap_capable+0x1d6/0x3e0
[ 1235.536508] ? threaded_store+0x132/0x220
[ 1235.541804] threaded_store+0x132/0x220
[ 1235.546906] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.552205] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.557504] ? sysfs_kf_write+0x65/0x160
[ 1235.562705] ? sysfs_file_ops+0x140/0x140
[ 1235.568004] kernfs_fop_write_iter+0x335/0x510
[ 1235.573787] ? kernfs_ops+0xf0/0xf0
[ 1235.578500] new_sync_write+0x333/0x750
[ 1235.583606] ? __lock_acquire+0x508/0xc10
[ 1235.588903] ? new_sync_read+0x740/0x740
[ 1235.594106] ? lock_acquire.part.0+0xbc/0x260
[ 1235.599794] ? ksys_write+0x116/0x250
[ 1235.604704] vfs_write+0x657/0x9b0
[ 1235.609322] ksys_write+0x116/0x250
[ 1235.614036] ? __ia32_sys_read+0xc0/0xc0
[ 1235.619238] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.624829] ? rcu_is_watching+0x15/0xd0
[ 1235.630031] do_syscall_64+0xf3/0x650
[ 1235.634941] ? trace_hardirqs_off+0xd/0x30
[ 1235.640337] ? exc_page_fault+0xda/0xf0
[ 1235.645432] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.651899] RIP: 0033:0x7f9a9f66f22e
[ 1235.656701] RSP: 002b:00007ffffde13b50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.666002] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9a9f66f22e
[ 1235.674810] RDX: 0000000000000001 RSI: 000056054e18dcd0 RDI: 0000000000000007
[ 1235.683616] RBP: 00007ffffde13b60 R08: 0000000000000000 R09: 0000000000000000
[ 1235.692424] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9a9f4e7390
[ 1235.701224] R13: 000056054e18dcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.710039]
[ 1235.713327] INFO: task python3:5853 is blocked on a mutex likely owned by task python3:3821.
[ 1235.723599] task:python3 state:D stack:25672 pid:3821 tgid:3821 ppid:3820 task_flags:0x400100 flags:0x00080802
[ 1235.736801] Call Trace:
[ 1235.740339]
[ 1235.743491] __schedule+0xcc3/0x19c0
[ 1235.748305] ? io_schedule_timeout+0x140/0x140
[ 1235.754092] ? find_held_lock+0x2b/0x80
[ 1235.759197] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.765380] schedule+0xd4/0x260
[ 1235.769798] schedule_timeout+0x13f/0x290
[ 1235.775088] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.781449] ? trace_contention_end+0xb3/0x1a0
[ 1235.787233] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.792925] ? ww_mutex_lock+0x160/0x160
[ 1235.798126] msleep+0x1b/0x30
[ 1235.802253] napi_set_threaded+0x3a5/0x440
[ 1235.807647] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.813141] netif_set_threaded+0x257/0x400
[ 1235.818635] threaded_store+0x1b8/0x220
[ 1235.823738] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.829036] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.834331] ? sysfs_kf_write+0x65/0x160
[ 1235.839532] ? sysfs_file_ops+0x140/0x140
[ 1235.844830] kernfs_fop_write_iter+0x335/0x510
[ 1235.850604] ? kernfs_ops+0xf0/0xf0
[ 1235.855316] new_sync_write+0x333/0x750
[ 1235.860421] ? __lock_acquire+0x508/0xc10
[ 1235.865717] ? new_sync_read+0x740/0x740
[ 1235.870920] ? lock_acquire.part.0+0xbc/0x260
[ 1235.876607] ? ksys_write+0x116/0x250
[ 1235.881518] vfs_write+0x657/0x9b0
[ 1235.886136] ksys_write+0x116/0x250
[ 1235.890850] ? __ia32_sys_read+0xc0/0xc0
[ 1235.896051] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.901642] ? rcu_is_watching+0x15/0xd0
[ 1235.906841] do_syscall_64+0xf3/0x650
[ 1235.911750] ? trace_hardirqs_off+0xd/0x30
[ 1235.917145] ? exc_page_fault+0xda/0xf0
[ 1235.922247] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.928715] RIP: 0033:0x7fa272e9422e
[ 1235.933526] RSP: 002b:00007ffe1ebe1bf0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.942825] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa272e9422e
[ 1235.951638] RDX: 0000000000000001 RSI: 0000562a54b26cd0 RDI: 0000000000000007
[ 1235.960448] RBP: 00007ffe1ebe1c00 R08: 0000000000000000 R09: 0000000000000000
[ 1235.969258] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa272d0c390
[ 1235.978067] R13: 0000562a54b26cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.986885]
[ 1235.990156] \x0aShowing all locks held in the system:
[ 1235.998697] 1 lock held by khungtaskd/185:
[ 1236.004080] #0: ffffffffb16fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.015566] 4 locks held by python3/3821:
[ 1236.020863] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.030754] #1: ff11000172b9fa80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.041716] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.052863] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.062960] 4 locks held by python3/5853:
[ 1236.068256] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.078146] #1: ff1100020d7c2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.089107] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.100256] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.110340]
[ 1236.112809] =============================================\x0a
[ 1267.173914] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0
[ 1358.312741] INFO: task python3:5853 blocked for more than 491 seconds.
[ 1358.320887] Not tainted 7.1.0-rc3-nfvp-g0c01f800133a #1
[ 1358.328138] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.337729] task:python3 state:D stack:25672 pid:5853 tgid:5853 ppid:5852 task_flags:0x400100 flags:0x00080002
[ 1358.350934] Call Trace:
[ 1358.354474]
[ 1358.357621] __schedule+0xcc3/0x19c0
[ 1358.362435] ? io_schedule_timeout+0x140/0x140
[ 1358.368219] ? find_held_lock+0x2b/0x80
[ 1358.373321] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.379206] schedule+0xd4/0x260
[ 1358.383623] schedule_preempt_disabled+0x18/0x30
[ 1358.389600] __mutex_lock+0x6a3/0x2000
[ 1358.394605] ? threaded_store+0x132/0x220
[ 1358.399904] ? ww_mutex_lock+0x160/0x160
[ 1358.405100] ? selinux_inode_free_security+0x330/0x330
[ 1358.411669] ? cap_capable+0x1d6/0x3e0
[ 1358.416674] ? threaded_store+0x132/0x220
[ 1358.421967] threaded_store+0x132/0x220
[ 1358.427067] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.432367] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.437665] ? sysfs_kf_write+0x65/0x160
[ 1358.442863] ? sysfs_file_ops+0x140/0x140
[ 1358.448156] kernfs_fop_write_iter+0x335/0x510
[ 1358.453939] ? kernfs_ops+0xf0/0xf0
[ 1358.458647] new_sync_write+0x333/0x750
[ 1358.463749] ? __lock_acquire+0x508/0xc10
[ 1358.469045] ? new_sync_read+0x740/0x740
[ 1358.474243] ? lock_acquire.part.0+0xbc/0x260
[ 1358.479930] ? ksys_write+0x116/0x250
[ 1358.484832] vfs_write+0x657/0x9b0
[ 1358.489445] ksys_write+0x116/0x250
[ 1358.494153] ? __ia32_sys_read+0xc0/0xc0
[ 1358.499351] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.504939] ? rcu_is_watching+0x15/0xd0
[ 1358.510141] do_syscall_64+0xf3/0x650
[ 1358.515047] ? trace_hardirqs_off+0xd/0x30
[ 1358.520439] ? exc_page_fault+0xda/0xf0
[ 1358.525538] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.532005] RIP: 0033:0x7f9a9f66f22e
[ 1358.536813] RSP: 002b:00007ffffde13b50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.546108] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f9a9f66f22e
[ 1358.554915] RDX: 0000000000000001 RSI: 000056054e18dcd0 RDI: 0000000000000007
[ 1358.563722] RBP: 00007ffffde13b60 R08: 0000000000000000 R09: 0000000000000000
[ 1358.572527] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f9a9f4e7390
[ 1358.581335] R13: 000056054e18dcd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.590147]
[ 1358.593444] INFO: task python3:5853 is blocked on a mutex likely owned by task python3:3821.
[ 1358.603717] task:python3 state:D stack:25672 pid:3821 tgid:3821 ppid:3820 task_flags:0x400100 flags:0x00080802
[ 1358.616918] Call Trace:
[ 1358.620457]
[ 1358.623605] __schedule+0xcc3/0x19c0
[ 1358.628415] ? io_schedule_timeout+0x140/0x140
[ 1358.634195] ? find_held_lock+0x2b/0x80
[ 1358.639296] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.645473] schedule+0xd4/0x260
[ 1358.649892] schedule_timeout+0x13f/0x290
[ 1358.655190] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.661551] ? trace_contention_end+0xb3/0x1a0
[ 1358.667326] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.673015] ? ww_mutex_lock+0x160/0x160
[ 1358.678213] msleep+0x1b/0x30
[ 1358.682337] napi_set_threaded+0x3a5/0x440
[ 1358.687731] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.693223] netif_set_threaded+0x257/0x400
[ 1358.698713] threaded_store+0x1b8/0x220
[ 1358.703815] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.709111] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.714404] ? sysfs_kf_write+0x65/0x160
[ 1358.719602] ? sysfs_file_ops+0x140/0x140
[ 1358.724896] kernfs_fop_write_iter+0x335/0x510
[ 1358.730681] ? kernfs_ops+0xf0/0xf0
[ 1358.735393] new_sync_write+0x333/0x750
[ 1358.740493] ? __lock_acquire+0x508/0xc10
[ 1358.745789] ? new_sync_read+0x740/0x740
[ 1358.750988] ? lock_acquire.part.0+0xbc/0x260
[ 1358.756672] ? ksys_write+0x116/0x250
[ 1358.761588] vfs_write+0x657/0x9b0
[ 1358.766203] ksys_write+0x116/0x250
[ 1358.770905] ? __ia32_sys_read+0xc0/0xc0
[ 1358.776105] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.781684] ? rcu_is_watching+0x15/0xd0
[ 1358.786882] do_syscall_64+0xf3/0x650
[ 1358.791784] ? trace_hardirqs_off+0xd/0x30
[ 1358.797176] ? exc_page_fault+0xda/0xf0
[ 1358.802276] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.808743] RIP: 0033:0x7fa272e9422e
[ 1358.813550] RSP: 002b:00007ffe1ebe1bf0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.822847] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa272e9422e
[ 1358.831651] RDX: 0000000000000001 RSI: 0000562a54b26cd0 RDI: 0000000000000007
[ 1358.840456] RBP: 00007ffe1ebe1c00 R08: 0000000000000000 R09: 0000000000000000
[ 1358.849261] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa272d0c390
[ 1358.858069] R13: 0000562a54b26cd0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.866882]
[ 1358.870151] \x0aShowing all locks held in the system:
[ 1358.878693] 1 lock held by khungtaskd/185:
[ 1358.884078] #0: ffffffffb16fe060 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.895561] 4 locks held by python3/3821:
[ 1358.900847] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.910738] #1: ff11000172b9fa80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.921698] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.932855] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.942944] 4 locks held by python3/5853:
[ 1358.948238] #0: ff1100011ed243e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.958126] #1: ff1100020d7c2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.969086] #2: ff11000123a82b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.980242] #3: ff11000123ac9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.990336]
[ 1358.992798] =============================================\x0a