[ 781.627151] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 781.630712] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 781.757957] kselftest: Running tests in drivers/net
[ 989.678022] INFO: task python3:5772 blocked for more than 122 seconds.
[ 989.686248] Not tainted 7.1.0-rc2-tboi-g442817b5d4df #1
[ 989.693498] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.703719] task:python3 state:D stack:25032 pid:5772 tgid:5772 ppid:5771 task_flags:0x400100 flags:0x00080000
[ 989.716917] Call Trace:
[ 989.720455]
[ 989.723607] __schedule+0xcba/0x1940
[ 989.728428] ? io_schedule_timeout+0x140/0x140
[ 989.734215] ? find_held_lock+0x2b/0x80
[ 989.739321] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.745206] schedule+0xd4/0x260
[ 989.749627] schedule_preempt_disabled+0x18/0x30
[ 989.755605] __mutex_lock+0x6a3/0x2000
[ 989.760612] ? threaded_store+0x132/0x220
[ 989.765911] ? ww_mutex_lock+0x160/0x160
[ 989.771109] ? selinux_inode_free_security+0x330/0x330
[ 989.777681] ? cap_capable+0x1d6/0x3e0
[ 989.782691] ? threaded_store+0x132/0x220
[ 989.787989] threaded_store+0x132/0x220
[ 989.793084] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.798379] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.803680] ? sysfs_kf_write+0x65/0x160
[ 989.808880] ? sysfs_file_ops+0x140/0x140
[ 989.814174] kernfs_fop_write_iter+0x335/0x510
[ 989.819960] ? kernfs_ops+0xf0/0xf0
[ 989.824672] new_sync_write+0x333/0x750
[ 989.829776] ? __lock_acquire+0x508/0xc10
[ 989.835073] ? new_sync_read+0x740/0x740
[ 989.840272] ? lock_acquire.part.0+0xbc/0x260
[ 989.845960] ? ksys_write+0x116/0x250
[ 989.850863] vfs_write+0x657/0x9b0
[ 989.855477] ksys_write+0x116/0x250
[ 989.860191] ? __ia32_sys_read+0xc0/0xc0
[ 989.865389] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.870983] ? rcu_is_watching+0x15/0xd0
[ 989.876183] do_syscall_64+0xf3/0x650
[ 989.881083] ? trace_hardirqs_off+0xd/0x30
[ 989.886477] ? exc_page_fault+0xda/0xf0
[ 989.891581] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.898058] RIP: 0033:0x7fcfa9e1d22e
[ 989.902871] RSP: 002b:00007ffdce58b570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.912167] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfa9e1d22e
[ 989.920969] RDX: 0000000000000001 RSI: 0000560f0b975fe0 RDI: 0000000000000007
[ 989.929779] RBP: 00007ffdce58b580 R08: 0000000000000000 R09: 0000000000000000
[ 989.938591] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfa9c95390
[ 989.947400] R13: 0000560f0b975fe0 R14: 0000000000000007 R15: 0000000000000003
[ 989.956215]
[ 989.959516] INFO: task python3:5772 is blocked on a mutex likely owned by task python3:3688.
[ 989.969790] task:python3 state:D stack:25672 pid:3688 tgid:3688 ppid:3687 task_flags:0x400100 flags:0x00080802
[ 989.982994] Call Trace:
[ 989.986531]
[ 989.989685] __schedule+0xcba/0x1940
[ 989.994496] ? io_schedule_timeout+0x140/0x140
[ 990.000283] ? find_held_lock+0x2b/0x80
[ 990.005388] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.011564] schedule+0xd4/0x260
[ 990.015983] schedule_timeout+0x13f/0x290
[ 990.021281] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.027649] ? trace_contention_end+0xb3/0x1a0
[ 990.033437] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.039125] ? ww_mutex_lock+0x160/0x160
[ 990.044326] msleep+0x1b/0x30
[ 990.048452] napi_set_threaded+0x3a5/0x440
[ 990.053842] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.059339] netif_set_threaded+0x257/0x400
[ 990.064833] threaded_store+0x1b8/0x220
[ 990.069935] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.075232] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.080522] ? sysfs_kf_write+0x65/0x160
[ 990.085723] ? sysfs_file_ops+0x140/0x140
[ 990.091019] kernfs_fop_write_iter+0x335/0x510
[ 990.096803] ? kernfs_ops+0xf0/0xf0
[ 990.101512] new_sync_write+0x333/0x750
[ 990.106614] ? __lock_acquire+0x508/0xc10
[ 990.111909] ? new_sync_read+0x740/0x740
[ 990.117112] ? lock_acquire.part.0+0xbc/0x260
[ 990.122799] ? ksys_write+0x116/0x250
[ 990.127708] vfs_write+0x657/0x9b0
[ 990.132323] ksys_write+0x116/0x250
[ 990.137039] ? __ia32_sys_read+0xc0/0xc0
[ 990.142238] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.147829] ? rcu_is_watching+0x15/0xd0
[ 990.153022] do_syscall_64+0xf3/0x650
[ 990.157928] ? trace_hardirqs_off+0xd/0x30
[ 990.163323] ? exc_page_fault+0xda/0xf0
[ 990.168425] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.174893] RIP: 0033:0x7f678c0da22e
[ 990.179701] RSP: 002b:00007fff7dbb6c60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.188997] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f678c0da22e
[ 990.197807] RDX: 0000000000000001 RSI: 000056429dff7fe0 RDI: 0000000000000007
[ 990.206617] RBP: 00007fff7dbb6c70 R08: 0000000000000000 R09: 0000000000000000
[ 990.215426] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f678bf52390
[ 990.224235] R13: 000056429dff7fe0 R14: 0000000000000007 R15: 0000000000000003
[ 990.233047]
[ 990.236306] \x0aShowing all locks held in the system:
[ 990.244848] 1 lock held by khungtaskd/185:
[ 990.250242] #0: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.261730] 4 locks held by python3/3688:
[ 990.267026] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.276918] #1: ff1100012e6d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.287878] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.299037] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.309140] 4 locks held by python3/5772:
[ 990.314436] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.324327] #1: ff1100019356ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.335288] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.346444] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.356535]
[ 990.359002] =============================================\x0a
[ 1112.559271] INFO: task python3:5772 blocked for more than 245 seconds.
[ 1112.567420] Not tainted 7.1.0-rc2-tboi-g442817b5d4df #1
[ 1112.574669] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.584260] task:python3 state:D stack:25032 pid:5772 tgid:5772 ppid:5771 task_flags:0x400100 flags:0x00080000
[ 1112.597465] Call Trace:
[ 1112.601005]
[ 1112.604153] __schedule+0xcba/0x1940
[ 1112.608969] ? io_schedule_timeout+0x140/0x140
[ 1112.614753] ? find_held_lock+0x2b/0x80
[ 1112.619859] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.625741] schedule+0xd4/0x260
[ 1112.630159] schedule_preempt_disabled+0x18/0x30
[ 1112.636137] __mutex_lock+0x6a3/0x2000
[ 1112.641141] ? threaded_store+0x132/0x220
[ 1112.646431] ? ww_mutex_lock+0x160/0x160
[ 1112.651620] ? selinux_inode_free_security+0x330/0x330
[ 1112.658187] ? cap_capable+0x1d6/0x3e0
[ 1112.663194] ? threaded_store+0x132/0x220
[ 1112.668489] threaded_store+0x132/0x220
[ 1112.673587] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.678884] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.684185] ? sysfs_kf_write+0x65/0x160
[ 1112.689382] ? sysfs_file_ops+0x140/0x140
[ 1112.694678] kernfs_fop_write_iter+0x335/0x510
[ 1112.700463] ? kernfs_ops+0xf0/0xf0
[ 1112.705173] new_sync_write+0x333/0x750
[ 1112.710274] ? __lock_acquire+0x508/0xc10
[ 1112.715568] ? new_sync_read+0x740/0x740
[ 1112.720768] ? lock_acquire.part.0+0xbc/0x260
[ 1112.726452] ? ksys_write+0x116/0x250
[ 1112.731361] vfs_write+0x657/0x9b0
[ 1112.735973] ksys_write+0x116/0x250
[ 1112.740683] ? __ia32_sys_read+0xc0/0xc0
[ 1112.745881] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.751470] ? rcu_is_watching+0x15/0xd0
[ 1112.756670] do_syscall_64+0xf3/0x650
[ 1112.761576] ? trace_hardirqs_off+0xd/0x30
[ 1112.766969] ? exc_page_fault+0xda/0xf0
[ 1112.772069] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.778534] RIP: 0033:0x7fcfa9e1d22e
[ 1112.783343] RSP: 002b:00007ffdce58b570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.792640] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfa9e1d22e
[ 1112.801449] RDX: 0000000000000001 RSI: 0000560f0b975fe0 RDI: 0000000000000007
[ 1112.810258] RBP: 00007ffdce58b580 R08: 0000000000000000 R09: 0000000000000000
[ 1112.819067] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfa9c95390
[ 1112.827876] R13: 0000560f0b975fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.836687]
[ 1112.839965] INFO: task python3:5772 is blocked on a mutex likely owned by task python3:3688.
[ 1112.850236] task:python3 state:D stack:25672 pid:3688 tgid:3688 ppid:3687 task_flags:0x400100 flags:0x00080802
[ 1112.863437] Call Trace:
[ 1112.866975]
[ 1112.870124] __schedule+0xcba/0x1940
[ 1112.874935] ? io_schedule_timeout+0x140/0x140
[ 1112.880719] ? find_held_lock+0x2b/0x80
[ 1112.885822] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.891998] schedule+0xd4/0x260
[ 1112.896407] schedule_timeout+0x13f/0x290
[ 1112.901705] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.908074] ? trace_contention_end+0xb3/0x1a0
[ 1112.913856] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.919546] ? ww_mutex_lock+0x160/0x160
[ 1112.924745] msleep+0x1b/0x30
[ 1112.928871] napi_set_threaded+0x3a5/0x440
[ 1112.934265] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.939759] netif_set_threaded+0x257/0x400
[ 1112.945250] threaded_store+0x1b8/0x220
[ 1112.950353] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.955650] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.960944] ? sysfs_kf_write+0x65/0x160
[ 1112.966144] ? sysfs_file_ops+0x140/0x140
[ 1112.971427] kernfs_fop_write_iter+0x335/0x510
[ 1112.977210] ? kernfs_ops+0xf0/0xf0
[ 1112.981919] new_sync_write+0x333/0x750
[ 1112.987020] ? __lock_acquire+0x508/0xc10
[ 1112.992316] ? new_sync_read+0x740/0x740
[ 1112.997516] ? lock_acquire.part.0+0xbc/0x260
[ 1113.003203] ? ksys_write+0x116/0x250
[ 1113.008117] vfs_write+0x657/0x9b0
[ 1113.012729] ksys_write+0x116/0x250
[ 1113.017441] ? __ia32_sys_read+0xc0/0xc0
[ 1113.022637] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.028226] ? rcu_is_watching+0x15/0xd0
[ 1113.033416] do_syscall_64+0xf3/0x650
[ 1113.038323] ? trace_hardirqs_off+0xd/0x30
[ 1113.043718] ? exc_page_fault+0xda/0xf0
[ 1113.048821] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.055276] RIP: 0033:0x7f678c0da22e
[ 1113.060082] RSP: 002b:00007fff7dbb6c60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.069381] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f678c0da22e
[ 1113.078194] RDX: 0000000000000001 RSI: 000056429dff7fe0 RDI: 0000000000000007
[ 1113.087001] RBP: 00007fff7dbb6c70 R08: 0000000000000000 R09: 0000000000000000
[ 1113.095806] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f678bf52390
[ 1113.104611] R13: 000056429dff7fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.113422]
[ 1113.116686] \x0aShowing all locks held in the system:
[ 1113.125225] 1 lock held by khungtaskd/185:
[ 1113.130618] #0: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.142074] 4 locks held by kworker/15:1/212:
[ 1113.147757] #0: ff1100180e1b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.157645] #1: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: __update_idle_core+0x5a/0x700
[ 1113.168315] #2: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: unwind_next_frame+0x17e/0x1ea0
[ 1113.179081] #3: ff11000188f449a8 (&s->s_inode_wblist_lock){-.-.}-{3:3}, at: sb_clear_inode_writeback+0x7f/0x390
[ 1113.191336] 2 locks held by systemd-journal/802:
[ 1113.197315] #0: ff1100180e8b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.207203] #1: ffffffff8ed032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1113.219448] 4 locks held by python3/3688:
[ 1113.224744] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.234636] #1: ff1100012e6d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.245597] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.256753] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.266857] 4 locks held by python3/5772:
[ 1113.272150] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.282038] #1: ff1100019356ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.292996] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.304150] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.314239]
[ 1113.316705] =============================================\x0a
[ 1235.439526] INFO: task python3:5772 blocked for more than 368 seconds.
[ 1235.447677] Not tainted 7.1.0-rc2-tboi-g442817b5d4df #1
[ 1235.454926] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.464519] task:python3 state:D stack:25032 pid:5772 tgid:5772 ppid:5771 task_flags:0x400100 flags:0x00080002
[ 1235.477728] Call Trace:
[ 1235.481268]
[ 1235.484419] __schedule+0xcba/0x1940
[ 1235.489238] ? io_schedule_timeout+0x140/0x140
[ 1235.495023] ? find_held_lock+0x2b/0x80
[ 1235.500128] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.506017] schedule+0xd4/0x260
[ 1235.510436] schedule_preempt_disabled+0x18/0x30
[ 1235.516416] __mutex_lock+0x6a3/0x2000
[ 1235.521414] ? threaded_store+0x132/0x220
[ 1235.526716] ? ww_mutex_lock+0x160/0x160
[ 1235.531915] ? selinux_inode_free_security+0x330/0x330
[ 1235.538475] ? cap_capable+0x1d6/0x3e0
[ 1235.543489] ? threaded_store+0x132/0x220
[ 1235.548787] threaded_store+0x132/0x220
[ 1235.553893] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.559188] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.564477] ? sysfs_kf_write+0x65/0x160
[ 1235.569678] ? sysfs_file_ops+0x140/0x140
[ 1235.574973] kernfs_fop_write_iter+0x335/0x510
[ 1235.580759] ? kernfs_ops+0xf0/0xf0
[ 1235.585473] new_sync_write+0x333/0x750
[ 1235.590579] ? __lock_acquire+0x508/0xc10
[ 1235.595874] ? new_sync_read+0x740/0x740
[ 1235.601076] ? lock_acquire.part.0+0xbc/0x260
[ 1235.606762] ? ksys_write+0x116/0x250
[ 1235.611673] vfs_write+0x657/0x9b0
[ 1235.616288] ksys_write+0x116/0x250
[ 1235.620999] ? __ia32_sys_read+0xc0/0xc0
[ 1235.626198] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.631787] ? rcu_is_watching+0x15/0xd0
[ 1235.636993] do_syscall_64+0xf3/0x650
[ 1235.641901] ? trace_hardirqs_off+0xd/0x30
[ 1235.647294] ? exc_page_fault+0xda/0xf0
[ 1235.652398] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.658866] RIP: 0033:0x7fcfa9e1d22e
[ 1235.663678] RSP: 002b:00007ffdce58b570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.672976] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfa9e1d22e
[ 1235.681784] RDX: 0000000000000001 RSI: 0000560f0b975fe0 RDI: 0000000000000007
[ 1235.690592] RBP: 00007ffdce58b580 R08: 0000000000000000 R09: 0000000000000000
[ 1235.699402] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfa9c95390
[ 1235.708210] R13: 0000560f0b975fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.717015]
[ 1235.720294] INFO: task python3:5772 is blocked on a mutex likely owned by task python3:3688.
[ 1235.730570] task:python3 state:D stack:25672 pid:3688 tgid:3688 ppid:3687 task_flags:0x400100 flags:0x00080802
[ 1235.743776] Call Trace:
[ 1235.747314]
[ 1235.750465] __schedule+0xcba/0x1940
[ 1235.755277] ? io_schedule_timeout+0x140/0x140
[ 1235.761060] ? find_held_lock+0x2b/0x80
[ 1235.766163] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.772342] schedule+0xd4/0x260
[ 1235.776762] schedule_timeout+0x13f/0x290
[ 1235.782052] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.788421] ? trace_contention_end+0xb3/0x1a0
[ 1235.794209] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.799898] ? ww_mutex_lock+0x160/0x160
[ 1235.805098] msleep+0x1b/0x30
[ 1235.809216] napi_set_threaded+0x3a5/0x440
[ 1235.814603] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.820090] netif_set_threaded+0x257/0x400
[ 1235.825586] threaded_store+0x1b8/0x220
[ 1235.830690] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.835987] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.841284] ? sysfs_kf_write+0x65/0x160
[ 1235.846483] ? sysfs_file_ops+0x140/0x140
[ 1235.851780] kernfs_fop_write_iter+0x335/0x510
[ 1235.857566] ? kernfs_ops+0xf0/0xf0
[ 1235.862280] new_sync_write+0x333/0x750
[ 1235.867383] ? __lock_acquire+0x508/0xc10
[ 1235.872670] ? new_sync_read+0x740/0x740
[ 1235.877873] ? lock_acquire.part.0+0xbc/0x260
[ 1235.883557] ? ksys_write+0x116/0x250
[ 1235.888468] vfs_write+0x657/0x9b0
[ 1235.893085] ksys_write+0x116/0x250
[ 1235.897798] ? __ia32_sys_read+0xc0/0xc0
[ 1235.902988] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.908579] ? rcu_is_watching+0x15/0xd0
[ 1235.913781] do_syscall_64+0xf3/0x650
[ 1235.918689] ? trace_hardirqs_off+0xd/0x30
[ 1235.924073] ? exc_page_fault+0xda/0xf0
[ 1235.929176] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.935644] RIP: 0033:0x7f678c0da22e
[ 1235.940453] RSP: 002b:00007fff7dbb6c60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.949751] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f678c0da22e
[ 1235.958560] RDX: 0000000000000001 RSI: 000056429dff7fe0 RDI: 0000000000000007
[ 1235.967371] RBP: 00007fff7dbb6c70 R08: 0000000000000000 R09: 0000000000000000
[ 1235.976182] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f678bf52390
[ 1235.984989] R13: 000056429dff7fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.993794]
[ 1235.997061] \x0aShowing all locks held in the system:
[ 1236.005597] 1 lock held by khungtaskd/185:
[ 1236.010992] #0: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.022456] 3 locks held by systemd-journal/802:
[ 1236.028434] #0: ff1100180e2b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1236.038321] #1: ffffffff8ed032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.051140] #2: ff110001bb762440 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.062699] 4 locks held by python3/3688:
[ 1236.067992] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.077880] #1: ff1100012e6d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.088842] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.099992] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.110088] 4 locks held by python3/5772:
[ 1236.115382] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.125271] #1: ff1100019356ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.136233] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.147391] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.157481]
[ 1236.159947] =============================================\x0a
[ 1358.320781] INFO: task python3:5772 blocked for more than 491 seconds.
[ 1358.328929] Not tainted 7.1.0-rc2-tboi-g442817b5d4df #1
[ 1358.336170] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.345759] task:python3 state:D stack:25032 pid:5772 tgid:5772 ppid:5771 task_flags:0x400100 flags:0x00080002
[ 1358.358965] Call Trace:
[ 1358.362506]
[ 1358.365660] __schedule+0xcba/0x1940
[ 1358.370477] ? io_schedule_timeout+0x140/0x140
[ 1358.376262] ? find_held_lock+0x2b/0x80
[ 1358.381366] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.387255] schedule+0xd4/0x260
[ 1358.391671] schedule_preempt_disabled+0x18/0x30
[ 1358.397652] __mutex_lock+0x6a3/0x2000
[ 1358.402657] ? threaded_store+0x132/0x220
[ 1358.407959] ? ww_mutex_lock+0x160/0x160
[ 1358.413160] ? selinux_inode_free_security+0x330/0x330
[ 1358.419730] ? cap_capable+0x1d6/0x3e0
[ 1358.424747] ? threaded_store+0x132/0x220
[ 1358.430053] threaded_store+0x132/0x220
[ 1358.435156] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.440452] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.445742] ? sysfs_kf_write+0x65/0x160
[ 1358.450941] ? sysfs_file_ops+0x140/0x140
[ 1358.456236] kernfs_fop_write_iter+0x335/0x510
[ 1358.462023] ? kernfs_ops+0xf0/0xf0
[ 1358.466733] new_sync_write+0x333/0x750
[ 1358.471836] ? __lock_acquire+0x508/0xc10
[ 1358.477134] ? new_sync_read+0x740/0x740
[ 1358.482335] ? lock_acquire.part.0+0xbc/0x260
[ 1358.488021] ? ksys_write+0x116/0x250
[ 1358.492933] vfs_write+0x657/0x9b0
[ 1358.497551] ksys_write+0x116/0x250
[ 1358.502264] ? __ia32_sys_read+0xc0/0xc0
[ 1358.507464] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.513044] ? rcu_is_watching+0x15/0xd0
[ 1358.518238] do_syscall_64+0xf3/0x650
[ 1358.523145] ? trace_hardirqs_off+0xd/0x30
[ 1358.528540] ? exc_page_fault+0xda/0xf0
[ 1358.533645] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.540114] RIP: 0033:0x7fcfa9e1d22e
[ 1358.544924] RSP: 002b:00007ffdce58b570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.554222] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcfa9e1d22e
[ 1358.563030] RDX: 0000000000000001 RSI: 0000560f0b975fe0 RDI: 0000000000000007
[ 1358.571839] RBP: 00007ffdce58b580 R08: 0000000000000000 R09: 0000000000000000
[ 1358.580647] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fcfa9c95390
[ 1358.589455] R13: 0000560f0b975fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.598269]
[ 1358.601552] INFO: task python3:5772 is blocked on a mutex likely owned by task python3:3688.
[ 1358.611826] task:python3 state:D stack:25672 pid:3688 tgid:3688 ppid:3687 task_flags:0x400100 flags:0x00080802
[ 1358.625029] Call Trace:
[ 1358.628566]
[ 1358.631717] __schedule+0xcba/0x1940
[ 1358.636520] ? io_schedule_timeout+0x140/0x140
[ 1358.642303] ? find_held_lock+0x2b/0x80
[ 1358.647406] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.653585] schedule+0xd4/0x260
[ 1358.658004] schedule_timeout+0x13f/0x290
[ 1358.663300] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.669669] ? trace_contention_end+0xb3/0x1a0
[ 1358.675455] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.681146] ? ww_mutex_lock+0x160/0x160
[ 1358.686346] msleep+0x1b/0x30
[ 1358.690471] napi_set_threaded+0x3a5/0x440
[ 1358.695869] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.701366] netif_set_threaded+0x257/0x400
[ 1358.706858] threaded_store+0x1b8/0x220
[ 1358.711961] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.717258] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.722555] ? sysfs_kf_write+0x65/0x160
[ 1358.727746] ? sysfs_file_ops+0x140/0x140
[ 1358.733043] kernfs_fop_write_iter+0x335/0x510
[ 1358.738829] ? kernfs_ops+0xf0/0xf0
[ 1358.743541] new_sync_write+0x333/0x750
[ 1358.748637] ? __lock_acquire+0x508/0xc10
[ 1358.753932] ? new_sync_read+0x740/0x740
[ 1358.759131] ? lock_acquire.part.0+0xbc/0x260
[ 1358.764817] ? ksys_write+0x116/0x250
[ 1358.769725] vfs_write+0x657/0x9b0
[ 1358.774341] ksys_write+0x116/0x250
[ 1358.779053] ? __ia32_sys_read+0xc0/0xc0
[ 1358.784254] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.789842] ? rcu_is_watching+0x15/0xd0
[ 1358.795041] do_syscall_64+0xf3/0x650
[ 1358.799948] ? trace_hardirqs_off+0xd/0x30
[ 1358.805341] ? exc_page_fault+0xda/0xf0
[ 1358.810441] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.816910] RIP: 0033:0x7f678c0da22e
[ 1358.821716] RSP: 002b:00007fff7dbb6c60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.831012] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f678c0da22e
[ 1358.839821] RDX: 0000000000000001 RSI: 000056429dff7fe0 RDI: 0000000000000007
[ 1358.848632] RBP: 00007fff7dbb6c70 R08: 0000000000000000 R09: 0000000000000000
[ 1358.857442] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f678bf52390
[ 1358.866250] R13: 000056429dff7fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.875063]
[ 1358.878330] \x0aShowing all locks held in the system:
[ 1358.886867] 1 lock held by khungtaskd/185:
[ 1358.892260] #0: ffffffff8eafdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.903730] 3 locks held by systemd-journal/802:
[ 1358.909709] #0: ff1100180db372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1358.919598] #1: ffffffff8ed032a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.932415] #2: ff1100020a461410 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.943972] 4 locks held by python3/3688:
[ 1358.949266] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.959153] #1: ff1100012e6d7a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.970114] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.981273] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.991369] 4 locks held by python3/5772:
[ 1358.996661] #0: ff110017d9d8c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.006549] #1: ff1100019356ac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.017511] #2: ff1100012328cb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.028670] #3: ff110001233a1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.038761]
[ 1359.041227] =============================================\x0a