[ 789.277070] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 789.279473] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 789.431591] kselftest: Running tests in drivers/net
[ 989.698238] INFO: task python3:5816 blocked for more than 122 seconds.
[ 989.706477] Not tainted 7.1.0-rc4-yvvk-gc6fc3bdd5ba0 #1
[ 989.714430] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.724018] task:python3 state:D stack:25672 pid:5816 tgid:5816 ppid:5815 task_flags:0x400100 flags:0x00080000
[ 989.737225] Call Trace:
[ 989.740763]
[ 989.743912] __schedule+0xcc3/0x19c0
[ 989.748731] ? io_schedule_timeout+0x140/0x140
[ 989.754514] ? find_held_lock+0x2b/0x80
[ 989.759611] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.765495] schedule+0xd8/0x260
[ 989.769911] schedule_preempt_disabled+0x18/0x30
[ 989.775890] __mutex_lock+0x6af/0x2010
[ 989.780893] ? threaded_store+0x132/0x220
[ 989.786184] ? ww_mutex_lock+0x160/0x160
[ 989.791382] ? selinux_inode_free_security+0x330/0x330
[ 989.797942] ? cap_capable+0x1d6/0x3e0
[ 989.802951] ? threaded_store+0x132/0x220
[ 989.808246] threaded_store+0x132/0x220
[ 989.813346] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.818642] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.823939] ? sysfs_kf_write+0x65/0x160
[ 989.829138] ? sysfs_file_ops+0x140/0x140
[ 989.834434] kernfs_fop_write_iter+0x335/0x510
[ 989.840216] ? kernfs_ops+0xf0/0xf0
[ 989.844926] new_sync_write+0x333/0x750
[ 989.850028] ? __lock_acquire+0x508/0xc10
[ 989.855322] ? new_sync_read+0x740/0x740
[ 989.860523] ? lock_acquire.part.0+0xbc/0x260
[ 989.866210] ? ksys_write+0x116/0x250
[ 989.871109] vfs_write+0x657/0x9b0
[ 989.875725] ksys_write+0x116/0x250
[ 989.880434] ? __ia32_sys_read+0xc0/0xc0
[ 989.885631] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.891218] ? rcu_is_watching+0x15/0xd0
[ 989.896421] do_syscall_64+0xf3/0x650
[ 989.901328] ? trace_hardirqs_off+0xd/0x30
[ 989.906720] ? exc_page_fault+0xda/0xf0
[ 989.911823] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.918289] RIP: 0033:0x7f07b773822e
[ 989.923100] RSP: 002b:00007ffcc5030d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.932399] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07b773822e
[ 989.941205] RDX: 0000000000000001 RSI: 000055e31c8a6a90 RDI: 0000000000000007
[ 989.950010] RBP: 00007ffcc5030d40 R08: 0000000000000000 R09: 0000000000000000
[ 989.958816] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07b75b0390
[ 989.967623] R13: 000055e31c8a6a90 R14: 0000000000000007 R15: 0000000000000003
[ 989.976442]
[ 989.979754] INFO: task python3:5816 is blocked on a mutex likely owned by task python3:3758.
[ 989.990023] task:python3 state:D stack:25672 pid:3758 tgid:3758 ppid:3757 task_flags:0x400100 flags:0x00080802
[ 990.003227] Call Trace:
[ 990.006765]
[ 990.009912] __schedule+0xcc3/0x19c0
[ 990.014722] ? io_schedule_timeout+0x140/0x140
[ 990.020497] ? find_held_lock+0x2b/0x80
[ 990.025601] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.031779] schedule+0xd8/0x260
[ 990.036198] schedule_timeout+0x13f/0x290
[ 990.041495] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.047864] ? trace_contention_end+0xb3/0x1a0
[ 990.053649] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.059340] ? ww_mutex_lock+0x160/0x160
[ 990.064541] msleep+0x1b/0x30
[ 990.068667] napi_set_threaded+0x3a5/0x440
[ 990.074059] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.079554] netif_set_threaded+0x257/0x400
[ 990.085047] threaded_store+0x1b8/0x220
[ 990.090148] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.095445] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.100741] ? sysfs_kf_write+0x65/0x160
[ 990.105940] ? sysfs_file_ops+0x140/0x140
[ 990.111234] kernfs_fop_write_iter+0x335/0x510
[ 990.117019] ? kernfs_ops+0xf0/0xf0
[ 990.121730] new_sync_write+0x333/0x750
[ 990.126831] ? __lock_acquire+0x508/0xc10
[ 990.132126] ? new_sync_read+0x740/0x740
[ 990.137327] ? lock_acquire.part.0+0xbc/0x260
[ 990.143013] ? ksys_write+0x116/0x250
[ 990.147924] vfs_write+0x657/0x9b0
[ 990.152539] ksys_write+0x116/0x250
[ 990.157247] ? __ia32_sys_read+0xc0/0xc0
[ 990.162446] ? rcu_is_watching+0x15/0xd0
[ 990.167644] ? rcu_is_watching+0x15/0xd0
[ 990.172841] do_syscall_64+0xf3/0x650
[ 990.177745] ? irq_exit_rcu+0x1a/0x30
[ 990.182652] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.189116] RIP: 0033:0x7fa89843022e
[ 990.193924] RSP: 002b:00007ffc2f3494a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.203220] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa89843022e
[ 990.212017] RDX: 0000000000000001 RSI: 000055685f2cca90 RDI: 0000000000000007
[ 990.220823] RBP: 00007ffc2f3494b0 R08: 0000000000000000 R09: 0000000000000000
[ 990.229628] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8982a8390
[ 990.238433] R13: 000055685f2cca90 R14: 0000000000000007 R15: 0000000000000003
[ 990.247247]
[ 990.250504] \x0aShowing all locks held in the system:
[ 990.259042] 1 lock held by khungtaskd/184:
[ 990.264435] #0: ffffffffb82fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.275917] 4 locks held by python3/3758:
[ 990.281213] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.291101] #1: ff1100016e994480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.302063] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.313225] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.323322] 4 locks held by python3/5816:
[ 990.328618] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.338505] #1: ff110001f06adc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.349463] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.360619] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.370710]
[ 990.373175] =============================================\x0a
[ 1112.580236] INFO: task python3:5816 blocked for more than 245 seconds.
[ 1112.588384] Not tainted 7.1.0-rc4-yvvk-gc6fc3bdd5ba0 #1
[ 1112.595636] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.605226] task:python3 state:D stack:25672 pid:5816 tgid:5816 ppid:5815 task_flags:0x400100 flags:0x00080000
[ 1112.618434] Call Trace:
[ 1112.621975]
[ 1112.625129] __schedule+0xcc3/0x19c0
[ 1112.629947] ? io_schedule_timeout+0x140/0x140
[ 1112.635732] ? find_held_lock+0x2b/0x80
[ 1112.640840] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.646724] schedule+0xd8/0x260
[ 1112.651144] schedule_preempt_disabled+0x18/0x30
[ 1112.657126] __mutex_lock+0x6af/0x2010
[ 1112.662132] ? threaded_store+0x132/0x220
[ 1112.667435] ? ww_mutex_lock+0x160/0x160
[ 1112.672637] ? selinux_inode_free_security+0x330/0x330
[ 1112.679206] ? cap_capable+0x1d6/0x3e0
[ 1112.684218] ? threaded_store+0x132/0x220
[ 1112.689516] threaded_store+0x132/0x220
[ 1112.694617] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.699914] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.705214] ? sysfs_kf_write+0x65/0x160
[ 1112.710415] ? sysfs_file_ops+0x140/0x140
[ 1112.715713] kernfs_fop_write_iter+0x335/0x510
[ 1112.721489] ? kernfs_ops+0xf0/0xf0
[ 1112.726191] new_sync_write+0x333/0x750
[ 1112.731295] ? __lock_acquire+0x508/0xc10
[ 1112.736594] ? new_sync_read+0x740/0x740
[ 1112.741795] ? lock_acquire.part.0+0xbc/0x260
[ 1112.747480] ? ksys_write+0x116/0x250
[ 1112.752393] vfs_write+0x657/0x9b0
[ 1112.757011] ksys_write+0x116/0x250
[ 1112.761724] ? __ia32_sys_read+0xc0/0xc0
[ 1112.766925] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.772515] ? rcu_is_watching+0x15/0xd0
[ 1112.777717] do_syscall_64+0xf3/0x650
[ 1112.782625] ? trace_hardirqs_off+0xd/0x30
[ 1112.788021] ? exc_page_fault+0xda/0xf0
[ 1112.793123] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.799583] RIP: 0033:0x7f07b773822e
[ 1112.804393] RSP: 002b:00007ffcc5030d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.813690] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07b773822e
[ 1112.822500] RDX: 0000000000000001 RSI: 000055e31c8a6a90 RDI: 0000000000000007
[ 1112.831307] RBP: 00007ffcc5030d40 R08: 0000000000000000 R09: 0000000000000000
[ 1112.840118] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07b75b0390
[ 1112.848926] R13: 000055e31c8a6a90 R14: 0000000000000007 R15: 0000000000000003
[ 1112.857732]
[ 1112.861022] INFO: task python3:5816 is blocked on a mutex likely owned by task python3:3758.
[ 1112.871297] task:python3 state:D stack:25672 pid:3758 tgid:3758 ppid:3757 task_flags:0x400100 flags:0x00080802
[ 1112.884492] Call Trace:
[ 1112.888030]
[ 1112.891183] __schedule+0xcc3/0x19c0
[ 1112.895994] ? io_schedule_timeout+0x140/0x140
[ 1112.901778] ? find_held_lock+0x2b/0x80
[ 1112.906882] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.913060] schedule+0xd8/0x260
[ 1112.917479] schedule_timeout+0x13f/0x290
[ 1112.922777] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.929138] ? trace_contention_end+0xb3/0x1a0
[ 1112.934924] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.940616] ? ww_mutex_lock+0x160/0x160
[ 1112.945816] msleep+0x1b/0x30
[ 1112.949940] napi_set_threaded+0x3a5/0x440
[ 1112.955338] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.960831] netif_set_threaded+0x257/0x400
[ 1112.966316] threaded_store+0x1b8/0x220
[ 1112.971421] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.976720] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.982015] ? sysfs_kf_write+0x65/0x160
[ 1112.987214] ? sysfs_file_ops+0x140/0x140
[ 1112.992509] kernfs_fop_write_iter+0x335/0x510
[ 1112.998295] ? kernfs_ops+0xf0/0xf0
[ 1113.003006] new_sync_write+0x333/0x750
[ 1113.008110] ? __lock_acquire+0x508/0xc10
[ 1113.013406] ? new_sync_read+0x740/0x740
[ 1113.018605] ? lock_acquire.part.0+0xbc/0x260
[ 1113.024293] ? ksys_write+0x116/0x250
[ 1113.029203] vfs_write+0x657/0x9b0
[ 1113.033820] ksys_write+0x116/0x250
[ 1113.038533] ? __ia32_sys_read+0xc0/0xc0
[ 1113.043735] ? rcu_is_watching+0x15/0xd0
[ 1113.048935] ? rcu_is_watching+0x15/0xd0
[ 1113.054127] do_syscall_64+0xf3/0x650
[ 1113.059036] ? irq_exit_rcu+0x1a/0x30
[ 1113.063942] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.070410] RIP: 0033:0x7fa89843022e
[ 1113.075219] RSP: 002b:00007ffc2f3494a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.084516] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa89843022e
[ 1113.093324] RDX: 0000000000000001 RSI: 000055685f2cca90 RDI: 0000000000000007
[ 1113.102125] RBP: 00007ffc2f3494b0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.110935] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8982a8390
[ 1113.119743] R13: 000055685f2cca90 R14: 0000000000000007 R15: 0000000000000003
[ 1113.128557]
[ 1113.131821] \x0aShowing all locks held in the system:
[ 1113.140361] 1 lock held by khungtaskd/184:
[ 1113.145757] #0: ffffffffb82fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.157229] 2 locks held by systemd-journal/802:
[ 1113.163209] #0: ff1100180e4372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.173098] #1: ffffffffb85029a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.185925] 4 locks held by python3/3758:
[ 1113.191223] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.201111] #1: ff1100016e994480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.212079] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.223237] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.233330] 4 locks held by python3/5816:
[ 1113.238627] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.248515] #1: ff110001f06adc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.259476] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.270624] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.280716]
[ 1113.283183] =============================================\x0a
[ 1210.884043] kworker/u129:1 (185) used greatest stack depth: 22176 bytes left
[ 1235.461267] INFO: task python3:5816 blocked for more than 368 seconds.
[ 1235.469408] Not tainted 7.1.0-rc4-yvvk-gc6fc3bdd5ba0 #1
[ 1235.476658] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.486251] task:python3 state:D stack:25672 pid:5816 tgid:5816 ppid:5815 task_flags:0x400100 flags:0x00080002
[ 1235.499459] Call Trace:
[ 1235.503000]
[ 1235.506153] __schedule+0xcc3/0x19c0
[ 1235.510968] ? io_schedule_timeout+0x140/0x140
[ 1235.516754] ? find_held_lock+0x2b/0x80
[ 1235.521863] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.527746] schedule+0xd8/0x260
[ 1235.532167] schedule_preempt_disabled+0x18/0x30
[ 1235.538147] __mutex_lock+0x6af/0x2010
[ 1235.543154] ? threaded_store+0x132/0x220
[ 1235.548453] ? ww_mutex_lock+0x160/0x160
[ 1235.553653] ? selinux_inode_free_security+0x330/0x330
[ 1235.560226] ? cap_capable+0x1d6/0x3e0
[ 1235.565236] ? threaded_store+0x132/0x220
[ 1235.570536] threaded_store+0x132/0x220
[ 1235.575642] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.580939] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.586240] ? sysfs_kf_write+0x65/0x160
[ 1235.591440] ? sysfs_file_ops+0x140/0x140
[ 1235.596737] kernfs_fop_write_iter+0x335/0x510
[ 1235.602523] ? kernfs_ops+0xf0/0xf0
[ 1235.607234] new_sync_write+0x333/0x750
[ 1235.612339] ? __lock_acquire+0x508/0xc10
[ 1235.617629] ? new_sync_read+0x740/0x740
[ 1235.622831] ? lock_acquire.part.0+0xbc/0x260
[ 1235.628517] ? ksys_write+0x116/0x250
[ 1235.633429] vfs_write+0x657/0x9b0
[ 1235.638045] ksys_write+0x116/0x250
[ 1235.642758] ? __ia32_sys_read+0xc0/0xc0
[ 1235.647950] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.653541] ? rcu_is_watching+0x15/0xd0
[ 1235.658745] do_syscall_64+0xf3/0x650
[ 1235.663652] ? trace_hardirqs_off+0xd/0x30
[ 1235.669047] ? exc_page_fault+0xda/0xf0
[ 1235.674148] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.680618] RIP: 0033:0x7f07b773822e
[ 1235.685427] RSP: 002b:00007ffcc5030d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.694726] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07b773822e
[ 1235.703535] RDX: 0000000000000001 RSI: 000055e31c8a6a90 RDI: 0000000000000007
[ 1235.712345] RBP: 00007ffcc5030d40 R08: 0000000000000000 R09: 0000000000000000
[ 1235.721156] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07b75b0390
[ 1235.729967] R13: 000055e31c8a6a90 R14: 0000000000000007 R15: 0000000000000003
[ 1235.738781]
[ 1235.742058] INFO: task python3:5816 is blocked on a mutex likely owned by task python3:3758.
[ 1235.752333] task:python3 state:D stack:25672 pid:3758 tgid:3758 ppid:3757 task_flags:0x400100 flags:0x00080802
[ 1235.765535] Call Trace:
[ 1235.769073]
[ 1235.772226] __schedule+0xcc3/0x19c0
[ 1235.777037] ? io_schedule_timeout+0x140/0x140
[ 1235.782821] ? find_held_lock+0x2b/0x80
[ 1235.787924] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.794106] schedule+0xd8/0x260
[ 1235.798523] schedule_timeout+0x13f/0x290
[ 1235.803821] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.810190] ? trace_contention_end+0xb3/0x1a0
[ 1235.815976] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.821666] ? ww_mutex_lock+0x160/0x160
[ 1235.826866] msleep+0x1b/0x30
[ 1235.830991] napi_set_threaded+0x3a5/0x440
[ 1235.836390] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.841885] netif_set_threaded+0x257/0x400
[ 1235.847378] threaded_store+0x1b8/0x220
[ 1235.852483] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.857779] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.863077] ? sysfs_kf_write+0x65/0x160
[ 1235.868276] ? sysfs_file_ops+0x140/0x140
[ 1235.873572] kernfs_fop_write_iter+0x335/0x510
[ 1235.879358] ? kernfs_ops+0xf0/0xf0
[ 1235.884071] new_sync_write+0x333/0x750
[ 1235.889173] ? __lock_acquire+0x508/0xc10
[ 1235.894468] ? new_sync_read+0x740/0x740
[ 1235.899668] ? lock_acquire.part.0+0xbc/0x260
[ 1235.905352] ? ksys_write+0x116/0x250
[ 1235.910264] vfs_write+0x657/0x9b0
[ 1235.914881] ksys_write+0x116/0x250
[ 1235.919595] ? __ia32_sys_read+0xc0/0xc0
[ 1235.924796] ? rcu_is_watching+0x15/0xd0
[ 1235.929996] ? rcu_is_watching+0x15/0xd0
[ 1235.935188] do_syscall_64+0xf3/0x650
[ 1235.940102] ? irq_exit_rcu+0x1a/0x30
[ 1235.945011] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.951478] RIP: 0033:0x7fa89843022e
[ 1235.956289] RSP: 002b:00007ffc2f3494a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.965588] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa89843022e
[ 1235.974398] RDX: 0000000000000001 RSI: 000055685f2cca90 RDI: 0000000000000007
[ 1235.983206] RBP: 00007ffc2f3494b0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.992013] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8982a8390
[ 1236.000821] R13: 000055685f2cca90 R14: 0000000000000007 R15: 0000000000000003
[ 1236.009634]
[ 1236.012905] \x0aShowing all locks held in the system:
[ 1236.021440] 1 lock held by khungtaskd/184:
[ 1236.026834] #0: ffffffffb82fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.038309] 2 locks held by systemd-journal/802:
[ 1236.044288] #0: ff1100180e3372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.054175] #1: ffffffffb85029a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.067004] 4 locks held by python3/3758:
[ 1236.072301] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.082191] #1: ff1100016e994480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.093156] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.104316] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.114411] 4 locks held by python3/5816:
[ 1236.119709] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.129600] #1: ff110001f06adc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.140560] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.151717] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.161811]
[ 1236.164276] =============================================\x0a
[ 1358.342293] INFO: task python3:5816 blocked for more than 491 seconds.
[ 1358.350444] Not tainted 7.1.0-rc4-yvvk-gc6fc3bdd5ba0 #1
[ 1358.357694] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.367285] task:python3 state:D stack:25672 pid:5816 tgid:5816 ppid:5815 task_flags:0x400100 flags:0x00080002
[ 1358.380493] Call Trace:
[ 1358.384031]
[ 1358.387181] __schedule+0xcc3/0x19c0
[ 1358.391996] ? io_schedule_timeout+0x140/0x140
[ 1358.397783] ? find_held_lock+0x2b/0x80
[ 1358.402888] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.408767] schedule+0xd8/0x260
[ 1358.413185] schedule_preempt_disabled+0x18/0x30
[ 1358.419162] __mutex_lock+0x6af/0x2010
[ 1358.424170] ? threaded_store+0x132/0x220
[ 1358.429468] ? ww_mutex_lock+0x160/0x160
[ 1358.434668] ? selinux_inode_free_security+0x330/0x330
[ 1358.441237] ? cap_capable+0x1d6/0x3e0
[ 1358.446243] ? threaded_store+0x132/0x220
[ 1358.451541] threaded_store+0x132/0x220
[ 1358.456643] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.461943] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.467245] ? sysfs_kf_write+0x65/0x160
[ 1358.472447] ? sysfs_file_ops+0x140/0x140
[ 1358.477744] kernfs_fop_write_iter+0x335/0x510
[ 1358.483523] ? kernfs_ops+0xf0/0xf0
[ 1358.488235] new_sync_write+0x333/0x750
[ 1358.493340] ? __lock_acquire+0x508/0xc10
[ 1358.498637] ? new_sync_read+0x740/0x740
[ 1358.503829] ? lock_acquire.part.0+0xbc/0x260
[ 1358.509515] ? ksys_write+0x116/0x250
[ 1358.514428] vfs_write+0x657/0x9b0
[ 1358.519045] ksys_write+0x116/0x250
[ 1358.523757] ? __ia32_sys_read+0xc0/0xc0
[ 1358.528957] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.534548] ? rcu_is_watching+0x15/0xd0
[ 1358.539749] do_syscall_64+0xf3/0x650
[ 1358.544657] ? trace_hardirqs_off+0xd/0x30
[ 1358.550053] ? exc_page_fault+0xda/0xf0
[ 1358.555156] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.561626] RIP: 0033:0x7f07b773822e
[ 1358.566435] RSP: 002b:00007ffcc5030d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.575735] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07b773822e
[ 1358.584546] RDX: 0000000000000001 RSI: 000055e31c8a6a90 RDI: 0000000000000007
[ 1358.593353] RBP: 00007ffcc5030d40 R08: 0000000000000000 R09: 0000000000000000
[ 1358.602163] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07b75b0390
[ 1358.610973] R13: 000055e31c8a6a90 R14: 0000000000000007 R15: 0000000000000003
[ 1358.619788]
[ 1358.623061] INFO: task python3:5816 is blocked on a mutex likely owned by task python3:3758.
[ 1358.633333] task:python3 state:D stack:25672 pid:3758 tgid:3758 ppid:3757 task_flags:0x400100 flags:0x00080802
[ 1358.646537] Call Trace:
[ 1358.650075]
[ 1358.653224] __schedule+0xcc3/0x19c0
[ 1358.658035] ? io_schedule_timeout+0x140/0x140
[ 1358.663820] ? find_held_lock+0x2b/0x80
[ 1358.668915] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.675093] schedule+0xd8/0x260
[ 1358.679512] schedule_timeout+0x13f/0x290
[ 1358.684809] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.691179] ? trace_contention_end+0xb3/0x1a0
[ 1358.696959] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.702652] ? ww_mutex_lock+0x160/0x160
[ 1358.707852] msleep+0x1b/0x30
[ 1358.711978] napi_set_threaded+0x3a5/0x440
[ 1358.717376] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.722869] netif_set_threaded+0x257/0x400
[ 1358.728364] threaded_store+0x1b8/0x220
[ 1358.733470] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.738769] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.744064] ? sysfs_kf_write+0x65/0x160
[ 1358.749263] ? sysfs_file_ops+0x140/0x140
[ 1358.754558] kernfs_fop_write_iter+0x335/0x510
[ 1358.760345] ? kernfs_ops+0xf0/0xf0
[ 1358.765056] new_sync_write+0x333/0x750
[ 1358.770158] ? __lock_acquire+0x508/0xc10
[ 1358.775453] ? new_sync_read+0x740/0x740
[ 1358.780655] ? lock_acquire.part.0+0xbc/0x260
[ 1358.786341] ? ksys_write+0x116/0x250
[ 1358.791254] vfs_write+0x657/0x9b0
[ 1358.795872] ksys_write+0x116/0x250
[ 1358.800585] ? __ia32_sys_read+0xc0/0xc0
[ 1358.805787] ? rcu_is_watching+0x15/0xd0
[ 1358.810986] ? rcu_is_watching+0x15/0xd0
[ 1358.816188] do_syscall_64+0xf3/0x650
[ 1358.821097] ? irq_exit_rcu+0x1a/0x30
[ 1358.826007] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.832474] RIP: 0033:0x7fa89843022e
[ 1358.837275] RSP: 002b:00007ffc2f3494a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.846572] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa89843022e
[ 1358.855383] RDX: 0000000000000001 RSI: 000055685f2cca90 RDI: 0000000000000007
[ 1358.864194] RBP: 00007ffc2f3494b0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.873004] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa8982a8390
[ 1358.881812] R13: 000055685f2cca90 R14: 0000000000000007 R15: 0000000000000003
[ 1358.890627]
[ 1358.893897] \x0aShowing all locks held in the system:
[ 1358.902434] 1 lock held by khungtaskd/184:
[ 1358.907830] #0: ffffffffb82fdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.919297] 3 locks held by systemd-journal/802:
[ 1358.925278] #0: ff1100180dbb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.935165] #1: ffffffffb85029a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.947982] #2: ff110001a3ab6ff0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.959542] 4 locks held by python3/3758:
[ 1358.964836] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.974723] #1: ff1100016e994480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.985688] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.996845] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.006929] 4 locks held by python3/5816:
[ 1359.012224] #0: ff1100012fed93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.022116] #1: ff110001f06adc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.033076] #2: ff11000123504b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.044235] #3: ff11000123579d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.054325]
[ 1359.056784] =============================================\x0a