[ 787.416392] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 787.418904] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 787.551419] kselftest: Running tests in drivers/net
[ 989.716747] INFO: task python3:5840 blocked for more than 122 seconds.
[ 989.724977] Not tainted 7.1.0-rc4-ibhq-gad6a1ad5271b #1
[ 989.732909] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.742491] task:python3 state:D stack:25672 pid:5840 tgid:5840 ppid:5839 task_flags:0x400100 flags:0x00080000
[ 989.755697] Call Trace:
[ 989.759238]
[ 989.762391] __schedule+0xcc3/0x19c0
[ 989.767207] ? io_schedule_timeout+0x140/0x140
[ 989.772990] ? find_held_lock+0x2b/0x80
[ 989.778087] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.783973] schedule+0xd8/0x260
[ 989.788392] schedule_preempt_disabled+0x18/0x30
[ 989.794370] __mutex_lock+0x6af/0x2010
[ 989.799373] ? threaded_store+0x132/0x220
[ 989.804664] ? ww_mutex_lock+0x160/0x160
[ 989.809862] ? selinux_inode_free_security+0x330/0x330
[ 989.816434] ? cap_capable+0x1d6/0x3e0
[ 989.821444] ? threaded_store+0x132/0x220
[ 989.826739] threaded_store+0x132/0x220
[ 989.831842] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.837141] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.842442] ? sysfs_kf_write+0x65/0x160
[ 989.847640] ? sysfs_file_ops+0x140/0x140
[ 989.852935] kernfs_fop_write_iter+0x335/0x510
[ 989.858722] ? kernfs_ops+0xf0/0xf0
[ 989.863435] new_sync_write+0x333/0x750
[ 989.868541] ? __lock_acquire+0x508/0xc10
[ 989.873838] ? new_sync_read+0x740/0x740
[ 989.879039] ? lock_acquire.part.0+0xbc/0x260
[ 989.884717] ? ksys_write+0x116/0x250
[ 989.889631] vfs_write+0x657/0x9b0
[ 989.894245] ksys_write+0x116/0x250
[ 989.898956] ? __ia32_sys_read+0xc0/0xc0
[ 989.904156] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.909737] ? rcu_is_watching+0x15/0xd0
[ 989.914929] do_syscall_64+0xf3/0x650
[ 989.919836] ? irq_exit_rcu+0x1a/0x30
[ 989.924735] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.931202] RIP: 0033:0x7fce21fc122e
[ 989.936012] RSP: 002b:00007ffd37dfed60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.945312] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fce21fc122e
[ 989.954120] RDX: 0000000000000001 RSI: 000056428a46c6b0 RDI: 0000000000000007
[ 989.962927] RBP: 00007ffd37dfed70 R08: 0000000000000000 R09: 0000000000000000
[ 989.971725] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fce21e39390
[ 989.980534] R13: 000056428a46c6b0 R14: 0000000000000007 R15: 0000000000000003
[ 989.989354]
[ 989.992651] INFO: task python3:5840 is blocked on a mutex likely owned by task python3:3819.
[ 990.002925] task:python3 state:D stack:25672 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802
[ 990.016130] Call Trace:
[ 990.019670]
[ 990.022820] __schedule+0xcc3/0x19c0
[ 990.027629] ? io_schedule_timeout+0x140/0x140
[ 990.033413] ? find_held_lock+0x2b/0x80
[ 990.038517] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.044698] schedule+0xd8/0x260
[ 990.049118] schedule_timeout+0x13f/0x290
[ 990.054417] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.060784] ? trace_contention_end+0xb3/0x1a0
[ 990.066570] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.072259] ? ww_mutex_lock+0x160/0x160
[ 990.077462] msleep+0x1b/0x30
[ 990.081590] napi_set_threaded+0x3a5/0x440
[ 990.086988] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.092483] netif_set_threaded+0x257/0x400
[ 990.097980] threaded_store+0x1b8/0x220
[ 990.103081] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.108382] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.113678] ? sysfs_kf_write+0x65/0x160
[ 990.118876] ? sysfs_file_ops+0x140/0x140
[ 990.124171] kernfs_fop_write_iter+0x335/0x510
[ 990.129955] ? kernfs_ops+0xf0/0xf0
[ 990.134667] new_sync_write+0x333/0x750
[ 990.139760] ? __lock_acquire+0x508/0xc10
[ 990.145056] ? new_sync_read+0x740/0x740
[ 990.150255] ? lock_acquire.part.0+0xbc/0x260
[ 990.155943] ? ksys_write+0x116/0x250
[ 990.160854] vfs_write+0x657/0x9b0
[ 990.165467] ksys_write+0x116/0x250
[ 990.170178] ? __ia32_sys_read+0xc0/0xc0
[ 990.175379] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.180958] ? rcu_is_watching+0x15/0xd0
[ 990.186157] do_syscall_64+0xf3/0x650
[ 990.191065] ? irq_exit_rcu+0x1a/0x30
[ 990.195973] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.202439] RIP: 0033:0x7f7e3c4ce22e
[ 990.207248] RSP: 002b:00007ffdf85acd80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.216546] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e3c4ce22e
[ 990.225354] RDX: 0000000000000001 RSI: 000055cde50cc6b0 RDI: 0000000000000007
[ 990.234161] RBP: 00007ffdf85acd90 R08: 0000000000000000 R09: 0000000000000000
[ 990.242968] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e3c346390
[ 990.251767] R13: 000055cde50cc6b0 R14: 0000000000000007 R15: 0000000000000003
[ 990.260570]
[ 990.263829] \x0aShowing all locks held in the system:
[ 990.272366] 1 lock held by khungtaskd/184:
[ 990.277760] #0: ffffffffb8cfdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.289235] 3 locks held by systemd-journal/805:
[ 990.295217] #0: ff1100180e3372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.305106] #1: ff110001a2a23350 (&ep->lock){-.-.}-{3:3}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.315387] #2: ff110001a06dab20 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.326949] 4 locks held by python3/3819:
[ 990.332246] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.342136] #1: ff11000172dce880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.353099] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.364255] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.374356] 4 locks held by python3/5840:
[ 990.379654] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.389543] #1: ff110001f5f3d680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.400504] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.411661] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.421742]
[ 990.424210] =============================================\x0a
[ 1112.597803] INFO: task python3:5840 blocked for more than 245 seconds.
[ 1112.605952] Not tainted 7.1.0-rc4-ibhq-gad6a1ad5271b #1
[ 1112.613204] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.622792] task:python3 state:D stack:25672 pid:5840 tgid:5840 ppid:5839 task_flags:0x400100 flags:0x00080000
[ 1112.635998] Call Trace:
[ 1112.639539]
[ 1112.642689] __schedule+0xcc3/0x19c0
[ 1112.647508] ? io_schedule_timeout+0x140/0x140
[ 1112.653295] ? find_held_lock+0x2b/0x80
[ 1112.658400] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.664284] schedule+0xd8/0x260
[ 1112.668703] schedule_preempt_disabled+0x18/0x30
[ 1112.674681] __mutex_lock+0x6af/0x2010
[ 1112.679685] ? threaded_store+0x132/0x220
[ 1112.684988] ? ww_mutex_lock+0x160/0x160
[ 1112.690188] ? selinux_inode_free_security+0x330/0x330
[ 1112.696759] ? cap_capable+0x1d6/0x3e0
[ 1112.701769] ? threaded_store+0x132/0x220
[ 1112.707068] threaded_store+0x132/0x220
[ 1112.712169] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.717468] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.722770] ? sysfs_kf_write+0x65/0x160
[ 1112.727972] ? sysfs_file_ops+0x140/0x140
[ 1112.733270] kernfs_fop_write_iter+0x335/0x510
[ 1112.739049] ? kernfs_ops+0xf0/0xf0
[ 1112.743761] new_sync_write+0x333/0x750
[ 1112.748868] ? __lock_acquire+0x508/0xc10
[ 1112.754163] ? new_sync_read+0x740/0x740
[ 1112.759365] ? lock_acquire.part.0+0xbc/0x260
[ 1112.765052] ? ksys_write+0x116/0x250
[ 1112.769962] vfs_write+0x657/0x9b0
[ 1112.774580] ksys_write+0x116/0x250
[ 1112.779293] ? __ia32_sys_read+0xc0/0xc0
[ 1112.784494] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.790086] ? rcu_is_watching+0x15/0xd0
[ 1112.795287] do_syscall_64+0xf3/0x650
[ 1112.800183] ? irq_exit_rcu+0x1a/0x30
[ 1112.805093] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.811563] RIP: 0033:0x7fce21fc122e
[ 1112.816374] RSP: 002b:00007ffd37dfed60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.825672] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fce21fc122e
[ 1112.834483] RDX: 0000000000000001 RSI: 000056428a46c6b0 RDI: 0000000000000007
[ 1112.843289] RBP: 00007ffd37dfed70 R08: 0000000000000000 R09: 0000000000000000
[ 1112.852098] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fce21e39390
[ 1112.860909] R13: 000056428a46c6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.869726]
[ 1112.872987] INFO: task python3:5840 is blocked on a mutex likely owned by task python3:3819.
[ 1112.883261] task:python3 state:D stack:25672 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802
[ 1112.896464] Call Trace:
[ 1112.900002]
[ 1112.903153] __schedule+0xcc3/0x19c0
[ 1112.907967] ? io_schedule_timeout+0x140/0x140
[ 1112.913753] ? find_held_lock+0x2b/0x80
[ 1112.918858] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.925038] schedule+0xd8/0x260
[ 1112.929457] schedule_timeout+0x13f/0x290
[ 1112.934754] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.941125] ? trace_contention_end+0xb3/0x1a0
[ 1112.946913] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.952606] ? ww_mutex_lock+0x160/0x160
[ 1112.957806] msleep+0x1b/0x30
[ 1112.961933] napi_set_threaded+0x3a5/0x440
[ 1112.967328] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.972825] netif_set_threaded+0x257/0x400
[ 1112.978321] threaded_store+0x1b8/0x220
[ 1112.983426] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.988724] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.994020] ? sysfs_kf_write+0x65/0x160
[ 1112.999220] ? sysfs_file_ops+0x140/0x140
[ 1113.004508] kernfs_fop_write_iter+0x335/0x510
[ 1113.010295] ? kernfs_ops+0xf0/0xf0
[ 1113.015006] new_sync_write+0x333/0x750
[ 1113.020109] ? __lock_acquire+0x508/0xc10
[ 1113.025396] ? new_sync_read+0x740/0x740
[ 1113.030597] ? lock_acquire.part.0+0xbc/0x260
[ 1113.036284] ? ksys_write+0x116/0x250
[ 1113.041193] vfs_write+0x657/0x9b0
[ 1113.045810] ksys_write+0x116/0x250
[ 1113.050523] ? __ia32_sys_read+0xc0/0xc0
[ 1113.055723] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.061314] ? rcu_is_watching+0x15/0xd0
[ 1113.066514] do_syscall_64+0xf3/0x650
[ 1113.071421] ? irq_exit_rcu+0x1a/0x30
[ 1113.076327] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.082795] RIP: 0033:0x7f7e3c4ce22e
[ 1113.087605] RSP: 002b:00007ffdf85acd80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.096895] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e3c4ce22e
[ 1113.105705] RDX: 0000000000000001 RSI: 000055cde50cc6b0 RDI: 0000000000000007
[ 1113.114515] RBP: 00007ffdf85acd90 R08: 0000000000000000 R09: 0000000000000000
[ 1113.123325] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e3c346390
[ 1113.132133] R13: 000055cde50cc6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.140947]
[ 1113.144209] \x0aShowing all locks held in the system:
[ 1113.152749] 1 lock held by khungtaskd/184:
[ 1113.158141] #0: ffffffffb8cfdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.169628] 4 locks held by python3/3819:
[ 1113.174924] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.184816] #1: ff11000172dce880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.195779] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.206936] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.217033] 4 locks held by python3/5840:
[ 1113.222328] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.232218] #1: ff110001f5f3d680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.243179] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.254328] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.264421]
[ 1113.266888] =============================================\x0a
[ 1235.478865] INFO: task python3:5840 blocked for more than 368 seconds.
[ 1235.487013] Not tainted 7.1.0-rc4-ibhq-gad6a1ad5271b #1
[ 1235.494264] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.503852] task:python3 state:D stack:25672 pid:5840 tgid:5840 ppid:5839 task_flags:0x400100 flags:0x00080002
[ 1235.517055] Call Trace:
[ 1235.520595]
[ 1235.523743] __schedule+0xcc3/0x19c0
[ 1235.528560] ? io_schedule_timeout+0x140/0x140
[ 1235.534346] ? find_held_lock+0x2b/0x80
[ 1235.539454] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.545336] schedule+0xd8/0x260
[ 1235.549753] schedule_preempt_disabled+0x18/0x30
[ 1235.555730] __mutex_lock+0x6af/0x2010
[ 1235.560733] ? threaded_store+0x132/0x220
[ 1235.566023] ? ww_mutex_lock+0x160/0x160
[ 1235.571218] ? selinux_inode_free_security+0x330/0x330
[ 1235.577790] ? cap_capable+0x1d6/0x3e0
[ 1235.582799] ? threaded_store+0x132/0x220
[ 1235.588093] threaded_store+0x132/0x220
[ 1235.593194] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.598492] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.603793] ? sysfs_kf_write+0x65/0x160
[ 1235.608995] ? sysfs_file_ops+0x140/0x140
[ 1235.614291] kernfs_fop_write_iter+0x335/0x510
[ 1235.620073] ? kernfs_ops+0xf0/0xf0
[ 1235.624784] new_sync_write+0x333/0x750
[ 1235.629879] ? __lock_acquire+0x508/0xc10
[ 1235.635167] ? new_sync_read+0x740/0x740
[ 1235.640365] ? lock_acquire.part.0+0xbc/0x260
[ 1235.646048] ? ksys_write+0x116/0x250
[ 1235.650958] vfs_write+0x657/0x9b0
[ 1235.655571] ksys_write+0x116/0x250
[ 1235.660281] ? __ia32_sys_read+0xc0/0xc0
[ 1235.665477] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.671066] ? rcu_is_watching+0x15/0xd0
[ 1235.676266] do_syscall_64+0xf3/0x650
[ 1235.681171] ? irq_exit_rcu+0x1a/0x30
[ 1235.686079] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.692536] RIP: 0033:0x7fce21fc122e
[ 1235.697347] RSP: 002b:00007ffd37dfed60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.706645] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fce21fc122e
[ 1235.715452] RDX: 0000000000000001 RSI: 000056428a46c6b0 RDI: 0000000000000007
[ 1235.724260] RBP: 00007ffd37dfed70 R08: 0000000000000000 R09: 0000000000000000
[ 1235.733066] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fce21e39390
[ 1235.741874] R13: 000056428a46c6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.750686]
[ 1235.753957] INFO: task python3:5840 is blocked on a mutex likely owned by task python3:3819.
[ 1235.764229] task:python3 state:D stack:25672 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802
[ 1235.777431] Call Trace:
[ 1235.780969]
[ 1235.784118] __schedule+0xcc3/0x19c0
[ 1235.788927] ? io_schedule_timeout+0x140/0x140
[ 1235.794710] ? find_held_lock+0x2b/0x80
[ 1235.799813] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.805993] schedule+0xd8/0x260
[ 1235.810407] schedule_timeout+0x13f/0x290
[ 1235.815710] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.822079] ? trace_contention_end+0xb3/0x1a0
[ 1235.827864] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.833552] ? ww_mutex_lock+0x160/0x160
[ 1235.838740] msleep+0x1b/0x30
[ 1235.842864] napi_set_threaded+0x3a5/0x440
[ 1235.848258] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.853754] netif_set_threaded+0x257/0x400
[ 1235.859247] threaded_store+0x1b8/0x220
[ 1235.864348] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.869647] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.874944] ? sysfs_kf_write+0x65/0x160
[ 1235.880140] ? sysfs_file_ops+0x140/0x140
[ 1235.885433] kernfs_fop_write_iter+0x335/0x510
[ 1235.891217] ? kernfs_ops+0xf0/0xf0
[ 1235.895928] new_sync_write+0x333/0x750
[ 1235.901029] ? __lock_acquire+0x508/0xc10
[ 1235.906326] ? new_sync_read+0x740/0x740
[ 1235.911524] ? lock_acquire.part.0+0xbc/0x260
[ 1235.917207] ? ksys_write+0x116/0x250
[ 1235.922118] vfs_write+0x657/0x9b0
[ 1235.926731] ksys_write+0x116/0x250
[ 1235.931442] ? __ia32_sys_read+0xc0/0xc0
[ 1235.936643] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.942222] ? rcu_is_watching+0x15/0xd0
[ 1235.947422] do_syscall_64+0xf3/0x650
[ 1235.952319] ? irq_exit_rcu+0x1a/0x30
[ 1235.957223] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.963690] RIP: 0033:0x7f7e3c4ce22e
[ 1235.968488] RSP: 002b:00007ffdf85acd80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.977782] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e3c4ce22e
[ 1235.986590] RDX: 0000000000000001 RSI: 000055cde50cc6b0 RDI: 0000000000000007
[ 1235.995396] RBP: 00007ffdf85acd90 R08: 0000000000000000 R09: 0000000000000000
[ 1236.004203] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e3c346390
[ 1236.013009] R13: 000055cde50cc6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1236.021824]
[ 1236.025091] \x0aShowing all locks held in the system:
[ 1236.033629] 1 lock held by khungtaskd/184:
[ 1236.039023] #0: ffffffffb8cfdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.050502] 4 locks held by python3/3819:
[ 1236.055795] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.065683] #1: ff11000172dce880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.076643] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.087797] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.097891] 4 locks held by python3/5840:
[ 1236.103184] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.113073] #1: ff110001f5f3d680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.124023] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.135179] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.145273]
[ 1236.147740] =============================================\x0a
[ 1358.358959] INFO: task python3:5840 blocked for more than 491 seconds.
[ 1358.367110] Not tainted 7.1.0-rc4-ibhq-gad6a1ad5271b #1
[ 1358.374362] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.383952] task:python3 state:D stack:25672 pid:5840 tgid:5840 ppid:5839 task_flags:0x400100 flags:0x00080002
[ 1358.397158] Call Trace:
[ 1358.400700]
[ 1358.403849] __schedule+0xcc3/0x19c0
[ 1358.408666] ? io_schedule_timeout+0x140/0x140
[ 1358.414453] ? find_held_lock+0x2b/0x80
[ 1358.419556] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.425442] schedule+0xd8/0x260
[ 1358.429860] schedule_preempt_disabled+0x18/0x30
[ 1358.435838] __mutex_lock+0x6af/0x2010
[ 1358.440842] ? threaded_store+0x132/0x220
[ 1358.446142] ? ww_mutex_lock+0x160/0x160
[ 1358.451340] ? selinux_inode_free_security+0x330/0x330
[ 1358.457902] ? cap_capable+0x1d6/0x3e0
[ 1358.462915] ? threaded_store+0x132/0x220
[ 1358.468212] threaded_store+0x132/0x220
[ 1358.473315] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.478613] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.483910] ? sysfs_kf_write+0x65/0x160
[ 1358.489109] ? sysfs_file_ops+0x140/0x140
[ 1358.494406] kernfs_fop_write_iter+0x335/0x510
[ 1358.500181] ? kernfs_ops+0xf0/0xf0
[ 1358.504893] new_sync_write+0x333/0x750
[ 1358.509995] ? __lock_acquire+0x508/0xc10
[ 1358.515293] ? new_sync_read+0x740/0x740
[ 1358.520496] ? lock_acquire.part.0+0xbc/0x260
[ 1358.526185] ? ksys_write+0x116/0x250
[ 1358.531096] vfs_write+0x657/0x9b0
[ 1358.535714] ksys_write+0x116/0x250
[ 1358.540427] ? __ia32_sys_read+0xc0/0xc0
[ 1358.545626] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.551217] ? rcu_is_watching+0x15/0xd0
[ 1358.556419] do_syscall_64+0xf3/0x650
[ 1358.561324] ? irq_exit_rcu+0x1a/0x30
[ 1358.566233] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.572700] RIP: 0033:0x7fce21fc122e
[ 1358.577501] RSP: 002b:00007ffd37dfed60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.587580] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fce21fc122e
[ 1358.596392] RDX: 0000000000000001 RSI: 000056428a46c6b0 RDI: 0000000000000007
[ 1358.605199] RBP: 00007ffd37dfed70 R08: 0000000000000000 R09: 0000000000000000
[ 1358.614008] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fce21e39390
[ 1358.622816] R13: 000056428a46c6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.631628]
[ 1358.634895] INFO: task python3:5840 is blocked on a mutex likely owned by task python3:3819.
[ 1358.645169] task:python3 state:D stack:25672 pid:3819 tgid:3819 ppid:3818 task_flags:0x400100 flags:0x00080802
[ 1358.658375] Call Trace:
[ 1358.661915]
[ 1358.665065] __schedule+0xcc3/0x19c0
[ 1358.669878] ? io_schedule_timeout+0x140/0x140
[ 1358.675664] ? find_held_lock+0x2b/0x80
[ 1358.680769] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.686949] schedule+0xd8/0x260
[ 1358.691369] schedule_timeout+0x13f/0x290
[ 1358.696667] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.703029] ? trace_contention_end+0xb3/0x1a0
[ 1358.708814] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.714503] ? ww_mutex_lock+0x160/0x160
[ 1358.719705] msleep+0x1b/0x30
[ 1358.723832] napi_set_threaded+0x3a5/0x440
[ 1358.729229] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.734724] netif_set_threaded+0x257/0x400
[ 1358.740220] threaded_store+0x1b8/0x220
[ 1358.745322] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.750621] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.755917] ? sysfs_kf_write+0x65/0x160
[ 1358.761119] ? sysfs_file_ops+0x140/0x140
[ 1358.766416] kernfs_fop_write_iter+0x335/0x510
[ 1358.772200] ? kernfs_ops+0xf0/0xf0
[ 1358.776913] new_sync_write+0x333/0x750
[ 1358.782009] ? __lock_acquire+0x508/0xc10
[ 1358.787306] ? new_sync_read+0x740/0x740
[ 1358.792508] ? lock_acquire.part.0+0xbc/0x260
[ 1358.798195] ? ksys_write+0x116/0x250
[ 1358.803106] vfs_write+0x657/0x9b0
[ 1358.807724] ksys_write+0x116/0x250
[ 1358.812437] ? __ia32_sys_read+0xc0/0xc0
[ 1358.817636] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.823224] ? rcu_is_watching+0x15/0xd0
[ 1358.828423] do_syscall_64+0xf3/0x650
[ 1358.833330] ? irq_exit_rcu+0x1a/0x30
[ 1358.838236] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.844702] RIP: 0033:0x7f7e3c4ce22e
[ 1358.849511] RSP: 002b:00007ffdf85acd80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.858810] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e3c4ce22e
[ 1358.867617] RDX: 0000000000000001 RSI: 000055cde50cc6b0 RDI: 0000000000000007
[ 1358.876424] RBP: 00007ffdf85acd90 R08: 0000000000000000 R09: 0000000000000000
[ 1358.885231] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e3c346390
[ 1358.894042] R13: 000055cde50cc6b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.902854]
[ 1358.906125] \x0aShowing all locks held in the system:
[ 1358.914666] 1 lock held by khungtaskd/184:
[ 1358.920062] #0: ffffffffb8cfdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.931534] 3 locks held by systemd-journal/805:
[ 1358.937515] #0: ff1100180e2372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.947403] #1: ffffffffb8f02b20 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.960218] #2: ff110001a06da9c0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.971780] 4 locks held by python3/3819:
[ 1358.977075] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.986966] #1: ff11000172dce880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.997930] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.009086] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.019179] 4 locks held by python3/5840:
[ 1359.024477] #0: ff1100012f6dc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.034367] #1: ff110001f5f3d680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.045328] #2: ff110017d9288b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.056483] #3: ff110017d9261d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.066575]
[ 1359.069042] =============================================\x0a