[ 783.981631] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 783.984536] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 784.146458] kselftest: Running tests in drivers/net
[ 989.691404] INFO: task python3:5739 blocked for more than 122 seconds.
[ 989.699627] Not tainted 7.1.0-rc5-vvlv-ge4fd72917a21 #1
[ 989.707483] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.717072] task:python3 state:D stack:25672 pid:5739 tgid:5739 ppid:5738 task_flags:0x400100 flags:0x00080000
[ 989.730275] Call Trace:
[ 989.733812]
[ 989.736961] __schedule+0xcc3/0x19c0
[ 989.741777] ? io_schedule_timeout+0x140/0x140
[ 989.747560] ? find_held_lock+0x2b/0x80
[ 989.752655] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.758542] schedule+0xd8/0x260
[ 989.762958] schedule_preempt_disabled+0x18/0x30
[ 989.768936] __mutex_lock+0x6af/0x2010
[ 989.773940] ? threaded_store+0x132/0x220
[ 989.779240] ? ww_mutex_lock+0x160/0x160
[ 989.784436] ? selinux_inode_free_security+0x330/0x330
[ 989.791008] ? cap_capable+0x1d6/0x3e0
[ 989.796014] ? threaded_store+0x132/0x220
[ 989.801310] threaded_store+0x132/0x220
[ 989.806415] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.811710] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.817007] ? sysfs_kf_write+0x65/0x160
[ 989.822206] ? sysfs_file_ops+0x140/0x140
[ 989.827502] kernfs_fop_write_iter+0x335/0x510
[ 989.833286] ? kernfs_ops+0xf0/0xf0
[ 989.837994] new_sync_write+0x333/0x750
[ 989.843096] ? __lock_acquire+0x508/0xc10
[ 989.848394] ? new_sync_read+0x740/0x740
[ 989.853594] ? lock_acquire.part.0+0xbc/0x260
[ 989.859278] ? ksys_write+0x116/0x250
[ 989.864189] vfs_write+0x657/0x9b0
[ 989.868804] ksys_write+0x116/0x250
[ 989.873514] ? __ia32_sys_read+0xc0/0xc0
[ 989.878714] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.884303] ? rcu_is_watching+0x15/0xd0
[ 989.889501] do_syscall_64+0xf3/0x650
[ 989.894408] ? trace_hardirqs_off+0xd/0x30
[ 989.899793] ? exc_page_fault+0xda/0xf0
[ 989.904893] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.911360] RIP: 0033:0x7f91cdefb22e
[ 989.916159] RSP: 002b:00007ffd33f67110 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.925454] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91cdefb22e
[ 989.934261] RDX: 0000000000000001 RSI: 0000559f3e2026b0 RDI: 0000000000000007
[ 989.943068] RBP: 00007ffd33f67120 R08: 0000000000000000 R09: 0000000000000000
[ 989.951877] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91cdd73390
[ 989.960685] R13: 0000559f3e2026b0 R14: 0000000000000007 R15: 0000000000000003
[ 989.969506]
[ 989.972790] INFO: task python3:5739 is blocked on a mutex likely owned by task python3:3699.
[ 989.983054] task:python3 state:D stack:25672 pid:3699 tgid:3699 ppid:3698 task_flags:0x400100 flags:0x00080802
[ 989.996255] Call Trace:
[ 989.999791]
[ 990.002942] __schedule+0xcc3/0x19c0
[ 990.007742] ? io_schedule_timeout+0x140/0x140
[ 990.013525] ? find_held_lock+0x2b/0x80
[ 990.018629] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.024804] schedule+0xd8/0x260
[ 990.029221] schedule_timeout+0x13f/0x290
[ 990.034517] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.040885] ? trace_contention_end+0xb3/0x1a0
[ 990.046670] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.052360] ? ww_mutex_lock+0x160/0x160
[ 990.057560] msleep+0x1b/0x30
[ 990.061684] napi_set_threaded+0x3a5/0x440
[ 990.067080] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.072571] netif_set_threaded+0x257/0x400
[ 990.078063] threaded_store+0x1b8/0x220
[ 990.083157] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.088454] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.093747] ? sysfs_kf_write+0x65/0x160
[ 990.098946] ? sysfs_file_ops+0x140/0x140
[ 990.104241] kernfs_fop_write_iter+0x335/0x510
[ 990.110023] ? kernfs_ops+0xf0/0xf0
[ 990.114732] new_sync_write+0x333/0x750
[ 990.119833] ? __lock_acquire+0x508/0xc10
[ 990.125127] ? new_sync_read+0x740/0x740
[ 990.130323] ? lock_acquire.part.0+0xbc/0x260
[ 990.136009] ? ksys_write+0x116/0x250
[ 990.140921] vfs_write+0x657/0x9b0
[ 990.145537] ksys_write+0x116/0x250
[ 990.150249] ? __ia32_sys_read+0xc0/0xc0
[ 990.155436] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.161027] ? rcu_is_watching+0x15/0xd0
[ 990.166234] do_syscall_64+0xf3/0x650
[ 990.171136] ? irq_exit_rcu+0x1a/0x30
[ 990.176043] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.182510] RIP: 0033:0x7fc7b769122e
[ 990.187319] RSP: 002b:00007ffdbd756670 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.196614] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc7b769122e
[ 990.205417] RDX: 0000000000000001 RSI: 000055e2670956b0 RDI: 0000000000000007
[ 990.214227] RBP: 00007ffdbd756680 R08: 0000000000000000 R09: 0000000000000000
[ 990.223037] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc7b7509390
[ 990.231844] R13: 000055e2670956b0 R14: 0000000000000007 R15: 0000000000000003
[ 990.240658]
[ 990.243912] \x0aShowing all locks held in the system:
[ 990.252451] 1 lock held by khungtaskd/185:
[ 990.257842] #0: ffffffff87efdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.269324] 4 locks held by python3/3699:
[ 990.274619] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.284509] #1: ff1100017035a680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.295468] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.306622] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.316716] 4 locks held by python3/5739:
[ 990.322012] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.331900] #1: ff110001e3e12680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.342859] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.354016] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.364094]
[ 990.366562] =============================================\x0a
[ 1112.572600] INFO: task python3:5739 blocked for more than 245 seconds.
[ 1112.580749] Not tainted 7.1.0-rc5-vvlv-ge4fd72917a21 #1
[ 1112.587990] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.597582] task:python3 state:D stack:25672 pid:5739 tgid:5739 ppid:5738 task_flags:0x400100 flags:0x00080000
[ 1112.610789] Call Trace:
[ 1112.614329]
[ 1112.617482] __schedule+0xcc3/0x19c0
[ 1112.622301] ? io_schedule_timeout+0x140/0x140
[ 1112.628087] ? find_held_lock+0x2b/0x80
[ 1112.633195] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.639080] schedule+0xd8/0x260
[ 1112.643497] schedule_preempt_disabled+0x18/0x30
[ 1112.649478] __mutex_lock+0x6af/0x2010
[ 1112.654486] ? threaded_store+0x132/0x220
[ 1112.659788] ? ww_mutex_lock+0x160/0x160
[ 1112.664989] ? selinux_inode_free_security+0x330/0x330
[ 1112.671558] ? cap_capable+0x1d6/0x3e0
[ 1112.676567] ? threaded_store+0x132/0x220
[ 1112.681865] threaded_store+0x132/0x220
[ 1112.686971] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.692268] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.697571] ? sysfs_kf_write+0x65/0x160
[ 1112.702774] ? sysfs_file_ops+0x140/0x140
[ 1112.708070] kernfs_fop_write_iter+0x335/0x510
[ 1112.713857] ? kernfs_ops+0xf0/0xf0
[ 1112.718569] new_sync_write+0x333/0x750
[ 1112.723673] ? __lock_acquire+0x508/0xc10
[ 1112.728960] ? new_sync_read+0x740/0x740
[ 1112.734161] ? lock_acquire.part.0+0xbc/0x260
[ 1112.739847] ? ksys_write+0x116/0x250
[ 1112.744751] vfs_write+0x657/0x9b0
[ 1112.749368] ksys_write+0x116/0x250
[ 1112.754082] ? __ia32_sys_read+0xc0/0xc0
[ 1112.759284] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.764874] ? rcu_is_watching+0x15/0xd0
[ 1112.770078] do_syscall_64+0xf3/0x650
[ 1112.774986] ? trace_hardirqs_off+0xd/0x30
[ 1112.780382] ? exc_page_fault+0xda/0xf0
[ 1112.785484] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.791945] RIP: 0033:0x7f91cdefb22e
[ 1112.796756] RSP: 002b:00007ffd33f67110 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.806053] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91cdefb22e
[ 1112.814861] RDX: 0000000000000001 RSI: 0000559f3e2026b0 RDI: 0000000000000007
[ 1112.823669] RBP: 00007ffd33f67120 R08: 0000000000000000 R09: 0000000000000000
[ 1112.832478] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91cdd73390
[ 1112.841288] R13: 0000559f3e2026b0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.850101]
[ 1112.853367] INFO: task python3:5739 is blocked on a mutex likely owned by task python3:3699.
[ 1112.863642] task:python3 state:D stack:25672 pid:3699 tgid:3699 ppid:3698 task_flags:0x400100 flags:0x00080802
[ 1112.876844] Call Trace:
[ 1112.880381]
[ 1112.883533] __schedule+0xcc3/0x19c0
[ 1112.888335] ? io_schedule_timeout+0x140/0x140
[ 1112.894120] ? find_held_lock+0x2b/0x80
[ 1112.899225] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.905405] schedule+0xd8/0x260
[ 1112.909824] schedule_timeout+0x13f/0x290
[ 1112.915113] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.921474] ? trace_contention_end+0xb3/0x1a0
[ 1112.927261] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.932952] ? ww_mutex_lock+0x160/0x160
[ 1112.938153] msleep+0x1b/0x30
[ 1112.942279] napi_set_threaded+0x3a5/0x440
[ 1112.947673] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.953166] netif_set_threaded+0x257/0x400
[ 1112.958651] threaded_store+0x1b8/0x220
[ 1112.963752] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.969051] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.974346] ? sysfs_kf_write+0x65/0x160
[ 1112.979546] ? sysfs_file_ops+0x140/0x140
[ 1112.984834] kernfs_fop_write_iter+0x335/0x510
[ 1112.990617] ? kernfs_ops+0xf0/0xf0
[ 1112.995329] new_sync_write+0x333/0x750
[ 1113.000438] ? __lock_acquire+0x508/0xc10
[ 1113.005733] ? new_sync_read+0x740/0x740
[ 1113.010935] ? lock_acquire.part.0+0xbc/0x260
[ 1113.016622] ? ksys_write+0x116/0x250
[ 1113.021522] vfs_write+0x657/0x9b0
[ 1113.026141] ksys_write+0x116/0x250
[ 1113.030855] ? __ia32_sys_read+0xc0/0xc0
[ 1113.036053] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.041644] ? rcu_is_watching+0x15/0xd0
[ 1113.046844] do_syscall_64+0xf3/0x650
[ 1113.051749] ? irq_exit_rcu+0x1a/0x30
[ 1113.056658] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.063126] RIP: 0033:0x7fc7b769122e
[ 1113.067936] RSP: 002b:00007ffdbd756670 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.077224] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc7b769122e
[ 1113.086023] RDX: 0000000000000001 RSI: 000055e2670956b0 RDI: 0000000000000007
[ 1113.094833] RBP: 00007ffdbd756680 R08: 0000000000000000 R09: 0000000000000000
[ 1113.103643] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc7b7509390
[ 1113.112451] R13: 000055e2670956b0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.121266]
[ 1113.124524] \x0aShowing all locks held in the system:
[ 1113.133064] 1 lock held by khungtaskd/185:
[ 1113.138456] #0: ffffffff87efdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.149939] 4 locks held by python3/3699:
[ 1113.155237] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.165129] #1: ff1100017035a680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.176089] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.187244] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.197338] 4 locks held by python3/5739:
[ 1113.202635] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.212525] #1: ff110001e3e12680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.223486] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.234643] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.244736]
[ 1113.247204] =============================================\x0a
[ 1235.453770] INFO: task python3:5739 blocked for more than 368 seconds.
[ 1235.461919] Not tainted 7.1.0-rc5-vvlv-ge4fd72917a21 #1
[ 1235.469169] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.478757] task:python3 state:D stack:25672 pid:5739 tgid:5739 ppid:5738 task_flags:0x400100 flags:0x00080002
[ 1235.491964] Call Trace:
[ 1235.495504]
[ 1235.498658] __schedule+0xcc3/0x19c0
[ 1235.503476] ? io_schedule_timeout+0x140/0x140
[ 1235.509263] ? find_held_lock+0x2b/0x80
[ 1235.514370] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.520255] schedule+0xd8/0x260
[ 1235.524672] schedule_preempt_disabled+0x18/0x30
[ 1235.530653] __mutex_lock+0x6af/0x2010
[ 1235.535661] ? threaded_store+0x132/0x220
[ 1235.540963] ? ww_mutex_lock+0x160/0x160
[ 1235.546163] ? selinux_inode_free_security+0x330/0x330
[ 1235.552732] ? cap_capable+0x1d6/0x3e0
[ 1235.557742] ? threaded_store+0x132/0x220
[ 1235.563042] threaded_store+0x132/0x220
[ 1235.568146] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.573443] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.578745] ? sysfs_kf_write+0x65/0x160
[ 1235.583946] ? sysfs_file_ops+0x140/0x140
[ 1235.589242] kernfs_fop_write_iter+0x335/0x510
[ 1235.595028] ? kernfs_ops+0xf0/0xf0
[ 1235.599739] new_sync_write+0x333/0x750
[ 1235.604844] ? __lock_acquire+0x508/0xc10
[ 1235.610134] ? new_sync_read+0x740/0x740
[ 1235.615336] ? lock_acquire.part.0+0xbc/0x260
[ 1235.621022] ? ksys_write+0x116/0x250
[ 1235.625935] vfs_write+0x657/0x9b0
[ 1235.630552] ksys_write+0x116/0x250
[ 1235.635264] ? __ia32_sys_read+0xc0/0xc0
[ 1235.640465] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.646055] ? rcu_is_watching+0x15/0xd0
[ 1235.651259] do_syscall_64+0xf3/0x650
[ 1235.656168] ? trace_hardirqs_off+0xd/0x30
[ 1235.661563] ? exc_page_fault+0xda/0xf0
[ 1235.666666] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.673126] RIP: 0033:0x7f91cdefb22e
[ 1235.677937] RSP: 002b:00007ffd33f67110 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.687237] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91cdefb22e
[ 1235.696046] RDX: 0000000000000001 RSI: 0000559f3e2026b0 RDI: 0000000000000007
[ 1235.704845] RBP: 00007ffd33f67120 R08: 0000000000000000 R09: 0000000000000000
[ 1235.713645] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91cdd73390
[ 1235.722454] R13: 0000559f3e2026b0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.731267]
[ 1235.734532] INFO: task python3:5739 is blocked on a mutex likely owned by task python3:3699.
[ 1235.744806] task:python3 state:D stack:25672 pid:3699 tgid:3699 ppid:3698 task_flags:0x400100 flags:0x00080802
[ 1235.758010] Call Trace:
[ 1235.761547]
[ 1235.764701] __schedule+0xcc3/0x19c0
[ 1235.769513] ? io_schedule_timeout+0x140/0x140
[ 1235.775298] ? find_held_lock+0x2b/0x80
[ 1235.780403] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.786580] schedule+0xd8/0x260
[ 1235.790998] schedule_timeout+0x13f/0x290
[ 1235.796296] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.802665] ? trace_contention_end+0xb3/0x1a0
[ 1235.808453] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.814145] ? ww_mutex_lock+0x160/0x160
[ 1235.819345] msleep+0x1b/0x30
[ 1235.823461] napi_set_threaded+0x3a5/0x440
[ 1235.828856] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.834353] netif_set_threaded+0x257/0x400
[ 1235.839846] threaded_store+0x1b8/0x220
[ 1235.844948] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.850246] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.855544] ? sysfs_kf_write+0x65/0x160
[ 1235.860745] ? sysfs_file_ops+0x140/0x140
[ 1235.866042] kernfs_fop_write_iter+0x335/0x510
[ 1235.871829] ? kernfs_ops+0xf0/0xf0
[ 1235.876539] new_sync_write+0x333/0x750
[ 1235.881642] ? __lock_acquire+0x508/0xc10
[ 1235.886936] ? new_sync_read+0x740/0x740
[ 1235.892138] ? lock_acquire.part.0+0xbc/0x260
[ 1235.897823] ? ksys_write+0x116/0x250
[ 1235.902733] vfs_write+0x657/0x9b0
[ 1235.907350] ksys_write+0x116/0x250
[ 1235.912065] ? __ia32_sys_read+0xc0/0xc0
[ 1235.917266] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.922857] ? rcu_is_watching+0x15/0xd0
[ 1235.928060] do_syscall_64+0xf3/0x650
[ 1235.932969] ? irq_exit_rcu+0x1a/0x30
[ 1235.937877] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.944345] RIP: 0033:0x7fc7b769122e
[ 1235.949152] RSP: 002b:00007ffdbd756670 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.958450] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc7b769122e
[ 1235.967258] RDX: 0000000000000001 RSI: 000055e2670956b0 RDI: 0000000000000007
[ 1235.976058] RBP: 00007ffdbd756680 R08: 0000000000000000 R09: 0000000000000000
[ 1235.984866] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc7b7509390
[ 1235.993677] R13: 000055e2670956b0 R14: 0000000000000007 R15: 0000000000000003
[ 1236.002491]
[ 1236.005757] \x0aShowing all locks held in the system:
[ 1236.014287] 1 lock held by khungtaskd/185:
[ 1236.019683] #0: ffffffff87efdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.031157] 2 locks held by systemd-journal/806:
[ 1236.037136] #0: ff1100180e4b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.047023] #1: ffffffff88102c20 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.059848] 4 locks held by python3/3699:
[ 1236.065141] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.075033] #1: ff1100017035a680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.085996] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.097145] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.107236] 4 locks held by python3/5739:
[ 1236.112527] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.122415] #1: ff110001e3e12680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.133376] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.144532] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.154629]
[ 1236.157093] =============================================\x0a
[ 1358.334962] INFO: task python3:5739 blocked for more than 491 seconds.
[ 1358.343109] Not tainted 7.1.0-rc5-vvlv-ge4fd72917a21 #1
[ 1358.350359] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.359950] task:python3 state:D stack:25672 pid:5739 tgid:5739 ppid:5738 task_flags:0x400100 flags:0x00080002
[ 1358.373156] Call Trace:
[ 1358.376696]
[ 1358.379849] __schedule+0xcc3/0x19c0
[ 1358.384667] ? io_schedule_timeout+0x140/0x140
[ 1358.390453] ? find_held_lock+0x2b/0x80
[ 1358.395560] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.401444] schedule+0xd8/0x260
[ 1358.405862] schedule_preempt_disabled+0x18/0x30
[ 1358.411842] __mutex_lock+0x6af/0x2010
[ 1358.416850] ? threaded_store+0x132/0x220
[ 1358.422153] ? ww_mutex_lock+0x160/0x160
[ 1358.427352] ? selinux_inode_free_security+0x330/0x330
[ 1358.433921] ? cap_capable+0x1d6/0x3e0
[ 1358.438930] ? threaded_store+0x132/0x220
[ 1358.444230] threaded_store+0x132/0x220
[ 1358.449333] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.454629] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.459921] ? sysfs_kf_write+0x65/0x160
[ 1358.465122] ? sysfs_file_ops+0x140/0x140
[ 1358.470420] kernfs_fop_write_iter+0x335/0x510
[ 1358.476206] ? kernfs_ops+0xf0/0xf0
[ 1358.480918] new_sync_write+0x333/0x750
[ 1358.486022] ? __lock_acquire+0x508/0xc10
[ 1358.491317] ? new_sync_read+0x740/0x740
[ 1358.496519] ? lock_acquire.part.0+0xbc/0x260
[ 1358.502205] ? ksys_write+0x116/0x250
[ 1358.507118] vfs_write+0x657/0x9b0
[ 1358.511735] ksys_write+0x116/0x250
[ 1358.516449] ? __ia32_sys_read+0xc0/0xc0
[ 1358.521651] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.527240] ? rcu_is_watching+0x15/0xd0
[ 1358.532444] do_syscall_64+0xf3/0x650
[ 1358.537353] ? trace_hardirqs_off+0xd/0x30
[ 1358.542748] ? exc_page_fault+0xda/0xf0
[ 1358.547851] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.554320] RIP: 0033:0x7f91cdefb22e
[ 1358.559130] RSP: 002b:00007ffd33f67110 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.568428] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f91cdefb22e
[ 1358.577236] RDX: 0000000000000001 RSI: 0000559f3e2026b0 RDI: 0000000000000007
[ 1358.586035] RBP: 00007ffd33f67120 R08: 0000000000000000 R09: 0000000000000000
[ 1358.594845] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f91cdd73390
[ 1358.603653] R13: 0000559f3e2026b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.612466]
[ 1358.615733] INFO: task python3:5739 is blocked on a mutex likely owned by task python3:3699.
[ 1358.626007] task:python3 state:D stack:25672 pid:3699 tgid:3699 ppid:3698 task_flags:0x400100 flags:0x00080802
[ 1358.639209] Call Trace:
[ 1358.642747]
[ 1358.645899] __schedule+0xcc3/0x19c0
[ 1358.650711] ? io_schedule_timeout+0x140/0x140
[ 1358.656496] ? find_held_lock+0x2b/0x80
[ 1358.661590] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.667768] schedule+0xd8/0x260
[ 1358.672187] schedule_timeout+0x13f/0x290
[ 1358.677475] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.683844] ? trace_contention_end+0xb3/0x1a0
[ 1358.689631] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.695323] ? ww_mutex_lock+0x160/0x160
[ 1358.700524] msleep+0x1b/0x30
[ 1358.704649] napi_set_threaded+0x3a5/0x440
[ 1358.710048] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.715541] netif_set_threaded+0x257/0x400
[ 1358.721035] threaded_store+0x1b8/0x220
[ 1358.726140] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.731437] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.736736] ? sysfs_kf_write+0x65/0x160
[ 1358.741936] ? sysfs_file_ops+0x140/0x140
[ 1358.747223] kernfs_fop_write_iter+0x335/0x510
[ 1358.753007] ? kernfs_ops+0xf0/0xf0
[ 1358.757718] new_sync_write+0x333/0x750
[ 1358.762822] ? __lock_acquire+0x508/0xc10
[ 1358.768116] ? new_sync_read+0x740/0x740
[ 1358.773318] ? lock_acquire.part.0+0xbc/0x260
[ 1358.779004] ? ksys_write+0x116/0x250
[ 1358.783913] vfs_write+0x657/0x9b0
[ 1358.788530] ksys_write+0x116/0x250
[ 1358.793244] ? __ia32_sys_read+0xc0/0xc0
[ 1358.798445] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.804036] ? rcu_is_watching+0x15/0xd0
[ 1358.809238] do_syscall_64+0xf3/0x650
[ 1358.814147] ? irq_exit_rcu+0x1a/0x30
[ 1358.819054] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.825521] RIP: 0033:0x7fc7b769122e
[ 1358.830319] RSP: 002b:00007ffdbd756670 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.839615] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc7b769122e
[ 1358.848423] RDX: 0000000000000001 RSI: 000055e2670956b0 RDI: 0000000000000007
[ 1358.857231] RBP: 00007ffdbd756680 R08: 0000000000000000 R09: 0000000000000000
[ 1358.866039] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc7b7509390
[ 1358.874838] R13: 000055e2670956b0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.883651]
[ 1358.886921] \x0aShowing all locks held in the system:
[ 1358.895458] 1 lock held by khungtaskd/185:
[ 1358.900853] #0: ffffffff87efdde0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.912325] 2 locks held by systemd-journal/806:
[ 1358.918305] #0: ff1100180dab72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.928192] #1: ffffffff88102c20 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.941016] 4 locks held by python3/3699:
[ 1358.946313] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.956196] #1: ff1100017035a680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.967158] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.978316] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.988407] 4 locks held by python3/5739:
[ 1358.993694] #0: ff1100011c8d93e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.003586] #1: ff110001e3e12680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.014548] #2: ff110001122ccb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.025704] #3: ff11000112271d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.035795]
[ 1359.038252] =============================================\x0a