[ 782.341275] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 782.344850] nipa-hw-worker: [5/46] Retrying drivers/net:napi_threaded.py
[ 782.499472] kselftest: Running tests in drivers/net
[ 989.675249] INFO: task python3:5726 blocked for more than 122 seconds.
[ 989.683485] Not tainted 7.1.0-rc2-mnel-g764262def20b #1
[ 989.691477] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.701069] task:python3 state:D stack:25672 pid:5726 tgid:5726 ppid:5725 task_flags:0x400100 flags:0x00080000
[ 989.714278] Call Trace:
[ 989.717819]
[ 989.720969] __schedule+0xcba/0x1940
[ 989.725785] ? io_schedule_timeout+0x140/0x140
[ 989.731571] ? find_held_lock+0x2b/0x80
[ 989.736678] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.742565] schedule+0xd4/0x260
[ 989.746984] schedule_preempt_disabled+0x18/0x30
[ 989.752964] __mutex_lock+0x6a3/0x2000
[ 989.757969] ? threaded_store+0x132/0x220
[ 989.763269] ? ww_mutex_lock+0x160/0x160
[ 989.768466] ? selinux_inode_free_security+0x330/0x330
[ 989.775039] ? cap_capable+0x1d6/0x3e0
[ 989.780045] ? threaded_store+0x132/0x220
[ 989.785331] threaded_store+0x132/0x220
[ 989.790430] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.795730] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.801031] ? sysfs_kf_write+0x65/0x160
[ 989.806229] ? sysfs_file_ops+0x140/0x140
[ 989.811524] kernfs_fop_write_iter+0x335/0x510
[ 989.817310] ? kernfs_ops+0xf0/0xf0
[ 989.822024] new_sync_write+0x333/0x750
[ 989.827128] ? __lock_acquire+0x508/0xc10
[ 989.832419] ? new_sync_read+0x740/0x740
[ 989.837622] ? lock_acquire.part.0+0xbc/0x260
[ 989.843308] ? ksys_write+0x116/0x250
[ 989.848219] vfs_write+0x657/0x9b0
[ 989.852836] ksys_write+0x116/0x250
[ 989.857547] ? __ia32_sys_read+0xc0/0xc0
[ 989.862747] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.868338] ? rcu_is_watching+0x15/0xd0
[ 989.873541] do_syscall_64+0xf3/0x650
[ 989.878448] ? trace_hardirqs_off+0xd/0x30
[ 989.883835] ? exc_page_fault+0xda/0xf0
[ 989.888937] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.895397] RIP: 0033:0x7f04fbcc022e
[ 989.900209] RSP: 002b:00007fff9ede3f40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.909508] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f04fbcc022e
[ 989.918318] RDX: 0000000000000001 RSI: 0000558612a47fe0 RDI: 0000000000000007
[ 989.927128] RBP: 00007fff9ede3f50 R08: 0000000000000000 R09: 0000000000000000
[ 989.935940] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f04fbb38390
[ 989.944747] R13: 0000558612a47fe0 R14: 0000000000000007 R15: 0000000000000003
[ 989.953573]
[ 989.956874] INFO: task python3:5726 is blocked on a mutex likely owned by task python3:3676.
[ 989.967145] task:python3 state:D stack:25672 pid:3676 tgid:3676 ppid:3675 task_flags:0x400100 flags:0x00080802
[ 989.980347] Call Trace:
[ 989.983887]
[ 989.987039] __schedule+0xcba/0x1940
[ 989.991849] ? io_schedule_timeout+0x140/0x140
[ 989.997631] ? find_held_lock+0x2b/0x80
[ 990.002737] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.008917] schedule+0xd4/0x260
[ 990.013336] schedule_timeout+0x13f/0x290
[ 990.018637] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.025009] ? trace_contention_end+0xb3/0x1a0
[ 990.030793] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.036483] ? ww_mutex_lock+0x160/0x160
[ 990.041684] msleep+0x1b/0x30
[ 990.045811] napi_set_threaded+0x3a5/0x440
[ 990.051206] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.056703] netif_set_threaded+0x257/0x400
[ 990.062199] threaded_store+0x1b8/0x220
[ 990.067302] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.072597] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.077893] ? sysfs_kf_write+0x65/0x160
[ 990.083096] ? sysfs_file_ops+0x140/0x140
[ 990.088393] kernfs_fop_write_iter+0x335/0x510
[ 990.094179] ? kernfs_ops+0xf0/0xf0
[ 990.098892] new_sync_write+0x333/0x750
[ 990.103995] ? __lock_acquire+0x508/0xc10
[ 990.109292] ? new_sync_read+0x740/0x740
[ 990.114491] ? lock_acquire.part.0+0xbc/0x260
[ 990.120177] ? ksys_write+0x116/0x250
[ 990.125092] vfs_write+0x657/0x9b0
[ 990.129707] ksys_write+0x116/0x250
[ 990.134418] ? __ia32_sys_read+0xc0/0xc0
[ 990.139618] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.145206] ? rcu_is_watching+0x15/0xd0
[ 990.150407] do_syscall_64+0xf3/0x650
[ 990.155313] ? trace_hardirqs_off+0xd/0x30
[ 990.160706] ? exc_page_fault+0xda/0xf0
[ 990.165806] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.172272] RIP: 0033:0x7f772cda722e
[ 990.177078] RSP: 002b:00007ffd8cecefb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.186379] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f772cda722e
[ 990.195187] RDX: 0000000000000001 RSI: 000055e8e078efe0 RDI: 0000000000000007
[ 990.203998] RBP: 00007ffd8cecefc0 R08: 0000000000000000 R09: 0000000000000000
[ 990.212805] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f772cc1f390
[ 990.221612] R13: 000055e8e078efe0 R14: 0000000000000007 R15: 0000000000000003
[ 990.230424]
[ 990.233685] \x0aShowing all locks held in the system:
[ 990.242226] 1 lock held by khungtaskd/185:
[ 990.247620] #0: ffffffffad4fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.259099] 3 locks held by systemd-journal/802:
[ 990.265075] #0: ff1100180ddb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 990.274970] #1: ffffffffad4fdf60 (rcu_read_lock){....}-{1:3}, at: __update_idle_core+0x5a/0x700
[ 990.285638] #2: ff110001a552ade0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.297187] 4 locks held by python3/3676:
[ 990.302483] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.312375] #1: ff11000170015c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.323345] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.334519] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.344622] 4 locks held by python3/5726:
[ 990.349916] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.359804] #1: ff110001ea7c2c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.370768] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.381926] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.392015]
[ 990.394482] =============================================\x0a
[ 1112.555539] INFO: task python3:5726 blocked for more than 245 seconds.
[ 1112.563687] Not tainted 7.1.0-rc2-mnel-g764262def20b #1
[ 1112.570936] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.580518] task:python3 state:D stack:25672 pid:5726 tgid:5726 ppid:5725 task_flags:0x400100 flags:0x00080000
[ 1112.593724] Call Trace:
[ 1112.597261]
[ 1112.600410] __schedule+0xcba/0x1940
[ 1112.605224] ? io_schedule_timeout+0x140/0x140
[ 1112.611009] ? find_held_lock+0x2b/0x80
[ 1112.616114] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.621998] schedule+0xd4/0x260
[ 1112.626414] schedule_preempt_disabled+0x18/0x30
[ 1112.632390] __mutex_lock+0x6a3/0x2000
[ 1112.637395] ? threaded_store+0x132/0x220
[ 1112.642693] ? ww_mutex_lock+0x160/0x160
[ 1112.647891] ? selinux_inode_free_security+0x330/0x330
[ 1112.654451] ? cap_capable+0x1d6/0x3e0
[ 1112.659457] ? threaded_store+0x132/0x220
[ 1112.664751] threaded_store+0x132/0x220
[ 1112.669849] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.675147] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.680443] ? sysfs_kf_write+0x65/0x160
[ 1112.685643] ? sysfs_file_ops+0x140/0x140
[ 1112.690929] kernfs_fop_write_iter+0x335/0x510
[ 1112.696712] ? kernfs_ops+0xf0/0xf0
[ 1112.701422] new_sync_write+0x333/0x750
[ 1112.706518] ? __lock_acquire+0x508/0xc10
[ 1112.711805] ? new_sync_read+0x740/0x740
[ 1112.717006] ? lock_acquire.part.0+0xbc/0x260
[ 1112.722692] ? ksys_write+0x116/0x250
[ 1112.727599] vfs_write+0x657/0x9b0
[ 1112.732211] ksys_write+0x116/0x250
[ 1112.736922] ? __ia32_sys_read+0xc0/0xc0
[ 1112.742119] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.747706] ? rcu_is_watching+0x15/0xd0
[ 1112.752908] do_syscall_64+0xf3/0x650
[ 1112.757811] ? trace_hardirqs_off+0xd/0x30
[ 1112.763205] ? exc_page_fault+0xda/0xf0
[ 1112.768307] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.774775] RIP: 0033:0x7f04fbcc022e
[ 1112.779582] RSP: 002b:00007fff9ede3f40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.788877] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f04fbcc022e
[ 1112.797683] RDX: 0000000000000001 RSI: 0000558612a47fe0 RDI: 0000000000000007
[ 1112.806489] RBP: 00007fff9ede3f50 R08: 0000000000000000 R09: 0000000000000000
[ 1112.815295] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f04fbb38390
[ 1112.824101] R13: 0000558612a47fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1112.832915]
[ 1112.836200] INFO: task python3:5726 is blocked on a mutex likely owned by task python3:3676.
[ 1112.846471] task:python3 state:D stack:25672 pid:3676 tgid:3676 ppid:3675 task_flags:0x400100 flags:0x00080802
[ 1112.859674] Call Trace:
[ 1112.863209]
[ 1112.866356] __schedule+0xcba/0x1940
[ 1112.871165] ? io_schedule_timeout+0x140/0x140
[ 1112.876949] ? find_held_lock+0x2b/0x80
[ 1112.882049] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.888228] schedule+0xd4/0x260
[ 1112.892645] schedule_timeout+0x13f/0x290
[ 1112.897941] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.904311] ? trace_contention_end+0xb3/0x1a0
[ 1112.910092] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.915780] ? ww_mutex_lock+0x160/0x160
[ 1112.920980] msleep+0x1b/0x30
[ 1112.925103] napi_set_threaded+0x3a5/0x440
[ 1112.930498] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.935992] netif_set_threaded+0x257/0x400
[ 1112.941482] threaded_store+0x1b8/0x220
[ 1112.946582] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.951881] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.957177] ? sysfs_kf_write+0x65/0x160
[ 1112.962380] ? sysfs_file_ops+0x140/0x140
[ 1112.967674] kernfs_fop_write_iter+0x335/0x510
[ 1112.973457] ? kernfs_ops+0xf0/0xf0
[ 1112.978168] new_sync_write+0x333/0x750
[ 1112.983271] ? __lock_acquire+0x508/0xc10
[ 1112.988566] ? new_sync_read+0x740/0x740
[ 1112.993763] ? lock_acquire.part.0+0xbc/0x260
[ 1112.999450] ? ksys_write+0x116/0x250
[ 1113.004359] vfs_write+0x657/0x9b0
[ 1113.008971] ksys_write+0x116/0x250
[ 1113.013680] ? __ia32_sys_read+0xc0/0xc0
[ 1113.018879] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.024466] ? rcu_is_watching+0x15/0xd0
[ 1113.029666] do_syscall_64+0xf3/0x650
[ 1113.034562] ? trace_hardirqs_off+0xd/0x30
[ 1113.039954] ? exc_page_fault+0xda/0xf0
[ 1113.045053] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.051519] RIP: 0033:0x7f772cda722e
[ 1113.056326] RSP: 002b:00007ffd8cecefb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.065620] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f772cda722e
[ 1113.074427] RDX: 0000000000000001 RSI: 000055e8e078efe0 RDI: 0000000000000007
[ 1113.083235] RBP: 00007ffd8cecefc0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.092032] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f772cc1f390
[ 1113.100838] R13: 000055e8e078efe0 R14: 0000000000000007 R15: 0000000000000003
[ 1113.109649]
[ 1113.112908] \x0aShowing all locks held in the system:
[ 1113.121445] 1 lock held by khungtaskd/185:
[ 1113.126838] #0: ffffffffad4fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.138316] 4 locks held by python3/3676:
[ 1113.143610] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.153496] #1: ff11000170015c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.164457] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.175613] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.185706] 4 locks held by python3/5726:
[ 1113.190999] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.200888] #1: ff110001ea7c2c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.211839] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.222994] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.233082]
[ 1113.235546] =============================================\x0a
[ 1235.436822] INFO: task python3:5726 blocked for more than 368 seconds.
[ 1235.444971] Not tainted 7.1.0-rc2-mnel-g764262def20b #1
[ 1235.452221] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.461811] task:python3 state:D stack:25672 pid:5726 tgid:5726 ppid:5725 task_flags:0x400100 flags:0x00080002
[ 1235.475016] Call Trace:
[ 1235.478553]
[ 1235.481702] __schedule+0xcba/0x1940
[ 1235.486515] ? io_schedule_timeout+0x140/0x140
[ 1235.492301] ? find_held_lock+0x2b/0x80
[ 1235.497404] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.503279] schedule+0xd4/0x260
[ 1235.507696] schedule_preempt_disabled+0x18/0x30
[ 1235.513672] __mutex_lock+0x6a3/0x2000
[ 1235.518678] ? threaded_store+0x132/0x220
[ 1235.523978] ? ww_mutex_lock+0x160/0x160
[ 1235.529175] ? selinux_inode_free_security+0x330/0x330
[ 1235.535742] ? cap_capable+0x1d6/0x3e0
[ 1235.540740] ? threaded_store+0x132/0x220
[ 1235.546036] threaded_store+0x132/0x220
[ 1235.551138] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.556438] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.561734] ? sysfs_kf_write+0x65/0x160
[ 1235.566936] ? sysfs_file_ops+0x140/0x140
[ 1235.572231] kernfs_fop_write_iter+0x335/0x510
[ 1235.578016] ? kernfs_ops+0xf0/0xf0
[ 1235.582717] new_sync_write+0x333/0x750
[ 1235.587819] ? __lock_acquire+0x508/0xc10
[ 1235.593116] ? new_sync_read+0x740/0x740
[ 1235.598318] ? lock_acquire.part.0+0xbc/0x260
[ 1235.604003] ? ksys_write+0x116/0x250
[ 1235.608909] vfs_write+0x657/0x9b0
[ 1235.613524] ksys_write+0x116/0x250
[ 1235.618236] ? __ia32_sys_read+0xc0/0xc0
[ 1235.623427] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.629012] ? rcu_is_watching+0x15/0xd0
[ 1235.634213] do_syscall_64+0xf3/0x650
[ 1235.639118] ? trace_hardirqs_off+0xd/0x30
[ 1235.644513] ? exc_page_fault+0xda/0xf0
[ 1235.649614] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.656074] RIP: 0033:0x7f04fbcc022e
[ 1235.660884] RSP: 002b:00007fff9ede3f40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.670174] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f04fbcc022e
[ 1235.678980] RDX: 0000000000000001 RSI: 0000558612a47fe0 RDI: 0000000000000007
[ 1235.687785] RBP: 00007fff9ede3f50 R08: 0000000000000000 R09: 0000000000000000
[ 1235.696590] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f04fbb38390
[ 1235.705396] R13: 0000558612a47fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.714206]
[ 1235.717490] INFO: task python3:5726 is blocked on a mutex likely owned by task python3:3676.
[ 1235.727760] task:python3 state:D stack:25672 pid:3676 tgid:3676 ppid:3675 task_flags:0x400100 flags:0x00080802
[ 1235.740963] Call Trace:
[ 1235.744499]
[ 1235.747646] __schedule+0xcba/0x1940
[ 1235.752454] ? io_schedule_timeout+0x140/0x140
[ 1235.758235] ? find_held_lock+0x2b/0x80
[ 1235.763338] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.769515] schedule+0xd4/0x260
[ 1235.773933] schedule_timeout+0x13f/0x290
[ 1235.779230] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.785600] ? trace_contention_end+0xb3/0x1a0
[ 1235.791373] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.797061] ? ww_mutex_lock+0x160/0x160
[ 1235.802261] msleep+0x1b/0x30
[ 1235.806377] napi_set_threaded+0x3a5/0x440
[ 1235.811773] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.817267] netif_set_threaded+0x257/0x400
[ 1235.822761] threaded_store+0x1b8/0x220
[ 1235.827862] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.833157] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.838453] ? sysfs_kf_write+0x65/0x160
[ 1235.843652] ? sysfs_file_ops+0x140/0x140
[ 1235.848936] kernfs_fop_write_iter+0x335/0x510
[ 1235.854720] ? kernfs_ops+0xf0/0xf0
[ 1235.859429] new_sync_write+0x333/0x750
[ 1235.864529] ? __lock_acquire+0x508/0xc10
[ 1235.869825] ? new_sync_read+0x740/0x740
[ 1235.875022] ? lock_acquire.part.0+0xbc/0x260
[ 1235.880707] ? ksys_write+0x116/0x250
[ 1235.885616] vfs_write+0x657/0x9b0
[ 1235.890231] ksys_write+0x116/0x250
[ 1235.894941] ? __ia32_sys_read+0xc0/0xc0
[ 1235.900141] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.905728] ? rcu_is_watching+0x15/0xd0
[ 1235.910928] do_syscall_64+0xf3/0x650
[ 1235.915823] ? trace_hardirqs_off+0xd/0x30
[ 1235.921216] ? exc_page_fault+0xda/0xf0
[ 1235.926315] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.932781] RIP: 0033:0x7f772cda722e
[ 1235.937588] RSP: 002b:00007ffd8cecefb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.946884] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f772cda722e
[ 1235.955692] RDX: 0000000000000001 RSI: 000055e8e078efe0 RDI: 0000000000000007
[ 1235.964500] RBP: 00007ffd8cecefc0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.973297] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f772cc1f390
[ 1235.982104] R13: 000055e8e078efe0 R14: 0000000000000007 R15: 0000000000000003
[ 1235.990916]
[ 1235.994181] \x0aShowing all locks held in the system:
[ 1236.002719] 1 lock held by khungtaskd/185:
[ 1236.008113] #0: ffffffffad4fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.019593] 4 locks held by python3/3676:
[ 1236.024886] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.034773] #1: ff11000170015c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.045732] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.056885] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.066979] 4 locks held by python3/5726:
[ 1236.072273] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.082162] #1: ff110001ea7c2c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.093122] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.104280] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.114369]
[ 1236.116833] =============================================\x0a
[ 1358.319103] INFO: task python3:5726 blocked for more than 491 seconds.
[ 1358.327269] Not tainted 7.1.0-rc2-mnel-g764262def20b #1
[ 1358.334522] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.344111] task:python3 state:D stack:25672 pid:5726 tgid:5726 ppid:5725 task_flags:0x400100 flags:0x00080002
[ 1358.357318] Call Trace:
[ 1358.360856]
[ 1358.364009] __schedule+0xcba/0x1940
[ 1358.368825] ? io_schedule_timeout+0x140/0x140
[ 1358.374609] ? find_held_lock+0x2b/0x80
[ 1358.379707] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.385593] schedule+0xd4/0x260
[ 1358.390012] schedule_preempt_disabled+0x18/0x30
[ 1358.395992] __mutex_lock+0x6a3/0x2000
[ 1358.400989] ? threaded_store+0x132/0x220
[ 1358.406286] ? ww_mutex_lock+0x160/0x160
[ 1358.411484] ? selinux_inode_free_security+0x330/0x330
[ 1358.418051] ? cap_capable+0x1d6/0x3e0
[ 1358.423049] ? threaded_store+0x132/0x220
[ 1358.428345] threaded_store+0x132/0x220
[ 1358.433446] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.438741] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.444035] ? sysfs_kf_write+0x65/0x160
[ 1358.449235] ? sysfs_file_ops+0x140/0x140
[ 1358.454522] kernfs_fop_write_iter+0x335/0x510
[ 1358.460304] ? kernfs_ops+0xf0/0xf0
[ 1358.465016] new_sync_write+0x333/0x750
[ 1358.470120] ? __lock_acquire+0x508/0xc10
[ 1358.475407] ? new_sync_read+0x740/0x740
[ 1358.480607] ? lock_acquire.part.0+0xbc/0x260
[ 1358.486295] ? ksys_write+0x116/0x250
[ 1358.491202] vfs_write+0x657/0x9b0
[ 1358.495815] ksys_write+0x116/0x250
[ 1358.500526] ? __ia32_sys_read+0xc0/0xc0
[ 1358.505723] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.511312] ? rcu_is_watching+0x15/0xd0
[ 1358.516502] do_syscall_64+0xf3/0x650
[ 1358.521406] ? trace_hardirqs_off+0xd/0x30
[ 1358.526800] ? exc_page_fault+0xda/0xf0
[ 1358.531902] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.538368] RIP: 0033:0x7f04fbcc022e
[ 1358.543178] RSP: 002b:00007fff9ede3f40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.552465] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f04fbcc022e
[ 1358.561272] RDX: 0000000000000001 RSI: 0000558612a47fe0 RDI: 0000000000000007
[ 1358.570079] RBP: 00007fff9ede3f50 R08: 0000000000000000 R09: 0000000000000000
[ 1358.578885] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f04fbb38390
[ 1358.587692] R13: 0000558612a47fe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.596499]
[ 1358.599793] INFO: task python3:5726 is blocked on a mutex likely owned by task python3:3676.
[ 1358.610067] task:python3 state:D stack:25672 pid:3676 tgid:3676 ppid:3675 task_flags:0x400100 flags:0x00080802
[ 1358.623268] Call Trace:
[ 1358.626807]
[ 1358.629958] __schedule+0xcba/0x1940
[ 1358.634767] ? io_schedule_timeout+0x140/0x140
[ 1358.640551] ? find_held_lock+0x2b/0x80
[ 1358.645651] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.651828] schedule+0xd4/0x260
[ 1358.656246] schedule_timeout+0x13f/0x290
[ 1358.661544] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.667904] ? trace_contention_end+0xb3/0x1a0
[ 1358.673690] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.679381] ? ww_mutex_lock+0x160/0x160
[ 1358.684582] msleep+0x1b/0x30
[ 1358.688708] napi_set_threaded+0x3a5/0x440
[ 1358.694101] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.699595] netif_set_threaded+0x257/0x400
[ 1358.705088] threaded_store+0x1b8/0x220
[ 1358.710189] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.715486] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.720779] ? sysfs_kf_write+0x65/0x160
[ 1358.725979] ? sysfs_file_ops+0x140/0x140
[ 1358.731266] kernfs_fop_write_iter+0x335/0x510
[ 1358.737049] ? kernfs_ops+0xf0/0xf0
[ 1358.741757] new_sync_write+0x333/0x750
[ 1358.746859] ? __lock_acquire+0x508/0xc10
[ 1358.752156] ? new_sync_read+0x740/0x740
[ 1358.757356] ? lock_acquire.part.0+0xbc/0x260
[ 1358.763040] ? ksys_write+0x116/0x250
[ 1358.767954] vfs_write+0x657/0x9b0
[ 1358.772568] ksys_write+0x116/0x250
[ 1358.777277] ? __ia32_sys_read+0xc0/0xc0
[ 1358.782474] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.788061] ? rcu_is_watching+0x15/0xd0
[ 1358.793259] do_syscall_64+0xf3/0x650
[ 1358.798163] ? trace_hardirqs_off+0xd/0x30
[ 1358.803554] ? exc_page_fault+0xda/0xf0
[ 1358.808653] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.815120] RIP: 0033:0x7f772cda722e
[ 1358.819925] RSP: 002b:00007ffd8cecefb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.829220] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f772cda722e
[ 1358.838026] RDX: 0000000000000001 RSI: 000055e8e078efe0 RDI: 0000000000000007
[ 1358.846833] RBP: 00007ffd8cecefc0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.855640] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f772cc1f390
[ 1358.864446] R13: 000055e8e078efe0 R14: 0000000000000007 R15: 0000000000000003
[ 1358.873258]
[ 1358.876526] \x0aShowing all locks held in the system:
[ 1358.885064] 1 lock held by khungtaskd/185:
[ 1358.890457] #0: ffffffffad4fdf60 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.901939] 4 locks held by python3/3676:
[ 1358.907234] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.917124] #1: ff11000170015c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.928086] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.939232] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.949324] 4 locks held by python3/5726:
[ 1358.954617] #0: ff110017d97443e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.964505] #1: ff110001ea7c2c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.975464] #2: ff11000121060b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.986620] #3: ff110001210c9d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.996711]
[ 1358.999176] =============================================\x0a