Revision 6e2df0581f569038719cf2bc2b3baa3fcc83cab4 authored by Peter Zijlstra on 08 November 2019, 10:11:52 UTC, committed by Peter Zijlstra on 08 November 2019, 21:34:14 UTC
Commit 67692435c411 ("sched: Rework pick_next_task() slow-path") inadvertly introduced a race because it changed a previously unexplored dependency between dropping the rq->lock and sched_class::put_prev_task(). The comments about dropping rq->lock, in for example newidle_balance(), only mentions the task being current and ->on_cpu being set. But when we look at the 'change' pattern (in for example sched_setnuma()): queued = task_on_rq_queued(p); /* p->on_rq == TASK_ON_RQ_QUEUED */ running = task_current(rq, p); /* rq->curr == p */ if (queued) dequeue_task(...); if (running) put_prev_task(...); /* change task properties */ if (queued) enqueue_task(...); if (running) set_next_task(...); It becomes obvious that if we do this after put_prev_task() has already been called on @p, things go sideways. This is exactly what the commit in question allows to happen when it does: prev->sched_class->put_prev_task(rq, prev, rf); if (!rq->nr_running) newidle_balance(rq, rf); The newidle_balance() call will drop rq->lock after we've called put_prev_task() and that allows the above 'change' pattern to interleave and mess up the state. Furthermore, it turns out we lost the RT-pull when we put the last DL task. Fix both problems by extracting the balancing from put_prev_task() and doing a multi-class balance() pass before put_prev_task(). Fixes: 67692435c411 ("sched: Rework pick_next_task() slow-path") Reported-by: Quentin Perret <qperret@google.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Quentin Perret <qperret@google.com> Tested-by: Valentin Schneider <valentin.schneider@arm.com>
1 parent e3b8b6a
File | Mode | Size |
---|---|---|
Kconfig | -rw-r--r-- | 5.5 KB |
LICENCE | -rw-r--r-- | 1.4 KB |
Makefile | -rw-r--r-- | 786 bytes |
README.Locking | -rw-r--r-- | 6.9 KB |
TODO | -rw-r--r-- | 1.9 KB |
acl.c | -rw-r--r-- | 6.7 KB |
acl.h | -rw-r--r-- | 966 bytes |
background.c | -rw-r--r-- | 4.2 KB |
build.c | -rw-r--r-- | 12.6 KB |
compr.c | -rw-r--r-- | 11.6 KB |
compr.h | -rw-r--r-- | 3.1 KB |
compr_lzo.c | -rw-r--r-- | 2.2 KB |
compr_rtime.c | -rw-r--r-- | 2.8 KB |
compr_rubin.c | -rw-r--r-- | 8.7 KB |
compr_zlib.c | -rw-r--r-- | 5.5 KB |
debug.c | -rw-r--r-- | 25.6 KB |
debug.h | -rw-r--r-- | 7.9 KB |
dir.c | -rw-r--r-- | 22.6 KB |
erase.c | -rw-r--r-- | 13.4 KB |
file.c | -rw-r--r-- | 9.1 KB |
fs.c | -rw-r--r-- | 19.3 KB |
gc.c | -rw-r--r-- | 44.0 KB |
ioctl.c | -rw-r--r-- | 557 bytes |
jffs2_fs_i.h | -rw-r--r-- | 1.6 KB |
jffs2_fs_sb.h | -rw-r--r-- | 5.7 KB |
malloc.c | -rw-r--r-- | 7.3 KB |
nodelist.c | -rw-r--r-- | 21.3 KB |
nodelist.h | -rw-r--r-- | 17.9 KB |
nodemgmt.c | -rw-r--r-- | 28.4 KB |
os-linux.h | -rw-r--r-- | 7.4 KB |
read.c | -rw-r--r-- | 6.7 KB |
readinode.c | -rw-r--r-- | 42.8 KB |
scan.c | -rw-r--r-- | 35.3 KB |
security.c | -rw-r--r-- | 1.9 KB |
summary.c | -rw-r--r-- | 23.6 KB |
summary.h | -rw-r--r-- | 6.3 KB |
super.c | -rw-r--r-- | 10.2 KB |
symlink.c | -rw-r--r-- | 417 bytes |
wbuf.c | -rw-r--r-- | 36.9 KB |
write.c | -rw-r--r-- | 21.0 KB |
writev.c | -rw-r--r-- | 1.1 KB |
xattr.c | -rw-r--r-- | 38.3 KB |
xattr.h | -rw-r--r-- | 4.1 KB |
xattr_trusted.c | -rw-r--r-- | 1.2 KB |
xattr_user.c | -rw-r--r-- | 1.1 KB |
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...