Revision 6e2df0581f569038719cf2bc2b3baa3fcc83cab4 authored by Peter Zijlstra on 08 November 2019, 10:11:52 UTC, committed by Peter Zijlstra on 08 November 2019, 21:34:14 UTC
Commit 67692435c411 ("sched: Rework pick_next_task() slow-path") inadvertly introduced a race because it changed a previously unexplored dependency between dropping the rq->lock and sched_class::put_prev_task(). The comments about dropping rq->lock, in for example newidle_balance(), only mentions the task being current and ->on_cpu being set. But when we look at the 'change' pattern (in for example sched_setnuma()): queued = task_on_rq_queued(p); /* p->on_rq == TASK_ON_RQ_QUEUED */ running = task_current(rq, p); /* rq->curr == p */ if (queued) dequeue_task(...); if (running) put_prev_task(...); /* change task properties */ if (queued) enqueue_task(...); if (running) set_next_task(...); It becomes obvious that if we do this after put_prev_task() has already been called on @p, things go sideways. This is exactly what the commit in question allows to happen when it does: prev->sched_class->put_prev_task(rq, prev, rf); if (!rq->nr_running) newidle_balance(rq, rf); The newidle_balance() call will drop rq->lock after we've called put_prev_task() and that allows the above 'change' pattern to interleave and mess up the state. Furthermore, it turns out we lost the RT-pull when we put the last DL task. Fix both problems by extracting the balancing from put_prev_task() and doing a multi-class balance() pass before put_prev_task(). Fixes: 67692435c411 ("sched: Rework pick_next_task() slow-path") Reported-by: Quentin Perret <qperret@google.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Quentin Perret <qperret@google.com> Tested-by: Valentin Schneider <valentin.schneider@arm.com>
1 parent e3b8b6a
File | Mode | Size |
---|---|---|
LSM | ||
acpi | ||
aoe | ||
auxdisplay | ||
blockdev | ||
cgroup-v1 | ||
cifs | ||
device-mapper | ||
gpio | ||
hw-vuln | ||
kdump | ||
laptops | ||
mm | ||
namespaces | ||
perf | ||
pm | ||
sysctl | ||
wimax | ||
README.rst | -rw-r--r-- | 18.0 KB |
bcache.rst | -rw-r--r-- | 23.7 KB |
binderfs.rst | -rw-r--r-- | 2.9 KB |
binfmt-misc.rst | -rw-r--r-- | 7.2 KB |
braille-console.rst | -rw-r--r-- | 1.7 KB |
btmrvl.rst | -rw-r--r-- | 2.7 KB |
bug-bisect.rst | -rw-r--r-- | 2.1 KB |
bug-hunting.rst | -rw-r--r-- | 15.0 KB |
cgroup-v2.rst | -rw-r--r-- | 94.0 KB |
clearing-warn-once.rst | -rw-r--r-- | 275 bytes |
cpu-load.rst | -rw-r--r-- | 3.0 KB |
cputopology.rst | -rw-r--r-- | 5.7 KB |
devices.rst | -rw-r--r-- | 11.9 KB |
devices.txt | -rw-r--r-- | 104.6 KB |
dynamic-debug-howto.rst | -rw-r--r-- | 13.0 KB |
efi-stub.rst | -rw-r--r-- | 3.8 KB |
ext4.rst | -rw-r--r-- | 26.8 KB |
highuid.rst | -rw-r--r-- | 2.6 KB |
hw_random.rst | -rw-r--r-- | 3.8 KB |
index.rst | -rw-r--r-- | 2.2 KB |
init.rst | -rw-r--r-- | 2.6 KB |
initrd.rst | -rw-r--r-- | 14.3 KB |
iostats.rst | -rw-r--r-- | 8.9 KB |
java.rst | -rw-r--r-- | 10.9 KB |
jfs.rst | -rw-r--r-- | 2.6 KB |
kernel-parameters.rst | -rw-r--r-- | 8.0 KB |
kernel-parameters.txt | -rw-r--r-- | 188.1 KB |
kernel-per-CPU-kthreads.rst | -rw-r--r-- | 13.3 KB |
lcd-panel-cgram.rst | -rw-r--r-- | 1.2 KB |
ldm.rst | -rw-r--r-- | 4.6 KB |
lockup-watchdogs.rst | -rw-r--r-- | 4.1 KB |
md.rst | -rw-r--r-- | 27.0 KB |
module-signing.rst | -rw-r--r-- | 11.1 KB |
mono.rst | -rw-r--r-- | 2.5 KB |
numastat.rst | -rw-r--r-- | 1.0 KB |
parport.rst | -rw-r--r-- | 9.3 KB |
perf-security.rst | -rw-r--r-- | 10.3 KB |
pnp.rst | -rw-r--r-- | 7.0 KB |
ramoops.rst | -rw-r--r-- | 5.7 KB |
rapidio.rst | -rw-r--r-- | 2.2 KB |
ras.rst | -rw-r--r-- | 41.7 KB |
reporting-bugs.rst | -rw-r--r-- | 7.5 KB |
rtc.rst | -rw-r--r-- | 7.0 KB |
security-bugs.rst | -rw-r--r-- | 3.9 KB |
serial-console.rst | -rw-r--r-- | 4.3 KB |
svga.rst | -rw-r--r-- | 11.8 KB |
sysfs-rules.rst | -rw-r--r-- | 9.4 KB |
sysrq.rst | -rw-r--r-- | 12.1 KB |
tainted-kernels.rst | -rw-r--r-- | 7.6 KB |
thunderbolt.rst | -rw-r--r-- | 11.3 KB |
ufs.rst | -rw-r--r-- | 1.3 KB |
unicode.rst | -rw-r--r-- | 7.1 KB |
vga-softcursor.rst | -rw-r--r-- | 1.9 KB |
video-output.rst | -rw-r--r-- | 1.1 KB |
xfs.rst | -rw-r--r-- | 17.8 KB |
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...