Revision 6e2df0581f569038719cf2bc2b3baa3fcc83cab4 authored by Peter Zijlstra on 08 November 2019, 10:11:52 UTC, committed by Peter Zijlstra on 08 November 2019, 21:34:14 UTC
Commit 67692435c411 ("sched: Rework pick_next_task() slow-path") inadvertly introduced a race because it changed a previously unexplored dependency between dropping the rq->lock and sched_class::put_prev_task(). The comments about dropping rq->lock, in for example newidle_balance(), only mentions the task being current and ->on_cpu being set. But when we look at the 'change' pattern (in for example sched_setnuma()): queued = task_on_rq_queued(p); /* p->on_rq == TASK_ON_RQ_QUEUED */ running = task_current(rq, p); /* rq->curr == p */ if (queued) dequeue_task(...); if (running) put_prev_task(...); /* change task properties */ if (queued) enqueue_task(...); if (running) set_next_task(...); It becomes obvious that if we do this after put_prev_task() has already been called on @p, things go sideways. This is exactly what the commit in question allows to happen when it does: prev->sched_class->put_prev_task(rq, prev, rf); if (!rq->nr_running) newidle_balance(rq, rf); The newidle_balance() call will drop rq->lock after we've called put_prev_task() and that allows the above 'change' pattern to interleave and mess up the state. Furthermore, it turns out we lost the RT-pull when we put the last DL task. Fix both problems by extracting the balancing from put_prev_task() and doing a multi-class balance() pass before put_prev_task(). Fixes: 67692435c411 ("sched: Rework pick_next_task() slow-path") Reported-by: Quentin Perret <qperret@google.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Quentin Perret <qperret@google.com> Tested-by: Valentin Schneider <valentin.schneider@arm.com>
1 parent e3b8b6a
dentry.c
// SPDX-License-Identifier: GPL-2.0
/*
* linux/fs/hpfs/dentry.c
*
* Mikulas Patocka (mikulas@artax.karlin.mff.cuni.cz), 1998-1999
*
* dcache operations
*/
#include "hpfs_fn.h"
/*
* Note: the dentry argument is the parent dentry.
*/
static int hpfs_hash_dentry(const struct dentry *dentry, struct qstr *qstr)
{
unsigned long hash;
int i;
unsigned l = qstr->len;
if (l == 1) if (qstr->name[0]=='.') goto x;
if (l == 2) if (qstr->name[0]=='.' || qstr->name[1]=='.') goto x;
hpfs_adjust_length(qstr->name, &l);
/*if (hpfs_chk_name(qstr->name,&l))*/
/*return -ENAMETOOLONG;*/
/*return -ENOENT;*/
x:
hash = init_name_hash(dentry);
for (i = 0; i < l; i++)
hash = partial_name_hash(hpfs_upcase(hpfs_sb(dentry->d_sb)->sb_cp_table,qstr->name[i]), hash);
qstr->hash = end_name_hash(hash);
return 0;
}
static int hpfs_compare_dentry(const struct dentry *dentry,
unsigned int len, const char *str, const struct qstr *name)
{
unsigned al = len;
unsigned bl = name->len;
hpfs_adjust_length(str, &al);
/*hpfs_adjust_length(b->name, &bl);*/
/*
* 'str' is the nane of an already existing dentry, so the name
* must be valid. 'name' must be validated first.
*/
if (hpfs_chk_name(name->name, &bl))
return 1;
if (hpfs_compare_names(dentry->d_sb, str, al, name->name, bl, 0))
return 1;
return 0;
}
const struct dentry_operations hpfs_dentry_operations = {
.d_hash = hpfs_hash_dentry,
.d_compare = hpfs_compare_dentry,
};
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...