https://github.com/torvalds/linux
Revision 0bf3924bfabd13ba21aa702344fc00b3b3263e5a authored by Thomas Gleixner on 12 June 2020, 13:55:00 UTC, committed by Thomas Gleixner on 12 June 2020, 19:36:33 UTC
The idea of conditionally calling into rcu_irq_enter() only when RCU is
not watching turned out to be not completely thought through.

Paul noticed occasional premature end of grace periods in RCU torture
testing. Bisection led to the commit which made the invocation of
rcu_irq_enter() conditional on !rcu_is_watching().

It turned out that this conditional breaks RCU assumptions about the idle
task when the scheduler tick happens to be a nested interrupt. Nested
interrupts can happen when the first interrupt invokes softirq processing
on return which enables interrupts.

If that nested tick interrupt does not invoke rcu_irq_enter() then the
RCU's irq-nesting checks will believe that this interrupt came directly
from idle, which will cause RCU to report a quiescent state.  Because this
interrupt instead came from a softirq handler which might have been
executing an RCU read-side critical section, this can cause the grace
period to end prematurely.

Change the condition from !rcu_is_watching() to is_idle_task(current) which
enforces that interrupts in the idle task unconditionally invoke
rcu_irq_enter() independent of the RCU state.

This is also correct vs. user mode entries in NOHZ full scenarios because
user mode entries bring RCU out of EQS and force the RCU irq nesting state
accounting to nested. As only the first interrupt can enter from user mode
a nested tick interrupt will enter from kernel mode and as the nesting
state accounting is forced to nesting it will not do anything stupid even
if rcu_irq_enter() has not been invoked.

Fixes: 3eeec3858488 ("x86/entry: Provide idtentry_entry/exit_cond_rcu()")
Reported-by: "Paul E. McKenney" <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: "Paul E. McKenney" <paulmck@kernel.org>
Reviewed-by: "Paul E. McKenney" <paulmck@kernel.org>
Acked-by: Andy Lutomirski <luto@kernel.org>
Acked-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lkml.kernel.org/r/87wo4cxubv.fsf@nanos.tec.linutronix.de

1 parent 71ed49d
History
Tip revision: 0bf3924bfabd13ba21aa702344fc00b3b3263e5a authored by Thomas Gleixner on 12 June 2020, 13:55:00 UTC
x86/entry: Force rcu_irq_enter() when in idle task
Tip revision: 0bf3924
File Mode Size
Documentation
LICENSES
arch
block
certs
crypto
drivers
fs
include
init
ipc
kernel
lib
mm
net
samples
scripts
security
sound
tools
usr
virt
.clang-format -rw-r--r-- 15.8 KB
.cocciconfig -rw-r--r-- 59 bytes
.get_maintainer.ignore -rw-r--r-- 71 bytes
.gitattributes -rw-r--r-- 62 bytes
.gitignore -rw-r--r-- 1.8 KB
.mailmap -rw-r--r-- 15.4 KB
COPYING -rw-r--r-- 496 bytes
CREDITS -rw-r--r-- 97.4 KB
Kbuild -rw-r--r-- 1.3 KB
Kconfig -rw-r--r-- 555 bytes
MAINTAINERS -rw-r--r-- 546.7 KB
Makefile -rw-r--r-- 60.9 KB
README -rw-r--r-- 727 bytes

README

back to top