https://github.com/torvalds/linux
Revision e4eed03fd06578571c01d4f1478c874bb432c815 authored by Andrea Arcangeli on 20 June 2012, 19:52:57 UTC, committed by Linus Torvalds on 20 June 2012, 21:39:35 UTC
In the x86 32bit PAE CONFIG_TRANSPARENT_HUGEPAGE=y case while holding the
mmap_sem for reading, cmpxchg8b cannot be used to read pmd contents under
Xen.

So instead of dealing only with "consistent" pmdvals in
pmd_none_or_trans_huge_or_clear_bad() (which would be conceptually
simpler) we let pmd_none_or_trans_huge_or_clear_bad() deal with pmdvals
where the low 32bit and high 32bit could be inconsistent (to avoid having
to use cmpxchg8b).

The only guarantee we get from pmd_read_atomic is that if the low part of
the pmd was found null, the high part will be null too (so the pmd will be
considered unstable).  And if the low part of the pmd is found "stable"
later, then it means the whole pmd was read atomically (because after a
pmd is stable, neither MADV_DONTNEED nor page faults can alter it anymore,
and we read the high part after the low part).

In the 32bit PAE x86 case, it is enough to read the low part of the pmdval
atomically to declare the pmd as "stable" and that's true for THP and no
THP, furthermore in the THP case we also have a barrier() that will
prevent any inconsistent pmdvals to be cached by a later re-read of the
*pmd.

Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Jonathan Nieder <jrnieder@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Cc: Larry Woodman <lwoodman@redhat.com>
Cc: Petr Matousek <pmatouse@redhat.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
Tested-by: Andrew Jones <drjones@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
1 parent abca7c4
History
Tip revision: e4eed03fd06578571c01d4f1478c874bb432c815 authored by Andrea Arcangeli on 20 June 2012, 19:52:57 UTC
thp: avoid atomic64_read in pmd_read_atomic for 32bit PAE
Tip revision: e4eed03
File Mode Size
ABI
DocBook
EDID
PCI
RCU
accounting
acpi
aoe
arm
auxdisplay
backlight
blackfin
block
blockdev
cdrom
cgroups
connector
console
cpu-freq
cpuidle
cris
crypto
development-process
device-mapper
devicetree
driver-model
dvb
early-userspace
extcon
fault-injection
fb
filesystems
firmware_class
frv
hid
hwmon
i2c
i2o
ia64
ide
infiniband
input
ioctl
isdn
ja_JP
kbuild
kdump
ko_KR
laptops
leds
m68k
make
memory-devices
mips
misc-devices
mmc
mn10300
mtd
namespaces
netlabel
networking
nfc
parisc
pcmcia
power
powerpc
pps
prctl
pti
ptp
rapidio
s390
scheduler
scsi
security
serial
sh
sound
spi
sysctl
target
telephony
thermal
timers
trace
usb
vDSO
video4linux
virtual
vm
w1
watchdog
wimax
x86
zh_CN
.gitignore -rw-r--r-- 107 bytes
00-INDEX -rw-r--r-- 11.8 KB
BUG-HUNTING -rw-r--r-- 8.1 KB
Changes -rw-r--r-- 11.7 KB
CodingStyle -rw-r--r-- 30.6 KB
DMA-API-HOWTO.txt -rw-r--r-- 28.0 KB
DMA-API.txt -rw-r--r-- 26.7 KB
DMA-ISA-LPC.txt -rw-r--r-- 5.2 KB
DMA-attributes.txt -rw-r--r-- 1.9 KB
HOWTO -rw-r--r-- 27.2 KB
IPMI.txt -rw-r--r-- 28.5 KB
IRQ-affinity.txt -rw-r--r-- 2.5 KB
IRQ-domain.txt -rw-r--r-- 5.1 KB
IRQ.txt -rw-r--r-- 962 bytes
Intel-IOMMU.txt -rw-r--r-- 3.8 KB
Makefile -rw-r--r-- 174 bytes
ManagementStyle -rw-r--r-- 12.9 KB
SAK.txt -rw-r--r-- 2.8 KB
SM501.txt -rw-r--r-- 2.8 KB
SecurityBugs -rw-r--r-- 1.8 KB
SubmitChecklist -rw-r--r-- 4.4 KB
SubmittingDrivers -rw-r--r-- 6.3 KB
SubmittingPatches -rw-r--r-- 28.6 KB
VGA-softcursor.txt -rw-r--r-- 2.0 KB
applying-patches.txt -rw-r--r-- 19.5 KB
atomic_ops.txt -rw-r--r-- 21.6 KB
bad_memory.txt -rw-r--r-- 1.1 KB
basic_profiling.txt -rw-r--r-- 1.7 KB
binfmt_misc.txt -rw-r--r-- 5.9 KB
braille-console.txt -rw-r--r-- 1.4 KB
bt8xxgpio.txt -rw-r--r-- 4.3 KB
btmrvl.txt -rw-r--r-- 2.9 KB
bus-virt-phys-mapping.txt -rw-r--r-- 7.9 KB
cachetlb.txt -rw-r--r-- 17.1 KB
circular-buffers.txt -rw-r--r-- 7.6 KB
clk.txt -rw-r--r-- 8.5 KB
coccinelle.txt -rw-r--r-- 8.1 KB
cpu-hotplug.txt -rw-r--r-- 14.6 KB
cpu-load.txt -rw-r--r-- 3.0 KB
cputopology.txt -rw-r--r-- 3.8 KB
crc32.txt -rw-r--r-- 8.5 KB
dcdbas.txt -rw-r--r-- 3.6 KB
debugging-modules.txt -rw-r--r-- 954 bytes
debugging-via-ohci1394.txt -rw-r--r-- 7.4 KB
dell_rbu.txt -rw-r--r-- 4.9 KB
devices.txt -rw-r--r-- 115.8 KB
digsig.txt -rw-r--r-- 2.8 KB
dma-buf-sharing.txt -rw-r--r-- 19.5 KB
dmaengine.txt -rw-r--r-- 7.6 KB
dontdiff -rw-r--r-- 2.5 KB
dynamic-debug-howto.txt -rw-r--r-- 11.8 KB
edac.txt -rw-r--r-- 26.9 KB
eisa.txt -rw-r--r-- 7.1 KB
email-clients.txt -rw-r--r-- 8.6 KB
feature-removal-schedule.txt -rw-r--r-- 22.1 KB
flexible-arrays.txt -rw-r--r-- 5.5 KB
futex-requeue-pi.txt -rw-r--r-- 5.0 KB
gcov.txt -rw-r--r-- 7.5 KB
gpio.txt -rw-r--r-- 32.4 KB
highuid.txt -rw-r--r-- 2.4 KB
hw_random.txt -rw-r--r-- 3.5 KB
hwspinlock.txt -rw-r--r-- 12.1 KB
init.txt -rw-r--r-- 2.5 KB
initrd.txt -rw-r--r-- 14.1 KB
intel_txt.txt -rw-r--r-- 10.2 KB
io-mapping.txt -rw-r--r-- 3.2 KB
io_ordering.txt -rw-r--r-- 1.9 KB
iostats.txt -rw-r--r-- 7.9 KB
irqflags-tracing.txt -rw-r--r-- 2.6 KB
isapnp.txt -rw-r--r-- 433 bytes
java.txt -rw-r--r-- 10.7 KB
kernel-doc-nano-HOWTO.txt -rw-r--r-- 11.4 KB
kernel-docs.txt -rw-r--r-- 33.1 KB
kernel-parameters.txt -rw-r--r-- 106.3 KB
kmemcheck.txt -rw-r--r-- 29.8 KB
kmemleak.txt -rw-r--r-- 7.9 KB
kobject.txt -rw-r--r-- 17.6 KB
kprobes.txt -rw-r--r-- 29.5 KB
kref.txt -rw-r--r-- 6.1 KB
ldm.txt -rw-r--r-- 3.8 KB
local_ops.txt -rw-r--r-- 6.1 KB
lockdep-design.txt -rw-r--r-- 11.6 KB
lockstat.txt -rw-r--r-- 10.7 KB
lockup-watchdogs.txt -rw-r--r-- 3.1 KB
logo.gif -rw-r--r-- 16.0 KB
logo.txt -rw-r--r-- 563 bytes
magic-number.txt -rw-r--r-- 9.7 KB
md.txt -rw-r--r-- 25.4 KB
media-framework.txt -rw-r--r-- 14.7 KB
memory-barriers.txt -rw-r--r-- 82.3 KB
memory-hotplug.txt -rw-r--r-- 15.0 KB
memory.txt -rw-r--r-- 1.2 KB
mono.txt -rw-r--r-- 2.5 KB
mutex-design.txt -rw-r--r-- 5.8 KB
nommu-mmap.txt -rw-r--r-- 12.7 KB
numastat.txt -rw-r--r-- 836 bytes
oops-tracing.txt -rw-r--r-- 12.5 KB
padata.txt -rw-r--r-- 7.3 KB
parport-lowlevel.txt -rw-r--r-- 32.2 KB
parport.txt -rw-r--r-- 8.8 KB
pi-futex.txt -rw-r--r-- 5.7 KB
pinctrl.txt -rw-r--r-- 37.5 KB
pnp.txt -rw-r--r-- 6.8 KB
preempt-locking.txt -rw-r--r-- 5.2 KB
printk-formats.txt -rw-r--r-- 4.9 KB
prio_tree.txt -rw-r--r-- 5.2 KB
ramoops.txt -rw-r--r-- 2.7 KB
rbtree.txt -rw-r--r-- 9.1 KB
remoteproc.txt -rw-r--r-- 13.8 KB
rfkill.txt -rw-r--r-- 4.7 KB
robust-futex-ABI.txt -rw-r--r-- 8.7 KB
robust-futexes.txt -rw-r--r-- 9.4 KB
rpmsg.txt -rw-r--r-- 13.6 KB
rt-mutex-design.txt -rw-r--r-- 32.8 KB
rt-mutex.txt -rw-r--r-- 3.5 KB
rtc.txt -rw-r--r-- 15.5 KB
serial-console.txt -rw-r--r-- 4.0 KB
sgi-ioc4.txt -rw-r--r-- 2.0 KB
sgi-visws.txt -rw-r--r-- 678 bytes
sparse.txt -rw-r--r-- 3.0 KB
spinlocks.txt -rw-r--r-- 6.5 KB
stable_api_nonsense.txt -rw-r--r-- 9.2 KB
stable_kernel_rules.txt -rw-r--r-- 3.7 KB
static-keys.txt -rw-r--r-- 11.9 KB
svga.txt -rw-r--r-- 14.1 KB
sysfs-rules.txt -rw-r--r-- 8.1 KB
sysrq.txt -rw-r--r-- 11.5 KB
unaligned-memory-access.txt -rw-r--r-- 10.0 KB
unicode.txt -rw-r--r-- 6.5 KB
unshare.txt -rw-r--r-- 13.1 KB
vgaarbiter.txt -rw-r--r-- 8.1 KB
video-output.txt -rw-r--r-- 1.1 KB
vme_api.txt -rw-r--r-- 13.1 KB
volatile-considered-harmful.txt -rw-r--r-- 5.6 KB
workqueue.txt -rw-r--r-- 15.6 KB
xz.txt -rw-r--r-- 5.7 KB
zorro.txt -rw-r--r-- 2.8 KB

back to top