Revision e2093926a098a8ccf0f1d10f6df8dad452cb28d3 authored by Ross Zwisler on 02 June 2017, 21:46:37 UTC, committed by Linus Torvalds on 02 June 2017, 22:07:37 UTC
We currently have two related PMD vs PTE races in the DAX code. These can both be easily triggered by having two threads reading and writing simultaneously to the same private mapping, with the key being that private mapping reads can be handled with PMDs but private mapping writes are always handled with PTEs so that we can COW. Here is the first race: CPU 0 CPU 1 (private mapping write) __handle_mm_fault() create_huge_pmd() - FALLBACK handle_pte_fault() passes check for pmd_devmap() (private mapping read) __handle_mm_fault() create_huge_pmd() dax_iomap_pmd_fault() inserts PMD dax_iomap_pte_fault() does a PTE fault, but we already have a DAX PMD installed in our page tables at this spot. Here's the second race: CPU 0 CPU 1 (private mapping read) __handle_mm_fault() passes check for pmd_none() create_huge_pmd() dax_iomap_pmd_fault() inserts PMD (private mapping write) __handle_mm_fault() create_huge_pmd() - FALLBACK (private mapping read) __handle_mm_fault() passes check for pmd_none() create_huge_pmd() handle_pte_fault() dax_iomap_pte_fault() inserts PTE dax_iomap_pmd_fault() inserts PMD, but we already have a PTE at this spot. The core of the issue is that while there is isolation between faults to the same range in the DAX fault handlers via our DAX entry locking, there is no isolation between faults in the code in mm/memory.c. This means for instance that this code in __handle_mm_fault() can run: if (pmd_none(*vmf.pmd) && transparent_hugepage_enabled(vma)) { ret = create_huge_pmd(&vmf); But by the time we actually get to run the fault handler called by create_huge_pmd(), the PMD is no longer pmd_none() because a racing PTE fault has installed a normal PMD here as a parent. This is the cause of the 2nd race. The first race is similar - there is the following check in handle_pte_fault(): } else { /* See comment in pte_alloc_one_map() */ if (pmd_devmap(*vmf->pmd) || pmd_trans_unstable(vmf->pmd)) return 0; So if a pmd_devmap() PMD (a DAX PMD) has been installed at vmf->pmd, we will bail and retry the fault. This is correct, but there is nothing preventing the PMD from being installed after this check but before we actually get to the DAX PTE fault handlers. In my testing these races result in the following types of errors: BUG: Bad rss-counter state mm:ffff8800a817d280 idx:1 val:1 BUG: non-zero nr_ptes on freeing mm: 15 Fix this issue by having the DAX fault handlers verify that it is safe to continue their fault after they have taken an entry lock to block other racing faults. [ross.zwisler@linux.intel.com: improve fix for colliding PMD & PTE entries] Link: http://lkml.kernel.org/r/20170526195932.32178-1-ross.zwisler@linux.intel.com Link: http://lkml.kernel.org/r/20170522215749.23516-2-ross.zwisler@linux.intel.com Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> Reported-by: Pawel Lebioda <pawel.lebioda@intel.com> Reviewed-by: Jan Kara <jack@suse.cz> Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: Alexander Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@lst.de> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Matthew Wilcox <mawilcox@microsoft.com> Cc: "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com> Cc: Pawel Lebioda <pawel.lebioda@intel.com> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Xiong Zhou <xzhou@redhat.com> Cc: Eryu Guan <eguan@redhat.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
1 parent d0f0931
patch-kernel
#! /bin/sh
# Script to apply kernel patches.
# usage: patch-kernel [ sourcedir [ patchdir [ stopversion ] [ -acxx ] ] ]
# The source directory defaults to /usr/src/linux, and the patch
# directory defaults to the current directory.
# e.g.
# scripts/patch-kernel . ..
# Update the kernel tree in the current directory using patches in the
# directory above to the latest Linus kernel
# scripts/patch-kernel . .. -ac
# Get the latest Linux kernel and patch it with the latest ac patch
# scripts/patch-kernel . .. 2.4.9
# Gets standard kernel 2.4.9
# scripts/patch-kernel . .. 2.4.9 -ac
# Gets 2.4.9 with latest ac patches
# scripts/patch-kernel . .. 2.4.9 -ac11
# Gets 2.4.9 with ac patch ac11
# Note: It uses the patches relative to the Linus kernels, not the
# ac to ac relative patches
#
# It determines the current kernel version from the top-level Makefile.
# It then looks for patches for the next sublevel in the patch directory.
# This is applied using "patch -p1 -s" from within the kernel directory.
# A check is then made for "*.rej" files to see if the patch was
# successful. If it is, then all of the "*.orig" files are removed.
#
# Nick Holloway <Nick.Holloway@alfie.demon.co.uk>, 2nd January 1995.
#
# Added support for handling multiple types of compression. What includes
# gzip, bzip, bzip2, zip, compress, and plaintext.
#
# Adam Sulmicki <adam@cfar.umd.edu>, 1st January 1997.
#
# Added ability to stop at a given version number
# Put the full version number (i.e. 2.3.31) as the last parameter
# Dave Gilbert <linux@treblig.org>, 11th December 1999.
# Fixed previous patch so that if we are already at the correct version
# not to patch up.
#
# Added -ac option, use -ac or -ac9 (say) to stop at a particular version
# Dave Gilbert <linux@treblig.org>, 29th September 2001.
#
# Add support for (use of) EXTRAVERSION (to support 2.6.8.x, e.g.);
# update usage message;
# fix some whitespace damage;
# be smarter about stopping when current version is larger than requested;
# Randy Dunlap <rdunlap@xenotime.net>, 2004-AUG-18.
#
# Add better support for (non-incremental) 2.6.x.y patches;
# If an ending version number if not specified, the script automatically
# increments the SUBLEVEL (x in 2.6.x.y) until no more patch files are found;
# however, EXTRAVERSION (y in 2.6.x.y) is never automatically incremented
# but must be specified fully.
#
# patch-kernel does not normally support reverse patching, but does so when
# applying EXTRAVERSION (x.y) patches, so that moving from 2.6.11.y to 2.6.11.z
# is easy and handled by the script (reverse 2.6.11.y and apply 2.6.11.z).
# Randy Dunlap <rdunlap@xenotime.net>, 2005-APR-08.
PNAME=patch-kernel
# Set directories from arguments, or use defaults.
sourcedir=${1-/usr/src/linux}
patchdir=${2-.}
stopvers=${3-default}
if [ "$1" = -h -o "$1" = --help -o ! -r "$sourcedir/Makefile" ]; then
cat << USAGE
usage: $PNAME [-h] [ sourcedir [ patchdir [ stopversion ] [ -acxx ] ] ]
source directory defaults to /usr/src/linux,
patch directory defaults to the current directory,
stopversion defaults to <all in patchdir>.
USAGE
exit 1
fi
# See if we have any -ac options
for PARM in $*
do
case $PARM in
-ac*)
gotac=$PARM;
esac;
done
# ---------------------------------------------------------------------------
# arg1 is filename
noFile () {
echo "cannot find patch file: ${patch}"
exit 1
}
# ---------------------------------------------------------------------------
backwards () {
echo "$PNAME does not support reverse patching"
exit 1
}
# ---------------------------------------------------------------------------
# Find a file, first parameter is basename of file
# it tries many compression mechanisms and sets variables to say how to get it
findFile () {
filebase=$1;
if [ -r ${filebase}.gz ]; then
ext=".gz"
name="gzip"
uncomp="gunzip -dc"
elif [ -r ${filebase}.bz ]; then
ext=".bz"
name="bzip"
uncomp="bunzip -dc"
elif [ -r ${filebase}.bz2 ]; then
ext=".bz2"
name="bzip2"
uncomp="bunzip2 -dc"
elif [ -r ${filebase}.xz ]; then
ext=".xz"
name="xz"
uncomp="xz -dc"
elif [ -r ${filebase}.zip ]; then
ext=".zip"
name="zip"
uncomp="unzip -d"
elif [ -r ${filebase}.Z ]; then
ext=".Z"
name="uncompress"
uncomp="uncompress -c"
elif [ -r ${filebase} ]; then
ext=""
name="plaintext"
uncomp="cat"
else
return 1;
fi
return 0;
}
# ---------------------------------------------------------------------------
# Apply a patch and check it goes in cleanly
# First param is patch name (e.g. patch-2.4.9-ac5) - without path or extension
applyPatch () {
echo -n "Applying $1 (${name})... "
if $uncomp ${patchdir}/$1${ext} | patch -p1 -s -N -E -d $sourcedir
then
echo "done."
else
echo "failed. Clean up yourself."
return 1;
fi
if [ "`find $sourcedir/ '(' -name '*.rej' -o -name '.*.rej' ')' -print`" ]
then
echo "Aborting. Reject files found."
return 1;
fi
# Remove backup files
find $sourcedir/ '(' -name '*.orig' -o -name '.*.orig' ')' -exec rm -f {} \;
return 0;
}
# ---------------------------------------------------------------------------
# arg1 is patch filename
reversePatch () {
echo -n "Reversing $1 (${name}) ... "
if $uncomp ${patchdir}/"$1"${ext} | patch -p1 -Rs -N -E -d $sourcedir
then
echo "done."
else
echo "failed. Clean it up."
exit 1
fi
if [ "`find $sourcedir/ '(' -name '*.rej' -o -name '.*.rej' ')' -print`" ]
then
echo "Aborting. Reject files found."
return 1
fi
# Remove backup files
find $sourcedir/ '(' -name '*.orig' -o -name '.*.orig' ')' -exec rm -f {} \;
return 0
}
# set current VERSION, PATCHLEVEL, SUBLEVEL, EXTRAVERSION
# force $TMPFILEs below to be in local directory: a slash character prevents
# the dot command from using the search path.
TMPFILE=`mktemp ./.tmpver.XXXXXX` || { echo "cannot make temp file" ; exit 1; }
grep -E "^(VERSION|PATCHLEVEL|SUBLEVEL|EXTRAVERSION)" $sourcedir/Makefile > $TMPFILE
tr -d [:blank:] < $TMPFILE > $TMPFILE.1
. $TMPFILE.1
rm -f $TMPFILE*
if [ -z "$VERSION" -o -z "$PATCHLEVEL" -o -z "$SUBLEVEL" ]
then
echo "unable to determine current kernel version" >&2
exit 1
fi
NAME=`grep ^NAME $sourcedir/Makefile`
NAME=${NAME##*=}
echo "Current kernel version is $VERSION.$PATCHLEVEL.$SUBLEVEL${EXTRAVERSION} ($NAME)"
# strip EXTRAVERSION to just a number (drop leading '.' and trailing additions)
EXTRAVER=
if [ x$EXTRAVERSION != "x" ]
then
EXTRAVER=${EXTRAVERSION#.}
EXTRAVER=${EXTRAVER%%[[:punct:]]*}
#echo "$PNAME: changing EXTRAVERSION from $EXTRAVERSION to $EXTRAVER"
fi
#echo "stopvers=$stopvers"
if [ $stopvers != "default" ]; then
STOPSUBLEVEL=`echo $stopvers | cut -d. -f3`
STOPEXTRA=`echo $stopvers | cut -d. -f4`
STOPFULLVERSION=${stopvers%%.$STOPEXTRA}
#echo "#___STOPSUBLEVEL=/$STOPSUBLEVEL/, STOPEXTRA=/$STOPEXTRA/"
else
STOPSUBLEVEL=9999
STOPEXTRA=9999
fi
# This all assumes a 2.6.x[.y] kernel tree.
# Don't allow backwards/reverse patching.
if [ $STOPSUBLEVEL -lt $SUBLEVEL ]; then
backwards
fi
if [ x$EXTRAVER != "x" ]; then
CURRENTFULLVERSION="$VERSION.$PATCHLEVEL.$SUBLEVEL.$EXTRAVER"
else
CURRENTFULLVERSION="$VERSION.$PATCHLEVEL.$SUBLEVEL"
fi
if [ x$EXTRAVER != "x" ]; then
echo "backing up to: $VERSION.$PATCHLEVEL.$SUBLEVEL"
patch="patch-${CURRENTFULLVERSION}"
findFile $patchdir/${patch} || noFile ${patch}
reversePatch ${patch} || exit 1
fi
# now current is 2.6.x, with no EXTRA applied,
# so update to target SUBLEVEL (2.6.SUBLEVEL)
# and then to target EXTRAVER (2.6.SUB.EXTRAVER) if requested.
# If not ending sublevel is specified, it is incremented until
# no further sublevels are found.
if [ $STOPSUBLEVEL -gt $SUBLEVEL ]; then
while : # incrementing SUBLEVEL (s in v.p.s)
do
CURRENTFULLVERSION="$VERSION.$PATCHLEVEL.$SUBLEVEL"
EXTRAVER=
if [ x$STOPFULLVERSION = x$CURRENTFULLVERSION ]; then
echo "Stopping at $CURRENTFULLVERSION base as requested."
break
fi
SUBLEVEL=$(($SUBLEVEL + 1))
FULLVERSION="$VERSION.$PATCHLEVEL.$SUBLEVEL"
#echo "#___ trying $FULLVERSION ___"
if [ $(($SUBLEVEL)) -gt $(($STOPSUBLEVEL)) ]; then
echo "Stopping since sublevel ($SUBLEVEL) is beyond stop-sublevel ($STOPSUBLEVEL)"
exit 1
fi
patch=patch-$FULLVERSION
# See if the file exists and find extension
findFile $patchdir/${patch} || noFile ${patch}
# Apply the patch and check all is OK
applyPatch $patch || break
done
#echo "#___sublevel all done"
fi
# There is no incremental searching for extraversion...
if [ "$STOPEXTRA" != "" ]; then
while : # just to allow break
do
# apply STOPEXTRA directly (not incrementally) (x in v.p.s.x)
FULLVERSION="$VERSION.$PATCHLEVEL.$SUBLEVEL.$STOPEXTRA"
#echo "#... trying $FULLVERSION ..."
patch=patch-$FULLVERSION
# See if the file exists and find extension
findFile $patchdir/${patch} || noFile ${patch}
# Apply the patch and check all is OK
applyPatch $patch || break
#echo "#___extraver all done"
break
done
fi
if [ x$gotac != x ]; then
# Out great user wants the -ac patches
# They could have done -ac (get latest) or -acxx where xx=version they want
if [ $gotac = "-ac" ]; then
# They want the latest version
HIGHESTPATCH=0
for PATCHNAMES in $patchdir/patch-${CURRENTFULLVERSION}-ac*\.*
do
ACVALUE=`echo $PATCHNAMES | sed -e 's/^.*patch-[0-9.]*-ac\([0-9]*\).*/\1/'`
# Check it is actually a recognised patch type
findFile $patchdir/patch-${CURRENTFULLVERSION}-ac${ACVALUE} || break
if [ $ACVALUE -gt $HIGHESTPATCH ]; then
HIGHESTPATCH=$ACVALUE
fi
done
if [ $HIGHESTPATCH -ne 0 ]; then
findFile $patchdir/patch-${CURRENTFULLVERSION}-ac${HIGHESTPATCH} || break
applyPatch patch-${CURRENTFULLVERSION}-ac${HIGHESTPATCH}
else
echo "No -ac patches found"
fi
else
# They want an exact version
findFile $patchdir/patch-${CURRENTFULLVERSION}${gotac} || {
echo "Sorry, I couldn't find the $gotac patch for $CURRENTFULLVERSION. Hohum."
exit 1
}
applyPatch patch-${CURRENTFULLVERSION}${gotac}
fi
fi
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...