Revision 6e2df0581f569038719cf2bc2b3baa3fcc83cab4 authored by Peter Zijlstra on 08 November 2019, 10:11:52 UTC, committed by Peter Zijlstra on 08 November 2019, 21:34:14 UTC
Commit 67692435c411 ("sched: Rework pick_next_task() slow-path")
inadvertly introduced a race because it changed a previously
unexplored dependency between dropping the rq->lock and
sched_class::put_prev_task().
The comments about dropping rq->lock, in for example
newidle_balance(), only mentions the task being current and ->on_cpu
being set. But when we look at the 'change' pattern (in for example
sched_setnuma()):
queued = task_on_rq_queued(p); /* p->on_rq == TASK_ON_RQ_QUEUED */
running = task_current(rq, p); /* rq->curr == p */
if (queued)
dequeue_task(...);
if (running)
put_prev_task(...);
/* change task properties */
if (queued)
enqueue_task(...);
if (running)
set_next_task(...);
It becomes obvious that if we do this after put_prev_task() has
already been called on @p, things go sideways. This is exactly what
the commit in question allows to happen when it does:
prev->sched_class->put_prev_task(rq, prev, rf);
if (!rq->nr_running)
newidle_balance(rq, rf);
The newidle_balance() call will drop rq->lock after we've called
put_prev_task() and that allows the above 'change' pattern to
interleave and mess up the state.
Furthermore, it turns out we lost the RT-pull when we put the last DL
task.
Fix both problems by extracting the balancing from put_prev_task() and
doing a multi-class balance() pass before put_prev_task().
Fixes: 67692435c411 ("sched: Rework pick_next_task() slow-path")
Reported-by: Quentin Perret <qperret@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Quentin Perret <qperret@google.com>
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
1 parent e3b8b6a
scif_overview.rst
========================================
Symmetric Communication Interface (SCIF)
========================================
The Symmetric Communication Interface (SCIF (pronounced as skiff)) is a low
level communications API across PCIe currently implemented for MIC. Currently
SCIF provides inter-node communication within a single host platform, where a
node is a MIC Coprocessor or Xeon based host. SCIF abstracts the details of
communicating over the PCIe bus while providing an API that is symmetric
across all the nodes in the PCIe network. An important design objective for SCIF
is to deliver the maximum possible performance given the communication
abilities of the hardware. SCIF has been used to implement an offload compiler
runtime and OFED support for MPI implementations for MIC coprocessors.
SCIF API Components
===================
The SCIF API has the following parts:
1. Connection establishment using a client server model
2. Byte stream messaging intended for short messages
3. Node enumeration to determine online nodes
4. Poll semantics for detection of incoming connections and messages
5. Memory registration to pin down pages
6. Remote memory mapping for low latency CPU accesses via mmap
7. Remote DMA (RDMA) for high bandwidth DMA transfers
8. Fence APIs for RDMA synchronization
SCIF exposes the notion of a connection which can be used by peer processes on
nodes in a SCIF PCIe "network" to share memory "windows" and to communicate. A
process in a SCIF node initiates a SCIF connection to a peer process on a
different node via a SCIF "endpoint". SCIF endpoints support messaging APIs
which are similar to connection oriented socket APIs. Connected SCIF endpoints
can also register local memory which is followed by data transfer using either
DMA, CPU copies or remote memory mapping via mmap. SCIF supports both user and
kernel mode clients which are functionally equivalent.
SCIF Performance for MIC
========================
DMA bandwidth comparison between the TCP (over ethernet over PCIe) stack versus
SCIF shows the performance advantages of SCIF for HPC applications and
runtimes::
Comparison of TCP and SCIF based BW
Throughput (GB/sec)
8 + PCIe Bandwidth ******
+ TCP ######
7 + ************************************** SCIF %%%%%%
| %%%%%%%%%%%%%%%%%%%
6 + %%%%
| %%
| %%%
5 + %%
| %%
4 + %%
| %%
3 + %%
| %
2 + %%
| %%
| %
1 +
+ ######################################
0 +++---+++--+--+-+--+--+-++-+--+-++-+--+-++-+-
1 10 100 1000 10000 100000
Transfer Size (KBytes)
SCIF allows memory sharing via mmap(..) between processes on different PCIe
nodes and thus provides bare-metal PCIe latency. The round trip SCIF mmap
latency from the host to an x100 MIC for an 8 byte message is 0.44 usecs.
SCIF has a user space library which is a thin IOCTL wrapper providing a user
space API similar to the kernel API in scif.h. The SCIF user space library
is distributed @ https://software.intel.com/en-us/mic-developer
Here is some pseudo code for an example of how two applications on two PCIe
nodes would typically use the SCIF API::
Process A (on node A) Process B (on node B)
/* get online node information */
scif_get_node_ids(..) scif_get_node_ids(..)
scif_open(..) scif_open(..)
scif_bind(..) scif_bind(..)
scif_listen(..)
scif_accept(..) scif_connect(..)
/* SCIF connection established */
/* Send and receive short messages */
scif_send(..)/scif_recv(..) scif_send(..)/scif_recv(..)
/* Register memory */
scif_register(..) scif_register(..)
/* RDMA */
scif_readfrom(..)/scif_writeto(..) scif_readfrom(..)/scif_writeto(..)
/* Fence DMAs */
scif_fence_signal(..) scif_fence_signal(..)
mmap(..) mmap(..)
/* Access remote registered memory */
/* Close the endpoints */
scif_close(..) scif_close(..)
Computing file changes ...
