Skip to main content

Browse the archive

https://github.com/torvalds/linux
23 October 2021, 00:29:24 UTC
Revision 928501344fc645f80390afc12708c81b3595745d authored by Andrey Konovalov on 26 February 2021, 01:19:55 UTC, committed by Linus Torvalds on 26 February 2021, 17:41:02 UTC 
kasan, mm: don't save alloc stacks twice
 
Patch series "kasan: optimizations and fixes for HW_TAGS", v4.

This patchset makes the HW_TAGS mode more efficient, mostly by reworking
poisoning approaches and simplifying/inlining some internal helpers.

With this change, the overhead of HW_TAGS annotations excluding setting
and checking memory tags is ~3%.  The performance impact caused by tags
will be unknown until we have hardware that supports MTE.

As a side-effect, this patchset speeds up generic KASAN by ~15%.

This patch (of 13):

Currently KASAN saves allocation stacks in both kasan_slab_alloc() and
kasan_kmalloc() annotations.  This patch changes KASAN to save allocation
stacks for slab objects from kmalloc caches in kasan_kmalloc() only, and
stacks for other slab objects in kasan_slab_alloc() only.

This change requires ____kasan_kmalloc() knowing whether the object
belongs to a kmalloc cache.  This is implemented by adding a flag field to
the kasan_info structure.  That flag is only set for kmalloc caches via a
new kasan_cache_create_kmalloc() annotation.

Link: https://lkml.kernel.org/r/cover.1612546384.git.andreyknvl@google.com
Link: https://lkml.kernel.org/r/7c673ebca8d00f40a7ad6f04ab9a2bddeeae2097.1612546384.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
1 parent d3a61f7
Raw File
Tip revision: 928501344fc645f80390afc12708c81b3595745d authored by Andrey Konovalov on 26 February 2021, 01:19:55 UTC
kasan, mm: don't save alloc stacks twice
Tip revision: 9285013
blake2b.h 
/* SPDX-License-Identifier: GPL-2.0 OR MIT */

#ifndef _CRYPTO_BLAKE2B_H
#define _CRYPTO_BLAKE2B_H

#include <linux/bug.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>

enum blake2b_lengths {
	BLAKE2B_BLOCK_SIZE = 128,
	BLAKE2B_HASH_SIZE = 64,
	BLAKE2B_KEY_SIZE = 64,

	BLAKE2B_160_HASH_SIZE = 20,
	BLAKE2B_256_HASH_SIZE = 32,
	BLAKE2B_384_HASH_SIZE = 48,
	BLAKE2B_512_HASH_SIZE = 64,
};

struct blake2b_state {
	/* 'h', 't', and 'f' are used in assembly code, so keep them as-is. */
	u64 h[8];
	u64 t[2];
	u64 f[2];
	u8 buf[BLAKE2B_BLOCK_SIZE];
	unsigned int buflen;
	unsigned int outlen;
};

enum blake2b_iv {
	BLAKE2B_IV0 = 0x6A09E667F3BCC908ULL,
	BLAKE2B_IV1 = 0xBB67AE8584CAA73BULL,
	BLAKE2B_IV2 = 0x3C6EF372FE94F82BULL,
	BLAKE2B_IV3 = 0xA54FF53A5F1D36F1ULL,
	BLAKE2B_IV4 = 0x510E527FADE682D1ULL,
	BLAKE2B_IV5 = 0x9B05688C2B3E6C1FULL,
	BLAKE2B_IV6 = 0x1F83D9ABFB41BD6BULL,
	BLAKE2B_IV7 = 0x5BE0CD19137E2179ULL,
};

static inline void __blake2b_init(struct blake2b_state *state, size_t outlen,
				  const void *key, size_t keylen)
{
	state->h[0] = BLAKE2B_IV0 ^ (0x01010000 | keylen << 8 | outlen);
	state->h[1] = BLAKE2B_IV1;
	state->h[2] = BLAKE2B_IV2;
	state->h[3] = BLAKE2B_IV3;
	state->h[4] = BLAKE2B_IV4;
	state->h[5] = BLAKE2B_IV5;
	state->h[6] = BLAKE2B_IV6;
	state->h[7] = BLAKE2B_IV7;
	state->t[0] = 0;
	state->t[1] = 0;
	state->f[0] = 0;
	state->f[1] = 0;
	state->buflen = 0;
	state->outlen = outlen;
	if (keylen) {
		memcpy(state->buf, key, keylen);
		memset(&state->buf[keylen], 0, BLAKE2B_BLOCK_SIZE - keylen);
		state->buflen = BLAKE2B_BLOCK_SIZE;
	}
}

#endif /* _CRYPTO_BLAKE2B_H */
back to top