Commit graph

975,215 commits

Author SHA1 Message Date
Vlastimil Babka
e871c7feeb FROMGIT: mm, page_poison: use static key more efficiently
Commit 11c9c7edae ("mm/page_poison.c: replace bool variable with static
key") changed page_poisoning_enabled() to a static key check.  However,
the function is not inlined, so each check still involves a function call
with overhead not eliminated when page poisoning is disabled.

Analogically to how debug_pagealloc is handled, this patch converts
page_poisoning_enabled() back to boolean check, and introduces
page_poisoning_enabled_static() for fast paths.  Both functions are
inlined.

The function kernel_poison_pages() is also called unconditionally and does
the static key check inside.  Remove it from there and put it to callers.
Also split it to two functions kernel_poison_pages() and
kernel_unpoison_pages() instead of the confusing bool parameter.

Also optimize the check that enables page poisoning instead of
debug_pagealloc for architectures without proper debug_pagealloc support.
Move the check to init_mem_debugging_and_hardening() to enable a single
static key instead of having two static branches in
page_poisoning_enabled_static().

Link: https://lkml.kernel.org/r/20201113104033.22907-3-vbabka@suse.cz
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: David Hildenbrand <david@redhat.com>
Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Laura Abbott <labbott@kernel.org>
Cc: Mateusz Nosek <mateusznosek0@gmail.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

(cherry picked from commit 8db26a3d47
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ifc3fdf5cd58f3b8346bf81480df3836811e7458b
2021-03-24 15:09:16 -07:00
Vlastimil Babka
0879d44ddd BACKPORT: mm, page_alloc: do not rely on the order of page_poison and init_on_alloc/free parameters
Patch series "cleanup page poisoning", v3.

I have identified a number of issues and opportunities for cleanup with
CONFIG_PAGE_POISON and friends:

 - interaction with init_on_alloc and init_on_free parameters depends on
   the order of parameters (Patch 1)

 - the boot time enabling uses static key, but inefficienty (Patch 2)

 - sanity checking is incompatible with hibernation (Patch 3)

 - CONFIG_PAGE_POISONING_NO_SANITY can be removed now that we have
   init_on_free (Patch 4)

 - CONFIG_PAGE_POISONING_ZERO can be most likely removed now that we
   have init_on_free (Patch 5)

This patch (of 5):

Enabling page_poison=1 together with init_on_alloc=1 or init_on_free=1
produces a warning in dmesg that page_poison takes precedence.  However,
as these warnings are printed in early_param handlers for
init_on_alloc/free, they are not printed if page_poison is enabled later
on the command line (handlers are called in the order of their
parameters), or when init_on_alloc/free is always enabled by the
respective config option - before the page_poison early param handler is
called, it is not considered to be enabled.  This is inconsistent.

We can remove the dependency on order by making the init_on_* parameters
only set a boolean variable, and postponing the evaluation after all early
params have been processed.  Introduce a new
init_mem_debugging_and_hardening() function for that, and move the related
debug_pagealloc processing there as well.

As a result init_mem_debugging_and_hardening() knows always accurately if
init_on_* and/or page_poison options were enabled.  Thus we can also
optimize want_init_on_alloc() and want_init_on_free().  We don't need to
check page_poisoning_enabled() there, we can instead not enable the
init_on_* static keys at all, if page poisoning is enabled.  This results
in a simpler and more effective code.

Link: https://lkml.kernel.org/r/20201113104033.22907-1-vbabka@suse.cz
Link: https://lkml.kernel.org/r/20201113104033.22907-2-vbabka@suse.cz
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Mateusz Nosek <mateusznosek0@gmail.com>
Cc: Laura Abbott <labbott@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

(cherry picked from commit 04013513cc
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
[glider: resolved a minor conflict in init/main.c - API change]
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I6c0ffcb0d8e2f56a688986aa1dc201adf89de067
2021-03-24 15:09:16 -07:00
Andrey Konovalov
7186ac0c43 FROMGIT: kasan: init memory in kasan_(un)poison for HW_TAGS
This change adds an argument to kasan_poison() and kasan_unpoison() that
allows initializing memory along with setting the tags for HW_TAGS.

Combining setting allocation tags with memory initialization will improve
HW_TAGS KASAN performance when init_on_alloc/free is enabled.

This change doesn't integrate memory initialization with KASAN, this is
done is subsequent patches in this series.

Link: https://lkml.kernel.org/r/3054314039fa64510947e674180d675cab1b4c41.1615296150.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit b5feba92b2290c2216281d2863891d587d131d06
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ie28bfd789f67a02901c2dd42b115a1c24d89e9da
2021-03-24 15:09:16 -07:00
Andrey Konovalov
0de58767b7 FROMGIT: arm64: kasan: allow to init memory when setting tags
Patch series "kasan: integrate with init_on_alloc/free", v3.

This patch series integrates HW_TAGS KASAN with init_on_alloc/free by
initializing memory via the same arm64 instruction that sets memory tags.

This is expected to improve HW_TAGS KASAN performance when
init_on_alloc/free is enabled.  The exact perfomance numbers are unknown
as MTE-enabled hardware doesn't exist yet.

This patch (of 5):

This change adds an argument to mte_set_mem_tag_range() that allows to
enable memory initialization when settinh the allocation tags.  The
implementation uses stzg instruction instead of stg when this argument
indicates to initialize memory.

Combining setting allocation tags with memory initialization will improve
HW_TAGS KASAN performance when init_on_alloc/free is enabled.

This change doesn't integrate memory initialization with KASAN, this is
done is subsequent patches in this series.

Link: https://lkml.kernel.org/r/cover.1615296150.git.andreyknvl@google.com
Link: https://lkml.kernel.org/r/d04ae90cc36be3fe246ea8025e5085495681c3d7.1615296150.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Marco Elver <elver@google.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit 75393a0acbc3c0fd2307622ef2b1d49e5f64916a
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I986bcd4a94e6d75793dab04fcca2f2e9e3ebc4f9
2021-03-24 15:09:15 -07:00
Andrey Konovalov
e1675ffaf8 FROMGIT: mm, kasan: don't poison boot memory with tag-based modes
During boot, all non-reserved memblock memory is exposed to page_alloc via
memblock_free_pages->__free_pages_core().  This results in
kasan_free_pages() being called, which poisons that memory.

Poisoning all that memory lengthens boot time.  The most noticeable effect
is observed with the HW_TAGS mode.  A boot-time impact may potentially
also affect systems with large amount of RAM.

This patch changes the tag-based modes to not poison the memory during the
memblock->page_alloc transition.

An exception is made for KASAN_GENERIC.  Since it marks all new memory as
accessible, not poisoning the memory released from memblock will lead to
KASAN missing invalid boot-time accesses to that memory.

With KASAN_SW_TAGS, as it uses the invalid 0xFE tag as the default tag for
all memory, it won't miss bad boot-time accesses even if the poisoning of
memblock memory is removed.

With KASAN_HW_TAGS, the default memory tags values are unspecified.
Therefore, if memblock poisoning is removed, this KASAN mode will miss the
mentioned type of boot-time bugs with a 1/16 probability.  This is taken
as an acceptable trafe-off.

Internally, the poisoning is removed as follows.  __free_pages_core() is
used when exposing fresh memory during system boot and when onlining
memory during hotplug.  This patch adds a new FPI_SKIP_KASAN_POISON flag
and passes it to __free_pages_ok() through free_pages_prepare() from
__free_pages_core().  If FPI_SKIP_KASAN_POISON is set, kasan_free_pages()
is not called.

All memory allocated normally when the boot is over keeps getting poisoned
as usual.

Link: https://lkml.kernel.org/r/a0570dc1e3a8f39a55aa343a1fc08cd5c2d4cad6.1613692950.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit d07a05953b6067f4295dc1a81ee4cda10615f784
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I3b26994bb3e2b9d8e85d001d817a37892bf0477c
2021-03-24 15:09:15 -07:00
Andrey Konovalov
b1df3e8de9 FROMGIT: kasan: initialize shadow to TAG_INVALID for SW_TAGS
Currently, KASAN_SW_TAGS uses 0xFF as the default tag value for
unallocated memory.  The underlying idea is that since that memory hasn't
been allocated yet, it's only supposed to be dereferenced through a
pointer with the native 0xFF tag.

While this is a good idea in terms on consistency, practically it doesn't
bring any benefit.  Since the 0xFF pointer tag is a match-all tag, it
doesn't matter what tag the accessed memory has.  No accesses through
0xFF-tagged pointers are considered buggy by KASAN.

This patch changes the default tag value for unallocated memory to 0xFE,
which is the tag KASAN uses for inaccessible memory.  This doesn't affect
accesses through 0xFF-tagged pointer to this memory, but this allows KASAN
to detect wild and large out-of-bounds invalid memory accesses through
otherwise-tagged pointers.

This is a prepatory patch for the next one, which changes the tag-based
KASAN modes to not poison the boot memory.

Link: https://lkml.kernel.org/r/c8e93571c18b3528aac5eb33ade213bf133d10ad.1613692950.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit 7b40fcc93f6139e8979d791580f84f79ea67c9d8
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Ia06c03058a77e30ddd875fa0e189e679b8404333
2021-03-24 15:09:15 -07:00
Zhiyuan Dai
fcf7fb524a FROMGIT: mm/kasan: switch from strlcpy to strscpy
strlcpy is marked as deprecated in Documentation/process/deprecated.rst,
and there is no functional difference when the caller expects truncation
(when not checking the return value).  strscpy is relatively better as it
also avoids scanning the whole source string.

Link: https://lkml.kernel.org/r/1613970647-23272-1-git-send-email-daizhiyuan@phytium.com.cn
Signed-off-by: Zhiyuan Dai <daizhiyuan@phytium.com.cn>
Acked-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit e5bbe620e7a10d60161638cb0feece9466c8f511
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I34a961043c1d0505e67547f1200d768e4f82a851
2021-03-24 15:09:15 -07:00
Walter Wu
86a1ff2750 BACKPORT: kasan: remove redundant config option
CONFIG_KASAN_STACK and CONFIG_KASAN_STACK_ENABLE both enable KASAN stack
instrumentation, but we should only need one config, so that we remove
CONFIG_KASAN_STACK_ENABLE and make CONFIG_KASAN_STACK workable.  see [1].

When enable KASAN stack instrumentation, then for gcc we could do no
prompt and default value y, and for clang prompt and default value n.

[1]: https://bugzilla.kernel.org/show_bug.cgi?id=210221

Link: https://lkml.kernel.org/r/20210226012531.29231-1-walter-zh.wu@mediatek.com
Signed-off-by: Walter Wu <walter-zh.wu@mediatek.com>
Suggested-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Alexander Potapenko <glider@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit 3bc29a8e251a9469ce69e62118d70eaf0caa5acb
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
(cherry picked from commit 0d9e60b66271414a18a1d4f1fe2c923245f1e3a8
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
[glider: resolved a minor merge conflict, squashed two patches together]

Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I1b3d2cdcf45727a25d36ac7f417a4f026152d6a8
2021-03-24 15:09:15 -07:00
Andrey Konovalov
ea1ffe7053 FROMGIT: kasan: fix per-page tags for non-page_alloc pages
To allow performing tag checks on page_alloc addresses obtained via
page_address(), tag-based KASAN modes store tags for page_alloc
allocations in page->flags.

Currently, the default tag value stored in page->flags is 0x00.
Therefore, page_address() returns a 0x00ffff...  address for pages that
were not allocated via page_alloc.

This might cause problems.  A particular case we encountered is a conflict
with KFENCE.  If a KFENCE-allocated slab object is being freed via
kfree(page_address(page) + offset), the address passed to kfree() will get
tagged with 0x00 (as slab pages keep the default per-page tags).  This
leads to is_kfence_address() check failing, and a KFENCE object ending up
in normal slab freelist, which causes memory corruptions.

This patch changes the way KASAN stores tag in page-flags: they are now
stored xor'ed with 0xff.  This way, KASAN doesn't need to initialize
per-page flags for every created page, which might be slow.

With this change, page_address() returns natively-tagged (with 0xff)
pointers for pages that didn't have tags set explicitly.

This patch fixes the encountered conflict with KFENCE and prevents more
similar issues that can occur in the future.

Link: https://lkml.kernel.org/r/1a41abb11c51b264511d9e71c303bb16d5cb367b.1615475452.git.andreyknvl@google.com
Fixes: 2813b9c029 ("kasan, mm, arm64: tag non slab memory allocated via pagealloc")
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

(cherry picked from commit e671110d7acf7936013d309ff47ebe645f880992
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I3cd514ed31d2742cf800ff85074c69930aec2771
2021-03-24 15:09:14 -07:00
Andrey Konovalov
cfcec8bc2a FROMGIT: kasan: fix KASAN_STACK dependency for HW_TAGS
There's a runtime failure when running HW_TAGS-enabled kernel built with
GCC on hardware that doesn't support MTE.  GCC-built kernels always have
CONFIG_KASAN_STACK enabled, even though stack instrumentation isn't
supported by HW_TAGS.  Having that config enabled causes KASAN to issue
MTE-only instructions to unpoison kernel stacks, which causes the failure.

Fix the issue by disallowing CONFIG_KASAN_STACK when HW_TAGS is used.

(The commit that introduced CONFIG_KASAN_HW_TAGS specified proper
 dependency for CONFIG_KASAN_STACK_ENABLE but not for CONFIG_KASAN_STACK.)

Link: https://lkml.kernel.org/r/59e75426241dbb5611277758c8d4d6f5f9298dac.1615215441.git.andreyknvl@google.com
Fixes: 6a63a63ff1 ("kasan: introduce CONFIG_KASAN_HW_TAGS")
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reported-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: <stable@vger.kernel.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

(cherry picked from commit d9b571c885
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: I35c6446be4c7193007c1b96eeddaab3923500c61
2021-03-24 15:09:14 -07:00
Andrey Konovalov
23ba14e38e FROMGIT: kasan, mm: fix crash with HW_TAGS and DEBUG_PAGEALLOC
Currently, kasan_free_nondeferred_pages()->kasan_free_pages() is called
after debug_pagealloc_unmap_pages(). This causes a crash when
debug_pagealloc is enabled, as HW_TAGS KASAN can't set tags on an
unmapped page.

This patch puts kasan_free_nondeferred_pages() before
debug_pagealloc_unmap_pages() and arch_free_page(), which can also make
the page unavailable.

Link: https://lkml.kernel.org/r/24cd7db274090f0e5bc3adcdc7399243668e3171.1614987311.git.andreyknvl@google.com
Fixes: 94ab5b61ee ("kasan, arm64: enable CONFIG_KASAN_HW_TAGS")
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

(cherry picked from commit f9d79e8dce
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iab88e1e7ab63e6e5ec4b5e0160baf16e0335ee9e
2021-03-24 15:09:14 -07:00
Andrey Konovalov
9bd391fac8 FROMGIT: arm64: kasan: fix page_alloc tagging with DEBUG_VIRTUAL
When CONFIG_DEBUG_VIRTUAL is enabled, the default page_to_virt() macro
implementation from include/linux/mm.h is used. That definition doesn't
account for KASAN tags, which leads to no tags on page_alloc allocations.

Provide an arm64-specific definition for page_to_virt() when
CONFIG_DEBUG_VIRTUAL is enabled that takes care of KASAN tags.

Fixes: 2813b9c029 ("kasan, mm, arm64: tag non slab memory allocated via pagealloc")
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/4b55b35202706223d3118230701c6a59749d9b72.1615219501.git.andreyknvl@google.com
Signed-off-by: Will Deacon <will@kernel.org>

(cherry picked from commit 86c83365ab
 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)
Bug: 182930667
Signed-off-by: Alexander Potapenko <glider@google.com>
Change-Id: Iae14c4b803ca62753992a26628c32840c19cce76
2021-03-24 15:09:14 -07:00
Maciej Żenczykowski
fa535cfd78 FROMLIST: configfs: make directories inherit uid/gid from creator
Currently a non-root user may have the rights to create directories
in configfs, but they default to being owned by root, so you can't
create anything inside of the directories you yourself created.

phone:/config/usb_gadget/g1/configs $ id; mkdir b.2; ls -lZ; chown system:system b.2
uid=1000(system) gid=1000(system) groups=1000(system),1004(input),1007(log),1011(adb),...
drwxr-xr-x 3 system  system u:object_r:configfs:s0  0 2020-12-28 06:03 b.1
drwxr-xr-x 3 root    root   u:object_r:configfs:s0  0 2020-12-28 06:51 b.2
chown: 'b.2' to 'system:system': Operation not permitted

phone:/config/usb_gadget/g1/configs $ ln -s ../../../../usb_gadget/g1/functions/ffs.adb b.2/function0
ln: cannot create symbolic link from '../../../../usb_gadget/g1/functions/ffs.adb' to 'b.2/function0': Permission denied

Test: With this change b.2 is owned by system:system and the ln succeeds.
Link: https://lore.kernel.org/lkml/20210123205516.2738060-1-zenczykowski@gmail.com/
Bug: 172793258
Signed-off-by: Maciej Żenczykowski <maze@google.com>
Change-Id: Ia907b2def940197b44aa87b337d37c5dde9c5b91
2021-03-24 21:32:54 +00:00
Saravana Kannan
54539dfef4 ANDROID: GKI: add some padding to some driver core structures
These structures are fundamental to implementing fw_devlink and
sync_state(). Since they are still evolving, add some padding in case we
need to backport any important bug fixes.

	struct device_link
	struct class
	struct fwnode_handle
	struct fwnode_link

Bug: 183615740
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: Id9daf7cf9ae5d94fb0134144f8220a241ccbaef8
2021-03-24 20:43:25 +00:00
Todd Kjos
2fa0951b66 ANDROID: Initial Android 12 OWNERS for abi metafiles
Require OWNERS approval for changes to abi metafiles.

Bug: 183615388
Signed-off-by: Todd Kjos <tkjos@google.com>
Change-Id: I42e57e2cd32ae830ec32fccdb78744e8beb8f317
2021-03-24 19:56:04 +00:00
Robin Murphy
aa036a0e3d UPSTREAM: iommu/msm: Hook up iotlb_sync_map
The core API can now accommodate invalidate-on-map style behaviour in a
single efficient call, so hook that up instead of having io-pgatble do
it piecemeal.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/e95223a0abf129230a0bec6743f837075f0a2fcb.1611764372.git.robin.murphy@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit c867c78aca)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I6b757abc9c64bdcfb5a05def95eda416461aec4e
2021-03-24 12:45:16 -07:00
Yong Wu
1a0226d0f3 UPSTREAM: memory: mtk-smi: Allow building as module
Add support for building the SMI driver as module. Switch MTK_SMI to
tristate, and add module_exit/module_license.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20210126060055.11050-1-yong.wu@mediatek.com
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
(cherry picked from commit 50fc8d9232)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I6d4c9e14304d6d42e4a1e8ef441ae8591495b2df
2021-03-24 12:45:16 -07:00
Yong Wu
ea79f21727 UPSTREAM: memory: mtk-smi: Use platform_register_drivers
In this file, we have 2 drivers, smi-common and smi-larb.
Use platform_register_drivers.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20210121062429.26504-2-yong.wu@mediatek.com
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
(cherry picked from commit 1821203150)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Idf1b0a9bbe6460bcdc92ae597a19b2190f7a655a
2021-03-24 12:45:16 -07:00
Dan Carpenter
9a2ce7cfa2 UPSTREAM: iommu/mediatek: Fix error code in probe()
This error path is supposed to return -EINVAL.  It used to return
directly but we added some clean up and accidentally removed the
error code.  Also I fixed a typo in the error message.

Fixes: c0b57581b7 ("iommu/mediatek: Add power-domain operation")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/YB0+GU5akSdu29Vu@mwanda
Signed-off-by: Joerg Roedel <jroedel@suse.de>
(cherry picked from commit a92a90ac62)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I65e9f85408625df6c827ef835089d847a8ab7257
2021-03-24 12:45:16 -07:00
Colin Ian King
2095b0d898 UPSTREAM: iommu/mediatek: Fix unsigned domid comparison with less than zero
Currently the check for domid < 0 is always false because domid
is unsigned. Fix this by casting domid to an int before making
the comparison.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210204150001.102672-1-colin.king@canonical.com
Addresses-Coverity: ("Unsigned comparison against 0")
Signed-off-by: Joerg Roedel <jroedel@suse.de>
(cherry picked from commit 7a5661739d)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ic975f030d0ae1409c9ef79bc4d82099fa715e0fd
2021-03-24 12:45:15 -07:00
Yong Wu
9e0929221b UPSTREAM: iommu/mediatek: Add mt8192 support
Add mt8192 iommu support.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-33-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 9e3489e06f)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I5a35da29cb9512d432cf476d2342d96359d45d4f
2021-03-24 12:45:15 -07:00
Yong Wu
f85b76e778 UPSTREAM: memory: mtk-smi: Add mt8192 support
Add mt8192 SMI support.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20201103054200.21386-4-yong.wu@mediatek.com
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
(cherry picked from commit 02c02ddce4)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I6658247f1d165a2bf26268bdd21b750ed6efa104
2021-03-24 12:45:15 -07:00
Yong Wu
b28fe378d1 UPSTREAM: iommu/mediatek: Remove unnecessary check in attach_device
This priv_data is set in the of_xlate. if of_xlate failed, it should
not enter attach_device. remove the unnecessary check.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-32-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 23357572be)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Icd2d561abd0dcf1c580c2cbca89df4bc06b8299f
2021-03-24 12:45:15 -07:00
Yong Wu
f513c83eca UPSTREAM: iommu/mediatek: Support master use iova over 32bit
After extending v7s, our pagetable already support iova reach
16GB(34bit). the master got the iova via dma_alloc_attrs may reach
34bits, but its HW register still is 32bit. then how to set the
bit32/bit33 iova? this depend on a SMI larb setting(bank_sel).

we separate whole 16GB iova to four banks:
bank: 0: 0~4G; 1: 4~8G; 2: 8-12G; 3: 12-16G;
The bank number is (iova >> 32).

We will preassign which bank the larbs belong to. currently we don't
have a interface for master to adjust its bank number.

Each a bank is a iova_region which is a independent iommu-domain.
the iova range for each iommu-domain can't cross 4G.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org> #for memory part
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-31-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 8d2c749e52)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I0d397b100026b04747c753233f0dea0a1d5cf2ee
2021-03-24 12:45:15 -07:00
Yong Wu
1131fdff70 UPSTREAM: iommu/mediatek: Add iova reserved function
For multiple iommu_domains, we need to reserve some iova regions. Take a
example, If the default iova region is 0 ~ 4G, but the 0x4000_0000 ~
0x43ff_ffff is only for the special CCU0 domain. Thus we should exclude
this region for the default iova region.

Signed-off-by: Anan sun <anan.sun@mediatek.com>
Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-30-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit ab1d5281a6)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I9ea4789c525f140bc2326f26adc7c2b899ad62aa
2021-03-24 12:45:15 -07:00
Yong Wu
97b17dc794 UPSTREAM: iommu/mediatek: Support for multi domains
Some HW IP(ex: CCU) require the special iova range. That means the iova
got from dma_alloc_attrs for that devices must locate in his special range.
In this patch, we prepare a iommu group(domain) for each a iova range
requirement.

Meanwhile we still use one pagetable which support 16GB iova.

After this patch, If the iova range of a master is over 4G, the master
should:
a) Declare its special dma-ranges in its dtsi node. For example, If we
   preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should
   add this:
   /*
    * iova start at 0x1_0000_0000, pa still start at 0x4000_0000
    * size is 0x1_0000_0000.
    */
   dma-ranges = <0x1 0x0 0x0 0x40000000 0x1 0x0>;  /* 4G ~ 8G */
 Note: we don't have a actual bus concept here. the master doesn't have its
 special parent node, thus this dma-ranges can only be put in the master's
 node.

b) Update the dma_mask:
  dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33));

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-29-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit c3045f3924)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Id104c9ffa0ffae47b13be5f9d71029bead0cd926
2021-03-24 12:45:14 -07:00
Yong Wu
a73718de28 UPSTREAM: iommu/mediatek: Add get_domain_id from dev->dma_range_map
Add a new interface _get_domain_id from dev->dma_range_map,
The iommu consumer device will use dma-ranges in dtsi node to indicate
its dma address region requirement. In this iommu driver, we will get
the requirement and decide which iova domain it should locate.

In the lastest SoC, there will be several iova-regions(domains), we will
compare and calculate which domain is right. If the start/end of device
requirement equal some region. it is best fit of course. If it is inside
some region, it is also ok. the iova requirement of a device should not
be inside two or more regions.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-28-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 803cf9e5a6)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ic41106a406bcbb375aa8f507232ab847a2d9c4e0
2021-03-24 12:45:14 -07:00
Yong Wu
d91ad14e31 UPSTREAM: iommu/mediatek: Add iova_region structure
Add a new structure for the iova_region. Each a region will be a
independent iommu domain.

For the previous SoC, there is single iova region(0~4G). For the SoC
that need support multi-domains, there will be several regions.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-27-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 585e58f498)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I16dafc5e799d2b656b7d92749bb8412e7cc045e7
2021-03-24 12:45:14 -07:00
Yong Wu
ce83c68a10 UPSTREAM: iommu/mediatek: Move geometry.aperture updating into domain_finalise
Move the domain geometry.aperture updating into domain_finalise.
This is a preparing patch for updating the domain region. We know the
detailed iova region in the attach_device.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-26-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit b7875eb945)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I9291bfcccc315b470dbe756454a8793396ef58e1
2021-03-24 12:45:14 -07:00
Yong Wu
35dbaf1367 UPSTREAM: iommu/mediatek: Move domain_finalise into attach_device
Currently domain_alloc only has a parameter(type), We have no chance to
input some special data. This patch moves the domain_finalise into
attach_device which has the device information, then could update
the domain's geometry.aperture ranges for each a device.

Strictly, I should use the data from mtk_iommu_get_m4u_data as the
parameter of mtk_iommu_domain_finalise in this patch. but dom->data
only is used in tlb ops in which the data is get from the m4u_list, thus
it is ok here.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-25-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 4f956c97d2)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I1abc9d7a17eb7a4df5fae01b422525fd4f012384
2021-03-24 12:45:14 -07:00
Yong Wu
9716e44f2d UPSTREAM: iommu/mediatek: Adjust the structure
Add "struct mtk_iommu_data *" in the "struct mtk_iommu_domain",
reduce the call mtk_iommu_get_m4u_data().
No functional change.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-24-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 08500c43d4)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ie710275a785bde078390541541069fd5370e3c42
2021-03-24 12:45:14 -07:00
Yong Wu
0003b1eeb9 UPSTREAM: iommu/mediatek: Support report iova 34bit translation fault in ISR
If the iova is over 32bit, the fault status register bit is a little
different.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-23-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit ef0f0986b6)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I3e14fec49d2e2afe1786a44bb0bd57c294c83a30
2021-03-24 12:45:13 -07:00
Yong Wu
02faad4388 UPSTREAM: iommu/mediatek: Support up to 34bit iova in tlb flush
If the iova is 34bit, the iova[32][33] is the bit0/1 in the tlb flush
register. Add a new macro for this.

In the macro, since (iova + size - 1) may be end with 0xfff, then the
bit0/1 always is 1, thus add a mask.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-22-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit bfed873114)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I715de5715282d4122b9d8d1aef4a9b36ec9278a1
2021-03-24 12:45:13 -07:00
Yong Wu
3f2b9b566b UPSTREAM: iommu/mediatek: Add power-domain operation
In the previous SoC, the M4U HW is in the EMI power domain which is
always on. the latest M4U is in the display power domain which may be
turned on/off, thus we have to add pm_runtime interface for it.

When the engine work, the engine always enable the power and clocks for
smi-larb/smi-common, then the M4U's power will always be powered on
automatically via the device link with smi-common.

Note: we don't enable the M4U power in iommu_map/unmap for tlb flush.
If its power already is on, of course it is ok. if the power is off,
the main tlb will be reset while M4U power on, thus the tlb flush while
m4u power off is unnecessary, just skip it.
Therefore, we increase the ref_count for pm when pm status is ACTIVE,
otherwise, skip it. Meanwhile, the tlb_flush_range is called so often,
thus, update pm ref_count while the SoC has power-domain to avoid touch the
dev->power.lock. and the tlb_flush_all only is called when boot, so no
need check if the SoC has power-domain to keep code clean.

There will be one case that pm runctime status is not expected when tlb
flush. After boot, the display may call dma_alloc_attrs before it call
pm_runtime_get(disp-dev), then the m4u's pm status is not active inside
the dma_alloc_attrs. Since it only happens after boot, the tlb is clean
at that time, I also think this is ok.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-21-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit c0b57581b7)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ice851a821fdc1c04cb2835a495e852a61bc9536f
2021-03-24 12:45:13 -07:00
Yong Wu
1076657742 UPSTREAM: iommu/mediatek: Add pm runtime callback
In pm runtime case, all the registers backup/restore and bclk are
controlled in the pm_runtime callback, Rename the original
suspend/resume to the runtime_suspend/resume.

Use pm_runtime_force_suspend/resume as the normal suspend/resume.
iommu should suspend after iommu consumer devices, thus use _LATE_.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-20-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 34665c7929)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ib0eff40d127500ad0d05aee767130728831405a3
2021-03-24 12:45:13 -07:00
Yong Wu
a7750f55ff UPSTREAM: iommu/mediatek: Add device link for smi-common and m4u
In the lastest SoC, M4U has its special power domain. thus, If the engine
begin to work, it should help enable the power for M4U firstly.
Currently if the engine work, it always enable the power/clocks for
smi-larbs/smi-common. This patch adds device_link for smi-common and M4U.
then, if smi-common power is enabled, the M4U power also is powered on
automatically.

Normally M4U connect with several smi-larbs and their smi-common always
are the same, In this patch it get smi-common dev from the last smi-larb
device, then add the device_link.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-19-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit baf94e6ebf)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ie57e065de705464cdec6ef7fc4a34a0abfa2572c
2021-03-24 12:45:13 -07:00
Yong Wu
251379f7ae UPSTREAM: iommu/mediatek: Add error handle for mtk_iommu_probe
In the original code, we lack the error handle. This patch adds them.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-18-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 986d9ec5f1)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: If94b6bc5afd3ca8f6bc25086f27c509f58e569d4
2021-03-24 12:45:13 -07:00
Yong Wu
ac13148f28 UPSTREAM: iommu/mediatek: Move hw_init into attach_device
In attach device, it will update the pagetable base address register.
Move the hw_init function also here. Then it only need call
pm_runtime_get/put one time here if m4u has power domain.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-17-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 7f37a91dc8)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: If3f8d688abe8f3f438f3de6b1fbfc37ce2893ee5
2021-03-24 12:45:12 -07:00
Yong Wu
c13006d851 UPSTREAM: iommu/mediatek: Update oas for v7s
This patch only updates oas in different SoCs.

If the SoC supports 4GB-mode and current dram size is 4GB, the oas is 33.
otherwise, it's still 32. In the lastest SoC, the oas is 35bits.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-16-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 9bdfe4c175)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I2f06cef2cbecbf60b6a597afa26472ed19c797c6
2021-03-24 12:45:12 -07:00
Yong Wu
68f8fba54e UPSTREAM: iommu/mediatek: Add a flag for iova 34bits case
Add a HW flag for if the HW support 34bit IOVA. the previous SoC
still use 32bit. normally the lvl1 pgtable size is 16KB when ias == 32.
if ias == 34, lvl1 pgtable size is 16KB * 4. The purpose of this patch
is to save 16KB*3 continuous memory for the previous SoC.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-15-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 2f317da433)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I1b648641c580e875d84dc2c36bcc8e24b536fccd
2021-03-24 12:45:12 -07:00
Yong Wu
2c4a1e84fe UPSTREAM: iommu/io-pgtable-arm-v7s: Quad lvl1 pgtable for MediaTek
The standard input iova bits is 32. MediaTek quad the lvl1 pagetable
(4 * lvl1). No change for lvl2 pagetable. Then the iova bits can reach
34bit.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-14-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit f3a8a46d71)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I106a2511e3123412345712cc40be77856629d0c0
2021-03-24 12:45:12 -07:00
Yong Wu
7b75eb0b4f UPSTREAM: iommu/io-pgtable-arm-v7s: Add cfg as a param in some macros
Add "cfg" as a parameter for some macros. This is a preparing patch for
mediatek extend the lvl1 pgtable. No functional change.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-13-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 468ea0bfae)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I55aa053db8212ea3384a82a448dc523be08ee1f9
2021-03-24 12:45:12 -07:00
Yong Wu
65e264a3b3 UPSTREAM: iommu/io-pgtable-arm-v7s: Clarify LVL_SHIFT/BITS macro
The current _ARM_V7S_LVL_BITS/ARM_V7S_LVL_SHIFT use a formula to calculate
the corresponding value for level1 and level2 to pretend the code sane.
Actually their level1 and level2 values are different from each other.
This patch only clarify the two macro. No functional change.

Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-12-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 00ab6f2d61)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I974422e463f9b88a2954b82755e0a8240fe1d7f7
2021-03-24 12:45:12 -07:00
Yong Wu
a1799e4fda UPSTREAM: iommu/io-pgtable-arm-v7s: Use ias to check the valid iova in unmap
Use the ias for the valid iova checking in arm_v7s_unmap. This is a
preparing patch for supporting iova 34bit for MediaTek.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-10-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 859da21112)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ie1d6fb1b25db14fecef5d7742fa88f259449a37c
2021-03-24 12:45:11 -07:00
Yong Wu
04e8c94822 UPSTREAM: iommu/io-pgtable-arm-v7s: Extend PA34 for MediaTek
MediaTek extend the bit5 in lvl1 and lvl2 descriptor as PA34.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-11-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 40596d2f2b)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I126769349c1eec5503338304981bdc1f4eae0787
2021-03-24 12:45:11 -07:00
Yong Wu
54ef996792 UPSTREAM: iommu/mediatek: Use the common mtk-memory-port.h
Use the common memory header(larb-port) in the source code.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-9-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 66a2891554)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Iaf7565a8cb635d93897c5182fe75e94547a41b54
2021-03-24 12:45:11 -07:00
Yong Wu
127a9ea1da UPSTREAM: dt-bindings: mediatek: Add binding for mt8192 IOMMU
This patch adds decriptions for mt8192 IOMMU and SMI.

mt8192 also is MTK IOMMU gen2 which uses ARM Short-Descriptor translation
table format. The M4U-SMI HW diagram is as below:

                          EMI
                           |
                          M4U
                           |
                      ------------
                       SMI Common
                      ------------
                           |
  +-------+------+------+----------------------+-------+
  |       |      |      |       ......         |       |
  |       |      |      |                      |       |
larb0   larb1  larb2  larb4     ......      larb19   larb20
disp0   disp1   mdp    vdec                   IPE      IPE

All the connections are HW fixed, SW can NOT adjust it.

mt8192 M4U support 0~16GB iova range. we preassign different engines
into different iova ranges:

domain-id  module     iova-range                  larbs
   0       disp        0 ~ 4G                      larb0/1
   1       vcodec      4G ~ 8G                     larb4/5/7
   2       cam/mdp     8G ~ 12G             larb2/9/11/13/14/16/17/18/19/20
   3       CCU0    0x4000_0000 ~ 0x43ff_ffff     larb13: port 9/10
   4       CCU1    0x4400_0000 ~ 0x47ff_ffff     larb14: port 4/5

The iova range for CCU0/1(camera control unit) is HW requirement.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-6-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit fc3734698a)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I61d4b5024722d1784f66f96484ba5b821e73e9d6
2021-03-24 12:45:11 -07:00
Yong Wu
03538000e1 UPSTREAM: dt-bindings: memory: mediatek: Rename header guard for SMI header file
Only rename the header guard for all the SoC larb port header file.
No funtional change.

Suggested-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Acked-by: Rob Herring <robh@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-5-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit ddd3e349b8)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Idbe649857b926f3d568b8b693f955f9575b174f9
2021-03-24 12:45:11 -07:00
Yong Wu
ad29253b1f UPSTREAM: dt-bindings: memory: mediatek: Extend LARB_NR_MAX to 32
Extend the max larb number definition as mt8192 has larb_nr over 16.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Rob Herring <robh@kernel.org>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-4-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit ca49a4b4c9)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: Ic1ec210b8ef992bb20207958290cb8b0b5dde8dc
2021-03-24 12:45:11 -07:00
Yong Wu
29f3ad04e6 UPSTREAM: dt-bindings: memory: mediatek: Add a common memory header file
Put all the macros about smi larb/port togethers.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Rob Herring <robh@kernel.org>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-3-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 5cf482f2f7)

BUG=b:174513569

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Change-Id: I2266a66b48e329343b8a29d50a2bf4ea891750a8
2021-03-24 12:45:10 -07:00