linux-toradex.git/include/linux/pmem.h, branch v4.11-rc3

pmem: kill __pmem address space

2016-07-13T02:25:38+00:00

The __pmem address space was meant to annotate codepaths that touch
persistent memory and need to coordinate a call to wmb_pmem().  Now that
wmb_pmem() is gone, there is little need to keep this annotation.

Cc: Christoph Hellwig 
Cc: Ross Zwisler 
Signed-off-by: Dan Williams

pmem: kill wmb_pmem()

2016-07-12T22:13:48+00:00

All users have been replaced with flushing in the pmem driver.

Cc: Ross Zwisler 
Signed-off-by: Dan Williams

pmem: fix BUG() error in pmem.h:48 on X86_32

2016-04-14T15:01:47+00:00

After 'commit fc0c2028135c ("x86, pmem: use memcpy_mcsafe()
for memcpy_from_pmem()")', probing a PMEM device hits the BUG()
error below on X86_32 kernel.

 kernel BUG at include/linux/pmem.h:48!

memcpy_from_pmem() calls arch_memcpy_from_pmem(), which is
unimplemented since CONFIG_ARCH_HAS_PMEM_API is undefined on
X86_32.

Fix the BUG() error by adding default_memcpy_from_pmem().

Acked-by: Dan Williams 
Signed-off-by: Toshi Kani 
Signed-off-by: Ross Zwisler 
Cc: Dan Williams 
Cc: Ross Zwisler

x86, pmem: use memcpy_mcsafe() for memcpy_from_pmem()

2016-03-29T00:19:31+00:00

Update the definition of memcpy_from_pmem() to return 0 or a negative
error code.  Implement x86/arch_memcpy_from_pmem() with memcpy_mcsafe().

Cc: Borislav Petkov 
Cc: Tony Luck 
Cc: Thomas Gleixner 
Cc: Andy Lutomirski 
Cc: Peter Zijlstra 
Cc: Andrew Morton 
Cc: Linus Torvalds 
Acked-by: Ingo Molnar 
Reviewed-by: Ross Zwisler 
Signed-off-by: Dan Williams

libnvdimm, pmem: clear poison on write

2016-03-09T23:15:32+00:00

If a write is directed at a known bad block perform the following:

1/ write the data

2/ send a clear poison command

3/ invalidate the poison out of the cache hierarchy

Cc: 
Cc: Ross Zwisler 
Reviewed-by: Vishal Verma 
Signed-off-by: Dan Williams

pmem: add wb_cache_pmem() to the PMEM API

2016-01-23T01:02:18+00:00

__arch_wb_cache_pmem() was already an internal implementation detail of
the x86 PMEM API, but this functionality needs to be exported as part of
the general PMEM API to handle the fsync/msync case for DAX mmaps.

One thing worth noting is that we really do want this to be part of the
PMEM API as opposed to a stand-alone function like clflush_cache_range()
because of ordering restrictions.  By having wb_cache_pmem() as part of
the PMEM API we can leave it unordered, call it multiple times to write
back large amounts of memory, and then order the multiple calls with a
single wmb_pmem().

Signed-off-by: Ross Zwisler 
Cc: "H. Peter Anvin" 
Cc: "J. Bruce Fields" 
Cc: "Theodore Ts'o" 
Cc: Alexander Viro 
Cc: Andreas Dilger 
Cc: Dave Chinner 
Cc: Ingo Molnar 
Cc: Jan Kara 
Cc: Jeff Layton 
Cc: Matthew Wilcox 
Cc: Thomas Gleixner 
Cc: Dan Williams 
Cc: Matthew Wilcox 
Cc: Dave Hansen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

pmem: kill memremap_pmem()

2015-10-09T21:00:32+00:00

Now that the pmem-api is defined as "a set of apis that enables access
to WB mapped pmem",  the mapping type is implied.  Remove the wrapper
and push the functionality down into the pmem driver in preparation for
adding support for direct-mapped pmem.

Reviewed-by: Christoph Hellwig 
Signed-off-by: Dan Williams

x86, pmem: clarify that ARCH_HAS_PMEM_API implies PMEM mapped WB

2015-08-27T23:40:59+00:00

Given that a write-back (WB) mapping plus non-temporal stores is
expected to be the most efficient way to access PMEM, update the
definition of ARCH_HAS_PMEM_API to imply arch support for
WB-mapped-PMEM.  This is needed as a pre-requisite for adding PMEM to
the direct map and mapping it with struct page.

The above clarification for X86_64 means that memcpy_to_pmem() is
permitted to use the non-temporal arch_memcpy_to_pmem() rather than
needlessly fall back to default_memcpy_to_pmem() when the pcommit
instruction is not available.  When arch_memcpy_to_pmem() is not
guaranteed to flush writes out of cache, i.e. on older X86_32
implementations where non-temporal stores may just dirty cache,
ARCH_HAS_PMEM_API is simply disabled.

The default fall back for persistent memory handling remains.  Namely,
map it with the WT (write-through) cache-type and hope for the best.

arch_has_pmem_api() is updated to only indicate whether the arch
provides the proper helpers to meet the minimum "writes are visible
outside the cache hierarchy after memcpy_to_pmem() + wmb_pmem()".  Code
that cares whether wmb_pmem() actually flushes writes to pmem must now
call arch_has_wmb_pmem() directly.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Reviewed-by: Ross Zwisler 
[hch: set ARCH_HAS_PMEM_API=n on x86_32]
Reviewed-by: Christoph Hellwig 
[toshi: x86_32 compile fixes]
Signed-off-by: Toshi Kani 
Signed-off-by: Dan Williams

pmem: add copy_from_iter_pmem() and clear_pmem()

2015-08-20T18:07:23+00:00

Add support for two new PMEM APIs, copy_from_iter_pmem() and
clear_pmem().  copy_from_iter_pmem() is used to copy data from an
iterator into a PMEM buffer.  clear_pmem() zeros a PMEM memory range.

Both of these new APIs must be explicitly ordered using a wmb_pmem()
function call and are implemented in such a way that the wmb_pmem()
will make the stores to PMEM durable.  Because both APIs are unordered
they can be called as needed without introducing any unwanted memory
barriers.

Signed-off-by: Ross Zwisler 
Signed-off-by: Dan Williams

pmem: remove layer when calling arch_has_wmb_pmem()

2015-08-20T18:07:23+00:00

Prior to this change arch_has_wmb_pmem() was only called by
arch_has_pmem_api().  Both arch_has_wmb_pmem() and arch_has_pmem_api()
checked to make sure that CONFIG_ARCH_HAS_PMEM_API was enabled.

Instead, remove the old arch_has_wmb_pmem() wrapper to be rid of one
extra layer of indirection and the redundant CONFIG_ARCH_HAS_PMEM_API
check. Rename __arch_has_wmb_pmem() to arch_has_wmb_pmem() since we no
longer have a wrapper, and just have arch_has_pmem_api() call the
architecture specific arch_has_wmb_pmem() directly.

Signed-off-by: Ross Zwisler 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Dan Williams