diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-17 19:04:16 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-17 19:04:16 -0700 |
| commit | 1d7443e4dca1e8637930f3baf64e2fe82033e669 (patch) | |
| tree | 5800e3a841ff00d6f547d9c23f02e9afc5650fda | |
| parent | 63c070cba0c6a586d33730ddbf63065cf28276b7 (diff) | |
| parent | 7dd38d9dd7a05329825fe2324d4d8e27ad4b3cec (diff) | |
Merge tag 'fscrypt-for-linus' of git://git.kernel.org/pub/scm/fs/fscrypt/linux
Pull fscrypt updates from Eric Biggers:
"The main change this cycle is a significant simplification that's been
overdue for a while now: standardizing on a single file contents
encryption implementation in ext4 and f2fs, instead of having two.
Specifically, the original filesystem-layer file contents encryption
implementation is removed, and the blk-crypto implementation is now
used unconditionally. blk-crypto delegates either to inline crypto
hardware or to the CPU via blk-crypto-fallback. The latter is
functionally equivalent to the original filesystem-layer code.
The blk-crypto implementation already existed, but previously it was
used only when the filesystem was mounted with "-o inlinecrypt". Now,
"-o inlinecrypt" just selects whether inline crypto hardware is used.
To allow maintaining that user control over hardware use, the
blk-crypto API is extended with a new flag BLK_CRYPTO_CFG_ALLOW_HW.
Overall, this removes quite a bit of redundant code from ext4, f2fs,
and fs/crypto/. It should make things easier for ongoing filesystem
efforts such as iomap support, large folios, and btrfs encryption
(btrfs had already been planning to use blk-crypto exclusively.)
There are two small behavior changes of note:
- Direct I/O now works on encrypted files even without "-o inlinecrypt",
rather than falling back to buffered I/O. This is effectively a
bugfix, though I'll continue to keep an eye out for any user that
may have been depending on the buffered I/O fallback.
- IV_INO_LBLK_32 policies are no longer supported in certain cases
that didn't make sense and have no known uses.
This has been in linux-next since July 22 with no reported issues. All
encryption xfstests pass on ext4 and f2fs. As usual I've also been
using it on a system with an fscrypt-encrypted home directory. Of
course, the blk-crypto code paths also aren't new and were already
being used on many systems via the inlinecrypt mount option.
In addition to the main change described above, there are a few other
cleanups such as using lock guards for mutexes, improving
documentation, and removing a workaround for outdated gcc versions"
* tag 'fscrypt-for-linus' of git://git.kernel.org/pub/scm/fs/fscrypt/linux: (29 commits)
blk-crypto: Update docs for blk-crypto-fallback motivation
blk-crypto: Remove unused function blk_crypto_config_supported()
fscrypt: Update docs for data path
fscrypt: Remove unused function fscrypt_finalize_bounce_page()
f2fs: Update outdated comment in f2fs_write_begin()
fs: Update outdated comment for SB_INLINECRYPT
fscrypt: Update encryption policy version docs
fscrypt: Replace some variable-size memsets with fixed-size
fscrypt: Add safety checks to non-block-based en/decryption
fscrypt: Merge bio.c and inline_crypt.c into block.c
fscrypt: Remove unused functions and workqueue
fscrypt: Remove fs-layer zeroout code
fscrypt: Remove fscrypt_dio_supported()
fscrypt: Replace calls to fscrypt_inode_uses_inline_crypto()
fs/buffer: Remove fs-layer decryption code
f2fs: Remove fs-layer file contents en/decryption code
ext4: Further de-generalize the bio postprocessing code
ext4: Make ext4_bio_write_folio() return void
ext4: Remove fs-layer file contents en/decryption code
Documentation: fscrypt: Update docs for inlinecrypt
...
39 files changed, 494 insertions, 1246 deletions
diff --git a/Documentation/admin-guide/ext4.rst b/Documentation/admin-guide/ext4.rst index ac0c709ea9e7..742a48e6fc0c 100644 --- a/Documentation/admin-guide/ext4.rst +++ b/Documentation/admin-guide/ext4.rst @@ -385,11 +385,9 @@ When mounting an ext4 filesystem, the following option are accepted: incompatible with data=journal. inlinecrypt - When possible, encrypt/decrypt the contents of encrypted files using the - blk-crypto framework rather than filesystem-layer encryption. This - allows the use of inline encryption hardware. The on-disk format is - unaffected. For more details, see - Documentation/block/inline-encryption.rst. + When possible, encrypt/decrypt the contents of encrypted files using + inline encryption hardware rather than the CPU. For more details, see + Documentation/filesystems/fscrypt.rst. Data Mode ========= diff --git a/Documentation/block/inline-encryption.rst b/Documentation/block/inline-encryption.rst index cae23949a626..0df964507f76 100644 --- a/Documentation/block/inline-encryption.rst +++ b/Documentation/block/inline-encryption.rst @@ -37,12 +37,12 @@ initialization vector for each sector, and can be tested for correctness. Objective ========= -We want to support inline encryption in the kernel. To make testing easier, we -also want support for falling back to the kernel crypto API when actual inline -encryption hardware is absent. We also want inline encryption to work with -layered devices like device-mapper and loopback (i.e. we want to be able to use -the inline encryption hardware of the underlying devices if present, or else -fall back to crypto API en/decryption). +We want to support inline encryption hardware in the kernel. The API for using +such hardware should also support a fallback to the CPU, so that users only need +to use a single API and more of the code can be tested without actual hardware. +We also want inline encryption to work with layered devices like device-mapper +and loopback (i.e. we want to be able to use the inline encryption hardware of +the underlying devices if present, or else fall back to the CPU). Constraints and notes ===================== @@ -185,20 +185,12 @@ blk-crypto-fallback is optional and is controlled by the API presented to users of the block layer ========================================= -``blk_crypto_config_supported()`` allows users to check ahead of time whether -inline encryption with particular crypto settings will work on a particular -block_device -- either via hardware or via blk-crypto-fallback. This function -takes in a ``struct blk_crypto_config`` which is like blk_crypto_key, but omits -the actual bytes of the key and instead just contains the algorithm, data unit -size, etc. This function can be useful if blk-crypto-fallback is disabled. - ``blk_crypto_init_key()`` allows users to initialize a blk_crypto_key. Users must call ``blk_crypto_start_using_key()`` before actually starting to use -a blk_crypto_key on a block_device (even if ``blk_crypto_config_supported()`` -was called earlier). This is needed to initialize blk-crypto-fallback if it -will be needed. This must not be called from the data path, as this may have to -allocate resources, which may deadlock in that case. +a blk_crypto_key on a block_device. This is needed to initialize +blk-crypto-fallback if it will be needed. This must not be called from the data +path, as this may have to allocate resources, which may deadlock in that case. Next, to attach an encryption context to a bio, users should call ``bio_crypt_set_ctx()``. This function allocates a bio_crypt_ctx and attaches @@ -220,16 +212,15 @@ any kernel data structures it may be linked into. In summary, for users of the block layer, the lifecycle of a blk_crypto_key is as follows: -1. ``blk_crypto_config_supported()`` (optional) -2. ``blk_crypto_init_key()`` -3. ``blk_crypto_start_using_key()`` -4. ``bio_crypt_set_ctx()`` (potentially many times) -5. ``blk_crypto_evict_key()`` (after all I/O has completed) -6. Zeroize the blk_crypto_key (this has no dedicated function) +1. ``blk_crypto_init_key()`` +2. ``blk_crypto_start_using_key()`` +3. ``bio_crypt_set_ctx()`` (potentially many times) +4. ``blk_crypto_evict_key()`` (after all I/O has completed) +5. Zeroize the blk_crypto_key (this has no dedicated function) If a blk_crypto_key is being used on multiple block_devices, then -``blk_crypto_config_supported()`` (if used), ``blk_crypto_start_using_key()``, -and ``blk_crypto_evict_key()`` must be called on each block_device. +``blk_crypto_start_using_key()`` and ``blk_crypto_evict_key()`` must be called +on each block_device. API presented to device drivers =============================== @@ -304,7 +295,7 @@ hardware implementations might not implement both features together correctly, and disallow the combination for now. Whenever a device supports integrity, the kernel will pretend that the device does not support hardware inline encryption (by setting the blk_crypto_profile in the request_queue of the device to NULL). -When the crypto API fallback is enabled, this means that all bios with and +When the crypto API fallback is enabled, this means that all bios with an encryption context will use the fallback, and IO will complete as usual. When the fallback is disabled, a bio with an encryption context will be failed. diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst index 8c4a14ae444f..b45d7a687625 100644 --- a/Documentation/filesystems/f2fs.rst +++ b/Documentation/filesystems/f2fs.rst @@ -351,12 +351,10 @@ compress_mode=%s Control file compression mode. This supports "fs" and "user" compress_cache Support to use address space of a filesystem managed inode to cache compressed block, in order to improve cache hit ratio of random read. -inlinecrypt When possible, encrypt/decrypt the contents of encrypted - files using the blk-crypto framework rather than - filesystem-layer encryption. This allows the use of - inline encryption hardware. The on-disk format is - unaffected. For more details, see - Documentation/block/inline-encryption.rst. +inlinecrypt When possible, encrypt/decrypt the contents of + encrypted files using inline encryption hardware rather + than the CPU. For more details, see + Documentation/filesystems/fscrypt.rst. atgc Enable age-threshold garbage collection, it provides high effectiveness and efficiency on background GC. discard_unit=%s Control discard unit, the argument can be "block", "segment" diff --git a/Documentation/filesystems/fscrypt.rst b/Documentation/filesystems/fscrypt.rst index c0dd35f1af12..f309337fe110 100644 --- a/Documentation/filesystems/fscrypt.rst +++ b/Documentation/filesystems/fscrypt.rst @@ -188,8 +188,8 @@ attacks: - Non-root users cannot securely remove encryption keys. All the above problems are fixed with v2 encryption policies. For -this reason among others, it is recommended to use v2 encryption -policies on all new encrypted directories. +this reason among others, v1 encryption policies are deprecated. Use +v2 encryption policies on all new encrypted directories. Key hierarchy ============= @@ -305,7 +305,8 @@ included in the IV. Moreover: - For v2 encryption policies, the encryption is done with a per-mode key derived using the KDF. Users may use the same master key for - other v2 encryption policies. + other v2 encryption policies. However, using a distinct master key + for each policy is still the best practice and normal usage. IV_INO_LBLK_64 policies ----------------------- @@ -336,6 +337,9 @@ per I/O request and may have only a small number of keyslots. This format results in some level of IV reuse, so it should only be used when necessary due to hardware limitations. +IV_INO_LBLK_32 is supported only when the filesystem block size is +equal to the page size. + Key identifiers --------------- @@ -601,7 +605,9 @@ This structure must be initialized as follows: struct fscrypt_policy_v1 is used or FSCRYPT_POLICY_V2 (2) if struct fscrypt_policy_v2 is used. (Note: we refer to the original policy version as "v1", though its version code is really 0.) - For new encrypted directories, use v2 policies. + For new encrypted directories, use v2 policies, which are supported + since Linux v5.4. v1 policies are deprecated and have several + usability and security problems. - ``contents_encryption_mode`` and ``filenames_encryption_mode`` must be set to constants from ``<linux/fscrypt.h>`` which identify the @@ -736,17 +742,6 @@ FS_IOC_SET_ENCRYPTION_POLICY can fail with the following errors: Getting an encryption policy ---------------------------- -Two ioctls are available to get a file's encryption policy: - -- `FS_IOC_GET_ENCRYPTION_POLICY_EX`_ -- `FS_IOC_GET_ENCRYPTION_POLICY`_ - -The extended (_EX) version of the ioctl is more general and is -recommended to use when possible. However, on older kernels only the -original ioctl is available. Applications should try the extended -version, and if it fails with ENOTTY fall back to the original -version. - FS_IOC_GET_ENCRYPTION_POLICY_EX ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -780,7 +775,6 @@ FS_IOC_GET_ENCRYPTION_POLICY_EX can fail with the following errors: - ``ENODATA``: the file is not encrypted - ``ENOTTY``: this type of filesystem does not implement encryption, or this kernel is too old to support FS_IOC_GET_ENCRYPTION_POLICY_EX - (try FS_IOC_GET_ENCRYPTION_POLICY instead) - ``EOPNOTSUPP``: the kernel was not configured with encryption support for this filesystem, or the filesystem superblock has not had encryption enabled on it @@ -796,12 +790,13 @@ check for STATX_ATTR_ENCRYPTED in stx_attributes. FS_IOC_GET_ENCRYPTION_POLICY ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The FS_IOC_GET_ENCRYPTION_POLICY ioctl can also retrieve the -encryption policy, if any, for a directory or regular file. However, -unlike `FS_IOC_GET_ENCRYPTION_POLICY_EX`_, -FS_IOC_GET_ENCRYPTION_POLICY only supports the original policy -version. It takes in a pointer directly to struct fscrypt_policy_v1 -rather than struct fscrypt_get_policy_ex_arg. +The FS_IOC_GET_ENCRYPTION_POLICY ioctl is deprecated. It supports +only v1 encryption policies, which themselves are deprecated. Use +`FS_IOC_GET_ENCRYPTION_POLICY_EX`_ instead. + +FS_IOC_GET_ENCRYPTION_POLICY retrieves the encryption policy for a +directory or regular file, but only if it uses a v1 policy. It takes +in a pointer directly to struct fscrypt_policy_v1. The error codes for FS_IOC_GET_ENCRYPTION_POLICY are the same as those for FS_IOC_GET_ENCRYPTION_POLICY_EX, except that @@ -884,6 +879,10 @@ as follows: To add this type of key, the calling process must have the CAP_SYS_ADMIN capability in the initial user namespace. + (Note that v1 encryption policies are deprecated. The ability to + add a key for v1 encryption policies remains only for compatibility + with existing encrypted directories.) + Alternatively, if the key is being added for use by v2 encryption policies, then ``key_spec.type`` must contain FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER, and ``key_spec.u.identifier`` is @@ -1315,32 +1314,20 @@ Inline encryption support Many newer systems (especially mobile SoCs) have *inline encryption hardware* that can encrypt/decrypt data while it is on its way to/from -the storage device. Linux supports inline encryption through a set of -extensions to the block layer called *blk-crypto*. blk-crypto allows -filesystems to attach encryption contexts to bios (I/O requests) to -specify how the data will be encrypted or decrypted in-line. For more -information about blk-crypto, see -:ref:`Documentation/block/inline-encryption.rst <inline_encryption>`. +the storage device. On supported filesystems (currently ext4 and f2fs), fscrypt can use -blk-crypto instead of the kernel crypto API to encrypt/decrypt file -contents. To enable this, set CONFIG_FS_ENCRYPTION_INLINE_CRYPT=y in -the kernel configuration, and specify the "inlinecrypt" mount option -when mounting the filesystem. - -Note that the "inlinecrypt" mount option just specifies to use inline -encryption when possible; it doesn't force its use. fscrypt will -still fall back to using the kernel crypto API on files where the -inline encryption hardware doesn't have the needed crypto capabilities -(e.g. support for the needed encryption algorithm and data unit size) -and where blk-crypto-fallback is unusable. (For blk-crypto-fallback -to be usable, it must be enabled in the kernel configuration with -CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK=y, and the file must be -protected by a raw key rather than a hardware-wrapped key.) - -Currently fscrypt always uses the filesystem block size (which is -usually 4096 bytes) as the data unit size. Therefore, it can only use -inline encryption hardware that supports that data unit size. +inline encryption hardware instead of the CPU to encrypt/decrypt file +contents. To enable this, specify the "inlinecrypt" mount option when +mounting the filesystem. + +This causes the filesystem to use inline encryption hardware whenever +possible, falling back to the CPU only if such hardware is absent or +doesn't provide the needed crypto capabilities. + +For more information about the kernel's support for inline encryption +hardware, see :ref:`Documentation/block/inline-encryption.rst +<inline_encryption>`. Inline encryption doesn't affect the ciphertext or other aspects of the on-disk format, so users may freely switch back and forth between @@ -1422,10 +1409,8 @@ For direct I/O on an encrypted file to work, the following conditions must be met (in addition to the conditions for direct I/O on an unencrypted file): -* The file must be using inline encryption. Usually this means that - the filesystem must be mounted with ``-o inlinecrypt`` and inline - encryption hardware must be present. However, a software fallback - is also available. For details, see `Inline encryption support`_. +* The filesystem must be block-based. (Before Linux v7.3, the + filesystem also needed to be mounted with ``-o inlinecrypt``.) * The I/O request must be fully aligned to the filesystem block size. This means that the file position the I/O is targeting, the lengths @@ -1486,25 +1471,43 @@ keys`_ and `DIRECT_KEY policies`_. Data path changes ----------------- -When inline encryption is used, filesystems just need to associate -encryption contexts with bios to specify how the block layer or the -inline encryption hardware will encrypt/decrypt the file contents. - -When inline encryption isn't used, filesystems must encrypt/decrypt -the file contents themselves, as described below: - -For the read path (->read_folio()) of regular files, filesystems can -read the ciphertext into the page cache and decrypt it in-place. The -folio lock must be held until decryption has finished, to prevent the -folio from becoming visible to userspace prematurely. - -For the write path (->writepages()) of regular files, filesystems -cannot encrypt data in-place in the page cache, since the cached -plaintext must be preserved. Instead, filesystems must encrypt into a -temporary buffer or "bounce page", then write out the temporary -buffer. Some filesystems, such as UBIFS, already use temporary -buffers regardless of encryption. Other filesystems, such as ext4 and -F2FS, have to allocate bounce pages specially for encryption. +The block-based filesystems that support fscrypt, such as ext4 and +f2fs, use blk-crypto (:ref:`inline_encryption`) to implement file +contents encryption and decryption. With blk-crypto, the filesystem +assigns an encryption context to each I/O request it issues to the +contents of an encrypted file. The encryption (for writes) or +decryption (for reads) is handled by the block layer transparently to +the filesystem, using either the CPU or inline encryption hardware. + +Non-block-based filesystems can't use blk-crypto, so they make the +calls to the cryptographic algorithms at the filesystem layer instead. + +Regardless of the layer in which they occur (blk-crypto-fallback or the +filesystem), for CPU-based encryption and decryption of file contents: + +- For reads, the ciphertext data is read from the storage backend + (block device, network, UBI device, etc.) into the destination + buffers, then decrypted in-place. The destination buffers are + pagecache folios for buffered reads, or application-provided buffers + for direct reads. In either case, the filesystem reports success + only after decryption has successfully completed. + +- For writes, the plaintext data is encrypted from the source buffers + (which cannot be modified) into bounce buffers. Then, the + ciphertext in the bounce buffers is written to the storage backend. + + The source buffers are usually pagecache folios for buffered writes, + or application-provided buffers for direct writes. There are also + some cases (all files on UBIFS, and compressed files on f2fs) where + the filesystem already uses bounce buffers for writes for other + reasons; in these cases the source plaintext data is already in + bounce buffers. UBIFS optimizes this case by encrypting the data + in-place in its existing bounce buffers. + +When inline encryption hardware is used instead of the CPU, reads from +the storage backend logically return plaintext data, and writes accept +plaintext data. In that case the flow is simplified: there's no +scheduling of decryption work, and no bounce buffers are used. Filename hashing and encoding ----------------------------- @@ -1552,14 +1555,11 @@ Tests To test fscrypt, use xfstests, which is Linux's de facto standard filesystem test suite. First, run all the tests in the "encrypt" -group on the relevant filesystem(s). One can also run the tests -with the 'inlinecrypt' mount option to test the implementation for -inline encryption support. For example, to test ext4 and +group on the relevant filesystem(s). For example, to test ext4 and f2fs encryption using `kvm-xfstests <https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_:: kvm-xfstests -c ext4,f2fs -g encrypt - kvm-xfstests -c ext4,f2fs -g encrypt -m inlinecrypt UBIFS encryption can also be tested this way, but it should be done in a separate command, and it takes some time for kvm-xfstests to set up @@ -1581,7 +1581,6 @@ This tests the encrypted I/O paths more thoroughly. To do this with kvm-xfstests, use the "encrypt" filesystem configuration:: kvm-xfstests -c ext4/encrypt,f2fs/encrypt -g auto - kvm-xfstests -c ext4/encrypt,f2fs/encrypt -g auto -m inlinecrypt Because this runs many more tests than "-g encrypt" does, it takes much longer to run; so also consider using `gce-xfstests @@ -1589,4 +1588,9 @@ much longer to run; so also consider using `gce-xfstests instead of kvm-xfstests:: gce-xfstests -c ext4/encrypt,f2fs/encrypt -g auto - gce-xfstests -c ext4/encrypt,f2fs/encrypt -g auto -m inlinecrypt + +To test inline encryption hardware on a platform that supports such +hardware, run xfstests directly with the ``inlinecrypt`` mount option +enabled. For example:: + + EXT_MOUNT_OPTIONS="-o inlinecrypt" ./check -g encrypt diff --git a/arch/loongarch/configs/loongson32_defconfig b/arch/loongarch/configs/loongson32_defconfig index 7c8f01513ed2..6bf2867dbdc6 100644 --- a/arch/loongarch/configs/loongson32_defconfig +++ b/arch/loongarch/configs/loongson32_defconfig @@ -969,7 +969,6 @@ CONFIG_F2FS_FS_SECURITY=y CONFIG_F2FS_CHECK_FS=y CONFIG_F2FS_FS_COMPRESSION=y CONFIG_FS_ENCRYPTION=y -CONFIG_FS_ENCRYPTION_INLINE_CRYPT=y CONFIG_FS_VERITY=y CONFIG_FANOTIFY=y CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y diff --git a/arch/loongarch/configs/loongson64_defconfig b/arch/loongarch/configs/loongson64_defconfig index 8e3906d3bd70..def104c9d405 100644 --- a/arch/loongarch/configs/loongson64_defconfig +++ b/arch/loongarch/configs/loongson64_defconfig @@ -1000,7 +1000,6 @@ CONFIG_F2FS_FS_SECURITY=y CONFIG_F2FS_CHECK_FS=y CONFIG_F2FS_FS_COMPRESSION=y CONFIG_FS_ENCRYPTION=y -CONFIG_FS_ENCRYPTION_INLINE_CRYPT=y CONFIG_FS_VERITY=y CONFIG_FANOTIFY=y CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y diff --git a/block/blk-crypto-fallback.c b/block/blk-crypto-fallback.c index 2a5c52ab74b4..2a8f40a65158 100644 --- a/block/blk-crypto-fallback.c +++ b/block/blk-crypto-fallback.c @@ -496,8 +496,7 @@ bool blk_crypto_fallback_bio_prep(struct bio *bio) return false; } - if (!__blk_crypto_cfg_supported(blk_crypto_fallback_profile, - &bc->bc_key->crypto_cfg)) { + if (bc->bc_key->crypto_cfg.key_type != BLK_CRYPTO_KEY_TYPE_RAW) { bio_endio_status(bio, BLK_STS_NOTSUPP); return false; } diff --git a/block/blk-crypto-internal.h b/block/blk-crypto-internal.h index 742694213529..2c7a0446572a 100644 --- a/block/blk-crypto-internal.h +++ b/block/blk-crypto-internal.h @@ -80,9 +80,6 @@ void blk_crypto_put_keyslot(struct blk_crypto_keyslot *slot); int __blk_crypto_evict_key(struct blk_crypto_profile *profile, const struct blk_crypto_key *key); -bool __blk_crypto_cfg_supported(struct blk_crypto_profile *profile, - const struct blk_crypto_config *cfg); - int blk_crypto_ioctl(struct block_device *bdev, unsigned int cmd, void __user *argp); diff --git a/block/blk-crypto-profile.c b/block/blk-crypto-profile.c index cf447ba4a66e..53126c091b0b 100644 --- a/block/blk-crypto-profile.c +++ b/block/blk-crypto-profile.c @@ -335,28 +335,6 @@ void blk_crypto_put_keyslot(struct blk_crypto_keyslot *slot) } } -/** - * __blk_crypto_cfg_supported() - Check whether the given crypto profile - * supports the given crypto configuration. - * @profile: the crypto profile to check - * @cfg: the crypto configuration to check for - * - * Return: %true if @profile supports the given @cfg. - */ -bool __blk_crypto_cfg_supported(struct blk_crypto_profile *profile, - const struct blk_crypto_config *cfg) -{ - if (!profile) - return false; - if (!(profile->modes_supported[cfg->crypto_mode] & cfg->data_unit_size)) - return false; - if (profile->max_dun_bytes_supported < cfg->dun_bytes) - return false; - if (!(profile->key_types_supported & cfg->key_type)) - return false; - return true; -} - /* * This is an internal function that evicts a key from an inline encryption * device that can be either a real device or the blk-crypto-fallback "device". diff --git a/block/blk-crypto.c b/block/blk-crypto.c index 15e25e41b166..bc3a9f59574b 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -300,6 +300,7 @@ int __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, * @dun_bytes: number of bytes that will be used to specify the DUN when this * key is used * @data_unit_size: the data unit size to use for en/decryption + * @flags: BLK_CRYPTO_CFG_* flags * * Return: 0 on success, -errno on failure. The caller is responsible for * zeroizing both blk_key and key_bytes when done with them. @@ -309,7 +310,7 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, enum blk_crypto_key_type key_type, enum blk_crypto_mode_num crypto_mode, unsigned int dun_bytes, - unsigned int data_unit_size) + unsigned int data_unit_size, int flags) { const struct blk_crypto_mode *mode; @@ -318,6 +319,9 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, if (crypto_mode >= ARRAY_SIZE(blk_crypto_modes)) return -EINVAL; + if (flags & ~BLK_CRYPTO_CFG_ALLOW_HW) + return -EINVAL; + mode = &blk_crypto_modes[crypto_mode]; switch (key_type) { case BLK_CRYPTO_KEY_TYPE_RAW: @@ -328,6 +332,8 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, if (key_size < mode->security_strength || key_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) return -EINVAL; + if (!(flags & BLK_CRYPTO_CFG_ALLOW_HW)) + return -EINVAL; break; default: return -EINVAL; @@ -343,6 +349,7 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, blk_key->crypto_cfg.dun_bytes = dun_bytes; blk_key->crypto_cfg.data_unit_size = data_unit_size; blk_key->crypto_cfg.key_type = key_type; + blk_key->crypto_cfg.flags = flags; blk_key->data_unit_size_bits = ilog2(data_unit_size); blk_key->size = key_size; memcpy(blk_key->bytes, key_bytes, key_size); @@ -351,25 +358,32 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, } EXPORT_SYMBOL_GPL(blk_crypto_init_key); +/** + * blk_crypto_config_supported_natively() - Check whether a block device + * supports hardware inline encryption + * with the given configuration. + * @bdev: the block device + * @cfg: the crypto configuration to check for + * + * Return: %true if @bdev supports hardware inline encryption with @cfg. + */ bool blk_crypto_config_supported_natively(struct block_device *bdev, const struct blk_crypto_config *cfg) { - return __blk_crypto_cfg_supported(bdev_get_queue(bdev)->crypto_profile, - cfg); -} + struct blk_crypto_profile *profile = + bdev_get_queue(bdev)->crypto_profile; -/* - * Check if bios with @cfg can be en/decrypted by blk-crypto (i.e. either the - * block_device it's submitted to supports inline crypto, or the - * blk-crypto-fallback is enabled and supports the cfg). - */ -bool blk_crypto_config_supported(struct block_device *bdev, - const struct blk_crypto_config *cfg) -{ - if (IS_ENABLED(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) && - cfg->key_type == BLK_CRYPTO_KEY_TYPE_RAW) - return true; - return blk_crypto_config_supported_natively(bdev, cfg); + if (!profile) + return false; + if (!(cfg->flags & BLK_CRYPTO_CFG_ALLOW_HW)) + return false; + if (!(profile->modes_supported[cfg->crypto_mode] & cfg->data_unit_size)) + return false; + if (profile->max_dun_bytes_supported < cfg->dun_bytes) + return false; + if (!(profile->key_types_supported & cfg->key_type)) + return false; + return true; } /** diff --git a/drivers/md/dm-inlinecrypt.c b/drivers/md/dm-inlinecrypt.c index 41293c18d10f..f50970db0f94 100644 --- a/drivers/md/dm-inlinecrypt.c +++ b/drivers/md/dm-inlinecrypt.c @@ -408,7 +408,8 @@ static int inlinecrypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) err = blk_crypto_init_key(&ctx->key, key_bytes, ctx->key_size, ctx->key_type, cipher->mode_num, - dun_bytes, ctx->sector_size); + dun_bytes, ctx->sector_size, + BLK_CRYPTO_CFG_ALLOW_HW); if (err) { ti->error = "Error initializing blk-crypto key"; goto bad; diff --git a/fs/buffer.c b/fs/buffer.c index be8b57a635cd..65a2893b2cbb 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -336,7 +336,7 @@ still_busy: spin_unlock_irqrestore(&first->b_uptodate_lock, flags); } -struct postprocess_bh_ctx { +struct verify_bh_ctx { struct work_struct work; struct buffer_head *bh; struct fsverity_info *vi; @@ -344,8 +344,8 @@ struct postprocess_bh_ctx { static void verify_bh(struct work_struct *work) { - struct postprocess_bh_ctx *ctx = - container_of(work, struct postprocess_bh_ctx, work); + struct verify_bh_ctx *ctx = + container_of(work, struct verify_bh_ctx, work); struct buffer_head *bh = ctx->bh; bool valid; @@ -355,29 +355,6 @@ static void verify_bh(struct work_struct *work) kfree(ctx); } -static void decrypt_bh(struct work_struct *work) -{ - struct postprocess_bh_ctx *ctx = - container_of(work, struct postprocess_bh_ctx, work); - struct buffer_head *bh = ctx->bh; - int err; - - err = fscrypt_decrypt_pagecache_blocks(bh->b_folio, bh->b_size, - bh_offset(bh)); - if (err == 0 && ctx->vi) { - /* - * We use different work queues for decryption and for verity - * because verity may require reading metadata pages that need - * decryption, and we shouldn't recurse to the same workqueue. - */ - INIT_WORK(&ctx->work, verify_bh); - fsverity_enqueue_verify_work(&ctx->work); - return; - } - end_buffer_async_read(bh, err == 0); - kfree(ctx); -} - /* * I/O completion handler for block_read_full_folio() - folios * which come unlocked at the end of I/O. @@ -387,27 +364,21 @@ static void bh_end_async_read(struct bio *bio) struct buffer_head *bh; bool uptodate = bio_endio_bh(bio, &bh); struct inode *inode = bh->b_folio->mapping->host; - bool decrypt = fscrypt_inode_uses_fs_layer_crypto(inode); struct fsverity_info *vi = NULL; /* needed by ext4 */ if (bh->b_folio->index < DIV_ROUND_UP(inode->i_size, PAGE_SIZE)) vi = fsverity_get_info(inode); - /* Decrypt (with fscrypt) and/or verify (with fsverity) if needed. */ - if (uptodate && (decrypt || vi)) { - struct postprocess_bh_ctx *ctx = kmalloc_obj(*ctx, GFP_ATOMIC); + /* Verify (with fsverity) if needed. */ + if (vi && uptodate) { + struct verify_bh_ctx *ctx = kmalloc_obj(*ctx, GFP_ATOMIC); if (ctx) { ctx->bh = bh; ctx->vi = vi; - if (decrypt) { - INIT_WORK(&ctx->work, decrypt_bh); - fscrypt_enqueue_decrypt_work(&ctx->work); - } else { - INIT_WORK(&ctx->work, verify_bh); - fsverity_enqueue_verify_work(&ctx->work); - } + INIT_WORK(&ctx->work, verify_bh); + fsverity_enqueue_verify_work(&ctx->work); return; } uptodate = false; diff --git a/fs/crypto/Kconfig b/fs/crypto/Kconfig index 983d8ad1f417..cd934e31dec4 100644 --- a/fs/crypto/Kconfig +++ b/fs/crypto/Kconfig @@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-only config FS_ENCRYPTION bool "FS Encryption (Per-file encryption)" + select BLK_INLINE_ENCRYPTION if BLOCK + select BLK_INLINE_ENCRYPTION_FALLBACK if BLOCK select CRYPTO select CRYPTO_SKCIPHER select CRYPTO_LIB_AES @@ -34,7 +36,5 @@ config FS_ENCRYPTION_ALGS select CRYPTO_XTS config FS_ENCRYPTION_INLINE_CRYPT - bool "Enable fscrypt to use inline crypto" - depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION - help - Enable fscrypt to use inline encryption hardware if available. + bool + default y if FS_ENCRYPTION && BLOCK diff --git a/fs/crypto/Makefile b/fs/crypto/Makefile index 652c7180ec6d..b03e02f0f09d 100644 --- a/fs/crypto/Makefile +++ b/fs/crypto/Makefile @@ -10,5 +10,4 @@ fscrypto-y := crypto.o \ keysetup_v1.o \ policy.o -fscrypto-$(CONFIG_BLOCK) += bio.o -fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o +fscrypto-$(CONFIG_BLOCK) += block.o diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c deleted file mode 100644 index d07740680602..000000000000 --- a/fs/crypto/bio.c +++ /dev/null @@ -1,216 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Utility functions for file contents encryption/decryption on - * block device-based filesystems. - * - * Copyright (C) 2015, Google, Inc. - * Copyright (C) 2015, Motorola Mobility - */ - -#include <linux/bio.h> -#include <linux/export.h> -#include <linux/module.h> -#include <linux/namei.h> -#include <linux/pagemap.h> - -#include "fscrypt_private.h" - -/** - * fscrypt_decrypt_bio() - decrypt the contents of a bio - * @bio: the bio to decrypt - * - * Decrypt the contents of a "read" bio following successful completion of the - * underlying disk read. The bio must be reading a whole number of blocks of an - * encrypted file directly into the page cache. If the bio is reading the - * ciphertext into bounce pages instead of the page cache (for example, because - * the file is also compressed, so decompression is required after decryption), - * then this function isn't applicable. This function may sleep, so it must be - * called from a workqueue rather than from the bio's bi_end_io callback. - * - * Return: %true on success; %false on failure. On failure, bio->bi_status is - * also set to an error status. - */ -bool fscrypt_decrypt_bio(struct bio *bio) -{ - struct folio_iter fi; - - bio_for_each_folio_all(fi, bio) { - int err = fscrypt_decrypt_pagecache_blocks(fi.folio, fi.length, - fi.offset); - - if (err) { - bio->bi_status = errno_to_blk_status(err); - return false; - } - } - return true; -} -EXPORT_SYMBOL(fscrypt_decrypt_bio); - -struct fscrypt_zero_done { - atomic_t pending; - blk_status_t status; - struct completion done; -}; - -static void fscrypt_zeroout_range_done(struct fscrypt_zero_done *done) -{ - if (atomic_dec_and_test(&done->pending)) - complete(&done->done); -} - -static void fscrypt_zeroout_range_end_io(struct bio *bio) -{ - struct fscrypt_zero_done *done = bio->bi_private; - - if (bio->bi_status) - cmpxchg(&done->status, 0, bio->bi_status); - fscrypt_zeroout_range_done(done); - bio_put(bio); -} - -static int fscrypt_zeroout_range_inline_crypt(const struct inode *inode, - loff_t pos, sector_t sector, - u64 len) -{ - struct fscrypt_zero_done done = { - .pending = ATOMIC_INIT(1), - .done = COMPLETION_INITIALIZER_ONSTACK(done.done), - }; - - while (len) { - struct bio *bio; - unsigned int n; - - bio = bio_alloc(inode->i_sb->s_bdev, BIO_MAX_VECS, REQ_OP_WRITE, - GFP_NOFS); - bio->bi_iter.bi_sector = sector; - bio->bi_private = &done; - bio->bi_end_io = fscrypt_zeroout_range_end_io; - fscrypt_set_bio_crypt_ctx(bio, inode, pos, GFP_NOFS); - - for (n = 0; n < BIO_MAX_VECS; n++) { - unsigned int bytes_this_page = min(len, PAGE_SIZE); - - __bio_add_page(bio, ZERO_PAGE(0), bytes_this_page, 0); - len -= bytes_this_page; - pos += bytes_this_page; - sector += (bytes_this_page >> SECTOR_SHIFT); - if (!len || !fscrypt_mergeable_bio(bio, inode, pos)) - break; - } - - atomic_inc(&done.pending); - blk_crypto_submit_bio(bio); - } - - fscrypt_zeroout_range_done(&done); - - wait_for_completion(&done.done); - return blk_status_to_errno(done.status); -} - -/** - * fscrypt_zeroout_range() - zero out a range of blocks in an encrypted file - * @inode: the file's inode - * @pos: the first file position (in bytes) to zero out - * @sector: the first sector to zero out - * @len: bytes to zero out - * - * Zero out filesystem blocks in an encrypted regular file on-disk, i.e. write - * ciphertext blocks which decrypt to the all-zeroes block. The blocks must be - * both logically and physically contiguous. It's also assumed that the - * filesystem only uses a single block device, ->s_bdev. @len must be a - * multiple of the file system logical block size. - * - * Note that since each block uses a different IV, this involves writing a - * different ciphertext to each block; we can't simply reuse the same one. - * - * Return: 0 on success; -errno on failure. - */ -int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, - sector_t sector, u64 len) -{ - const struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode); - const unsigned int du_bits = ci->ci_data_unit_bits; - const unsigned int du_size = 1U << du_bits; - const unsigned int du_per_page_bits = PAGE_SHIFT - du_bits; - const unsigned int du_per_page = 1U << du_per_page_bits; - u64 du_index = pos >> du_bits; - u64 du_remaining = len >> du_bits; - struct page *pages[16]; /* write up to 16 pages at a time */ - unsigned int nr_pages; - unsigned int i; - unsigned int offset; - struct bio *bio; - int ret, err; - - if (len == 0) - return 0; - - if (fscrypt_inode_uses_inline_crypto(inode)) - return fscrypt_zeroout_range_inline_crypt(inode, pos, sector, - len); - - BUILD_BUG_ON(ARRAY_SIZE(pages) > BIO_MAX_VECS); - nr_pages = min_t(u64, ARRAY_SIZE(pages), - (du_remaining + du_per_page - 1) >> du_per_page_bits); - - /* - * We need at least one page for ciphertext. Allocate the first one - * from a mempool, with __GFP_DIRECT_RECLAIM set so that it can't fail. - * - * Any additional page allocations are allowed to fail, as they only - * help performance, and waiting on the mempool for them could deadlock. - */ - for (i = 0; i < nr_pages; i++) { - pages[i] = fscrypt_alloc_bounce_page(i == 0 ? GFP_NOFS : - GFP_NOWAIT); - if (!pages[i]) - break; - } - nr_pages = i; - if (WARN_ON_ONCE(nr_pages <= 0)) - return -EINVAL; - - /* This always succeeds since __GFP_DIRECT_RECLAIM is set. */ - bio = bio_alloc(inode->i_sb->s_bdev, nr_pages, REQ_OP_WRITE, GFP_NOFS); - - do { - bio->bi_iter.bi_sector = sector; - - i = 0; - offset = 0; - do { - err = fscrypt_crypt_data_unit(ci, FS_ENCRYPT, du_index, - ZERO_PAGE(0), pages[i], - du_size, offset); - if (err) - goto out; - du_index++; - sector += 1U << (du_bits - SECTOR_SHIFT); - du_remaining--; - offset += du_size; - if (offset == PAGE_SIZE || du_remaining == 0) { - ret = bio_add_page(bio, pages[i++], offset, 0); - if (WARN_ON_ONCE(ret != offset)) { - err = -EIO; - goto out; - } - offset = 0; - } - } while (i != nr_pages && du_remaining != 0); - - err = submit_bio_wait(bio); - if (err) - goto out; - bio_reset(bio, inode->i_sb->s_bdev, REQ_OP_WRITE); - } while (du_remaining != 0); - err = 0; -out: - bio_put(bio); - for (i = 0; i < nr_pages; i++) - fscrypt_free_bounce_page(pages[i]); - return err; -} -EXPORT_SYMBOL(fscrypt_zeroout_range); diff --git a/fs/crypto/inline_crypt.c b/fs/crypto/block.c index 66b9c9150fed..5193f8ba3ee0 100644 --- a/fs/crypto/inline_crypt.c +++ b/fs/crypto/block.c @@ -1,20 +1,20 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Inline encryption support for fscrypt + * File contents en/decryption on block-based filesystems * * Copyright 2019 Google LLC */ /* - * With "inline encryption", the block layer handles the decryption/encryption - * as part of the bio, instead of the filesystem doing the crypto itself via - * crypto API. See Documentation/block/inline-encryption.rst. fscrypt still - * provides the key and IV to use. + * This file implements fscrypt's file contents en/decryption using blk-crypto + * (Documentation/block/inline-encryption.rst). fscrypt assigns a bio_crypt_ctx + * with a key and IV to each bio, and the block layer does the en/decryption. + * + * This file's exported functions are called only by block-based filesystems. */ #include <linux/blk-crypto.h> #include <linux/blkdev.h> -#include <linux/buffer_head.h> #include <linux/export.h> #include <linux/sched/mm.h> #include <linux/slab.h> @@ -62,84 +62,19 @@ static unsigned int fscrypt_get_dun_bytes(const struct fscrypt_inode_info *ci) * helpful for debugging problems where the "wrong" implementation is used. */ static void fscrypt_log_blk_crypto_impl(struct fscrypt_mode *mode, - struct block_device **devs, - unsigned int num_devs, - const struct blk_crypto_config *cfg) + struct block_device *dev, + const struct blk_crypto_key *blk_key) { - unsigned int i; - - for (i = 0; i < num_devs; i++) { - if (!IS_ENABLED(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) || - blk_crypto_config_supported_natively(devs[i], cfg)) { - if (!xchg(&mode->logged_blk_crypto_native, 1)) - pr_info("fscrypt: %s using blk-crypto (native)\n", - mode->friendly_name); - } else if (!xchg(&mode->logged_blk_crypto_fallback, 1)) { - pr_info("fscrypt: %s using blk-crypto-fallback\n", + if (blk_crypto_config_supported_natively(dev, &blk_key->crypto_cfg)) { + if (!xchg(&mode->logged_blk_crypto_native, 1)) + pr_info("fscrypt: %s using blk-crypto (native)\n", mode->friendly_name); - } + } else if (!xchg(&mode->logged_blk_crypto_fallback, 1)) { + pr_info("fscrypt: %s using blk-crypto-fallback\n", + mode->friendly_name); } } -/* Enable inline encryption for this file if supported. */ -int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci, - bool is_hw_wrapped_key) -{ - const struct inode *inode = ci->ci_inode; - struct super_block *sb = inode->i_sb; - struct blk_crypto_config crypto_cfg; - struct block_device *devs[FSCRYPT_MAX_DEVICES]; - unsigned int num_devs; - unsigned int i; - - /* The file must need contents encryption, not filenames encryption */ - if (!S_ISREG(inode->i_mode)) - return 0; - - /* The crypto mode must have a blk-crypto counterpart */ - if (ci->ci_mode->blk_crypto_mode == BLK_ENCRYPTION_MODE_INVALID) - return 0; - - /* The filesystem must be mounted with -o inlinecrypt */ - if (!(sb->s_flags & SB_INLINECRYPT)) - return 0; - - /* - * When a page contains multiple logically contiguous filesystem blocks, - * some filesystem code only calls fscrypt_mergeable_bio() for the first - * block in the page. This is fine for most of fscrypt's IV generation - * strategies, where contiguous blocks imply contiguous IVs. But it - * doesn't work with IV_INO_LBLK_32. For now, simply exclude - * IV_INO_LBLK_32 with blocksize != PAGE_SIZE from inline encryption. - */ - if ((fscrypt_policy_flags(&ci->ci_policy) & - FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) && - sb->s_blocksize != PAGE_SIZE) - return 0; - - /* - * On all the filesystem's block devices, blk-crypto must support the - * crypto configuration that the file would use. - */ - crypto_cfg.crypto_mode = ci->ci_mode->blk_crypto_mode; - crypto_cfg.data_unit_size = 1U << ci->ci_data_unit_bits; - crypto_cfg.dun_bytes = fscrypt_get_dun_bytes(ci); - crypto_cfg.key_type = is_hw_wrapped_key ? - BLK_CRYPTO_KEY_TYPE_HW_WRAPPED : BLK_CRYPTO_KEY_TYPE_RAW; - - num_devs = fscrypt_get_devices(sb, devs); - for (i = 0; i < num_devs; i++) { - if (!blk_crypto_config_supported(devs[i], &crypto_cfg)) - return 0; - } - - fscrypt_log_blk_crypto_impl(ci->ci_mode, devs, num_devs, &crypto_cfg); - - ci->ci_inlinecrypt = true; - - return 0; -} - int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, const u8 *key_bytes, size_t key_size, bool is_hw_wrapped, @@ -147,7 +82,8 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, { const struct inode *inode = ci->ci_inode; struct super_block *sb = inode->i_sb; - enum blk_crypto_mode_num crypto_mode = ci->ci_mode->blk_crypto_mode; + bool inlinecrypt = sb->s_flags & SB_INLINECRYPT; + struct fscrypt_mode *mode = ci->ci_mode; enum blk_crypto_key_type key_type = is_hw_wrapped ? BLK_CRYPTO_KEY_TYPE_HW_WRAPPED : BLK_CRYPTO_KEY_TYPE_RAW; struct blk_crypto_key *blk_key; @@ -156,15 +92,28 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, unsigned int i; int err; + if (is_hw_wrapped && !inlinecrypt) { + /* + * blk_crypto_init_key() would catch this anyway, but this + * provides a clearer error message. + */ + fscrypt_err( + inode, + "Hardware-wrapped keys require inline encryption (-o inlinecrypt)"); + return -EINVAL; + } + blk_key = kmalloc_obj(*blk_key); if (!blk_key) return -ENOMEM; err = blk_crypto_init_key(blk_key, key_bytes, key_size, key_type, - crypto_mode, fscrypt_get_dun_bytes(ci), - 1U << ci->ci_data_unit_bits); + mode->blk_crypto_mode, + fscrypt_get_dun_bytes(ci), + 1U << ci->ci_data_unit_bits, + inlinecrypt ? BLK_CRYPTO_CFG_ALLOW_HW : 0); if (err) { - fscrypt_err(inode, "error %d initializing blk-crypto key", err); + fscrypt_err(inode, "Error %d initializing blk-crypto key", err); goto fail; } @@ -174,9 +123,16 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, err = blk_crypto_start_using_key(devs[i], blk_key); if (err) break; + fscrypt_log_blk_crypto_impl(mode, devs[i], blk_key); } if (err) { - fscrypt_err(inode, "error %d starting to use blk-crypto", err); + if (err == -EOPNOTSUPP && is_hw_wrapped) + fscrypt_err( + inode, + "Hardware-wrapped key required, but no suitable inline encryption capabilities are available"); + else + fscrypt_err(inode, + "Error %d starting to use blk-crypto", err); goto fail; } @@ -238,12 +194,6 @@ int fscrypt_derive_sw_secret(struct super_block *sb, return err; } -bool __fscrypt_inode_uses_inline_crypto(const struct inode *inode) -{ - return fscrypt_get_inode_info_raw(inode)->ci_inlinecrypt; -} -EXPORT_SYMBOL_GPL(__fscrypt_inode_uses_inline_crypto); - static void fscrypt_generate_dun(const struct fscrypt_inode_info *ci, loff_t pos, u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE]) { @@ -266,8 +216,8 @@ static void fscrypt_generate_dun(const struct fscrypt_inode_info *ci, * @gfp_mask: memory allocation flags - these must be a waiting mask so that * bio_crypt_set_ctx can't fail. * - * If the contents of the file should be encrypted (or decrypted) with inline - * encryption, then assign the appropriate encryption context to the bio. + * If the contents of the file should be encrypted (or decrypted), then assign + * the appropriate encryption context to the bio. * * Normally the bio should be newly allocated (i.e. no pages added yet), as * otherwise fscrypt_mergeable_bio() won't work as intended. @@ -280,7 +230,7 @@ void fscrypt_set_bio_crypt_ctx(struct bio *bio, const struct inode *inode, const struct fscrypt_inode_info *ci; u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE]; - if (!fscrypt_inode_uses_inline_crypto(inode)) + if (!fscrypt_needs_contents_encryption(inode)) return; ci = fscrypt_get_inode_info_raw(inode); @@ -295,12 +245,12 @@ EXPORT_SYMBOL_GPL(fscrypt_set_bio_crypt_ctx); * @inode: the inode for the next part of the I/O * @pos: the next file position (in bytes) in the I/O * - * When building a bio which may contain data which should undergo inline - * encryption (or decryption) via fscrypt, filesystems should call this function - * to ensure that the resulting bio contains only contiguous data unit numbers. - * This will return false if the next part of the I/O cannot be merged with the - * bio because either the encryption key would be different or the encryption - * data unit numbers would be discontiguous. + * When building a bio which may contain data which should undergo encryption + * (or decryption) via fscrypt, filesystems should call this function to ensure + * that the resulting bio contains only contiguous data unit numbers. This will + * return false if the next part of the I/O cannot be merged with the bio + * because either the encryption key would be different or the encryption data + * unit numbers would be discontiguous. * * fscrypt_set_bio_crypt_ctx() must have already been called on the bio. * @@ -317,7 +267,7 @@ bool fscrypt_mergeable_bio(struct bio *bio, const struct inode *inode, const struct fscrypt_inode_info *ci; u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE]; - if (!!bc != fscrypt_inode_uses_inline_crypto(inode)) + if (!!bc != fscrypt_needs_contents_encryption(inode)) return false; if (!bc) return true; @@ -337,49 +287,6 @@ bool fscrypt_mergeable_bio(struct bio *bio, const struct inode *inode, EXPORT_SYMBOL_GPL(fscrypt_mergeable_bio); /** - * fscrypt_dio_supported() - check whether DIO (direct I/O) is supported on an - * inode, as far as encryption is concerned - * @inode: the inode in question - * - * Return: %true if there are no encryption constraints that prevent DIO from - * being supported; %false if DIO is unsupported. (Note that in the - * %true case, the filesystem might have other, non-encryption-related - * constraints that prevent DIO from actually being supported. Also, on - * encrypted files the filesystem is still responsible for only allowing - * DIO when requests are filesystem-block-aligned.) - */ -bool fscrypt_dio_supported(struct inode *inode) -{ - int err; - - /* If the file is unencrypted, no veto from us. */ - if (!fscrypt_needs_contents_encryption(inode)) - return true; - - /* - * We only support DIO with inline crypto, not fs-layer crypto. - * - * To determine whether the inode is using inline crypto, we have to set - * up the key if it wasn't already done. This is because in the current - * design of fscrypt, the decision of whether to use inline crypto or - * not isn't made until the inode's encryption key is being set up. In - * the DIO read/write case, the key will always be set up already, since - * the file will be open. But in the case of statx(), the key might not - * be set up yet, as the file might not have been opened yet. - */ - err = fscrypt_require_key(inode); - if (err) { - /* - * Key unavailable or couldn't be set up. This edge case isn't - * worth worrying about; just report that DIO is unsupported. - */ - return false; - } - return fscrypt_inode_uses_inline_crypto(inode); -} -EXPORT_SYMBOL_GPL(fscrypt_dio_supported); - -/** * fscrypt_limit_io_blocks() - limit I/O blocks to avoid discontiguous DUNs * @inode: the file on which I/O is being done * @lblk: the block at which the I/O is being started from @@ -404,7 +311,7 @@ u64 fscrypt_limit_io_blocks(const struct inode *inode, u64 lblk, u64 nr_blocks) const struct fscrypt_inode_info *ci; u32 dun; - if (!fscrypt_inode_uses_inline_crypto(inode)) + if (!fscrypt_needs_contents_encryption(inode)) return nr_blocks; if (nr_blocks <= 1) @@ -422,3 +329,87 @@ u64 fscrypt_limit_io_blocks(const struct inode *inode, u64 lblk, u64 nr_blocks) return min_t(u64, nr_blocks, (u64)U32_MAX + 1 - dun); } EXPORT_SYMBOL_GPL(fscrypt_limit_io_blocks); + +struct fscrypt_zero_done { + atomic_t pending; + blk_status_t status; + struct completion done; +}; + +static void fscrypt_zeroout_range_done(struct fscrypt_zero_done *done) +{ + if (atomic_dec_and_test(&done->pending)) + complete(&done->done); +} + +static void fscrypt_zeroout_range_end_io(struct bio *bio) +{ + struct fscrypt_zero_done *done = bio->bi_private; + + if (bio->bi_status) + cmpxchg(&done->status, 0, bio->bi_status); + fscrypt_zeroout_range_done(done); + bio_put(bio); +} + +/** + * fscrypt_zeroout_range() - zero out a range of blocks in an encrypted file + * @inode: the file's inode + * @pos: the first file position (in bytes) to zero out + * @sector: the first sector to zero out + * @len: bytes to zero out + * + * Zero out filesystem blocks in an encrypted regular file on-disk, i.e. write + * ciphertext blocks which decrypt to the all-zeroes block. The blocks must be + * both logically and physically contiguous. It's also assumed that the + * filesystem only uses a single block device, ->s_bdev. @len must be a + * multiple of the file system logical block size. + * + * Note that since each block uses a different IV, this involves writing a + * different ciphertext to each block; we can't simply reuse the same one. + * + * Return: 0 on success; -errno on failure. + */ +int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, + sector_t sector, u64 len) +{ + struct fscrypt_zero_done done = { + .pending = ATOMIC_INIT(1), + .done = COMPLETION_INITIALIZER_ONSTACK(done.done), + }; + + if (len == 0) + return 0; + + do { + struct bio *bio; + unsigned int n; + + bio = bio_alloc(inode->i_sb->s_bdev, BIO_MAX_VECS, REQ_OP_WRITE, + GFP_NOFS); + bio->bi_iter.bi_sector = sector; + bio->bi_private = &done; + bio->bi_end_io = fscrypt_zeroout_range_end_io; + fscrypt_set_bio_crypt_ctx(bio, inode, pos, GFP_NOFS); + + for (n = 0; n < BIO_MAX_VECS; n++) { + unsigned int bytes_this_page = min(len, PAGE_SIZE); + + __bio_add_page(bio, ZERO_PAGE(0), bytes_this_page, 0); + len -= bytes_this_page; + pos += bytes_this_page; + sector += (bytes_this_page >> SECTOR_SHIFT); + if (!len || !fscrypt_mergeable_bio(bio, inode, pos)) + break; + } + + atomic_inc(&done.pending); + blk_crypto_submit_bio(bio); + } while (len); + + fscrypt_zeroout_range_done(&done); + + wait_for_completion(&done.done); + return blk_status_to_errno(done.status); +} +EXPORT_SYMBOL(fscrypt_zeroout_range); diff --git a/fs/crypto/crypto.c b/fs/crypto/crypto.c index 570a2231c945..5286a124b0d9 100644 --- a/fs/crypto/crypto.c +++ b/fs/crypto/crypto.c @@ -38,18 +38,11 @@ MODULE_PARM_DESC(num_prealloc_crypto_pages, static mempool_t *fscrypt_bounce_page_pool = NULL; -static struct workqueue_struct *fscrypt_read_workqueue; static DEFINE_MUTEX(fscrypt_init_mutex); struct kmem_cache *fscrypt_inode_info_cachep; -void fscrypt_enqueue_decrypt_work(struct work_struct *work) -{ - queue_work(fscrypt_read_workqueue, work); -} -EXPORT_SYMBOL(fscrypt_enqueue_decrypt_work); - -struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags) +static struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags) { if (WARN_ON_ONCE(!fscrypt_bounce_page_pool)) { /* @@ -65,8 +58,7 @@ struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags) * fscrypt_free_bounce_page() - free a ciphertext bounce page * @bounce_page: the bounce page to free, or NULL * - * Free a bounce page that was allocated by fscrypt_encrypt_pagecache_blocks(), - * or by fscrypt_alloc_bounce_page() directly. + * Free a bounce page that was allocated by fscrypt_encrypt_pagecache_blocks(). */ void fscrypt_free_bounce_page(struct page *bounce_page) { @@ -91,7 +83,7 @@ void fscrypt_generate_iv(union fscrypt_iv *iv, u64 index, { u8 flags = fscrypt_policy_flags(&ci->ci_policy); - memset(iv, 0, ci->ci_mode->ivsize); + memset(iv, 0, sizeof(*iv)); if (flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64) { WARN_ON_ONCE(index > U32_MAX); @@ -107,17 +99,23 @@ void fscrypt_generate_iv(union fscrypt_iv *iv, u64 index, } /* Encrypt or decrypt a single "data unit" of file contents. */ -int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci, - fscrypt_direction_t rw, u64 index, - struct page *src_page, struct page *dest_page, - unsigned int len, unsigned int offs) +static int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci, + fscrypt_direction_t rw, u64 index, + struct page *src_page, + struct page *dest_page, unsigned int len, + unsigned int offs) { - struct crypto_sync_skcipher *tfm = ci->ci_enc_key.tfm; - SYNC_SKCIPHER_REQUEST_ON_STACK(req, tfm); + struct crypto_sync_skcipher *tfm; union fscrypt_iv iv; struct scatterlist dst, src; int err; + if (WARN_ON_ONCE(ci == NULL)) /* File hasn't been opened yet? */ + return -ENOKEY; + tfm = ci->ci_enc_key.tfm; + if (WARN_ON_ONCE(tfm == NULL)) /* Called on block-based filesystem? */ + return -ENOKEY; + if (WARN_ON_ONCE(len <= 0)) return -EINVAL; if (WARN_ON_ONCE(len % FSCRYPT_CONTENTS_ALIGNMENT != 0)) @@ -125,18 +123,22 @@ int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci, fscrypt_generate_iv(&iv, index, ci); - skcipher_request_set_callback( - req, CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP, - NULL, NULL); - sg_init_table(&dst, 1); - sg_set_page(&dst, dest_page, len, offs); - sg_init_table(&src, 1); - sg_set_page(&src, src_page, len, offs); - skcipher_request_set_crypt(req, &src, &dst, len, &iv); - if (rw == FS_DECRYPT) - err = crypto_skcipher_decrypt(req); - else - err = crypto_skcipher_encrypt(req); + { + SYNC_SKCIPHER_REQUEST_ON_STACK(req, tfm); + skcipher_request_set_callback(req, + CRYPTO_TFM_REQ_MAY_BACKLOG | + CRYPTO_TFM_REQ_MAY_SLEEP, + NULL, NULL); + sg_init_table(&dst, 1); + sg_set_page(&dst, dest_page, len, offs); + sg_init_table(&src, 1); + sg_set_page(&src, src_page, len, offs); + skcipher_request_set_crypt(req, &src, &dst, len, &iv); + if (rw == FS_DECRYPT) + err = crypto_skcipher_decrypt(req); + else + err = crypto_skcipher_encrypt(req); + } if (err) fscrypt_err(ci->ci_inode, "%scryption failed for data unit %llu: %d", @@ -160,7 +162,7 @@ int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci, * which the plaintext data was located in the source page. Any other parts of * the bounce page will be left uninitialized. * - * This is for use by the filesystem's ->writepages() method. + * This is for use by the ->writepages() method of non-block-based filesystems. * * The bounce page allocation is mempool-backed, so it will always succeed when * @gfp_flags includes __GFP_DIRECT_RECLAIM, e.g. when it's GFP_NOFS. However, @@ -174,14 +176,20 @@ struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio, { const struct inode *inode = folio->mapping->host; const struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode); - const unsigned int du_bits = ci->ci_data_unit_bits; - const unsigned int du_size = 1U << du_bits; + unsigned int du_bits; + unsigned int du_size; struct page *ciphertext_page; - u64 index = ((u64)folio->index << (PAGE_SHIFT - du_bits)) + - (offs >> du_bits); + u64 index; unsigned int i; int err; + if (WARN_ON_ONCE(ci == NULL)) /* File hasn't been opened yet? */ + return ERR_PTR(-ENOKEY); + + du_bits = ci->ci_data_unit_bits; + du_size = 1U << du_bits; + index = (folio_pos(folio) + offs) >> du_bits; + VM_BUG_ON_FOLIO(folio_test_large(folio), folio); if (WARN_ON_ONCE(!folio_test_locked(folio))) return ERR_PTR(-EINVAL); @@ -222,7 +230,8 @@ EXPORT_SYMBOL(fscrypt_encrypt_pagecache_blocks); * arbitrary page, not necessarily in the original pagecache page. The @inode * and @lblk_num must be specified, as they can't be determined from @page. * - * This is not compatible with fscrypt_operations::supports_subblock_data_units. + * This function only supports non-block-based filesystems that don't support + * sub-block data units (as indicated by the fscrypt_operations fields). * * Return: 0 on success; -errno on failure */ @@ -239,50 +248,6 @@ int fscrypt_encrypt_block_inplace(const struct inode *inode, struct page *page, EXPORT_SYMBOL(fscrypt_encrypt_block_inplace); /** - * fscrypt_decrypt_pagecache_blocks() - Decrypt data from a pagecache folio - * @folio: the pagecache folio containing the data to decrypt - * @len: size of the data to decrypt, in bytes - * @offs: offset within @folio of the data to decrypt, in bytes - * - * Decrypt data that has just been read from an encrypted file. The data must - * be located in a pagecache folio that is still locked and not yet uptodate. - * The length and offset of the data must be aligned to the file's crypto data - * unit size. Alignment to the filesystem block size fulfills this requirement, - * as the filesystem block size is always a multiple of the data unit size. - * - * Return: 0 on success; -errno on failure - */ -int fscrypt_decrypt_pagecache_blocks(struct folio *folio, size_t len, - size_t offs) -{ - const struct inode *inode = folio->mapping->host; - const struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode); - const unsigned int du_bits = ci->ci_data_unit_bits; - const unsigned int du_size = 1U << du_bits; - u64 index = ((u64)folio->index << (PAGE_SHIFT - du_bits)) + - (offs >> du_bits); - size_t i; - int err; - - if (WARN_ON_ONCE(!folio_test_locked(folio))) - return -EINVAL; - - if (WARN_ON_ONCE(len <= 0 || !IS_ALIGNED(len | offs, du_size))) - return -EINVAL; - - for (i = offs; i < offs + len; i += du_size, index++) { - struct page *page = folio_page(folio, i >> PAGE_SHIFT); - - err = fscrypt_crypt_data_unit(ci, FS_DECRYPT, index, page, - page, du_size, i & ~PAGE_MASK); - if (err) - return err; - } - return 0; -} -EXPORT_SYMBOL(fscrypt_decrypt_pagecache_blocks); - -/** * fscrypt_decrypt_block_inplace() - Decrypt a filesystem block in-place * @inode: The inode to which this block belongs * @page: The page containing the block to decrypt @@ -296,7 +261,8 @@ EXPORT_SYMBOL(fscrypt_decrypt_pagecache_blocks); * arbitrary page, not necessarily in the original pagecache page. The @inode * and @lblk_num must be specified, as they can't be determined from @page. * - * This is not compatible with fscrypt_operations::supports_subblock_data_units. + * This function only supports non-block-based filesystems that don't support + * sub-block data units (as indicated by the fscrypt_operations fields). * * Return: 0 on success; -errno on failure */ @@ -323,31 +289,26 @@ EXPORT_SYMBOL(fscrypt_decrypt_block_inplace); */ int fscrypt_initialize(struct super_block *sb) { - int err = 0; mempool_t *pool; /* pairs with smp_store_release() below */ - if (likely(smp_load_acquire(&fscrypt_bounce_page_pool))) + if (smp_load_acquire(&fscrypt_bounce_page_pool)) return 0; /* No need to allocate a bounce page pool if this FS won't use it. */ if (!sb->s_cop->needs_bounce_pages) return 0; - mutex_lock(&fscrypt_init_mutex); + guard(mutex)(&fscrypt_init_mutex); if (fscrypt_bounce_page_pool) - goto out_unlock; + return 0; - err = -ENOMEM; pool = mempool_create_page_pool(num_prealloc_crypto_pages, 0); if (!pool) - goto out_unlock; + return -ENOMEM; /* pairs with smp_load_acquire() above */ smp_store_release(&fscrypt_bounce_page_pool, pool); - err = 0; -out_unlock: - mutex_unlock(&fscrypt_init_mutex); - return err; + return 0; } void fscrypt_msg(const struct inode *inode, const char *level, @@ -374,45 +335,12 @@ void fscrypt_msg(const struct inode *inode, const char *level, va_end(args); } -/** - * fscrypt_init() - Set up for fs encryption. - * - * Return: 0 on success; -errno on failure - */ static int __init fscrypt_init(void) { - int err = -ENOMEM; - - /* - * Use an unbound workqueue to allow bios to be decrypted in parallel - * even when they happen to complete on the same CPU. This sacrifices - * locality, but it's worthwhile since decryption is CPU-intensive. - * - * Also use a high-priority workqueue to prioritize decryption work, - * which blocks reads from completing, over regular application tasks. - */ - fscrypt_read_workqueue = alloc_workqueue("fscrypt_read_queue", - WQ_UNBOUND | WQ_HIGHPRI, - num_online_cpus()); - if (!fscrypt_read_workqueue) - goto fail; - fscrypt_inode_info_cachep = KMEM_CACHE(fscrypt_inode_info, - SLAB_RECLAIM_ACCOUNT); - if (!fscrypt_inode_info_cachep) - goto fail_free_queue; - - err = fscrypt_init_keyring(); - if (err) - goto fail_free_inode_info; - + SLAB_RECLAIM_ACCOUNT | + SLAB_PANIC); + fscrypt_init_keyring(); return 0; - -fail_free_inode_info: - kmem_cache_destroy(fscrypt_inode_info_cachep); -fail_free_queue: - destroy_workqueue(fscrypt_read_workqueue); -fail: - return err; } late_initcall(fscrypt_init) diff --git a/fs/crypto/fscrypt_private.h b/fs/crypto/fscrypt_private.h index 0053b5c45412..74329e0953d1 100644 --- a/fs/crypto/fscrypt_private.h +++ b/fs/crypto/fscrypt_private.h @@ -66,9 +66,6 @@ #define FSCRYPT_CONTEXT_V1 1 #define FSCRYPT_CONTEXT_V2 2 -/* Keep this in sync with include/uapi/linux/fscrypt.h */ -#define FSCRYPT_MODE_MAX FSCRYPT_MODE_AES_256_HCTR2 - struct fscrypt_context_v1 { u8 version; /* FSCRYPT_CONTEXT_V1 */ u8 contents_encryption_mode; @@ -269,14 +266,6 @@ struct fscrypt_inode_info { /* True if ci_enc_key should be freed when this struct is freed */ u8 ci_owns_key : 1; -#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT - /* - * True if this inode will use inline encryption (blk-crypto) instead of - * the traditional filesystem-layer encryption. - */ - u8 ci_inlinecrypt : 1; -#endif - /* True if ci_dirhash_key is initialized */ u8 ci_dirhash_key_initialized : 1; @@ -340,11 +329,6 @@ typedef enum { /* crypto.c */ extern struct kmem_cache *fscrypt_inode_info_cachep; int fscrypt_initialize(struct super_block *sb); -int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci, - fscrypt_direction_t rw, u64 index, - struct page *src_page, struct page *dest_page, - unsigned int len, unsigned int offs); -struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags); void __printf(3, 4) __cold fscrypt_msg(const struct inode *inode, const char *level, const char *fmt, ...); @@ -411,15 +395,14 @@ void fscrypt_hkdf_expand(const struct hmac_sha512_key *hkdf, u8 context, const u8 *info, unsigned int infolen, u8 *okm, unsigned int okmlen); -/* inline_crypt.c */ +/* block.c */ #ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT -int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci, - bool is_hw_wrapped_key); - static inline bool fscrypt_using_inline_encryption(const struct fscrypt_inode_info *ci) { - return ci->ci_inlinecrypt; + const struct inode *inode = ci->ci_inode; + + return S_ISREG(inode->i_mode) && inode->i_sb->s_cop->is_block_based; } int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, @@ -449,12 +432,6 @@ fscrypt_is_key_prepared(const struct fscrypt_prepared_key *prep_key, #else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */ -static inline int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci, - bool is_hw_wrapped_key) -{ - return 0; -} - static inline bool fscrypt_using_inline_encryption(const struct fscrypt_inode_info *ci) { @@ -718,7 +695,7 @@ int fscrypt_add_test_dummy_key(struct super_block *sb, int fscrypt_verify_key_added(struct super_block *sb, const u8 identifier[FSCRYPT_KEY_IDENTIFIER_SIZE]); -int __init fscrypt_init_keyring(void); +void __init fscrypt_init_keyring(void); /* keysetup.c */ diff --git a/fs/crypto/keyring.c b/fs/crypto/keyring.c index 38b73e703073..76e28d1e0064 100644 --- a/fs/crypto/keyring.c +++ b/fs/crypto/keyring.c @@ -497,7 +497,7 @@ static int do_add_master_key(struct super_block *sb, struct fscrypt_master_key *mk; int err; - mutex_lock(&fscrypt_add_key_mutex); /* serialize find + link */ + guard(mutex)(&fscrypt_add_key_mutex); /* serialize find + link */ mk = fscrypt_find_master_key(sb, mk_spec); if (!mk) { @@ -524,7 +524,6 @@ static int do_add_master_key(struct super_block *sb, } fscrypt_put_master_key(mk); } - mutex_unlock(&fscrypt_add_key_mutex); return err; } @@ -1221,21 +1220,19 @@ out: } EXPORT_SYMBOL_GPL(fscrypt_ioctl_get_key_status); -int __init fscrypt_init_keyring(void) +void __init fscrypt_init_keyring(void) { int err; + /* + * Note that register_key_type() fails only if a key type with the same + * name already exists, which should never happen here. + */ err = register_key_type(&key_type_fscrypt_user); if (err) - return err; - + panic("failed to register .fscrypt key type (%d)", err); err = register_key_type(&key_type_fscrypt_provisioning); if (err) - goto err_unregister_fscrypt_user; - - return 0; - -err_unregister_fscrypt_user: - unregister_key_type(&key_type_fscrypt_user); - return err; + panic("failed to register fscrypt-provisioning key type (%d)", + err); } diff --git a/fs/crypto/keysetup.c b/fs/crypto/keysetup.c index f905f9f94bdd..892044ebcaca 100644 --- a/fs/crypto/keysetup.c +++ b/fs/crypto/keysetup.c @@ -83,8 +83,6 @@ static struct fscrypt_mode * select_encryption_mode(const union fscrypt_policy *policy, const struct inode *inode) { - BUILD_BUG_ON(ARRAY_SIZE(fscrypt_modes) != FSCRYPT_MODE_MAX + 1); - if (S_ISREG(inode->i_mode)) return &fscrypt_modes[fscrypt_policy_contents_mode(policy)]; @@ -146,9 +144,9 @@ err_free_tfm: /* * Prepare the crypto transform object or blk-crypto key in @prep_key, given the - * raw key, encryption mode (@ci->ci_mode), flag indicating which encryption - * implementation (fs-layer or blk-crypto) will be used (@ci->ci_inlinecrypt), - * and IV generation method (@ci->ci_policy.flags). + * raw key, encryption mode (@ci->ci_mode), predicate indicating which style of + * key is needed (fscrypt_using_inline_encryption(ci)), IV generation method + * (@ci->ci_policy.flags), and data unit size (@ci->ci_data_unit_bits). */ int fscrypt_prepare_key(struct fscrypt_prepared_key *prep_key, const u8 *raw_key, const struct fscrypt_inode_info *ci) @@ -226,26 +224,8 @@ static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci, u8 raw_mode_key[FSCRYPT_MAX_RAW_KEY_SIZE]; u8 hkdf_info[sizeof(mode_num) + sizeof(sb->s_uuid)]; unsigned int hkdf_infolen = 0; - bool use_hw_wrapped_key = false; int err; - if (WARN_ON_ONCE(mode_num > FSCRYPT_MODE_MAX)) - return -EINVAL; - - if (mk->mk_secret.is_hw_wrapped && S_ISREG(inode->i_mode)) { - /* Using a hardware-wrapped key for file contents encryption */ - if (!fscrypt_using_inline_encryption(ci)) { - if (sb->s_flags & SB_INLINECRYPT) - fscrypt_warn(ci->ci_inode, - "Hardware-wrapped key required, but no suitable inline encryption capabilities are available"); - else - fscrypt_warn(ci->ci_inode, - "Hardware-wrapped keys require inline encryption (-o inlinecrypt)"); - return -EINVAL; - } - use_hw_wrapped_key = true; - } - prep_key = fscrypt_find_mode_key(mk, hkdf_context, mode_num, ci); if (prep_key) { ci->ci_enc_key = *prep_key; @@ -268,7 +248,7 @@ static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci, new_node->data_unit_bits = ci->ci_data_unit_bits; prep_key = &new_node->key; - if (use_hw_wrapped_key) { + if (mk->mk_secret.is_hw_wrapped && S_ISREG(inode->i_mode)) { err = fscrypt_prepare_inline_crypt_key(prep_key, mk->mk_secret.bytes, mk->mk_secret.size, true, @@ -287,7 +267,7 @@ static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci, hkdf_info, hkdf_infolen, raw_mode_key, mode->keysize); err = fscrypt_prepare_key(prep_key, raw_mode_key, ci); - memzero_explicit(raw_mode_key, mode->keysize); + memzero_explicit(raw_mode_key, sizeof(raw_mode_key)); } if (err) { kfree(new_node); @@ -349,18 +329,17 @@ static int fscrypt_setup_iv_ino_lblk_32_key(struct fscrypt_inode_info *ci, /* pairs with smp_store_release() below */ if (!smp_load_acquire(&mk->mk_ino_hash_key_initialized)) { - - mutex_lock(&fscrypt_mode_key_setup_mutex); - - if (mk->mk_ino_hash_key_initialized) - goto unlock; - - fscrypt_derive_siphash_key(mk, HKDF_CONTEXT_INODE_HASH_KEY, - NULL, 0, &mk->mk_ino_hash_key); - /* pairs with smp_load_acquire() above */ - smp_store_release(&mk->mk_ino_hash_key_initialized, true); -unlock: - mutex_unlock(&fscrypt_mode_key_setup_mutex); + guard(mutex)(&fscrypt_mode_key_setup_mutex); + + if (!mk->mk_ino_hash_key_initialized) { + fscrypt_derive_siphash_key(mk, + HKDF_CONTEXT_INODE_HASH_KEY, + NULL, 0, + &mk->mk_ino_hash_key); + /* pairs with smp_load_acquire() above */ + smp_store_release(&mk->mk_ino_hash_key_initialized, + true); + } } /* @@ -418,7 +397,7 @@ static int fscrypt_setup_v2_file_key(struct fscrypt_inode_info *ci, ci->ci_nonce, FSCRYPT_FILE_NONCE_SIZE, derived_key, ci->ci_mode->keysize); err = fscrypt_set_per_file_enc_key(ci, derived_key); - memzero_explicit(derived_key, ci->ci_mode->keysize); + memzero_explicit(derived_key, sizeof(derived_key)); } if (err) return err; @@ -515,10 +494,6 @@ static int setup_file_encryption_key(struct fscrypt_inode_info *ci, if (ci->ci_policy.version != FSCRYPT_POLICY_V1) return -ENOKEY; - err = fscrypt_select_encryption_impl(ci, false); - if (err) - return err; - /* * As a legacy fallback for v1 policies, search for the key in * the current task's subscribed keyrings too. Don't move this @@ -540,10 +515,6 @@ static int setup_file_encryption_key(struct fscrypt_inode_info *ci, goto out_release_key; } - err = fscrypt_select_encryption_impl(ci, mk->mk_secret.is_hw_wrapped); - if (err) - goto out_release_key; - switch (ci->ci_policy.version) { case FSCRYPT_POLICY_V1: if (WARN_ON_ONCE(mk->mk_secret.is_hw_wrapped)) { diff --git a/fs/crypto/keysetup_v1.c b/fs/crypto/keysetup_v1.c index 7e3a58dc4b56..87fe13ccb253 100644 --- a/fs/crypto/keysetup_v1.c +++ b/fs/crypto/keysetup_v1.c @@ -251,7 +251,7 @@ static int setup_v1_file_key_derived(struct fscrypt_inode_info *ci, err = fscrypt_set_per_file_enc_key(ci, derived_key); - memzero_explicit(derived_key, derived_keysize); + memzero_explicit(derived_key, sizeof(derived_key)); /* No need to zeroize 'aes', as its key is not secret. */ return err; } diff --git a/fs/crypto/policy.c b/fs/crypto/policy.c index c80b24a941ad..6dd510f93e6d 100644 --- a/fs/crypto/policy.c +++ b/fs/crypto/policy.c @@ -177,6 +177,23 @@ static bool supported_iv_ino_lblk_policy(const struct fscrypt_policy_v2 *policy, type, sb->s_id); return false; } + + /* + * IV_INO_LBLK_32 isn't compatible with inline encryption when + * s_blocksize != PAGE_SIZE. In that case the DUN can wrap around in + * the middle of a page, but sometimes fscrypt_mergeable_bio() is called + * only for the first block per page. Since IV_INO_LBLK_32 exists only + * to support inline encryption hardware that is limited to 32-bit DUNs, + * just disallow IV_INO_LBLK_32 with s_blocksize != PAGE_SIZE entirely. + */ + if ((policy->flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) && + sb->s_blocksize != PAGE_SIZE) { + fscrypt_warn(inode, + "Can't use %s policy on filesystem '%s' with block size != PAGE_SIZE", + type, sb->s_id); + return false; + } + return true; } @@ -507,7 +524,6 @@ int fscrypt_ioctl_set_policy(struct file *filp, const void __user *arg) union fscrypt_policy policy; union fscrypt_policy existing_policy; struct inode *inode = file_inode(filp); - u8 version; int size; int ret; @@ -518,21 +534,9 @@ int fscrypt_ioctl_set_policy(struct file *filp, const void __user *arg) if (size <= 0) return -EINVAL; - /* - * We should just copy the remaining 'size - 1' bytes here, but a - * bizarre bug in gcc 7 and earlier (fixed by gcc r255731) causes gcc to - * think that size can be 0 here (despite the check above!) *and* that - * it's a compile-time constant. Thus it would think copy_from_user() - * is passed compile-time constant ULONG_MAX, causing the compile-time - * buffer overflow check to fail, breaking the build. This only occurred - * when building an i386 kernel with -Os and branch profiling enabled. - * - * Work around it by just copying the first byte again... - */ - version = policy.version; - if (copy_from_user(&policy, arg, size)) + if (copy_from_user((u8 *)&policy + 1, (const u8 __user *)arg + 1, + size - 1)) return -EFAULT; - policy.version = version; if (!inode_owner_or_capable(file_mnt_idmap(filp), inode)) return -EACCES; diff --git a/fs/ext4/crypto.c b/fs/ext4/crypto.c index f41f320f4437..9265cfe62c83 100644 --- a/fs/ext4/crypto.c +++ b/fs/ext4/crypto.c @@ -236,7 +236,7 @@ static bool ext4_has_stable_inodes(struct super_block *sb) const struct fscrypt_operations ext4_cryptops = { .inode_info_offs = (int)offsetof(struct ext4_inode_info, i_crypt_info) - (int)offsetof(struct ext4_inode_info, vfs_inode), - .needs_bounce_pages = 1, + .is_block_based = 1, .has_32bit_inodes = 1, .supports_subblock_data_units = 1, .legacy_key_prefix = "ext4:", diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index f1866532782a..2fdff50526e9 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -3841,8 +3841,8 @@ static inline void ext4_set_de_type(struct super_block *sb, /* readpages.c */ int ext4_read_folio(struct file *file, struct folio *folio); void ext4_readahead(struct readahead_control *rac); -extern int __init ext4_init_post_read_processing(void); -extern void ext4_exit_post_read_processing(void); +int __init ext4_init_verity_caches(void); +void ext4_exit_verity_caches(void); /* symlink.c */ extern const struct inode_operations ext4_encrypted_symlink_inode_operations; @@ -3957,7 +3957,7 @@ extern void ext4_io_submit_init(struct ext4_io_submit *io, struct writeback_control *wbc); extern void ext4_end_io_rsv_work(struct work_struct *work); extern void ext4_io_submit(struct ext4_io_submit *io); -int ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *page, +void ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *page, size_t len); extern struct ext4_io_end_vec *ext4_alloc_io_end_vec(ext4_io_end_t *io_end); extern struct ext4_io_end_vec *ext4_last_io_end_vec(ext4_io_end_t *io_end); diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 2ad221a50630..3fc235eb65f2 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1264,17 +1264,6 @@ int ext4_block_write_begin(handle_t *handle, struct folio *folio, from, to); else folio_zero_new_buffers(folio, from, to); - } else if (fscrypt_inode_uses_fs_layer_crypto(inode)) { - for (i = 0; i < nr_wait; i++) { - int err2; - - err2 = fscrypt_decrypt_pagecache_blocks(folio, - blocksize, bh_offset(wait[i])); - if (err2) { - clear_buffer_uptodate(wait[i]); - err = err2; - } - } } return err; @@ -2077,11 +2066,10 @@ static void mpage_folio_done(struct mpage_da_data *mpd, struct folio *folio) folio_unlock(folio); } -static int mpage_submit_folio(struct mpage_da_data *mpd, struct folio *folio) +static void mpage_submit_folio(struct mpage_da_data *mpd, struct folio *folio) { size_t len; loff_t size; - int err; WARN_ON_ONCE(folio_pos(folio) != mpd->start_pos); folio_clear_dirty_for_io(folio); @@ -2103,9 +2091,7 @@ static int mpage_submit_folio(struct mpage_da_data *mpd, struct folio *folio) if (folio_pos(folio) + len > size && !ext4_verity_in_progress(mpd->inode)) len = size & (len - 1); - err = ext4_bio_write_folio(&mpd->io_submit, folio, len); - - return err; + ext4_bio_write_folio(&mpd->io_submit, folio, len); } #define BH_FLAGS (BIT(BH_Unwritten) | BIT(BH_Delay)) @@ -2182,8 +2168,7 @@ static bool mpage_add_bh_to_extent(struct mpage_da_data *mpd, ext4_lblk_t lblk, * accumulated extent of buffers to map or add buffers in the page to the * extent of buffers to map. The function returns 1 if the caller can continue * by processing the next page, 0 if it should stop adding buffers to the - * extent to map because we cannot extend it anymore. It can also return value - * < 0 in case of error during IO submission. + * extent to map because we cannot extend it anymore. */ static int mpage_process_page_bufs(struct mpage_da_data *mpd, struct buffer_head *head, @@ -2191,7 +2176,6 @@ static int mpage_process_page_bufs(struct mpage_da_data *mpd, ext4_lblk_t lblk) { struct inode *inode = mpd->inode; - int err; ext4_lblk_t blocks = (i_size_read(inode) + i_blocksize(inode) - 1) >> inode->i_blkbits; @@ -2214,9 +2198,7 @@ static int mpage_process_page_bufs(struct mpage_da_data *mpd, } while (lblk++, (bh = bh->b_this_page) != head); /* So far everything mapped? Submit the page for IO. */ if (mpd->map.m_len == 0) { - err = mpage_submit_folio(mpd, head->b_folio); - if (err < 0) - return err; + mpage_submit_folio(mpd, head->b_folio); mpage_folio_done(mpd, head->b_folio); } if (lblk >= blocks) { @@ -2346,9 +2328,7 @@ static int mpage_map_and_submit_buffers(struct mpage_da_data *mpd) if (err < 0 || map_bh) goto out; /* Page fully mapped - let IO run! */ - err = mpage_submit_folio(mpd, folio); - if (err < 0) - goto out; + mpage_submit_folio(mpd, folio); mpage_folio_done(mpd, folio); } folio_batch_release(&fbatch); @@ -2421,7 +2401,6 @@ static int mpage_submit_partial_folio(struct mpage_da_data *mpd) struct inode *inode = mpd->inode; struct folio *folio; loff_t pos; - int ret; folio = filemap_get_folio(inode->i_mapping, mpd->start_pos >> PAGE_SHIFT); @@ -2436,9 +2415,7 @@ static int mpage_submit_partial_folio(struct mpage_da_data *mpd) !folio_contains(folio, pos >> PAGE_SHIFT))) return -EINVAL; - ret = mpage_submit_folio(mpd, folio); - if (ret) - goto out; + mpage_submit_folio(mpd, folio); /* * Update start_pos to prevent this folio from being released in * mpage_release_unused_pages(), it will be reset to the aligned folio @@ -2447,10 +2424,9 @@ static int mpage_submit_partial_folio(struct mpage_da_data *mpd) * entire folio has finished processing. */ mpd->start_pos = pos; -out: folio_unlock(folio); folio_put(folio); - return ret; + return 0; } /* @@ -2737,9 +2713,8 @@ static int mpage_prepare_extent_to_map(struct mpage_da_data *mpd) * through a pin. */ if (!mpd->can_map) { - err = mpage_submit_folio(mpd, folio); - if (err < 0) - goto out; + mpage_submit_folio(mpd, folio); + err = 0; /* Pending dirtying of journalled data? */ if (folio_test_checked(folio)) { err = mpage_journal_page_buffers(handle, @@ -3833,9 +3808,9 @@ int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, return ret; out: /* - * When inline encryption is enabled, sometimes I/O to an encrypted file - * has to be broken up to guarantee DUN contiguity. Handle this by - * limiting the length of the mapping returned. + * Sometimes I/O to an encrypted file has to be broken up to guarantee + * DUN contiguity. Handle this by limiting the length of the mapping + * returned. */ map.m_len = fscrypt_limit_io_blocks(inode, map.m_lblk, map.m_len); @@ -4089,17 +4064,6 @@ static struct buffer_head *ext4_load_tail_bh(struct inode *inode, loff_t from) err = ext4_read_bh_lock(bh, 0, true); if (err) goto unlock; - if (fscrypt_inode_uses_fs_layer_crypto(inode)) { - /* We expect the key to be set. */ - BUG_ON(!fscrypt_has_encryption_key(inode)); - err = fscrypt_decrypt_pagecache_blocks(folio, - blocksize, - bh_offset(bh)); - if (err) { - clear_buffer_uptodate(bh); - goto unlock; - } - } } return bh; @@ -6207,11 +6171,8 @@ u32 ext4_dio_alignment(struct inode *inode) return 0; if (ext4_has_inline_data(inode)) return 0; - if (IS_ENCRYPTED(inode)) { - if (!fscrypt_dio_supported(inode)) - return 0; + if (IS_ENCRYPTED(inode)) return i_blocksize(inode); - } return 1; /* use the iomap defaults */ } @@ -6230,11 +6191,7 @@ int ext4_getattr(struct mnt_idmap *idmap, const struct path *path, stat->btime.tv_nsec = ei->i_crtime.tv_nsec; } - /* - * Return the DIO alignment restrictions if requested. We only return - * this information when requested, since on encrypted files it might - * take a fair bit of work to get if the file wasn't opened recently. - */ + /* Return the DIO alignment restrictions if requested. */ if ((request_mask & STATX_DIOALIGN) && S_ISREG(inode->i_mode)) { u32 dio_align = ext4_dio_alignment(inode); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index bc674aa4a656..0236b6b9785a 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -103,18 +103,12 @@ static void ext4_finish_bio(struct bio *bio) bio_for_each_folio_all(fi, bio) { struct folio *folio = fi.folio; - struct folio *io_folio = NULL; struct buffer_head *bh, *head; size_t bio_start = fi.offset; size_t bio_end = bio_start + fi.length; unsigned under_io = 0; unsigned long flags; - if (fscrypt_is_bounce_folio(folio)) { - io_folio = folio; - folio = fscrypt_pagecache_folio(folio); - } - if (bio->bi_status) { int err = blk_status_to_errno(bio->bi_status); mapping_set_error(folio->mapping, err); @@ -139,10 +133,8 @@ static void ext4_finish_bio(struct bio *bio) } } while ((bh = bh->b_this_page) != head); spin_unlock_irqrestore(&head->b_uptodate_lock, flags); - if (!under_io) { - fscrypt_free_bounce_page(&io_folio->page); + if (!under_io) folio_end_writeback(folio); - } } } @@ -453,7 +445,6 @@ static bool io_submit_need_new_bio(struct ext4_io_submit *io, static void io_submit_add_bh(struct ext4_io_submit *io, struct inode *inode, struct folio *folio, - struct folio *io_folio, struct buffer_head *bh) { if (io->io_bio && io_submit_need_new_bio(io, inode, folio, bh)) { @@ -462,20 +453,18 @@ submit_and_retry: } if (io->io_bio == NULL) io_submit_init_bio(io, inode, folio, bh); - if (!bio_add_folio(io->io_bio, io_folio, bh->b_size, bh_offset(bh))) + if (!bio_add_folio(io->io_bio, folio, bh->b_size, bh_offset(bh))) goto submit_and_retry; wbc_account_cgroup_owner(io->io_wbc, folio, bh->b_size); io->io_next_block++; } -int ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *folio, +void ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *folio, size_t len) { - struct folio *io_folio = folio; struct inode *inode = folio->mapping->host; unsigned block_start; struct buffer_head *bh, *head; - int ret = 0; int nr_to_submit = 0; struct writeback_control *wbc = io->io_wbc; bool keep_towrite = false; @@ -544,70 +533,17 @@ int ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *folio, */ __folio_start_writeback(folio, keep_towrite); folio_end_writeback(folio); - return 0; + return; } bh = head = folio_buffers(folio); - /* - * If any blocks are being written to an encrypted file, encrypt them - * into a bounce page. For simplicity, just encrypt until the last - * block which might be needed. This may cause some unneeded blocks - * (e.g. holes) to be unnecessarily encrypted, but this is rare and - * can't happen in the common case of blocksize == PAGE_SIZE. - */ - if (fscrypt_inode_uses_fs_layer_crypto(inode)) { - gfp_t gfp_flags = GFP_NOFS; - unsigned int enc_bytes = round_up(len, i_blocksize(inode)); - struct page *bounce_page; - - /* - * Since bounce page allocation uses a mempool, we can only use - * a waiting mask (i.e. request guaranteed allocation) on the - * first page of the bio. Otherwise it can deadlock. - */ - if (io->io_bio) - gfp_flags = GFP_NOWAIT; - retry_encrypt: - bounce_page = fscrypt_encrypt_pagecache_blocks(folio, - enc_bytes, 0, gfp_flags); - if (IS_ERR(bounce_page)) { - ret = PTR_ERR(bounce_page); - if (ret == -ENOMEM && - (io->io_bio || wbc->sync_mode == WB_SYNC_ALL)) { - gfp_t new_gfp_flags = GFP_NOFS; - if (io->io_bio) - ext4_io_submit(io); - else - new_gfp_flags |= __GFP_NOFAIL; - memalloc_retry_wait(gfp_flags); - gfp_flags = new_gfp_flags; - goto retry_encrypt; - } - - printk_ratelimited(KERN_ERR "%s: ret = %d\n", __func__, ret); - folio_redirty_for_writepage(wbc, folio); - do { - if (buffer_async_write(bh)) { - clear_buffer_async_write(bh); - set_buffer_dirty(bh); - } - bh = bh->b_this_page; - } while (bh != head); - - return ret; - } - io_folio = page_folio(bounce_page); - } - __folio_start_writeback(folio, keep_towrite); /* Now submit buffers to write */ do { if (!buffer_async_write(bh)) continue; - io_submit_add_bh(io, inode, folio, io_folio, bh); + io_submit_add_bh(io, inode, folio, bh); } while ((bh = bh->b_this_page) != head); - - return 0; } diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c index dd3627c71732..c7b6cdb2e124 100644 --- a/fs/ext4/readpage.c +++ b/fs/ext4/readpage.c @@ -47,25 +47,15 @@ #include "ext4.h" #include <trace/events/ext4.h> -#define NUM_PREALLOC_POST_READ_CTXS 128 +#define NUM_VERITY_WORKS 128 -static struct kmem_cache *bio_post_read_ctx_cache; -static mempool_t *bio_post_read_ctx_pool; +static struct kmem_cache *ext4_verity_work_cache; +static mempool_t *ext4_verity_work_pool; -/* postprocessing steps for read bios */ -enum bio_post_read_step { - STEP_INITIAL = 0, - STEP_DECRYPT, - STEP_VERITY, - STEP_MAX, -}; - -struct bio_post_read_ctx { +struct ext4_verity_work { struct bio *bio; struct fsverity_info *vi; struct work_struct work; - unsigned int cur_step; - unsigned int enabled_steps; }; static void __read_end_io(struct bio *bio) @@ -75,40 +65,22 @@ static void __read_end_io(struct bio *bio) bio_for_each_folio_all(fi, bio) folio_end_read(fi.folio, bio->bi_status == 0); if (bio->bi_private) - mempool_free(bio->bi_private, bio_post_read_ctx_pool); + mempool_free(bio->bi_private, ext4_verity_work_pool); bio_put(bio); } -static void bio_post_read_processing(struct bio_post_read_ctx *ctx); - -static void decrypt_work(struct work_struct *work) -{ - struct bio_post_read_ctx *ctx = - container_of(work, struct bio_post_read_ctx, work); - struct bio *bio = ctx->bio; - - if (fscrypt_decrypt_bio(bio)) - bio_post_read_processing(ctx); - else - __read_end_io(bio); -} - static void verity_work(struct work_struct *work) { - struct bio_post_read_ctx *ctx = - container_of(work, struct bio_post_read_ctx, work); + struct ext4_verity_work *ctx = + container_of(work, struct ext4_verity_work, work); struct bio *bio = ctx->bio; struct fsverity_info *vi = ctx->vi; /* - * fsverity_verify_bio() may call readahead() again, and although verity - * will be disabled for that, decryption may still be needed, causing - * another bio_post_read_ctx to be allocated. So to guarantee that - * mempool_alloc() never deadlocks we must free the current ctx first. - * This is safe because verity is the last post-read step. + * Free the ext4_verity_work right away, since it's no longer needed. + * This relieves the pressure on the mempool as much as possible. */ - BUILD_BUG_ON(STEP_VERITY + 1 != STEP_MAX); - mempool_free(ctx, bio_post_read_ctx_pool); + mempool_free(ctx, ext4_verity_work_pool); bio->bi_private = NULL; fsverity_verify_bio(vi, bio); @@ -116,41 +88,6 @@ static void verity_work(struct work_struct *work) __read_end_io(bio); } -static void bio_post_read_processing(struct bio_post_read_ctx *ctx) -{ - /* - * We use different work queues for decryption and for verity because - * verity may require reading metadata pages that need decryption, and - * we shouldn't recurse to the same workqueue. - */ - switch (++ctx->cur_step) { - case STEP_DECRYPT: - if (ctx->enabled_steps & (1 << STEP_DECRYPT)) { - INIT_WORK(&ctx->work, decrypt_work); - fscrypt_enqueue_decrypt_work(&ctx->work); - return; - } - ctx->cur_step++; - fallthrough; - case STEP_VERITY: - if (IS_ENABLED(CONFIG_FS_VERITY) && - ctx->enabled_steps & (1 << STEP_VERITY)) { - INIT_WORK(&ctx->work, verity_work); - fsverity_enqueue_verify_work(&ctx->work); - return; - } - ctx->cur_step++; - fallthrough; - default: - __read_end_io(ctx->bio); - } -} - -static bool bio_post_read_required(struct bio *bio) -{ - return bio->bi_private && !bio->bi_status; -} - /* * I/O completion handler for multipage BIOs. * @@ -165,36 +102,26 @@ static bool bio_post_read_required(struct bio *bio) */ static void mpage_end_io(struct bio *bio) { - if (bio_post_read_required(bio)) { - struct bio_post_read_ctx *ctx = bio->bi_private; + if (IS_ENABLED(CONFIG_FS_VERITY) && bio->bi_private && + !bio->bi_status) { + struct ext4_verity_work *ctx = bio->bi_private; - ctx->cur_step = STEP_INITIAL; - bio_post_read_processing(ctx); + INIT_WORK(&ctx->work, verity_work); + fsverity_enqueue_verify_work(&ctx->work); return; } __read_end_io(bio); } -static void ext4_set_bio_post_read_ctx(struct bio *bio, - const struct inode *inode, - struct fsverity_info *vi) +static void ext4_set_verity_work(struct bio *bio, struct fsverity_info *vi) { - unsigned int post_read_steps = 0; - - if (fscrypt_inode_uses_fs_layer_crypto(inode)) - post_read_steps |= 1 << STEP_DECRYPT; - - if (vi) - post_read_steps |= 1 << STEP_VERITY; - - if (post_read_steps) { + if (vi) { /* Due to the mempool, this never fails. */ - struct bio_post_read_ctx *ctx = - mempool_alloc(bio_post_read_ctx_pool, GFP_NOFS); + struct ext4_verity_work *ctx = + mempool_alloc(ext4_verity_work_pool, GFP_NOFS); ctx->bio = bio; ctx->vi = vi; - ctx->enabled_steps = post_read_steps; bio->bi_private = ctx; } } @@ -355,7 +282,7 @@ static int ext4_mpage_readpages(struct inode *inode, struct fsverity_info *vi, bio = bio_alloc(bdev, bio_max_segs(nr_pages), REQ_OP_READ, GFP_KERNEL); fscrypt_set_bio_crypt_ctx(bio, inode, pos, GFP_KERNEL); - ext4_set_bio_post_read_ctx(bio, inode, vi); + ext4_set_verity_work(bio, vi); bio->bi_iter.bi_sector = first_block << (blkbits - 9); bio->bi_end_io = mpage_end_io; if (rac) @@ -429,27 +356,31 @@ void ext4_readahead(struct readahead_control *rac) ext4_mpage_readpages(inode, vi, rac, NULL); } -int __init ext4_init_post_read_processing(void) +int __init ext4_init_verity_caches(void) { - bio_post_read_ctx_cache = KMEM_CACHE(bio_post_read_ctx, SLAB_RECLAIM_ACCOUNT); + if (!IS_ENABLED(CONFIG_FS_VERITY)) + return 0; + ext4_verity_work_cache = + KMEM_CACHE(ext4_verity_work, SLAB_RECLAIM_ACCOUNT); - if (!bio_post_read_ctx_cache) + if (!ext4_verity_work_cache) goto fail; - bio_post_read_ctx_pool = - mempool_create_slab_pool(NUM_PREALLOC_POST_READ_CTXS, - bio_post_read_ctx_cache); - if (!bio_post_read_ctx_pool) + ext4_verity_work_pool = mempool_create_slab_pool( + NUM_VERITY_WORKS, ext4_verity_work_cache); + if (!ext4_verity_work_pool) goto fail_free_cache; return 0; fail_free_cache: - kmem_cache_destroy(bio_post_read_ctx_cache); + kmem_cache_destroy(ext4_verity_work_cache); fail: return -ENOMEM; } -void ext4_exit_post_read_processing(void) +void ext4_exit_verity_caches(void) { - mempool_destroy(bio_post_read_ctx_pool); - kmem_cache_destroy(bio_post_read_ctx_cache); + if (!IS_ENABLED(CONFIG_FS_VERITY)) + return; + mempool_destroy(ext4_verity_work_pool); + kmem_cache_destroy(ext4_verity_work_cache); } diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 2877a6cf6d09..4b6112e5d6c5 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -7539,7 +7539,7 @@ static int __init ext4_init_fs(void) if (err) goto out7; - err = ext4_init_post_read_processing(); + err = ext4_init_verity_caches(); if (err) goto out6; @@ -7588,7 +7588,7 @@ out3: out4: ext4_exit_pageio(); out5: - ext4_exit_post_read_processing(); + ext4_exit_verity_caches(); out6: ext4_exit_pending(); out7: @@ -7609,7 +7609,7 @@ static void __exit ext4_exit_fs(void) ext4_exit_sysfs(); ext4_exit_system_zone(); ext4_exit_pageio(); - ext4_exit_post_read_processing(); + ext4_exit_verity_caches(); ext4_exit_es(); ext4_exit_pending(); } diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c index 91855d91bbdd..ce88092d9ce2 100644 --- a/fs/f2fs/compress.c +++ b/fs/f2fs/compress.c @@ -1286,8 +1286,6 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc, .compressed_page = NULL, .io_type = io_type, .io_wbc = wbc, - .encrypted = fscrypt_inode_uses_fs_layer_crypto(cc->inode) ? - 1 : 0, }; struct folio *folio; struct dnode_of_data dn; @@ -1361,14 +1359,6 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc, /* wait for GCed page writeback via META_MAPPING */ f2fs_wait_on_block_writeback(inode, fio.old_blkaddr); - - if (fio.encrypted) { - fio.page = cc->rpages[i + 1]; - err = f2fs_encrypt_one_page(&fio); - if (err) - goto out_destroy_crypt; - cc->cpages[i] = fio.encrypted_page; - } } set_cluster_writeback(cc); @@ -1406,21 +1396,15 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc, f2fs_bug_on(fio.sbi, blkaddr == NULL_ADDR); - if (fio.encrypted) - fio.encrypted_page = cc->cpages[i - 1]; - else - fio.compressed_page = cc->cpages[i - 1]; + fio.compressed_page = cc->cpages[i - 1]; cc->cpages[i - 1] = NULL; fio.submitted = 0; f2fs_outplace_write_data(&dn, &fio); if (unlikely(!fio.submitted)) { cancel_cluster_writeback(cc, cic, i); - - /* To call fscrypt_finalize_bounce_page */ - i = cc->valid_nr_cpages; *submitted = 0; - goto out_destroy_crypt; + goto out_free_page_array; } (*submitted)++; unlock_continue: @@ -1452,17 +1436,8 @@ unlock_continue: f2fs_destroy_compress_ctx(cc, false); return 0; -out_destroy_crypt: +out_free_page_array: page_array_free(sbi, cic->rpages, cc->cluster_size); - - if (!fio.encrypted) - goto out_put_cic; - - for (--i; i >= 0; i--) { - if (!cc->cpages[i]) - continue; - fscrypt_finalize_bounce_page(&cc->cpages[i]); - } out_put_cic: kmem_cache_free(cic_entry_slab, cic); out_put_dnode: diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 8977ad379f50..dad5c2bd5888 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -65,9 +65,6 @@ bool f2fs_is_cp_guaranteed(const struct folio *folio) struct inode *inode; struct f2fs_sb_info *sbi; - if (fscrypt_is_bounce_folio(folio)) - return folio_test_f2fs_gcing(fscrypt_pagecache_folio(folio)); - inode = mapping->host; sbi = F2FS_I_SB(inode); @@ -101,11 +98,6 @@ static enum count_type __read_io_type(struct folio *folio) /* postprocessing steps for read bios */ enum bio_post_read_step { -#ifdef CONFIG_FS_ENCRYPTION - STEP_DECRYPT = BIT(0), -#else - STEP_DECRYPT = 0, /* compile out the decryption-related code */ -#endif #ifdef CONFIG_F2FS_FS_COMPRESSION STEP_DECOMPRESS = BIT(1), #else @@ -301,11 +293,6 @@ static void f2fs_post_read_work(struct work_struct *work) container_of(work, struct bio_post_read_ctx, work); struct bio *bio = ctx->bio; - if ((ctx->enabled_steps & STEP_DECRYPT) && !fscrypt_decrypt_bio(bio)) { - f2fs_finish_read_bio(bio, true); - return; - } - if (ctx->enabled_steps & STEP_DECOMPRESS) f2fs_handle_step_decompress(ctx, true); @@ -329,18 +316,11 @@ static void f2fs_read_end_io(struct bio *bio) return; } - if (ctx) { - unsigned int enabled_steps = ctx->enabled_steps & - (STEP_DECRYPT | STEP_DECOMPRESS); - - /* - * If we have only decompression step between decompression and - * decrypt, we don't need post processing for this. - */ - if (enabled_steps == STEP_DECOMPRESS && - !f2fs_low_mem_mode(sbi)) { + if (ctx && (ctx->enabled_steps & STEP_DECOMPRESS)) { + if (!f2fs_low_mem_mode(sbi)) { + /* Decompress inline. */ f2fs_handle_step_decompress(ctx, intask); - } else if (enabled_steps) { + } else { INIT_WORK(&ctx->work, f2fs_post_read_work); queue_work(ctx->sbi->wq, &ctx->work); return; @@ -362,13 +342,6 @@ static void f2fs_write_end_bio(struct bio *bio) struct folio *folio = fi.folio; enum count_type type; - if (fscrypt_is_bounce_folio(folio)) { - struct folio *io_folio = folio; - - folio = fscrypt_pagecache_folio(io_folio); - fscrypt_free_bounce_page(&io_folio->page); - } - #ifdef CONFIG_F2FS_FS_COMPRESSION if (f2fs_is_compressed_page(folio)) { f2fs_compress_write_end_io(bio, folio); @@ -614,11 +587,6 @@ static bool __has_merged_page(struct bio *bio, struct inode *inode, bio_for_each_folio_all(fi, bio) { struct folio *target = fi.folio; - if (fscrypt_is_bounce_folio(target)) { - target = fscrypt_pagecache_folio(target); - if (IS_ERR(target)) - continue; - } if (f2fs_is_compressed_page(target)) { target = f2fs_compress_control_folio(target); if (IS_ERR(target)) @@ -1161,9 +1129,6 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode, f2fs_set_bio_crypt_ctx(bio, inode, first_idx, NULL, GFP_NOFS); bio->bi_end_io = f2fs_read_end_io; - if (fscrypt_inode_uses_fs_layer_crypto(inode)) - post_read_steps |= STEP_DECRYPT; - if (vi) post_read_steps |= STEP_VERITY; @@ -2852,35 +2817,6 @@ static void f2fs_readahead(struct readahead_control *rac) f2fs_mpage_readpages(inode, vi, rac, NULL); } -int f2fs_encrypt_one_page(struct f2fs_io_info *fio) -{ - struct inode *inode = fio_inode(fio); - struct folio *mfolio; - struct page *page; - - if (!f2fs_encrypted_file(inode)) - return 0; - - page = fio->compressed_page ? fio->compressed_page : fio->page; - - if (fscrypt_inode_uses_inline_crypto(inode)) - return 0; - - fio->encrypted_page = fscrypt_encrypt_pagecache_blocks(page_folio(page), - PAGE_SIZE, 0, GFP_NOFS); - if (IS_ERR(fio->encrypted_page)) - return PTR_ERR(fio->encrypted_page); - - mfolio = filemap_lock_folio(META_MAPPING(fio->sbi), fio->old_blkaddr); - if (!IS_ERR(mfolio)) { - if (folio_test_uptodate(mfolio)) - memcpy(folio_address(mfolio), - page_address(fio->encrypted_page), PAGE_SIZE); - f2fs_folio_put(mfolio, true); - } - return 0; -} - static inline bool check_inplace_update_policy(struct inode *inode, struct f2fs_io_info *fio) { @@ -3053,22 +2989,15 @@ got_it: if (ipu_force || (__is_valid_data_blkaddr(fio->old_blkaddr) && need_inplace_update(fio))) { - err = f2fs_encrypt_one_page(fio); - if (err) - goto out_writepage; - folio_start_writeback(folio); f2fs_put_dnode(&dn); if (fio->need_lock == LOCK_REQ) f2fs_unlock_op(fio->sbi, &lc); err = f2fs_inplace_write_data(fio); - if (err) { - if (fscrypt_inode_uses_fs_layer_crypto(inode)) - fscrypt_finalize_bounce_page(&fio->encrypted_page); + if (err) folio_end_writeback(folio); - } else { + else set_inode_flag(inode, FI_UPDATE_WRITE); - } trace_f2fs_do_write_data_page(folio, IPU); return err; } @@ -3087,10 +3016,6 @@ got_it: fio->version = ni.version; - err = f2fs_encrypt_one_page(fio); - if (err) - goto out_writepage; - folio_start_writeback(folio); if (fio->compr_blocks && fio->old_blkaddr == COMPRESS_ADDR) @@ -4031,7 +3956,7 @@ repeat: /* * Although the block may be stored in the COW inode, the folio * belongs to @inode and its data was encrypted (or not) using - * @inode's context (see f2fs_encrypt_one_page()). Read with + * @inode's context (see f2fs_set_bio_crypt_ctx()). Read with * @inode so the post-read decryption decision matches the * folio's owner; otherwise an unencrypted @inode whose COW inode * is encrypted hits a NULL ->i_crypt_info on decryption. @@ -4602,9 +4527,9 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, iomap->offset = F2FS_BLK_TO_BYTES(map.m_lblk); /* - * When inline encryption is enabled, sometimes I/O to an encrypted file - * has to be broken up to guarantee DUN contiguity. Handle this by - * limiting the length of the mapping returned. + * Sometimes I/O to an encrypted file has to be broken up to guarantee + * DUN contiguity. Handle this by limiting the length of the mapping + * returned. */ map.m_len = fscrypt_limit_io_blocks(inode, map.m_lblk, map.m_len); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 8f3e632f315c..8376bbe58ee3 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -1364,7 +1364,6 @@ struct f2fs_io_info { unsigned int submitted:1; /* indicate IO submission */ unsigned int in_list:1; /* indicate fio is in io_list */ unsigned int is_por:1; /* indicate IO is from recovery or not */ - unsigned int encrypted:1; /* indicate file is encrypted */ unsigned int meta_gc:1; /* require meta inode GC */ enum iostat_type io_type; /* io type */ struct writeback_control *io_wbc; /* writeback control */ @@ -4199,7 +4198,6 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio); int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, int flag); int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, u64 start, u64 len); -int f2fs_encrypt_one_page(struct f2fs_io_info *fio); bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio); bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio); int f2fs_write_single_data_page(struct folio *folio, int *submitted, diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 4b52c56d71f0..089759366cdc 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -950,8 +950,6 @@ static bool f2fs_force_buffered_io(struct inode *inode, int rw) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - if (!fscrypt_dio_supported(inode)) - return true; if (fsverity_active(inode)) return true; if (f2fs_compressed_file(inode)) @@ -996,9 +994,7 @@ int f2fs_getattr(struct mnt_idmap *idmap, const struct path *path, } /* - * Return the DIO alignment restrictions if requested. We only return - * this information when requested, since on encrypted files it might - * take a fair bit of work to get if the file wasn't opened recently. + * Return the DIO alignment restrictions if requested. * * f2fs sometimes supports DIO reads but not DIO writes. STATX_DIOALIGN * cannot represent that, so in that case we report no DIO support. diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index d71ddb3ee918..642668a1847c 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -3986,8 +3986,6 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio) "%s Failed to allocate data block, ino:%u, index:%lu, type:%d, old_blkaddr:0x%x, new_blkaddr:0x%x, err:%d", __func__, fio->ino, folio->index, type, fio->old_blkaddr, fio->new_blkaddr, err); - if (fscrypt_inode_uses_fs_layer_crypto(folio->mapping->host)) - fscrypt_finalize_bounce_page(&fio->encrypted_page); folio_end_writeback(folio); if (f2fs_in_warm_node_list(folio)) f2fs_del_fsync_node_entry(fio->sbi, folio); diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index a7df36986616..be22c31015ef 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -3775,7 +3775,7 @@ f2fs_get_devices(struct super_block *sb, static const struct fscrypt_operations f2fs_cryptops = { .inode_info_offs = (int)offsetof(struct f2fs_inode_info, i_crypt_info) - (int)offsetof(struct f2fs_inode_info, vfs_inode), - .needs_bounce_pages = 1, + .is_block_based = 1, .has_32bit_inodes = 1, .supports_subblock_data_units = 1, .legacy_key_prefix = "f2fs:", diff --git a/include/linux/blk-crypto.h b/include/linux/blk-crypto.h index f7c3cb4a342f..938ff536838c 100644 --- a/include/linux/blk-crypto.h +++ b/include/linux/blk-crypto.h @@ -68,6 +68,15 @@ enum blk_crypto_key_type { */ #define BLK_CRYPTO_SW_SECRET_SIZE 32 +/* Flags for blk_crypto_config::flags: */ + +/* + * If set, inline encryption hardware will be used if available. + * If unset, CPU-based encryption will always be used (requires + * CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) + */ +#define BLK_CRYPTO_CFG_ALLOW_HW (1 << 0) + /** * struct blk_crypto_config - an inline encryption key's crypto configuration * @crypto_mode: encryption algorithm this key is for @@ -77,12 +86,14 @@ enum blk_crypto_key_type { * filesystem block size or the disk sector size. * @dun_bytes: the maximum number of bytes of DUN used when using this key * @key_type: the type of this key -- either raw or hardware-wrapped + * @flags: BLK_CRYPTO_CFG_* flags */ struct blk_crypto_config { enum blk_crypto_mode_num crypto_mode; unsigned int data_unit_size; unsigned int dun_bytes; enum blk_crypto_key_type key_type; + int flags; }; /** @@ -150,7 +161,7 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, enum blk_crypto_key_type key_type, enum blk_crypto_mode_num crypto_mode, unsigned int dun_bytes, - unsigned int data_unit_size); + unsigned int data_unit_size, int flags); int blk_crypto_start_using_key(struct block_device *bdev, const struct blk_crypto_key *key); @@ -160,8 +171,6 @@ void blk_crypto_evict_key(struct block_device *bdev, bool blk_crypto_config_supported_natively(struct block_device *bdev, const struct blk_crypto_config *cfg); -bool blk_crypto_config_supported(struct block_device *bdev, - const struct blk_crypto_config *cfg); int blk_crypto_derive_sw_secret(struct block_device *bdev, const u8 *eph_key, size_t eph_key_size, diff --git a/include/linux/fs/super_types.h b/include/linux/fs/super_types.h index a8da89a51401..ecd96aeb1cee 100644 --- a/include/linux/fs/super_types.h +++ b/include/linux/fs/super_types.h @@ -304,7 +304,7 @@ struct super_block { #define SB_NODIRATIME BIT(11) /* Do not update directory access times */ #define SB_SILENT BIT(15) #define SB_POSIXACL BIT(16) /* Supports POSIX ACLs */ -#define SB_INLINECRYPT BIT(17) /* Use blk-crypto for encrypted files */ +#define SB_INLINECRYPT BIT(17) /* Use inline crypto hardware if available */ #define SB_KERNMOUNT BIT(22) /* this is a kern_mount call */ #define SB_I_VERSION BIT(23) /* Update inode I_version field */ #define SB_LAZYTIME BIT(25) /* Update the on-disk [acm]times lazily */ diff --git a/include/linux/fscrypt.h b/include/linux/fscrypt.h index f6b235cd72b4..28f2108e29f3 100644 --- a/include/linux/fscrypt.h +++ b/include/linux/fscrypt.h @@ -72,14 +72,15 @@ struct fscrypt_operations { ptrdiff_t inode_info_offs; /* - * If set, then fs/crypto/ will allocate a global bounce page pool the - * first time an encryption key is set up for a file. The bounce page - * pool is required by the following functions: - * - * - fscrypt_encrypt_pagecache_blocks() - * - fscrypt_zeroout_range() for files not using inline crypto - * - * If the filesystem doesn't use those, it doesn't need to set this. + * Set to 1 if the filesystem is block-based. This causes fs/crypto/ to + * set up the key for regular files as a blk_crypto_key. The filesystem + * then uses fscrypt_set_bio_crypt_ctx() and similar functions. + */ + unsigned int is_block_based : 1; + + /* + * Set to 1 if the filesystem uses fscrypt_encrypt_pagecache_blocks(). + * This enables the allocation of the bounce page pool it requires. */ unsigned int needs_bounce_pages : 1; @@ -344,7 +345,6 @@ static inline void fscrypt_prepare_dentry(struct dentry *dentry, } /* crypto.c */ -void fscrypt_enqueue_decrypt_work(struct work_struct *); struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio, size_t len, size_t offs, gfp_t gfp_flags); @@ -352,8 +352,6 @@ int fscrypt_encrypt_block_inplace(const struct inode *inode, struct page *page, unsigned int len, unsigned int offs, u64 lblk_num); -int fscrypt_decrypt_pagecache_blocks(struct folio *folio, size_t len, - size_t offs); int fscrypt_decrypt_block_inplace(const struct inode *inode, struct page *page, unsigned int len, unsigned int offs, u64 lblk_num); @@ -450,11 +448,6 @@ bool fscrypt_match_name(const struct fscrypt_name *fname, const u8 *de_name, u32 de_name_len); u64 fscrypt_fname_siphash(const struct inode *dir, const struct qstr *name); -/* bio.c */ -bool fscrypt_decrypt_bio(struct bio *bio); -int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, - sector_t sector, u64 len); - /* hooks.c */ int fscrypt_file_open(struct inode *inode, struct file *filp); int __fscrypt_prepare_link(struct inode *inode, struct inode *dir, @@ -511,9 +504,6 @@ static inline void fscrypt_prepare_dentry(struct dentry *dentry, } /* crypto.c */ -static inline void fscrypt_enqueue_decrypt_work(struct work_struct *work) -{ -} static inline struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio, size_t len, size_t offs, gfp_t gfp_flags) @@ -529,12 +519,6 @@ static inline int fscrypt_encrypt_block_inplace(const struct inode *inode, return -EOPNOTSUPP; } -static inline int fscrypt_decrypt_pagecache_blocks(struct folio *folio, - size_t len, size_t offs) -{ - return -EOPNOTSUPP; -} - static inline int fscrypt_decrypt_block_inplace(const struct inode *inode, struct page *page, unsigned int len, @@ -751,18 +735,6 @@ static inline int fscrypt_d_revalidate(struct inode *dir, const struct qstr *nam return 1; } -/* bio.c */ -static inline bool fscrypt_decrypt_bio(struct bio *bio) -{ - return true; -} - -static inline int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, - sector_t sector, u64 len) -{ - return -EOPNOTSUPP; -} - /* hooks.c */ static inline int fscrypt_file_open(struct inode *inode, struct file *filp) @@ -862,28 +834,21 @@ static inline void fscrypt_set_ops(struct super_block *sb, #endif /* !CONFIG_FS_ENCRYPTION */ -/* inline_crypt.c */ +/* block.c */ #ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT -bool __fscrypt_inode_uses_inline_crypto(const struct inode *inode); - void fscrypt_set_bio_crypt_ctx(struct bio *bio, const struct inode *inode, loff_t pos, gfp_t gfp_mask); bool fscrypt_mergeable_bio(struct bio *bio, const struct inode *inode, loff_t pos); -bool fscrypt_dio_supported(struct inode *inode); - u64 fscrypt_limit_io_blocks(const struct inode *inode, u64 lblk, u64 nr_blocks); +int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, + sector_t sector, u64 len); #else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */ -static inline bool __fscrypt_inode_uses_inline_crypto(const struct inode *inode) -{ - return false; -} - static inline void fscrypt_set_bio_crypt_ctx(struct bio *bio, const struct inode *inode, loff_t pos, gfp_t gfp_mask) { } @@ -895,47 +860,18 @@ static inline bool fscrypt_mergeable_bio(struct bio *bio, return true; } -static inline bool fscrypt_dio_supported(struct inode *inode) -{ - return !fscrypt_needs_contents_encryption(inode); -} - static inline u64 fscrypt_limit_io_blocks(const struct inode *inode, u64 lblk, u64 nr_blocks) { return nr_blocks; } -#endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */ -/** - * fscrypt_inode_uses_inline_crypto() - test whether an inode uses inline - * encryption - * @inode: an inode. If encrypted, its key must be set up. - * - * Return: true if the inode requires file contents encryption and if the - * encryption should be done in the block layer via blk-crypto rather - * than in the filesystem layer. - */ -static inline bool fscrypt_inode_uses_inline_crypto(const struct inode *inode) -{ - return fscrypt_needs_contents_encryption(inode) && - __fscrypt_inode_uses_inline_crypto(inode); -} - -/** - * fscrypt_inode_uses_fs_layer_crypto() - test whether an inode uses fs-layer - * encryption - * @inode: an inode. If encrypted, its key must be set up. - * - * Return: true if the inode requires file contents encryption and if the - * encryption should be done in the filesystem layer rather than in the - * block layer via blk-crypto. - */ -static inline bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode) +static inline int fscrypt_zeroout_range(const struct inode *inode, loff_t pos, + sector_t sector, u64 len) { - return fscrypt_needs_contents_encryption(inode) && - !__fscrypt_inode_uses_inline_crypto(inode); + return -EOPNOTSUPP; } +#endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */ /** * fscrypt_has_encryption_key() - check whether an inode has had its key set up @@ -1123,15 +1059,4 @@ static inline int fscrypt_encrypt_symlink(struct inode *inode, return 0; } -/* If *pagep is a bounce page, free it and set *pagep to the pagecache page */ -static inline void fscrypt_finalize_bounce_page(struct page **pagep) -{ - struct page *page = *pagep; - - if (fscrypt_is_bounce_page(page)) { - *pagep = fscrypt_pagecache_page(page); - fscrypt_free_bounce_page(page); - } -} - #endif /* _LINUX_FSCRYPT_H */ diff --git a/include/uapi/linux/fscrypt.h b/include/uapi/linux/fscrypt.h index 3aff99f2696a..84507280b3ea 100644 --- a/include/uapi/linux/fscrypt.h +++ b/include/uapi/linux/fscrypt.h @@ -30,7 +30,6 @@ #define FSCRYPT_MODE_SM4_CTS 8 #define FSCRYPT_MODE_ADIANTUM 9 #define FSCRYPT_MODE_AES_256_HCTR2 10 -/* If adding a mode number > 10, update FSCRYPT_MODE_MAX in fscrypt_private.h */ /* * Legacy policy version; ad-hoc KDF and no key verification. diff --git a/tools/include/uapi/linux/fscrypt.h b/tools/include/uapi/linux/fscrypt.h index 3aff99f2696a..84507280b3ea 100644 --- a/tools/include/uapi/linux/fscrypt.h +++ b/tools/include/uapi/linux/fscrypt.h @@ -30,7 +30,6 @@ #define FSCRYPT_MODE_SM4_CTS 8 #define FSCRYPT_MODE_ADIANTUM 9 #define FSCRYPT_MODE_AES_256_HCTR2 10 -/* If adding a mode number > 10, update FSCRYPT_MODE_MAX in fscrypt_private.h */ /* * Legacy policy version; ad-hoc KDF and no key verification. |