Merge __{get,put}_user() removal branch

This is long over-due: the __get_user() and __put_user() functions are
entirely historical, and are never the rigth thing to use any more.  And
when mis-used (read: every case) they are not only slower than the right
thing, but also insecure because they don't check the user address
range.

Most of the usage sites have been converted long ago, but various
left-overs remained.  So this started out as a single patch that I wrote
as a "show people how to convert __{get,put}_user() to modern code".

In particular, Thomas last year implemented the very nice new scoped
user access region helpers (see commit e497310b4ffb: "uaccess: Provide
scoped user access regions") that makes the modern interfaces
particularly easy to use and really removes any excuse for the old
horrors.

That small example patch then grew to this, partly using Claude as a
glorified coccinelle script to apply the pattern to other places.

There were a few different reasons for using __put_user() and
__get_user() rather than the alternatives, but the original historical
one was performance: when doing multiple accesses off the same base
pointer, doing a single range check followed by multiple non-checked
accesses used to create smaller and faster code.

The keyword here is "used to".  It hasn't been true for years, because
the range check isn't actually the real cost these days.  The real cost
of user accesses on modern hardware is turning the user accesses on and
off in the first place (on x86, using the STAC/CLAC instructions).

Those sequences get turned into 'scoped_user_read_access() { }' blocks,
see for example that first patch in the series: "signal: use scoped user
access for old-style sigaction".

Another, historical reason for using __get_user() and friends is that
you actually want to do a kernel pointer access, but the pointer might
not be entirely reliable (typically during fault handling), so you want
to have exception handling for the kernel access.

The modern way to do that kind of non-faulting kernel access is using
"get_kernel_nofault()", but some code predates that helper existing at
all, and used __get_user() as a way to just get a memory access with
exception handling.  For that pattern, see for example "alpha: use
get_kernel_nofault() in dik_show_code()"

A third bad reason for using __get_user() and friends was that the
proper get_user() had a "might_fault()" in it, and code that ran in
atomic regions but hadn't disabled page faults would result in a
warning.  That kind of code needs to explicitly disable page faults,
because "atomic region" is not well defined at run-time and can be
invisible (eg a preempt-off region in a non-preemptible kernel).

See "vdpa_sim: disable page faults around vringh_need_notify_iotlb()"
for an example of that pattern, where the local_bh_disable() now also
grows a pagefault_disable(), which makes the normal get_user() no longer
warn about the user access in non-process context, because it can see
that the potential page fault will not cause problems.

And finally, the really nasty reason to use __get_user() and friends:
pointers that can be either kernel pointers or user pointers.  We've
gotten rid of almost all such cases, but a couple remained.  See for
example "x86/sev: Use explicit user-or-kernel accessors for #VC memory
operands".

NOTE! This series removes all __get/put_user() calls from generic code
but only does the final removal of the whole interface on x86.  Other
architectures have not been entirely converted, for a variety of reasons:

 - arm64 makes __{get,put}_user() be exactly the same as
   {get,put}_user() so it's harmless and didn't have the dangers of the
   legacy use cases.

 - any architecture that uses <asm-generic/uaccess.h> will still use
   __{get,put}_user as the underlying helper function name

 - several architectures had some legacy uses that weren't entirely
   trivial and I didn't want to extend this patch series to things that
   I don't know and that don't really matter.

but I did do a couple of the obvious "follows the exact same pattern"
just because it could be done mindlessly and automatically.

Let's see if I broke anything, but we should have done this years ago.

* uaccess-cleanups: (48 commits)
  drm/i915/gem: ignore errors on presumed_offset write-back
  powerpc/align: use get_user_instr() for a user-mode instruction
  powerpc/signal64: use scoped user access in setup_trampoline()
  powerpc/signal64: use get_user() for the single context reads
  powerpc/signal32: use get_user() for the single context reads
  KVM: PPC: Book3S HV: use get_user()/put_user() for the HPT fd
  powerpc/powernv: use get_user()/put_user() in the LPC debugfs files
  s390: use get_user()/put_user() instead of the unchecked variants
  ARM: OABI: use scoped user access in epoll_put_uevent()
  ARM: signal: stop using __put_user() for the signal frame
  ARM: signal: check the FDPIC function descriptor address
  alpha: split the prefetch check in do_page_fault() by mode
  openrisc: use get_kernel_nofault() in show_registers()
  alpha: use get_kernel_nofault() in dik_show_code()
  nios2: use scoped user access for the signal frame
  nios2: check the signal frame address before writing to it
  hexagon: use scoped user access for the signal frame
  alpha: use scoped user access when setting up signal frames
  alpha: use scoped user access in the sigreturn path
  alpha: use scoped user access in osf_sigaction()
  ...

6 files changed, 70 insertions, 70 deletions

diff --git a/kernel/compat.c b/kernel/compat.c
index fb50f29d9b36..c754793c7d65 100644
--- a/kernel/compat.c
+++ b/kernel/compat.c
@@ -180,14 +180,18 @@ int get_compat_sigevent(struct sigevent *event,
 		const struct compat_sigevent __user *u_event)
 {
 	memset(event, 0, sizeof(*event));
-	return (!access_ok(u_event, sizeof(*u_event)) ||
-		__get_user(event->sigev_value.sival_int,
-			&u_event->sigev_value.sival_int) ||
-		__get_user(event->sigev_signo, &u_event->sigev_signo) ||
-		__get_user(event->sigev_notify, &u_event->sigev_notify) ||
-		__get_user(event->sigev_notify_thread_id,
-			&u_event->sigev_notify_thread_id))
-		? -EFAULT : 0;
+	scoped_user_read_access(u_event, Efault) {
+		unsafe_get_user(event->sigev_value.sival_int,
+				&u_event->sigev_value.sival_int, Efault);
+		unsafe_get_user(event->sigev_signo, &u_event->sigev_signo, Efault);
+		unsafe_get_user(event->sigev_notify, &u_event->sigev_notify, Efault);
+		unsafe_get_user(event->sigev_notify_thread_id,
+				&u_event->sigev_notify_thread_id, Efault);
+	}
+	return 0;
+
+Efault:
+	return -EFAULT;
 }
 
 long compat_get_bitmap(unsigned long *mask, const compat_ulong_t __user *umask,
diff --git a/kernel/entry/syscall_user_dispatch.c b/kernel/entry/syscall_user_dispatch.c
index 59c861866941..7d1dd7f1bf38 100644
--- a/kernel/entry/syscall_user_dispatch.c
+++ b/kernel/entry/syscall_user_dispatch.c
@@ -45,11 +45,7 @@ bool syscall_user_dispatch(struct pt_regs *regs)
 		return false;
 
 	if (likely(sd->selector)) {
-		/*
-		 * access_ok() is performed once, at prctl time, when
-		 * the selector is loaded by userspace.
-		 */
-		if (unlikely(__get_user(state, sd->selector))) {
+		if (unlikely(get_user(state, untagged_addr(sd->selector)))) {
 			force_exit_sig(SIGSEGV);
 			return true;
 		}
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 7709ea882477..3192806eebfe 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -2404,7 +2404,7 @@ static int is_trap_at_addr(struct mm_struct *mm, unsigned long vaddr)
 		return -EINVAL;
 
 	pagefault_disable();
-	result = __get_user(opcode, (uprobe_opcode_t __user *)vaddr);
+	result = get_user(opcode, (uprobe_opcode_t __user *)vaddr);
 	pagefault_enable();
 
 	if (likely(result == 0))
diff --git a/kernel/ptrace.c b/kernel/ptrace.c
index 4e9822a87aab..3f2fe68ccb14 100644
--- a/kernel/ptrace.c
+++ b/kernel/ptrace.c
@@ -1365,16 +1365,13 @@ int ptrace_request(struct task_struct *child, long request,
 		struct iovec kiov;
 		struct iovec __user *uiov = datavp;
 
-		if (!access_ok(uiov, sizeof(*uiov)))
-			return -EFAULT;
-
-		if (__get_user(kiov.iov_base, &uiov->iov_base) ||
-		    __get_user(kiov.iov_len, &uiov->iov_len))
+		if (get_user(kiov.iov_base, &uiov->iov_base) ||
+		    get_user(kiov.iov_len, &uiov->iov_len))
 			return -EFAULT;
 
 		ret = ptrace_regset(child, request, addr, &kiov);
 		if (!ret)
-			ret = __put_user(kiov.iov_len, &uiov->iov_len);
+			ret = put_user(kiov.iov_len, &uiov->iov_len);
 		break;
 	}
 
@@ -1531,11 +1528,8 @@ int compat_ptrace_request(struct task_struct *child, compat_long_t request,
 		compat_uptr_t ptr;
 		compat_size_t len;
 
-		if (!access_ok(uiov, sizeof(*uiov)))
-			return -EFAULT;
-
-		if (__get_user(ptr, &uiov->iov_base) ||
-		    __get_user(len, &uiov->iov_len))
+		if (get_user(ptr, &uiov->iov_base) ||
+		    get_user(len, &uiov->iov_len))
 			return -EFAULT;
 
 		kiov.iov_base = compat_ptr(ptr);
@@ -1543,7 +1537,7 @@ int compat_ptrace_request(struct task_struct *child, compat_long_t request,
 
 		ret = ptrace_regset(child, request, addr, &kiov);
 		if (!ret)
-			ret = __put_user(kiov.iov_len, &uiov->iov_len);
+			ret = put_user(kiov.iov_len, &uiov->iov_len);
 		break;
 	}
 #endif
diff --git a/kernel/signal.c b/kernel/signal.c
index e433de93b430..cb6ec7054eee 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -4553,11 +4553,12 @@ int restore_altstack(const stack_t __user *uss)
 
 int __save_altstack(stack_t __user *uss, unsigned long sp)
 {
-	struct task_struct *t = current;
-	int err = __put_user((void __user *)t->sas_ss_sp, &uss->ss_sp) |
-		__put_user(t->sas_ss_flags, &uss->ss_flags) |
-		__put_user(t->sas_ss_size, &uss->ss_size);
-	return err;
+	scoped_user_write_access(uss, Efault)
+		unsafe_save_altstack(uss, sp, Efault);
+	return 0;
+
+Efault:
+	return -EFAULT;
 }
 
 #ifdef CONFIG_COMPAT
@@ -4606,13 +4607,12 @@ int compat_restore_altstack(const compat_stack_t __user *uss)
 
 int __compat_save_altstack(compat_stack_t __user *uss, unsigned long sp)
 {
-	int err;
-	struct task_struct *t = current;
-	err = __put_user(ptr_to_compat((void __user *)t->sas_ss_sp),
-			 &uss->ss_sp) |
-		__put_user(t->sas_ss_flags, &uss->ss_flags) |
-		__put_user(t->sas_ss_size, &uss->ss_size);
-	return err;
+	scoped_user_write_access(uss, Efault)
+		unsafe_compat_save_altstack(uss, sp, Efault);
+	return 0;
+
+Efault:
+	return -EFAULT;
 }
 #endif
 
@@ -4790,12 +4790,12 @@ SYSCALL_DEFINE3(sigaction, int, sig,
 
 	if (act) {
 		old_sigset_t mask;
-		if (!access_ok(act, sizeof(*act)) ||
-		    __get_user(new_ka.sa.sa_handler, &act->sa_handler) ||
-		    __get_user(new_ka.sa.sa_restorer, &act->sa_restorer) ||
-		    __get_user(new_ka.sa.sa_flags, &act->sa_flags) ||
-		    __get_user(mask, &act->sa_mask))
-			return -EFAULT;
+		scoped_user_read_access(act, Efault) {
+			unsafe_get_user(new_ka.sa.sa_handler, &act->sa_handler, Efault);
+			unsafe_get_user(new_ka.sa.sa_restorer, &act->sa_restorer, Efault);
+			unsafe_get_user(new_ka.sa.sa_flags, &act->sa_flags, Efault);
+			unsafe_get_user(mask, &act->sa_mask, Efault);
+		}
 #ifdef __ARCH_HAS_KA_RESTORER
 		new_ka.ka_restorer = NULL;
 #endif
@@ -4805,15 +4805,18 @@ SYSCALL_DEFINE3(sigaction, int, sig,
 	ret = do_sigaction(sig, act ? &new_ka : NULL, oact ? &old_ka : NULL);
 
 	if (!ret && oact) {
-		if (!access_ok(oact, sizeof(*oact)) ||
-		    __put_user(old_ka.sa.sa_handler, &oact->sa_handler) ||
-		    __put_user(old_ka.sa.sa_restorer, &oact->sa_restorer) ||
-		    __put_user(old_ka.sa.sa_flags, &oact->sa_flags) ||
-		    __put_user(old_ka.sa.sa_mask.sig[0], &oact->sa_mask))
-			return -EFAULT;
+		scoped_user_write_access(oact, Efault) {
+			unsafe_put_user(old_ka.sa.sa_handler, &oact->sa_handler, Efault);
+			unsafe_put_user(old_ka.sa.sa_restorer, &oact->sa_restorer, Efault);
+			unsafe_put_user(old_ka.sa.sa_flags, &oact->sa_flags, Efault);
+			unsafe_put_user(old_ka.sa.sa_mask.sig[0], &oact->sa_mask, Efault);
+		}
 	}
 
 	return ret;
+
+Efault:
+	return -EFAULT;
 }
 #endif
 #ifdef CONFIG_COMPAT_OLD_SIGACTION
@@ -4827,12 +4830,12 @@ COMPAT_SYSCALL_DEFINE3(sigaction, int, sig,
 	compat_uptr_t handler, restorer;
 
 	if (act) {
-		if (!access_ok(act, sizeof(*act)) ||
-		    __get_user(handler, &act->sa_handler) ||
-		    __get_user(restorer, &act->sa_restorer) ||
-		    __get_user(new_ka.sa.sa_flags, &act->sa_flags) ||
-		    __get_user(mask, &act->sa_mask))
-			return -EFAULT;
+		scoped_user_read_access(act, Efault) {
+			unsafe_get_user(handler, &act->sa_handler, Efault);
+			unsafe_get_user(restorer, &act->sa_restorer, Efault);
+			unsafe_get_user(new_ka.sa.sa_flags, &act->sa_flags, Efault);
+			unsafe_get_user(mask, &act->sa_mask, Efault);
+		}
 
 #ifdef __ARCH_HAS_KA_RESTORER
 		new_ka.ka_restorer = NULL;
@@ -4845,16 +4848,19 @@ COMPAT_SYSCALL_DEFINE3(sigaction, int, sig,
 	ret = do_sigaction(sig, act ? &new_ka : NULL, oact ? &old_ka : NULL);
 
 	if (!ret && oact) {
-		if (!access_ok(oact, sizeof(*oact)) ||
-		    __put_user(ptr_to_compat(old_ka.sa.sa_handler),
-			       &oact->sa_handler) ||
-		    __put_user(ptr_to_compat(old_ka.sa.sa_restorer),
-			       &oact->sa_restorer) ||
-		    __put_user(old_ka.sa.sa_flags, &oact->sa_flags) ||
-		    __put_user(old_ka.sa.sa_mask.sig[0], &oact->sa_mask))
-			return -EFAULT;
+		scoped_user_write_access(oact, Efault) {
+			unsafe_put_user(ptr_to_compat(old_ka.sa.sa_handler),
+					&oact->sa_handler, Efault);
+			unsafe_put_user(ptr_to_compat(old_ka.sa.sa_restorer),
+					&oact->sa_restorer, Efault);
+			unsafe_put_user(old_ka.sa.sa_flags, &oact->sa_flags, Efault);
+			unsafe_put_user(old_ka.sa.sa_mask.sig[0], &oact->sa_mask, Efault);
+		}
 	}
 	return ret;
+
+Efault:
+	return -EFAULT;
 }
 #endif
 
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 29260951aa87..88c3b5b513ab 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -2388,7 +2388,7 @@ static int copy_user_syms(struct user_syms *us, unsigned long __user *usyms, u32
 		goto error;
 
 	for (p = buf, i = 0; i < cnt; i++) {
-		if (__get_user(usymbol, usyms + i)) {
+		if (get_user(usymbol, usyms + i)) {
 			err = -EFAULT;
 			goto error;
 		}
@@ -3295,8 +3295,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 
 	/*
 	 * All uoffsets/uref_ctr_offsets/ucookies arrays have the same value
-	 * size, we need to check their address range is safe for __get_user
-	 * calls.
+	 * size.  Check their address ranges up front, so that a bad pointer
+	 * fails before anything is looked up or allocated.
 	 */
 	size = sizeof(*uoffsets) * cnt;
 	if (!access_ok(uoffsets, size) || !access_ok(uref_ctr_offsets, size) ||
@@ -3331,7 +3331,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 		goto error_free;
 
 	for (i = 0; i < cnt; i++) {
-		if (__get_user(uprobes[i].offset, uoffsets + i)) {
+		if (get_user(uprobes[i].offset, uoffsets + i)) {
 			err = -EFAULT;
 			goto error_free;
 		}
@@ -3339,11 +3339,11 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 			err = -EINVAL;
 			goto error_free;
 		}
-		if (uref_ctr_offsets && __get_user(uprobes[i].ref_ctr_offset, uref_ctr_offsets + i)) {
+		if (uref_ctr_offsets && get_user(uprobes[i].ref_ctr_offset, uref_ctr_offsets + i)) {
 			err = -EFAULT;
 			goto error_free;
 		}
-		if (ucookies && __get_user(uprobes[i].cookie, ucookies + i)) {
+		if (ucookies && get_user(uprobes[i].cookie, ucookies + i)) {
 			err = -EFAULT;
 			goto error_free;
 		}