linux-toradex.git/arch/x86/net, branch v3.14.52

x86: bpf_jit: fix compilation of large bpf programs

2015-06-23T00:01:16+00:00

[ Upstream commit 3f7352bf21f8fd7ba3e2fcef9488756f188e12be ]

x86 has variable length encoding. x86 JIT compiler is trying
to pick the shortest encoding for given bpf instruction.
While doing so the jump targets are changing, so JIT is doing
multiple passes over the program. Typical program needs 3 passes.
Some very short programs converge with 2 passes. Large programs
may need 4 or 5. But specially crafted bpf programs may hit the
pass limit and if the program converges on the last iteration
the JIT compiler will be producing an image full of 'int 3' insns.
Fix this corner case by doing final iteration over bpf program.

Fixes: 0a14842f5a3c ("net: filter: Just In Time compiler for x86-64")
Reported-by: Daniel Borkmann 
Signed-off-by: Alexei Starovoitov 
Tested-by: Daniel Borkmann 
Acked-by: Daniel Borkmann 
Signed-off-by: David S. Miller 
Signed-off-by: Greg Kroah-Hartman

net: filter: x86: fix JIT address randomization

2014-05-31T20:20:38+00:00

[ Upstream commit 773cd38f40b8834be991dbfed36683acc1dd41ee ]

bpf_alloc_binary() adds 128 bytes of room to JITed program image
and rounds it up to the nearest page size. If image size is close
to page size (like 4000), it is rounded to two pages:
round_up(4000 + 4 + 128) == 8192
then 'hole' is computed as 8192 - (4000 + 4) = 4188
If prandom_u32() % hole selects a number >= PAGE_SIZE - sizeof(*header)
then kernel will crash during bpf_jit_free():

kernel BUG at arch/x86/mm/pageattr.c:887!
Call Trace:
 [] change_page_attr_set_clr+0x135/0x460
 [] ? _raw_spin_unlock_irq+0x30/0x50
 [] set_memory_rw+0x2f/0x40
 [] bpf_jit_free_deferred+0x2d/0x60
 [] process_one_work+0x1d8/0x6a0
 [] ? process_one_work+0x178/0x6a0
 [] worker_thread+0x11c/0x370

since bpf_jit_free() does:
  unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK;
  struct bpf_binary_header *header = (void *)addr;
to compute start address of 'bpf_binary_header'
and header->pages will pass junk to:
  set_memory_rw(addr, header->pages);

Fix it by making sure that &header->image[prandom_u32() % hole] and &header
are in the same page

Fixes: 314beb9bcabfd ("x86: bpf_jit_comp: secure bpf jit against spraying attacks")
Signed-off-by: Alexei Starovoitov 
Acked-by: Eric Dumazet 
Signed-off-by: David S. Miller 
Signed-off-by: Greg Kroah-Hartman

x86: bpf_jit: support negative offsets

2014-03-12T03:25:22+00:00

Commit a998d4342337 claimed to introduce negative offset support to x86 jit,
but it couldn't be working, since at the time of the execution
of LD+ABS or LD+IND instructions via call into
bpf_internal_load_pointer_neg_helper() the %edx (3rd argument of this func)
had junk value instead of access size in bytes (1 or 2 or 4).

Store size into %edx instead of %ecx (what original commit intended to do)

Fixes: a998d4342337 ("bpf jit: Let the x86 jit handle negative offsets")
Signed-off-by: Alexei Starovoitov 
Cc: Jan Seiffert 
Cc: Eric Dumazet 
Acked-by: Eric Dumazet 
Signed-off-by: David S. Miller

bpf: do not use reciprocal divide

2014-01-16T01:02:08+00:00

At first Jakub Zawadzki noticed that some divisions by reciprocal_divide
were not correct. (off by one in some cases)
http://www.wireshark.org/~darkjames/reciprocal-buggy.c

He could also show this with BPF:
http://www.wireshark.org/~darkjames/set-and-dump-filter-k-bug.c

The reciprocal divide in linux kernel is not generic enough,
lets remove its use in BPF, as it is not worth the pain with
current cpus.

Signed-off-by: Eric Dumazet 
Reported-by: Jakub Zawadzki 
Cc: Mircea Gherzan 
Cc: Daniel Borkmann 
Cc: Hannes Frederic Sowa 
Cc: Matt Evans 
Cc: Martin Schwidefsky 
Cc: Heiko Carstens 
Cc: David S. Miller 
Signed-off-by: David S. Miller

net: x86: bpf: don't forget to free sk_filter (v2)

2013-11-08T00:06:52+00:00

sk_filter isn't freed if bpf_func is equal to sk_run_filter.

This memory leak was introduced by v3.12-rc3-224-gd45ed4a4
"net: fix unsafe set_memory_rw from softirq".

Before this patch sk_filter was freed in sk_filter_release_rcu,
now it should be freed in bpf_jit_free.

Here is output of kmemleak:
unreferenced object 0xffff8800b774eab0 (size 128):
  comm "systemd", pid 1, jiffies 4294669014 (age 124.062s)
  hex dump (first 32 bytes):
    00 00 00 00 0b 00 00 00 20 63 7f b7 00 88 ff ff  ........ c......
    60 d4 55 81 ff ff ff ff 30 d9 55 81 ff ff ff ff  `.U.....0.U.....
  backtrace:
    [] kmemleak_alloc+0x4e/0xb0
    [] __kmalloc+0xef/0x260
    [] sock_kmalloc+0x38/0x60
    [] sk_attach_filter+0x5d/0x190
    [] sock_setsockopt+0x991/0x9e0
    [] SyS_setsockopt+0xb6/0xd0
    [] system_call_fastpath+0x16/0x1b
    [] 0xffffffffffffffff

v2: add extra { } after else

Fixes: d45ed4a4e33a ("net: fix unsafe set_memory_rw from softirq")
Acked-by: Daniel Borkmann 
Cc: Alexei Starovoitov 
Cc: Eric Dumazet 
Cc: "David S. Miller" 
Signed-off-by: Andrey Vagin 
Acked-by: Alexei Starovoitov 
Signed-off-by: David S. Miller

net: fix unsafe set_memory_rw from softirq

2013-10-07T19:16:45+00:00

on x86 system with net.core.bpf_jit_enable = 1

sudo tcpdump -i eth1 'tcp port 22'

causes the warning:
[   56.766097]  Possible unsafe locking scenario:
[   56.766097]
[   56.780146]        CPU0
[   56.786807]        ----
[   56.793188]   lock(&(&vb->lock)->rlock);
[   56.799593]   
[   56.805889]     lock(&(&vb->lock)->rlock);
[   56.812266]
[   56.812266]  *** DEADLOCK ***
[   56.812266]
[   56.830670] 1 lock held by ksoftirqd/1/13:
[   56.836838]  #0:  (rcu_read_lock){.+.+..}, at: [] vm_unmap_aliases+0x8c/0x380
[   56.849757]
[   56.849757] stack backtrace:
[   56.862194] CPU: 1 PID: 13 Comm: ksoftirqd/1 Not tainted 3.12.0-rc3+ #45
[   56.868721] Hardware name: System manufacturer System Product Name/P8Z77 WS, BIOS 3007 07/26/2012
[   56.882004]  ffffffff821944c0 ffff88080bbdb8c8 ffffffff8175a145 0000000000000007
[   56.895630]  ffff88080bbd5f40 ffff88080bbdb928 ffffffff81755b14 0000000000000001
[   56.909313]  ffff880800000001 ffff880800000000 ffffffff8101178f 0000000000000001
[   56.923006] Call Trace:
[   56.929532]  [] dump_stack+0x55/0x76
[   56.936067]  [] print_usage_bug+0x1f7/0x208
[   56.942445]  [] ? save_stack_trace+0x2f/0x50
[   56.948932]  [] ? check_usage_backwards+0x150/0x150
[   56.955470]  [] mark_lock+0x282/0x2c0
[   56.961945]  [] __lock_acquire+0x45d/0x1d50
[   56.968474]  [] ? __lock_acquire+0x2de/0x1d50
[   56.975140]  [] ? cpumask_next_and+0x55/0x90
[   56.981942]  [] lock_acquire+0x92/0x1d0
[   56.988745]  [] ? vm_unmap_aliases+0x16a/0x380
[   56.995619]  [] _raw_spin_lock+0x41/0x50
[   57.002493]  [] ? vm_unmap_aliases+0x16a/0x380
[   57.009447]  [] vm_unmap_aliases+0x16a/0x380
[   57.016477]  [] ? vm_unmap_aliases+0x8c/0x380
[   57.023607]  [] change_page_attr_set_clr+0xc0/0x460
[   57.030818]  [] ? trace_hardirqs_on+0xd/0x10
[   57.037896]  [] ? kmem_cache_free+0xb0/0x2b0
[   57.044789]  [] ? free_object_rcu+0x93/0xa0
[   57.051720]  [] set_memory_rw+0x2f/0x40
[   57.058727]  [] bpf_jit_free+0x2c/0x40
[   57.065577]  [] sk_filter_release_rcu+0x1a/0x30
[   57.072338]  [] rcu_process_callbacks+0x202/0x7c0
[   57.078962]  [] __do_softirq+0xf7/0x3f0
[   57.085373]  [] run_ksoftirqd+0x35/0x70

cannot reuse jited filter memory, since it's readonly,
so use original bpf insns memory to hold work_struct

defer kfree of sk_filter until jit completed freeing

tested on x86_64 and i386

Signed-off-by: Alexei Starovoitov 
Acked-by: Eric Dumazet 
Signed-off-by: David S. Miller

x86: bpf_jit_comp: secure bpf jit against spraying attacks

2013-05-20T06:55:41+00:00

hpa bringed into my attention some security related issues
with BPF JIT on x86.

This patch makes sure the bpf generated code is marked read only,
as other kernel text sections.

It also splits the unused space (we vmalloc() and only use a fraction of
the page) in two parts, so that the generated bpf code not starts at a
known offset in the page, but a pseudo random one.

Refs:
http://mainisusuallyafunction.blogspot.com/2012/11/attacking-hardened-linux-systems-with.html

Reported-by: H. Peter Anvin 
Signed-off-by: Eric Dumazet 
Reviewed-by: Daniel Borkmann 
Signed-off-by: David S. Miller

x86: bpf_jit_comp: can call module_free() from any context

2013-05-18T00:37:19+00:00

It looks like we can call module_free()/vfree() from softirq context,
so no longer need a wrapper and a work_struct.

Signed-off-by: Eric Dumazet 
Cc: Al Viro 
Signed-off-by: David S. Miller

filter: bpf_jit_comp: refactor and unify BPF JIT image dump output

2013-03-21T21:25:56+00:00

If bpf_jit_enable > 1, then we dump the emitted JIT compiled image
after creation. Currently, only SPARC and PowerPC has similar output
as in the reference implementation on x86_64. Make a small helper
function in order to reduce duplicated code and make the dump output
uniform across architectures x86_64, SPARC, PPC, ARM (e.g. on ARM
flen, pass and proglen are currently not shown, but would be
interesting to know as well), also for future BPF JIT implementations
on other archs.

Cc: Mircea Gherzan 
Cc: Matt Evans 
Cc: Eric Dumazet 
Cc: David S. Miller 
Signed-off-by: Daniel Borkmann 
Signed-off-by: David S. Miller

x86: bpf_jit_comp: add pkt_type support

2013-01-31T03:38:34+00:00

Supporting access to skb->pkt_type is a bit tricky if we want
to have a generic code, allowing pkt_type to be moved in struct sk_buff

pkt_type is a bit field, so compiler cannot really help us to find
its offset. Let's use a helper for this : It will throw a one time
message if pkt_type no longer starts at a byte boundary or is
no longer a 3bit field.

Reported-by: Willem de Bruijn 
Signed-off-by: Eric Dumazet 
Cc: Maciej Żenczykowski 
Signed-off-by: David S. Miller