linux-toradex.git/lib/decompress_inflate.c, branch v3.19.1

initramfs: support initramfs that is bigger than 2GiB

2014-08-08T22:57:26+00:00

Now with 64bit bzImage and kexec tools, we support ramdisk that size is
bigger than 2g, as we could put it above 4G.

Found compressed initramfs image could not be decompressed properly.  It
turns out that image length is int during decompress detection, and it
will become < 0 when length is more than 2G.  Furthermore, during
decompressing len as int is used for inbuf count, that has problem too.

Change len to long, that should be ok as on 32 bit platform long is
32bits.

Tested with following compressed initramfs image as root with kexec.
	gzip, bzip2, xz, lzma, lzop, lz4.
run time for populate_rootfs():
   size        name       Nehalem-EX  Westmere-EX  Ivybridge-EX
 9034400256 root_img     :   26s           24s          30s
 3561095057 root_img.lz4 :   28s           27s          27s
 3459554629 root_img.lzo :   29s           29s          28s
 3219399480 root_img.gz  :   64s           62s          49s
 2251594592 root_img.xz  :  262s          260s         183s
 2226366598 root_img.lzma:  386s          376s         277s
 2901482513 root_img.bz2 :  635s          599s

Signed-off-by: Yinghai Lu 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Rashika Kheria 
Cc: Josh Triplett 
Cc: Kyungsik Lee 
Cc: P J P 
Cc: Al Viro 
Cc: Tetsuo Handa 
Cc: "Daniel M. Weeks" 
Cc: Alexandre Courbot 
Cc: Jan Beulich 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/decompress_inflate.c: include appropriate header file

2014-04-03T23:21:12+00:00

Include appropriate header file include/linux/decompress/inflate.h in
lib/decompress_inflate.c because it has prototype declaration of
function defined in lib/decompress_inflate.c.

Also, fix the guard around the header file
include/linux/decompress/inflate.h to use a more unique guard symbol.
This avoids conflict with the INFLATE_H defined by
zlib_inflate/inflate.h.

This eliminates the following warning in lib/decompress_inflate.c:

  lib/decompress_inflate.c:35:17: warning: no previous prototype for `gunzip' [-Wmissing-prototypes]

Signed-off-by: Rashika Kheria 
Reviewed-by: Josh Triplett 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/decompressors: fix "no limit" output buffer length

2013-09-11T22:58:38+00:00

When decompressing into memory, the output buffer length is set to some
arbitrarily high value (0x7fffffff) to indicate the output is, virtually,
unlimited in size.

The problem with this is that some platforms have their physical memory at
high physical addresses (0x80000000 or more), and that the output buffer
address and its "unlimited" length cannot be added without overflowing.
An example of this can be found in inflate_fast():

/* next_out is the output buffer address */
out = strm->next_out - OFF;
/* avail_out is the output buffer size. end will overflow if the output
 * address is >= 0x80000104 */
end = out + (strm->avail_out - 257);

This has huge consequences on the performance of kernel decompression,
since the following exit condition of inflate_fast() will be always true:

} while (in < last && out < end);

Indeed, "end" has overflowed and is now always lower than "out".  As a
result, inflate_fast() will return after processing one single byte of
input data, and will thus need to be called an unreasonably high number of
times.  This probably went unnoticed because kernel decompression is fast
enough even with this issue.

Nonetheless, adjusting the output buffer length in such a way that the
above pointer arithmetic never overflows results in a kernel decompression
that is about 3 times faster on affected machines.

Signed-off-by: Alexandre Courbot 
Tested-by: Jon Medhurst 
Cc: Stephen Warren 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

decompressors: check input size in decompress_inflate.c

2011-01-13T16:03:25+00:00

Check for end of the input buffer when skipping over the filename field in
the .gz file header.

Signed-off-by: Lasse Collin 
Cc: "H. Peter Anvin" 
Cc: Alain Knaff 
Cc: Albin Tonnerre 
Cc: Phillip Lougher 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Decompressors: include in

2011-01-13T16:03:23+00:00

Currently users of mm.h need to include  to use the macros
malloc() and free() provided by mm.h.  This fixes it.

Signed-off-by: Lasse Collin 
Cc: "H. Peter Anvin" 
Cc: Alain Knaff 
Cc: Albin Tonnerre 
Cc: Phillip Lougher 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Decompressors: get rid of set_error_fn() macro

2011-01-13T16:03:23+00:00

set_error_fn() has become a useless complication after c1e7c3ae59
("bzip2/lzma/gzip: pre-boot malloc doesn't return NULL on failure") fixed
the use of error() in malloc().  Only decompress_unlzma.c had some use for
it and that was easy to change too.

This also gets rid of the static function pointer "error", which
should have been marked as __initdata.

Signed-off-by: Lasse Collin 
Cc: "H. Peter Anvin" 
Cc: Alain Knaff 
Cc: Albin Tonnerre 
Cc: Phillip Lougher 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Decompressors: add missing INIT (i.e. __init)

2011-01-13T16:03:23+00:00

Signed-off-by: Lasse Collin 
Cc: "H. Peter Anvin" 
Cc: Alain Knaff 
Cc: Albin Tonnerre 
Cc: Phillip Lougher 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lzma/gzip: fix potential oops when input data is truncated

2009-09-24T14:21:05+00:00

If the lzma/gzip decompressors are called with insufficient input data
(len > 0 & fill = NULL), they will attempt to call the fill function to
obtain more data, leading to a kernel oops.

Signed-off-by: Phillip Lougher 
Cc: "H. Peter Anvin" 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/decompress_*: only include if STATIC is not defined

2009-08-07T17:39:56+00:00

These includes were added by 079effb6933f34b9b1b67b08bd4fd7fb672d16ef
("kmemtrace, kbuild: fix slab.h dependency problem in
lib/decompress_inflate.c") to fix the build when using kmemtrace.  However
this is not necessary when used to create a compressed kernel, and
actually creates issues (brings a lot of things unavailable in the
decompression environment), so don't include it if STATIC is defined.

Signed-off-by: Albin Tonnerre 
Cc: Sam Ravnborg 
Cc: Russell King 
Cc: Ingo Molnar 
Cc: Thomas Gleixner 
Cc: "H. Peter Anvin" 
Cc: Pekka Enberg 
Cc: Eduard - Gabriel Munteanu 
Cc: Phillip Lougher 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bzip2/lzma/gzip: fix comments describing decompressor API

2009-08-07T17:39:56+00:00

Fix and improve comments in decompress/generic.h that describe the
decompressor API.  Also remove an unused definition, and rename INBUF_LEN
in lib/decompress_inflate.c to conform to bzip2/lzma naming.

Signed-off-by: Phillip Lougher 
Cc: "H. Peter Anvin" 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds