linux-toradex.git/lib/string_helpers.c, branch v4.15-rc1

mm: treewide: remove GFP_TEMPORARY allocation flag

2017-09-14T01:53:16+00:00

GFP_TEMPORARY was introduced by commit e12ba74d8ff3 ("Group short-lived
and reclaimable kernel allocations") along with __GFP_RECLAIMABLE.  It's
primary motivation was to allow users to tell that an allocation is
short lived and so the allocator can try to place such allocations close
together and prevent long term fragmentation.  As much as this sounds
like a reasonable semantic it becomes much less clear when to use the
highlevel GFP_TEMPORARY allocation flag.  How long is temporary? Can the
context holding that memory sleep? Can it take locks? It seems there is
no good answer for those questions.

The current implementation of GFP_TEMPORARY is basically GFP_KERNEL |
__GFP_RECLAIMABLE which in itself is tricky because basically none of
the existing caller provide a way to reclaim the allocated memory.  So
this is rather misleading and hard to evaluate for any benefits.

I have checked some random users and none of them has added the flag
with a specific justification.  I suspect most of them just copied from
other existing users and others just thought it might be a good idea to
use without any measuring.  This suggests that GFP_TEMPORARY just
motivates for cargo cult usage without any reasoning.

I believe that our gfp flags are quite complex already and especially
those with highlevel semantic should be clearly defined to prevent from
confusion and abuse.  Therefore I propose dropping GFP_TEMPORARY and
replace all existing users to simply use GFP_KERNEL.  Please note that
SLAB users with shrinkers will still get __GFP_RECLAIMABLE heuristic and
so they will be placed properly for memory fragmentation prevention.

I can see reasons we might want some gfp flag to reflect shorterm
allocations but I propose starting from a clear semantic definition and
only then add users with proper justification.

This was been brought up before LSF this year by Matthew [1] and it
turned out that GFP_TEMPORARY really doesn't have a clear semantic.  It
seems to be a heuristic without any measured advantage for most (if not
all) its current users.  The follow up discussion has revealed that
opinions on what might be temporary allocation differ a lot between
developers.  So rather than trying to tweak existing users into a
semantic which they haven't expected I propose to simply remove the flag
and start from scratch if we really need a semantic for short term
allocations.

[1] http://lkml.kernel.org/r/20170118054945.GD18349@bombadil.infradead.org

[akpm@linux-foundation.org: fix typo]
[akpm@linux-foundation.org: coding-style fixes]
[sfr@canb.auug.org.au: drm/i915: fix up]
  Link: http://lkml.kernel.org/r/20170816144703.378d4f4d@canb.auug.org.au
Link: http://lkml.kernel.org/r/20170728091904.14627-1-mhocko@kernel.org
Signed-off-by: Michal Hocko 
Signed-off-by: Stephen Rothwell 
Acked-by: Mel Gorman 
Acked-by: Vlastimil Babka 
Cc: Matthew Wilcox 
Cc: Neil Brown 
Cc: "Theodore Ts'o" 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

string_helpers: add kstrdup_quotable_file

2016-04-21T00:47:26+00:00

Allocate a NULL-terminated file path with special characters escaped,
safe for logging.

Signed-off-by: Kees Cook 
Signed-off-by: James Morris

string_helpers: add kstrdup_quotable_cmdline

2016-04-21T00:47:26+00:00

Provide an escaped (but readable: no inter-argument NULLs) commandline
safe for logging.

Signed-off-by: Kees Cook 
Signed-off-by: James Morris

string_helpers: add kstrdup_quotable

2016-04-21T00:47:25+00:00

Handle allocating and escaping a string safe for logging.

Signed-off-by: Kees Cook 
Signed-off-by: James Morris

string_helpers: fix precision loss for some inputs

2016-01-21T01:09:18+00:00

It was noticed that we lose precision in the final calculation for some
inputs.  The most egregious example is size=3000 blk_size=1900 in units
of 10 should yield 5.70 MB but in fact yields 3.00 MB (oops).

This is because the current algorithm doesn't correctly account for
all the remainders in the logarithms.  Fix this by doing a correct
calculation in the remainders based on napier's algorithm.

Additionally, now we have the correct result, we have to account for
arithmetic rounding because we're printing 3 digits of precision.  This
means that if the fourth digit is five or greater, we have to round up,
so add a section to ensure correct rounding.  Finally account for all
possible inputs correctly, including zero for block size.

Fixes: b9f28d863594c429e1df35a0474d2663ca28b307
Signed-off-by: James Bottomley 
Reported-by: Vitaly Kuznetsov 
Cc: 	[delay until after 4.4 release]
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/string_helpers.c: fix infinite loop in string_get_size()

2015-09-18T04:16:07+00:00

Some string_get_size() calls (e.g.:
 string_get_size(1, 512, STRING_UNITS_10, ..., ...)
 string_get_size(15, 64, STRING_UNITS_10, ..., ...)
) result in an infinite loop. The problem is that if size is equal to
divisor[units]/blk_size and is smaller than divisor[units] we'll end
up with size == 0 when we start doing sf_cap calculations:

For string_get_size(1, 512, STRING_UNITS_10, ..., ...) case:
   ...
   remainder = do_div(size, divisor[units]); -> size is 0, remainder is 1
   remainder *= blk_size; -> remainder is 512
   ...
   size *= blk_size; -> size is still 0
   size += remainder / divisor[units]; -> size is still 0

The caller causing the issue is sd_read_capacity(), the problem was
noticed on Hyper-V, such weird size was reported by host when scanning
collides with device removal.  This is probably a separate issue worth
fixing, this patch is intended to prevent the library routine from
infinite looping.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: James Bottomley 
Cc: Andy Shevchenko 
Cc: Rasmus Villemoes 
Cc: "K. Y. Srinivasan" 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/string_helpers: rename "esc" arg to "only"

2015-09-10T20:29:01+00:00

To further clarify the purpose of the "esc" argument, rename it to "only"
to reflect that it is a limit, not a list of additional characters to
escape.

Signed-off-by: Kees Cook 
Suggested-by: Rasmus Villemoes 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/string_helpers: clarify esc arg in string_escape_mem

2015-09-10T20:29:01+00:00

The esc argument is used to reduce which characters will be escaped.  For
example, using " " with ESCAPE_SPACE will not produce any escaped spaces.

Signed-off-by: Kees Cook 
Cc: Andy Shevchenko 
Cc: Rasmus Villemoes 
Cc: Mathias Krause 
Cc: James Bottomley 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

2015-04-16T23:02:04+00:00

Pull SCSI updates from James Bottomley:
"This is the usual grab bag of driver updates (lpfc, qla2xxx, storvsc,
aacraid, ipr) plus an assortment of minor updates. There's also a
major update to aic1542 which moves the driver into this millenium"

* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (106 commits)
change SCSI Maintainer email
sd, mmc, virtio_blk, string_helpers: fix block size units
ufs: add support to allow non standard behaviours (quirks)
ufs-qcom: save controller revision info in internal structure
qla2xxx: Update driver version to 8.07.00.18-k
qla2xxx: Restore physical port WWPN only, when port down detected for FA-WWPN port.
qla2xxx: Fix virtual port configuration, when switch port is disabled/enabled.
qla2xxx: Prevent multiple firmware dump collection for ISP27XX.
qla2xxx: Disable Interrupt handshake for ISP27XX.
qla2xxx: Add debugging info for MBX timeout.
qla2xxx: Add serdes read/write support for ISP27XX
qla2xxx: Add udev notification to save fw dump for ISP27XX
qla2xxx: Add message for sucessful FW dump collected for ISP27XX.
qla2xxx: Add support to load firmware from file for ISP 26XX/27XX.
qla2xxx: Fix beacon blink for ISP27XX.
qla2xxx: Increase the wait time for firmware to be ready for P3P.
qla2xxx: Fix crash due to wrong casting of reg for ISP27XX.
qla2xxx: Fix warnings reported by static checker.
lpfc: Update version to 10.5.0.0 for upstream patch set
lpfc: Update copyright to 2015
...

lib/string_helpers.c: change semantics of string_escape_mem

2015-04-15T23:35:24+00:00

The current semantics of string_escape_mem are inadequate for one of its
current users, vsnprintf().  If that is to honour its contract, it must
know how much space would be needed for the entire escaped buffer, and
string_escape_mem provides no way of obtaining that (short of allocating a
large enough buffer (~4 times input string) to let it play with, and
that's definitely a big no-no inside vsnprintf).

So change the semantics for string_escape_mem to be more snprintf-like:
Return the size of the output that would be generated if the destination
buffer was big enough, but of course still only write to the part of dst
it is allowed to, and (contrary to snprintf) don't do '\0'-termination.
It is then up to the caller to detect whether output was truncated and to
append a '\0' if desired.  Also, we must output partial escape sequences,
otherwise a call such as snprintf(buf, 3, "%1pE", "\123") would cause
printf to write a \0 to buf[2] but leaving buf[0] and buf[1] with whatever
they previously contained.

This also fixes a bug in the escaped_string() helper function, which used
to unconditionally pass a length of "end-buf" to string_escape_mem();
since the latter doesn't check osz for being insanely large, it would
happily write to dst.  For example, kasprintf(GFP_KERNEL, "something and
then %pE", ...); is an easy way to trigger an oops.

In test-string_helpers.c, the -ENOMEM test is replaced with testing for
getting the expected return value even if the buffer is too small.  We
also ensure that nothing is written (by relying on a NULL pointer deref)
if the output size is 0 by passing NULL - this has to work for
kasprintf("%pE") to work.

In net/sunrpc/cache.c, I think qword_add still has the same semantics.
Someone should definitely double-check this.

In fs/proc/array.c, I made the minimum possible change, but longer-term it
should stop poking around in seq_file internals.

[andriy.shevchenko@linux.intel.com: simplify qword_add]
[andriy.shevchenko@linux.intel.com: add missed curly braces]
Signed-off-by: Rasmus Villemoes 
Acked-by: Andy Shevchenko 
Signed-off-by: Andy Shevchenko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds