Btrfs: fix locking bugs when defragging leaves

2015-12-18T02:51:32+00:00

When running fstests btrfs/070, with a higher number of fsstress
operations, I ran frequently into two different locking bugs when
defragging directories.

The first bug produced the following traces:

[133860.229792] ------------[ cut here ]------------
[133860.251062] WARNING: CPU: 2 PID: 26057 at fs/btrfs/locking.c:46 btrfs_set_lock_blocking_rw+0x57/0xbd [btrfs]()
[133860.253576] Modules linked in: btrfs crc32c_generic xor raid6_pq nfsd auth_rpcgss oid_registry nfs_acl nfs lockd grace fscache sunrpc loop fuse parport_pc i2c_piix4 psmouse parport
[133860.282566] CPU: 2 PID: 26057 Comm: btrfs Tainted: G        W       4.3.0-rc5-btrfs-next-17+ #1
[133860.284393] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.1-0-g4adadbd-20150316_085822-nilsson.home.kraxel.org 04/01/2014
[133860.286827]  0000000000000000 ffff880207697b78 ffffffff812566f4 0000000000000000
[133860.288341]  ffff880207697bb0 ffffffff8104d0a6 ffffffffa052d4c1 ffff880178f60e00
[133860.294219]  ffff880178f60e00 0000000000000000 00000000000000f6 ffff880207697bc0
[133860.295831] Call Trace:
[133860.306518]  [] dump_stack+0x4e/0x79
[133860.307473]  [] warn_slowpath_common+0x9f/0xb8
[133860.308619]  [] ? btrfs_set_lock_blocking_rw+0x57/0xbd [btrfs]
[133860.310068]  [] warn_slowpath_null+0x1a/0x1c
[133860.312552]  [] btrfs_set_lock_blocking_rw+0x57/0xbd [btrfs]
[133860.314630]  [] btrfs_set_lock_blocking+0xe/0x10 [btrfs]
[133860.323596]  [] btrfs_realloc_node+0xb3/0x341 [btrfs]
[133860.325233]  [] btrfs_defrag_leaves+0x239/0x2fa [btrfs]
[133860.332427]  [] btrfs_defrag_root+0x63/0xca [btrfs]
[133860.337259]  [] btrfs_ioctl_defrag+0x78/0x14e [btrfs]
[133860.340147]  [] btrfs_ioctl+0x746/0x24c6 [btrfs]
[133860.344833]  [] ? arch_local_irq_save+0x9/0xc
[133860.346343]  [] ? __might_fault+0x4c/0xa7
[133860.353248]  [] ? __might_fault+0x4c/0xa7
[133860.354242]  [] ? __might_fault+0xa5/0xa7
[133860.355232]  [] ? cp_new_stat+0x15d/0x174
[133860.356237]  [] do_vfs_ioctl+0x427/0x4e6
[133860.358587]  [] ? SYSC_newfstat+0x25/0x2e
[133860.360195]  [] ? __fget_light+0x4d/0x71
[133860.361380]  [] SyS_ioctl+0x57/0x79
[133860.363578]  [] entry_SYSCALL_64_fastpath+0x12/0x6f
[133860.366217] ---[ end trace 2cadb2f653437e49 ]---
[133860.367399] ------------[ cut here ]------------
[133860.368162] kernel BUG at fs/btrfs/locking.c:307!
[133860.369430] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[133860.370205] Modules linked in: btrfs crc32c_generic xor raid6_pq nfsd auth_rpcgss oid_registry nfs_acl nfs lockd grace fscache sunrpc loop fuse parport_pc i2c_piix4 psmouse parport
[133860.370205] CPU: 2 PID: 26057 Comm: btrfs Tainted: G        W       4.3.0-rc5-btrfs-next-17+ #1
[133860.370205] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.1-0-g4adadbd-20150316_085822-nilsson.home.kraxel.org 04/01/2014
[133860.370205] task: ffff8800aec6db40 ti: ffff880207694000 task.ti: ffff880207694000
[133860.370205] RIP: 0010:[]  [] btrfs_assert_tree_locked+0x10/0x14 [btrfs]
[133860.370205] RSP: 0018:ffff880207697bc0  EFLAGS: 00010246
[133860.370205] RAX: 0000000000000000 RBX: ffff880178f60e00 RCX: 0000000000000000
[133860.370205] RDX: ffff88023ec4fb50 RSI: 00000000ffffffff RDI: ffff880178f60e00
[133860.370205] RBP: ffff880207697bc0 R08: 0000000000000001 R09: 0000000000000000
[133860.370205] R10: 0000160000000000 R11: ffffffff81651000 R12: ffff880178f60e00
[133860.370205] R13: 0000000000000000 R14: 00000000000000f6 R15: ffff8801ff409000
[133860.370205] FS:  00007f763efd48c0(0000) GS:ffff88023ec40000(0000) knlGS:0000000000000000
[133860.370205] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[133860.370205] CR2: 0000000002158048 CR3: 000000003fd6c000 CR4: 00000000000006e0
[133860.370205] Stack:
[133860.370205]  ffff880207697bd8 ffffffffa052d4d0 0000000000000000 ffff880207697be8
[133860.370205]  ffffffffa04d5787 ffff880207697c80 ffffffffa04d99cb ffff8801ff409590
[133860.370205]  ffff880207697ca8 000000f507697c80 ffff880183c11bb8 0000000000000000
[133860.370205] Call Trace:
[133860.370205]  [] btrfs_set_lock_blocking_rw+0x66/0xbd [btrfs]
[133860.370205]  [] btrfs_set_lock_blocking+0xe/0x10 [btrfs]
[133860.370205]  [] btrfs_realloc_node+0xb3/0x341 [btrfs]
[133860.370205]  [] btrfs_defrag_leaves+0x239/0x2fa [btrfs]
[133860.370205]  [] btrfs_defrag_root+0x63/0xca [btrfs]
[133860.370205]  [] btrfs_ioctl_defrag+0x78/0x14e [btrfs]
[133860.370205]  [] btrfs_ioctl+0x746/0x24c6 [btrfs]
[133860.370205]  [] ? arch_local_irq_save+0x9/0xc
[133860.370205]  [] ? __might_fault+0x4c/0xa7
[133860.370205]  [] ? __might_fault+0x4c/0xa7
[133860.370205]  [] ? __might_fault+0xa5/0xa7
[133860.370205]  [] ? cp_new_stat+0x15d/0x174
[133860.370205]  [] do_vfs_ioctl+0x427/0x4e6
[133860.370205]  [] ? SYSC_newfstat+0x25/0x2e
[133860.370205]  [] ? __fget_light+0x4d/0x71
[133860.370205]  [] SyS_ioctl+0x57/0x79
[133860.370205]  [] entry_SYSCALL_64_fastpath+0x12/0x6f

This bug happened because we assumed that by setting keep_locks to 1 in
our search path, our path after a call to btrfs_search_slot() would have
all nodes locked, which is not always true because unlock_up() (called by
btrfs_search_slot()) will unlock a node in a path if the slot of the node
below it doesn't point to the last item or beyond the last item. For
example, when the tree has a heigth of 2 and path->slots[0] has a value
smaller than btrfs_header_nritems(path->nodes[0]) - 1, the node at level 2
will be unlocked (also because lowest_unlock is set to 1 due to the fact
that the value passed as ins_len to btrfs_search_slot is 0).
This resulted in btrfs_find_next_key(), called before btrfs_realloc_node(),
to release out path and call again btrfs_search_slot(), but this time with
the cow parameter set to 0, meaning the resulting path got only read locks.
Therefore when we called btrfs_realloc_node(), with path->nodes[1] having
a read lock, it resulted in the warning and BUG_ON when calling
btrfs_set_lock_blocking() against the node, as that function expects the
node to have a write lock.

The second bug happened often when the first bug didn't happen, and made
us hang and hitting the following warning at fs/btrfs/locking.c:

   251  void btrfs_tree_lock(struct extent_buffer *eb)
   252  {
   253          WARN_ON(eb->lock_owner == current->pid);

This happened because the tree search we made at btrfs_defrag_leaves()
before calling btrfs_find_next_key() locked a leaf and all the other
nodes in the path, so btrfs_find_next_key() had no need to release the
path and make a new search (with path->lowest_level set to 1). This
made btrfs_realloc_node() attempt to write lock the same leaf again,
resulting in a hang/deadlock.

So fix these issues by calling btrfs_find_next_key() after calling
btrfs_realloc_node() and setting the search path's lowest_level to 1
to avoid the hang/deadlock when attempting to write lock the leaves
at btrfs_realloc_node().

Signed-off-by: Filipe Manana

Btrfs: cleanup: remove unnecessary check before btrfs_free_path is called

2015-08-31T18:46:41+00:00

We need not check path before btrfs_free_path() is called because
path is checked in btrfs_free_path().

Signed-off-by: Tsutomu Itoh 
Reviewed-by: Qu Wenruo 
Signed-off-by: Chris Mason

btrfs: let tree defrag work in SSD mode

2015-06-03T02:34:33+00:00

Long time ago (2008) the defrag was automatic for new b-tree writes but
has been disabled after performance problems. There was a leftover in
tree-defrag.c that effectively stops any defragmentation on b-trees.
This is a bit unexpected and IMHO undesired. The SSD mode is an
optimization and defrag is supposed to work if the users asks for it.

Related commits:

6702ed490ca0bb44e17131818a5a18b773957c5a
Btrfs: Add run time btree defrag, and an ioctl to force btree defrag

e18e4809b10e6c9efb5fe10c1ddcb4ebb690d517
Btrfs: Add mount -o ssd, which includes optimizations for seek free
storage

b3236e68bf86b3ae87f58984a1822369225211cb
Btrfs: Leave on the tree defragger in mount -o ssd, it still helps there

9afbb0b752ef30a429c45b9de6706e28ad1a36e1
Btrfs: Disable tree defrag in SSD mode

The last three commits switch the defrag+ssd off/on/off and the last one

3f157a2fd2ad731e1ed9964fecdc5f459f04a4a4
Btrfs: Online btree defragmentation fixes

misses the bits from tree-defrag.c to revert to the behaviour introduced
in e18e4809b10e.

Signed-off-by: David Sterba 
Signed-off-by: Chris Mason

Btrfs: use bitfield instead of integer data type for the some variants in btrfs_root

2014-06-10T00:20:40+00:00

Signed-off-by: Miao Xie 
Signed-off-by: Wang Shilong 
Signed-off-by: Chris Mason

Btrfs: cleanup dead code of defragment

2013-11-12T02:59:45+00:00

@is_extent is no more needed since we don't defrag extent root.

Signed-off-by: Liu Bo 
Signed-off-by: Josef Bacik 
Signed-off-by: Chris Mason

Btrfs: remove unused max_key arg from btrfs_search_forward

2013-11-12T02:54:57+00:00

It is not used for anything.

Signed-off-by: Filipe David Borba Manana 
Signed-off-by: Josef Bacik 
Signed-off-by: Chris Mason

btrfs: remove cache only arguments from defrag path

2013-02-20T17:59:36+00:00

The entry point at the defrag ioctl always sets "cache only" to 0;
the codepaths haven't run for a long time as far as I can
tell.  Chris says they're dead code, so remove them.

Signed-off-by: Eric Sandeen 
Signed-off-by: Josef Bacik

btrfs: drop unused parameter from btrfs_release_path

2011-05-02T11:57:22+00:00

parameter tree root it's not used since commit
5f39d397dfbe140a14edecd4e73c34ce23c4f9ee ("Btrfs: Create extent_buffer
interface for large blocksizes")

Signed-off-by: David Sterba

Btrfs: cleanup warnings from gcc 4.6 (nonbugs)

2010-10-29T19:14:37+00:00

These are all the cases where a variable is set, but not read which are
not bugs as far as I can see, but simply leftovers.

Still needs more review.

Found by gcc 4.6's new warnings

Signed-off-by: Andi Kleen 
Cc: Chris Mason 
Signed-off-by: Andrew Morton 
Signed-off-by: Chris Mason

Btrfs: Introduce global metadata reservation

2010-05-25T14:34:52+00:00

Reserve metadata space for extent tree, checksum tree and root tree

Signed-off-by: Yan Zheng 
Signed-off-by: Chris Mason

linux-toradex.git/fs/btrfs/tree-defrag.c, branch v4.9.21