From 68ca406930d6380b3be7ada5f15fcf85bfcbd552 Mon Sep 17 00:00:00 2001 From: Len Brown Date: Fri, 19 Feb 2010 00:09:22 -0500 Subject: ACPI: delete the "acpi=ht" boot option acpi=ht was important in 2003 -- before ACPI was universally deployed and enabled by default in the major Linux distributions. At that time, there were a fair number of people who or chose to, or needed to, run with acpi=off, yet also wanted access to Hyper-threading. Today we find that many invocations of "acpi=ht" are accidental, and thus is it possible that it is doing more harm than good. In 2.6.34, we warn on invocation of acpi=ht. In 2.6.35, we delete the boot option. Signed-off-by: Len Brown --- Documentation/kernel-parameters.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 3bc48b0bd3a9..41b924ff0b51 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -143,11 +143,10 @@ and is between 256 and 4096 characters. It is defined in the file acpi= [HW,ACPI,X86] Advanced Configuration and Power Interface - Format: { force | off | ht | strict | noirq | rsdt } + Format: { force | off | strict | noirq | rsdt } force -- enable ACPI if default was off off -- disable ACPI if default was on noirq -- do not use ACPI for IRQ routing - ht -- run only enough ACPI to enable Hyper Threading strict -- Be less tolerant of platforms that are not strictly ACPI specification compliant. rsdt -- prefer RSDT over (default) XSDT -- cgit v1.2.3 From ea8c071cad789b1919355fc7a67182a5c9994e6b Mon Sep 17 00:00:00 2001 From: Huang Ying Date: Tue, 18 May 2010 14:35:15 +0800 Subject: ACPI, APEI, Document for APEI Add document for APEI, including kernel parameters and EINJ debug file sytem interface. Signed-off-by: Huang Ying Signed-off-by: Andi Kleen Signed-off-by: Len Brown --- Documentation/acpi/apei/einj.txt | 49 +++++++++++++++++++++++++++++++++++++ Documentation/kernel-parameters.txt | 5 ++++ 2 files changed, 54 insertions(+) create mode 100644 Documentation/acpi/apei/einj.txt (limited to 'Documentation') diff --git a/Documentation/acpi/apei/einj.txt b/Documentation/acpi/apei/einj.txt new file mode 100644 index 000000000000..838b7f0d5e11 --- /dev/null +++ b/Documentation/acpi/apei/einj.txt @@ -0,0 +1,49 @@ + APEI Error INJection + ~~~~~~~~~~~~~~~~~~~~ + +EINJ provides a hardware error injection mechanism +It is very useful for debugging and testing of other APEI and RAS features. + +To use EINJ, make sure the following are enabled in your kernel +configuration: + +CONFIG_DEBUG_FS +CONFIG_ACPI_APEI +CONFIG_ACPI_APEI_EINJ + +The user interface of EINJ is debug file system, under the +directory apei/einj. The following files are provided. + +- available_error_type + Reading this file returns the error injection capability of the + platform, that is, which error types are supported. The error type + definition is as follow, the left field is the error type value, the + right field is error description. + + 0x00000001 Processor Correctable + 0x00000002 Processor Uncorrectable non-fatal + 0x00000004 Processor Uncorrectable fatal + 0x00000008 Memory Correctable + 0x00000010 Memory Uncorrectable non-fatal + 0x00000020 Memory Uncorrectable fatal + 0x00000040 PCI Express Correctable + 0x00000080 PCI Express Uncorrectable fatal + 0x00000100 PCI Express Uncorrectable non-fatal + 0x00000200 Platform Correctable + 0x00000400 Platform Uncorrectable non-fatal + 0x00000800 Platform Uncorrectable fatal + + The format of file contents are as above, except there are only the + available error type lines. + +- error_type + This file is used to set the error type value. The error type value + is defined in "available_error_type" description. + +- error_inject + Write any integer to this file to trigger the error + injection. Before this, please specify all necessary error + parameters. + +For more information about EINJ, please refer to ACPI specification +version 4.0, section 17.5. diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 839b21b0699a..47a0e1ba5e64 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -843,6 +843,11 @@ and is between 256 and 4096 characters. It is defined in the file hd= [EIDE] (E)IDE hard drive subsystem geometry Format: ,, + hest_disable [ACPI] + Disable Hardware Error Source Table (HEST) support; + corresponding firmware-first mode error processing + logic will be disabled. + highmem=nn[KMG] [KNL,BOOT] forces the highmem zone to have an exact size of . This works even on boxes that have no highmem otherwise. This also works to reduce highmem -- cgit v1.2.3 From a08f82d08053fb6e3aa3635c2c26456d96337c8b Mon Sep 17 00:00:00 2001 From: Huang Ying Date: Tue, 18 May 2010 14:35:21 +0800 Subject: ACPI, APEI, Error Record Serialization Table (ERST) support ERST is a way provided by APEI to save and retrieve hardware error record to and from some simple persistent storage (such as flash). The Linux kernel support implementation is quite simple and workable in NMI context. So it can be used to save hardware error record into flash in hardware error exception or NMI handler, where other more complex persistent storage such as disk is not usable. After saving hardware error records via ERST in hardware error exception or NMI handler, the error records can be retrieved and logged into disk or network after a clean reboot. For more information about ERST, please refer to ACPI Specification version 4.0, section 17.4. This patch incorporate fixes from Jin Dongming. Signed-off-by: Huang Ying Signed-off-by: Andi Kleen CC: Jin Dongming Signed-off-by: Len Brown --- Documentation/kernel-parameters.txt | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'Documentation') diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 47a0e1ba5e64..42d77735fd45 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -750,6 +750,10 @@ and is between 256 and 4096 characters. It is defined in the file Default value is 0. Value can be changed at runtime via /selinux/enforce. + erst_disable [ACPI] + Disable Error Record Serialization Table (ERST) + support. + ether= [HW,NET] Ethernet cards parameters This option is obsoleted by the "netdev=" option, which has equivalent usage. See its documentation for details. -- cgit v1.2.3 From 6e320ec1d98f9eb93d5b2a5d70e2f40dce923f1b Mon Sep 17 00:00:00 2001 From: Huang Ying Date: Tue, 18 May 2010 14:35:24 +0800 Subject: ACPI, APEI, EINJ injection parameters support Some hardware error injection needs parameters, for example, it is useful to specify memory address and memory address mask for memory errors. Some BIOSes allow parameters to be specified via an unpublished extension. This patch adds support to it. The parameters will be ignored on machines without necessary BIOS support. Signed-off-by: Huang Ying Signed-off-by: Andi Kleen Signed-off-by: Len Brown --- Documentation/acpi/apei/einj.txt | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'Documentation') diff --git a/Documentation/acpi/apei/einj.txt b/Documentation/acpi/apei/einj.txt index 838b7f0d5e11..dfab71848dc8 100644 --- a/Documentation/acpi/apei/einj.txt +++ b/Documentation/acpi/apei/einj.txt @@ -45,5 +45,15 @@ directory apei/einj. The following files are provided. injection. Before this, please specify all necessary error parameters. +- param1 + This file is used to set the first error parameter value. Effect of + parameter depends on error_type specified. For memory error, this is + physical memory address. + +- param2 + This file is used to set the second error parameter value. Effect of + parameter depends on error_type specified. For memory error, this is + physical memory address mask. + For more information about EINJ, please refer to ACPI specification version 4.0, section 17.5. -- cgit v1.2.3 From a9a745daadab26f13884ff26a50fa38247c11ce9 Mon Sep 17 00:00:00 2001 From: Dave Chinner Date: Fri, 14 May 2010 21:43:11 +1000 Subject: xfs: Delayed logging design documentation Document the design of the delayed logging implementation. This includes assumptions made, dead ends followed, the reasoning behind the structuring of the code, the layout of various structures, how things fit together, traps and pit-falls avoided, etc. This is all too much to document in the code itself, so do it in a separate file. Signed-off-by: Dave Chinner Signed-off-by: Alex Elder --- .../filesystems/xfs-delayed-logging-design.txt | 816 +++++++++++++++++++++ 1 file changed, 816 insertions(+) create mode 100644 Documentation/filesystems/xfs-delayed-logging-design.txt (limited to 'Documentation') diff --git a/Documentation/filesystems/xfs-delayed-logging-design.txt b/Documentation/filesystems/xfs-delayed-logging-design.txt new file mode 100644 index 000000000000..d8119e9d2d60 --- /dev/null +++ b/Documentation/filesystems/xfs-delayed-logging-design.txt @@ -0,0 +1,816 @@ +XFS Delayed Logging Design +-------------------------- + +Introduction to Re-logging in XFS +--------------------------------- + +XFS logging is a combination of logical and physical logging. Some objects, +such as inodes and dquots, are logged in logical format where the details +logged are made up of the changes to in-core structures rather than on-disk +structures. Other objects - typically buffers - have their physical changes +logged. The reason for these differences is to reduce the amount of log space +required for objects that are frequently logged. Some parts of inodes are more +frequently logged than others, and inodes are typically more frequently logged +than any other object (except maybe the superblock buffer) so keeping the +amount of metadata logged low is of prime importance. + +The reason that this is such a concern is that XFS allows multiple separate +modifications to a single object to be carried in the log at any given time. +This allows the log to avoid needing to flush each change to disk before +recording a new change to the object. XFS does this via a method called +"re-logging". Conceptually, this is quite simple - all it requires is that any +new change to the object is recorded with a *new copy* of all the existing +changes in the new transaction that is written to the log. + +That is, if we have a sequence of changes A through to F, and the object was +written to disk after change D, we would see in the log the following series +of transactions, their contents and the log sequence number (LSN) of the +transaction: + + Transaction Contents LSN + A A X + B A+B X+n + C A+B+C X+n+m + D A+B+C+D X+n+m+o + + E E Y (> X+n+m+o) + F E+F Yٍ+p + +In other words, each time an object is relogged, the new transaction contains +the aggregation of all the previous changes currently held only in the log. + +This relogging technique also allows objects to be moved forward in the log so +that an object being relogged does not prevent the tail of the log from ever +moving forward. This can be seen in the table above by the changing +(increasing) LSN of each subsquent transaction - the LSN is effectively a +direct encoding of the location in the log of the transaction. + +This relogging is also used to implement long-running, multiple-commit +transactions. These transaction are known as rolling transactions, and require +a special log reservation known as a permanent transaction reservation. A +typical example of a rolling transaction is the removal of extents from an +inode which can only be done at a rate of two extents per transaction because +of reservation size limitations. Hence a rolling extent removal transaction +keeps relogging the inode and btree buffers as they get modified in each +removal operation. This keeps them moving forward in the log as the operation +progresses, ensuring that current operation never gets blocked by itself if the +log wraps around. + +Hence it can be seen that the relogging operation is fundamental to the correct +working of the XFS journalling subsystem. From the above description, most +people should be able to see why the XFS metadata operations writes so much to +the log - repeated operations to the same objects write the same changes to +the log over and over again. Worse is the fact that objects tend to get +dirtier as they get relogged, so each subsequent transaction is writing more +metadata into the log. + +Another feature of the XFS transaction subsystem is that most transactions are +asynchronous. That is, they don't commit to disk until either a log buffer is +filled (a log buffer can hold multiple transactions) or a synchronous operation +forces the log buffers holding the transactions to disk. This means that XFS is +doing aggregation of transactions in memory - batching them, if you like - to +minimise the impact of the log IO on transaction throughput. + +The limitation on asynchronous transaction throughput is the number and size of +log buffers made available by the log manager. By default there are 8 log +buffers available and the size of each is 32kB - the size can be increased up +to 256kB by use of a mount option. + +Effectively, this gives us the maximum bound of outstanding metadata changes +that can be made to the filesystem at any point in time - if all the log +buffers are full and under IO, then no more transactions can be committed until +the current batch completes. It is now common for a single current CPU core to +be to able to issue enough transactions to keep the log buffers full and under +IO permanently. Hence the XFS journalling subsystem can be considered to be IO +bound. + +Delayed Logging: Concepts +------------------------- + +The key thing to note about the asynchronous logging combined with the +relogging technique XFS uses is that we can be relogging changed objects +multiple times before they are committed to disk in the log buffers. If we +return to the previous relogging example, it is entirely possible that +transactions A through D are committed to disk in the same log buffer. + +That is, a single log buffer may contain multiple copies of the same object, +but only one of those copies needs to be there - the last one "D", as it +contains all the changes from the previous changes. In other words, we have one +necessary copy in the log buffer, and three stale copies that are simply +wasting space. When we are doing repeated operations on the same set of +objects, these "stale objects" can be over 90% of the space used in the log +buffers. It is clear that reducing the number of stale objects written to the +log would greatly reduce the amount of metadata we write to the log, and this +is the fundamental goal of delayed logging. + +From a conceptual point of view, XFS is already doing relogging in memory (where +memory == log buffer), only it is doing it extremely inefficiently. It is using +logical to physical formatting to do the relogging because there is no +infrastructure to keep track of logical changes in memory prior to physically +formatting the changes in a transaction to the log buffer. Hence we cannot avoid +accumulating stale objects in the log buffers. + +Delayed logging is the name we've given to keeping and tracking transactional +changes to objects in memory outside the log buffer infrastructure. Because of +the relogging concept fundamental to the XFS journalling subsystem, this is +actually relatively easy to do - all the changes to logged items are already +tracked in the current infrastructure. The big problem is how to accumulate +them and get them to the log in a consistent, recoverable manner. +Describing the problems and how they have been solved is the focus of this +document. + +One of the key changes that delayed logging makes to the operation of the +journalling subsystem is that it disassociates the amount of outstanding +metadata changes from the size and number of log buffers available. In other +words, instead of there only being a maximum of 2MB of transaction changes not +written to the log at any point in time, there may be a much greater amount +being accumulated in memory. Hence the potential for loss of metadata on a +crash is much greater than for the existing logging mechanism. + +It should be noted that this does not change the guarantee that log recovery +will result in a consistent filesystem. What it does mean is that as far as the +recovered filesystem is concerned, there may be many thousands of transactions +that simply did not occur as a result of the crash. This makes it even more +important that applications that care about their data use fsync() where they +need to ensure application level data integrity is maintained. + +It should be noted that delayed logging is not an innovative new concept that +warrants rigorous proofs to determine whether it is correct or not. The method +of accumulating changes in memory for some period before writing them to the +log is used effectively in many filesystems including ext3 and ext4. Hence +no time is spent in this document trying to convince the reader that the +concept is sound. Instead it is simply considered a "solved problem" and as +such implementing it in XFS is purely an exercise in software engineering. + +The fundamental requirements for delayed logging in XFS are simple: + + 1. Reduce the amount of metadata written to the log by at least + an order of magnitude. + 2. Supply sufficient statistics to validate Requirement #1. + 3. Supply sufficient new tracing infrastructure to be able to debug + problems with the new code. + 4. No on-disk format change (metadata or log format). + 5. Enable and disable with a mount option. + 6. No performance regressions for synchronous transaction workloads. + +Delayed Logging: Design +----------------------- + +Storing Changes + +The problem with accumulating changes at a logical level (i.e. just using the +existing log item dirty region tracking) is that when it comes to writing the +changes to the log buffers, we need to ensure that the object we are formatting +is not changing while we do this. This requires locking the object to prevent +concurrent modification. Hence flushing the logical changes to the log would +require us to lock every object, format them, and then unlock them again. + +This introduces lots of scope for deadlocks with transactions that are already +running. For example, a transaction has object A locked and modified, but needs +the delayed logging tracking lock to commit the transaction. However, the +flushing thread has the delayed logging tracking lock already held, and is +trying to get the lock on object A to flush it to the log buffer. This appears +to be an unsolvable deadlock condition, and it was solving this problem that +was the barrier to implementing delayed logging for so long. + +The solution is relatively simple - it just took a long time to recognise it. +Put simply, the current logging code formats the changes to each item into an +vector array that points to the changed regions in the item. The log write code +simply copies the memory these vectors point to into the log buffer during +transaction commit while the item is locked in the transaction. Instead of +using the log buffer as the destination of the formatting code, we can use an +allocated memory buffer big enough to fit the formatted vector. + +If we then copy the vector into the memory buffer and rewrite the vector to +point to the memory buffer rather than the object itself, we now have a copy of +the changes in a format that is compatible with the log buffer writing code. +that does not require us to lock the item to access. This formatting and +rewriting can all be done while the object is locked during transaction commit, +resulting in a vector that is transactionally consistent and can be accessed +without needing to lock the owning item. + +Hence we avoid the need to lock items when we need to flush outstanding +asynchronous transactions to the log. The differences between the existing +formatting method and the delayed logging formatting can be seen in the +diagram below. + +Current format log vector: + +Object +---------------------------------------------+ +Vector 1 +----+ +Vector 2 +----+ +Vector 3 +----------+ + +After formatting: + +Log Buffer +-V1-+-V2-+----V3----+ + +Delayed logging vector: + +Object +---------------------------------------------+ +Vector 1 +----+ +Vector 2 +----+ +Vector 3 +----------+ + +After formatting: + +Memory Buffer +-V1-+-V2-+----V3----+ +Vector 1 +----+ +Vector 2 +----+ +Vector 3 +----------+ + +The memory buffer and associated vector need to be passed as a single object, +but still need to be associated with the parent object so if the object is +relogged we can replace the current memory buffer with a new memory buffer that +contains the latest changes. + +The reason for keeping the vector around after we've formatted the memory +buffer is to support splitting vectors across log buffer boundaries correctly. +If we don't keep the vector around, we do not know where the region boundaries +are in the item, so we'd need a new encapsulation method for regions in the log +buffer writing (i.e. double encapsulation). This would be an on-disk format +change and as such is not desirable. It also means we'd have to write the log +region headers in the formatting stage, which is problematic as there is per +region state that needs to be placed into the headers during the log write. + +Hence we need to keep the vector, but by attaching the memory buffer to it and +rewriting the vector addresses to point at the memory buffer we end up with a +self-describing object that can be passed to the log buffer write code to be +handled in exactly the same manner as the existing log vectors are handled. +Hence we avoid needing a new on-disk format to handle items that have been +relogged in memory. + + +Tracking Changes + +Now that we can record transactional changes in memory in a form that allows +them to be used without limitations, we need to be able to track and accumulate +them so that they can be written to the log at some later point in time. The +log item is the natural place to store this vector and buffer, and also makes sense +to be the object that is used to track committed objects as it will always +exist once the object has been included in a transaction. + +The log item is already used to track the log items that have been written to +the log but not yet written to disk. Such log items are considered "active" +and as such are stored in the Active Item List (AIL) which is a LSN-ordered +double linked list. Items are inserted into this list during log buffer IO +completion, after which they are unpinned and can be written to disk. An object +that is in the AIL can be relogged, which causes the object to be pinned again +and then moved forward in the AIL when the log buffer IO completes for that +transaction. + +Essentially, this shows that an item that is in the AIL can still be modified +and relogged, so any tracking must be separate to the AIL infrastructure. As +such, we cannot reuse the AIL list pointers for tracking committed items, nor +can we store state in any field that is protected by the AIL lock. Hence the +committed item tracking needs it's own locks, lists and state fields in the log +item. + +Similar to the AIL, tracking of committed items is done through a new list +called the Committed Item List (CIL). The list tracks log items that have been +committed and have formatted memory buffers attached to them. It tracks objects +in transaction commit order, so when an object is relogged it is removed from +it's place in the list and re-inserted at the tail. This is entirely arbitrary +and done to make it easy for debugging - the last items in the list are the +ones that are most recently modified. Ordering of the CIL is not necessary for +transactional integrity (as discussed in the next section) so the ordering is +done for convenience/sanity of the developers. + + +Delayed Logging: Checkpoints + +When we have a log synchronisation event, commonly known as a "log force", +all the items in the CIL must be written into the log via the log buffers. +We need to write these items in the order that they exist in the CIL, and they +need to be written as an atomic transaction. The need for all the objects to be +written as an atomic transaction comes from the requirements of relogging and +log replay - all the changes in all the objects in a given transaction must +either be completely replayed during log recovery, or not replayed at all. If +a transaction is not replayed because it is not complete in the log, then +no later transactions should be replayed, either. + +To fulfill this requirement, we need to write the entire CIL in a single log +transaction. Fortunately, the XFS log code has no fixed limit on the size of a +transaction, nor does the log replay code. The only fundamental limit is that +the transaction cannot be larger than just under half the size of the log. The +reason for this limit is that to find the head and tail of the log, there must +be at least one complete transaction in the log at any given time. If a +transaction is larger than half the log, then there is the possibility that a +crash during the write of a such a transaction could partially overwrite the +only complete previous transaction in the log. This will result in a recovery +failure and an inconsistent filesystem and hence we must enforce the maximum +size of a checkpoint to be slightly less than a half the log. + +Apart from this size requirement, a checkpoint transaction looks no different +to any other transaction - it contains a transaction header, a series of +formatted log items and a commit record at the tail. From a recovery +perspective, the checkpoint transaction is also no different - just a lot +bigger with a lot more items in it. The worst case effect of this is that we +might need to tune the recovery transaction object hash size. + +Because the checkpoint is just another transaction and all the changes to log +items are stored as log vectors, we can use the existing log buffer writing +code to write the changes into the log. To do this efficiently, we need to +minimise the time we hold the CIL locked while writing the checkpoint +transaction. The current log write code enables us to do this easily with the +way it separates the writing of the transaction contents (the log vectors) from +the transaction commit record, but tracking this requires us to have a +per-checkpoint context that travels through the log write process through to +checkpoint completion. + +Hence a checkpoint has a context that tracks the state of the current +checkpoint from initiation to checkpoint completion. A new context is initiated +at the same time a checkpoint transaction is started. That is, when we remove +all the current items from the CIL during a checkpoint operation, we move all +those changes into the current checkpoint context. We then initialise a new +context and attach that to the CIL for aggregation of new transactions. + +This allows us to unlock the CIL immediately after transfer of all the +committed items and effectively allow new transactions to be issued while we +are formatting the checkpoint into the log. It also allows concurrent +checkpoints to be written into the log buffers in the case of log force heavy +workloads, just like the existing transaction commit code does. This, however, +requires that we strictly order the commit records in the log so that +checkpoint sequence order is maintained during log replay. + +To ensure that we can be writing an item into a checkpoint transaction at +the same time another transaction modifies the item and inserts the log item +into the new CIL, then checkpoint transaction commit code cannot use log items +to store the list of log vectors that need to be written into the transaction. +Hence log vectors need to be able to be chained together to allow them to be +detatched from the log items. That is, when the CIL is flushed the memory +buffer and log vector attached to each log item needs to be attached to the +checkpoint context so that the log item can be released. In diagrammatic form, +the CIL would look like this before the flush: + + CIL Head + | + V + Log Item <-> log vector 1 -> memory buffer + | -> vector array + V + Log Item <-> log vector 2 -> memory buffer + | -> vector array + V + ...... + | + V + Log Item <-> log vector N-1 -> memory buffer + | -> vector array + V + Log Item <-> log vector N -> memory buffer + -> vector array + +And after the flush the CIL head is empty, and the checkpoint context log +vector list would look like: + + Checkpoint Context + | + V + log vector 1 -> memory buffer + | -> vector array + | -> Log Item + V + log vector 2 -> memory buffer + | -> vector array + | -> Log Item + V + ...... + | + V + log vector N-1 -> memory buffer + | -> vector array + | -> Log Item + V + log vector N -> memory buffer + -> vector array + -> Log Item + +Once this transfer is done, the CIL can be unlocked and new transactions can +start, while the checkpoint flush code works over the log vector chain to +commit the checkpoint. + +Once the checkpoint is written into the log buffers, the checkpoint context is +attached to the log buffer that the commit record was written to along with a +completion callback. Log IO completion will call that callback, which can then +run transaction committed processing for the log items (i.e. insert into AIL +and unpin) in the log vector chain and then free the log vector chain and +checkpoint context. + +Discussion Point: I am uncertain as to whether the log item is the most +efficient way to track vectors, even though it seems like the natural way to do +it. The fact that we walk the log items (in the CIL) just to chain the log +vectors and break the link between the log item and the log vector means that +we take a cache line hit for the log item list modification, then another for +the log vector chaining. If we track by the log vectors, then we only need to +break the link between the log item and the log vector, which means we should +dirty only the log item cachelines. Normally I wouldn't be concerned about one +vs two dirty cachelines except for the fact I've seen upwards of 80,000 log +vectors in one checkpoint transaction. I'd guess this is a "measure and +compare" situation that can be done after a working and reviewed implementation +is in the dev tree.... + +Delayed Logging: Checkpoint Sequencing + +One of the key aspects of the XFS transaction subsystem is that it tags +committed transactions with the log sequence number of the transaction commit. +This allows transactions to be issued asynchronously even though there may be +future operations that cannot be completed until that transaction is fully +committed to the log. In the rare case that a dependent operation occurs (e.g. +re-using a freed metadata extent for a data extent), a special, optimised log +force can be issued to force the dependent transaction to disk immediately. + +To do this, transactions need to record the LSN of the commit record of the +transaction. This LSN comes directly from the log buffer the transaction is +written into. While this works just fine for the existing transaction +mechanism, it does not work for delayed logging because transactions are not +written directly into the log buffers. Hence some other method of sequencing +transactions is required. + +As discussed in the checkpoint section, delayed logging uses per-checkpoint +contexts, and as such it is simple to assign a sequence number to each +checkpoint. Because the switching of checkpoint contexts must be done +atomically, it is simple to ensure that each new context has a monotonically +increasing sequence number assigned to it without the need for an external +atomic counter - we can just take the current context sequence number and add +one to it for the new context. + +Then, instead of assigning a log buffer LSN to the transaction commit LSN +during the commit, we can assign the current checkpoint sequence. This allows +operations that track transactions that have not yet completed know what +checkpoint sequence needs to be committed before they can continue. As a +result, the code that forces the log to a specific LSN now needs to ensure that +the log forces to a specific checkpoint. + +To ensure that we can do this, we need to track all the checkpoint contexts +that are currently committing to the log. When we flush a checkpoint, the +context gets added to a "committing" list which can be searched. When a +checkpoint commit completes, it is removed from the committing list. Because +the checkpoint context records the LSN of the commit record for the checkpoint, +we can also wait on the log buffer that contains the commit record, thereby +using the existing log force mechanisms to execute synchronous forces. + +It should be noted that the synchronous forces may need to be extended with +mitigation algorithms similar to the current log buffer code to allow +aggregation of multiple synchronous transactions if there are already +synchronous transactions being flushed. Investigation of the performance of the +current design is needed before making any decisions here. + +The main concern with log forces is to ensure that all the previous checkpoints +are also committed to disk before the one we need to wait for. Therefore we +need to check that all the prior contexts in the committing list are also +complete before waiting on the one we need to complete. We do this +synchronisation in the log force code so that we don't need to wait anywhere +else for such serialisation - it only matters when we do a log force. + +The only remaining complexity is that a log force now also has to handle the +case where the forcing sequence number is the same as the current context. That +is, we need to flush the CIL and potentially wait for it to complete. This is a +simple addition to the existing log forcing code to check the sequence numbers +and push if required. Indeed, placing the current sequence checkpoint flush in +the log force code enables the current mechanism for issuing synchronous +transactions to remain untouched (i.e. commit an asynchronous transaction, then +force the log at the LSN of that transaction) and so the higher level code +behaves the same regardless of whether delayed logging is being used or not. + +Delayed Logging: Checkpoint Log Space Accounting + +The big issue for a checkpoint transaction is the log space reservation for the +transaction. We don't know how big a checkpoint transaction is going to be +ahead of time, nor how many log buffers it will take to write out, nor the +number of split log vector regions are going to be used. We can track the +amount of log space required as we add items to the commit item list, but we +still need to reserve the space in the log for the checkpoint. + +A typical transaction reserves enough space in the log for the worst case space +usage of the transaction. The reservation accounts for log record headers, +transaction and region headers, headers for split regions, buffer tail padding, +etc. as well as the actual space for all the changed metadata in the +transaction. While some of this is fixed overhead, much of it is dependent on +the size of the transaction and the number of regions being logged (the number +of log vectors in the transaction). + +An example of the differences would be logging directory changes versus logging +inode changes. If you modify lots of inode cores (e.g. chmod -R g+w *), then +there are lots of transactions that only contain an inode core and an inode log +format structure. That is, two vectors totaling roughly 150 bytes. If we modify +10,000 inodes, we have about 1.5MB of metadata to write in 20,000 vectors. Each +vector is 12 bytes, so the total to be logged is approximately 1.75MB. In +comparison, if we are logging full directory buffers, they are typically 4KB +each, so we in 1.5MB of directory buffers we'd have roughly 400 buffers and a +buffer format structure for each buffer - roughly 800 vectors or 1.51MB total +space. From this, it should be obvious that a static log space reservation is +not particularly flexible and is difficult to select the "optimal value" for +all workloads. + +Further, if we are going to use a static reservation, which bit of the entire +reservation does it cover? We account for space used by the transaction +reservation by tracking the space currently used by the object in the CIL and +then calculating the increase or decrease in space used as the object is +relogged. This allows for a checkpoint reservation to only have to account for +log buffer metadata used such as log header records. + +However, even using a static reservation for just the log metadata is +problematic. Typically log record headers use at least 16KB of log space per +1MB of log space consumed (512 bytes per 32k) and the reservation needs to be +large enough to handle arbitrary sized checkpoint transactions. This +reservation needs to be made before the checkpoint is started, and we need to +be able to reserve the space without sleeping. For a 8MB checkpoint, we need a +reservation of around 150KB, which is a non-trivial amount of space. + +A static reservation needs to manipulate the log grant counters - we can take a +permanent reservation on the space, but we still need to make sure we refresh +the write reservation (the actual space available to the transaction) after +every checkpoint transaction completion. Unfortunately, if this space is not +available when required, then the regrant code will sleep waiting for it. + +The problem with this is that it can lead to deadlocks as we may need to commit +checkpoints to be able to free up log space (refer back to the description of +rolling transactions for an example of this). Hence we *must* always have +space available in the log if we are to use static reservations, and that is +very difficult and complex to arrange. It is possible to do, but there is a +simpler way. + +The simpler way of doing this is tracking the entire log space used by the +items in the CIL and using this to dynamically calculate the amount of log +space required by the log metadata. If this log metadata space changes as a +result of a transaction commit inserting a new memory buffer into the CIL, then +the difference in space required is removed from the transaction that causes +the change. Transactions at this level will *always* have enough space +available in their reservation for this as they have already reserved the +maximal amount of log metadata space they require, and such a delta reservation +will always be less than or equal to the maximal amount in the reservation. + +Hence we can grow the checkpoint transaction reservation dynamically as items +are added to the CIL and avoid the need for reserving and regranting log space +up front. This avoids deadlocks and removes a blocking point from the +checkpoint flush code. + +As mentioned early, transactions can't grow to more than half the size of the +log. Hence as part of the reservation growing, we need to also check the size +of the reservation against the maximum allowed transaction size. If we reach +the maximum threshold, we need to push the CIL to the log. This is effectively +a "background flush" and is done on demand. This is identical to +a CIL push triggered by a log force, only that there is no waiting for the +checkpoint commit to complete. This background push is checked and executed by +transaction commit code. + +If the transaction subsystem goes idle while we still have items in the CIL, +they will be flushed by the periodic log force issued by the xfssyncd. This log +force will push the CIL to disk, and if the transaction subsystem stays idle, +allow the idle log to be covered (effectively marked clean) in exactly the same +manner that is done for the existing logging method. A discussion point is +whether this log force needs to be done more frequently than the current rate +which is once every 30s. + + +Delayed Logging: Log Item Pinning + +Currently log items are pinned during transaction commit while the items are +still locked. This happens just after the items are formatted, though it could +be done any time before the items are unlocked. The result of this mechanism is +that items get pinned once for every transaction that is committed to the log +buffers. Hence items that are relogged in the log buffers will have a pin count +for every outstanding transaction they were dirtied in. When each of these +transactions is completed, they will unpin the item once. As a result, the item +only becomes unpinned when all the transactions complete and there are no +pending transactions. Thus the pinning and unpinning of a log item is symmetric +as there is a 1:1 relationship with transaction commit and log item completion. + +For delayed logging, however, we have an assymetric transaction commit to +completion relationship. Every time an object is relogged in the CIL it goes +through the commit process without a corresponding completion being registered. +That is, we now have a many-to-one relationship between transaction commit and +log item completion. The result of this is that pinning and unpinning of the +log items becomes unbalanced if we retain the "pin on transaction commit, unpin +on transaction completion" model. + +To keep pin/unpin symmetry, the algorithm needs to change to a "pin on +insertion into the CIL, unpin on checkpoint completion". In other words, the +pinning and unpinning becomes symmetric around a checkpoint context. We have to +pin the object the first time it is inserted into the CIL - if it is already in +the CIL during a transaction commit, then we do not pin it again. Because there +can be multiple outstanding checkpoint contexts, we can still see elevated pin +counts, but as each checkpoint completes the pin count will retain the correct +value according to it's context. + +Just to make matters more slightly more complex, this checkpoint level context +for the pin count means that the pinning of an item must take place under the +CIL commit/flush lock. If we pin the object outside this lock, we cannot +guarantee which context the pin count is associated with. This is because of +the fact pinning the item is dependent on whether the item is present in the +current CIL or not. If we don't pin the CIL first before we check and pin the +object, we have a race with CIL being flushed between the check and the pin +(or not pinning, as the case may be). Hence we must hold the CIL flush/commit +lock to guarantee that we pin the items correctly. + +Delayed Logging: Concurrent Scalability + +A fundamental requirement for the CIL is that accesses through transaction +commits must scale to many concurrent commits. The current transaction commit +code does not break down even when there are transactions coming from 2048 +processors at once. The current transaction code does not go any faster than if +there was only one CPU using it, but it does not slow down either. + +As a result, the delayed logging transaction commit code needs to be designed +for concurrency from the ground up. It is obvious that there are serialisation +points in the design - the three important ones are: + + 1. Locking out new transaction commits while flushing the CIL + 2. Adding items to the CIL and updating item space accounting + 3. Checkpoint commit ordering + +Looking at the transaction commit and CIL flushing interactions, it is clear +that we have a many-to-one interaction here. That is, the only restriction on +the number of concurrent transactions that can be trying to commit at once is +the amount of space available in the log for their reservations. The practical +limit here is in the order of several hundred concurrent transactions for a +128MB log, which means that it is generally one per CPU in a machine. + +The amount of time a transaction commit needs to hold out a flush is a +relatively long period of time - the pinning of log items needs to be done +while we are holding out a CIL flush, so at the moment that means it is held +across the formatting of the objects into memory buffers (i.e. while memcpy()s +are in progress). Ultimately a two pass algorithm where the formatting is done +separately to the pinning of objects could be used to reduce the hold time of +the transaction commit side. + +Because of the number of potential transaction commit side holders, the lock +really needs to be a sleeping lock - if the CIL flush takes the lock, we do not +want every other CPU in the machine spinning on the CIL lock. Given that +flushing the CIL could involve walking a list of tens of thousands of log +items, it will get held for a significant time and so spin contention is a +significant concern. Preventing lots of CPUs spinning doing nothing is the +main reason for choosing a sleeping lock even though nothing in either the +transaction commit or CIL flush side sleeps with the lock held. + +It should also be noted that CIL flushing is also a relatively rare operation +compared to transaction commit for asynchronous transaction workloads - only +time will tell if using a read-write semaphore for exclusion will limit +transaction commit concurrency due to cache line bouncing of the lock on the +read side. + +The second serialisation point is on the transaction commit side where items +are inserted into the CIL. Because transactions can enter this code +concurrently, the CIL needs to be protected separately from the above +commit/flush exclusion. It also needs to be an exclusive lock but it is only +held for a very short time and so a spin lock is appropriate here. It is +possible that this lock will become a contention point, but given the short +hold time once per transaction I think that contention is unlikely. + +The final serialisation point is the checkpoint commit record ordering code +that is run as part of the checkpoint commit and log force sequencing. The code +path that triggers a CIL flush (i.e. whatever triggers the log force) will enter +an ordering loop after writing all the log vectors into the log buffers but +before writing the commit record. This loop walks the list of committing +checkpoints and needs to block waiting for checkpoints to complete their commit +record write. As a result it needs a lock and a wait variable. Log force +sequencing also requires the same lock, list walk, and blocking mechanism to +ensure completion of checkpoints. + +These two sequencing operations can use the mechanism even though the +events they are waiting for are different. The checkpoint commit record +sequencing needs to wait until checkpoint contexts contain a commit LSN +(obtained through completion of a commit record write) while log force +sequencing needs to wait until previous checkpoint contexts are removed from +the committing list (i.e. they've completed). A simple wait variable and +broadcast wakeups (thundering herds) has been used to implement these two +serialisation queues. They use the same lock as the CIL, too. If we see too +much contention on the CIL lock, or too many context switches as a result of +the broadcast wakeups these operations can be put under a new spinlock and +given separate wait lists to reduce lock contention and the number of processes +woken by the wrong event. + + +Lifecycle Changes + +The existing log item life cycle is as follows: + + 1. Transaction allocate + 2. Transaction reserve + 3. Lock item + 4. Join item to transaction + If not already attached, + Allocate log item + Attach log item to owner item + Attach log item to transaction + 5. Modify item + Record modifications in log item + 6. Transaction commit + Pin item in memory + Format item into log buffer + Write commit LSN into transaction + Unlock item + Attach transaction to log buffer + + + + + 7. Transaction completion + Mark log item committed + Insert log item into AIL + Write commit LSN into log item + Unpin log item + 8. AIL traversal + Lock item + Mark log item clean + Flush item to disk + + + + 9. Log item removed from AIL + Moves log tail + Item unlocked + +Essentially, steps 1-6 operate independently from step 7, which is also +independent of steps 8-9. An item can be locked in steps 1-6 or steps 8-9 +at the same time step 7 is occurring, but only steps 1-6 or 8-9 can occur +at the same time. If the log item is in the AIL or between steps 6 and 7 +and steps 1-6 are re-entered, then the item is relogged. Only when steps 8-9 +are entered and completed is the object considered clean. + +With delayed logging, there are new steps inserted into the life cycle: + + 1. Transaction allocate + 2. Transaction reserve + 3. Lock item + 4. Join item to transaction + If not already attached, + Allocate log item + Attach log item to owner item + Attach log item to transaction + 5. Modify item + Record modifications in log item + 6. Transaction commit + Pin item in memory if not pinned in CIL + Format item into log vector + buffer + Attach log vector and buffer to log item + Insert log item into CIL + Write CIL context sequence into transaction + Unlock item + + + + 7. CIL push + lock CIL flush + Chain log vectors and buffers together + Remove items from CIL + unlock CIL flush + write log vectors into log + sequence commit records + attach checkpoint context to log buffer + + + + + 8. Checkpoint completion + Mark log item committed + Insert item into AIL + Write commit LSN into log item + Unpin log item + 9. AIL traversal + Lock item + Mark log item clean + Flush item to disk + + 10. Log item removed from AIL + Moves log tail + Item unlocked + +From this, it can be seen that the only life cycle differences between the two +logging methods are in the middle of the life cycle - they still have the same +beginning and end and execution constraints. The only differences are in the +commiting of the log items to the log itself and the completion processing. +Hence delayed logging should not introduce any constraints on log item +behaviour, allocation or freeing that don't already exist. + +As a result of this zero-impact "insertion" of delayed logging infrastructure +and the design of the internal structures to avoid on disk format changes, we +can basically switch between delayed logging and the existing mechanism with a +mount option. Fundamentally, there is no reason why the log manager would not +be able to swap methods automatically and transparently depending on load +characteristics, but this should not be necessary if delayed logging works as +designed. + +Roadmap: + +2.6.35 Inclusion in mainline as an experimental mount option + => approximately 2-3 months to merge window + => needs to be in xfs-dev tree in 4-6 weeks + => code is nearing readiness for review + +2.6.37 Remove experimental tag from mount option + => should be roughly 6 months after initial merge + => enough time to: + => gain confidence and fix problems reported by early + adopters (a.k.a. guinea pigs) + => address worst performance regressions and undesired + behaviours + => start tuning/optimising code for parallelism + => start tuning/optimising algorithms consuming + excessive CPU time + +2.6.39 Switch default mount option to use delayed logging + => should be roughly 12 months after initial merge + => enough time to shake out remaining problems before next round of + enterprise distro kernel rebases -- cgit v1.2.3 From 1dcf57ceb461fe2ddeb622fe84d9918aadefe7ba Mon Sep 17 00:00:00 2001 From: Prarit Bhargava Date: Wed, 19 May 2010 19:30:28 +0000 Subject: Documentation/spi/* compile warning fix Fix this warning: Documentation/spi/spidev_fdx.c: In function `do_msg': Documentation/spi/spidev_fdx.c:61: warning: cast from pointer to integer of different size Documentation/spi/spidev_fdx.c:64: warning: cast from pointer to integer of different size buf should be unsigned long to match native arch on 32-bit x86 compile. Signed-off-by: Prarit Bhargava Signed-off-by: Grant Likely --- Documentation/spi/spidev_fdx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/spi/spidev_fdx.c b/Documentation/spi/spidev_fdx.c index fc354f760384..36ec0774ca0b 100644 --- a/Documentation/spi/spidev_fdx.c +++ b/Documentation/spi/spidev_fdx.c @@ -58,10 +58,10 @@ static void do_msg(int fd, int len) len = sizeof buf; buf[0] = 0xaa; - xfer[0].tx_buf = (__u64) buf; + xfer[0].tx_buf = (unsigned long)buf; xfer[0].len = 1; - xfer[1].rx_buf = (__u64) buf; + xfer[1].rx_buf = (unsigned long) buf; xfer[1].len = len; status = ioctl(fd, SPI_IOC_MESSAGE(2), xfer); -- cgit v1.2.3 From 011f23a3c2f20ae15b7664d3942493af107fe39b Mon Sep 17 00:00:00 2001 From: Mika Westerberg Date: Thu, 6 May 2010 04:47:04 +0000 Subject: spi/ep93xx: implemented driver for Cirrus EP93xx SPI controller This patch adds an SPI master driver for the Cirrus EP93xx SPI controller found in EP93xx chips. Signed-off-by: Mika Westerberg Signed-off-by: H Hartley Sweeten Acked-by: H Hartley Sweeten Signed-off-by: Grant Likely --- Documentation/spi/ep93xx_spi | 95 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 Documentation/spi/ep93xx_spi (limited to 'Documentation') diff --git a/Documentation/spi/ep93xx_spi b/Documentation/spi/ep93xx_spi new file mode 100644 index 000000000000..6325f5b48635 --- /dev/null +++ b/Documentation/spi/ep93xx_spi @@ -0,0 +1,95 @@ +Cirrus EP93xx SPI controller driver HOWTO +========================================= + +ep93xx_spi driver brings SPI master support for EP93xx SPI controller. Chip +selects are implemented with GPIO lines. + +NOTE: If possible, don't use SFRMOUT (SFRM1) signal as a chip select. It will +not work correctly (it cannot be controlled by software). Use GPIO lines +instead. + +Sample configuration +==================== + +Typically driver configuration is done in platform board files (the files under +arch/arm/mach-ep93xx/*.c). In this example we configure MMC over SPI through +this driver on TS-7260 board. You can adapt the code to suit your needs. + +This example uses EGPIO9 as SD/MMC card chip select (this is wired in DIO1 +header on the board). + +You need to select CONFIG_MMC_SPI to use mmc_spi driver. + +arch/arm/mach-ep93xx/ts72xx.c: + +... +#include +#include + +#include + +/* this is our GPIO line used for chip select */ +#define MMC_CHIP_SELECT_GPIO EP93XX_GPIO_LINE_EGPIO9 + +static int ts72xx_mmc_spi_setup(struct spi_device *spi) +{ + int err; + + err = gpio_request(MMC_CHIP_SELECT_GPIO, spi->modalias); + if (err) + return err; + + gpio_direction_output(MMC_CHIP_SELECT_GPIO, 1); + + return 0; +} + +static void ts72xx_mmc_spi_cleanup(struct spi_device *spi) +{ + gpio_set_value(MMC_CHIP_SELECT_GPIO, 1); + gpio_direction_input(MMC_CHIP_SELECT_GPIO); + gpio_free(MMC_CHIP_SELECT_GPIO); +} + +static void ts72xx_mmc_spi_cs_control(struct spi_device *spi, int value) +{ + gpio_set_value(MMC_CHIP_SELECT_GPIO, value); +} + +static struct ep93xx_spi_chip_ops ts72xx_mmc_spi_ops = { + .setup = ts72xx_mmc_spi_setup, + .cleanup = ts72xx_mmc_spi_cleanup, + .cs_control = ts72xx_mmc_spi_cs_control, +}; + +static struct spi_board_info ts72xx_spi_devices[] __initdata = { + { + .modalias = "mmc_spi", + .controller_data = &ts72xx_mmc_spi_ops, + /* + * We use 10 MHz even though the maximum is 7.4 MHz. The driver + * will limit it automatically to max. frequency. + */ + .max_speed_hz = 10 * 1000 * 1000, + .bus_num = 0, + .chip_select = 0, + .mode = SPI_MODE_0, + }, +}; + +static struct ep93xx_spi_info ts72xx_spi_info = { + .num_chipselect = ARRAY_SIZE(ts72xx_spi_devices), +}; + +static void __init ts72xx_init_machine(void) +{ + ... + ep93xx_register_spi(&ts72xx_spi_info, ts72xx_spi_devices, + ARRAY_SIZE(ts72xx_spi_devices)); +} + +Thanks to +========= +Martin Guy, H. Hartley Sweeten and others who helped me during development of +the driver. Simplemachines.it donated me a Sim.One board which I used testing +the driver on EP9307. -- cgit v1.2.3 From 4724ba575ea73431aba8a48be6f5a382c470418d Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Mon, 3 May 2010 11:42:52 -0700 Subject: watchdog: update/improve/consolidate watchdog driver Move the limited watchdog driver help from kernel-parameters.txt to Documentation/watchdog/watchdog-parameters.txt and add info to it for all watchdog drivers except the ones that have driver-specific files already. Correct minor comments and MODULE_PARM_DESC() text in 2 places. Signed-off-by: Randy Dunlap Signed-off-by: Wim Van Sebroeck --- Documentation/kernel-parameters.txt | 15 +- Documentation/watchdog/00-INDEX | 5 + Documentation/watchdog/watchdog-parameters.txt | 390 +++++++++++++++++++++++++ Documentation/watchdog/wdt.txt | 15 +- 4 files changed, 410 insertions(+), 15 deletions(-) create mode 100644 Documentation/watchdog/watchdog-parameters.txt (limited to 'Documentation') diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index f5fce483930c..b56ea860da21 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -290,9 +290,6 @@ and is between 256 and 4096 characters. It is defined in the file advansys= [HW,SCSI] See header of drivers/scsi/advansys.c. - advwdt= [HW,WDT] Advantech WDT - Format: , - aedsp16= [HW,OSS] Audio Excel DSP 16 Format: ,,,,, See also header of sound/oss/aedsp16.c. @@ -765,9 +762,6 @@ and is between 256 and 4096 characters. It is defined in the file This option is obsoleted by the "netdev=" option, which has equivalent usage. See its documentation for details. - eurwdt= [HW,WDT] Eurotech CPU-1220/1410 onboard watchdog. - Format: [,] - failslab= fail_page_alloc= fail_make_request=[KNL] @@ -2267,9 +2261,6 @@ and is between 256 and 4096 characters. It is defined in the file sched_debug [KNL] Enables verbose scheduler debug messages. - sc1200wdt= [HW,WDT] SC1200 WDT (watchdog) driver - Format: [,[,]] - scsi_debug_*= [SCSI] See drivers/scsi/scsi_debug.c. @@ -2858,8 +2849,10 @@ and is between 256 and 4096 characters. It is defined in the file wd7000= [HW,SCSI] See header of drivers/scsi/wd7000.c. - wdt= [WDT] Watchdog - See Documentation/watchdog/wdt.txt. + watchdog timers [HW,WDT] For information on watchdog timers, + see Documentation/watchdog/watchdog-parameters.txt + or other driver-specific files in the + Documentation/watchdog/ directory. x2apic_phys [X86-64,APIC] Use x2apic physical mode instead of default x2apic cluster mode on platforms diff --git a/Documentation/watchdog/00-INDEX b/Documentation/watchdog/00-INDEX index c3ea47e507fe..ee994513a9b1 100644 --- a/Documentation/watchdog/00-INDEX +++ b/Documentation/watchdog/00-INDEX @@ -1,10 +1,15 @@ 00-INDEX - this file. +hpwdt.txt + - information on the HP iLO2 NMI watchdog pcwd-watchdog.txt - documentation for Berkshire Products PC Watchdog ISA cards. src/ - directory holding watchdog related example programs. watchdog-api.txt - description of the Linux Watchdog driver API. +watchdog-parameters.txt + - information on driver parameters (for drivers other than + the ones that have driver-specific files here) wdt.txt - description of the Watchdog Timer Interfaces for Linux. diff --git a/Documentation/watchdog/watchdog-parameters.txt b/Documentation/watchdog/watchdog-parameters.txt new file mode 100644 index 000000000000..41c95cc1dc1f --- /dev/null +++ b/Documentation/watchdog/watchdog-parameters.txt @@ -0,0 +1,390 @@ +This file provides information on the module parameters of many of +the Linux watchdog drivers. Watchdog driver parameter specs should +be listed here unless the driver has its own driver-specific information +file. + + +See Documentation/kernel-parameters.txt for information on +providing kernel parameters for builtin drivers versus loadable +modules. + + +------------------------------------------------- +acquirewdt: +wdt_stop: Acquire WDT 'stop' io port (default 0x43) +wdt_start: Acquire WDT 'start' io port (default 0x443) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +advantechwdt: +wdt_stop: Advantech WDT 'stop' io port (default 0x443) +wdt_start: Advantech WDT 'start' io port (default 0x443) +timeout: Watchdog timeout in seconds. 1<= timeout <=63, default=60. +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +alim1535_wdt: +timeout: Watchdog timeout in seconds. (0 < timeout < 18000, default=60 +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +alim7101_wdt: +timeout: Watchdog timeout in seconds. (1<=timeout<=3600, default=30 +use_gpio: Use the gpio watchdog (required by old cobalt boards). + default=0/off/no +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +ar7_wdt: +margin: Watchdog margin in seconds (default=60) +nowayout: Disable watchdog shutdown on close + (default=kernel config parameter) +------------------------------------------------- +at32ap700x_wdt: +timeout: Timeout value. Limited to be 1 or 2 seconds. (default=2) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +at91rm9200_wdt: +wdt_time: Watchdog time in seconds. (default=5) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +at91sam9_wdt: +heartbeat: Watchdog heartbeats in seconds. (default = 15) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +bcm47xx_wdt: +wdt_time: Watchdog time in seconds. (default=30) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +bfin_wdt: +timeout: Watchdog timeout in seconds. (1<=timeout<=((2^32)/SCLK), default=20) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +coh901327_wdt: +margin: Watchdog margin in seconds (default 60s) +------------------------------------------------- +cpu5wdt: +port: base address of watchdog card, default is 0x91 +verbose: be verbose, default is 0 (no) +ticks: count down ticks, default is 10000 +------------------------------------------------- +cpwd: +wd0_timeout: Default watchdog0 timeout in 1/10secs +wd1_timeout: Default watchdog1 timeout in 1/10secs +wd2_timeout: Default watchdog2 timeout in 1/10secs +------------------------------------------------- +davinci_wdt: +heartbeat: Watchdog heartbeat period in seconds from 1 to 600, default 60 +------------------------------------------------- +ep93xx_wdt: +nowayout: Watchdog cannot be stopped once started +timeout: Watchdog timeout in seconds. (1<=timeout<=3600, default=TBD) +------------------------------------------------- +eurotechwdt: +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +io: Eurotech WDT io port (default=0x3f0) +irq: Eurotech WDT irq (default=10) +ev: Eurotech WDT event type (default is `int') +------------------------------------------------- +gef_wdt: +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +geodewdt: +timeout: Watchdog timeout in seconds. 1<= timeout <=131, default=60. +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +i6300esb: +heartbeat: Watchdog heartbeat in seconds. (11 for debug, (default 0) +------------------------------------------------- +sa1100_wdt: +margin: Watchdog margin in seconds (default 60s) +------------------------------------------------- +sb_wdog: +timeout: Watchdog timeout in microseconds (max/default 8388607 or 8.3ish secs) +------------------------------------------------- +sbc60xxwdt: +wdt_stop: SBC60xx WDT 'stop' io port (default 0x45) +wdt_start: SBC60xx WDT 'start' io port (default 0x443) +timeout: Watchdog timeout in seconds. (1<=timeout<=3600, default=30) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +sbc7240_wdt: +timeout: Watchdog timeout in seconds. (1<=timeout<=255, default=30) +nowayout: Disable watchdog when closing device file +------------------------------------------------- +sbc8360: +timeout: Index into timeout table (0-63) (default=27 (60s)) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +sbc_epx_c3: +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +sbc_fitpc2_wdt: +margin: Watchdog margin in seconds (default 60s) +nowayout: Watchdog cannot be stopped once started +------------------------------------------------- +sc1200wdt: +isapnp: When set to 0 driver ISA PnP support will be disabled (default=1) +io: io port +timeout: range is 0-255 minutes, default is 1 +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +sc520_wdt: +timeout: Watchdog timeout in seconds. (1 <= timeout <= 3600, default=30) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +sch311x_wdt: +force_id: Override the detected device ID +therm_trip: Should a ThermTrip trigger the reset generator +timeout: Watchdog timeout in seconds. 1<= timeout <=15300, default=60 +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +scx200_wdt: +margin: Watchdog margin in seconds +nowayout: Disable watchdog shutdown on close +------------------------------------------------- +shwdt: +clock_division_ratio: Clock division ratio. Valid ranges are from 0x5 (1.31ms) + to 0x7 (5.25ms). (default=7) +heartbeat: Watchdog heartbeat in seconds. (1 <= heartbeat <= 3600, default=30 +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +smsc37b787_wdt: +timeout: range is 1-255 units, default is 60 +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +softdog: +soft_margin: Watchdog soft_margin in seconds. + (0 < soft_margin < 65536, default=60) +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +soft_noboot: Softdog action, set to 1 to ignore reboots, 0 to reboot + (default=0) +------------------------------------------------- +stmp3xxx_wdt: +heartbeat: Watchdog heartbeat period in seconds from 1 to 4194304, default 19 +------------------------------------------------- +ts72xx_wdt: +timeout: Watchdog timeout in seconds. (1 <= timeout <= 8, default=8) +nowayout: Disable watchdog shutdown on close +------------------------------------------------- +twl4030_wdt: +nowayout: Watchdog cannot be stopped once started + (default=kernel config parameter) +------------------------------------------------- +txx9wdt: +timeout: Watchdog timeout in seconds. (0 Date: Mon, 24 May 2010 14:32:05 -0700 Subject: mempolicy: document cpuset interaction with tmpfs mpol mount option Update Documentation/filesystems/tmpfs.txt to describe the interaction of tmpfs mount option memory policy with tasks' cpuset mems_allowed. Note: the mount(8) man page [in the util-linux-ng package] requires similiar updates. Signed-off-by: Lee Schermerhorn Cc: Hugh Dickins Cc: Ravikiran Thirumalai Cc: KOSAKI Motohiro Cc: Christoph Lameter Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/filesystems/tmpfs.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/tmpfs.txt b/Documentation/filesystems/tmpfs.txt index fe09a2cb1858..98ef55124158 100644 --- a/Documentation/filesystems/tmpfs.txt +++ b/Documentation/filesystems/tmpfs.txt @@ -94,11 +94,19 @@ NodeList format is a comma-separated list of decimal numbers and ranges, a range being two hyphen-separated decimal numbers, the smallest and largest node numbers in the range. For example, mpol=bind:0-3,5,7,9-15 +A memory policy with a valid NodeList will be saved, as specified, for +use at file creation time. When a task allocates a file in the file +system, the mount option memory policy will be applied with a NodeList, +if any, modified by the calling task's cpuset constraints +[See Documentation/cgroups/cpusets.txt] and any optional flags, listed +below. If the resulting NodeLists is the empty set, the effective memory +policy for the file will revert to "default" policy. + NUMA memory allocation policies have optional flags that can be used in conjunction with their modes. These optional flags can be specified when tmpfs is mounted by appending them to the mode before the NodeList. See Documentation/vm/numa_memory_policy.txt for a list of all available -memory allocation policy mode flags. +memory allocation policy mode flags and their effect on memory policy. =static is equivalent to MPOL_F_STATIC_NODES =relative is equivalent to MPOL_F_RELATIVE_NODES -- cgit v1.2.3 From 76ab0f530e4a01d4dc20cdc1d5e87753c579dc18 Mon Sep 17 00:00:00 2001 From: Mel Gorman Date: Mon, 24 May 2010 14:32:28 -0700 Subject: mm: compaction: add /proc trigger for memory compaction Add a proc file /proc/sys/vm/compact_memory. When an arbitrary value is written to the file, all zones are compacted. The expected user of such a trigger is a job scheduler that prepares the system before the target application runs. Signed-off-by: Mel Gorman Acked-by: Rik van Riel Reviewed-by: KAMEZAWA Hiroyuki Reviewed-by: Minchan Kim Reviewed-by: KOSAKI Motohiro Reviewed-by: Christoph Lameter Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/sysctl/vm.txt | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'Documentation') diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt index 6c7d18c53f84..56dd29b97a91 100644 --- a/Documentation/sysctl/vm.txt +++ b/Documentation/sysctl/vm.txt @@ -19,6 +19,7 @@ files can be found in mm/swap.c. Currently, these files are in /proc/sys/vm: - block_dump +- compact_memory - dirty_background_bytes - dirty_background_ratio - dirty_bytes @@ -64,6 +65,15 @@ information on block I/O debugging is in Documentation/laptops/laptop-mode.txt. ============================================================== +compact_memory + +Available only when CONFIG_COMPACTION is set. When 1 is written to the file, +all zones are compacted such that free memory is available in contiguous +blocks where possible. This can be important for example in the allocation of +huge pages although processes will also directly compact memory as required. + +============================================================== + dirty_background_bytes Contains the amount of dirty memory at which the pdflush background writeback -- cgit v1.2.3 From ed4a6d7f0676db50b5023cc01f6cda82a2f2a307 Mon Sep 17 00:00:00 2001 From: Mel Gorman Date: Mon, 24 May 2010 14:32:29 -0700 Subject: mm: compaction: add /sys trigger for per-node memory compaction Add a per-node sysfs file called compact. When the file is written to, each zone in that node is compacted. The intention that this would be used by something like a job scheduler in a batch system before a job starts so that the job can allocate the maximum number of hugepages without significant start-up cost. Signed-off-by: Mel Gorman Acked-by: Rik van Riel Reviewed-by: KOSAKI Motohiro Reviewed-by: Christoph Lameter Reviewed-by: Minchan Kim Reviewed-by: KAMEZAWA Hiroyuki Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/ABI/testing/sysfs-devices-node | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-devices-node (limited to 'Documentation') diff --git a/Documentation/ABI/testing/sysfs-devices-node b/Documentation/ABI/testing/sysfs-devices-node new file mode 100644 index 000000000000..453a210c3ceb --- /dev/null +++ b/Documentation/ABI/testing/sysfs-devices-node @@ -0,0 +1,7 @@ +What: /sys/devices/system/node/nodeX/compact +Date: February 2010 +Contact: Mel Gorman +Description: + When this file is written to, all memory within that node + will be compacted. When it completes, memory will be freed + into blocks which have as many contiguous pages as possible -- cgit v1.2.3 From 5e7719058079a1423ccce56148b0aaa56b2df821 Mon Sep 17 00:00:00 2001 From: Mel Gorman Date: Mon, 24 May 2010 14:32:31 -0700 Subject: mm: compaction: add a tunable that decides when memory should be compacted and when it should be reclaimed The kernel applies some heuristics when deciding if memory should be compacted or reclaimed to satisfy a high-order allocation. One of these is based on the fragmentation. If the index is below 500, memory will not be compacted. This choice is arbitrary and not based on data. To help optimise the system and set a sensible default for this value, this patch adds a sysctl extfrag_threshold. The kernel will only compact memory if the fragmentation index is above the extfrag_threshold. [randy.dunlap@oracle.com: Fix build errors when proc fs is not configured] Signed-off-by: Mel Gorman Signed-off-by: Randy Dunlap Cc: Rik van Riel Cc: Minchan Kim Cc: KOSAKI Motohiro Cc: Christoph Lameter Cc: KAMEZAWA Hiroyuki Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/sysctl/vm.txt | 15 +++++++++++++++ 1 file changed, 15 insertions(+) (limited to 'Documentation') diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt index 56dd29b97a91..5fdbb612aeb8 100644 --- a/Documentation/sysctl/vm.txt +++ b/Documentation/sysctl/vm.txt @@ -27,6 +27,7 @@ Currently, these files are in /proc/sys/vm: - dirty_ratio - dirty_writeback_centisecs - drop_caches +- extfrag_threshold - hugepages_treat_as_movable - hugetlb_shm_group - laptop_mode @@ -149,6 +150,20 @@ user should run `sync' first. ============================================================== +extfrag_threshold + +This parameter affects whether the kernel will compact memory or direct +reclaim to satisfy a high-order allocation. /proc/extfrag_index shows what +the fragmentation index for each order is in each zone in the system. Values +tending towards 0 imply allocations would fail due to lack of memory, +values towards 1000 imply failures are due to fragmentation and -1 implies +that the allocation will succeed as long as watermarks are met. + +The kernel will not compact memory in a zone if the +fragmentation index is <= extfrag_threshold. The default value is 500. + +============================================================== + hugepages_treat_as_movable This parameter is only useful when kernelcore= is specified at boot time to -- cgit v1.2.3 From 3014420b6b5d0a6483cf5e56c10df180a33e957e Mon Sep 17 00:00:00 2001 From: Stefan Richter Date: Mon, 24 May 2010 21:31:23 +0200 Subject: ieee1394: schedule for removal All application domains that are supported by the old ieee1394 driver stack are supported by the newer firewire driver stack too. There is now good and extensive experience with the newer stack from deployment in Fedora since F7 as well as by enthusiast users of other distributions. The new drivers have consequently been recommended as the default ones since 2.6.33, in order to fix some severe usability problems of FireWire on Linux due to limitations of the old stack. It is now high time to announce when the obsolete drivers will be removed. Signed-off-by: Stefan Richter Acked-by: Jarod Wilson --- Documentation/feature-removal-schedule.txt | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'Documentation') diff --git a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt index 0a46833c1b76..88a216d18092 100644 --- a/Documentation/feature-removal-schedule.txt +++ b/Documentation/feature-removal-schedule.txt @@ -542,3 +542,13 @@ Why: Duplicate functionality with the gspca_zc3xx driver, zc0301 only sensors) wich are also supported by the gspca_zc3xx driver (which supports 53 USB-ID's in total) Who: Hans de Goede + +---------------------------- + +What: old ieee1394 subsystem (CONFIG_IEEE1394) +When: 2.6.37 +Files: drivers/ieee1394/ except init_ohci1394_dma.c +Why: superseded by drivers/firewire/ (CONFIG_FIREWIRE) which offers more + features, better performance, and better security, all with smaller + and more modern code base +Who: Stefan Richter -- cgit v1.2.3 From 578454ff7eab61d13a26b568f99a89a2c9edc881 Mon Sep 17 00:00:00 2001 From: Kay Sievers Date: Thu, 20 May 2010 18:07:20 +0200 Subject: driver core: add devname module aliases to allow module on-demand auto-loading This adds: alias: devname: to some common kernel modules, which will allow the on-demand loading of the kernel module when the device node is accessed. Ideally all these modules would be compiled-in, but distros seems too much in love with their modularization that we need to cover the common cases with this new facility. It will allow us to remove a bunch of pretty useless init scripts and modprobes from init scripts. The static device node aliases will be carried in the module itself. The program depmod will extract this information to a file in the module directory: $ cat /lib/modules/2.6.34-00650-g537b60d-dirty/modules.devname # Device nodes to trigger on-demand module loading. microcode cpu/microcode c10:184 fuse fuse c10:229 ppp_generic ppp c108:0 tun net/tun c10:200 dm_mod mapper/control c10:235 Udev will pick up the depmod created file on startup and create all the static device nodes which the kernel modules specify, so that these modules get automatically loaded when the device node is accessed: $ /sbin/udevd --debug ... static_dev_create_from_modules: mknod '/dev/cpu/microcode' c10:184 static_dev_create_from_modules: mknod '/dev/fuse' c10:229 static_dev_create_from_modules: mknod '/dev/ppp' c108:0 static_dev_create_from_modules: mknod '/dev/net/tun' c10:200 static_dev_create_from_modules: mknod '/dev/mapper/control' c10:235 udev_rules_apply_static_dev_perms: chmod '/dev/net/tun' 0666 udev_rules_apply_static_dev_perms: chmod '/dev/fuse' 0666 A few device nodes are switched to statically allocated numbers, to allow the static nodes to work. This might also useful for systems which still run a plain static /dev, which is completely unsafe to use with any dynamic minor numbers. Note: The devname aliases must be limited to the *common* and *single*instance* device nodes, like the misc devices, and never be used for conceptually limited systems like the loop devices, which should rather get fixed properly and get a control node for losetup to talk to, instead of creating a random number of device nodes in advance, regardless if they are ever used. This facility is to hide the mess distros are creating with too modualized kernels, and just to hide that these modules are not compiled-in, and not to paper-over broken concepts. Thanks! :) Cc: Greg Kroah-Hartman Cc: David S. Miller Cc: Miklos Szeredi Cc: Chris Mason Cc: Alasdair G Kergon Cc: Tigran Aivazian Cc: Ian Kent Signed-Off-By: Kay Sievers Signed-off-by: Greg Kroah-Hartman --- Documentation/devices.txt | 2 ++ 1 file changed, 2 insertions(+) (limited to 'Documentation') diff --git a/Documentation/devices.txt b/Documentation/devices.txt index 53d64d382343..1d83d124056c 100644 --- a/Documentation/devices.txt +++ b/Documentation/devices.txt @@ -443,6 +443,8 @@ Your cooperation is appreciated. 231 = /dev/snapshot System memory snapshot device 232 = /dev/kvm Kernel-based virtual machine (hardware virtualization extensions) 233 = /dev/kmview View-OS A process with a view + 234 = /dev/btrfs-control Btrfs control device + 235 = /dev/autofs Autofs control device 240-254 Reserved for local use 255 Reserved for MISC_DYNAMIC_MINOR -- cgit v1.2.3 From 43c9c59185eec7caaff6e9dd8d4c93a4d9836a86 Mon Sep 17 00:00:00 2001 From: Tejun Heo Date: Sun, 23 May 2010 12:59:11 +0200 Subject: libata: implement dump_id force param Add dump_id libata.force parameter. If specified, libata dumps full IDENTIFY data during device configuration. This is to aid debugging. Signed-off-by: Tejun Heo Cc: Larry Baker Signed-off-by: Jeff Garzik --- Documentation/kernel-parameters.txt | 2 ++ 1 file changed, 2 insertions(+) (limited to 'Documentation') diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index b56ea860da21..a5c055066764 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -1252,6 +1252,8 @@ and is between 256 and 4096 characters. It is defined in the file * nohrst, nosrst, norst: suppress hard, soft and both resets. + * dump_id: dump IDENTIFY data. + If there are multiple matching configurations changing the same attribute, the last one is used. -- cgit v1.2.3 From 899f4530334da9292556e1f8f5791468e0136ff1 Mon Sep 17 00:00:00 2001 From: Phillip Lougher Date: Tue, 25 May 2010 02:47:00 +0100 Subject: squashfs: update documentation to include description of xattr layout Signed-off-by: Phillip Lougher --- Documentation/filesystems/squashfs.txt | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/squashfs.txt b/Documentation/filesystems/squashfs.txt index b324c033035a..203f7202cc9e 100644 --- a/Documentation/filesystems/squashfs.txt +++ b/Documentation/filesystems/squashfs.txt @@ -38,7 +38,8 @@ Hard link support: yes no Real inode numbers: yes no 32-bit uids/gids: yes no File creation time: yes no -Xattr and ACL support: no no +Xattr support: yes no +ACL support: no no Squashfs compresses data, inodes and directories. In addition, inode and directory data are highly compacted, and packed on byte boundaries. Each @@ -58,7 +59,7 @@ obtained from this site also. 3. SQUASHFS FILESYSTEM DESIGN ----------------------------- -A squashfs filesystem consists of seven parts, packed together on a byte +A squashfs filesystem consists of a maximum of eight parts, packed together on a byte alignment: --------------- @@ -80,6 +81,9 @@ alignment: |---------------| | uid/gid | | lookup table | + |---------------| + | xattr | + | table | --------------- Compressed data blocks are written to the filesystem as files are read from @@ -192,6 +196,26 @@ This table is stored compressed into metadata blocks. A second index table is used to locate these. This second index table for speed of access (and because it is small) is read at mount time and cached in memory. +3.7 Xattr table +--------------- + +The xattr table contains extended attributes for each inode. The xattrs +for each inode are stored in a list, each list entry containing a type, +name and value field. The type field encodes the xattr prefix +("user.", "trusted." etc) and it also encodes how the name/value fields +should be interpreted. Currently the type indicates whether the value +is stored inline (in which case the value field contains the xattr value), +or if it is stored out of line (in which case the value field stores a +reference to where the actual value is stored). This allows large values +to be stored out of line improving scanning and lookup performance and it +also allows values to be de-duplicated, the value being stored once, and +all other occurences holding an out of line reference to that value. + +The xattr lists are packed into compressed 8K metadata blocks. +To reduce overhead in inodes, rather than storing the on-disk +location of the xattr list inside each inode, a 32-bit xattr id +is stored. This xattr id is mapped into the location of the xattr +list using a second xattr id lookup table. 4. TODOS AND OUTSTANDING ISSUES ------------------------------- @@ -199,9 +223,7 @@ it is small) is read at mount time and cached in memory. 4.1 Todo list ------------- -Implement Xattr and ACL support. The Squashfs 4.0 filesystem layout has hooks -for these but the code has not been written. Once the code has been written -the existing layout should not require modification. +Implement ACL support. 4.2 Squashfs internal cache --------------------------- -- cgit v1.2.3 From 6d06b81bce5f325055e4cdf6621e827b512b7f89 Mon Sep 17 00:00:00 2001 From: "Arce, Abraham" Date: Wed, 26 May 2010 14:42:34 -0700 Subject: Documentation: SubmittingDrivers: Resources - Add additional location (Git) for the kernel master tree - Add reference to Git Project Signed-off-by: Abraham Arce Acked-by: Randy Dunlap Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/SubmittingDrivers | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'Documentation') diff --git a/Documentation/SubmittingDrivers b/Documentation/SubmittingDrivers index 99e72a81fa2f..4947fd8fb182 100644 --- a/Documentation/SubmittingDrivers +++ b/Documentation/SubmittingDrivers @@ -130,6 +130,8 @@ Linux kernel master tree: ftp.??.kernel.org:/pub/linux/kernel/... ?? == your country code, such as "us", "uk", "fr", etc. + http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git + Linux kernel mailing list: linux-kernel@vger.kernel.org [mail majordomo@vger.kernel.org to subscribe] @@ -160,3 +162,6 @@ How to NOT write kernel driver by Arjan van de Ven: Kernel Janitor: http://janitor.kernelnewbies.org/ + +GIT, Fast Version Control System: + http://git-scm.com/ -- cgit v1.2.3 From 595f4b694c2424f2af6e1de74c955bbe9cd7b73d Mon Sep 17 00:00:00 2001 From: Trevor Woerner Date: Wed, 26 May 2010 14:42:35 -0700 Subject: Documentation/cgroups/cgroups.txt: fix reference to "numtasks" Signed-off-by: Trevor Woerner Cc: Paul Menage Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/cgroups/cgroups.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/cgroups/cgroups.txt b/Documentation/cgroups/cgroups.txt index 57444c2609fc..b34823ff1646 100644 --- a/Documentation/cgroups/cgroups.txt +++ b/Documentation/cgroups/cgroups.txt @@ -339,7 +339,7 @@ To mount a cgroup hierarchy with all available subsystems, type: The "xxx" is not interpreted by the cgroup code, but will appear in /proc/mounts so may be any useful identifying string that you like. -To mount a cgroup hierarchy with just the cpuset and numtasks +To mount a cgroup hierarchy with just the cpuset and memory subsystems, type: # mount -t cgroup -o cpuset,memory hier1 /dev/cgroup -- cgit v1.2.3 From 9490ff275606da012d5b373342a49610ad61cb81 Mon Sep 17 00:00:00 2001 From: KAMEZAWA Hiroyuki Date: Wed, 26 May 2010 14:42:36 -0700 Subject: memcg: oom notifier Considering containers or other resource management softwares in userland, event notification of OOM in memcg should be implemented. Now, memcg has "threshold" notifier which uses eventfd, we can make use of it for oom notification. This patch adds oom notification eventfd callback for memcg. The usage is very similar to threshold notifier, but control file is memory.oom_control and no arguments other than eventfd is required. % cgroup_event_notifier /cgroup/A/memory.oom_control dummy (About cgroup_event_notifier, see Documentation/cgroup/) Signed-off-by: KAMEZAWA Hiroyuki Cc: Daisuke Nishimura Cc: Balbir Singh Cc: Daisuke Nishimura Cc: David Rientjes Cc: Davide Libenzi Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/cgroups/memory.txt | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index 6cab1f29da4c..eac22d3b2f7b 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -184,6 +184,9 @@ limits on the root cgroup. Note2: When panic_on_oom is set to "2", the whole system will panic. +When oom event notifier is registered, event will be delivered. +(See oom_control section) + 2. Locking The memory controller uses the following hierarchy @@ -488,7 +491,22 @@ threshold in any direction. It's applicable for root and non-root cgroup. -10. TODO +10. OOM Control + +Memory controler implements oom notifier using cgroup notification +API (See cgroups.txt). It allows to register multiple oom notification +delivery and gets notification when oom happens. + +To register a notifier, application need: + - create an eventfd using eventfd(2) + - open memory.oom_control file + - write string like " " to cgroup.event_control + +Application will be notifier through eventfd when oom happens. +OOM notification doesn't work for root cgroup. + + +11. TODO 1. Add support for accounting huge pages (as a separate controller) 2. Make per-cgroup scanner reclaim not-shared pages first -- cgit v1.2.3 From 3c11ecf448eff8f12922c498b8274ce98587eb74 Mon Sep 17 00:00:00 2001 From: KAMEZAWA Hiroyuki Date: Wed, 26 May 2010 14:42:37 -0700 Subject: memcg: oom kill disable and oom status This adds a feature to disable oom-killer for memcg, if disabled, of course, tasks under memcg will stop. But now, we have oom-notifier for memcg. And the world around memcg is not under out-of-memory. memcg's out-of-memory just shows memcg hits limit. Then, administrator or management daemon can recover the situation by - kill some process - enlarge limit, add more swap. - migrate some tasks - remove file cache on tmps (difficult ?) Unlike oom-killer, you can take enough information before killing tasks. (by gcore, or, ps etc.) [akpm@linux-foundation.org: coding-style fixes] Signed-off-by: KAMEZAWA Hiroyuki Cc: Daisuke Nishimura Cc: Balbir Singh Cc: Daisuke Nishimura Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/cgroups/memory.txt | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) (limited to 'Documentation') diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index eac22d3b2f7b..44e7ded33448 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -493,6 +493,8 @@ It's applicable for root and non-root cgroup. 10. OOM Control +memory.oom_control file is for OOM notification and other controls. + Memory controler implements oom notifier using cgroup notification API (See cgroups.txt). It allows to register multiple oom notification delivery and gets notification when oom happens. @@ -505,6 +507,27 @@ To register a notifier, application need: Application will be notifier through eventfd when oom happens. OOM notification doesn't work for root cgroup. +You can disable oom-killer by writing "1" to memory.oom_control file. +As. + #echo 1 > memory.oom_control + +This operation is only allowed to the top cgroup of subhierarchy. +If oom-killer is disabled, tasks under cgroup will hang/sleep +in memcg's oom-waitq when they request accountable memory. + +For running them, you have to relax the memcg's oom sitaution by + * enlarge limit or reduce usage. +To reduce usage, + * kill some tasks. + * move some tasks to other group with account migration. + * remove some files (on tmpfs?) + +Then, stopped tasks will work again. + +At reading, current status of OOM is shown. + oom_kill_disable 0 or 1 (if 1, oom-killer is disabled) + under_oom 0 or 1 (if 1, the memcg is under OOM,tasks may + be stopped.) 11. TODO -- cgit v1.2.3 From 87946a72283be3de936adc754b7007df7d3e6aeb Mon Sep 17 00:00:00 2001 From: Daisuke Nishimura Date: Wed, 26 May 2010 14:42:39 -0700 Subject: memcg: move charge of file pages This patch adds support for moving charge of file pages, which include normal file, tmpfs file and swaps of tmpfs file. It's enabled by setting bit 1 of /memory.move_charge_at_immigrate. Unlike the case of anonymous pages, file pages(and swaps) in the range mmapped by the task will be moved even if the task hasn't done page fault, i.e. they might not be the task's "RSS", but other task's "RSS" that maps the same file. And mapcount of the page is ignored(the page can be moved even if page_mapcount(page) > 1). So, conditions that the page/swap should be met to be moved is that it must be in the range mmapped by the target task and it must be charged to the old cgroup. [akpm@linux-foundation.org: coding-style fixes] [akpm@linux-foundation.org: fix warning] Signed-off-by: Daisuke Nishimura Acked-by: KAMEZAWA Hiroyuki Cc: Balbir Singh Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/cgroups/memory.txt | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) (limited to 'Documentation') diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index 44e7ded33448..5e028870ee8a 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -454,21 +454,27 @@ And if you want disable it again: 8.2 Type of charges which can be move Each bits of move_charge_at_immigrate has its own meaning about what type of -charges should be moved. +charges should be moved. But in any cases, it must be noted that an account of +a page or a swap can be moved only when it is charged to the task's current(old) +memory cgroup. bit | what type of charges would be moved ? -----+------------------------------------------------------------------------ 0 | A charge of an anonymous page(or swap of it) used by the target task. | Those pages and swaps must be used only by the target task. You must | enable Swap Extension(see 2.4) to enable move of swap charges. - -Note: Those pages and swaps must be charged to the old cgroup. -Note: More type of pages(e.g. file cache, shmem,) will be supported by other - bits in future. + -----+------------------------------------------------------------------------ + 1 | A charge of file pages(normal file, tmpfs file(e.g. ipc shared memory) + | and swaps of tmpfs file) mmaped by the target task. Unlike the case of + | anonymous pages, file pages(and swaps) in the range mmapped by the task + | will be moved even if the task hasn't done page fault, i.e. they might + | not be the task's "RSS", but other task's "RSS" that maps the same file. + | And mapcount of the page is ignored(the page can be moved even if + | page_mapcount(page) > 1). You must enable Swap Extension(see 2.4) to + | enable move of swap charges. 8.3 TODO -- Add support for other types of pages(e.g. file cache, shmem, etc.). - Implement madvise(2) to let users decide the vma to be moved or not to be moved. - All of moving charge operations are done under cgroup_mutex. It's not good -- cgit v1.2.3 From dc10e281f5fc42e288ab979294d1d5dc9743ae1b Mon Sep 17 00:00:00 2001 From: KAMEZAWA Hiroyuki Date: Wed, 26 May 2010 14:42:40 -0700 Subject: memcg: update documentation Some information are old, and I think current document doesn't work as "a guide for users". We need summary of all of our controls, at least. Signed-off-by: KAMEZAWA Hiroyuki Reviewed-by: Randy Dunlap Cc: Balbir Singh Cc: Daisuke Nishimura Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/cgroups/memory.txt | 291 ++++++++++++++++++++++++++------------- 1 file changed, 198 insertions(+), 93 deletions(-) (limited to 'Documentation') diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index 5e028870ee8a..7781857dc940 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -1,18 +1,15 @@ Memory Resource Controller NOTE: The Memory Resource Controller has been generically been referred -to as the memory controller in this document. Do not confuse memory controller -used here with the memory controller that is used in hardware. + to as the memory controller in this document. Do not confuse memory + controller used here with the memory controller that is used in hardware. -Salient features - -a. Enable control of Anonymous, Page Cache (mapped and unmapped) and - Swap Cache memory pages. -b. The infrastructure allows easy addition of other types of memory to control -c. Provides *zero overhead* for non memory controller users -d. Provides a double LRU: global memory pressure causes reclaim from the - global LRU; a cgroup on hitting a limit, reclaims from the per - cgroup LRU +(For editors) +In this document: + When we mention a cgroup (cgroupfs's directory) with memory controller, + we call it "memory cgroup". When you see git-log and source code, you'll + see patch's title and function names tend to use "memcg". + In this document, we avoid using it. Benefits and Purpose of the memory controller @@ -33,6 +30,45 @@ d. A CD/DVD burner could control the amount of memory used by the e. There are several other use cases, find one or use the controller just for fun (to learn and hack on the VM subsystem). +Current Status: linux-2.6.34-mmotm(development version of 2010/April) + +Features: + - accounting anonymous pages, file caches, swap caches usage and limiting them. + - private LRU and reclaim routine. (system's global LRU and private LRU + work independently from each other) + - optionally, memory+swap usage can be accounted and limited. + - hierarchical accounting + - soft limit + - moving(recharging) account at moving a task is selectable. + - usage threshold notifier + - oom-killer disable knob and oom-notifier + - Root cgroup has no limit controls. + + Kernel memory and Hugepages are not under control yet. We just manage + pages on LRU. To add more controls, we have to take care of performance. + +Brief summary of control files. + + tasks # attach a task(thread) and show list of threads + cgroup.procs # show list of processes + cgroup.event_control # an interface for event_fd() + memory.usage_in_bytes # show current memory(RSS+Cache) usage. + memory.memsw.usage_in_bytes # show current memory+Swap usage + memory.limit_in_bytes # set/show limit of memory usage + memory.memsw.limit_in_bytes # set/show limit of memory+Swap usage + memory.failcnt # show the number of memory usage hits limits + memory.memsw.failcnt # show the number of memory+Swap hits limits + memory.max_usage_in_bytes # show max memory usage recorded + memory.memsw.usage_in_bytes # show max memory+Swap usage recorded + memory.soft_limit_in_bytes # set/show soft limit of memory usage + memory.stat # show various statistics + memory.use_hierarchy # set/show hierarchical account enabled + memory.force_empty # trigger forced move charge to parent + memory.swappiness # set/show swappiness parameter of vmscan + (See sysctl's vm.swappiness) + memory.move_charge_at_immigrate # set/show controls of moving charges + memory.oom_control # set/show oom controls. + 1. History The memory controller has a long history. A request for comments for the memory @@ -106,14 +142,14 @@ the necessary data structures and check if the cgroup that is being charged is over its limit. If it is then reclaim is invoked on the cgroup. More details can be found in the reclaim section of this document. If everything goes well, a page meta-data-structure called page_cgroup is -allocated and associated with the page. This routine also adds the page to -the per cgroup LRU. +updated. page_cgroup has its own LRU on cgroup. +(*) page_cgroup structure is allocated at boot/memory-hotplug time. 2.2.1 Accounting details All mapped anon pages (RSS) and cache pages (Page Cache) are accounted. -(some pages which never be reclaimable and will not be on global LRU - are not accounted. we just accounts pages under usual vm management.) +Some pages which are never reclaimable and will not be on the global LRU +are not accounted. We just account pages under usual VM management. RSS pages are accounted at page_fault unless they've already been accounted for earlier. A file page will be accounted for as Page Cache when it's @@ -121,12 +157,19 @@ inserted into inode (radix-tree). While it's mapped into the page tables of processes, duplicate accounting is carefully avoided. A RSS page is unaccounted when it's fully unmapped. A PageCache page is -unaccounted when it's removed from radix-tree. +unaccounted when it's removed from radix-tree. Even if RSS pages are fully +unmapped (by kswapd), they may exist as SwapCache in the system until they +are really freed. Such SwapCaches also also accounted. +A swapped-in page is not accounted until it's mapped. + +Note: The kernel does swapin-readahead and read multiple swaps at once. +This means swapped-in pages may contain pages for other tasks than a task +causing page fault. So, we avoid accounting at swap-in I/O. At page migration, accounting information is kept. -Note: we just account pages-on-lru because our purpose is to control amount -of used pages. not-on-lru pages are tend to be out-of-control from vm view. +Note: we just account pages-on-LRU because our purpose is to control amount +of used pages; not-on-LRU pages tend to be out-of-control from VM view. 2.3 Shared Page Accounting @@ -143,6 +186,7 @@ caller of swapoff rather than the users of shmem. 2.4 Swap Extension (CONFIG_CGROUP_MEM_RES_CTLR_SWAP) + Swap Extension allows you to record charge for swap. A swapped-in page is charged back to original page allocator if possible. @@ -150,13 +194,20 @@ When swap is accounted, following files are added. - memory.memsw.usage_in_bytes. - memory.memsw.limit_in_bytes. -usage of mem+swap is limited by memsw.limit_in_bytes. +memsw means memory+swap. Usage of memory+swap is limited by +memsw.limit_in_bytes. + +Example: Assume a system with 4G of swap. A task which allocates 6G of memory +(by mistake) under 2G memory limitation will use all swap. +In this case, setting memsw.limit_in_bytes=3G will prevent bad use of swap. +By using memsw limit, you can avoid system OOM which can be caused by swap +shortage. -* why 'mem+swap' rather than swap. +* why 'memory+swap' rather than swap. The global LRU(kswapd) can swap out arbitrary pages. Swap-out means to move account from memory to swap...there is no change in usage of -mem+swap. In other words, when we want to limit the usage of swap without -affecting global LRU, mem+swap limit is better than just limiting swap from +memory+swap. In other words, when we want to limit the usage of swap without +affecting global LRU, memory+swap limit is better than just limiting swap from OS point of view. * What happens when a cgroup hits memory.memsw.limit_in_bytes @@ -168,12 +219,12 @@ it by cgroup. 2.5 Reclaim -Each cgroup maintains a per cgroup LRU that consists of an active -and inactive list. When a cgroup goes over its limit, we first try +Each cgroup maintains a per cgroup LRU which has the same structure as +global VM. When a cgroup goes over its limit, we first try to reclaim memory from the cgroup so as to make space for the new pages that the cgroup has touched. If the reclaim is unsuccessful, an OOM routine is invoked to select and kill the bulkiest task in the -cgroup. +cgroup. (See 10. OOM Control below.) The reclaim algorithm has not been modified for cgroups, except that pages that are selected for reclaiming come from the per cgroup LRU @@ -187,13 +238,19 @@ Note2: When panic_on_oom is set to "2", the whole system will panic. When oom event notifier is registered, event will be delivered. (See oom_control section) -2. Locking +2.6 Locking -The memory controller uses the following hierarchy + lock_page_cgroup()/unlock_page_cgroup() should not be called under + mapping->tree_lock. -1. zone->lru_lock is used for selecting pages to be isolated -2. mem->per_zone->lru_lock protects the per cgroup LRU (per zone) -3. lock_page_cgroup() is used to protect page->page_cgroup + Other lock order is following: + PG_locked. + mm->page_table_lock + zone->lru_lock + lock_page_cgroup. + In many cases, just lock_page_cgroup() is called. + per-zone-per-cgroup LRU (cgroup's private LRU) is just guarded by + zone->lru_lock, it has no lock of its own. 3. User Interface @@ -202,6 +259,7 @@ The memory controller uses the following hierarchy a. Enable CONFIG_CGROUPS b. Enable CONFIG_RESOURCE_COUNTERS c. Enable CONFIG_CGROUP_MEM_RES_CTLR +d. Enable CONFIG_CGROUP_MEM_RES_CTLR_SWAP (to use swap extension) 1. Prepare the cgroups # mkdir -p /cgroups @@ -209,31 +267,28 @@ c. Enable CONFIG_CGROUP_MEM_RES_CTLR 2. Make the new group and move bash into it # mkdir /cgroups/0 -# echo $$ > /cgroups/0/tasks +# echo $$ > /cgroups/0/tasks -Since now we're in the 0 cgroup, -We can alter the memory limit: +Since now we're in the 0 cgroup, we can alter the memory limit: # echo 4M > /cgroups/0/memory.limit_in_bytes NOTE: We can use a suffix (k, K, m, M, g or G) to indicate values in kilo, -mega or gigabytes. +mega or gigabytes. (Here, Kilo, Mega, Giga are Kibibytes, Mebibytes, Gibibytes.) + NOTE: We can write "-1" to reset the *.limit_in_bytes(unlimited). NOTE: We cannot set limits on the root cgroup any more. # cat /cgroups/0/memory.limit_in_bytes 4194304 -NOTE: The interface has now changed to display the usage in bytes -instead of pages - We can check the usage: # cat /cgroups/0/memory.usage_in_bytes 1216512 A successful write to this file does not guarantee a successful set of -this limit to the value written into the file. This can be due to a +this limit to the value written into the file. This can be due to a number of factors, such as rounding up to page boundaries or the total -availability of memory on the system. The user is required to re-read +availability of memory on the system. The user is required to re-read this file after a write to guarantee the value committed by the kernel. # echo 1 > memory.limit_in_bytes @@ -248,15 +303,23 @@ caches, RSS and Active pages/Inactive pages are shown. 4. Testing -Balbir posted lmbench, AIM9, LTP and vmmstress results [10] and [11]. -Apart from that v6 has been tested with several applications and regular -daily use. The controller has also been tested on the PPC64, x86_64 and -UML platforms. +For testing features and implementation, see memcg_test.txt. + +Performance test is also important. To see pure memory controller's overhead, +testing on tmpfs will give you good numbers of small overheads. +Example: do kernel make on tmpfs. + +Page-fault scalability is also important. At measuring parallel +page fault test, multi-process test may be better than multi-thread +test because it has noise of shared objects/status. + +But the above two are testing extreme situations. +Trying usual test under memory controller is always helpful. 4.1 Troubleshooting Sometimes a user might find that the application under a cgroup is -terminated. There are several causes for this: +terminated by OOM killer. There are several causes for this: 1. The cgroup limit is too low (just too low to do anything useful) 2. The user is using anonymous memory and swap is turned off or too low @@ -264,6 +327,9 @@ terminated. There are several causes for this: A sync followed by echo 1 > /proc/sys/vm/drop_caches will help get rid of some of the pages cached in the cgroup (page cache pages). +To know what happens, disable OOM_Kill by 10. OOM Control(see below) and +seeing what happens will be helpful. + 4.2 Task migration When a task migrates from one cgroup to another, its charge is not @@ -271,16 +337,19 @@ carried forward by default. The pages allocated from the original cgroup still remain charged to it, the charge is dropped when the page is freed or reclaimed. -Note: You can move charges of a task along with task migration. See 8. +You can move charges of a task along with task migration. +See 8. "Move charges at task migration" 4.3 Removing a cgroup A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a cgroup might have some charge associated with it, even though all -tasks have migrated away from it. -Such charges are freed(at default) or moved to its parent. When moved, -both of RSS and CACHES are moved to parent. -If both of them are busy, rmdir() returns -EBUSY. See 5.1 Also. +tasks have migrated away from it. (because we charge against pages, not +against tasks.) + +Such charges are freed or moved to their parent. At moving, both of RSS +and CACHES are moved to parent. +rmdir() may return -EBUSY if freeing/moving fails. See 5.1 also. Charges recorded in swap information is not updated at removal of cgroup. Recorded information is discarded and a cgroup which uses swap (swapcache) @@ -296,10 +365,10 @@ will be charged as a new owner of it. # echo 0 > memory.force_empty - Almost all pages tracked by this memcg will be unmapped and freed. Some of - pages cannot be freed because it's locked or in-use. Such pages are moved - to parent and this cgroup will be empty. But this may return -EBUSY in - some too busy case. + Almost all pages tracked by this memory cgroup will be unmapped and freed. + Some pages cannot be freed because they are locked or in-use. Such pages are + moved to parent and this cgroup will be empty. This may return -EBUSY if + VM is too busy to free/move all pages immediately. Typical use case of this interface is that calling this before rmdir(). Because rmdir() moves all pages to parent, some out-of-use page caches can be @@ -309,19 +378,41 @@ will be charged as a new owner of it. memory.stat file includes following statistics +# per-memory cgroup local status cache - # of bytes of page cache memory. rss - # of bytes of anonymous and swap cache memory. +mapped_file - # of bytes of mapped file (includes tmpfs/shmem) pgpgin - # of pages paged in (equivalent to # of charging events). pgpgout - # of pages paged out (equivalent to # of uncharging events). -active_anon - # of bytes of anonymous and swap cache memory on active - lru list. +swap - # of bytes of swap usage inactive_anon - # of bytes of anonymous memory and swap cache memory on - inactive lru list. -active_file - # of bytes of file-backed memory on active lru list. -inactive_file - # of bytes of file-backed memory on inactive lru list. + LRU list. +active_anon - # of bytes of anonymous and swap cache memory on active + inactive LRU list. +inactive_file - # of bytes of file-backed memory on inactive LRU list. +active_file - # of bytes of file-backed memory on active LRU list. unevictable - # of bytes of memory that cannot be reclaimed (mlocked etc). -The following additional stats are dependent on CONFIG_DEBUG_VM. +# status considering hierarchy (see memory.use_hierarchy settings) + +hierarchical_memory_limit - # of bytes of memory limit with regard to hierarchy + under which the memory cgroup is +hierarchical_memsw_limit - # of bytes of memory+swap limit with regard to + hierarchy under which memory cgroup is. + +total_cache - sum of all children's "cache" +total_rss - sum of all children's "rss" +total_mapped_file - sum of all children's "cache" +total_pgpgin - sum of all children's "pgpgin" +total_pgpgout - sum of all children's "pgpgout" +total_swap - sum of all children's "swap" +total_inactive_anon - sum of all children's "inactive_anon" +total_active_anon - sum of all children's "active_anon" +total_inactive_file - sum of all children's "inactive_file" +total_active_file - sum of all children's "active_file" +total_unevictable - sum of all children's "unevictable" + +# The following additional stats are dependent on CONFIG_DEBUG_VM. inactive_ratio - VM internal parameter. (see mm/page_alloc.c) recent_rotated_anon - VM internal parameter. (see mm/vmscan.c) @@ -330,24 +421,37 @@ recent_scanned_anon - VM internal parameter. (see mm/vmscan.c) recent_scanned_file - VM internal parameter. (see mm/vmscan.c) Memo: - recent_rotated means recent frequency of lru rotation. - recent_scanned means recent # of scans to lru. + recent_rotated means recent frequency of LRU rotation. + recent_scanned means recent # of scans to LRU. showing for better debug please see the code for meanings. Note: Only anonymous and swap cache memory is listed as part of 'rss' stat. This should not be confused with the true 'resident set size' or the - amount of physical memory used by the cgroup. Per-cgroup rss - accounting is not done yet. + amount of physical memory used by the cgroup. + 'rss + file_mapped" will give you resident set size of cgroup. + (Note: file and shmem may be shared among other cgroups. In that case, + file_mapped is accounted only when the memory cgroup is owner of page + cache.) 5.3 swappiness - Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only. - Following cgroups' swappiness can't be changed. - - root cgroup (uses /proc/sys/vm/swappiness). - - a cgroup which uses hierarchy and it has child cgroup. - - a cgroup which uses hierarchy and not the root of hierarchy. +Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only. +Following cgroups' swappiness can't be changed. +- root cgroup (uses /proc/sys/vm/swappiness). +- a cgroup which uses hierarchy and it has other cgroup(s) below it. +- a cgroup which uses hierarchy and not the root of hierarchy. + +5.4 failcnt + +A memory cgroup provides memory.failcnt and memory.memsw.failcnt files. +This failcnt(== failure count) shows the number of times that a usage counter +hit its limit. When a memory cgroup hits a limit, failcnt increases and +memory under it will be reclaimed. + +You can reset failcnt by writing 0 to failcnt file. +# echo 0 > .../memory.failcnt 6. Hierarchy support @@ -366,13 +470,13 @@ hierarchy In the diagram above, with hierarchical accounting enabled, all memory usage of e, is accounted to its ancestors up until the root (i.e, c and root), -that has memory.use_hierarchy enabled. If one of the ancestors goes over its +that has memory.use_hierarchy enabled. If one of the ancestors goes over its limit, the reclaim algorithm reclaims from the tasks in the ancestor and the children of the ancestor. 6.1 Enabling hierarchical accounting and reclaim -The memory controller by default disables the hierarchy feature. Support +A memory cgroup by default disables the hierarchy feature. Support can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup # echo 1 > memory.use_hierarchy @@ -382,10 +486,10 @@ The feature can be disabled by # echo 0 > memory.use_hierarchy NOTE1: Enabling/disabling will fail if the cgroup already has other -cgroups created below it. + cgroups created below it. NOTE2: When panic_on_oom is set to "2", the whole system will panic in -case of an oom event in any cgroup. + case of an OOM event in any cgroup. 7. Soft limits @@ -395,7 +499,7 @@ is to allow control groups to use as much of the memory as needed, provided a. There is no memory contention b. They do not exceed their hard limit -When the system detects memory contention or low memory control groups +When the system detects memory contention or low memory, control groups are pushed back to their soft limits. If the soft limit of each control group is very high, they are pushed back as much as possible to make sure that one control group does not starve the others of memory. @@ -409,7 +513,7 @@ it gets invoked from balance_pgdat (kswapd). 7.1 Interface Soft limits can be setup by using the following commands (in this example we -assume a soft limit of 256 megabytes) +assume a soft limit of 256 MiB) # echo 256M > memory.soft_limit_in_bytes @@ -445,7 +549,7 @@ Note: Charges are moved only when you move mm->owner, IOW, a leader of a thread Note: If we cannot find enough space for the task in the destination cgroup, we try to make space by reclaiming memory. Task migration may fail if we cannot make enough space. -Note: It can take several seconds if you move charges in giga bytes order. +Note: It can take several seconds if you move charges much. And if you want disable it again: @@ -465,7 +569,7 @@ memory cgroup. | enable Swap Extension(see 2.4) to enable move of swap charges. -----+------------------------------------------------------------------------ 1 | A charge of file pages(normal file, tmpfs file(e.g. ipc shared memory) - | and swaps of tmpfs file) mmaped by the target task. Unlike the case of + | and swaps of tmpfs file) mmapped by the target task. Unlike the case of | anonymous pages, file pages(and swaps) in the range mmapped by the task | will be moved even if the task hasn't done page fault, i.e. they might | not be the task's "RSS", but other task's "RSS" that maps the same file. @@ -482,15 +586,15 @@ memory cgroup. 9. Memory thresholds -Memory controler implements memory thresholds using cgroups notification +Memory cgroup implements memory thresholds using cgroups notification API (see cgroups.txt). It allows to register multiple memory and memsw thresholds and gets notifications when it crosses. To register a threshold application need: - - create an eventfd using eventfd(2); - - open memory.usage_in_bytes or memory.memsw.usage_in_bytes; - - write string like " " to - cgroup.event_control. +- create an eventfd using eventfd(2); +- open memory.usage_in_bytes or memory.memsw.usage_in_bytes; +- write string like " " to + cgroup.event_control. Application will be notified through eventfd when memory usage crosses threshold in any direction. @@ -501,27 +605,28 @@ It's applicable for root and non-root cgroup. memory.oom_control file is for OOM notification and other controls. -Memory controler implements oom notifier using cgroup notification -API (See cgroups.txt). It allows to register multiple oom notification -delivery and gets notification when oom happens. +Memory cgroup implements OOM notifier using cgroup notification +API (See cgroups.txt). It allows to register multiple OOM notification +delivery and gets notification when OOM happens. To register a notifier, application need: - create an eventfd using eventfd(2) - open memory.oom_control file - - write string like " " to cgroup.event_control + - write string like " " to + cgroup.event_control -Application will be notifier through eventfd when oom happens. +Application will be notified through eventfd when OOM happens. OOM notification doesn't work for root cgroup. -You can disable oom-killer by writing "1" to memory.oom_control file. -As. +You can disable OOM-killer by writing "1" to memory.oom_control file, as: + #echo 1 > memory.oom_control -This operation is only allowed to the top cgroup of subhierarchy. -If oom-killer is disabled, tasks under cgroup will hang/sleep -in memcg's oom-waitq when they request accountable memory. +This operation is only allowed to the top cgroup of sub-hierarchy. +If OOM-killer is disabled, tasks under cgroup will hang/sleep +in memory cgroup's OOM-waitqueue when they request accountable memory. -For running them, you have to relax the memcg's oom sitaution by +For running them, you have to relax the memory cgroup's OOM status by * enlarge limit or reduce usage. To reduce usage, * kill some tasks. @@ -532,7 +637,7 @@ Then, stopped tasks will work again. At reading, current status of OOM is shown. oom_kill_disable 0 or 1 (if 1, oom-killer is disabled) - under_oom 0 or 1 (if 1, the memcg is under OOM,tasks may + under_oom 0 or 1 (if 1, the memory cgroup is under OOM, tasks may be stopped.) 11. TODO -- cgit v1.2.3 From f671be04dcda09f8383ae26072c0e75a79090193 Mon Sep 17 00:00:00 2001 From: FUJITA Tomonori Date: Wed, 26 May 2010 14:44:19 -0700 Subject: Documentation: add networking driver's mapping error handling to DMA-API-HOWTO Adds the concrete DMA mapping error handling for Networking drivers on the transmit path. Signed-off-by: FUJITA Tomonori Acked-by: David S. Miller Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/DMA-API-HOWTO.txt | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'Documentation') diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt index 2e435adfbd6b..93d90870791c 100644 --- a/Documentation/DMA-API-HOWTO.txt +++ b/Documentation/DMA-API-HOWTO.txt @@ -740,6 +740,11 @@ failure can be determined by: */ } +Networking drivers must call dev_kfree_skb to free the socket buffer +and return NETDEV_TX_OK if the DMA mapping fails on the transmit hook +(ndo_start_xmit). This means that the socket buffer is just dropped in +the failure case. + Closing This document, and the API itself, would not be in its current -- cgit v1.2.3 From 7cdcc8e4762f0e4a94cead6a3b57b5eaa01bd5da Mon Sep 17 00:00:00 2001 From: FUJITA Tomonori Date: Wed, 26 May 2010 14:44:21 -0700 Subject: Documentation: add SCSI drivers' mapping error handling to DMA-API-HOWTO Add the concrete DMA mapping error handling for SCSI drivers on the queuecommand path. Signed-off-by: FUJITA Tomonori Cc: James Bottomley Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/DMA-API-HOWTO.txt | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'Documentation') diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt index 93d90870791c..a38ddadd208e 100644 --- a/Documentation/DMA-API-HOWTO.txt +++ b/Documentation/DMA-API-HOWTO.txt @@ -745,6 +745,10 @@ and return NETDEV_TX_OK if the DMA mapping fails on the transmit hook (ndo_start_xmit). This means that the socket buffer is just dropped in the failure case. +SCSI drivers must return SCSI_MLQUEUE_HOST_BUSY if the DMA mapping +fails in the queuecommand hook. This means that the SCSI subsystem +passes the command to the driver again later. + Closing This document, and the API itself, would not be in its current -- cgit v1.2.3 From b02de871617be353c941edce5f7c22bb18499b4e Mon Sep 17 00:00:00 2001 From: FUJITA Tomonori Date: Wed, 26 May 2010 14:44:21 -0700 Subject: Documentation: update scatterlist struct description in DMA-API-HOWTO Now we have . Architectures should use it instead of inventing the own scatterlist struct. Let's update the description. Signed-off-by: FUJITA Tomonori Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/DMA-API-HOWTO.txt | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) (limited to 'Documentation') diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt index a38ddadd208e..c3ab456422f3 100644 --- a/Documentation/DMA-API-HOWTO.txt +++ b/Documentation/DMA-API-HOWTO.txt @@ -703,19 +703,10 @@ to "Closing". 1) Struct scatterlist requirements. - Struct scatterlist must contain, at a minimum, the following - members: - - struct page *page; - unsigned int offset; - unsigned int length; - - The base address is specified by a "page+offset" pair. - - Previous versions of struct scatterlist contained a "void *address" - field that was sometimes used instead of page+offset. As of Linux - 2.5., page+offset is always used, and the "address" field has been - deleted. + Don't invent the architecture specific struct scatterlist; just use + . You need to enable + CONFIG_NEED_SG_DMA_LENGTH if the architecture supports IOMMUs + (including software IOMMU). 2) More to come... -- cgit v1.2.3 From 4ae9ca825e3e28441ef8155c1a81e4c14dfbf38d Mon Sep 17 00:00:00 2001 From: FUJITA Tomonori Date: Wed, 26 May 2010 14:44:22 -0700 Subject: Documentation: move the error handling to the better place in DMA-API-HOWTO Handing DMA mapping errors is essential. Let's put it in the more appropriate place rather than the end of the doc. Signed-off-by: FUJITA Tomonori Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/DMA-API-HOWTO.txt | 60 ++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 30 deletions(-) (limited to 'Documentation') diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt index c3ab456422f3..bfd94e90af51 100644 --- a/Documentation/DMA-API-HOWTO.txt +++ b/Documentation/DMA-API-HOWTO.txt @@ -639,6 +639,36 @@ is planned to completely remove virt_to_bus() and bus_to_virt() as they are entirely deprecated. Some ports already do not provide these as it is impossible to correctly support them. + Handling Errors + +DMA address space is limited on some architectures and an allocation +failure can be determined by: + +- checking if dma_alloc_coherent returns NULL or dma_map_sg returns 0 + +- checking the returned dma_addr_t of dma_map_single and dma_map_page + by using dma_mapping_error(): + + dma_addr_t dma_handle; + + dma_handle = dma_map_single(dev, addr, size, direction); + if (dma_mapping_error(dev, dma_handle)) { + /* + * reduce current DMA mapping usage, + * delay and try again later or + * reset driver. + */ + } + +Networking drivers must call dev_kfree_skb to free the socket buffer +and return NETDEV_TX_OK if the DMA mapping fails on the transmit hook +(ndo_start_xmit). This means that the socket buffer is just dropped in +the failure case. + +SCSI drivers must return SCSI_MLQUEUE_HOST_BUSY if the DMA mapping +fails in the queuecommand hook. This means that the SCSI subsystem +passes the command to the driver again later. + Optimizing Unmap State Space Consumption On many platforms, dma_unmap_{single,page}() is simply a nop. @@ -710,36 +740,6 @@ to "Closing". 2) More to come... - Handling Errors - -DMA address space is limited on some architectures and an allocation -failure can be determined by: - -- checking if dma_alloc_coherent returns NULL or dma_map_sg returns 0 - -- checking the returned dma_addr_t of dma_map_single and dma_map_page - by using dma_mapping_error(): - - dma_addr_t dma_handle; - - dma_handle = dma_map_single(dev, addr, size, direction); - if (dma_mapping_error(dev, dma_handle)) { - /* - * reduce current DMA mapping usage, - * delay and try again later or - * reset driver. - */ - } - -Networking drivers must call dev_kfree_skb to free the socket buffer -and return NETDEV_TX_OK if the DMA mapping fails on the transmit hook -(ndo_start_xmit). This means that the socket buffer is just dropped in -the failure case. - -SCSI drivers must return SCSI_MLQUEUE_HOST_BUSY if the DMA mapping -fails in the queuecommand hook. This means that the SCSI subsystem -passes the command to the driver again later. - Closing This document, and the API itself, would not be in its current -- cgit v1.2.3 From 2fd74e25df46ecb0b54700aba242dcaeb2d75f4a Mon Sep 17 00:00:00 2001 From: FUJITA Tomonori Date: Wed, 26 May 2010 14:44:23 -0700 Subject: Documentation/DMA-API-HOWTO: add ARCH_KMALLOC_MINALIGN description Add ARCH_KMALLOC_MINALIGN description in "Platform Issues" section. Signed-off-by: FUJITA Tomonori Cc: "David S. Miller" Cc: Matt Mackall Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/DMA-API-HOWTO.txt | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt index bfd94e90af51..98ce51796f71 100644 --- a/Documentation/DMA-API-HOWTO.txt +++ b/Documentation/DMA-API-HOWTO.txt @@ -738,7 +738,20 @@ to "Closing". CONFIG_NEED_SG_DMA_LENGTH if the architecture supports IOMMUs (including software IOMMU). -2) More to come... +2) ARCH_KMALLOC_MINALIGN + + Architectures must ensure that kmalloc'ed buffer is + DMA-safe. Drivers and subsystems depend on it. If an architecture + isn't fully DMA-coherent (i.e. hardware doesn't ensure that data in + the CPU cache is identical to data in main memory), + ARCH_KMALLOC_MINALIGN must be set so that the memory allocator + makes sure that kmalloc'ed buffer doesn't share a cache line with + the others. See arch/arm/include/asm/cache.h as an example. + + Note that ARCH_KMALLOC_MINALIGN is about DMA memory alignment + constraints. You don't need to worry about the architecture data + alignment constraints (e.g. the alignment constraints about 64-bit + objects). Closing -- cgit v1.2.3 From 866707fc2721df8fee637fcf0239628b9231f9ea Mon Sep 17 00:00:00 2001 From: Jan Blunck Date: Wed, 26 May 2010 14:44:54 -0700 Subject: Documentation/filesystems/Locking: update documentation on llseek() wrt BKL The inode's i_size is not protected by the big kernel lock. Therefore it does not make sense to recommend taking the BKL in filesystems llseek operations. Instead it should use the inode's mutex or use just use i_size_read() instead. Add a note that this is not protecting file->f_pos. Signed-off-by: Jan Blunck Acked-by: Alan Cox Cc: Arnd Bergmann Cc: Christoph Hellwig Cc: Frederic Weisbecker Cc: John Kacur Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/filesystems/Locking | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/Locking b/Documentation/filesystems/Locking index af1608070cd5..61c98f03baa1 100644 --- a/Documentation/filesystems/Locking +++ b/Documentation/filesystems/Locking @@ -429,8 +429,9 @@ check_flags: no implementations. If your fs is not using generic_file_llseek, you need to acquire and release the appropriate locks in your ->llseek(). For many filesystems, it is probably safe to acquire the inode -mutex. Note some filesystems (i.e. remote ones) provide no -protection for i_size so you will need to use the BKL. +mutex or just to use i_size_read() instead. +Note: this does not protect the file->f_pos against concurrent modifications +since this is something the userspace has to take care about. Note: ext2_release() was *the* source of contention on fs-intensive loads and dropping BKL on ->release() helps to get rid of that (we still -- cgit v1.2.3 From b9498bfe86530fd54fb855906383c0c905a52c80 Mon Sep 17 00:00:00 2001 From: Lee Schermerhorn Date: Wed, 26 May 2010 14:45:06 -0700 Subject: numa: update Documentation/vm/numa, add memoryless node info Kamezawa Hiroyuki requested documentation for the numa_mem_id() and slab related changes. He suggested Documentation/vm/numa for this documentation. Looking at this file, it seems to me to be hopelessly out of date relative to current Linux NUMA support. At the risk of going down a rathole, I have made an attempt to rewrite the doc at a slightly higher level [I think] and provide pointers to other in-tree documents and out-of-tree man pages that cover the details. Let the games begin. Signed-off-by: Lee Schermerhorn Cc: Tejun Heo Cc: Mel Gorman Cc: Christoph Lameter Cc: Nick Piggin Cc: David Rientjes Cc: Eric Whitney Cc: KAMEZAWA Hiroyuki Cc: Ingo Molnar Cc: Thomas Gleixner Cc: "H. Peter Anvin" Cc: "Luck, Tony" Cc: Pekka Enberg Cc: Randy Dunlap Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- Documentation/vm/numa | 186 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 147 insertions(+), 39 deletions(-) (limited to 'Documentation') diff --git a/Documentation/vm/numa b/Documentation/vm/numa index e93ad9425e2a..a200a386429d 100644 --- a/Documentation/vm/numa +++ b/Documentation/vm/numa @@ -1,41 +1,149 @@ Started Nov 1999 by Kanoj Sarcar -The intent of this file is to have an uptodate, running commentary -from different people about NUMA specific code in the Linux vm. - -What is NUMA? It is an architecture where the memory access times -for different regions of memory from a given processor varies -according to the "distance" of the memory region from the processor. -Each region of memory to which access times are the same from any -cpu, is called a node. On such architectures, it is beneficial if -the kernel tries to minimize inter node communications. Schemes -for this range from kernel text and read-only data replication -across nodes, and trying to house all the data structures that -key components of the kernel need on memory on that node. - -Currently, all the numa support is to provide efficient handling -of widely discontiguous physical memory, so architectures which -are not NUMA but can have huge holes in the physical address space -can use the same code. All this code is bracketed by CONFIG_DISCONTIGMEM. - -The initial port includes NUMAizing the bootmem allocator code by -encapsulating all the pieces of information into a bootmem_data_t -structure. Node specific calls have been added to the allocator. -In theory, any platform which uses the bootmem allocator should -be able to put the bootmem and mem_map data structures anywhere -it deems best. - -Each node's page allocation data structures have also been encapsulated -into a pg_data_t. The bootmem_data_t is just one part of this. To -make the code look uniform between NUMA and regular UMA platforms, -UMA platforms have a statically allocated pg_data_t too (contig_page_data). -For the sake of uniformity, the function num_online_nodes() is also defined -for all platforms. As we run benchmarks, we might decide to NUMAize -more variables like low_on_memory, nr_free_pages etc into the pg_data_t. - -The NUMA aware page allocation code currently tries to allocate pages -from different nodes in a round robin manner. This will be changed to -do concentratic circle search, starting from current node, once the -NUMA port achieves more maturity. The call alloc_pages_node has been -added, so that drivers can make the call and not worry about whether -it is running on a NUMA or UMA platform. +What is NUMA? + +This question can be answered from a couple of perspectives: the +hardware view and the Linux software view. + +From the hardware perspective, a NUMA system is a computer platform that +comprises multiple components or assemblies each of which may contain 0 +or more CPUs, local memory, and/or IO buses. For brevity and to +disambiguate the hardware view of these physical components/assemblies +from the software abstraction thereof, we'll call the components/assemblies +'cells' in this document. + +Each of the 'cells' may be viewed as an SMP [symmetric multi-processor] subset +of the system--although some components necessary for a stand-alone SMP system +may not be populated on any given cell. The cells of the NUMA system are +connected together with some sort of system interconnect--e.g., a crossbar or +point-to-point link are common types of NUMA system interconnects. Both of +these types of interconnects can be aggregated to create NUMA platforms with +cells at multiple distances from other cells. + +For Linux, the NUMA platforms of interest are primarily what is known as Cache +Coherent NUMA or ccNUMA systems. With ccNUMA systems, all memory is visible +to and accessible from any CPU attached to any cell and cache coherency +is handled in hardware by the processor caches and/or the system interconnect. + +Memory access time and effective memory bandwidth varies depending on how far +away the cell containing the CPU or IO bus making the memory access is from the +cell containing the target memory. For example, access to memory by CPUs +attached to the same cell will experience faster access times and higher +bandwidths than accesses to memory on other, remote cells. NUMA platforms +can have cells at multiple remote distances from any given cell. + +Platform vendors don't build NUMA systems just to make software developers' +lives interesting. Rather, this architecture is a means to provide scalable +memory bandwidth. However, to achieve scalable memory bandwidth, system and +application software must arrange for a large majority of the memory references +[cache misses] to be to "local" memory--memory on the same cell, if any--or +to the closest cell with memory. + +This leads to the Linux software view of a NUMA system: + +Linux divides the system's hardware resources into multiple software +abstractions called "nodes". Linux maps the nodes onto the physical cells +of the hardware platform, abstracting away some of the details for some +architectures. As with physical cells, software nodes may contain 0 or more +CPUs, memory and/or IO buses. And, again, memory accesses to memory on +"closer" nodes--nodes that map to closer cells--will generally experience +faster access times and higher effective bandwidth than accesses to more +remote cells. + +For some architectures, such as x86, Linux will "hide" any node representing a +physical cell that has no memory attached, and reassign any CPUs attached to +that cell to a node representing a cell that does have memory. Thus, on +these architectures, one cannot assume that all CPUs that Linux associates with +a given node will see the same local memory access times and bandwidth. + +In addition, for some architectures, again x86 is an example, Linux supports +the emulation of additional nodes. For NUMA emulation, linux will carve up +the existing nodes--or the system memory for non-NUMA platforms--into multiple +nodes. Each emulated node will manage a fraction of the underlying cells' +physical memory. NUMA emluation is useful for testing NUMA kernel and +application features on non-NUMA platforms, and as a sort of memory resource +management mechanism when used together with cpusets. +[see Documentation/cgroups/cpusets.txt] + +For each node with memory, Linux constructs an independent memory management +subsystem, complete with its own free page lists, in-use page lists, usage +statistics and locks to mediate access. In addition, Linux constructs for +each memory zone [one or more of DMA, DMA32, NORMAL, HIGH_MEMORY, MOVABLE], +an ordered "zonelist". A zonelist specifies the zones/nodes to visit when a +selected zone/node cannot satisfy the allocation request. This situation, +when a zone has no available memory to satisfy a request, is called +"overflow" or "fallback". + +Because some nodes contain multiple zones containing different types of +memory, Linux must decide whether to order the zonelists such that allocations +fall back to the same zone type on a different node, or to a different zone +type on the same node. This is an important consideration because some zones, +such as DMA or DMA32, represent relatively scarce resources. Linux chooses +a default zonelist order based on the sizes of the various zone types relative +to the total memory of the node and the total memory of the system. The +default zonelist order may be overridden using the numa_zonelist_order kernel +boot parameter or sysctl. [see Documentation/kernel-parameters.txt and +Documentation/sysctl/vm.txt] + +By default, Linux will attempt to satisfy memory allocation requests from the +node to which the CPU that executes the request is assigned. Specifically, +Linux will attempt to allocate from the first node in the appropriate zonelist +for the node where the request originates. This is called "local allocation." +If the "local" node cannot satisfy the request, the kernel will examine other +nodes' zones in the selected zonelist looking for the first zone in the list +that can satisfy the request. + +Local allocation will tend to keep subsequent access to the allocated memory +"local" to the underlying physical resources and off the system interconnect-- +as long as the task on whose behalf the kernel allocated some memory does not +later migrate away from that memory. The Linux scheduler is aware of the +NUMA topology of the platform--embodied in the "scheduling domains" data +structures [see Documentation/scheduler/sched-domains.txt]--and the scheduler +attempts to minimize task migration to distant scheduling domains. However, +the scheduler does not take a task's NUMA footprint into account directly. +Thus, under sufficient imbalance, tasks can migrate between nodes, remote +from their initial node and kernel data structures. + +System administrators and application designers can restrict a task's migration +to improve NUMA locality using various CPU affinity command line interfaces, +such as taskset(1) and numactl(1), and program interfaces such as +sched_setaffinity(2). Further, one can modify the kernel's default local +allocation behavior using Linux NUMA memory policy. +[see Documentation/vm/numa_memory_policy.] + +System administrators can restrict the CPUs and nodes' memories that a non- +privileged user can specify in the scheduling or NUMA commands and functions +using control groups and CPUsets. [see Documentation/cgroups/CPUsets.txt] + +On architectures that do not hide memoryless nodes, Linux will include only +zones [nodes] with memory in the zonelists. This means that for a memoryless +node the "local memory node"--the node of the first zone in CPU's node's +zonelist--will not be the node itself. Rather, it will be the node that the +kernel selected as the nearest node with memory when it built the zonelists. +So, default, local allocations will succeed with the kernel supplying the +closest available memory. This is a consequence of the same mechanism that +allows such allocations to fallback to other nearby nodes when a node that +does contain memory overflows. + +Some kernel allocations do not want or cannot tolerate this allocation fallback +behavior. Rather they want to be sure they get memory from the specified node +or get notified that the node has no free memory. This is usually the case when +a subsystem allocates per CPU memory resources, for example. + +A typical model for making such an allocation is to obtain the node id of the +node to which the "current CPU" is attached using one of the kernel's +numa_node_id() or CPU_to_node() functions and then request memory from only +the node id returned. When such an allocation fails, the requesting subsystem +may revert to its own fallback path. The slab kernel memory allocator is an +example of this. Or, the subsystem may choose to disable or not to enable +itself on allocation failure. The kernel profiling subsystem is an example of +this. + +If the architecture supports--does not hide--memoryless nodes, then CPUs +attached to memoryless nodes would always incur the fallback path overhead +or some subsystems would fail to initialize if they attempted to allocated +memory exclusively from a node without memory. To support such +architectures transparently, kernel subsystems can use the numa_mem_id() +or cpu_to_mem() function to locate the "local memory node" for the calling or +specified CPU. Again, this is the same node from which default, local page +allocations will be attempted. -- cgit v1.2.3 From dce80a56268fffd6b5ea57b3f6ba3d027a68f05e Mon Sep 17 00:00:00 2001 From: Feng Tang Date: Wed, 26 May 2010 11:28:08 +0800 Subject: SFI: add sysfs interface for SFI tables. Analogous to ACPI's /sys/firmware/acpi/tables/... create /sys/firmware/sfi/tables/ The tables are primariy for the kernel, but sometimes it is useful for user-space to be able to read them. Signed-off-by: Feng Tang Signed-off-by: Len Brown --- Documentation/ABI/testing/sysfs-firmware-sfi | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-firmware-sfi (limited to 'Documentation') diff --git a/Documentation/ABI/testing/sysfs-firmware-sfi b/Documentation/ABI/testing/sysfs-firmware-sfi new file mode 100644 index 000000000000..4be7d44aeacf --- /dev/null +++ b/Documentation/ABI/testing/sysfs-firmware-sfi @@ -0,0 +1,15 @@ +What: /sys/firmware/sfi/tables/ +Date: May 2010 +Contact: Len Brown +Description: + SFI defines a number of small static memory tables + so the kernel can get platform information from firmware. + + The tables are defined in the latest SFI specification: + http://simplefirmware.org/documentation + + While the tables are used by the kernel, user-space + can observe them this way: + + # cd /sys/firmware/sfi/tables + # cat $TABLENAME > $TABLENAME.bin -- cgit v1.2.3 From 10f2ed31aae11040dfd64cee10c47db79b4b4647 Mon Sep 17 00:00:00 2001 From: Matthew Garrett Date: Thu, 27 May 2010 19:58:38 +0200 Subject: hwmon: (lm63) Add basic support for LM64 The LM64 appears to be an LM63 with added GPIO lines. Add support for the hwmon functionality - GPIO can be added at some later stage if someone has a need for them. Signed-off-by: Matthew Garrett Signed-off-by: Jean Delvare --- Documentation/hwmon/lm63 | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'Documentation') diff --git a/Documentation/hwmon/lm63 b/Documentation/hwmon/lm63 index 31660bf97979..b9843eab1afb 100644 --- a/Documentation/hwmon/lm63 +++ b/Documentation/hwmon/lm63 @@ -7,6 +7,11 @@ Supported chips: Addresses scanned: I2C 0x4c Datasheet: Publicly available at the National Semiconductor website http://www.national.com/pf/LM/LM63.html + * National Semiconductor LM64 + Prefix: 'lm64' + Addresses scanned: I2C 0x18 and 0x4e + Datasheet: Publicly available at the National Semiconductor website + http://www.national.com/pf/LM/LM64.html Author: Jean Delvare @@ -55,3 +60,5 @@ The lm63 driver will not update its values more frequently than every second; reading them more often will do no harm, but will return 'old' values. +The LM64 is effectively an LM63 with GPIO lines. The driver does not +support these GPIO lines at present. -- cgit v1.2.3 From d2b847d489ee4e9921bc61f47c3d0e992692fc0f Mon Sep 17 00:00:00 2001 From: "Ira W. Snyder" Date: Thu, 27 May 2010 19:58:45 +0200 Subject: hwmon: Add description of the update_rate sysfs attribute The update_rate attribute can be used by drivers to let userspace choose the update rate of the chip, if it is configurable. Signed-off-by: Ira W. Snyder Signed-off-by: Jean Delvare --- Documentation/hwmon/sysfs-interface | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/hwmon/sysfs-interface b/Documentation/hwmon/sysfs-interface index 3de6b0bcb147..d4e2917c6f18 100644 --- a/Documentation/hwmon/sysfs-interface +++ b/Documentation/hwmon/sysfs-interface @@ -80,9 +80,9 @@ All entries (except name) are optional, and should only be created in a given driver if the chip has the feature. -******** -* Name * -******** +********************* +* Global attributes * +********************* name The chip name. This should be a short, lowercase string, not containing @@ -91,6 +91,13 @@ name The chip name. I2C devices get this attribute created automatically. RO +update_rate The rate at which the chip will update readings. + Unit: millisecond + RW + Some devices have a variable update rate. This attribute + can be used to change the update rate to the desired + frequency. + ************ * Voltages * -- cgit v1.2.3 From beb1b6bbf2b448b97b9611200eb4b5a555336c60 Mon Sep 17 00:00:00 2001 From: Steven King Date: Thu, 27 May 2010 19:58:56 +0200 Subject: hwmon: Driver for TI TMP102 temperature sensor Driver for the TI TMP102. The TI TMP102 is similar to the LM75. It differs from the LM75 by having a 16-bit conf register and the temp registers have a minimum resolution of 12 bits; the extended conf register can select 13-bit resolution (which this driver does) and also change the update rate (which this driver currently doesn't use). [JD: Fix tmp102_exit tag, must be __exit, not __init.] Signed-off-by: Steven King Signed-off-by: Jean Delvare --- Documentation/hwmon/tmp102 | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 Documentation/hwmon/tmp102 (limited to 'Documentation') diff --git a/Documentation/hwmon/tmp102 b/Documentation/hwmon/tmp102 new file mode 100644 index 000000000000..239dded8539f --- /dev/null +++ b/Documentation/hwmon/tmp102 @@ -0,0 +1,27 @@ +Kernel driver tmp102 +==================== + +Supported chips: + * Texas Instruments TMP102 + Prefix: 'tmp102' + Addresses scanned: I2C 0x48 0x49 0x4a 0x4b + Datasheet: http://focus.ti.com/docs/prod/folders/print/tmp102.html + +Author: + Steven King + +Description +----------- + +The Texas Instruments TMP102 implements one temperature sensor. Limits can be +set through the Overtemperature Shutdown register and Hysteresis register. The +sensor is accurate to 0.5 degrees over the range of -25 to +85 C, and to 1.0 +degrees from -40 to +125 C. Resolution of the sensor is 0.0625 degree. The +operating temperature has a minimum of -55 C and a maximum of +150 C. + +The TMP102 has a programmable update rate that can select between 8, 4, 1, and +0.5 Hz. (Currently the driver only supports the default of 4 Hz). + +The driver provides the common sysfs-interface for temperatures (see +/Documentation/hwmon/sysfs-interface under Temperatures). + -- cgit v1.2.3 From cff37c9e82e022068840b3d33167e64c6a0ecc06 Mon Sep 17 00:00:00 2001 From: Jean Delvare Date: Thu, 27 May 2010 19:58:57 +0200 Subject: hwmon: (tmp102) Various fixes Fixes from my driver review: http://lists.lm-sensors.org/pipermail/lm-sensors/2010-March/028051.html Only the small changes are in there, more important changes will come later separately as time permits. * Drop the remnants of the now gone detect function * The TMP102 has no known compatible chip * Include the right header files * Clarify why byte swapping of register values is needed * Strip resolution info bit from temperature register value * Set cache lifetime to 1/3 second * Don't arbitrarily reject limit values; clamp as needed * Make limit writing unconditional * Don't check for transaction types the driver doesn't use * Properly check for error when setting configuration * Report error on failed probe * Make the driver load automatically where needed * Various other minor fixes Signed-off-by: Jean Delvare Cc: Steven King --- Documentation/hwmon/tmp102 | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/hwmon/tmp102 b/Documentation/hwmon/tmp102 index 239dded8539f..8454a7763122 100644 --- a/Documentation/hwmon/tmp102 +++ b/Documentation/hwmon/tmp102 @@ -4,7 +4,7 @@ Kernel driver tmp102 Supported chips: * Texas Instruments TMP102 Prefix: 'tmp102' - Addresses scanned: I2C 0x48 0x49 0x4a 0x4b + Addresses scanned: none Datasheet: http://focus.ti.com/docs/prod/folders/print/tmp102.html Author: @@ -15,13 +15,12 @@ Description The Texas Instruments TMP102 implements one temperature sensor. Limits can be set through the Overtemperature Shutdown register and Hysteresis register. The -sensor is accurate to 0.5 degrees over the range of -25 to +85 C, and to 1.0 -degrees from -40 to +125 C. Resolution of the sensor is 0.0625 degree. The +sensor is accurate to 0.5 degree over the range of -25 to +85 C, and to 1.0 +degree from -40 to +125 C. Resolution of the sensor is 0.0625 degree. The operating temperature has a minimum of -55 C and a maximum of +150 C. The TMP102 has a programmable update rate that can select between 8, 4, 1, and 0.5 Hz. (Currently the driver only supports the default of 4 Hz). The driver provides the common sysfs-interface for temperatures (see -/Documentation/hwmon/sysfs-interface under Temperatures). - +Documentation/hwmon/sysfs-interface under Temperatures). -- cgit v1.2.3 From ea694431f9c862bd409c90ba1cb3bdc6fdde8635 Mon Sep 17 00:00:00 2001 From: Juerg Haefliger Date: Thu, 27 May 2010 19:59:01 +0200 Subject: hwmon: (dme1737) Add SCH5127 support Add support for the hardware monitoring capabilities of the SCH5127 chip to the dme1737 driver. Signed-off-by: Juerg Haefliger Signed-off-by: Jean Delvare Tested-by: Jeff Rickman --- Documentation/hwmon/dme1737 | 51 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 10 deletions(-) (limited to 'Documentation') diff --git a/Documentation/hwmon/dme1737 b/Documentation/hwmon/dme1737 index 001d2e70bc11..fc5df7654d63 100644 --- a/Documentation/hwmon/dme1737 +++ b/Documentation/hwmon/dme1737 @@ -9,11 +9,15 @@ Supported chips: * SMSC SCH3112, SCH3114, SCH3116 Prefix: 'sch311x' Addresses scanned: none, address read from Super-I/O config space - Datasheet: http://www.nuhorizons.com/FeaturedProducts/Volume1/SMSC/311x.pdf + Datasheet: Available on the Internet * SMSC SCH5027 Prefix: 'sch5027' Addresses scanned: I2C 0x2c, 0x2d, 0x2e Datasheet: Provided by SMSC upon request and under NDA + * SMSC SCH5127 + Prefix: 'sch5127' + Addresses scanned: none, address read from Super-I/O config space + Datasheet: Provided by SMSC upon request and under NDA Authors: Juerg Haefliger @@ -36,8 +40,8 @@ Description ----------- This driver implements support for the hardware monitoring capabilities of the -SMSC DME1737 and Asus A8000 (which are the same), SMSC SCH5027, and SMSC -SCH311x Super-I/O chips. These chips feature monitoring of 3 temp sensors +SMSC DME1737 and Asus A8000 (which are the same), SMSC SCH5027, SCH311x, +and SCH5127 Super-I/O chips. These chips feature monitoring of 3 temp sensors temp[1-3] (2 remote diodes and 1 internal), 7 voltages in[0-6] (6 external and 1 internal) and up to 6 fan speeds fan[1-6]. Additionally, the chips implement up to 5 PWM outputs pwm[1-3,5-6] for controlling fan speeds both manually and @@ -48,14 +52,14 @@ Fan[3-6] and pwm[3,5-6] are optional features and their availability depends on the configuration of the chip. The driver will detect which features are present during initialization and create the sysfs attributes accordingly. -For the SCH311x, fan[1-3] and pwm[1-3] are always present and fan[4-6] and -pwm[5-6] don't exist. +For the SCH311x and SCH5127, fan[1-3] and pwm[1-3] are always present and +fan[4-6] and pwm[5-6] don't exist. The hardware monitoring features of the DME1737, A8000, and SCH5027 are only -accessible via SMBus, while the SCH311x only provides access via the ISA bus. -The driver will therefore register itself as an I2C client driver if it detects -a DME1737, A8000, or SCH5027 and as a platform driver if it detects a SCH311x -chip. +accessible via SMBus, while the SCH311x and SCH5127 only provide access via +the ISA bus. The driver will therefore register itself as an I2C client driver +if it detects a DME1737, A8000, or SCH5027 and as a platform driver if it +detects a SCH311x or SCH5127 chip. Voltage Monitoring @@ -76,7 +80,7 @@ DME1737, A8000: in6: Vbat (+3.0V) 0V - 4.38V SCH311x: - in0: +2.5V 0V - 6.64V + in0: +2.5V 0V - 3.32V in1: Vccp (processor core) 0V - 2V in2: VCC (internal +3.3V) 0V - 4.38V in3: +5V 0V - 6.64V @@ -93,6 +97,15 @@ SCH5027: in5: VTR (+3.3V standby) 0V - 4.38V in6: Vbat (+3.0V) 0V - 4.38V +SCH5127: + in0: +2.5 0V - 3.32V + in1: Vccp (processor core) 0V - 3V + in2: VCC (internal +3.3V) 0V - 4.38V + in3: V2_IN 0V - 1.5V + in4: V1_IN 0V - 1.5V + in5: VTR (+3.3V standby) 0V - 4.38V + in6: Vbat (+3.0V) 0V - 4.38V + Each voltage input has associated min and max limits which trigger an alarm when crossed. @@ -293,3 +306,21 @@ pwm[1-3]_auto_point1_pwm RW Auto PWM pwm point. Auto_point1 is the pwm[1-3]_auto_point2_pwm RO Auto PWM pwm point. Auto_point2 is the full-speed duty-cycle which is hard- wired to 255 (100% duty-cycle). + +Chip Differences +---------------- + +Feature dme1737 sch311x sch5027 sch5127 +------------------------------------------------------- +temp[1-3]_offset yes yes +vid yes +zone3 yes yes yes +zone[1-3]_hyst yes yes +pwm min/off yes yes +fan3 opt yes opt yes +pwm3 opt yes opt yes +fan4 opt opt +fan5 opt opt +pwm5 opt opt +fan6 opt opt +pwm6 opt opt -- cgit v1.2.3 From df16dd53c575d0cb9dbee20a3149927c862a9ff6 Mon Sep 17 00:00:00 2001 From: "Ira W. Snyder" Date: Thu, 27 May 2010 19:59:02 +0200 Subject: hwmon: (ltc4245) Read only one GPIO pin Read only one of the GPIO pins as an analog voltage. The ADC can be switched to a different GPIO pin at runtime, but this is not supported. Previously, this driver would report the analog voltage of the currently selected GPIO pin as all three GPIO voltages: in9_input, in10_input and in11_input. Signed-off-by: Ira W. Snyder Signed-off-by: Jean Delvare Cc: stable@kernel.org --- Documentation/hwmon/ltc4245 | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/hwmon/ltc4245 b/Documentation/hwmon/ltc4245 index 02838a47d862..86b5880d8502 100644 --- a/Documentation/hwmon/ltc4245 +++ b/Documentation/hwmon/ltc4245 @@ -72,9 +72,7 @@ in6_min_alarm 5v output undervoltage alarm in7_min_alarm 3v output undervoltage alarm in8_min_alarm Vee (-12v) output undervoltage alarm -in9_input GPIO #1 voltage data -in10_input GPIO #2 voltage data -in11_input GPIO #3 voltage data +in9_input GPIO voltage data power1_input 12v power usage (mW) power2_input 5v power usage (mW) -- cgit v1.2.3 From 7ea8085910ef3dd4f3cad6845aaa2b580d39b115 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig Date: Wed, 26 May 2010 17:53:25 +0200 Subject: drop unused dentry argument to ->fsync Signed-off-by: Christoph Hellwig Signed-off-by: Al Viro --- Documentation/filesystems/Locking | 2 +- Documentation/filesystems/vfs.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/Locking b/Documentation/filesystems/Locking index 61c98f03baa1..96d4293607ec 100644 --- a/Documentation/filesystems/Locking +++ b/Documentation/filesystems/Locking @@ -380,7 +380,7 @@ prototypes: int (*open) (struct inode *, struct file *); int (*flush) (struct file *); int (*release) (struct inode *, struct file *); - int (*fsync) (struct file *, struct dentry *, int datasync); + int (*fsync) (struct file *, int datasync); int (*aio_fsync) (struct kiocb *, int datasync); int (*fasync) (int, struct file *, int); int (*lock) (struct file *, int, struct file_lock *); diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index b66858538df5..d4f5731dcbbb 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -729,7 +729,7 @@ struct file_operations { int (*open) (struct inode *, struct file *); int (*flush) (struct file *); int (*release) (struct inode *, struct file *); - int (*fsync) (struct file *, struct dentry *, int datasync); + int (*fsync) (struct file *, int datasync); int (*aio_fsync) (struct kiocb *, int datasync); int (*fasync) (int, struct file *, int); int (*lock) (struct file *, int, struct file_lock *); -- cgit v1.2.3 From 7bb46a6734a7e1ad4beaecc11cae7ed3ff81d30f Mon Sep 17 00:00:00 2001 From: "npiggin@suse.de" Date: Thu, 27 May 2010 01:05:33 +1000 Subject: fs: introduce new truncate sequence Introduce a new truncate calling sequence into fs/mm subsystems. Rather than setattr > vmtruncate > truncate, have filesystems call their truncate sequence from ->setattr if filesystem specific operations are required. vmtruncate is deprecated, and truncate_pagecache and inode_newsize_ok helpers introduced previously should be used. simple_setattr is introduced for simple in-ram filesystems to implement the new truncate sequence. Eventually all filesystems should be converted to implement a setattr, and the default code in notify_change should go away. simple_setsize is also introduced to perform just the ATTR_SIZE portion of simple_setattr (ie. changing i_size and trimming pagecache). To implement the new truncate sequence: - filesystem specific manipulations (eg freeing blocks) must be done in the setattr method rather than ->truncate. - vmtruncate can not be used by core code to trim blocks past i_size in the event of write failure after allocation, so this must be performed in the fs code. - convert usage of helpers block_write_begin, nobh_write_begin, cont_write_begin, and *blockdev_direct_IO* to use _newtrunc postfixed variants. These avoid calling vmtruncate to trim blocks (see previous). - inode_setattr should not be used. generic_setattr is a new function to be used to copy simple attributes into the generic inode. - make use of the better opportunity to handle errors with the new sequence. Big problem with the previous calling sequence: the filesystem is not called until i_size has already changed. This means it is not allowed to fail the call, and also it does not know what the previous i_size was. Also, generic code calling vmtruncate to truncate allocated blocks in case of error had no good way to return a meaningful error (or, for example, atomically handle block deallocation). Cc: Christoph Hellwig Acked-by: Jan Kara Signed-off-by: Nick Piggin Signed-off-by: Al Viro --- Documentation/filesystems/vfs.txt | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index d4f5731dcbbb..94677e7dcb13 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -401,11 +401,16 @@ otherwise noted. started might not be in the page cache at the end of the walk). - truncate: called by the VFS to change the size of a file. The + truncate: Deprecated. This will not be called if ->setsize is defined. + Called by the VFS to change the size of a file. The i_size field of the inode is set to the desired size by the VFS before this method is called. This method is called by the truncate(2) system call and related functionality. + Note: ->truncate and vmtruncate are deprecated. Do not add new + instances/calls of these. Filesystems should be converted to do their + truncate sequence via ->setattr(). + permission: called by the VFS to check for access rights on a POSIX-like filesystem. -- cgit v1.2.3 From 1378a7efe880b3b59a899eb60e9acae270b788d8 Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 10:33:50 +0900 Subject: ARM: SAMSUNG: Documentation: update the list of SoCs supported Update the list of SoCs supported for the plat-s5p range. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung/Overview.txt | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/arm/Samsung/Overview.txt b/Documentation/arm/Samsung/Overview.txt index 7cced1fea9c3..e8c5dfcd0e3b 100644 --- a/Documentation/arm/Samsung/Overview.txt +++ b/Documentation/arm/Samsung/Overview.txt @@ -13,9 +13,10 @@ Introduction - S3C24XX: See Documentation/arm/Samsung-S3C24XX/Overview.txt for full list - S3C64XX: S3C6400 and S3C6410 - - S5PC6440 - - S5PC100 and S5PC110 support is currently being merged + - S5P6440 + - S5P6442 + - S5PC100 + - S5PC110 / S5PV210 S3C24XX Systems @@ -35,7 +36,10 @@ Configuration unifying all the SoCs into one kernel. s5p6440_defconfig - S5P6440 specific default configuration + s5p6442_defconfig - S5P6442 specific default configuration s5pc100_defconfig - S5PC100 specific default configuration + s5pc110_defconfig - S5PC110 specific default configuration + s5pv210_defconfig - S5PV210 specific default configuration Layout -- cgit v1.2.3 From 1b03ef462b5e87f7669d2ef5ccc84e7cf3801b35 Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 10:48:02 +0900 Subject: ARM: SAMSUNG: Documentation: update directory layout Update the directory layout in Documentation/arm/Samsung/Overview.txt to reflect the changes that have been made in the latest kernel updates. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung/Overview.txt | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) (limited to 'Documentation') diff --git a/Documentation/arm/Samsung/Overview.txt b/Documentation/arm/Samsung/Overview.txt index e8c5dfcd0e3b..c3094ea51aa7 100644 --- a/Documentation/arm/Samsung/Overview.txt +++ b/Documentation/arm/Samsung/Overview.txt @@ -54,18 +54,27 @@ Layout specific information. It contains the base clock, GPIO and device definitions to get the system running. - plat-s3c is the s3c24xx/s3c64xx platform directory, although it is currently - involved in other builds this will be phased out once the relevant code is - moved elsewhere. - plat-s3c24xx is for s3c24xx specific builds, see the S3C24XX docs. - plat-s3c64xx is for the s3c64xx specific bits, see the S3C24XX docs. + plat-s5p is for s5p specific builds, and contains common support for the + S5P specific systems. Not all S5Ps use all the features in this directory + due to differences in the hardware. + + +Layout changes +-------------- + + The old plat-s3c and plat-s5pc1xx directories have been removed, with + support moved to either plat-samsung or plat-s5p as necessary. These moves + where to simplify the include and dependency issues involved with having + so many different platform directories. - plat-s5p is for s5p specific builds, more to be added. + It was decided to remove plat-s5pc1xx as some of the support was already + in plat-s5p or plat-samsung, with the S5PC110 support added with S5PV210 + the only user was the S5PC100. The S5PC100 specific items where moved to + arch/arm/mach-s5pc100. - [ to finish ] Port Contributors -- cgit v1.2.3 From b9f141acf2a6b456e262bc54fd69c19d76c319d3 Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 10:53:06 +0900 Subject: ARM: S3C24XX: Documentation: update documentation overview Add the two new SoCs added in this release cycle and update the notes on the gpiolib conversion. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung-S3C24XX/Overview.txt | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/arm/Samsung-S3C24XX/Overview.txt b/Documentation/arm/Samsung-S3C24XX/Overview.txt index 081892df4fda..0f11fcc1d51e 100644 --- a/Documentation/arm/Samsung-S3C24XX/Overview.txt +++ b/Documentation/arm/Samsung-S3C24XX/Overview.txt @@ -8,10 +8,16 @@ Introduction The Samsung S3C24XX range of ARM9 System-on-Chip CPUs are supported by the 's3c2410' architecture of ARM Linux. Currently the S3C2410, - S3C2412, S3C2413, S3C2440, S3C2442 and S3C2443 devices are supported. + S3C2412, S3C2413, S3C2416 S3C2440, S3C2442, S3C2443 and S3C2450 devices + are supported. Support for the S3C2400 and S3C24A0 series are in progress. + The S3C2416 and S3C2450 devices are very similar and S3C2450 support is + included under the arch/arm/mach-s3c2416 directory. Note, whilst core + support for these SoCs is in, work on some of the extra peripherals + and extra interrupts is still ongoing. + Configuration ------------- @@ -209,6 +215,9 @@ GPIO Newer kernels carry GPIOLIB, and support is being moved towards this with some of the older support in line to be removed. + As of v2.6.34, the move towards using gpiolib support is almost + complete, and very little of the old calls are left. + Clock Management ---------------- -- cgit v1.2.3 From 70f5c9a4ddcbce3e6d7d54ec890a4bca19b4fc7a Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 11:11:50 +0900 Subject: ARM: S3C24XX: Documentation: update GPIO documentation Upate the S3C24XX GPIO documentation after the changes for gpiolib and show which calls are being replaced by gpiolib or the new s3c generic calls. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung-S3C24XX/GPIO.txt | 35 ++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/arm/Samsung-S3C24XX/GPIO.txt b/Documentation/arm/Samsung-S3C24XX/GPIO.txt index 2af2cf39915f..9fe5feac212d 100644 --- a/Documentation/arm/Samsung-S3C24XX/GPIO.txt +++ b/Documentation/arm/Samsung-S3C24XX/GPIO.txt @@ -24,8 +24,16 @@ GPIOLIB listed below will be removed (they may be marked as __deprecated in the near future). - - s3c2410_gpio_getpin - - s3c2410_gpio_setpin + The following functions now either have a s3c_ specific variant + or are merged into gpiolib. See the definitions in + arch/arm/plat-samsung/include/plat/gpio-cfg.h: + + s3c2410_gpio_setpin() gpio_set_value() or gpio_direction_output() + s3c2410_gpio_getpin() gpio_get_value() or gpio_direction_input() + s3c2410_gpio_getirq() gpio_to_irq() + s3c2410_gpio_cfgpin() s3c_gpio_cfgpin() + s3c2410_gpio_getcfg() s3c_gpio_getcfg() + s3c2410_gpio_pullup() s3c_gpio_setpull() Headers @@ -54,6 +62,11 @@ PIN Numbers eg S3C2410_GPA(0) or S3C2410_GPF(1). These defines are used to tell the GPIO functions which pin is to be used. + With the conversion to gpiolib, there is no longer a direct conversion + from gpio pin number to register base address as in earlier kernels. This + is due to the number space required for newer SoCs where the later + GPIOs are not contiguous. + Configuring a pin ----------------- @@ -71,6 +84,8 @@ Configuring a pin which would turn GPA(0) into the lowest Address line A0, and set GPE(8) to be connected to the SDIO/MMC controller's SDDAT1 line. + The s3c_gpio_cfgpin() call is a functional replacement for this call. + Reading the current configuration --------------------------------- @@ -82,6 +97,9 @@ Reading the current configuration The return value will be from the same set of values which can be passed to s3c2410_gpio_cfgpin(). + The s3c_gpio_getcfg() call should be a functional replacement for + this call. + Configuring a pull-up resistor ------------------------------ @@ -95,6 +113,10 @@ Configuring a pull-up resistor Where the to value is zero to set the pull-up off, and 1 to enable the specified pull-up. Any other values are currently undefined. + The s3c_gpio_setpull() offers similar functionality, but with the + ability to encode whether the pull is up or down. Currently there + is no 'just on' state, so up or down must be selected. + Getting the state of a PIN -------------------------- @@ -106,6 +128,9 @@ Getting the state of a PIN This will return either zero or non-zero. Do not count on this function returning 1 if the pin is set. + This call is now implemented by the relevant gpiolib calls, convert + your board or driver to use gpiolib. + Setting the state of a PIN -------------------------- @@ -117,6 +142,9 @@ Setting the state of a PIN Which sets the given pin to the value. Use 0 to write 0, and 1 to set the output to 1. + This call is now implemented by the relevant gpiolib calls, convert + your board or driver to use gpiolib. + Getting the IRQ number associated with a PIN -------------------------------------------- @@ -128,6 +156,9 @@ Getting the IRQ number associated with a PIN Note, not all pins have an IRQ. + This call is now implemented by the relevant gpiolib calls, convert + your board or driver to use gpiolib. + Authour ------- -- cgit v1.2.3 From 0639b93d00812dc6a7c925a41ce3610d28f73d4e Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 11:23:53 +0900 Subject: ARM: S3C24XX: Documentation: add section on gpiolib changes Add section to the S3C24XX GPIO documentation on the recent changes to move towards gpiolib integration. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung-S3C24XX/GPIO.txt | 44 ++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) (limited to 'Documentation') diff --git a/Documentation/arm/Samsung-S3C24XX/GPIO.txt b/Documentation/arm/Samsung-S3C24XX/GPIO.txt index 9fe5feac212d..228ecb61d2fd 100644 --- a/Documentation/arm/Samsung-S3C24XX/GPIO.txt +++ b/Documentation/arm/Samsung-S3C24XX/GPIO.txt @@ -36,6 +36,50 @@ GPIOLIB s3c2410_gpio_pullup() s3c_gpio_setpull() +GPIOLIB conversion +------------------ + +If you need to convert your board or driver to use gpiolib from the exiting +s3c2410 api, then here are some notes on the process. + +1) If your board is exclusively using an GPIO, say to control peripheral + power, then it will require to claim the gpio with gpio_request() before + it can use it. + + It is recommended to check the return value, with at least WARN_ON() + during initialisation. + +2) The s3c2410_gpio_cfgpin() can be directly replaced with s3c_gpio_cfgpin() + as they have the same arguments, and can either take the pin specific + values, or the more generic special-function-number arguments. + +3) s3c2410_gpio_pullup() changs have the problem that whilst the + s3c2410_gpio_pullup(x, 1) can be easily translated to the + s3c_gpio_setpull(x, S3C_GPIO_PULL_NONE), the s3c2410_gpio_pullup(x, 0) + are not so easy. + + The s3c2410_gpio_pullup(x, 0) case enables the pull-up (or in the case + of some of the devices, a pull-down) and as such the new API distinguishes + between the UP and DOWN case. There is currently no 'just turn on' setting + which may be required if this becomes a problem. + +4) s3c2410_gpio_setpin() can be replaced by gpio_set_value(), the old call + does not implicitly configure the relevant gpio to output. The gpio + direction should be changed before using gpio_set_value(). + +5) s3c2410_gpio_getpin() is replaceable by gpio_get_value() if the pin + has been set to input. It is currently unknown what the behaviour is + when using gpio_get_value() on an output pin (s3c2410_gpio_getpin + would return the value the pin is supposed to be outputting). + +6) s3c2410_gpio_getirq() should be directly replacable with the + gpio_to_irq() call. + +The s3c2410_gpio and gpio_ calls have always operated on the same gpio +numberspace, so there is no problem with converting the gpio numbering +between the calls. + + Headers ------- -- cgit v1.2.3 From d192baeaa7a068b4e5151a34b4121d99938e396d Mon Sep 17 00:00:00 2001 From: Ben Dooks Date: Mon, 24 May 2010 12:13:30 +0900 Subject: ARM: SAMSUNG: Documentation: add documentation on GPIO code Add some documentation in Documentation/arm/Samsung for the GPIO code and where to look for the necessary functions. Update the S3C24XX case as well. Signed-off-by: Ben Dooks --- Documentation/arm/Samsung-S3C24XX/GPIO.txt | 2 ++ Documentation/arm/Samsung-S3C24XX/Overview.txt | 4 +++ Documentation/arm/Samsung/GPIO.txt | 42 ++++++++++++++++++++++++++ 3 files changed, 48 insertions(+) create mode 100644 Documentation/arm/Samsung/GPIO.txt (limited to 'Documentation') diff --git a/Documentation/arm/Samsung-S3C24XX/GPIO.txt b/Documentation/arm/Samsung-S3C24XX/GPIO.txt index 228ecb61d2fd..816d6071669e 100644 --- a/Documentation/arm/Samsung-S3C24XX/GPIO.txt +++ b/Documentation/arm/Samsung-S3C24XX/GPIO.txt @@ -12,6 +12,8 @@ Introduction of the s3c2410 GPIO system, please read the Samsung provided data-sheet/users manual to find out the complete list. + See Documentation/arm/Samsung/GPIO.txt for the core implemetation. + GPIOLIB ------- diff --git a/Documentation/arm/Samsung-S3C24XX/Overview.txt b/Documentation/arm/Samsung-S3C24XX/Overview.txt index 0f11fcc1d51e..c12bfc1a00c9 100644 --- a/Documentation/arm/Samsung-S3C24XX/Overview.txt +++ b/Documentation/arm/Samsung-S3C24XX/Overview.txt @@ -218,6 +218,10 @@ GPIO As of v2.6.34, the move towards using gpiolib support is almost complete, and very little of the old calls are left. + See Documentation/arm/Samsung-S3C24XX/GPIO.txt for the S3C24XX specific + support and Documentation/arm/Samsung/GPIO.txt for the core Samsung + implementation. + Clock Management ---------------- diff --git a/Documentation/arm/Samsung/GPIO.txt b/Documentation/arm/Samsung/GPIO.txt new file mode 100644 index 000000000000..05850c62abeb --- /dev/null +++ b/Documentation/arm/Samsung/GPIO.txt @@ -0,0 +1,42 @@ + Samsung GPIO implementation + =========================== + +Introduction +------------ + +This outlines the Samsung GPIO implementation and the architecture +specfic calls provided alongisde the drivers/gpio core. + + +S3C24XX (Legacy) +---------------- + +See Documentation/arm/Samsung-S3C24XX/GPIO.txt for more information +about these devices. Their implementation is being brought into line +with the core samsung implementation described in this document. + + +GPIOLIB integration +------------------- + +The gpio implementation uses gpiolib as much as possible, only providing +specific calls for the items that require Samsung specific handling, such +as pin special-function or pull resistor control. + +GPIO numbering is synchronised between the Samsung and gpiolib system. + + +PIN configuration +----------------- + +Pin configuration is specific to the Samsung architecutre, with each SoC +registering the necessary information for the core gpio configuration +implementation to configure pins as necessary. + +The s3c_gpio_cfgpin() and s3c_gpio_setpull() provide the means for a +driver or machine to change gpio configuration. + +See arch/arm/plat-samsung/include/plat/gpio-cfg.h for more information +on these functions. + + -- cgit v1.2.3