diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2020-10-15 10:01:51 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2020-10-15 10:01:51 -0700 |
| commit | 726eb70e0d34dc4bc4dada71f52bba8ed638431e (patch) | |
| tree | e49674616f4513c8c6a4746a08e93c9441708d34 | |
| parent | c6dbef7307629cce855aa6b482b60cbf7777ed88 (diff) | |
| parent | f3277cbfba763cd2826396521b9296de67cf1bbc (diff) | |
Merge tag 'char-misc-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
Pull char/misc driver updates from Greg KH:
"Here is the big set of char, misc, and other assorted driver subsystem
patches for 5.10-rc1.
There's a lot of different things in here, all over the drivers/
directory. Some summaries:
- soundwire driver updates
- habanalabs driver updates
- extcon driver updates
- nitro_enclaves new driver
- fsl-mc driver and core updates
- mhi core and bus updates
- nvmem driver updates
- eeprom driver updates
- binder driver updates and fixes
- vbox minor bugfixes
- fsi driver updates
- w1 driver updates
- coresight driver updates
- interconnect driver updates
- misc driver updates
- other minor driver updates
All of these have been in linux-next for a while with no reported
issues"
* tag 'char-misc-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (396 commits)
binder: fix UAF when releasing todo list
docs: w1: w1_therm: Fix broken xref, mistakes, clarify text
misc: Kconfig: fix a HISI_HIKEY_USB dependency
LSM: Fix type of id parameter in kernel_post_load_data prototype
misc: Kconfig: add a new dependency for HISI_HIKEY_USB
firmware_loader: fix a kernel-doc markup
w1: w1_therm: make w1_poll_completion static
binder: simplify the return expression of binder_mmap
test_firmware: Test partial read support
firmware: Add request_partial_firmware_into_buf()
firmware: Store opt_flags in fw_priv
fs/kernel_file_read: Add "offset" arg for partial reads
IMA: Add support for file reads without contents
LSM: Add "contents" flag to kernel_read_file hook
module: Call security_kernel_post_load_data()
firmware_loader: Use security_post_load_data()
LSM: Introduce kernel_post_load_data() hook
fs/kernel_read_file: Add file_size output argument
fs/kernel_read_file: Switch buffer size arg to size_t
fs/kernel_read_file: Remove redundant size argument
...
297 files changed, 24461 insertions, 10014 deletions
diff --git a/Documentation/ABI/stable/sysfs-bus-mhi b/Documentation/ABI/stable/sysfs-bus-mhi new file mode 100644 index 000000000000..ecfe7662f8d0 --- /dev/null +++ b/Documentation/ABI/stable/sysfs-bus-mhi @@ -0,0 +1,21 @@ +What: /sys/bus/mhi/devices/.../serialnumber +Date: Sept 2020 +KernelVersion: 5.10 +Contact: Bhaumik Bhatt <bbhatt@codeaurora.org> +Description: The file holds the serial number of the client device obtained + using a BHI (Boot Host Interface) register read after at least + one attempt to power up the device has been done. If read + without having the device power on at least once, the file will + read all 0's. +Users: Any userspace application or clients interested in device info. + +What: /sys/bus/mhi/devices/.../oem_pk_hash +Date: Sept 2020 +KernelVersion: 5.10 +Contact: Bhaumik Bhatt <bbhatt@codeaurora.org> +Description: The file holds the OEM PK Hash value of the endpoint device + obtained using a BHI (Boot Host Interface) register read after + at least one attempt to power up the device has been done. If + read without having the device power on at least once, the file + will read all 0's. +Users: Any userspace application or clients interested in device info. diff --git a/Documentation/ABI/testing/sysfs-bus-dfl b/Documentation/ABI/testing/sysfs-bus-dfl new file mode 100644 index 000000000000..23543be904f2 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-bus-dfl @@ -0,0 +1,15 @@ +What: /sys/bus/dfl/devices/dfl_dev.X/type +Date: Aug 2020 +KernelVersion: 5.10 +Contact: Xu Yilun <yilun.xu@intel.com> +Description: Read-only. It returns type of DFL FIU of the device. Now DFL + supports 2 FIU types, 0 for FME, 1 for PORT. + Format: 0x%x + +What: /sys/bus/dfl/devices/dfl_dev.X/feature_id +Date: Aug 2020 +KernelVersion: 5.10 +Contact: Xu Yilun <yilun.xu@intel.com> +Description: Read-only. It returns feature identifier local to its DFL FIU + type. + Format: 0x%x diff --git a/Documentation/ABI/testing/sysfs-bus-fsi b/Documentation/ABI/testing/sysfs-bus-fsi index 320697bdf41d..d148214181a1 100644 --- a/Documentation/ABI/testing/sysfs-bus-fsi +++ b/Documentation/ABI/testing/sysfs-bus-fsi @@ -36,3 +36,11 @@ Contact: linux-fsi@lists.ozlabs.org Description: Provides a means of reading/writing a 32 bit value from/to a specified FSI bus address. + +What: /sys/bus/platform/devices/../cfam_reset +Date: Sept 2020 +KernelVersion: 5.10 +Contact: linux-fsi@lists.ozlabs.org +Description: + Provides a means of resetting the cfam that is attached to the + FSI device. diff --git a/Documentation/ABI/testing/sysfs-bus-mei b/Documentation/ABI/testing/sysfs-bus-mei index 3d37e2796d5a..6e9a105fe5cb 100644 --- a/Documentation/ABI/testing/sysfs-bus-mei +++ b/Documentation/ABI/testing/sysfs-bus-mei @@ -41,6 +41,13 @@ Contact: Tomas Winkler <tomas.winkler@intel.com> Description: Stores mei client fixed address, if any Format: %d +What: /sys/bus/mei/devices/.../vtag +Date: Nov 2020 +KernelVersion: 5.9 +Contact: Tomas Winkler <tomas.winkler@intel.com> +Description: Stores mei client vtag support status + Format: %d + What: /sys/bus/mei/devices/.../max_len Date: Nov 2019 KernelVersion: 5.5 diff --git a/Documentation/ABI/testing/sysfs-bus-soundwire-slave b/Documentation/ABI/testing/sysfs-bus-soundwire-slave index db4c9511d1aa..d324aa0b678f 100644 --- a/Documentation/ABI/testing/sysfs-bus-soundwire-slave +++ b/Documentation/ABI/testing/sysfs-bus-soundwire-slave @@ -1,3 +1,21 @@ +What: /sys/bus/soundwire/devices/sdw:.../status + /sys/bus/soundwire/devices/sdw:.../device_number + +Date: September 2020 + +Contact: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com> + Bard Liao <yung-chuan.liao@linux.intel.com> + Vinod Koul <vkoul@kernel.org> + +Description: SoundWire Slave status + + These properties report the Slave status, e.g. if it + is UNATTACHED or not, and in the latter case show the + device_number. This status information is useful to + detect devices exposed by platform firmware but not + physically present on the bus, and conversely devices + not exposed in platform firmware but enumerated. + What: /sys/bus/soundwire/devices/sdw:.../dev-properties/mipi_revision /sys/bus/soundwire/devices/sdw:.../dev-properties/wake_capable /sys/bus/soundwire/devices/sdw:.../dev-properties/test_mode_capable diff --git a/Documentation/ABI/testing/sysfs-driver-habanalabs b/Documentation/ABI/testing/sysfs-driver-habanalabs index 1a14bf9b22ba..169ae4b2a180 100644 --- a/Documentation/ABI/testing/sysfs-driver-habanalabs +++ b/Documentation/ABI/testing/sysfs-driver-habanalabs @@ -2,13 +2,17 @@ What: /sys/class/habanalabs/hl<n>/armcp_kernel_ver Date: Jan 2019 KernelVersion: 5.1 Contact: oded.gabbay@gmail.com -Description: Version of the Linux kernel running on the device's CPU +Description: Version of the Linux kernel running on the device's CPU. + Will be DEPRECATED in Linux kernel version 5.10, and be + replaced with cpucp_kernel_ver What: /sys/class/habanalabs/hl<n>/armcp_ver Date: Jan 2019 KernelVersion: 5.1 Contact: oded.gabbay@gmail.com Description: Version of the application running on the device's CPU + Will be DEPRECATED in Linux kernel version 5.10, and be + replaced with cpucp_ver What: /sys/class/habanalabs/hl<n>/clk_max_freq_mhz Date: Jun 2019 @@ -33,6 +37,18 @@ KernelVersion: 5.1 Contact: oded.gabbay@gmail.com Description: Version of the Device's CPLD F/W +What: /sys/class/habanalabs/hl<n>/cpucp_kernel_ver +Date: Oct 2020 +KernelVersion: 5.10 +Contact: oded.gabbay@gmail.com +Description: Version of the Linux kernel running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/cpucp_ver +Date: Oct 2020 +KernelVersion: 5.10 +Contact: oded.gabbay@gmail.com +Description: Version of the application running on the device's CPU + What: /sys/class/habanalabs/hl<n>/device_type Date: Jan 2019 KernelVersion: 5.1 diff --git a/Documentation/ABI/testing/sysfs-driver-w1_therm b/Documentation/ABI/testing/sysfs-driver-w1_therm index 9b488c0afdfa..8873bbb075cb 100644 --- a/Documentation/ABI/testing/sysfs-driver-w1_therm +++ b/Documentation/ABI/testing/sysfs-driver-w1_therm @@ -49,10 +49,13 @@ Description: will be changed only in device RAM, so it will be cleared when power is lost. Trigger a 'save' to EEPROM command to keep values after power-on. Read or write are : - * '9..12': device resolution in bit + * '9..14': device resolution in bit or resolution to set in bit * '-xx': xx is kernel error when reading the resolution * Anything else: do nothing + Some DS18B20 clones are fixed in 12-bit resolution, so the + actual resolution is read back from the chip and verified. Error + is reported if the results differ. Users: any user space application which wants to communicate with w1_term device @@ -86,7 +89,7 @@ Description: *write* : * '0' : save the 2 or 3 bytes to the device EEPROM (i.e. TH, TL and config register) - * '9..12' : set the device resolution in RAM + * '9..14' : set the device resolution in RAM (if supported) * Anything else: do nothing refer to Documentation/w1/slaves/w1_therm.rst for detailed @@ -114,3 +117,47 @@ Description: of the bulk read command (not the current temperature). Users: any user space application which wants to communicate with w1_term device + + +What: /sys/bus/w1/devices/.../conv_time +Date: July 2020 +Contact: Ivan Zaentsev <ivan.zaentsev@wirenboard.ru> +Description: + (RW) Get, set, or measure a temperature conversion time. The + setting remains active until a resolution change. Then it is + reset to default (datasheet) conversion time for a new + resolution. + + *read*: Actual conversion time in milliseconds. *write*: + '0': Set the default conversion time from the datasheet. + '1': Measure and set the conversion time. Make a single + temperature conversion, measure an actual value. + Increase it by 20% for temperature range. A new + conversion time can be obtained by reading this + same attribute. + other positive value: + Set the conversion time in milliseconds. + +Users: An application using the w1_term device + + +What: /sys/bus/w1/devices/.../features +Date: July 2020 +Contact: Ivan Zaentsev <ivan.zaentsev@wirenboard.ru> +Description: + (RW) Control optional driver settings. + Bit masks to read/write (bitwise OR): + + 1: Enable check for conversion success. If byte 6 of + scratchpad memory is 0xC after conversion, and + temperature reads 85.00 (powerup value) or 127.94 + (insufficient power) - return a conversion error. + + 2: Enable poll for conversion completion. Generate read cycles + after the conversion start and wait for 1's. In parasite + power mode this feature is not available. + + *read*: Currently selected features. + *write*: Select features. + +Users: An application using the w1_term device diff --git a/Documentation/devicetree/bindings/extcon/extcon-ptn5150.txt b/Documentation/devicetree/bindings/extcon/extcon-ptn5150.txt deleted file mode 100644 index 936fbdf12815..000000000000 --- a/Documentation/devicetree/bindings/extcon/extcon-ptn5150.txt +++ /dev/null @@ -1,27 +0,0 @@ -* PTN5150 CC (Configuration Channel) Logic device - -PTN5150 is a small thin low power CC logic chip supporting the USB Type-C -connector application with CC control logic detection and indication functions. -It is interfaced to the host controller using an I2C interface. - -Required properties: -- compatible: should be "nxp,ptn5150" -- reg: specifies the I2C slave address of the device -- int-gpio: should contain a phandle and GPIO specifier for the GPIO pin - connected to the PTN5150's INTB pin. -- vbus-gpio: should contain a phandle and GPIO specifier for the GPIO pin which - is used to control VBUS. -- pinctrl-names : a pinctrl state named "default" must be defined. -- pinctrl-0 : phandle referencing pin configuration of interrupt and vbus - control. - -Example: - ptn5150@1d { - compatible = "nxp,ptn5150"; - reg = <0x1d>; - int-gpio = <&msmgpio 78 GPIO_ACTIVE_HIGH>; - vbus-gpio = <&msmgpio 148 GPIO_ACTIVE_HIGH>; - pinctrl-names = "default"; - pinctrl-0 = <&ptn5150_default>; - status = "okay"; - }; diff --git a/Documentation/devicetree/bindings/extcon/extcon-ptn5150.yaml b/Documentation/devicetree/bindings/extcon/extcon-ptn5150.yaml new file mode 100644 index 000000000000..4b0f414486d2 --- /dev/null +++ b/Documentation/devicetree/bindings/extcon/extcon-ptn5150.yaml @@ -0,0 +1,60 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/extcon/extcon-ptn5150.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: PTN5150 CC (Configuration Channel) Logic device + +maintainers: + - Krzysztof Kozlowski <krzk@kernel.org> + +description: | + PTN5150 is a small thin low power CC logic chip supporting the USB Type-C + connector application with CC control logic detection and indication + functions. It is interfaced to the host controller using an I2C interface. + +properties: + compatible: + const: nxp,ptn5150 + + int-gpios: + deprecated: true + description: + GPIO pin (input) connected to the PTN5150's INTB pin. + Use "interrupts" instead. + + interrupts: + maxItems: 1 + + reg: + maxItems: 1 + + vbus-gpios: + description: + GPIO pin (output) used to control VBUS. If skipped, no such control + takes place. + +required: + - compatible + - interrupts + - reg + +additionalProperties: false + +examples: + - | + #include <dt-bindings/gpio/gpio.h> + #include <dt-bindings/interrupt-controller/irq.h> + i2c { + #address-cells = <1>; + #size-cells = <0>; + + ptn5150@1d { + compatible = "nxp,ptn5150"; + reg = <0x1d>; + interrupt-parent = <&msmgpio>; + interrupts = <78 IRQ_TYPE_LEVEL_HIGH>; + vbus-gpios = <&msmgpio 148 GPIO_ACTIVE_HIGH>; + }; + }; diff --git a/Documentation/devicetree/bindings/fsi/fsi-master-aspeed.txt b/Documentation/devicetree/bindings/fsi/fsi-master-aspeed.txt index b758f91914f7..9853fefff5d8 100644 --- a/Documentation/devicetree/bindings/fsi/fsi-master-aspeed.txt +++ b/Documentation/devicetree/bindings/fsi/fsi-master-aspeed.txt @@ -12,6 +12,13 @@ Required properties: - pinctrl-0: phandle to pinctrl node - pinctrl-names: pinctrl state +Optional properties: + - cfam-reset-gpios: GPIO for CFAM reset + + - fsi-routing-gpios: GPIO for setting the FSI mux (internal or cabled) + - fsi-mux-gpios: GPIO for detecting the desired FSI mux state + + Examples: fsi-master { @@ -21,4 +28,9 @@ Examples: pinctrl-names = "default"; pinctrl-0 = <&pinctrl_fsi1_default>; clocks = <&syscon ASPEED_CLK_GATE_FSICLK>; + + fsi-routing-gpios = <&gpio0 ASPEED_GPIO(Q, 7) GPIO_ACTIVE_HIGH>; + fsi-mux-gpios = <&gpio0 ASPEED_GPIO(B, 0) GPIO_ACTIVE_HIGH>; + + cfam-reset-gpios = <&gpio0 ASPEED_GPIO(Q, 0) GPIO_ACTIVE_LOW>; }; diff --git a/Documentation/devicetree/bindings/interconnect/interconnect.txt b/Documentation/devicetree/bindings/interconnect/interconnect.txt index 6f5d23a605b7..138c544c8c8c 100644 --- a/Documentation/devicetree/bindings/interconnect/interconnect.txt +++ b/Documentation/devicetree/bindings/interconnect/interconnect.txt @@ -19,7 +19,8 @@ directly. Required properties: - compatible : contains the interconnect provider compatible string - #interconnect-cells : number of cells in a interconnect specifier needed to - encode the interconnect node id + encode the interconnect node id and optionally add a + path tag Example: @@ -44,6 +45,10 @@ components it has to interact with. Required properties: interconnects : Pairs of phandles and interconnect provider specifier to denote the edge source and destination ports of the interconnect path. + An optional path tag value could specified as additional argument + to both endpoints and in such cases, this information will be passed + to the interconnect framework to do aggregation based on the attached + tag. Optional properties: interconnect-names : List of interconnect path name strings sorted in the same @@ -62,3 +67,20 @@ Example: interconnects = <&pnoc MASTER_SDCC_1 &bimc SLAVE_EBI_CH0>; interconnect-names = "sdhc-mem"; }; + +Example with path tags: + + gnoc: interconnect@17900000 { + ... + interconnect-cells = <2>; + }; + + mnoc: interconnect@1380000 { + ... + interconnect-cells = <2>; + }; + + cpu@0 { + ... + interconnects = <&gnoc MASTER_APPSS_PROC 3 &mnoc SLAVE_EBI1 3>; + } diff --git a/Documentation/devicetree/bindings/interconnect/qcom,bcm-voter.yaml b/Documentation/devicetree/bindings/interconnect/qcom,bcm-voter.yaml index 5971fc1df08d..e23df4836c6f 100644 --- a/Documentation/devicetree/bindings/interconnect/qcom,bcm-voter.yaml +++ b/Documentation/devicetree/bindings/interconnect/qcom,bcm-voter.yaml @@ -21,6 +21,23 @@ properties: enum: - qcom,bcm-voter + qcom,tcs-wait: + description: | + Optional mask of which TCSs (Triggered Command Sets) wait for completion + upon triggering. If not specified, then the AMC and WAKE sets wait for + completion. The mask bits are available in the QCOM_ICC_TAG_* defines. + + The AMC TCS is triggered immediately when icc_set_bw() is called. The + WAKE/SLEEP TCSs are triggered when the RSC transitions between active and + sleep modes. + + In most cases, it's necessary to wait in both the AMC and WAKE sets to + ensure resources are available before use. If a specific RSC and its use + cases can ensure sufficient delay by other means, then this can be + overridden to reduce latencies. + + $ref: /schemas/types.yaml#/definitions/uint32 + required: - compatible @@ -39,7 +56,10 @@ examples: # as defined in Documentation/devicetree/bindings/soc/qcom/rpmh-rsc.txt - | + #include <dt-bindings/interconnect/qcom,icc.h> + disp_bcm_voter: bcm_voter { compatible = "qcom,bcm-voter"; + qcom,tcs-wait = <QCOM_ICC_TAG_AMC>; }; ... diff --git a/Documentation/devicetree/bindings/interconnect/qcom,osm-l3.yaml b/Documentation/devicetree/bindings/interconnect/qcom,osm-l3.yaml index 91f70c9067d1..d6a95c3cb26f 100644 --- a/Documentation/devicetree/bindings/interconnect/qcom,osm-l3.yaml +++ b/Documentation/devicetree/bindings/interconnect/qcom,osm-l3.yaml @@ -19,6 +19,8 @@ properties: enum: - qcom,sc7180-osm-l3 - qcom,sdm845-osm-l3 + - qcom,sm8150-osm-l3 + - qcom,sm8250-epss-l3 reg: maxItems: 1 diff --git a/Documentation/devicetree/bindings/interconnect/qcom,sdm845.yaml b/Documentation/devicetree/bindings/interconnect/qcom,rpmh.yaml index dab17c0716ce..30c2a092d2d3 100644 --- a/Documentation/devicetree/bindings/interconnect/qcom,sdm845.yaml +++ b/Documentation/devicetree/bindings/interconnect/qcom,rpmh.yaml @@ -1,16 +1,17 @@ # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) %YAML 1.2 --- -$id: http://devicetree.org/schemas/interconnect/qcom,sdm845.yaml# +$id: http://devicetree.org/schemas/interconnect/qcom,rpmh.yaml# $schema: http://devicetree.org/meta-schemas/core.yaml# -title: Qualcomm SDM845 Network-On-Chip Interconnect +title: Qualcomm RPMh Network-On-Chip Interconnect maintainers: - Georgi Djakov <georgi.djakov@linaro.org> + - Odelu Kukatla <okukatla@codeaurora.org> description: | - SDM845 interconnect providers support system bandwidth requirements through + RPMh interconnect providers support system bandwidth requirements through RPMh hardware accelerators known as Bus Clock Manager (BCM). The provider is able to communicate with the BCM through the Resource State Coordinator (RSC) associated with each execution environment. Provider nodes must point to at @@ -23,6 +24,19 @@ properties: compatible: enum: + - qcom,sc7180-aggre1-noc + - qcom,sc7180-aggre2-noc + - qcom,sc7180-camnoc-virt + - qcom,sc7180-compute-noc + - qcom,sc7180-config-noc + - qcom,sc7180-dc-noc + - qcom,sc7180-gem-noc + - qcom,sc7180-ipa-virt + - qcom,sc7180-mc-virt + - qcom,sc7180-mmss-noc + - qcom,sc7180-npu-noc + - qcom,sc7180-qup-virt + - qcom,sc7180-system-noc - qcom,sdm845-aggre1-noc - qcom,sdm845-aggre2-noc - qcom,sdm845-config-noc @@ -31,6 +45,28 @@ properties: - qcom,sdm845-mem-noc - qcom,sdm845-mmss-noc - qcom,sdm845-system-noc + - qcom,sm8150-aggre1-noc + - qcom,sm8150-aggre2-noc + - qcom,sm8150-camnoc-noc + - qcom,sm8150-compute-noc + - qcom,sm8150-config-noc + - qcom,sm8150-dc-noc + - qcom,sm8150-gem-noc + - qcom,sm8150-ipa-virt + - qcom,sm8150-mc-virt + - qcom,sm8150-mmss-noc + - qcom,sm8150-system-noc + - qcom,sm8250-aggre1-noc + - qcom,sm8250-aggre2-noc + - qcom,sm8250-compute-noc + - qcom,sm8250-config-noc + - qcom,sm8250-dc-noc + - qcom,sm8250-gem-noc + - qcom,sm8250-ipa-virt + - qcom,sm8250-mc-virt + - qcom,sm8250-mmss-noc + - qcom,sm8250-npu-noc + - qcom,sm8250-system-noc '#interconnect-cells': const: 1 diff --git a/Documentation/devicetree/bindings/interconnect/qcom,sc7180.yaml b/Documentation/devicetree/bindings/interconnect/qcom,sc7180.yaml deleted file mode 100644 index 8659048f92a7..000000000000 --- a/Documentation/devicetree/bindings/interconnect/qcom,sc7180.yaml +++ /dev/null @@ -1,85 +0,0 @@ -# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) -%YAML 1.2 ---- -$id: http://devicetree.org/schemas/interconnect/qcom,sc7180.yaml# -$schema: http://devicetree.org/meta-schemas/core.yaml# - -title: Qualcomm SC7180 Network-On-Chip Interconnect - -maintainers: - - Odelu Kukatla <okukatla@codeaurora.org> - -description: | - SC7180 interconnect providers support system bandwidth requirements through - RPMh hardware accelerators known as Bus Clock Manager (BCM). The provider is - able to communicate with the BCM through the Resource State Coordinator (RSC) - associated with each execution environment. Provider nodes must point to at - least one RPMh device child node pertaining to their RSC and each provider - can map to multiple RPMh resources. - -properties: - reg: - maxItems: 1 - - compatible: - enum: - - qcom,sc7180-aggre1-noc - - qcom,sc7180-aggre2-noc - - qcom,sc7180-camnoc-virt - - qcom,sc7180-compute-noc - - qcom,sc7180-config-noc - - qcom,sc7180-dc-noc - - qcom,sc7180-gem-noc - - qcom,sc7180-ipa-virt - - qcom,sc7180-mc-virt - - qcom,sc7180-mmss-noc - - qcom,sc7180-npu-noc - - qcom,sc7180-qup-virt - - qcom,sc7180-system-noc - - '#interconnect-cells': - const: 1 - - qcom,bcm-voters: - $ref: /schemas/types.yaml#/definitions/phandle-array - description: | - List of phandles to qcom,bcm-voter nodes that are required by - this interconnect to send RPMh commands. - - qcom,bcm-voter-names: - $ref: /schemas/types.yaml#/definitions/string-array - description: | - Names for each of the qcom,bcm-voters specified. - -required: - - compatible - - reg - - '#interconnect-cells' - - qcom,bcm-voters - -additionalProperties: false - -examples: - - | - #include <dt-bindings/interconnect/qcom,sc7180.h> - - config_noc: interconnect@1500000 { - compatible = "qcom,sc7180-config-noc"; - reg = <0x01500000 0x28000>; - #interconnect-cells = <1>; - qcom,bcm-voters = <&apps_bcm_voter>; - }; - - system_noc: interconnect@1620000 { - compatible = "qcom,sc7180-system-noc"; - reg = <0x01620000 0x17080>; - #interconnect-cells = <1>; - qcom,bcm-voters = <&apps_bcm_voter>; - }; - - mmss_noc: interconnect@1740000 { - compatible = "qcom,sc7180-mmss-noc"; - reg = <0x01740000 0x1c100>; - #interconnect-cells = <1>; - qcom,bcm-voters = <&apps_bcm_voter>; - }; diff --git a/Documentation/devicetree/bindings/soundwire/qcom,sdw.txt b/Documentation/devicetree/bindings/soundwire/qcom,sdw.txt index 436547f3b155..b104be131235 100644 --- a/Documentation/devicetree/bindings/soundwire/qcom,sdw.txt +++ b/Documentation/devicetree/bindings/soundwire/qcom,sdw.txt @@ -11,6 +11,7 @@ board specific bus parameters. Example: "qcom,soundwire-v1.3.0" "qcom,soundwire-v1.5.0" + "qcom,soundwire-v1.5.1" "qcom,soundwire-v1.6.0" - reg: Usage: required diff --git a/Documentation/driver-api/mei/mei.rst b/Documentation/driver-api/mei/mei.rst index c800d8e5f422..cea0b69ec216 100644 --- a/Documentation/driver-api/mei/mei.rst +++ b/Documentation/driver-api/mei/mei.rst @@ -42,6 +42,11 @@ The session is terminated calling :c:func:`close(int fd)`. A code snippet for an application communicating with Intel AMTHI client: +In order to support virtualization or sandboxing a trusted supervisor +can use :c:macro:`MEI_CONNECT_CLIENT_IOCTL_VTAG` to create +virtual channels with an Intel ME feature. Not all features support +virtual channels such client with answer EOPNOTSUPP. + .. code-block:: C struct mei_connect_client_data data; @@ -110,6 +115,38 @@ Connect to firmware Feature/Client. data that can be sent or received. (e.g. if MTU=2K, can send requests up to bytes 2k and received responses up to 2k bytes). +IOCTL_MEI_CONNECT_CLIENT_VTAG: +------------------------------ + +.. code-block:: none + + Usage: + + struct mei_connect_client_data_vtag client_data_vtag; + + ioctl(fd, IOCTL_MEI_CONNECT_CLIENT_VTAG, &client_data_vtag); + + Inputs: + + struct mei_connect_client_data_vtag - contain the following + Input field: + + in_client_uuid - GUID of the FW Feature that needs + to connect to. + vtag - virtual tag [1, 255] + + Outputs: + out_client_properties - Client Properties: MTU and Protocol Version. + + Error returns: + + ENOTTY No such client (i.e. wrong GUID) or connection is not allowed. + EINVAL Wrong IOCTL Number or tag == 0 + ENODEV Device or Connection is not initialized or ready. + ENOMEM Unable to allocate memory to client internal data. + EFAULT Fatal Error (e.g. Unable to access user input data) + EBUSY Connection Already Open + EOPNOTSUPP Vtag is not supported IOCTL_MEI_NOTIFY_SET --------------------- diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 2a198838fca9..5f7ff00f394e 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -328,8 +328,11 @@ Code Seq# Include File Comments 0xAC 00-1F linux/raw.h 0xAD 00 Netfilter device in development: <mailto:rusty@rustcorp.com.au> -0xAE all linux/kvm.h Kernel-based Virtual Machine +0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine <mailto:kvm@vger.kernel.org> +0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine + <mailto:kvm@vger.kernel.org> +0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves 0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor 0xB0 all RATIO devices in development: <mailto:vgo@ratio.de> diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index d20490292642..350f5c869b56 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -11,6 +11,7 @@ Linux Virtualization Support uml/user_mode_linux_howto_v2 paravirt_ops guest-halt-polling + ne_overview .. only:: html and subproject diff --git a/Documentation/virt/ne_overview.rst b/Documentation/virt/ne_overview.rst new file mode 100644 index 000000000000..39b0c8fe2654 --- /dev/null +++ b/Documentation/virt/ne_overview.rst @@ -0,0 +1,95 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============== +Nitro Enclaves +============== + +Overview +======== + +Nitro Enclaves (NE) is a new Amazon Elastic Compute Cloud (EC2) capability +that allows customers to carve out isolated compute environments within EC2 +instances [1]. + +For example, an application that processes sensitive data and runs in a VM, +can be separated from other applications running in the same VM. This +application then runs in a separate VM than the primary VM, namely an enclave. + +An enclave runs alongside the VM that spawned it. This setup matches low latency +applications needs. The resources that are allocated for the enclave, such as +memory and CPUs, are carved out of the primary VM. Each enclave is mapped to a +process running in the primary VM, that communicates with the NE driver via an +ioctl interface. + +In this sense, there are two components: + +1. An enclave abstraction process - a user space process running in the primary +VM guest that uses the provided ioctl interface of the NE driver to spawn an +enclave VM (that's 2 below). + +There is a NE emulated PCI device exposed to the primary VM. The driver for this +new PCI device is included in the NE driver. + +The ioctl logic is mapped to PCI device commands e.g. the NE_START_ENCLAVE ioctl +maps to an enclave start PCI command. The PCI device commands are then +translated into actions taken on the hypervisor side; that's the Nitro +hypervisor running on the host where the primary VM is running. The Nitro +hypervisor is based on core KVM technology. + +2. The enclave itself - a VM running on the same host as the primary VM that +spawned it. Memory and CPUs are carved out of the primary VM and are dedicated +for the enclave VM. An enclave does not have persistent storage attached. + +The memory regions carved out of the primary VM and given to an enclave need to +be aligned 2 MiB / 1 GiB physically contiguous memory regions (or multiple of +this size e.g. 8 MiB). The memory can be allocated e.g. by using hugetlbfs from +user space [2][3]. The memory size for an enclave needs to be at least 64 MiB. +The enclave memory and CPUs need to be from the same NUMA node. + +An enclave runs on dedicated cores. CPU 0 and its CPU siblings need to remain +available for the primary VM. A CPU pool has to be set for NE purposes by an +user with admin capability. See the cpu list section from the kernel +documentation [4] for how a CPU pool format looks. + +An enclave communicates with the primary VM via a local communication channel, +using virtio-vsock [5]. The primary VM has virtio-pci vsock emulated device, +while the enclave VM has a virtio-mmio vsock emulated device. The vsock device +uses eventfd for signaling. The enclave VM sees the usual interfaces - local +APIC and IOAPIC - to get interrupts from virtio-vsock device. The virtio-mmio +device is placed in memory below the typical 4 GiB. + +The application that runs in the enclave needs to be packaged in an enclave +image together with the OS ( e.g. kernel, ramdisk, init ) that will run in the +enclave VM. The enclave VM has its own kernel and follows the standard Linux +boot protocol [6]. + +The kernel bzImage, the kernel command line, the ramdisk(s) are part of the +Enclave Image Format (EIF); plus an EIF header including metadata such as magic +number, eif version, image size and CRC. + +Hash values are computed for the entire enclave image (EIF), the kernel and +ramdisk(s). That's used, for example, to check that the enclave image that is +loaded in the enclave VM is the one that was intended to be run. + +These crypto measurements are included in a signed attestation document +generated by the Nitro Hypervisor and further used to prove the identity of the +enclave; KMS is an example of service that NE is integrated with and that checks +the attestation doc. + +The enclave image (EIF) is loaded in the enclave memory at offset 8 MiB. The +init process in the enclave connects to the vsock CID of the primary VM and a +predefined port - 9000 - to send a heartbeat value - 0xb7. This mechanism is +used to check in the primary VM that the enclave has booted. The CID of the +primary VM is 3. + +If the enclave VM crashes or gracefully exits, an interrupt event is received by +the NE driver. This event is sent further to the user space enclave process +running in the primary VM via a poll notification mechanism. Then the user space +enclave process can exit. + +[1] https://aws.amazon.com/ec2/nitro/nitro-enclaves/ +[2] https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html +[3] https://lwn.net/Articles/807108/ +[4] https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html +[5] https://man7.org/linux/man-pages/man7/vsock.7.html +[6] https://www.kernel.org/doc/html/latest/x86/boot.html diff --git a/Documentation/w1/slaves/w1_therm.rst b/Documentation/w1/slaves/w1_therm.rst index cc4edae17751..e39202e2b000 100644 --- a/Documentation/w1/slaves/w1_therm.rst +++ b/Documentation/w1/slaves/w1_therm.rst @@ -6,6 +6,7 @@ Supported chips: * Maxim ds18*20 based temperature sensors. * Maxim ds1825 based temperature sensors. + * GXCAS GC20MH01 temperature sensor. Author: Evgeniy Polyakov <johnpol@2ka.mipt.ru> @@ -13,8 +14,8 @@ Author: Evgeniy Polyakov <johnpol@2ka.mipt.ru> Description ----------- -w1_therm provides basic temperature conversion for ds18*20 devices, and the -ds28ea00 device. +w1_therm provides basic temperature conversion for ds18*20, ds28ea00, GX20MH01 +devices. Supported family codes: @@ -26,59 +27,72 @@ W1_THERM_DS1825 0x3B W1_THERM_DS28EA00 0x42 ==================== ==== -Support is provided through the sysfs w1_slave file. Each open and -read sequence will initiate a temperature conversion then provide two +Support is provided through the sysfs entry ``w1_slave``. Each open and +read sequence will initiate a temperature conversion, then provide two lines of ASCII output. The first line contains the nine hex bytes read along with a calculated crc value and YES or NO if it matched. If the crc matched the returned values are retained. The second line displays the retained values along with a temperature in millidegrees Centigrade after t=. -Alternatively, temperature can be read using temperature sysfs, it -return only temperature in millidegrees Centigrade. +Alternatively, temperature can be read using ``temperature`` sysfs, it +returns only the temperature in millidegrees Centigrade. -A bulk read of all devices on the bus could be done writing 'trigger' -in the therm_bulk_read sysfs entry at w1_bus_master level. This will -sent the convert command on all devices on the bus, and if parasite -powered devices are detected on the bus (and strong pullup is enable +A bulk read of all devices on the bus could be done writing ``trigger`` +to ``therm_bulk_read`` entry at w1_bus_master level. This will +send the convert command to all devices on the bus, and if parasite +powered devices are detected on the bus (and strong pullup is enabled in the module), it will drive the line high during the longer conversion time required by parasited powered device on the line. Reading -therm_bulk_read will return 0 if no bulk conversion pending, +``therm_bulk_read`` will return 0 if no bulk conversion pending, -1 if at least one sensor still in conversion, 1 if conversion is complete but at least one sensor value has not been read yet. Result temperature is -then accessed by reading the temperature sysfs entry of each device, which +then accessed by reading the ``temperature`` entry of each device, which may return empty if conversion is still in progress. Note that if a bulk read is sent but one sensor is not read immediately, the next access to -temperature on this device will return the temperature measured at the +``temperature`` on this device will return the temperature measured at the time of issue of the bulk read command (not the current temperature). -Writing a value between 9 and 12 to the sysfs w1_slave file will change the -precision of the sensor for the next readings. This value is in (volatile) -SRAM, so it is reset when the sensor gets power-cycled. +A strong pullup will be applied during the conversion if required. -To store the current precision configuration into EEPROM, the value 0 -has to be written to the sysfs w1_slave file. Since the EEPROM has a limited -amount of writes (>50k), this command should be used wisely. +``conv_time`` is used to get current conversion time (read), and +adjust it (write). A temperature conversion time depends on the device type and +it's current resolution. Default conversion time is set by the driver according +to the device datasheet. A conversion time for many original device clones +deviate from datasheet specs. There are three options: 1) manually set the +correct conversion time by writing a value in milliseconds to ``conv_time``; 2) +auto measure and set a conversion time by writing ``1`` to +``conv_time``; 3) use ``features`` to enable poll for conversion +completion. Options 2, 3 can't be used in parasite power mode. To get back to +the default conversion time write ``0`` to ``conv_time``. -Alternatively, resolution can be set or read (value from 9 to 12) using the -dedicated resolution sysfs entry on each device. This sysfs entry is not -present for devices not supporting this feature. Driver will adjust the -correct conversion time for each device regarding to its resolution setting. -In particular, strong pullup will be applied if required during the conversion -duration. +Writing a resolution value (in bits) to ``w1_slave`` will change the +precision of the sensor for the next readings. Allowed resolutions are defined by +the sensor. Resolution is reset when the sensor gets power-cycled. -The write-only sysfs entry eeprom is an alternative for EEPROM operations: - * 'save': will save device RAM to EEPROM - * 'restore': will restore EEPROM data in device RAM. +To store the current resolution in EEPROM, write ``0`` to ``w1_slave``. +Since the EEPROM has a limited amount of writes (>50k), this command should be +used wisely. -ext_power syfs entry allow tho check the power status of each device. - * '0': device parasite powered - * '1': device externally powered +Alternatively, resolution can be read or written using the dedicated +``resolution`` entry on each device, if supported by the sensor. -sysfs alarms allow read or write TH and TL (Temperature High an Low) alarms. +Some non-genuine DS18B20 chips are fixed in 12-bit mode only, so the actual +resolution is read back from the chip and verified. + +Note: Changing the resolution reverts the conversion time to default. + +The write-only sysfs entry ``eeprom`` is an alternative for EEPROM operations. +Write ``save`` to save device RAM to EEPROM. Write ``restore`` to restore EEPROM +data in device RAM. + +``ext_power`` entry allows checking the power state of each device. Reads +``0`` if the device is parasite powered, ``1`` if the device is externally powered. + +Sysfs ``alarms`` allow read or write TH and TL (Temperature High an Low) alarms. Values shall be space separated and in the device range (typical -55 degC to 125 degC). Values are integer as they are store in a 8bit register in -the device. Lowest value is automatically put to TL.Once set, alarms could +the device. Lowest value is automatically put to TL. Once set, alarms could be search at master level. The module parameter strong_pullup can be set to 0 to disable the @@ -102,5 +116,24 @@ The DS28EA00 provides an additional two pins for implementing a sequence detection algorithm. This feature allows you to determine the physical location of the chip in the 1-wire bus without needing pre-existing knowledge of the bus ordering. Support is provided through the sysfs -w1_seq file. The file will contain a single line with an integer value +``w1_seq``. The file will contain a single line with an integer value representing the device index in the bus starting at 0. + +``features`` sysfs entry controls optional driver settings per device. +Insufficient power in parasite mode, line noise and insufficient conversion +time may lead to conversion failure. Original DS18B20 and some clones allow for +detection of invalid conversion. Write bit mask ``1`` to ``features`` to enable +checking the conversion success. If byte 6 of scratchpad memory is 0xC after +conversion and temperature reads 85.00 (powerup value) or 127.94 (insufficient +power), the driver returns a conversion error. Bit mask ``2`` enables poll for +conversion completion (normal power only) by generating read cycles on the bus +after conversion starts. In parasite power mode this feature is not available. +Feature bit masks may be combined (OR). More details in +Documentation/ABI/testing/sysfs-driver-w1_therm + +GX20MH01 device shares family number 0x28 with DS18*20. The device is generally +compatible with DS18B20. Added are lowest 2\ :sup:`-5`, 2\ :sup:`-6` temperature +bits in Config register; R2 bit in Config register enabling 13 and 14 bit +resolutions. The device is powered up in 14-bit resolution mode. The conversion +times specified in the datasheet are too low and have to be increased. The +device supports driver features ``1`` and ``2``. diff --git a/MAINTAINERS b/MAINTAINERS index 1975f409e684..2c59389f3455 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1725,6 +1725,7 @@ ARM/CORESIGHT FRAMEWORK AND DRIVERS M: Mathieu Poirier <mathieu.poirier@linaro.org> R: Suzuki K Poulose <suzuki.poulose@arm.com> R: Mike Leach <mike.leach@linaro.org> +L: coresight@lists.linaro.org (moderated for non-subscribers) L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained F: Documentation/ABI/testing/sysfs-bus-coresight-devices-* @@ -4101,6 +4102,11 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git F: drivers/char/ F: drivers/misc/ F: include/linux/miscdevice.h +X: drivers/char/agp/ +X: drivers/char/hw_random/ +X: drivers/char/ipmi/ +X: drivers/char/random.c +X: drivers/char/tpm/ CHECKPATCH M: Andy Whitcroft <apw@canonical.com> @@ -6828,14 +6834,17 @@ F: drivers/net/ethernet/nvidia/* FPGA DFL DRIVERS M: Wu Hao <hao.wu@intel.com> +R: Tom Rix <trix@redhat.com> L: linux-fpga@vger.kernel.org S: Maintained +F: Documentation/ABI/testing/sysfs-bus-dfl F: Documentation/fpga/dfl.rst F: drivers/fpga/dfl* F: include/uapi/linux/fpga-dfl.h FPGA MANAGER FRAMEWORK M: Moritz Fischer <mdf@kernel.org> +R: Tom Rix <trix@redhat.com> L: linux-fpga@vger.kernel.org S: Maintained W: http://www.rocketboards.org @@ -7885,6 +7894,13 @@ W: http://www.hisilicon.com F: Documentation/devicetree/bindings/net/hisilicon*.txt F: drivers/net/ethernet/hisilicon/ +HIKEY960 ONBOARD USB GPIO HUB DRIVER +M: John Stultz <john.stultz@linaro.org> +L: linux-kernel@vger.kernel.org +S: Maintained +F: drivers/misc/hisi_hikey_usb.c +F: Documentation/devicetree/bindings/misc/hisilicon-hikey-usb.yaml + HISILICON PMU DRIVER M: Shaokun Zhang <zhangshaokun@hisilicon.com> S: Supported @@ -11353,6 +11369,7 @@ M: Hemant Kumar <hemantk@codeaurora.org> L: linux-arm-msm@vger.kernel.org S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/mani/mhi.git +F: Documentation/ABI/stable/sysfs-bus-mhi F: Documentation/mhi/ F: drivers/bus/mhi/ F: include/linux/mhi.h @@ -12307,6 +12324,19 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/lftan/nios2.git F: arch/nios2/ +NITRO ENCLAVES (NE) +M: Andra Paraschiv <andraprs@amazon.com> +M: Alexandru Vasile <lexnv@amazon.com> +M: Alexandru Ciobotaru <alcioa@amazon.com> +L: linux-kernel@vger.kernel.org +S: Supported +W: https://aws.amazon.com/ec2/nitro/nitro-enclaves/ +F: Documentation/virt/ne_overview.rst +F: drivers/virt/nitro_enclaves/ +F: include/linux/nitro_enclaves.h +F: include/uapi/linux/nitro_enclaves.h +F: samples/nitro_enclaves/ + NOHZ, DYNTICKS SUPPORT M: Frederic Weisbecker <fweisbec@gmail.com> M: Thomas Gleixner <tglx@linutronix.de> @@ -12469,6 +12499,13 @@ F: drivers/iio/gyro/fxas21002c_core.c F: drivers/iio/gyro/fxas21002c_i2c.c F: drivers/iio/gyro/fxas21002c_spi.c +NXP PTN5150A CC LOGIC AND EXTCON DRIVER +M: Krzysztof Kozlowski <krzk@kernel.org> +L: linux-kernel@vger.kernel.org +S: Maintained +F: Documentation/devicetree/bindings/extcon/extcon-ptn5150.yaml +F: drivers/extcon/extcon-ptn5150.c + NXP SGTL5000 DRIVER M: Fabio Estevam <festevam@gmail.com> L: alsa-devel@alsa-project.org (moderated for non-subscribers) diff --git a/drivers/android/binder.c b/drivers/android/binder.c index f936530a19b0..4b9476521da6 100644 --- a/drivers/android/binder.c +++ b/drivers/android/binder.c @@ -223,7 +223,7 @@ static struct binder_transaction_log_entry *binder_transaction_log_add( struct binder_work { struct list_head entry; - enum { + enum binder_work_type { BINDER_WORK_TRANSACTION = 1, BINDER_WORK_TRANSACTION_COMPLETE, BINDER_WORK_RETURN_ERROR, @@ -885,27 +885,6 @@ static struct binder_work *binder_dequeue_work_head_ilocked( return w; } -/** - * binder_dequeue_work_head() - Dequeues the item at head of list - * @proc: binder_proc associated with list - * @list: list to dequeue head - * - * Removes the head of the list if there are items on the list - * - * Return: pointer dequeued binder_work, NULL if list was empty - */ -static struct binder_work *binder_dequeue_work_head( - struct binder_proc *proc, - struct list_head *list) -{ - struct binder_work *w; - - binder_inner_proc_lock(proc); - w = binder_dequeue_work_head_ilocked(list); - binder_inner_proc_unlock(proc); - return w; -} - static void binder_defer_work(struct binder_proc *proc, enum binder_deferred_state defer); static void binder_free_thread(struct binder_thread *thread); @@ -2344,8 +2323,6 @@ static void binder_transaction_buffer_release(struct binder_proc *proc, * file is done when the transaction is torn * down. */ - WARN_ON(failed_at && - proc->tsk == current->group_leader); } break; case BINDER_TYPE_PTR: /* @@ -3136,7 +3113,7 @@ static void binder_transaction(struct binder_proc *proc, t->buffer = binder_alloc_new_buf(&target_proc->alloc, tr->data_size, tr->offsets_size, extra_buffers_size, - !reply && (t->flags & TF_ONE_WAY)); + !reply && (t->flags & TF_ONE_WAY), current->tgid); if (IS_ERR(t->buffer)) { /* * -ESRCH indicates VMA cleared. The target is dying. @@ -4587,13 +4564,17 @@ static void binder_release_work(struct binder_proc *proc, struct list_head *list) { struct binder_work *w; + enum binder_work_type wtype; while (1) { - w = binder_dequeue_work_head(proc, list); + binder_inner_proc_lock(proc); + w = binder_dequeue_work_head_ilocked(list); + wtype = w ? w->type : 0; + binder_inner_proc_unlock(proc); if (!w) return; - switch (w->type) { + switch (wtype) { case BINDER_WORK_TRANSACTION: { struct binder_transaction *t; @@ -4627,9 +4608,11 @@ static void binder_release_work(struct binder_proc *proc, kfree(death); binder_stats_deleted(BINDER_STAT_DEATH); } break; + case BINDER_WORK_NODE: + break; default: pr_err("unexpected work type, %d, not freed\n", - w->type); + wtype); break; } } @@ -5182,9 +5165,7 @@ static const struct vm_operations_struct binder_vm_ops = { static int binder_mmap(struct file *filp, struct vm_area_struct *vma) { - int ret; struct binder_proc *proc = filp->private_data; - const char *failure_string; if (proc->tsk != current->group_leader) return -EINVAL; @@ -5196,9 +5177,9 @@ static int binder_mmap(struct file *filp, struct vm_area_struct *vma) (unsigned long)pgprot_val(vma->vm_page_prot)); if (vma->vm_flags & FORBIDDEN_MMAP_FLAGS) { - ret = -EPERM; - failure_string = "bad vm_flags"; - goto err_bad_arg; + pr_err("%s: %d %lx-%lx %s failed %d\n", __func__, + proc->pid, vma->vm_start, vma->vm_end, "bad vm_flags", -EPERM); + return -EPERM; } vma->vm_flags |= VM_DONTCOPY | VM_MIXEDMAP; vma->vm_flags &= ~VM_MAYWRITE; @@ -5206,15 +5187,7 @@ static int binder_mmap(struct file *filp, struct vm_area_struct *vma) vma->vm_ops = &binder_vm_ops; vma->vm_private_data = proc; - ret = binder_alloc_mmap_handler(&proc->alloc, vma); - if (ret) - return ret; - return 0; - -err_bad_arg: - pr_err("%s: %d %lx-%lx %s failed %d\n", __func__, - proc->pid, vma->vm_start, vma->vm_end, failure_string, ret); - return ret; + return binder_alloc_mmap_handler(&proc->alloc, vma); } static int binder_open(struct inode *nodp, struct file *filp) diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c index 69609696a843..2f846b7ae8b8 100644 --- a/drivers/android/binder_alloc.c +++ b/drivers/android/binder_alloc.c @@ -338,12 +338,50 @@ static inline struct vm_area_struct *binder_alloc_get_vma( return vma; } +static void debug_low_async_space_locked(struct binder_alloc *alloc, int pid) +{ + /* + * Find the amount and size of buffers allocated by the current caller; + * The idea is that once we cross the threshold, whoever is responsible + * for the low async space is likely to try to send another async txn, + * and at some point we'll catch them in the act. This is more efficient + * than keeping a map per pid. + */ + struct rb_node *n; + struct binder_buffer *buffer; + size_t total_alloc_size = 0; + size_t num_buffers = 0; + + for (n = rb_first(&alloc->allocated_buffers); n != NULL; + n = rb_next(n)) { + buffer = rb_entry(n, struct binder_buffer, rb_node); + if (buffer->pid != pid) + continue; + if (!buffer->async_transaction) + continue; + total_alloc_size += binder_alloc_buffer_size(alloc, buffer) + + sizeof(struct binder_buffer); + num_buffers++; + } + + /* + * Warn if this pid has more than 50 transactions, or more than 50% of + * async space (which is 25% of total buffer size). + */ + if (num_buffers > 50 || total_alloc_size > alloc->buffer_size / 4) { + binder_alloc_debug(BINDER_DEBUG_USER_ERROR, + "%d: pid %d spamming oneway? %zd buffers allocated for a total size of %zd\n", + alloc->pid, pid, num_buffers, total_alloc_size); + } +} + static struct binder_buffer *binder_alloc_new_buf_locked( struct binder_alloc *alloc, size_t data_size, size_t offsets_size, size_t extra_buffers_size, - int is_async) + int is_async, + int pid) { struct rb_node *n = alloc->free_buffers.rb_node; struct binder_buffer *buffer; @@ -486,11 +524,20 @@ static struct binder_buffer *binder_alloc_new_buf_locked( buffer->offsets_size = offsets_size; buffer->async_transaction = is_async; buffer->extra_buffers_size = extra_buffers_size; + buffer->pid = pid; if (is_async) { alloc->free_async_space -= size + sizeof(struct binder_buffer); binder_alloc_debug(BINDER_DEBUG_BUFFER_ALLOC_ASYNC, "%d: binder_alloc_buf size %zd async free %zd\n", alloc->pid, size, alloc->free_async_space); + if (alloc->free_async_space < alloc->buffer_size / 10) { + /* + * Start detecting spammers once we have less than 20% + * of async space left (which is less than 10% of total + * buffer size). + */ + debug_low_async_space_locked(alloc, pid); + } } return buffer; @@ -508,6 +555,7 @@ err_alloc_buf_struct_failed: * @offsets_size: user specified buffer offset * @extra_buffers_size: size of extra space for meta-data (eg, security context) * @is_async: buffer for async transaction + * @pid: pid to attribute allocation to (used for debugging) * * Allocate a new buffer given the requested sizes. Returns * the kernel version of the buffer pointer. The size allocated @@ -520,13 +568,14 @@ struct binder_buffer *binder_alloc_new_buf(struct binder_alloc *alloc, size_t data_size, size_t offsets_size, size_t extra_buffers_size, - int is_async) + int is_async, + int pid) { struct binder_buffer *buffer; mutex_lock(&alloc->mutex); buffer = binder_alloc_new_buf_locked(alloc, data_size, offsets_size, - extra_buffers_size, is_async); + extra_buffers_size, is_async, pid); mutex_unlock(&alloc->mutex); return buffer; } @@ -652,7 +701,7 @@ static void binder_free_buf_locked(struct binder_alloc *alloc, * @alloc: binder_alloc for this proc * @buffer: kernel pointer to buffer * - * Free the buffer allocated via binder_alloc_new_buffer() + * Free the buffer allocated via binder_alloc_new_buf() */ void binder_alloc_free_buf(struct binder_alloc *alloc, struct binder_buffer *buffer) diff --git a/drivers/android/binder_alloc.h b/drivers/android/binder_alloc.h index db9c1b984695..55d8b4106766 100644 --- a/drivers/android/binder_alloc.h +++ b/drivers/android/binder_alloc.h @@ -32,6 +32,7 @@ struct binder_transaction; * @offsets_size: size of array of offsets * @extra_buffers_size: size of space for other objects (like sg lists) * @user_data: user pointer to base of buffer space + * @pid: pid to attribute the buffer to (caller) * * Bookkeeping structure for binder transaction buffers */ @@ -51,6 +52,7 @@ struct binder_buffer { size_t offsets_size; size_t extra_buffers_size; void __user *user_data; + int pid; }; /** @@ -117,7 +119,8 @@ extern struct binder_buffer *binder_alloc_new_buf(struct binder_alloc *alloc, size_t data_size, size_t offsets_size, size_t extra_buffers_size, - int is_async); + int is_async, + int pid); extern void binder_alloc_init(struct binder_alloc *alloc); extern int binder_alloc_shrinker_init(void); extern void binder_alloc_vma_close(struct binder_alloc *alloc); diff --git a/drivers/android/binder_alloc_selftest.c b/drivers/android/binder_alloc_selftest.c index 4151d9938255..c2b323bc3b3a 100644 --- a/drivers/android/binder_alloc_selftest.c +++ b/drivers/android/binder_alloc_selftest.c @@ -119,7 +119,7 @@ static void binder_selftest_alloc_buf(struct binder_alloc *alloc, int i; for (i = 0; i < BUFFER_NUM; i++) { - buffers[i] = binder_alloc_new_buf(alloc, sizes[i], 0, 0, 0); + buffers[i] = binder_alloc_new_buf(alloc, sizes[i], 0, 0, 0, 0); if (IS_ERR(buffers[i]) || !check_buffer_pages_allocated(alloc, buffers[i], sizes[i])) { diff --git a/drivers/android/binderfs.c b/drivers/android/binderfs.c index 7b76fefde3f8..7b4f154f07e6 100644 --- a/drivers/android/binderfs.c +++ b/drivers/android/binderfs.c @@ -63,7 +63,7 @@ static const struct constant_table binderfs_param_stats[] = { {} }; -const struct fs_parameter_spec binderfs_fs_parameters[] = { +static const struct fs_parameter_spec binderfs_fs_parameters[] = { fsparam_u32("max", Opt_max), fsparam_enum("stats", Opt_stats_mode, binderfs_param_stats), {} diff --git a/drivers/base/firmware_loader/fallback.c b/drivers/base/firmware_loader/fallback.c index d60e6d8c967c..4dec4b79ae06 100644 --- a/drivers/base/firmware_loader/fallback.c +++ b/drivers/base/firmware_loader/fallback.c @@ -272,9 +272,9 @@ static ssize_t firmware_loading_store(struct device *dev, dev_err(dev, "%s: map pages failed\n", __func__); else - rc = security_kernel_post_read_file(NULL, - fw_priv->data, fw_priv->size, - READING_FIRMWARE); + rc = security_kernel_post_load_data(fw_priv->data, + fw_priv->size, + LOADING_FIRMWARE, "blob"); /* * Same logic as fw_load_abort, only the DONE bit @@ -490,13 +490,11 @@ exit: /** * fw_load_sysfs_fallback() - load a firmware via the sysfs fallback mechanism * @fw_sysfs: firmware sysfs information for the firmware to load - * @opt_flags: flags of options, FW_OPT_* * @timeout: timeout to wait for the load * * In charge of constructing a sysfs fallback interface for firmware loading. **/ -static int fw_load_sysfs_fallback(struct fw_sysfs *fw_sysfs, - u32 opt_flags, long timeout) +static int fw_load_sysfs_fallback(struct fw_sysfs *fw_sysfs, long timeout) { int retval = 0; struct device *f_dev = &fw_sysfs->dev; @@ -518,7 +516,7 @@ static int fw_load_sysfs_fallback(struct fw_sysfs *fw_sysfs, list_add(&fw_priv->pending_list, &pending_fw_head); mutex_unlock(&fw_lock); - if (opt_flags & FW_OPT_UEVENT) { + if (fw_priv->opt_flags & FW_OPT_UEVENT) { fw_priv->need_uevent = true; dev_set_uevent_suppress(f_dev, false); dev_dbg(f_dev, "firmware: requesting %s\n", fw_priv->fw_name); @@ -580,10 +578,10 @@ static int fw_load_from_user_helper(struct firmware *firmware, } fw_sysfs->fw_priv = firmware->priv; - ret = fw_load_sysfs_fallback(fw_sysfs, opt_flags, timeout); + ret = fw_load_sysfs_fallback(fw_sysfs, timeout); if (!ret) - ret = assign_fw(firmware, device, opt_flags); + ret = assign_fw(firmware, device); out_unlock: usermodehelper_read_unlock(); @@ -613,7 +611,7 @@ static bool fw_run_sysfs_fallback(u32 opt_flags) return false; /* Also permit LSMs and IMA to fail firmware sysfs fallback */ - ret = security_kernel_load_data(LOADING_FIRMWARE); + ret = security_kernel_load_data(LOADING_FIRMWARE, true); if (ret < 0) return false; @@ -625,7 +623,8 @@ static bool fw_run_sysfs_fallback(u32 opt_flags) * @fw: pointer to firmware image * @name: name of firmware file to look for * @device: device for which firmware is being loaded - * @opt_flags: options to control firmware loading behaviour + * @opt_flags: options to control firmware loading behaviour, as defined by + * &enum fw_opt * @ret: return value from direct lookup which triggered the fallback mechanism * * This function is called if direct lookup for the firmware failed, it enables diff --git a/drivers/base/firmware_loader/fallback.h b/drivers/base/firmware_loader/fallback.h index 2afdb6adb23f..3af7205b302f 100644 --- a/drivers/base/firmware_loader/fallback.h +++ b/drivers/base/firmware_loader/fallback.h @@ -67,10 +67,9 @@ static inline void unregister_sysfs_loader(void) #endif /* CONFIG_FW_LOADER_USER_HELPER */ #ifdef CONFIG_EFI_EMBEDDED_FIRMWARE -int firmware_fallback_platform(struct fw_priv *fw_priv, u32 opt_flags); +int firmware_fallback_platform(struct fw_priv *fw_priv); #else -static inline int firmware_fallback_platform(struct fw_priv *fw_priv, - u32 opt_flags) +static inline int firmware_fallback_platform(struct fw_priv *fw_priv) { return -ENOENT; } diff --git a/drivers/base/firmware_loader/fallback_platform.c b/drivers/base/firmware_loader/fallback_platform.c index 685edb7dd05a..00af99f0aff2 100644 --- a/drivers/base/firmware_loader/fallback_platform.c +++ b/drivers/base/firmware_loader/fallback_platform.c @@ -8,16 +8,16 @@ #include "fallback.h" #include "firmware.h" -int firmware_fallback_platform(struct fw_priv *fw_priv, u32 opt_flags) +int firmware_fallback_platform(struct fw_priv *fw_priv) { const u8 *data; size_t size; int rc; - if (!(opt_flags & FW_OPT_FALLBACK_PLATFORM)) + if (!(fw_priv->opt_flags & FW_OPT_FALLBACK_PLATFORM)) return -ENOENT; - rc = security_kernel_load_data(LOADING_FIRMWARE_EFI_EMBEDDED); + rc = security_kernel_load_data(LOADING_FIRMWARE, true); if (rc) return rc; @@ -27,6 +27,12 @@ int firmware_fallback_platform(struct fw_priv *fw_priv, u32 opt_flags) if (fw_priv->data && size > fw_priv->allocated_size) return -ENOMEM; + + rc = security_kernel_post_load_data((u8 *)data, size, LOADING_FIRMWARE, + "platform"); + if (rc) + return rc; + if (!fw_priv->data) fw_priv->data = vmalloc(size); if (!fw_priv->data) diff --git a/drivers/base/firmware_loader/firmware.h b/drivers/base/firmware_loader/firmware.h index d08efc77cf16..63bd29fdcb9c 100644 --- a/drivers/base/firmware_loader/firmware.h +++ b/drivers/base/firmware_loader/firmware.h @@ -32,6 +32,8 @@ * @FW_OPT_FALLBACK_PLATFORM: Enable fallback to device fw copy embedded in * the platform's main firmware. If both this fallback and the sysfs * fallback are enabled, then this fallback will be tried first. + * @FW_OPT_PARTIAL: Allow partial read of firmware instead of needing to read + * entire file. */ enum fw_opt { FW_OPT_UEVENT = BIT(0), @@ -41,6 +43,7 @@ enum fw_opt { FW_OPT_NOCACHE = BIT(4), FW_OPT_NOFALLBACK_SYSFS = BIT(5), FW_OPT_FALLBACK_PLATFORM = BIT(6), + FW_OPT_PARTIAL = BIT(7), }; enum fw_status { @@ -68,6 +71,8 @@ struct fw_priv { void *data; size_t size; size_t allocated_size; + size_t offset; + u32 opt_flags; #ifdef CONFIG_FW_LOADER_PAGED_BUF bool is_paged_buf; struct page **pages; @@ -136,7 +141,7 @@ static inline void fw_state_done(struct fw_priv *fw_priv) __fw_state_set(fw_priv, FW_STATUS_DONE); } -int assign_fw(struct firmware *fw, struct device *device, u32 opt_flags); +int assign_fw(struct firmware *fw, struct device *device); #ifdef CONFIG_FW_LOADER_PAGED_BUF void fw_free_paged_buf(struct fw_priv *fw_priv); diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c index 63b9714a0154..78355095e00d 100644 --- a/drivers/base/firmware_loader/main.c +++ b/drivers/base/firmware_loader/main.c @@ -12,6 +12,7 @@ #include <linux/capability.h> #include <linux/device.h> +#include <linux/kernel_read_file.h> #include <linux/module.h> #include <linux/init.h> #include <linux/timer.h> @@ -167,10 +168,21 @@ static int fw_cache_piggyback_on_request(const char *name); static struct fw_priv *__allocate_fw_priv(const char *fw_name, struct firmware_cache *fwc, - void *dbuf, size_t size) + void *dbuf, + size_t size, + size_t offset, + u32 opt_flags) { struct fw_priv *fw_priv; + /* For a partial read, the buffer must be preallocated. */ + if ((opt_flags & FW_OPT_PARTIAL) && !dbuf) + return NULL; + + /* Only partial reads are allowed to use an offset. */ + if (offset != 0 && !(opt_flags & FW_OPT_PARTIAL)) + return NULL; + fw_priv = kzalloc(sizeof(*fw_priv), GFP_ATOMIC); if (!fw_priv) return NULL; @@ -185,6 +197,8 @@ static struct fw_priv *__allocate_fw_priv(const char *fw_name, fw_priv->fwc = fwc; fw_priv->data = dbuf; fw_priv->allocated_size = size; + fw_priv->offset = offset; + fw_priv->opt_flags = opt_flags; fw_state_init(fw_priv); #ifdef CONFIG_FW_LOADER_USER_HELPER INIT_LIST_HEAD(&fw_priv->pending_list); @@ -209,13 +223,20 @@ static struct fw_priv *__lookup_fw_priv(const char *fw_name) /* Returns 1 for batching firmware requests with the same name */ static int alloc_lookup_fw_priv(const char *fw_name, struct firmware_cache *fwc, - struct fw_priv **fw_priv, void *dbuf, - size_t size, u32 opt_flags) + struct fw_priv **fw_priv, + void *dbuf, + size_t size, + size_t offset, + u32 opt_flags) { struct fw_priv *tmp; spin_lock(&fwc->lock); - if (!(opt_flags & FW_OPT_NOCACHE)) { + /* + * Do not merge requests that are marked to be non-cached or + * are performing partial reads. + */ + if (!(opt_flags & (FW_OPT_NOCACHE | FW_OPT_PARTIAL))) { tmp = __lookup_fw_priv(fw_name); if (tmp) { kref_get(&tmp->ref); @@ -226,7 +247,7 @@ static int alloc_lookup_fw_priv(const char *fw_name, } } - tmp = __allocate_fw_priv(fw_name, fwc, dbuf, size); + tmp = __allocate_fw_priv(fw_name, fwc, dbuf, size, offset, opt_flags); if (tmp) { INIT_LIST_HEAD(&tmp->list); if (!(opt_flags & FW_OPT_NOCACHE)) @@ -466,18 +487,16 @@ fw_get_filesystem_firmware(struct device *device, struct fw_priv *fw_priv, size_t in_size, const void *in_buffer)) { - loff_t size; + size_t size; int i, len; int rc = -ENOENT; char *path; - enum kernel_read_file_id id = READING_FIRMWARE; size_t msize = INT_MAX; void *buffer = NULL; /* Already populated data member means we're loading into a buffer */ if (!decompress && fw_priv->data) { buffer = fw_priv->data; - id = READING_FIRMWARE_PREALLOC_BUFFER; msize = fw_priv->allocated_size; } @@ -486,6 +505,9 @@ fw_get_filesystem_firmware(struct device *device, struct fw_priv *fw_priv, return -ENOMEM; for (i = 0; i < ARRAY_SIZE(fw_path); i++) { + size_t file_size = 0; + size_t *file_size_ptr = NULL; + /* skip the unset customized path */ if (!fw_path[i][0]) continue; @@ -499,10 +521,20 @@ fw_get_filesystem_firmware(struct device *device, struct fw_priv *fw_priv, fw_priv->size = 0; + /* + * The total file size is only examined when doing a partial + * read; the "full read" case needs to fail if the whole + * firmware was not completely loaded. + */ + if ((fw_priv->opt_flags & FW_OPT_PARTIAL) && buffer) + file_size_ptr = &file_size; + /* load firmware files from the mount namespace of init */ - rc = kernel_read_file_from_path_initns(path, &buffer, - &size, msize, id); - if (rc) { + rc = kernel_read_file_from_path_initns(path, fw_priv->offset, + &buffer, msize, + file_size_ptr, + READING_FIRMWARE); + if (rc < 0) { if (rc != -ENOENT) dev_warn(device, "loading %s failed with error %d\n", path, rc); @@ -511,6 +543,9 @@ fw_get_filesystem_firmware(struct device *device, struct fw_priv *fw_priv, path); continue; } + size = rc; + rc = 0; + dev_dbg(device, "Loading firmware from %s\n", path); if (decompress) { dev_dbg(device, "f/w decompressing %s\n", @@ -637,7 +672,7 @@ static int fw_add_devm_name(struct device *dev, const char *name) } #endif -int assign_fw(struct firmware *fw, struct device *device, u32 opt_flags) +int assign_fw(struct firmware *fw, struct device *device) { struct fw_priv *fw_priv = fw->priv; int ret; @@ -656,8 +691,8 @@ int assign_fw(struct firmware *fw, struct device *device, u32 opt_flags) * should be fixed in devres or driver core. */ /* don't cache firmware handled without uevent */ - if (device && (opt_flags & FW_OPT_UEVENT) && - !(opt_flags & FW_OPT_NOCACHE)) { + if (device && (fw_priv->opt_flags & FW_OPT_UEVENT) && + !(fw_priv->opt_flags & FW_OPT_NOCACHE)) { ret = fw_add_devm_name(device, fw_priv->fw_name); if (ret) { mutex_unlock(&fw_lock); @@ -669,7 +704,7 @@ int assign_fw(struct firmware *fw, struct device *device, u32 opt_flags) * After caching firmware image is started, let it piggyback * on request firmware. */ - if (!(opt_flags & FW_OPT_NOCACHE) && + if (!(fw_priv->opt_flags & FW_OPT_NOCACHE) && fw_priv->fwc->state == FW_LOADER_START_CACHE) { if (fw_cache_piggyback_on_request(fw_priv->fw_name)) kref_get(&fw_priv->ref); @@ -688,7 +723,7 @@ int assign_fw(struct firmware *fw, struct device *device, u32 opt_flags) static int _request_firmware_prepare(struct firmware **firmware_p, const char *name, struct device *device, void *dbuf, size_t size, - u32 opt_flags) + size_t offset, u32 opt_flags) { struct firmware *firmware; struct fw_priv *fw_priv; @@ -707,7 +742,7 @@ _request_firmware_prepare(struct firmware **firmware_p, const char *name, } ret = alloc_lookup_fw_priv(name, &fw_cache, &fw_priv, dbuf, size, - opt_flags); + offset, opt_flags); /* * bind with 'priv' now to avoid warning in failure path @@ -754,9 +789,10 @@ static void fw_abort_batch_reqs(struct firmware *fw) static int _request_firmware(const struct firmware **firmware_p, const char *name, struct device *device, void *buf, size_t size, - u32 opt_flags) + size_t offset, u32 opt_flags) { struct firmware *fw = NULL; + bool nondirect = false; int ret; if (!firmware_p) @@ -768,28 +804,34 @@ _request_firmware(const struct firmware **firmware_p, const char *name, } ret = _request_firmware_prepare(&fw, name, device, buf, size, - opt_flags); + offset, opt_flags); if (ret <= 0) /* error or already assigned */ goto out; ret = fw_get_filesystem_firmware(device, fw->priv, "", NULL); + + /* Only full reads can support decompression, platform, and sysfs. */ + if (!(opt_flags & FW_OPT_PARTIAL)) + nondirect = true; + #ifdef CONFIG_FW_LOADER_COMPRESS - if (ret == -ENOENT) + if (ret == -ENOENT && nondirect) ret = fw_get_filesystem_firmware(device, fw->priv, ".xz", fw_decompress_xz); #endif - - if (ret == -ENOENT) - ret = firmware_fallback_platform(fw->priv, opt_flags); + if (ret == -ENOENT && nondirect) + ret = firmware_fallback_platform(fw->priv); if (ret) { if (!(opt_flags & FW_OPT_NO_WARN)) dev_warn(device, "Direct firmware load for %s failed with error %d\n", name, ret); - ret = firmware_fallback_sysfs(fw, name, device, opt_flags, ret); + if (nondirect) + ret = firmware_fallback_sysfs(fw, name, device, + opt_flags, ret); } else - ret = assign_fw(fw, device, opt_flags); + ret = assign_fw(fw, device); out: if (ret < 0) { @@ -830,7 +872,7 @@ request_firmware(const struct firmware **firmware_p, const char *name, /* Need to pin this module until return */ __module_get(THIS_MODULE); - ret = _request_firmware(firmware_p, name, device, NULL, 0, + ret = _request_firmware(firmware_p, name, device, NULL, 0, 0, FW_OPT_UEVENT); module_put(THIS_MODULE); return ret; @@ -857,7 +899,7 @@ int firmware_request_nowarn(const struct firmware **firmware, const char *name, /* Need to pin this module until return */ __module_get(THIS_MODULE); - ret = _request_firmware(firmware, name, device, NULL, 0, + ret = _request_firmware(firmware, name, device, NULL, 0, 0, FW_OPT_UEVENT | FW_OPT_NO_WARN); module_put(THIS_MODULE); return ret; @@ -881,7 +923,7 @@ int request_firmware_direct(const struct firmware **firmware_p, int ret; __module_get(THIS_MODULE); - ret = _request_firmware(firmware_p, name, device, NULL, 0, + ret = _request_firmware(firmware_p, name, device, NULL, 0, 0, FW_OPT_UEVENT | FW_OPT_NO_WARN | FW_OPT_NOFALLBACK_SYSFS); module_put(THIS_MODULE); @@ -906,7 +948,7 @@ int firmware_request_platform(const struct firmware **firmware, /* Need to pin this module until return */ __module_get(THIS_MODULE); - ret = _request_firmware(firmware, name, device, NULL, 0, + ret = _request_firmware(firmware, name, device, NULL, 0, 0, FW_OPT_UEVENT | FW_OPT_FALLBACK_PLATFORM); module_put(THIS_MODULE); return ret; @@ -962,7 +1004,7 @@ request_firmware_into_buf(const struct firmware **firmware_p, const char *name, return -EOPNOTSUPP; __module_get(THIS_MODULE); - ret = _request_firmware(firmware_p, name, device, buf, size, + ret = _request_firmware(firmware_p, name, device, buf, size, 0, FW_OPT_UEVENT | FW_OPT_NOCACHE); module_put(THIS_MODULE); return ret; @@ -970,6 +1012,37 @@ request_firmware_into_buf(const struct firmware **firmware_p, const char *name, EXPORT_SYMBOL(request_firmware_into_buf); /** + * request_partial_firmware_into_buf() - load partial firmware into a previously allocated buffer + * @firmware_p: pointer to firmware image + * @name: name of firmware file + * @device: device for which firmware is being loaded and DMA region allocated + * @buf: address of buffer to load firmware into + * @size: size of buffer + * @offset: offset into file to read + * + * This function works pretty much like request_firmware_into_buf except + * it allows a partial read of the file. + */ +int +request_partial_firmware_into_buf(const struct firmware **firmware_p, + const char *name, struct device *device, + void *buf, size_t size, size_t offset) +{ + int ret; + + if (fw_cache_is_setup(device, name)) + return -EOPNOTSUPP; + + __module_get(THIS_MODULE); + ret = _request_firmware(firmware_p, name, device, buf, size, offset, + FW_OPT_UEVENT | FW_OPT_NOCACHE | + FW_OPT_PARTIAL); + module_put(THIS_MODULE); + return ret; +} +EXPORT_SYMBOL(request_partial_firmware_into_buf); + +/** * release_firmware() - release the resource associated with a firmware image * @fw: firmware resource to release **/ @@ -1001,7 +1074,7 @@ static void request_firmware_work_func(struct work_struct *work) fw_work = container_of(work, struct firmware_work, work); - _request_firmware(&fw, fw_work->name, fw_work->device, NULL, 0, + _request_firmware(&fw, fw_work->name, fw_work->device, NULL, 0, 0, fw_work->opt_flags); fw_work->cont(fw, fw_work->context); put_device(fw_work->device); /* taken in request_firmware_nowait() */ diff --git a/drivers/bus/fsl-mc/dprc-driver.c b/drivers/bus/fsl-mc/dprc-driver.c index 2a473c09bc33..91dc015963a8 100644 --- a/drivers/bus/fsl-mc/dprc-driver.c +++ b/drivers/bus/fsl-mc/dprc-driver.c @@ -3,6 +3,7 @@ * Freescale data path resource container (DPRC) driver * * Copyright (C) 2014-2016 Freescale Semiconductor, Inc. + * Copyright 2019-2020 NXP * Author: German Rivera <German.Rivera@freescale.com> * */ @@ -80,9 +81,9 @@ static int __fsl_mc_device_remove(struct device *dev, void *data) * the MC by removing devices that represent MC objects that have * been dynamically removed in the physical DPRC. */ -static void dprc_remove_devices(struct fsl_mc_device *mc_bus_dev, - struct fsl_mc_obj_desc *obj_desc_array, - int num_child_objects_in_mc) +void dprc_remove_devices(struct fsl_mc_device *mc_bus_dev, + struct fsl_mc_obj_desc *obj_desc_array, + int num_child_objects_in_mc) { if (num_child_objects_in_mc != 0) { /* @@ -104,6 +105,7 @@ static void dprc_remove_devices(struct fsl_mc_device *mc_bus_dev, __fsl_mc_device_remove); } } +EXPORT_SYMBOL_GPL(dprc_remove_devices); static int __fsl_mc_device_match(struct device *dev, void *data) { @@ -220,8 +222,8 @@ static void dprc_add_new_devices(struct fsl_mc_device *mc_bus_dev, * dprc_scan_objects - Discover objects in a DPRC * * @mc_bus_dev: pointer to the fsl-mc device that represents a DPRC object - * @total_irq_count: If argument is provided the function populates the - * total number of IRQs created by objects in the DPRC. + * @alloc_interrupts: if true the function allocates the interrupt pool, + * otherwise the interrupt allocation is delayed * * Detects objects added and removed from a DPRC and synchronizes the * state of the Linux bus driver, MC by adding and removing @@ -236,7 +238,7 @@ static void dprc_add_new_devices(struct fsl_mc_device *mc_bus_dev, * of the device drivers for the non-allocatable devices. */ static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev, - unsigned int *total_irq_count) + bool alloc_interrupts) { int num_child_objects; int dprc_get_obj_failures; @@ -317,22 +319,21 @@ static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev, * Allocate IRQ's before binding the scanned devices with their * respective drivers. */ - if (dev_get_msi_domain(&mc_bus_dev->dev) && !mc_bus->irq_resources) { + if (dev_get_msi_domain(&mc_bus_dev->dev)) { if (irq_count > FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS) { dev_warn(&mc_bus_dev->dev, "IRQs needed (%u) exceed IRQs preallocated (%u)\n", irq_count, FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS); } - error = fsl_mc_populate_irq_pool(mc_bus, - FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS); - if (error < 0) - return error; + if (alloc_interrupts && !mc_bus->irq_resources) { + error = fsl_mc_populate_irq_pool(mc_bus_dev, + FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS); + if (error < 0) + return error; + } } - if (total_irq_count) - *total_irq_count = irq_count; - dprc_remove_devices(mc_bus_dev, child_obj_desc_array, num_child_objects); @@ -354,9 +355,10 @@ static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev, * bus driver with the actual state of the MC by adding and removing * devices as appropriate. */ -static int dprc_scan_container(struct fsl_mc_device *mc_bus_dev) +int dprc_scan_container(struct fsl_mc_device *mc_bus_dev, + bool alloc_interrupts) { - int error; + int error = 0; struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_bus_dev); fsl_mc_init_all_resource_pools(mc_bus_dev); @@ -365,16 +367,12 @@ static int dprc_scan_container(struct fsl_mc_device *mc_bus_dev) * Discover objects in the DPRC: */ mutex_lock(&mc_bus->scan_mutex); - error = dprc_scan_objects(mc_bus_dev, NULL); + error = dprc_scan_objects(mc_bus_dev, alloc_interrupts); mutex_unlock(&mc_bus->scan_mutex); - if (error < 0) { - fsl_mc_cleanup_all_resource_pools(mc_bus_dev); - return error; - } - return 0; + return error; } - +EXPORT_SYMBOL_GPL(dprc_scan_container); /** * dprc_irq0_handler - Regular ISR for DPRC interrupt 0 * @@ -434,9 +432,8 @@ static irqreturn_t dprc_irq0_handler_thread(int irq_num, void *arg) DPRC_IRQ_EVENT_CONTAINER_DESTROYED | DPRC_IRQ_EVENT_OBJ_DESTROYED | DPRC_IRQ_EVENT_OBJ_CREATED)) { - unsigned int irq_count; - error = dprc_scan_objects(mc_dev, &irq_count); + error = dprc_scan_objects(mc_dev, true); if (error < 0) { /* * If the error is -ENXIO, we ignore it, as it indicates @@ -451,12 +448,6 @@ static irqreturn_t dprc_irq0_handler_thread(int irq_num, void *arg) goto out; } - - if (irq_count > FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS) { - dev_warn(dev, - "IRQs needed (%u) exceed IRQs preallocated (%u)\n", - irq_count, FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS); - } } out: @@ -597,25 +588,24 @@ error_free_irqs: } /** - * dprc_probe - callback invoked when a DPRC is being bound to this driver + * dprc_setup - opens and creates a mc_io for DPRC * * @mc_dev: Pointer to fsl-mc device representing a DPRC * * It opens the physical DPRC in the MC. - * It scans the DPRC to discover the MC objects contained in it. - * It creates the interrupt pool for the MC bus associated with the DPRC. - * It configures the interrupts for the DPRC device itself. + * It configures the DPRC portal used to communicate with MC */ -static int dprc_probe(struct fsl_mc_device *mc_dev) + +int dprc_setup(struct fsl_mc_device *mc_dev) { - int error; - size_t region_size; struct device *parent_dev = mc_dev->dev.parent; struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_dev); + struct irq_domain *mc_msi_domain; bool mc_io_created = false; bool msi_domain_set = false; u16 major_ver, minor_ver; - struct irq_domain *mc_msi_domain; + size_t region_size; + int error; if (!is_fsl_mc_bus_dprc(mc_dev)) return -EINVAL; @@ -690,37 +680,63 @@ static int dprc_probe(struct fsl_mc_device *mc_dev) goto error_cleanup_open; } - mutex_init(&mc_bus->scan_mutex); + return 0; + +error_cleanup_open: + (void)dprc_close(mc_dev->mc_io, 0, mc_dev->mc_handle); + +error_cleanup_msi_domain: + if (msi_domain_set) + dev_set_msi_domain(&mc_dev->dev, NULL); + + if (mc_io_created) { + fsl_destroy_mc_io(mc_dev->mc_io); + mc_dev->mc_io = NULL; + } + + return error; +} +EXPORT_SYMBOL_GPL(dprc_setup); + +/** + * dprc_probe - callback invoked when a DPRC is being bound to this driver + * + * @mc_dev: Pointer to fsl-mc device representing a DPRC + * + * It opens the physical DPRC in the MC. + * It scans the DPRC to discover the MC objects contained in it. + * It creates the interrupt pool for the MC bus associated with the DPRC. + * It configures the interrupts for the DPRC device itself. + */ +static int dprc_probe(struct fsl_mc_device *mc_dev) +{ + int error; + + error = dprc_setup(mc_dev); + if (error < 0) + return error; /* * Discover MC objects in DPRC object: */ - error = dprc_scan_container(mc_dev); + error = dprc_scan_container(mc_dev, true); if (error < 0) - goto error_cleanup_open; + goto dprc_cleanup; /* * Configure interrupt for the DPRC object associated with this MC bus: */ error = dprc_setup_irq(mc_dev); if (error < 0) - goto error_cleanup_open; + goto scan_cleanup; dev_info(&mc_dev->dev, "DPRC device bound to driver"); return 0; -error_cleanup_open: - (void)dprc_close(mc_dev->mc_io, 0, mc_dev->mc_handle); - -error_cleanup_msi_domain: - if (msi_domain_set) - dev_set_msi_domain(&mc_dev->dev, NULL); - - if (mc_io_created) { - fsl_destroy_mc_io(mc_dev->mc_io); - mc_dev->mc_io = NULL; - } - +scan_cleanup: + device_for_each_child(&mc_dev->dev, NULL, __fsl_mc_device_remove); +dprc_cleanup: + dprc_cleanup(mc_dev); return error; } @@ -739,40 +755,39 @@ static void dprc_teardown_irq(struct fsl_mc_device *mc_dev) } /** - * dprc_remove - callback invoked when a DPRC is being unbound from this driver + * dprc_cleanup - function that cleanups a DPRC * * @mc_dev: Pointer to fsl-mc device representing the DPRC * - * It removes the DPRC's child objects from Linux (not from the MC) and - * closes the DPRC device in the MC. - * It tears down the interrupts that were configured for the DPRC device. + * It closes the DPRC device in the MC. * It destroys the interrupt pool associated with this MC bus. */ -static int dprc_remove(struct fsl_mc_device *mc_dev) + +int dprc_cleanup(struct fsl_mc_device *mc_dev) { int error; - struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_dev); + /* this function should be called only for DPRCs, it + * is an error to call it for regular objects + */ if (!is_fsl_mc_bus_dprc(mc_dev)) return -EINVAL; - if (!mc_dev->mc_io) - return -EINVAL; - - if (!mc_bus->irq_resources) - return -EINVAL; - - if (dev_get_msi_domain(&mc_dev->dev)) - dprc_teardown_irq(mc_dev); - - device_for_each_child(&mc_dev->dev, NULL, __fsl_mc_device_remove); if (dev_get_msi_domain(&mc_dev->dev)) { - fsl_mc_cleanup_irq_pool(mc_bus); + fsl_mc_cleanup_irq_pool(mc_dev); dev_set_msi_domain(&mc_dev->dev, NULL); } fsl_mc_cleanup_all_resource_pools(mc_dev); + /* if this step fails we cannot go further with cleanup as there is no way of + * communicating with the firmware + */ + if (!mc_dev->mc_io) { + dev_err(&mc_dev->dev, "mc_io is NULL, tear down cannot be performed in firmware\n"); + return -EINVAL; + } + error = dprc_close(mc_dev->mc_io, 0, mc_dev->mc_handle); if (error < 0) dev_err(&mc_dev->dev, "dprc_close() failed: %d\n", error); @@ -782,6 +797,37 @@ static int dprc_remove(struct fsl_mc_device *mc_dev) mc_dev->mc_io = NULL; } + return 0; +} +EXPORT_SYMBOL_GPL(dprc_cleanup); + +/** + * dprc_remove - callback invoked when a DPRC is being unbound from this driver + * + * @mc_dev: Pointer to fsl-mc device representing the DPRC + * + * It removes the DPRC's child objects from Linux (not from the MC) and + * closes the DPRC device in the MC. + * It tears down the interrupts that were configured for the DPRC device. + * It destroys the interrupt pool associated with this MC bus. + */ +static int dprc_remove(struct fsl_mc_device *mc_dev) +{ + struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_dev); + + if (!is_fsl_mc_bus_dprc(mc_dev)) + return -EINVAL; + + if (!mc_bus->irq_resources) + return -EINVAL; + + if (dev_get_msi_domain(&mc_dev->dev)) + dprc_teardown_irq(mc_dev); + + device_for_each_child(&mc_dev->dev, NULL, __fsl_mc_device_remove); + + dprc_cleanup(mc_dev); + dev_info(&mc_dev->dev, "DPRC device unbound from driver"); return 0; } diff --git a/drivers/bus/fsl-mc/dprc.c b/drivers/bus/fsl-mc/dprc.c index 602f030d84eb..57b097caf255 100644 --- a/drivers/bus/fsl-mc/dprc.c +++ b/drivers/bus/fsl-mc/dprc.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) /* * Copyright 2013-2016 Freescale Semiconductor Inc. + * Copyright 2020 NXP * */ #include <linux/kernel.h> @@ -8,6 +9,13 @@ #include "fsl-mc-private.h" +/* + * cache the DPRC version to reduce the number of commands + * towards the mc firmware + */ +static u16 dprc_major_ver; +static u16 dprc_minor_ver; + /** * dprc_open() - Open DPRC object for use * @mc_io: Pointer to MC portal's I/O object @@ -73,6 +81,77 @@ int dprc_close(struct fsl_mc_io *mc_io, EXPORT_SYMBOL_GPL(dprc_close); /** + * dprc_reset_container - Reset child container. + * @mc_io: Pointer to MC portal's I/O object + * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_' + * @token: Token of DPRC object + * @child_container_id: ID of the container to reset + * @options: 32 bit options: + * - 0 (no bits set) - all the objects inside the container are + * reset. The child containers are entered recursively and the + * objects reset. All the objects (including the child containers) + * are closed. + * - bit 0 set - all the objects inside the container are reset. + * However the child containers are not entered recursively. + * This option is supported for API versions >= 6.5 + * In case a software context crashes or becomes non-responsive, the parent + * may wish to reset its resources container before the software context is + * restarted. + * + * This routine informs all objects assigned to the child container that the + * container is being reset, so they may perform any cleanup operations that are + * needed. All objects handles that were owned by the child container shall be + * closed. + * + * Note that such request may be submitted even if the child software context + * has not crashed, but the resulting object cleanup operations will not be + * aware of that. + * + * Return: '0' on Success; Error code otherwise. + */ +int dprc_reset_container(struct fsl_mc_io *mc_io, + u32 cmd_flags, + u16 token, + int child_container_id, + u32 options) +{ + struct fsl_mc_command cmd = { 0 }; + struct dprc_cmd_reset_container *cmd_params; + u32 cmdid = DPRC_CMDID_RESET_CONT; + int err; + + /* + * If the DPRC object version was not yet cached, cache it now. + * Otherwise use the already cached value. + */ + if (!dprc_major_ver && !dprc_minor_ver) { + err = dprc_get_api_version(mc_io, 0, + &dprc_major_ver, + &dprc_minor_ver); + if (err) + return err; + } + + /* + * MC API 6.5 introduced a new field in the command used to pass + * some flags. + * Bit 0 indicates that the child containers are not recursively reset. + */ + if (dprc_major_ver > 6 || (dprc_major_ver == 6 && dprc_minor_ver >= 5)) + cmdid = DPRC_CMDID_RESET_CONT_V2; + + /* prepare command */ + cmd.header = mc_encode_cmd_header(cmdid, cmd_flags, token); + cmd_params = (struct dprc_cmd_reset_container *)cmd.params; + cmd_params->child_container_id = cpu_to_le32(child_container_id); + cmd_params->options = cpu_to_le32(options); + + /* send command to mc*/ + return mc_send_command(mc_io, &cmd); +} +EXPORT_SYMBOL_GPL(dprc_reset_container); + +/** * dprc_set_irq() - Set IRQ information for the DPRC to trigger an interrupt. * @mc_io: Pointer to MC portal's I/O object * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_' @@ -281,7 +360,7 @@ int dprc_get_attributes(struct fsl_mc_io *mc_io, /* retrieve response parameters */ rsp_params = (struct dprc_rsp_get_attributes *)cmd.params; attr->container_id = le32_to_cpu(rsp_params->container_id); - attr->icid = le16_to_cpu(rsp_params->icid); + attr->icid = le32_to_cpu(rsp_params->icid); attr->options = le32_to_cpu(rsp_params->options); attr->portal_id = le32_to_cpu(rsp_params->portal_id); @@ -443,30 +522,44 @@ int dprc_get_obj_region(struct fsl_mc_io *mc_io, struct fsl_mc_command cmd = { 0 }; struct dprc_cmd_get_obj_region *cmd_params; struct dprc_rsp_get_obj_region *rsp_params; - u16 major_ver, minor_ver; int err; - /* prepare command */ - err = dprc_get_api_version(mc_io, 0, - &major_ver, - &minor_ver); - if (err) - return err; - - /** - * MC API version 6.3 introduced a new field to the region - * descriptor: base_address. If the older API is in use then the base - * address is set to zero to indicate it needs to be obtained elsewhere - * (typically the device tree). - */ - if (major_ver > 6 || (major_ver == 6 && minor_ver >= 3)) - cmd.header = - mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG_V2, - cmd_flags, token); - else - cmd.header = - mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG, - cmd_flags, token); + /* + * If the DPRC object version was not yet cached, cache it now. + * Otherwise use the already cached value. + */ + if (!dprc_major_ver && !dprc_minor_ver) { + err = dprc_get_api_version(mc_io, 0, + &dprc_major_ver, + &dprc_minor_ver); + if (err) + return err; + } + + if (dprc_major_ver > 6 || (dprc_major_ver == 6 && dprc_minor_ver >= 6)) { + /* + * MC API version 6.6 changed the size of the MC portals and software + * portals to 64K (as implemented by hardware). If older API is in use the + * size reported is less (64 bytes for mc portals and 4K for software + * portals). + */ + + cmd.header = mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG_V3, + cmd_flags, token); + + } else if (dprc_major_ver == 6 && dprc_minor_ver >= 3) { + /* + * MC API version 6.3 introduced a new field to the region + * descriptor: base_address. If the older API is in use then the base + * address is set to zero to indicate it needs to be obtained elsewhere + * (typically the device tree). + */ + cmd.header = mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG_V2, + cmd_flags, token); + } else { + cmd.header = mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG, + cmd_flags, token); + } cmd_params = (struct dprc_cmd_get_obj_region *)cmd.params; cmd_params->obj_id = cpu_to_le32(obj_id); @@ -483,7 +576,7 @@ int dprc_get_obj_region(struct fsl_mc_io *mc_io, rsp_params = (struct dprc_rsp_get_obj_region *)cmd.params; region_desc->base_offset = le64_to_cpu(rsp_params->base_offset); region_desc->size = le32_to_cpu(rsp_params->size); - if (major_ver > 6 || (major_ver == 6 && minor_ver >= 3)) + if (dprc_major_ver > 6 || (dprc_major_ver == 6 && dprc_minor_ver >= 3)) region_desc->base_address = le64_to_cpu(rsp_params->base_addr); else region_desc->base_address = 0; diff --git a/drivers/bus/fsl-mc/fsl-mc-allocator.c b/drivers/bus/fsl-mc/fsl-mc-allocator.c index cc7bb900f524..e71a6f52ea0c 100644 --- a/drivers/bus/fsl-mc/fsl-mc-allocator.c +++ b/drivers/bus/fsl-mc/fsl-mc-allocator.c @@ -344,7 +344,7 @@ EXPORT_SYMBOL_GPL(fsl_mc_object_free); * Initialize the interrupt pool associated with an fsl-mc bus. * It allocates a block of IRQs from the GIC-ITS. */ -int fsl_mc_populate_irq_pool(struct fsl_mc_bus *mc_bus, +int fsl_mc_populate_irq_pool(struct fsl_mc_device *mc_bus_dev, unsigned int irq_count) { unsigned int i; @@ -352,10 +352,14 @@ int fsl_mc_populate_irq_pool(struct fsl_mc_bus *mc_bus, struct fsl_mc_device_irq *irq_resources; struct fsl_mc_device_irq *mc_dev_irq; int error; - struct fsl_mc_device *mc_bus_dev = &mc_bus->mc_dev; + struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_bus_dev); struct fsl_mc_resource_pool *res_pool = &mc_bus->resource_pools[FSL_MC_POOL_IRQ]; + /* do nothing if the IRQ pool is already populated */ + if (mc_bus->irq_resources) + return 0; + if (irq_count == 0 || irq_count > FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS) return -EINVAL; @@ -407,9 +411,9 @@ EXPORT_SYMBOL_GPL(fsl_mc_populate_irq_pool); * Teardown the interrupt pool associated with an fsl-mc bus. * It frees the IRQs that were allocated to the pool, back to the GIC-ITS. */ -void fsl_mc_cleanup_irq_pool(struct fsl_mc_bus *mc_bus) +void fsl_mc_cleanup_irq_pool(struct fsl_mc_device *mc_bus_dev) { - struct fsl_mc_device *mc_bus_dev = &mc_bus->mc_dev; + struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc_bus_dev); struct fsl_mc_resource_pool *res_pool = &mc_bus->resource_pools[FSL_MC_POOL_IRQ]; diff --git a/drivers/bus/fsl-mc/fsl-mc-bus.c b/drivers/bus/fsl-mc/fsl-mc-bus.c index b69794e7364d..76a6ee505d33 100644 --- a/drivers/bus/fsl-mc/fsl-mc-bus.c +++ b/drivers/bus/fsl-mc/fsl-mc-bus.c @@ -3,6 +3,7 @@ * Freescale Management Complex (MC) bus driver * * Copyright (C) 2014-2016 Freescale Semiconductor, Inc. + * Copyright 2019-2020 NXP * Author: German Rivera <German.Rivera@freescale.com> * */ @@ -78,6 +79,12 @@ static int fsl_mc_bus_match(struct device *dev, struct device_driver *drv) struct fsl_mc_driver *mc_drv = to_fsl_mc_driver(drv); bool found = false; + /* When driver_override is set, only bind to the matching driver */ + if (mc_dev->driver_override) { + found = !strcmp(mc_dev->driver_override, mc_drv->driver.name); + goto out; + } + if (!mc_drv->match_id_table) goto out; @@ -147,8 +154,52 @@ static ssize_t modalias_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(modalias); +static ssize_t driver_override_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev); + char *driver_override, *old = mc_dev->driver_override; + char *cp; + + if (WARN_ON(dev->bus != &fsl_mc_bus_type)) + return -EINVAL; + + if (count >= (PAGE_SIZE - 1)) + return -EINVAL; + + driver_override = kstrndup(buf, count, GFP_KERNEL); + if (!driver_override) + return -ENOMEM; + + cp = strchr(driver_override, '\n'); + if (cp) + *cp = '\0'; + + if (strlen(driver_override)) { + mc_dev->driver_override = driver_override; + } else { + kfree(driver_override); + mc_dev->driver_override = NULL; + } + + kfree(old); + + return count; +} + +static ssize_t driver_override_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev); + + return snprintf(buf, PAGE_SIZE, "%s\n", mc_dev->driver_override); +} +static DEVICE_ATTR_RW(driver_override); + static struct attribute *fsl_mc_dev_attrs[] = { &dev_attr_modalias.attr, + &dev_attr_driver_override.attr, NULL, }; @@ -452,7 +503,7 @@ common_cleanup: } static int get_dprc_icid(struct fsl_mc_io *mc_io, - int container_id, u16 *icid) + int container_id, u32 *icid) { struct dprc_attributes attr; int error; @@ -564,11 +615,8 @@ static int fsl_mc_device_get_mmio_regions(struct fsl_mc_device *mc_dev, regions[i].end = regions[i].start + region_desc.size - 1; regions[i].name = "fsl-mc object MMIO region"; - regions[i].flags = IORESOURCE_IO; - if (region_desc.flags & DPRC_REGION_CACHEABLE) - regions[i].flags |= IORESOURCE_CACHEABLE; - if (region_desc.flags & DPRC_REGION_SHAREABLE) - regions[i].flags |= IORESOURCE_MEM; + regions[i].flags = region_desc.flags & IORESOURCE_BITS; + regions[i].flags |= IORESOURCE_MEM; } mc_dev->regions = regions; @@ -630,6 +678,7 @@ int fsl_mc_device_add(struct fsl_mc_obj_desc *obj_desc, if (!mc_bus) return -ENOMEM; + mutex_init(&mc_bus->scan_mutex); mc_dev = &mc_bus->mc_dev; } else { /* @@ -748,6 +797,9 @@ EXPORT_SYMBOL_GPL(fsl_mc_device_add); */ void fsl_mc_device_remove(struct fsl_mc_device *mc_dev) { + kfree(mc_dev->driver_override); + mc_dev->driver_override = NULL; + /* * The device-specific remove callback will get invoked by device_del() */ @@ -908,9 +960,6 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) u32 mc_portal_size, mc_stream_id; struct resource *plat_res; - if (!iommu_present(&fsl_mc_bus_type)) - return -EPROBE_DEFER; - mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL); if (!mc) return -ENOMEM; @@ -918,11 +967,11 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) platform_set_drvdata(pdev, mc); plat_res = platform_get_resource(pdev, IORESOURCE_MEM, 1); - mc->fsl_mc_regs = devm_ioremap_resource(&pdev->dev, plat_res); - if (IS_ERR(mc->fsl_mc_regs)) - return PTR_ERR(mc->fsl_mc_regs); + if (plat_res) + mc->fsl_mc_regs = devm_ioremap_resource(&pdev->dev, plat_res); - if (IS_ENABLED(CONFIG_ACPI) && !dev_of_node(&pdev->dev)) { + if (mc->fsl_mc_regs && IS_ENABLED(CONFIG_ACPI) && + !dev_of_node(&pdev->dev)) { mc_stream_id = readl(mc->fsl_mc_regs + FSL_MC_FAPR); /* * HW ORs the PL and BMT bit, places the result in bit 15 of diff --git a/drivers/bus/fsl-mc/fsl-mc-private.h b/drivers/bus/fsl-mc/fsl-mc-private.h index 7a46a12eb747..85ca5fdee581 100644 --- a/drivers/bus/fsl-mc/fsl-mc-private.h +++ b/drivers/bus/fsl-mc/fsl-mc-private.h @@ -80,10 +80,12 @@ int dpmcp_reset(struct fsl_mc_io *mc_io, /* DPRC command versioning */ #define DPRC_CMD_BASE_VERSION 1 #define DPRC_CMD_2ND_VERSION 2 +#define DPRC_CMD_3RD_VERSION 3 #define DPRC_CMD_ID_OFFSET 4 #define DPRC_CMD(id) (((id) << DPRC_CMD_ID_OFFSET) | DPRC_CMD_BASE_VERSION) #define DPRC_CMD_V2(id) (((id) << DPRC_CMD_ID_OFFSET) | DPRC_CMD_2ND_VERSION) +#define DPRC_CMD_V3(id) (((id) << DPRC_CMD_ID_OFFSET) | DPRC_CMD_3RD_VERSION) /* DPRC command IDs */ #define DPRC_CMDID_CLOSE DPRC_CMD(0x800) @@ -91,6 +93,8 @@ int dpmcp_reset(struct fsl_mc_io *mc_io, #define DPRC_CMDID_GET_API_VERSION DPRC_CMD(0xa05) #define DPRC_CMDID_GET_ATTR DPRC_CMD(0x004) +#define DPRC_CMDID_RESET_CONT DPRC_CMD(0x005) +#define DPRC_CMDID_RESET_CONT_V2 DPRC_CMD_V2(0x005) #define DPRC_CMDID_SET_IRQ DPRC_CMD(0x010) #define DPRC_CMDID_SET_IRQ_ENABLE DPRC_CMD(0x012) @@ -103,6 +107,7 @@ int dpmcp_reset(struct fsl_mc_io *mc_io, #define DPRC_CMDID_GET_OBJ DPRC_CMD(0x15A) #define DPRC_CMDID_GET_OBJ_REG DPRC_CMD(0x15E) #define DPRC_CMDID_GET_OBJ_REG_V2 DPRC_CMD_V2(0x15E) +#define DPRC_CMDID_GET_OBJ_REG_V3 DPRC_CMD_V3(0x15E) #define DPRC_CMDID_SET_OBJ_IRQ DPRC_CMD(0x15F) #define DPRC_CMDID_GET_CONNECTION DPRC_CMD(0x16C) @@ -111,6 +116,11 @@ struct dprc_cmd_open { __le32 container_id; }; +struct dprc_cmd_reset_container { + __le32 child_container_id; + __le32 options; +}; + struct dprc_cmd_set_irq { /* cmd word 0 */ __le32 irq_val; @@ -152,8 +162,7 @@ struct dprc_cmd_clear_irq_status { struct dprc_rsp_get_attributes { /* response word 0 */ __le32 container_id; - __le16 icid; - __le16 pad; + __le32 icid; /* response word 1 */ __le32 options; __le32 portal_id; @@ -330,7 +339,7 @@ int dprc_clear_irq_status(struct fsl_mc_io *mc_io, */ struct dprc_attributes { int container_id; - u16 icid; + u32 icid; int portal_id; u64 options; }; @@ -358,12 +367,6 @@ int dprc_set_obj_irq(struct fsl_mc_io *mc_io, int obj_id, u8 irq_index, struct dprc_irq_cfg *irq_cfg); - -/* Region flags */ -/* Cacheable - Indicates that region should be mapped as cacheable */ -#define DPRC_REGION_CACHEABLE 0x00000001 -#define DPRC_REGION_SHAREABLE 0x00000002 - /** * enum dprc_region_type - Region type * @DPRC_REGION_TYPE_MC_PORTAL: MC portal region @@ -518,11 +521,6 @@ struct dpcon_cmd_set_notification { __le64 user_ctx; }; -/** - * Maximum number of total IRQs that can be pre-allocated for an MC bus' - * IRQ pool - */ -#define FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS 256 /** * struct fsl_mc_resource_pool - Pool of MC resources of a given @@ -597,11 +595,6 @@ void fsl_mc_msi_domain_free_irqs(struct device *dev); struct irq_domain *fsl_mc_find_msi_domain(struct device *dev); -int fsl_mc_populate_irq_pool(struct fsl_mc_bus *mc_bus, - unsigned int irq_count); - -void fsl_mc_cleanup_irq_pool(struct fsl_mc_bus *mc_bus); - int __must_check fsl_create_mc_io(struct device *dev, phys_addr_t mc_portal_phys_addr, u32 mc_portal_size, diff --git a/drivers/bus/fsl-mc/mc-io.c b/drivers/bus/fsl-mc/mc-io.c index a30b53f1d87d..305015486b91 100644 --- a/drivers/bus/fsl-mc/mc-io.c +++ b/drivers/bus/fsl-mc/mc-io.c @@ -129,7 +129,12 @@ error_destroy_mc_io: */ void fsl_destroy_mc_io(struct fsl_mc_io *mc_io) { - struct fsl_mc_device *dpmcp_dev = mc_io->dpmcp_dev; + struct fsl_mc_device *dpmcp_dev; + + if (!mc_io) + return; + + dpmcp_dev = mc_io->dpmcp_dev; if (dpmcp_dev) fsl_mc_io_unset_dpmcp(mc_io); diff --git a/drivers/bus/mhi/Kconfig b/drivers/bus/mhi/Kconfig index a8bd9bd7db7c..e841c1097fb4 100644 --- a/drivers/bus/mhi/Kconfig +++ b/drivers/bus/mhi/Kconfig @@ -6,9 +6,17 @@ # config MHI_BUS - tristate "Modem Host Interface (MHI) bus" - help - Bus driver for MHI protocol. Modem Host Interface (MHI) is a - communication protocol used by the host processors to control - and communicate with modem devices over a high speed peripheral - bus or shared memory. + tristate "Modem Host Interface (MHI) bus" + help + Bus driver for MHI protocol. Modem Host Interface (MHI) is a + communication protocol used by the host processors to control + and communicate with modem devices over a high speed peripheral + bus or shared memory. + +config MHI_BUS_DEBUG + bool "Debugfs support for the MHI bus" + depends on MHI_BUS && DEBUG_FS + help + Enable debugfs support for use with the MHI transport. Allows + reading and/or modifying some values within the MHI controller + for debug and test purposes. diff --git a/drivers/bus/mhi/core/Makefile b/drivers/bus/mhi/core/Makefile index 66e2700c9032..c3feb4130aa3 100644 --- a/drivers/bus/mhi/core/Makefile +++ b/drivers/bus/mhi/core/Makefile @@ -1,3 +1,4 @@ -obj-$(CONFIG_MHI_BUS) := mhi.o +obj-$(CONFIG_MHI_BUS) += mhi.o mhi-y := init.o main.o pm.o boot.o +mhi-$(CONFIG_MHI_BUS_DEBUG) += debugfs.o diff --git a/drivers/bus/mhi/core/boot.c b/drivers/bus/mhi/core/boot.c index 0b38014d040e..24422f5c3d80 100644 --- a/drivers/bus/mhi/core/boot.c +++ b/drivers/bus/mhi/core/boot.c @@ -392,13 +392,28 @@ void mhi_fw_load_handler(struct mhi_controller *mhi_cntrl) void *buf; dma_addr_t dma_addr; size_t size; - int ret; + int i, ret; if (MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state)) { dev_err(dev, "Device MHI is not in valid state\n"); return; } + /* save hardware info from BHI */ + ret = mhi_read_reg(mhi_cntrl, mhi_cntrl->bhi, BHI_SERIALNU, + &mhi_cntrl->serial_number); + if (ret) + dev_err(dev, "Could not capture serial number via BHI\n"); + + for (i = 0; i < ARRAY_SIZE(mhi_cntrl->oem_pk_hash); i++) { + ret = mhi_read_reg(mhi_cntrl, mhi_cntrl->bhi, BHI_OEMPKHASH(i), + &mhi_cntrl->oem_pk_hash[i]); + if (ret) { + dev_err(dev, "Could not capture OEM PK HASH via BHI\n"); + break; + } + } + /* If device is in pass through, do reset to ready state transition */ if (mhi_cntrl->ee == MHI_EE_PTHRU) goto fw_load_ee_pthru; diff --git a/drivers/bus/mhi/core/debugfs.c b/drivers/bus/mhi/core/debugfs.c new file mode 100644 index 000000000000..3a48801e01f4 --- /dev/null +++ b/drivers/bus/mhi/core/debugfs.c @@ -0,0 +1,411 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + * + */ + +#include <linux/debugfs.h> +#include <linux/device.h> +#include <linux/interrupt.h> +#include <linux/list.h> +#include <linux/mhi.h> +#include <linux/module.h> +#include "internal.h" + +static int mhi_debugfs_states_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + + /* states */ + seq_printf(m, "PM state: %s Device: %s MHI state: %s EE: %s wake: %s\n", + to_mhi_pm_state_str(mhi_cntrl->pm_state), + mhi_is_active(mhi_cntrl) ? "Active" : "Inactive", + TO_MHI_STATE_STR(mhi_cntrl->dev_state), + TO_MHI_EXEC_STR(mhi_cntrl->ee), + mhi_cntrl->wake_set ? "true" : "false"); + + /* counters */ + seq_printf(m, "M0: %u M2: %u M3: %u", mhi_cntrl->M0, mhi_cntrl->M2, + mhi_cntrl->M3); + + seq_printf(m, " device wake: %u pending packets: %u\n", + atomic_read(&mhi_cntrl->dev_wake), + atomic_read(&mhi_cntrl->pending_pkts)); + + return 0; +} + +static int mhi_debugfs_events_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + struct mhi_event *mhi_event; + struct mhi_event_ctxt *er_ctxt; + int i; + + if (!mhi_is_active(mhi_cntrl)) { + seq_puts(m, "Device not ready\n"); + return -ENODEV; + } + + er_ctxt = mhi_cntrl->mhi_ctxt->er_ctxt; + mhi_event = mhi_cntrl->mhi_event; + for (i = 0; i < mhi_cntrl->total_ev_rings; + i++, er_ctxt++, mhi_event++) { + struct mhi_ring *ring = &mhi_event->ring; + + if (mhi_event->offload_ev) { + seq_printf(m, "Index: %d is an offload event ring\n", + i); + continue; + } + + seq_printf(m, "Index: %d intmod count: %lu time: %lu", + i, (er_ctxt->intmod & EV_CTX_INTMODC_MASK) >> + EV_CTX_INTMODC_SHIFT, + (er_ctxt->intmod & EV_CTX_INTMODT_MASK) >> + EV_CTX_INTMODT_SHIFT); + + seq_printf(m, " base: 0x%0llx len: 0x%llx", er_ctxt->rbase, + er_ctxt->rlen); + + seq_printf(m, " rp: 0x%llx wp: 0x%llx", er_ctxt->rp, + er_ctxt->wp); + + seq_printf(m, " local rp: 0x%pK db: 0x%pad\n", ring->rp, + &mhi_event->db_cfg.db_val); + } + + return 0; +} + +static int mhi_debugfs_channels_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + struct mhi_chan *mhi_chan; + struct mhi_chan_ctxt *chan_ctxt; + int i; + + if (!mhi_is_active(mhi_cntrl)) { + seq_puts(m, "Device not ready\n"); + return -ENODEV; + } + + mhi_chan = mhi_cntrl->mhi_chan; + chan_ctxt = mhi_cntrl->mhi_ctxt->chan_ctxt; + for (i = 0; i < mhi_cntrl->max_chan; i++, chan_ctxt++, mhi_chan++) { + struct mhi_ring *ring = &mhi_chan->tre_ring; + + if (mhi_chan->offload_ch) { + seq_printf(m, "%s(%u) is an offload channel\n", + mhi_chan->name, mhi_chan->chan); + continue; + } + + if (!mhi_chan->mhi_dev) + continue; + + seq_printf(m, + "%s(%u) state: 0x%lx brstmode: 0x%lx pollcfg: 0x%lx", + mhi_chan->name, mhi_chan->chan, (chan_ctxt->chcfg & + CHAN_CTX_CHSTATE_MASK) >> CHAN_CTX_CHSTATE_SHIFT, + (chan_ctxt->chcfg & CHAN_CTX_BRSTMODE_MASK) >> + CHAN_CTX_BRSTMODE_SHIFT, (chan_ctxt->chcfg & + CHAN_CTX_POLLCFG_MASK) >> CHAN_CTX_POLLCFG_SHIFT); + + seq_printf(m, " type: 0x%x event ring: %u", chan_ctxt->chtype, + chan_ctxt->erindex); + + seq_printf(m, " base: 0x%llx len: 0x%llx rp: 0x%llx wp: 0x%llx", + chan_ctxt->rbase, chan_ctxt->rlen, chan_ctxt->rp, + chan_ctxt->wp); + + seq_printf(m, " local rp: 0x%pK local wp: 0x%pK db: 0x%pad\n", + ring->rp, ring->wp, + &mhi_chan->db_cfg.db_val); + } + + return 0; +} + +static int mhi_device_info_show(struct device *dev, void *data) +{ + struct mhi_device *mhi_dev; + + if (dev->bus != &mhi_bus_type) + return 0; + + mhi_dev = to_mhi_device(dev); + + seq_printf((struct seq_file *)data, "%s: type: %s dev_wake: %u", + mhi_dev->name, mhi_dev->dev_type ? "Controller" : "Transfer", + mhi_dev->dev_wake); + + /* for transfer device types only */ + if (mhi_dev->dev_type == MHI_DEVICE_XFER) + seq_printf((struct seq_file *)data, " channels: %u(UL)/%u(DL)", + mhi_dev->ul_chan_id, mhi_dev->dl_chan_id); + + seq_puts((struct seq_file *)data, "\n"); + + return 0; +} + +static int mhi_debugfs_devices_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + + if (!mhi_is_active(mhi_cntrl)) { + seq_puts(m, "Device not ready\n"); + return -ENODEV; + } + + device_for_each_child(mhi_cntrl->cntrl_dev, m, mhi_device_info_show); + + return 0; +} + +static int mhi_debugfs_regdump_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + enum mhi_state state; + enum mhi_ee_type ee; + int i, ret = -EIO; + u32 val; + void __iomem *mhi_base = mhi_cntrl->regs; + void __iomem *bhi_base = mhi_cntrl->bhi; + void __iomem *bhie_base = mhi_cntrl->bhie; + void __iomem *wake_db = mhi_cntrl->wake_db; + struct { + const char *name; + int offset; + void __iomem *base; + } regs[] = { + { "MHI_REGLEN", MHIREGLEN, mhi_base}, + { "MHI_VER", MHIVER, mhi_base}, + { "MHI_CFG", MHICFG, mhi_base}, + { "MHI_CTRL", MHICTRL, mhi_base}, + { "MHI_STATUS", MHISTATUS, mhi_base}, + { "MHI_WAKE_DB", 0, wake_db}, + { "BHI_EXECENV", BHI_EXECENV, bhi_base}, + { "BHI_STATUS", BHI_STATUS, bhi_base}, + { "BHI_ERRCODE", BHI_ERRCODE, bhi_base}, + { "BHI_ERRDBG1", BHI_ERRDBG1, bhi_base}, + { "BHI_ERRDBG2", BHI_ERRDBG2, bhi_base}, + { "BHI_ERRDBG3", BHI_ERRDBG3, bhi_base}, + { "BHIE_TXVEC_DB", BHIE_TXVECDB_OFFS, bhie_base}, + { "BHIE_TXVEC_STATUS", BHIE_TXVECSTATUS_OFFS, bhie_base}, + { "BHIE_RXVEC_DB", BHIE_RXVECDB_OFFS, bhie_base}, + { "BHIE_RXVEC_STATUS", BHIE_RXVECSTATUS_OFFS, bhie_base}, + { NULL }, + }; + + if (!MHI_REG_ACCESS_VALID(mhi_cntrl->pm_state)) + return ret; + + seq_printf(m, "Host PM state: %s Device state: %s EE: %s\n", + to_mhi_pm_state_str(mhi_cntrl->pm_state), + TO_MHI_STATE_STR(mhi_cntrl->dev_state), + TO_MHI_EXEC_STR(mhi_cntrl->ee)); + + state = mhi_get_mhi_state(mhi_cntrl); + ee = mhi_get_exec_env(mhi_cntrl); + seq_printf(m, "Device EE: %s state: %s\n", TO_MHI_EXEC_STR(ee), + TO_MHI_STATE_STR(state)); + + for (i = 0; regs[i].name; i++) { + if (!regs[i].base) + continue; + ret = mhi_read_reg(mhi_cntrl, regs[i].base, regs[i].offset, + &val); + if (ret) + continue; + + seq_printf(m, "%s: 0x%x\n", regs[i].name, val); + } + + return 0; +} + +static int mhi_debugfs_device_wake_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + struct mhi_device *mhi_dev = mhi_cntrl->mhi_dev; + + if (!mhi_is_active(mhi_cntrl)) { + seq_puts(m, "Device not ready\n"); + return -ENODEV; + } + + seq_printf(m, + "Wake count: %d\n%s\n", mhi_dev->dev_wake, + "Usage: echo get/put > device_wake to vote/unvote for M0"); + + return 0; +} + +static ssize_t mhi_debugfs_device_wake_write(struct file *file, + const char __user *ubuf, + size_t count, loff_t *ppos) +{ + struct seq_file *m = file->private_data; + struct mhi_controller *mhi_cntrl = m->private; + struct mhi_device *mhi_dev = mhi_cntrl->mhi_dev; + char buf[16]; + int ret = -EINVAL; + + if (copy_from_user(&buf, ubuf, min_t(size_t, sizeof(buf) - 1, count))) + return -EFAULT; + + if (!strncmp(buf, "get", 3)) { + ret = mhi_device_get_sync(mhi_dev); + } else if (!strncmp(buf, "put", 3)) { + mhi_device_put(mhi_dev); + ret = 0; + } + + return ret ? ret : count; +} + +static int mhi_debugfs_timeout_ms_show(struct seq_file *m, void *d) +{ + struct mhi_controller *mhi_cntrl = m->private; + + seq_printf(m, "%u ms\n", mhi_cntrl->timeout_ms); + + return 0; +} + +static ssize_t mhi_debugfs_timeout_ms_write(struct file *file, + const char __user *ubuf, + size_t count, loff_t *ppos) +{ + struct seq_file *m = file->private_data; + struct mhi_controller *mhi_cntrl = m->private; + u32 timeout_ms; + + if (kstrtou32_from_user(ubuf, count, 0, &timeout_ms)) + return -EINVAL; + + mhi_cntrl->timeout_ms = timeout_ms; + + return count; +} + +static int mhi_debugfs_states_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_states_show, inode->i_private); +} + +static int mhi_debugfs_events_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_events_show, inode->i_private); +} + +static int mhi_debugfs_channels_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_channels_show, inode->i_private); +} + +static int mhi_debugfs_devices_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_devices_show, inode->i_private); +} + +static int mhi_debugfs_regdump_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_regdump_show, inode->i_private); +} + +static int mhi_debugfs_device_wake_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_device_wake_show, inode->i_private); +} + +static int mhi_debugfs_timeout_ms_open(struct inode *inode, struct file *fp) +{ + return single_open(fp, mhi_debugfs_timeout_ms_show, inode->i_private); +} + +static const struct file_operations debugfs_states_fops = { + .open = mhi_debugfs_states_open, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_events_fops = { + .open = mhi_debugfs_events_open, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_channels_fops = { + .open = mhi_debugfs_channels_open, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_devices_fops = { + .open = mhi_debugfs_devices_open, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_regdump_fops = { + .open = mhi_debugfs_regdump_open, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_device_wake_fops = { + .open = mhi_debugfs_device_wake_open, + .write = mhi_debugfs_device_wake_write, + .release = single_release, + .read = seq_read, +}; + +static const struct file_operations debugfs_timeout_ms_fops = { + .open = mhi_debugfs_timeout_ms_open, + .write = mhi_debugfs_timeout_ms_write, + .release = single_release, + .read = seq_read, +}; + +static struct dentry *mhi_debugfs_root; + +void mhi_create_debugfs(struct mhi_controller *mhi_cntrl) +{ + mhi_cntrl->debugfs_dentry = + debugfs_create_dir(dev_name(mhi_cntrl->cntrl_dev), + mhi_debugfs_root); + + debugfs_create_file("states", 0444, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_states_fops); + debugfs_create_file("events", 0444, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_events_fops); + debugfs_create_file("channels", 0444, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_channels_fops); + debugfs_create_file("devices", 0444, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_devices_fops); + debugfs_create_file("regdump", 0444, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_regdump_fops); + debugfs_create_file("device_wake", 0644, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_device_wake_fops); + debugfs_create_file("timeout_ms", 0644, mhi_cntrl->debugfs_dentry, + mhi_cntrl, &debugfs_timeout_ms_fops); +} + +void mhi_destroy_debugfs(struct mhi_controller *mhi_cntrl) +{ + debugfs_remove_recursive(mhi_cntrl->debugfs_dentry); + mhi_cntrl->debugfs_dentry = NULL; +} + +void mhi_debugfs_init(void) +{ + mhi_debugfs_root = debugfs_create_dir(mhi_bus_type.name, NULL); +} + +void mhi_debugfs_exit(void) +{ + debugfs_remove_recursive(mhi_debugfs_root); +} diff --git a/drivers/bus/mhi/core/init.c b/drivers/bus/mhi/core/init.c index e43a190a7a36..0ffdebde8265 100644 --- a/drivers/bus/mhi/core/init.c +++ b/drivers/bus/mhi/core/init.c @@ -4,6 +4,7 @@ * */ +#include <linux/debugfs.h> #include <linux/device.h> #include <linux/dma-direction.h> #include <linux/dma-mapping.h> @@ -75,6 +76,42 @@ const char *to_mhi_pm_state_str(enum mhi_pm_state state) return mhi_pm_state_str[index]; } +static ssize_t serial_number_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct mhi_device *mhi_dev = to_mhi_device(dev); + struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl; + + return snprintf(buf, PAGE_SIZE, "Serial Number: %u\n", + mhi_cntrl->serial_number); +} +static DEVICE_ATTR_RO(serial_number); + +static ssize_t oem_pk_hash_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct mhi_device *mhi_dev = to_mhi_device(dev); + struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl; + int i, cnt = 0; + + for (i = 0; i < ARRAY_SIZE(mhi_cntrl->oem_pk_hash); i++) + cnt += snprintf(buf + cnt, PAGE_SIZE - cnt, + "OEMPKHASH[%d]: 0x%x\n", i, + mhi_cntrl->oem_pk_hash[i]); + + return cnt; +} +static DEVICE_ATTR_RO(oem_pk_hash); + +static struct attribute *mhi_dev_attrs[] = { + &dev_attr_serial_number.attr, + &dev_attr_oem_pk_hash.attr, + NULL, +}; +ATTRIBUTE_GROUPS(mhi_dev); + /* MHI protocol requires the transfer ring to be aligned with ring length */ static int mhi_alloc_aligned_ring(struct mhi_controller *mhi_cntrl, struct mhi_ring *ring, @@ -125,6 +162,13 @@ int mhi_init_irq_setup(struct mhi_controller *mhi_cntrl) if (mhi_event->offload_ev) continue; + if (mhi_event->irq >= mhi_cntrl->nr_irqs) { + dev_err(dev, "irq %d not available for event ring\n", + mhi_event->irq); + ret = -EINVAL; + goto error_request; + } + ret = request_irq(mhi_cntrl->irq[mhi_event->irq], mhi_irq_handler, IRQF_SHARED | IRQF_NO_SUSPEND, @@ -562,10 +606,10 @@ int mhi_init_chan_ctxt(struct mhi_controller *mhi_cntrl, } static int parse_ev_cfg(struct mhi_controller *mhi_cntrl, - struct mhi_controller_config *config) + const struct mhi_controller_config *config) { struct mhi_event *mhi_event; - struct mhi_event_config *event_cfg; + const struct mhi_event_config *event_cfg; struct device *dev = &mhi_cntrl->mhi_dev->dev; int i, num; @@ -636,9 +680,6 @@ static int parse_ev_cfg(struct mhi_controller *mhi_cntrl, mhi_event++; } - /* We need IRQ for each event ring + additional one for BHI */ - mhi_cntrl->nr_irqs_req = mhi_cntrl->total_ev_rings + 1; - return 0; error_ev_cfg: @@ -648,9 +689,9 @@ error_ev_cfg: } static int parse_ch_cfg(struct mhi_controller *mhi_cntrl, - struct mhi_controller_config *config) + const struct mhi_controller_config *config) { - struct mhi_channel_config *ch_cfg; + const struct mhi_channel_config *ch_cfg; struct device *dev = &mhi_cntrl->mhi_dev->dev; int i; u32 chan; @@ -766,7 +807,7 @@ error_chan_cfg: } static int parse_config(struct mhi_controller *mhi_cntrl, - struct mhi_controller_config *config) + const struct mhi_controller_config *config) { int ret; @@ -803,7 +844,7 @@ error_ev_cfg: } int mhi_register_controller(struct mhi_controller *mhi_cntrl, - struct mhi_controller_config *config) + const struct mhi_controller_config *config) { struct mhi_event *mhi_event; struct mhi_chan *mhi_chan; @@ -904,6 +945,7 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl, mhi_dev->dev_type = MHI_DEVICE_CONTROLLER; mhi_dev->mhi_cntrl = mhi_cntrl; dev_set_name(&mhi_dev->dev, "%s", dev_name(mhi_cntrl->cntrl_dev)); + mhi_dev->name = dev_name(mhi_cntrl->cntrl_dev); /* Init wakeup source */ device_init_wakeup(&mhi_dev->dev, true); @@ -914,6 +956,8 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl, mhi_cntrl->mhi_dev = mhi_dev; + mhi_create_debugfs(mhi_cntrl); + return 0; error_add_dev: @@ -936,6 +980,8 @@ void mhi_unregister_controller(struct mhi_controller *mhi_cntrl) struct mhi_chan *mhi_chan = mhi_cntrl->mhi_chan; unsigned int i; + mhi_destroy_debugfs(mhi_cntrl); + kfree(mhi_cntrl->mhi_cmd); kfree(mhi_cntrl->mhi_event); @@ -953,6 +999,22 @@ void mhi_unregister_controller(struct mhi_controller *mhi_cntrl) } EXPORT_SYMBOL_GPL(mhi_unregister_controller); +struct mhi_controller *mhi_alloc_controller(void) +{ + struct mhi_controller *mhi_cntrl; + + mhi_cntrl = kzalloc(sizeof(*mhi_cntrl), GFP_KERNEL); + + return mhi_cntrl; +} +EXPORT_SYMBOL_GPL(mhi_alloc_controller); + +void mhi_free_controller(struct mhi_controller *mhi_cntrl) +{ + kfree(mhi_cntrl); +} +EXPORT_SYMBOL_GPL(mhi_free_controller); + int mhi_prepare_for_power_up(struct mhi_controller *mhi_cntrl) { struct device *dev = &mhi_cntrl->mhi_dev->dev; @@ -1249,7 +1311,7 @@ static int mhi_uevent(struct device *dev, struct kobj_uevent_env *env) struct mhi_device *mhi_dev = to_mhi_device(dev); return add_uevent_var(env, "MODALIAS=" MHI_DEVICE_MODALIAS_FMT, - mhi_dev->chan_name); + mhi_dev->name); } static int mhi_match(struct device *dev, struct device_driver *drv) @@ -1266,7 +1328,7 @@ static int mhi_match(struct device *dev, struct device_driver *drv) return 0; for (id = mhi_drv->id_table; id->chan[0]; id++) - if (!strcmp(mhi_dev->chan_name, id->chan)) { + if (!strcmp(mhi_dev->name, id->chan)) { mhi_dev->id = id; return 1; } @@ -1279,15 +1341,18 @@ struct bus_type mhi_bus_type = { .dev_name = "mhi", .match = mhi_match, .uevent = mhi_uevent, + .dev_groups = mhi_dev_groups, }; static int __init mhi_init(void) { + mhi_debugfs_init(); return bus_register(&mhi_bus_type); } static void __exit mhi_exit(void) { + mhi_debugfs_exit(); bus_unregister(&mhi_bus_type); } diff --git a/drivers/bus/mhi/core/internal.h b/drivers/bus/mhi/core/internal.h index b1f640b75a94..7989269ddd96 100644 --- a/drivers/bus/mhi/core/internal.h +++ b/drivers/bus/mhi/core/internal.h @@ -570,6 +570,30 @@ struct mhi_chan { /* Default MHI timeout */ #define MHI_TIMEOUT_MS (1000) +/* debugfs related functions */ +#ifdef CONFIG_MHI_BUS_DEBUG +void mhi_create_debugfs(struct mhi_controller *mhi_cntrl); +void mhi_destroy_debugfs(struct mhi_controller *mhi_cntrl); +void mhi_debugfs_init(void); +void mhi_debugfs_exit(void); +#else +static inline void mhi_create_debugfs(struct mhi_controller *mhi_cntrl) +{ +} + +static inline void mhi_destroy_debugfs(struct mhi_controller *mhi_cntrl) +{ +} + +static inline void mhi_debugfs_init(void) +{ +} + +static inline void mhi_debugfs_exit(void) +{ +} +#endif + struct mhi_device *mhi_alloc_device(struct mhi_controller *mhi_cntrl); int mhi_destroy_device(struct device *dev, void *data); @@ -592,13 +616,24 @@ void mhi_pm_st_worker(struct work_struct *work); void mhi_pm_sys_err_handler(struct mhi_controller *mhi_cntrl); void mhi_fw_load_worker(struct work_struct *work); int mhi_ready_state_transition(struct mhi_controller *mhi_cntrl); -void mhi_ctrl_ev_task(unsigned long data); int mhi_pm_m0_transition(struct mhi_controller *mhi_cntrl); void mhi_pm_m1_transition(struct mhi_controller *mhi_cntrl); int mhi_pm_m3_transition(struct mhi_controller *mhi_cntrl); int __mhi_device_get_sync(struct mhi_controller *mhi_cntrl); int mhi_send_cmd(struct mhi_controller *mhi_cntrl, struct mhi_chan *mhi_chan, enum mhi_cmd_type cmd); +static inline bool mhi_is_active(struct mhi_controller *mhi_cntrl) +{ + return (mhi_cntrl->dev_state >= MHI_STATE_M0 && + mhi_cntrl->dev_state <= MHI_STATE_M3_FAST); +} + +static inline void mhi_trigger_resume(struct mhi_controller *mhi_cntrl) +{ + pm_wakeup_event(&mhi_cntrl->mhi_dev->dev, 0); + mhi_cntrl->runtime_get(mhi_cntrl); + mhi_cntrl->runtime_put(mhi_cntrl); +} /* Register access methods */ void mhi_db_brstmode(struct mhi_controller *mhi_cntrl, struct db_cfg *db_cfg, diff --git a/drivers/bus/mhi/core/main.c b/drivers/bus/mhi/core/main.c index 1f622ce6be8b..2cff5ddff225 100644 --- a/drivers/bus/mhi/core/main.c +++ b/drivers/bus/mhi/core/main.c @@ -249,7 +249,7 @@ int mhi_destroy_device(struct device *dev, void *data) put_device(&mhi_dev->dl_chan->mhi_dev->dev); dev_dbg(&mhi_cntrl->mhi_dev->dev, "destroy device for chan:%s\n", - mhi_dev->chan_name); + mhi_dev->name); /* Notify the client and remove the device from MHI bus */ device_del(dev); @@ -327,10 +327,10 @@ void mhi_create_devices(struct mhi_controller *mhi_cntrl) } /* Channel name is same for both UL and DL */ - mhi_dev->chan_name = mhi_chan->name; + mhi_dev->name = mhi_chan->name; dev_set_name(&mhi_dev->dev, "%s_%s", dev_name(mhi_cntrl->cntrl_dev), - mhi_dev->chan_name); + mhi_dev->name); /* Init wakeup source if available */ if (mhi_dev->dl_chan && mhi_dev->dl_chan->wake_capable) @@ -909,8 +909,7 @@ void mhi_ctrl_ev_task(unsigned long data) * process it since we are probably in a suspended state, * so trigger a resume. */ - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); + mhi_trigger_resume(mhi_cntrl); return; } @@ -971,10 +970,8 @@ int mhi_queue_skb(struct mhi_device *mhi_dev, enum dma_data_direction dir, } /* we're in M3 or transitioning to M3 */ - if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) { - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); - } + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); /* Toggle wake to exit out of M2 */ mhi_cntrl->wake_toggle(mhi_cntrl); @@ -1032,10 +1029,8 @@ int mhi_queue_dma(struct mhi_device *mhi_dev, enum dma_data_direction dir, } /* we're in M3 or transitioning to M3 */ - if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) { - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); - } + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); /* Toggle wake to exit out of M2 */ mhi_cntrl->wake_toggle(mhi_cntrl); @@ -1147,10 +1142,8 @@ int mhi_queue_buf(struct mhi_device *mhi_dev, enum dma_data_direction dir, read_lock_irqsave(&mhi_cntrl->pm_lock, flags); /* we're in M3 or transitioning to M3 */ - if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) { - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); - } + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); /* Toggle wake to exit out of M2 */ mhi_cntrl->wake_toggle(mhi_cntrl); diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c index 796098078083..3de7b1639ec6 100644 --- a/drivers/bus/mhi/core/pm.c +++ b/drivers/bus/mhi/core/pm.c @@ -256,6 +256,7 @@ int mhi_pm_m0_transition(struct mhi_controller *mhi_cntrl) dev_err(dev, "Unable to transition to M0 state\n"); return -EIO; } + mhi_cntrl->M0++; /* Wake up the device */ read_lock_bh(&mhi_cntrl->pm_lock); @@ -326,6 +327,8 @@ void mhi_pm_m1_transition(struct mhi_controller *mhi_cntrl) mhi_cntrl->dev_state = MHI_STATE_M2; write_unlock_irq(&mhi_cntrl->pm_lock); + + mhi_cntrl->M2++; wake_up_all(&mhi_cntrl->state_event); /* If there are any pending resources, exit M2 immediately */ @@ -362,6 +365,7 @@ int mhi_pm_m3_transition(struct mhi_controller *mhi_cntrl) return -EIO; } + mhi_cntrl->M3++; wake_up_all(&mhi_cntrl->state_event); return 0; @@ -686,7 +690,8 @@ int mhi_pm_suspend(struct mhi_controller *mhi_cntrl) return -EIO; /* Return busy if there are any pending resources */ - if (atomic_read(&mhi_cntrl->dev_wake)) + if (atomic_read(&mhi_cntrl->dev_wake) || + atomic_read(&mhi_cntrl->pending_pkts)) return -EBUSY; /* Take MHI out of M2 state */ @@ -712,7 +717,8 @@ int mhi_pm_suspend(struct mhi_controller *mhi_cntrl) write_lock_irq(&mhi_cntrl->pm_lock); - if (atomic_read(&mhi_cntrl->dev_wake)) { + if (atomic_read(&mhi_cntrl->dev_wake) || + atomic_read(&mhi_cntrl->pending_pkts)) { write_unlock_irq(&mhi_cntrl->pm_lock); return -EBUSY; } @@ -822,11 +828,8 @@ int __mhi_device_get_sync(struct mhi_controller *mhi_cntrl) /* Wake up the device */ read_lock_bh(&mhi_cntrl->pm_lock); mhi_cntrl->wake_get(mhi_cntrl, true); - if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) { - pm_wakeup_event(&mhi_cntrl->mhi_dev->dev, 0); - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); - } + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); read_unlock_bh(&mhi_cntrl->pm_lock); ret = wait_event_timeout(mhi_cntrl->state_event, @@ -915,7 +918,7 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl) dev_info(dev, "Requested to power ON\n"); - if (mhi_cntrl->nr_irqs < mhi_cntrl->total_ev_rings) + if (mhi_cntrl->nr_irqs < 1) return -EINVAL; /* Supply default wake routines if not provided by controller driver */ @@ -1113,6 +1116,9 @@ void mhi_device_get(struct mhi_device *mhi_dev) mhi_dev->dev_wake++; read_lock_bh(&mhi_cntrl->pm_lock); + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); + mhi_cntrl->wake_get(mhi_cntrl, true); read_unlock_bh(&mhi_cntrl->pm_lock); } @@ -1137,10 +1143,8 @@ void mhi_device_put(struct mhi_device *mhi_dev) mhi_dev->dev_wake--; read_lock_bh(&mhi_cntrl->pm_lock); - if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) { - mhi_cntrl->runtime_get(mhi_cntrl); - mhi_cntrl->runtime_put(mhi_cntrl); - } + if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state)) + mhi_trigger_resume(mhi_cntrl); mhi_cntrl->wake_put(mhi_cntrl, false); read_unlock_bh(&mhi_cntrl->pm_lock); diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig index b1bd336761b1..d229a2d0c017 100644 --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -93,8 +93,9 @@ config PPDEV config VIRTIO_CONSOLE tristate "Virtio console" - depends on VIRTIO && TTY + depends on TTY select HVC_DRIVER + select VIRTIO help Virtio console for use with hypervisors. diff --git a/drivers/char/lp.c b/drivers/char/lp.c index 45932f05fd67..0ec73917d8dd 100644 --- a/drivers/char/lp.c +++ b/drivers/char/lp.c @@ -853,8 +853,10 @@ static void lp_console_write(struct console *co, const char *s, count--; do { written = parport_write(port, crlf, i); - if (written > 0) - i -= written, crlf += written; + if (written > 0) { + i -= written; + crlf += written; + } } while (i > 0 && (CONSOLE_LP_STRICT || written > 0)); } } while (count > 0 && (CONSOLE_LP_STRICT || written > 0)); diff --git a/drivers/char/mem.c b/drivers/char/mem.c index abd4ffdc8cde..94c2b556cf97 100644 --- a/drivers/char/mem.c +++ b/drivers/char/mem.c @@ -726,6 +726,33 @@ static ssize_t read_iter_zero(struct kiocb *iocb, struct iov_iter *iter) return written; } +static ssize_t read_zero(struct file *file, char __user *buf, + size_t count, loff_t *ppos) +{ + size_t cleared = 0; + + while (count) { + size_t chunk = min_t(size_t, count, PAGE_SIZE); + size_t left; + + left = clear_user(buf + cleared, chunk); + if (unlikely(left)) { + cleared += (chunk - left); + if (!cleared) + return -EFAULT; + break; + } + cleared += chunk; + count -= chunk; + + if (signal_pending(current)) + break; + cond_resched(); + } + + return cleared; +} + static int mmap_zero(struct file *file, struct vm_area_struct *vma) { #ifndef CONFIG_MMU @@ -921,6 +948,7 @@ static const struct file_operations zero_fops = { .llseek = zero_lseek, .write = write_zero, .read_iter = read_iter_zero, + .read = read_zero, .write_iter = write_iter_zero, .mmap = mmap_zero, .get_unmapped_area = get_unmapped_area_zero, diff --git a/drivers/char/mspec.c b/drivers/char/mspec.c index 0fae33319d2e..f8231e2e84be 100644 --- a/drivers/char/mspec.c +++ b/drivers/char/mspec.c @@ -195,10 +195,7 @@ mspec_mmap(struct file *file, struct vm_area_struct *vma, pages = vma_pages(vma); vdata_size = sizeof(struct vma_data) + pages * sizeof(long); - if (vdata_size <= PAGE_SIZE) - vdata = kzalloc(vdata_size, GFP_KERNEL); - else - vdata = vzalloc(vdata_size); + vdata = kvzalloc(vdata_size, GFP_KERNEL); if (!vdata) return -ENOMEM; diff --git a/drivers/extcon/extcon-axp288.c b/drivers/extcon/extcon-axp288.c index 525345367260..fdb31954cf2b 100644 --- a/drivers/extcon/extcon-axp288.c +++ b/drivers/extcon/extcon-axp288.c @@ -491,18 +491,7 @@ static struct platform_driver axp288_extcon_driver = { .pm = &axp288_extcon_pm_ops, }, }; - -static int __init axp288_extcon_init(void) -{ - return platform_driver_register(&axp288_extcon_driver); -} -module_init(axp288_extcon_init); - -static void __exit axp288_extcon_exit(void) -{ - platform_driver_unregister(&axp288_extcon_driver); -} -module_exit(axp288_extcon_exit); +module_platform_driver(axp288_extcon_driver); MODULE_AUTHOR("Ramakrishna Pallala <ramakrishna.pallala@intel.com>"); MODULE_AUTHOR("Hans de Goede <hdegoede@redhat.com>"); diff --git a/drivers/extcon/extcon-max14577.c b/drivers/extcon/extcon-max14577.c index cc47d626095c..ace523924e58 100644 --- a/drivers/extcon/extcon-max14577.c +++ b/drivers/extcon/extcon-max14577.c @@ -713,7 +713,7 @@ static int max14577_muic_probe(struct platform_device *pdev) max14577_extcon_cable); if (IS_ERR(info->edev)) { dev_err(&pdev->dev, "failed to allocate memory for extcon\n"); - return -ENOMEM; + return PTR_ERR(info->edev); } ret = devm_extcon_dev_register(&pdev->dev, info->edev); diff --git a/drivers/extcon/extcon-max77693.c b/drivers/extcon/extcon-max77693.c index 32fc5a66ffa9..4a410fd2ea9a 100644 --- a/drivers/extcon/extcon-max77693.c +++ b/drivers/extcon/extcon-max77693.c @@ -1157,7 +1157,7 @@ static int max77693_muic_probe(struct platform_device *pdev) max77693_extcon_cable); if (IS_ERR(info->edev)) { dev_err(&pdev->dev, "failed to allocate memory for extcon\n"); - return -ENOMEM; + return PTR_ERR(info->edev); } ret = devm_extcon_dev_register(&pdev->dev, info->edev); diff --git a/drivers/extcon/extcon-max77843.c b/drivers/extcon/extcon-max77843.c index e6b50ca83008..8e6e97ec65a8 100644 --- a/drivers/extcon/extcon-max77843.c +++ b/drivers/extcon/extcon-max77843.c @@ -845,7 +845,7 @@ static int max77843_muic_probe(struct platform_device *pdev) max77843_extcon_cable); if (IS_ERR(info->edev)) { dev_err(&pdev->dev, "Failed to allocate memory for extcon\n"); - ret = -ENODEV; + ret = PTR_ERR(info->edev); goto err_muic_irq; } diff --git a/drivers/extcon/extcon-max8997.c b/drivers/extcon/extcon-max8997.c index 172e116ac1ce..337b0eea4e62 100644 --- a/drivers/extcon/extcon-max8997.c +++ b/drivers/extcon/extcon-max8997.c @@ -674,7 +674,7 @@ static int max8997_muic_probe(struct platform_device *pdev) info->edev = devm_extcon_dev_allocate(&pdev->dev, max8997_extcon_cable); if (IS_ERR(info->edev)) { dev_err(&pdev->dev, "failed to allocate memory for extcon\n"); - ret = -ENOMEM; + ret = PTR_ERR(info->edev); goto err_irq; } diff --git a/drivers/extcon/extcon-palmas.c b/drivers/extcon/extcon-palmas.c index cea58d0cb457..a2852bcc5f0d 100644 --- a/drivers/extcon/extcon-palmas.c +++ b/drivers/extcon/extcon-palmas.c @@ -2,7 +2,7 @@ /* * Palmas USB transceiver driver * - * Copyright (C) 2013 Texas Instruments Incorporated - http://www.ti.com + * Copyright (C) 2013 Texas Instruments Incorporated - https://www.ti.com * Author: Graeme Gregory <gg@slimlogic.co.uk> * Author: Kishon Vijay Abraham I <kishon@ti.com> * Based on twl6030_usb.c @@ -205,21 +205,15 @@ static int palmas_usb_probe(struct platform_device *pdev) palmas_usb->id_gpiod = devm_gpiod_get_optional(&pdev->dev, "id", GPIOD_IN); - if (PTR_ERR(palmas_usb->id_gpiod) == -EPROBE_DEFER) { - return -EPROBE_DEFER; - } else if (IS_ERR(palmas_usb->id_gpiod)) { - dev_err(&pdev->dev, "failed to get id gpio\n"); - return PTR_ERR(palmas_usb->id_gpiod); - } + if (IS_ERR(palmas_usb->id_gpiod)) + return dev_err_probe(&pdev->dev, PTR_ERR(palmas_usb->id_gpiod), + "failed to get id gpio\n"); palmas_usb->vbus_gpiod = devm_gpiod_get_optional(&pdev->dev, "vbus", GPIOD_IN); - if (PTR_ERR(palmas_usb->vbus_gpiod) == -EPROBE_DEFER) { - return -EPROBE_DEFER; - } else if (IS_ERR(palmas_usb->vbus_gpiod)) { - dev_err(&pdev->dev, "failed to get vbus gpio\n"); - return PTR_ERR(palmas_usb->vbus_gpiod); - } + if (IS_ERR(palmas_usb->vbus_gpiod)) + return dev_err_probe(&pdev->dev, PTR_ERR(palmas_usb->vbus_gpiod), + "failed to get id gpio\n"); if (palmas_usb->enable_id_detection && palmas_usb->id_gpiod) { palmas_usb->enable_id_detection = false; diff --git a/drivers/extcon/extcon-ptn5150.c b/drivers/extcon/extcon-ptn5150.c index d1c997599390..5b9a3cf8df26 100644 --- a/drivers/extcon/extcon-ptn5150.c +++ b/drivers/extcon/extcon-ptn5150.c @@ -5,7 +5,9 @@ // Based on extcon-sm5502.c driver // Copyright (c) 2018-2019 by Vijai Kumar K // Author: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com> +// Copyright (c) 2020 Krzysztof Kozlowski <krzk@kernel.org> +#include <linux/bitfield.h> #include <linux/err.h> #include <linux/i2c.h> #include <linux/interrupt.h> @@ -17,46 +19,28 @@ #include <linux/gpio/consumer.h> /* PTN5150 registers */ -enum ptn5150_reg { - PTN5150_REG_DEVICE_ID = 0x01, - PTN5150_REG_CONTROL, - PTN5150_REG_INT_STATUS, - PTN5150_REG_CC_STATUS, - PTN5150_REG_CON_DET = 0x09, - PTN5150_REG_VCONN_STATUS, - PTN5150_REG_RESET, - PTN5150_REG_INT_MASK = 0x18, - PTN5150_REG_INT_REG_STATUS, - PTN5150_REG_END, -}; +#define PTN5150_REG_DEVICE_ID 0x01 +#define PTN5150_REG_CONTROL 0x02 +#define PTN5150_REG_INT_STATUS 0x03 +#define PTN5150_REG_CC_STATUS 0x04 +#define PTN5150_REG_CON_DET 0x09 +#define PTN5150_REG_VCONN_STATUS 0x0a +#define PTN5150_REG_RESET 0x0b +#define PTN5150_REG_INT_MASK 0x18 +#define PTN5150_REG_INT_REG_STATUS 0x19 +#define PTN5150_REG_END PTN5150_REG_INT_REG_STATUS #define PTN5150_DFP_ATTACHED 0x1 #define PTN5150_UFP_ATTACHED 0x2 /* Define PTN5150 MASK/SHIFT constant */ -#define PTN5150_REG_DEVICE_ID_VENDOR_SHIFT 0 -#define PTN5150_REG_DEVICE_ID_VENDOR_MASK \ - (0x3 << PTN5150_REG_DEVICE_ID_VENDOR_SHIFT) - -#define PTN5150_REG_DEVICE_ID_VERSION_SHIFT 3 -#define PTN5150_REG_DEVICE_ID_VERSION_MASK \ - (0x1f << PTN5150_REG_DEVICE_ID_VERSION_SHIFT) - -#define PTN5150_REG_CC_PORT_ATTACHMENT_SHIFT 2 -#define PTN5150_REG_CC_PORT_ATTACHMENT_MASK \ - (0x7 << PTN5150_REG_CC_PORT_ATTACHMENT_SHIFT) +#define PTN5150_REG_DEVICE_ID_VERSION GENMASK(7, 3) +#define PTN5150_REG_DEVICE_ID_VENDOR GENMASK(2, 0) -#define PTN5150_REG_CC_VBUS_DETECTION_SHIFT 7 -#define PTN5150_REG_CC_VBUS_DETECTION_MASK \ - (0x1 << PTN5150_REG_CC_VBUS_DETECTION_SHIFT) - -#define PTN5150_REG_INT_CABLE_ATTACH_SHIFT 0 -#define PTN5150_REG_INT_CABLE_ATTACH_MASK \ - (0x1 << PTN5150_REG_INT_CABLE_ATTACH_SHIFT) - -#define PTN5150_REG_INT_CABLE_DETACH_SHIFT 1 -#define PTN5150_REG_INT_CABLE_DETACH_MASK \ - (0x1 << PTN5150_REG_CC_CABLE_DETACH_SHIFT) +#define PTN5150_REG_CC_PORT_ATTACHMENT GENMASK(4, 2) +#define PTN5150_REG_CC_VBUS_DETECTION BIT(7) +#define PTN5150_REG_INT_CABLE_ATTACH_MASK BIT(0) +#define PTN5150_REG_INT_CABLE_DETACH_MASK BIT(1) struct ptn5150_info { struct device *dev; @@ -83,12 +67,45 @@ static const struct regmap_config ptn5150_regmap_config = { .max_register = PTN5150_REG_END, }; +static void ptn5150_check_state(struct ptn5150_info *info) +{ + unsigned int port_status, reg_data, vbus; + int ret; + + ret = regmap_read(info->regmap, PTN5150_REG_CC_STATUS, ®_data); + if (ret) { + dev_err(info->dev, "failed to read CC STATUS %d\n", ret); + return; + } + + port_status = FIELD_GET(PTN5150_REG_CC_PORT_ATTACHMENT, reg_data); + + switch (port_status) { + case PTN5150_DFP_ATTACHED: + extcon_set_state_sync(info->edev, EXTCON_USB_HOST, false); + gpiod_set_value_cansleep(info->vbus_gpiod, 0); + extcon_set_state_sync(info->edev, EXTCON_USB, true); + break; + case PTN5150_UFP_ATTACHED: + extcon_set_state_sync(info->edev, EXTCON_USB, false); + vbus = FIELD_GET(PTN5150_REG_CC_VBUS_DETECTION, reg_data); + if (vbus) + gpiod_set_value_cansleep(info->vbus_gpiod, 0); + else + gpiod_set_value_cansleep(info->vbus_gpiod, 1); + + extcon_set_state_sync(info->edev, EXTCON_USB_HOST, true); + break; + default: + break; + } +} + static void ptn5150_irq_work(struct work_struct *work) { struct ptn5150_info *info = container_of(work, struct ptn5150_info, irq_work); int ret = 0; - unsigned int reg_data; unsigned int int_status; if (!info->edev) @@ -96,13 +113,6 @@ static void ptn5150_irq_work(struct work_struct *work) mutex_lock(&info->mutex); - ret = regmap_read(info->regmap, PTN5150_REG_CC_STATUS, ®_data); - if (ret) { - dev_err(info->dev, "failed to read CC STATUS %d\n", ret); - mutex_unlock(&info->mutex); - return; - } - /* Clear interrupt. Read would clear the register */ ret = regmap_read(info->regmap, PTN5150_REG_INT_STATUS, &int_status); if (ret) { @@ -116,47 +126,13 @@ static void ptn5150_irq_work(struct work_struct *work) cable_attach = int_status & PTN5150_REG_INT_CABLE_ATTACH_MASK; if (cable_attach) { - unsigned int port_status; - unsigned int vbus; - - port_status = ((reg_data & - PTN5150_REG_CC_PORT_ATTACHMENT_MASK) >> - PTN5150_REG_CC_PORT_ATTACHMENT_SHIFT); - - switch (port_status) { - case PTN5150_DFP_ATTACHED: - extcon_set_state_sync(info->edev, - EXTCON_USB_HOST, false); - gpiod_set_value(info->vbus_gpiod, 0); - extcon_set_state_sync(info->edev, EXTCON_USB, - true); - break; - case PTN5150_UFP_ATTACHED: - extcon_set_state_sync(info->edev, EXTCON_USB, - false); - vbus = ((reg_data & - PTN5150_REG_CC_VBUS_DETECTION_MASK) >> - PTN5150_REG_CC_VBUS_DETECTION_SHIFT); - if (vbus) - gpiod_set_value(info->vbus_gpiod, 0); - else - gpiod_set_value(info->vbus_gpiod, 1); - - extcon_set_state_sync(info->edev, - EXTCON_USB_HOST, true); - break; - default: - dev_err(info->dev, - "Unknown Port status : %x\n", - port_status); - break; - } + ptn5150_check_state(info); } else { extcon_set_state_sync(info->edev, EXTCON_USB_HOST, false); extcon_set_state_sync(info->edev, EXTCON_USB, false); - gpiod_set_value(info->vbus_gpiod, 0); + gpiod_set_value_cansleep(info->vbus_gpiod, 0); } } @@ -194,13 +170,10 @@ static int ptn5150_init_dev_type(struct ptn5150_info *info) return -EINVAL; } - vendor_id = ((reg_data & PTN5150_REG_DEVICE_ID_VENDOR_MASK) >> - PTN5150_REG_DEVICE_ID_VENDOR_SHIFT); - version_id = ((reg_data & PTN5150_REG_DEVICE_ID_VERSION_MASK) >> - PTN5150_REG_DEVICE_ID_VERSION_SHIFT); - - dev_info(info->dev, "Device type: version: 0x%x, vendor: 0x%x\n", - version_id, vendor_id); + vendor_id = FIELD_GET(PTN5150_REG_DEVICE_ID_VENDOR, reg_data); + version_id = FIELD_GET(PTN5150_REG_DEVICE_ID_VERSION, reg_data); + dev_dbg(info->dev, "Device type: version: 0x%x, vendor: 0x%x\n", + version_id, vendor_id); /* Clear any existing interrupts */ ret = regmap_read(info->regmap, PTN5150_REG_INT_STATUS, ®_data); @@ -221,8 +194,7 @@ static int ptn5150_init_dev_type(struct ptn5150_info *info) return 0; } -static int ptn5150_i2c_probe(struct i2c_client *i2c, - const struct i2c_device_id *id) +static int ptn5150_i2c_probe(struct i2c_client *i2c) { struct device *dev = &i2c->dev; struct device_node *np = i2c->dev.of_node; @@ -239,20 +211,15 @@ static int ptn5150_i2c_probe(struct i2c_client *i2c, info->dev = &i2c->dev; info->i2c = i2c; - info->int_gpiod = devm_gpiod_get(&i2c->dev, "int", GPIOD_IN); - if (IS_ERR(info->int_gpiod)) { - dev_err(dev, "failed to get INT GPIO\n"); - return PTR_ERR(info->int_gpiod); - } - info->vbus_gpiod = devm_gpiod_get(&i2c->dev, "vbus", GPIOD_IN); + info->vbus_gpiod = devm_gpiod_get(&i2c->dev, "vbus", GPIOD_OUT_LOW); if (IS_ERR(info->vbus_gpiod)) { - dev_err(dev, "failed to get VBUS GPIO\n"); - return PTR_ERR(info->vbus_gpiod); - } - ret = gpiod_direction_output(info->vbus_gpiod, 0); - if (ret) { - dev_err(dev, "failed to set VBUS GPIO direction\n"); - return -EINVAL; + ret = PTR_ERR(info->vbus_gpiod); + if (ret == -ENOENT) { + dev_info(dev, "No VBUS GPIO, ignoring VBUS control\n"); + info->vbus_gpiod = NULL; + } else { + return dev_err_probe(dev, ret, "failed to get VBUS GPIO\n"); + } } mutex_init(&info->mutex); @@ -261,28 +228,34 @@ static int ptn5150_i2c_probe(struct i2c_client *i2c, info->regmap = devm_regmap_init_i2c(i2c, &ptn5150_regmap_config); if (IS_ERR(info->regmap)) { - ret = PTR_ERR(info->regmap); - dev_err(info->dev, "failed to allocate register map: %d\n", - ret); - return ret; + return dev_err_probe(info->dev, PTR_ERR(info->regmap), + "failed to allocate register map\n"); } - if (info->int_gpiod) { + if (i2c->irq > 0) { + info->irq = i2c->irq; + } else { + info->int_gpiod = devm_gpiod_get(&i2c->dev, "int", GPIOD_IN); + if (IS_ERR(info->int_gpiod)) { + return dev_err_probe(dev, PTR_ERR(info->int_gpiod), + "failed to get INT GPIO\n"); + } + info->irq = gpiod_to_irq(info->int_gpiod); if (info->irq < 0) { dev_err(dev, "failed to get INTB IRQ\n"); return info->irq; } + } - ret = devm_request_threaded_irq(dev, info->irq, NULL, - ptn5150_irq_handler, - IRQF_TRIGGER_FALLING | - IRQF_ONESHOT, - i2c->name, info); - if (ret < 0) { - dev_err(dev, "failed to request handler for INTB IRQ\n"); - return ret; - } + ret = devm_request_threaded_irq(dev, info->irq, NULL, + ptn5150_irq_handler, + IRQF_TRIGGER_FALLING | + IRQF_ONESHOT, + i2c->name, info); + if (ret < 0) { + dev_err(dev, "failed to request handler for INTB IRQ\n"); + return ret; } /* Allocate extcon device */ @@ -299,11 +272,26 @@ static int ptn5150_i2c_probe(struct i2c_client *i2c, return ret; } + extcon_set_property_capability(info->edev, EXTCON_USB, + EXTCON_PROP_USB_VBUS); + extcon_set_property_capability(info->edev, EXTCON_USB_HOST, + EXTCON_PROP_USB_VBUS); + extcon_set_property_capability(info->edev, EXTCON_USB_HOST, + EXTCON_PROP_USB_TYPEC_POLARITY); + /* Initialize PTN5150 device and print vendor id and version id */ ret = ptn5150_init_dev_type(info); if (ret) return -EINVAL; + /* + * Update current extcon state if for example OTG connection was there + * before the probe + */ + mutex_lock(&info->mutex); + ptn5150_check_state(info); + mutex_unlock(&info->mutex); + return 0; } @@ -324,16 +312,12 @@ static struct i2c_driver ptn5150_i2c_driver = { .name = "ptn5150", .of_match_table = ptn5150_dt_match, }, - .probe = ptn5150_i2c_probe, + .probe_new = ptn5150_i2c_probe, .id_table = ptn5150_i2c_id, }; - -static int __init ptn5150_i2c_init(void) -{ - return i2c_add_driver(&ptn5150_i2c_driver); -} -subsys_initcall(ptn5150_i2c_init); +module_i2c_driver(ptn5150_i2c_driver); MODULE_DESCRIPTION("NXP PTN5150 CC logic Extcon driver"); MODULE_AUTHOR("Vijai Kumar K <vijaikumar.kanagarajan@gmail.com>"); +MODULE_AUTHOR("Krzysztof Kozlowski <krzk@kernel.org>"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/extcon/extcon-usb-gpio.c b/drivers/extcon/extcon-usb-gpio.c index 98b5afa5b615..f06be6d4e2a9 100644 --- a/drivers/extcon/extcon-usb-gpio.c +++ b/drivers/extcon/extcon-usb-gpio.c @@ -2,7 +2,7 @@ /** * drivers/extcon/extcon-usb-gpio.c - USB GPIO extcon driver * - * Copyright (C) 2015 Texas Instruments Incorporated - http://www.ti.com + * Copyright (C) 2015 Texas Instruments Incorporated - https://www.ti.com * Author: Roger Quadros <rogerq@ti.com> */ diff --git a/drivers/fpga/dfl-fme-perf.c b/drivers/fpga/dfl-fme-perf.c index 6ce1ed222ea4..531266287eee 100644 --- a/drivers/fpga/dfl-fme-perf.c +++ b/drivers/fpga/dfl-fme-perf.c @@ -148,7 +148,7 @@ struct fme_perf_priv { struct device *dev; void __iomem *ioaddr; struct pmu pmu; - u64 id; + u16 id; u32 fab_users; u32 fab_port_id; diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index e220bec2927d..a2203d03c9e2 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -31,12 +31,12 @@ struct cci_drvdata { struct dfl_fpga_cdev *cdev; /* container device */ }; -static void __iomem *cci_pci_ioremap_bar(struct pci_dev *pcidev, int bar) +static void __iomem *cci_pci_ioremap_bar0(struct pci_dev *pcidev) { - if (pcim_iomap_regions(pcidev, BIT(bar), DRV_NAME)) + if (pcim_iomap_regions(pcidev, BIT(0), DRV_NAME)) return NULL; - return pcim_iomap_table(pcidev)[bar]; + return pcim_iomap_table(pcidev)[0]; } static int cci_pci_alloc_irq(struct pci_dev *pcidev) @@ -156,8 +156,8 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) goto irq_free_exit; } - /* start to find Device Feature List from Bar 0 */ - base = cci_pci_ioremap_bar(pcidev, 0); + /* start to find Device Feature List in Bar 0 */ + base = cci_pci_ioremap_bar0(pcidev); if (!base) { ret = -ENOMEM; goto irq_free_exit; @@ -172,7 +172,7 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) start = pci_resource_start(pcidev, 0); len = pci_resource_len(pcidev, 0); - dfl_fpga_enum_info_add_dfl(info, start, len, base); + dfl_fpga_enum_info_add_dfl(info, start, len); /* * find more Device Feature Lists (e.g. Ports) per information @@ -196,26 +196,24 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev) */ bar = FIELD_GET(FME_PORT_OFST_BAR_ID, v); offset = FIELD_GET(FME_PORT_OFST_DFH_OFST, v); - base = cci_pci_ioremap_bar(pcidev, bar); - if (!base) - continue; - start = pci_resource_start(pcidev, bar) + offset; len = pci_resource_len(pcidev, bar) - offset; - dfl_fpga_enum_info_add_dfl(info, start, len, - base + offset); + dfl_fpga_enum_info_add_dfl(info, start, len); } } else if (dfl_feature_is_port(base)) { start = pci_resource_start(pcidev, 0); len = pci_resource_len(pcidev, 0); - dfl_fpga_enum_info_add_dfl(info, start, len, base); + dfl_fpga_enum_info_add_dfl(info, start, len); } else { ret = -ENODEV; goto irq_free_exit; } + /* release I/O mappings for next step enumeration */ + pcim_iounmap_regions(pcidev, BIT(0)); + /* start enumeration with prepared enumeration information */ cdev = dfl_fpga_feature_devs_enumerate(info); if (IS_ERR(cdev)) { diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 649958a36e62..b450870b75ed 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -30,12 +30,6 @@ static DEFINE_MUTEX(dfl_id_mutex); * index to dfl_chardevs table. If no chardev support just set devt_type * as one invalid index (DFL_FPGA_DEVT_MAX). */ -enum dfl_id_type { - FME_ID, /* fme id allocation and mapping */ - PORT_ID, /* port id allocation and mapping */ - DFL_ID_MAX, -}; - enum dfl_fpga_devt_type { DFL_FPGA_DEVT_FME, DFL_FPGA_DEVT_PORT, @@ -58,7 +52,7 @@ static const char *dfl_pdata_key_strings[DFL_ID_MAX] = { */ struct dfl_dev_info { const char *name; - u32 dfh_id; + u16 dfh_id; struct idr id; enum dfl_fpga_devt_type devt_type; }; @@ -134,7 +128,7 @@ static enum dfl_id_type feature_dev_id_type(struct platform_device *pdev) return DFL_ID_MAX; } -static enum dfl_id_type dfh_id_to_type(u32 id) +static enum dfl_id_type dfh_id_to_type(u16 id) { int i; @@ -250,6 +244,249 @@ int dfl_fpga_check_port_id(struct platform_device *pdev, void *pport_id) } EXPORT_SYMBOL_GPL(dfl_fpga_check_port_id); +static DEFINE_IDA(dfl_device_ida); + +static const struct dfl_device_id * +dfl_match_one_device(const struct dfl_device_id *id, struct dfl_device *ddev) +{ + if (id->type == ddev->type && id->feature_id == ddev->feature_id) + return id; + + return NULL; +} + +static int dfl_bus_match(struct device *dev, struct device_driver *drv) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + struct dfl_driver *ddrv = to_dfl_drv(drv); + const struct dfl_device_id *id_entry; + + id_entry = ddrv->id_table; + if (id_entry) { + while (id_entry->feature_id) { + if (dfl_match_one_device(id_entry, ddev)) { + ddev->id_entry = id_entry; + return 1; + } + id_entry++; + } + } + + return 0; +} + +static int dfl_bus_probe(struct device *dev) +{ + struct dfl_driver *ddrv = to_dfl_drv(dev->driver); + struct dfl_device *ddev = to_dfl_dev(dev); + + return ddrv->probe(ddev); +} + +static int dfl_bus_remove(struct device *dev) +{ + struct dfl_driver *ddrv = to_dfl_drv(dev->driver); + struct dfl_device *ddev = to_dfl_dev(dev); + + if (ddrv->remove) + ddrv->remove(ddev); + + return 0; +} + +static int dfl_bus_uevent(struct device *dev, struct kobj_uevent_env *env) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + + /* The type has 4 valid bits and feature_id has 12 valid bits */ + return add_uevent_var(env, "MODALIAS=dfl:t%01Xf%03X", + ddev->type, ddev->feature_id); +} + +static ssize_t +type_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + + return sprintf(buf, "0x%x\n", ddev->type); +} +static DEVICE_ATTR_RO(type); + +static ssize_t +feature_id_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + + return sprintf(buf, "0x%x\n", ddev->feature_id); +} +static DEVICE_ATTR_RO(feature_id); + +static struct attribute *dfl_dev_attrs[] = { + &dev_attr_type.attr, + &dev_attr_feature_id.attr, + NULL, +}; +ATTRIBUTE_GROUPS(dfl_dev); + +static struct bus_type dfl_bus_type = { + .name = "dfl", + .match = dfl_bus_match, + .probe = dfl_bus_probe, + .remove = dfl_bus_remove, + .uevent = dfl_bus_uevent, + .dev_groups = dfl_dev_groups, +}; + +static void release_dfl_dev(struct device *dev) +{ + struct dfl_device *ddev = to_dfl_dev(dev); + + if (ddev->mmio_res.parent) + release_resource(&ddev->mmio_res); + + ida_simple_remove(&dfl_device_ida, ddev->id); + kfree(ddev->irqs); + kfree(ddev); +} + +static struct dfl_device * +dfl_dev_add(struct dfl_feature_platform_data *pdata, + struct dfl_feature *feature) +{ + struct platform_device *pdev = pdata->dev; + struct resource *parent_res; + struct dfl_device *ddev; + int id, i, ret; + + ddev = kzalloc(sizeof(*ddev), GFP_KERNEL); + if (!ddev) + return ERR_PTR(-ENOMEM); + + id = ida_simple_get(&dfl_device_ida, 0, 0, GFP_KERNEL); + if (id < 0) { + dev_err(&pdev->dev, "unable to get id\n"); + kfree(ddev); + return ERR_PTR(id); + } + + /* freeing resources by put_device() after device_initialize() */ + device_initialize(&ddev->dev); + ddev->dev.parent = &pdev->dev; + ddev->dev.bus = &dfl_bus_type; + ddev->dev.release = release_dfl_dev; + ddev->id = id; + ret = dev_set_name(&ddev->dev, "dfl_dev.%d", id); + if (ret) + goto put_dev; + + ddev->type = feature_dev_id_type(pdev); + ddev->feature_id = feature->id; + ddev->cdev = pdata->dfl_cdev; + + /* add mmio resource */ + parent_res = &pdev->resource[feature->resource_index]; + ddev->mmio_res.flags = IORESOURCE_MEM; + ddev->mmio_res.start = parent_res->start; + ddev->mmio_res.end = parent_res->end; + ddev->mmio_res.name = dev_name(&ddev->dev); + ret = insert_resource(parent_res, &ddev->mmio_res); + if (ret) { + dev_err(&pdev->dev, "%s failed to claim resource: %pR\n", + dev_name(&ddev->dev), &ddev->mmio_res); + goto put_dev; + } + + /* then add irq resource */ + if (feature->nr_irqs) { + ddev->irqs = kcalloc(feature->nr_irqs, + sizeof(*ddev->irqs), GFP_KERNEL); + if (!ddev->irqs) { + ret = -ENOMEM; + goto put_dev; + } + + for (i = 0; i < feature->nr_irqs; i++) + ddev->irqs[i] = feature->irq_ctx[i].irq; + + ddev->num_irqs = feature->nr_irqs; + } + + ret = device_add(&ddev->dev); + if (ret) + goto put_dev; + + dev_dbg(&pdev->dev, "add dfl_dev: %s\n", dev_name(&ddev->dev)); + return ddev; + +put_dev: + /* calls release_dfl_dev() which does the clean up */ + put_device(&ddev->dev); + return ERR_PTR(ret); +} + +static void dfl_devs_remove(struct dfl_feature_platform_data *pdata) +{ + struct dfl_feature *feature; + + dfl_fpga_dev_for_each_feature(pdata, feature) { + if (feature->ddev) { + device_unregister(&feature->ddev->dev); + feature->ddev = NULL; + } + } +} + +static int dfl_devs_add(struct dfl_feature_platform_data *pdata) +{ + struct dfl_feature *feature; + struct dfl_device *ddev; + int ret; + + dfl_fpga_dev_for_each_feature(pdata, feature) { + if (feature->ioaddr) + continue; + + if (feature->ddev) { + ret = -EEXIST; + goto err; + } + + ddev = dfl_dev_add(pdata, feature); + if (IS_ERR(ddev)) { + ret = PTR_ERR(ddev); + goto err; + } + + feature->ddev = ddev; + } + + return 0; + +err: + dfl_devs_remove(pdata); + return ret; +} + +int __dfl_driver_register(struct dfl_driver *dfl_drv, struct module *owner) +{ + if (!dfl_drv || !dfl_drv->probe || !dfl_drv->id_table) + return -EINVAL; + + dfl_drv->drv.owner = owner; + dfl_drv->drv.bus = &dfl_bus_type; + + return driver_register(&dfl_drv->drv); +} +EXPORT_SYMBOL(__dfl_driver_register); + +void dfl_driver_unregister(struct dfl_driver *dfl_drv) +{ + driver_unregister(&dfl_drv->drv); +} +EXPORT_SYMBOL(dfl_driver_unregister); + +#define is_header_feature(feature) ((feature)->id == FEATURE_ID_FIU_HEADER) + /** * dfl_fpga_dev_feature_uinit - uinit for sub features of dfl feature device * @pdev: feature device. @@ -259,12 +496,15 @@ void dfl_fpga_dev_feature_uinit(struct platform_device *pdev) struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev); struct dfl_feature *feature; - dfl_fpga_dev_for_each_feature(pdata, feature) + dfl_devs_remove(pdata); + + dfl_fpga_dev_for_each_feature(pdata, feature) { if (feature->ops) { if (feature->ops->uinit) feature->ops->uinit(pdev, feature); feature->ops = NULL; } + } } EXPORT_SYMBOL_GPL(dfl_fpga_dev_feature_uinit); @@ -273,8 +513,22 @@ static int dfl_feature_instance_init(struct platform_device *pdev, struct dfl_feature *feature, struct dfl_feature_driver *drv) { + void __iomem *base; int ret = 0; + if (!is_header_feature(feature)) { + base = devm_platform_ioremap_resource(pdev, + feature->resource_index); + if (IS_ERR(base)) { + dev_err(&pdev->dev, + "ioremap failed for feature 0x%x!\n", + feature->id); + return PTR_ERR(base); + } + + feature->ioaddr = base; + } + if (drv->ops->init) { ret = drv->ops->init(pdev, feature); if (ret) @@ -331,6 +585,10 @@ int dfl_fpga_dev_feature_init(struct platform_device *pdev, drv++; } + ret = dfl_devs_add(pdata); + if (ret) + goto exit; + return 0; exit: dfl_fpga_dev_feature_uinit(pdev); @@ -427,7 +685,9 @@ EXPORT_SYMBOL_GPL(dfl_fpga_dev_ops_unregister); * @irq_table: Linux IRQ numbers for all irqs, indexed by local irq index of * this device. * @feature_dev: current feature device. - * @ioaddr: header register region address of feature device in enumeration. + * @ioaddr: header register region address of current FIU in enumeration. + * @start: register resource start of current FIU. + * @len: max register resource length of current FIU. * @sub_features: a sub features linked list for feature device in enumeration. * @feature_num: number of sub features for feature device in enumeration. */ @@ -439,6 +699,8 @@ struct build_feature_devs_info { struct platform_device *feature_dev; void __iomem *ioaddr; + resource_size_t start; + resource_size_t len; struct list_head sub_features; int feature_num; }; @@ -454,7 +716,7 @@ struct build_feature_devs_info { * @nr_irqs: number of irqs of this sub feature. */ struct dfl_feature_info { - u64 fid; + u16 fid; struct resource mmio_res; void __iomem *ioaddr; struct list_head node; @@ -484,10 +746,7 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) struct dfl_feature_platform_data *pdata; struct dfl_feature_info *finfo, *p; enum dfl_id_type type; - int ret, index = 0; - - if (!fdev) - return 0; + int ret, index = 0, res_idx = 0; type = feature_dev_id_type(fdev); if (WARN_ON_ONCE(type >= DFL_ID_MAX)) @@ -530,16 +789,32 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) /* fill features and resource information for feature dev */ list_for_each_entry_safe(finfo, p, &binfo->sub_features, node) { - struct dfl_feature *feature = &pdata->features[index]; + struct dfl_feature *feature = &pdata->features[index++]; struct dfl_feature_irq_ctx *ctx; unsigned int i; /* save resource information for each feature */ feature->dev = fdev; feature->id = finfo->fid; - feature->resource_index = index; - feature->ioaddr = finfo->ioaddr; - fdev->resource[index++] = finfo->mmio_res; + + /* + * the FIU header feature has some fundamental functions (sriov + * set, port enable/disable) needed for the dfl bus device and + * other sub features. So its mmio resource should be mapped by + * DFL bus device. And we should not assign it to feature + * devices (dfl-fme/afu) again. + */ + if (is_header_feature(feature)) { + feature->resource_index = -1; + feature->ioaddr = + devm_ioremap_resource(binfo->dev, + &finfo->mmio_res); + if (IS_ERR(feature->ioaddr)) + return PTR_ERR(feature->ioaddr); + } else { + feature->resource_index = res_idx; + fdev->resource[res_idx++] = finfo->mmio_res; + } if (finfo->nr_irqs) { ctx = devm_kcalloc(binfo->dev, finfo->nr_irqs, @@ -582,19 +857,13 @@ static int build_info_commit_dev(struct build_feature_devs_info *binfo) static int build_info_create_dev(struct build_feature_devs_info *binfo, - enum dfl_id_type type, void __iomem *ioaddr) + enum dfl_id_type type) { struct platform_device *fdev; - int ret; if (type >= DFL_ID_MAX) return -EINVAL; - /* we will create a new device, commit current device first */ - ret = build_info_commit_dev(binfo); - if (ret) - return ret; - /* * we use -ENODEV as the initialization indicator which indicates * whether the id need to be reclaimed @@ -605,7 +874,7 @@ build_info_create_dev(struct build_feature_devs_info *binfo, binfo->feature_dev = fdev; binfo->feature_num = 0; - binfo->ioaddr = ioaddr; + INIT_LIST_HEAD(&binfo->sub_features); fdev->id = dfl_id_alloc(type, &fdev->dev); @@ -649,7 +918,7 @@ static inline u32 feature_size(void __iomem *start) return ofst ? ofst : 4096; } -static u64 feature_id(void __iomem *start) +static u16 feature_id(void __iomem *start) { u64 v = readq(start + DFH); u16 id = FIELD_GET(DFH_ID, v); @@ -667,7 +936,7 @@ static u64 feature_id(void __iomem *start) } static int parse_feature_irqs(struct build_feature_devs_info *binfo, - resource_size_t ofst, u64 fid, + resource_size_t ofst, u16 fid, unsigned int *irq_base, unsigned int *nr_irqs) { void __iomem *base = binfo->ioaddr + ofst; @@ -713,12 +982,12 @@ static int parse_feature_irqs(struct build_feature_devs_info *binfo, return 0; } - dev_dbg(binfo->dev, "feature: 0x%llx, irq_base: %u, nr_irqs: %u\n", + dev_dbg(binfo->dev, "feature: 0x%x, irq_base: %u, nr_irqs: %u\n", fid, ibase, inr); if (ibase + inr > binfo->nr_irqs) { dev_err(binfo->dev, - "Invalid interrupt number in feature 0x%llx\n", fid); + "Invalid interrupt number in feature 0x%x\n", fid); return -EINVAL; } @@ -726,7 +995,7 @@ static int parse_feature_irqs(struct build_feature_devs_info *binfo, virq = binfo->irq_table[ibase + i]; if (virq < 0 || virq > NR_IRQS) { dev_err(binfo->dev, - "Invalid irq table entry for feature 0x%llx\n", + "Invalid irq table entry for feature 0x%x\n", fid); return -EINVAL; } @@ -747,18 +1016,17 @@ static int parse_feature_irqs(struct build_feature_devs_info *binfo, */ static int create_feature_instance(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst, - resource_size_t size, u64 fid) + resource_size_t ofst, resource_size_t size, u16 fid) { unsigned int irq_base, nr_irqs; struct dfl_feature_info *finfo; int ret; /* read feature size and id if inputs are invalid */ - size = size ? size : feature_size(dfl->ioaddr + ofst); - fid = fid ? fid : feature_id(dfl->ioaddr + ofst); + size = size ? size : feature_size(binfo->ioaddr + ofst); + fid = fid ? fid : feature_id(binfo->ioaddr + ofst); - if (dfl->len - ofst < size) + if (binfo->len - ofst < size) return -EINVAL; ret = parse_feature_irqs(binfo, ofst, fid, &irq_base, &nr_irqs); @@ -770,12 +1038,11 @@ create_feature_instance(struct build_feature_devs_info *binfo, return -ENOMEM; finfo->fid = fid; - finfo->mmio_res.start = dfl->start + ofst; + finfo->mmio_res.start = binfo->start + ofst; finfo->mmio_res.end = finfo->mmio_res.start + size - 1; finfo->mmio_res.flags = IORESOURCE_MEM; finfo->irq_base = irq_base; finfo->nr_irqs = nr_irqs; - finfo->ioaddr = dfl->ioaddr + ofst; list_add_tail(&finfo->node, &binfo->sub_features); binfo->feature_num++; @@ -784,7 +1051,6 @@ create_feature_instance(struct build_feature_devs_info *binfo, } static int parse_feature_port_afu(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) { u64 v = readq(binfo->ioaddr + PORT_HDR_CAP); @@ -792,21 +1058,22 @@ static int parse_feature_port_afu(struct build_feature_devs_info *binfo, WARN_ON(!size); - return create_feature_instance(binfo, dfl, ofst, size, FEATURE_ID_AFU); + return create_feature_instance(binfo, ofst, size, FEATURE_ID_AFU); } +#define is_feature_dev_detected(binfo) (!!(binfo)->feature_dev) + static int parse_feature_afu(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) { - if (!binfo->feature_dev) { + if (!is_feature_dev_detected(binfo)) { dev_err(binfo->dev, "this AFU does not belong to any FIU.\n"); return -EINVAL; } switch (feature_dev_id_type(binfo->feature_dev)) { case PORT_ID: - return parse_feature_port_afu(binfo, dfl, ofst); + return parse_feature_port_afu(binfo, ofst); default: dev_info(binfo->dev, "AFU belonging to FIU %s is not supported yet.\n", binfo->feature_dev->name); @@ -815,35 +1082,79 @@ static int parse_feature_afu(struct build_feature_devs_info *binfo, return 0; } +static int build_info_prepare(struct build_feature_devs_info *binfo, + resource_size_t start, resource_size_t len) +{ + struct device *dev = binfo->dev; + void __iomem *ioaddr; + + if (!devm_request_mem_region(dev, start, len, dev_name(dev))) { + dev_err(dev, "request region fail, start:%pa, len:%pa\n", + &start, &len); + return -EBUSY; + } + + ioaddr = devm_ioremap(dev, start, len); + if (!ioaddr) { + dev_err(dev, "ioremap region fail, start:%pa, len:%pa\n", + &start, &len); + return -ENOMEM; + } + + binfo->start = start; + binfo->len = len; + binfo->ioaddr = ioaddr; + + return 0; +} + +static void build_info_complete(struct build_feature_devs_info *binfo) +{ + devm_iounmap(binfo->dev, binfo->ioaddr); + devm_release_mem_region(binfo->dev, binfo->start, binfo->len); +} + static int parse_feature_fiu(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) { - u32 id, offset; - u64 v; int ret = 0; + u32 offset; + u16 id; + u64 v; + + if (is_feature_dev_detected(binfo)) { + build_info_complete(binfo); + + ret = build_info_commit_dev(binfo); + if (ret) + return ret; - v = readq(dfl->ioaddr + ofst + DFH); + ret = build_info_prepare(binfo, binfo->start + ofst, + binfo->len - ofst); + if (ret) + return ret; + } + + v = readq(binfo->ioaddr + DFH); id = FIELD_GET(DFH_ID, v); /* create platform device for dfl feature dev */ - ret = build_info_create_dev(binfo, dfh_id_to_type(id), - dfl->ioaddr + ofst); + ret = build_info_create_dev(binfo, dfh_id_to_type(id)); if (ret) return ret; - ret = create_feature_instance(binfo, dfl, ofst, 0, 0); + ret = create_feature_instance(binfo, 0, 0, 0); if (ret) return ret; /* * find and parse FIU's child AFU via its NEXT_AFU register. * please note that only Port has valid NEXT_AFU pointer per spec. */ - v = readq(dfl->ioaddr + ofst + NEXT_AFU); + v = readq(binfo->ioaddr + NEXT_AFU); offset = FIELD_GET(NEXT_AFU_NEXT_DFH_OFST, v); if (offset) - return parse_feature_afu(binfo, dfl, ofst + offset); + return parse_feature_afu(binfo, offset); dev_dbg(binfo->dev, "No AFUs detected on FIU %d\n", id); @@ -851,41 +1162,39 @@ static int parse_feature_fiu(struct build_feature_devs_info *binfo, } static int parse_feature_private(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) { - if (!binfo->feature_dev) { - dev_err(binfo->dev, "the private feature %llx does not belong to any AFU.\n", - (unsigned long long)feature_id(dfl->ioaddr + ofst)); + if (!is_feature_dev_detected(binfo)) { + dev_err(binfo->dev, "the private feature 0x%x does not belong to any AFU.\n", + feature_id(binfo->ioaddr + ofst)); return -EINVAL; } - return create_feature_instance(binfo, dfl, ofst, 0, 0); + return create_feature_instance(binfo, ofst, 0, 0); } /** * parse_feature - parse a feature on given device feature list * * @binfo: build feature devices information. - * @dfl: device feature list to parse - * @ofst: offset to feature header on this device feature list + * @ofst: offset to current FIU header */ static int parse_feature(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl, resource_size_t ofst) + resource_size_t ofst) { u64 v; u32 type; - v = readq(dfl->ioaddr + ofst + DFH); + v = readq(binfo->ioaddr + ofst + DFH); type = FIELD_GET(DFH_TYPE, v); switch (type) { case DFH_TYPE_AFU: - return parse_feature_afu(binfo, dfl, ofst); + return parse_feature_afu(binfo, ofst); case DFH_TYPE_PRIVATE: - return parse_feature_private(binfo, dfl, ofst); + return parse_feature_private(binfo, ofst); case DFH_TYPE_FIU: - return parse_feature_fiu(binfo, dfl, ofst); + return parse_feature_fiu(binfo, ofst); default: dev_info(binfo->dev, "Feature Type %x is not supported.\n", type); @@ -895,14 +1204,17 @@ static int parse_feature(struct build_feature_devs_info *binfo, } static int parse_feature_list(struct build_feature_devs_info *binfo, - struct dfl_fpga_enum_dfl *dfl) + resource_size_t start, resource_size_t len) { - void __iomem *start = dfl->ioaddr; - void __iomem *end = dfl->ioaddr + dfl->len; + resource_size_t end = start + len; int ret = 0; u32 ofst = 0; u64 v; + ret = build_info_prepare(binfo, start, len); + if (ret) + return ret; + /* walk through the device feature list via DFH's next DFH pointer. */ for (; start < end; start += ofst) { if (end - start < DFH_SIZE) { @@ -910,11 +1222,11 @@ static int parse_feature_list(struct build_feature_devs_info *binfo, return -EINVAL; } - ret = parse_feature(binfo, dfl, start - dfl->ioaddr); + ret = parse_feature(binfo, start - binfo->start); if (ret) return ret; - v = readq(start + DFH); + v = readq(binfo->ioaddr + start - binfo->start + DFH); ofst = FIELD_GET(DFH_NEXT_HDR_OFST, v); /* stop parsing if EOL(End of List) is set or offset is 0 */ @@ -923,7 +1235,12 @@ static int parse_feature_list(struct build_feature_devs_info *binfo, } /* commit current feature device when reach the end of list */ - return build_info_commit_dev(binfo); + build_info_complete(binfo); + + if (is_feature_dev_detected(binfo)) + ret = build_info_commit_dev(binfo); + + return ret; } struct dfl_fpga_enum_info *dfl_fpga_enum_info_alloc(struct device *dev) @@ -976,7 +1293,6 @@ EXPORT_SYMBOL_GPL(dfl_fpga_enum_info_free); * @info: ptr to dfl_fpga_enum_info * @start: mmio resource address of the device feature list. * @len: mmio resource length of the device feature list. - * @ioaddr: mapped mmio resource address of the device feature list. * * One FPGA device may have one or more Device Feature Lists (DFLs), use this * function to add information of each DFL to common data structure for next @@ -985,8 +1301,7 @@ EXPORT_SYMBOL_GPL(dfl_fpga_enum_info_free); * Return: 0 on success, negative error code otherwise. */ int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, - resource_size_t start, resource_size_t len, - void __iomem *ioaddr) + resource_size_t start, resource_size_t len) { struct dfl_fpga_enum_dfl *dfl; @@ -996,7 +1311,6 @@ int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, dfl->start = start; dfl->len = len; - dfl->ioaddr = ioaddr; list_add_tail(&dfl->node, &info->dfls); @@ -1119,7 +1433,7 @@ dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enum_info *info) * Lists. */ list_for_each_entry(dfl, &info->dfls, node) { - ret = parse_feature_list(binfo, dfl); + ret = parse_feature_list(binfo, dfl->start, dfl->len); if (ret) { remove_feature_devs(cdev); build_info_free(binfo); @@ -1212,11 +1526,17 @@ static int __init dfl_fpga_init(void) { int ret; + ret = bus_register(&dfl_bus_type); + if (ret) + return ret; + dfl_ids_init(); ret = dfl_chardev_init(); - if (ret) + if (ret) { dfl_ids_destroy(); + bus_unregister(&dfl_bus_type); + } return ret; } @@ -1424,7 +1744,7 @@ static int do_set_irq_trigger(struct dfl_feature *feature, unsigned int idx, return 0; feature->irq_ctx[idx].name = - kasprintf(GFP_KERNEL, "fpga-irq[%u](%s-%llx)", idx, + kasprintf(GFP_KERNEL, "fpga-irq[%u](%s-%x)", idx, dev_name(&pdev->dev), feature->id); if (!feature->irq_ctx[idx].name) return -ENOMEM; @@ -1554,6 +1874,7 @@ static void __exit dfl_fpga_exit(void) { dfl_chardev_uinit(); dfl_ids_destroy(); + bus_unregister(&dfl_bus_type); } module_init(dfl_fpga_init); diff --git a/drivers/fpga/dfl.h b/drivers/fpga/dfl.h index a32dfba2a88b..5dc758f655b7 100644 --- a/drivers/fpga/dfl.h +++ b/drivers/fpga/dfl.h @@ -197,7 +197,7 @@ int dfl_fpga_check_port_id(struct platform_device *pdev, void *pport_id); * @id: unique dfl private feature id. */ struct dfl_feature_id { - u64 id; + u16 id; }; /** @@ -236,16 +236,18 @@ struct dfl_feature_irq_ctx { * @irq_ctx: interrupt context list. * @nr_irqs: number of interrupt contexts. * @ops: ops of this sub feature. + * @ddev: ptr to the dfl device of this sub feature. * @priv: priv data of this feature. */ struct dfl_feature { struct platform_device *dev; - u64 id; + u16 id; int resource_index; void __iomem *ioaddr; struct dfl_feature_irq_ctx *irq_ctx; unsigned int nr_irqs; const struct dfl_feature_ops *ops; + struct dfl_device *ddev; void *priv; }; @@ -365,7 +367,7 @@ struct platform_device *dfl_fpga_inode_to_feature_dev(struct inode *inode) (feature) < (pdata)->features + (pdata)->num; (feature)++) static inline -struct dfl_feature *dfl_get_feature_by_id(struct device *dev, u64 id) +struct dfl_feature *dfl_get_feature_by_id(struct device *dev, u16 id) { struct dfl_feature_platform_data *pdata = dev_get_platdata(dev); struct dfl_feature *feature; @@ -378,7 +380,7 @@ struct dfl_feature *dfl_get_feature_by_id(struct device *dev, u64 id) } static inline -void __iomem *dfl_get_feature_ioaddr_by_id(struct device *dev, u64 id) +void __iomem *dfl_get_feature_ioaddr_by_id(struct device *dev, u16 id) { struct dfl_feature *feature = dfl_get_feature_by_id(dev, id); @@ -389,7 +391,7 @@ void __iomem *dfl_get_feature_ioaddr_by_id(struct device *dev, u64 id) return NULL; } -static inline bool is_dfl_feature_present(struct device *dev, u64 id) +static inline bool is_dfl_feature_present(struct device *dev, u16 id) { return !!dfl_get_feature_ioaddr_by_id(dev, id); } @@ -441,22 +443,17 @@ struct dfl_fpga_enum_info { * * @start: base address of this device feature list. * @len: size of this device feature list. - * @ioaddr: mapped base address of this device feature list. * @node: node in list of device feature lists. */ struct dfl_fpga_enum_dfl { resource_size_t start; resource_size_t len; - - void __iomem *ioaddr; - struct list_head node; }; struct dfl_fpga_enum_info *dfl_fpga_enum_info_alloc(struct device *dev); int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, - resource_size_t start, resource_size_t len, - void __iomem *ioaddr); + resource_size_t start, resource_size_t len); int dfl_fpga_enum_info_add_irq(struct dfl_fpga_enum_info *info, unsigned int nr_irqs, int *irq_table); void dfl_fpga_enum_info_free(struct dfl_fpga_enum_info *info); @@ -519,4 +516,88 @@ long dfl_feature_ioctl_set_irq(struct platform_device *pdev, struct dfl_feature *feature, unsigned long arg); +/** + * enum dfl_id_type - define the DFL FIU types + */ +enum dfl_id_type { + FME_ID, + PORT_ID, + DFL_ID_MAX, +}; + +/** + * struct dfl_device_id - dfl device identifier + * @type: contains 4 bits DFL FIU type of the device. See enum dfl_id_type. + * @feature_id: contains 12 bits feature identifier local to its DFL FIU type. + * @driver_data: driver specific data. + */ +struct dfl_device_id { + u8 type; + u16 feature_id; + unsigned long driver_data; +}; + +/** + * struct dfl_device - represent an dfl device on dfl bus + * + * @dev: generic device interface. + * @id: id of the dfl device. + * @type: type of DFL FIU of the device. See enum dfl_id_type. + * @feature_id: 16 bits feature identifier local to its DFL FIU type. + * @mmio_res: mmio resource of this dfl device. + * @irqs: list of Linux IRQ numbers of this dfl device. + * @num_irqs: number of IRQs supported by this dfl device. + * @cdev: pointer to DFL FPGA container device this dfl device belongs to. + * @id_entry: matched id entry in dfl driver's id table. + */ +struct dfl_device { + struct device dev; + int id; + u8 type; + u16 feature_id; + struct resource mmio_res; + int *irqs; + unsigned int num_irqs; + struct dfl_fpga_cdev *cdev; + const struct dfl_device_id *id_entry; +}; + +/** + * struct dfl_driver - represent an dfl device driver + * + * @drv: driver model structure. + * @id_table: pointer to table of device IDs the driver is interested in. + * { } member terminated. + * @probe: mandatory callback for device binding. + * @remove: callback for device unbinding. + */ +struct dfl_driver { + struct device_driver drv; + const struct dfl_device_id *id_table; + + int (*probe)(struct dfl_device *dfl_dev); + void (*remove)(struct dfl_device *dfl_dev); +}; + +#define to_dfl_dev(d) container_of(d, struct dfl_device, dev) +#define to_dfl_drv(d) container_of(d, struct dfl_driver, drv) + +/* + * use a macro to avoid include chaining to get THIS_MODULE. + */ +#define dfl_driver_register(drv) \ + __dfl_driver_register(drv, THIS_MODULE) +int __dfl_driver_register(struct dfl_driver *dfl_drv, struct module *owner); +void dfl_driver_unregister(struct dfl_driver *dfl_drv); + +/* + * module_dfl_driver() - Helper macro for drivers that don't do + * anything special in module init/exit. This eliminates a lot of + * boilerplate. Each module may only use this macro once, and + * calling it replaces module_init() and module_exit(). + */ +#define module_dfl_driver(__dfl_driver) \ + module_driver(__dfl_driver, dfl_driver_register, \ + dfl_driver_unregister) + #endif /* __FPGA_DFL_H */ diff --git a/drivers/fpga/fpga-region.c b/drivers/fpga/fpga-region.c index bde5a9d460c5..c3134b89c3fe 100644 --- a/drivers/fpga/fpga-region.c +++ b/drivers/fpga/fpga-region.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * FPGA Region - Device Tree support for FPGA programming under Linux + * FPGA Region - Support for FPGA programming under Linux * * Copyright (C) 2013-2016 Altera Corporation * Copyright (C) 2017 Intel Corporation diff --git a/drivers/fpga/stratix10-soc.c b/drivers/fpga/stratix10-soc.c index 44b7c569d4dc..657a70c5fc99 100644 --- a/drivers/fpga/stratix10-soc.c +++ b/drivers/fpga/stratix10-soc.c @@ -196,17 +196,13 @@ static int s10_ops_write_init(struct fpga_manager *mgr, if (ret < 0) goto init_done; - ret = wait_for_completion_interruptible_timeout( + ret = wait_for_completion_timeout( &priv->status_return_completion, S10_RECONFIG_TIMEOUT); if (!ret) { dev_err(dev, "timeout waiting for RECONFIG_REQUEST\n"); ret = -ETIMEDOUT; goto init_done; } - if (ret < 0) { - dev_err(dev, "error (%d) waiting for RECONFIG_REQUEST\n", ret); - goto init_done; - } ret = 0; if (!test_and_clear_bit(SVC_STATUS_OK, &priv->status)) { @@ -318,7 +314,7 @@ static int s10_ops_write(struct fpga_manager *mgr, const char *buf, */ wait_status = 1; /* not timed out */ if (!priv->status) - wait_status = wait_for_completion_interruptible_timeout( + wait_status = wait_for_completion_timeout( &priv->status_return_completion, S10_BUFFER_TIMEOUT); @@ -340,13 +336,6 @@ static int s10_ops_write(struct fpga_manager *mgr, const char *buf, ret = -ETIMEDOUT; break; } - if (wait_status < 0) { - ret = wait_status; - dev_err(dev, - "error (%d) waiting for svc layer buffers\n", - ret); - break; - } } if (!s10_free_buffers(mgr)) @@ -372,7 +361,7 @@ static int s10_ops_write_complete(struct fpga_manager *mgr, if (ret < 0) break; - ret = wait_for_completion_interruptible_timeout( + ret = wait_for_completion_timeout( &priv->status_return_completion, timeout); if (!ret) { dev_err(dev, @@ -380,12 +369,6 @@ static int s10_ops_write_complete(struct fpga_manager *mgr, ret = -ETIMEDOUT; break; } - if (ret < 0) { - dev_err(dev, - "error (%d) waiting for RECONFIG_COMPLETED\n", - ret); - break; - } /* Not error or timeout, so ret is # of jiffies until timeout */ timeout = ret; ret = 0; diff --git a/drivers/fpga/xilinx-spi.c b/drivers/fpga/xilinx-spi.c index 2967aa2a74e2..824abbbd631e 100644 --- a/drivers/fpga/xilinx-spi.c +++ b/drivers/fpga/xilinx-spi.c @@ -27,11 +27,22 @@ struct xilinx_spi_conf { struct gpio_desc *done; }; -static enum fpga_mgr_states xilinx_spi_state(struct fpga_manager *mgr) +static int get_done_gpio(struct fpga_manager *mgr) { struct xilinx_spi_conf *conf = mgr->priv; + int ret; + + ret = gpiod_get_value(conf->done); + + if (ret < 0) + dev_err(&mgr->dev, "Error reading DONE (%d)\n", ret); + + return ret; +} - if (!gpiod_get_value(conf->done)) +static enum fpga_mgr_states xilinx_spi_state(struct fpga_manager *mgr) +{ + if (!get_done_gpio(mgr)) return FPGA_MGR_STATE_RESET; return FPGA_MGR_STATE_UNKNOWN; @@ -57,11 +68,21 @@ static int wait_for_init_b(struct fpga_manager *mgr, int value, if (conf->init_b) { while (time_before(jiffies, timeout)) { - /* dump_state(conf, "wait for init_d .."); */ - if (gpiod_get_value(conf->init_b) == value) + int ret = gpiod_get_value(conf->init_b); + + if (ret == value) return 0; + + if (ret < 0) { + dev_err(&mgr->dev, "Error reading INIT_B (%d)\n", ret); + return ret; + } + usleep_range(100, 400); } + + dev_err(&mgr->dev, "Timeout waiting for INIT_B to %s\n", + value ? "assert" : "deassert"); return -ETIMEDOUT; } @@ -78,7 +99,7 @@ static int xilinx_spi_write_init(struct fpga_manager *mgr, int err; if (info->flags & FPGA_MGR_PARTIAL_RECONFIG) { - dev_err(&mgr->dev, "Partial reconfiguration not supported.\n"); + dev_err(&mgr->dev, "Partial reconfiguration not supported\n"); return -EINVAL; } @@ -86,7 +107,6 @@ static int xilinx_spi_write_init(struct fpga_manager *mgr, err = wait_for_init_b(mgr, 1, 1); /* min is 500 ns */ if (err) { - dev_err(&mgr->dev, "INIT_B pin did not go low\n"); gpiod_set_value(conf->prog_b, 0); return err; } @@ -94,12 +114,10 @@ static int xilinx_spi_write_init(struct fpga_manager *mgr, gpiod_set_value(conf->prog_b, 0); err = wait_for_init_b(mgr, 0, 0); - if (err) { - dev_err(&mgr->dev, "INIT_B pin did not go high\n"); + if (err) return err; - } - if (gpiod_get_value(conf->done)) { + if (get_done_gpio(mgr)) { dev_err(&mgr->dev, "Unexpected DONE pin state...\n"); return -EIO; } @@ -152,25 +170,46 @@ static int xilinx_spi_write_complete(struct fpga_manager *mgr, struct fpga_image_info *info) { struct xilinx_spi_conf *conf = mgr->priv; - unsigned long timeout; + unsigned long timeout = jiffies + usecs_to_jiffies(info->config_complete_timeout_us); + bool expired = false; + int done; int ret; - if (gpiod_get_value(conf->done)) - return xilinx_spi_apply_cclk_cycles(conf); + /* + * This loop is carefully written such that if the driver is + * scheduled out for more than 'timeout', we still check for DONE + * before giving up and we apply 8 extra CCLK cycles in all cases. + */ + while (!expired) { + expired = time_after(jiffies, timeout); - timeout = jiffies + usecs_to_jiffies(info->config_complete_timeout_us); - - while (time_before(jiffies, timeout)) { + done = get_done_gpio(mgr); + if (done < 0) + return done; ret = xilinx_spi_apply_cclk_cycles(conf); if (ret) return ret; - if (gpiod_get_value(conf->done)) - return xilinx_spi_apply_cclk_cycles(conf); + if (done) + return 0; + } + + if (conf->init_b) { + ret = gpiod_get_value(conf->init_b); + + if (ret < 0) { + dev_err(&mgr->dev, "Error reading INIT_B (%d)\n", ret); + return ret; + } + + dev_err(&mgr->dev, + ret ? "CRC error or invalid device\n" + : "Missing sync word or incomplete bitstream\n"); + } else { + dev_err(&mgr->dev, "Timeout after config data transfer\n"); } - dev_err(&mgr->dev, "Timeout after config data transfer.\n"); return -ETIMEDOUT; } diff --git a/drivers/fsi/fsi-core.c b/drivers/fsi/fsi-core.c index 8244da8a7241..4e60e84cd17a 100644 --- a/drivers/fsi/fsi-core.c +++ b/drivers/fsi/fsi-core.c @@ -50,6 +50,7 @@ static const int engine_page_size = 0x400; #define FSI_SMODE 0x0 /* R/W: Mode register */ #define FSI_SISC 0x8 /* R/W: Interrupt condition */ #define FSI_SSTAT 0x14 /* R : Slave status */ +#define FSI_SLBUS 0x30 /* W : LBUS Ownership */ #define FSI_LLMODE 0x100 /* R/W: Link layer mode register */ /* @@ -67,6 +68,11 @@ static const int engine_page_size = 0x400; #define FSI_SMODE_LBCRR_MASK 0xf /* Clk ratio mask */ /* + * SLBUS fields + */ +#define FSI_SLBUS_FORCE 0x80000000 /* Force LBUS ownership */ + +/* * LLMODE fields */ #define FSI_LLMODE_ASYNC 0x1 @@ -981,7 +987,7 @@ static int fsi_slave_init(struct fsi_master *master, int link, uint8_t id) uint32_t cfam_id; struct fsi_slave *slave; uint8_t crc; - __be32 data, llmode; + __be32 data, llmode, slbus; int rc; /* Currently, we only support single slaves on a link, and use the @@ -1052,6 +1058,14 @@ static int fsi_slave_init(struct fsi_master *master, int link, uint8_t id) } + slbus = cpu_to_be32(FSI_SLBUS_FORCE); + rc = fsi_master_write(master, link, id, FSI_SLAVE_BASE + FSI_SLBUS, + &slbus, sizeof(slbus)); + if (rc) + dev_warn(&master->dev, + "can't set slbus on slave:%02x:%02x %d\n", link, id, + rc); + rc = fsi_slave_set_smode(slave); if (rc) { dev_warn(&master->dev, @@ -1154,10 +1168,18 @@ static int fsi_master_write(struct fsi_master *master, int link, return rc; } +static int fsi_master_link_disable(struct fsi_master *master, int link) +{ + if (master->link_enable) + return master->link_enable(master, link, false); + + return 0; +} + static int fsi_master_link_enable(struct fsi_master *master, int link) { if (master->link_enable) - return master->link_enable(master, link); + return master->link_enable(master, link, true); return 0; } @@ -1192,12 +1214,15 @@ static int fsi_master_scan(struct fsi_master *master) } rc = fsi_master_break(master, link); if (rc) { + fsi_master_link_disable(master, link); dev_dbg(&master->dev, "break to link %d failed: %d\n", link, rc); continue; } - fsi_slave_init(master, link, 0); + rc = fsi_slave_init(master, link, 0); + if (rc) + fsi_master_link_disable(master, link); } return 0; diff --git a/drivers/fsi/fsi-master-aspeed.c b/drivers/fsi/fsi-master-aspeed.c index f49742b310c2..c006ec008a1a 100644 --- a/drivers/fsi/fsi-master-aspeed.c +++ b/drivers/fsi/fsi-master-aspeed.c @@ -13,6 +13,7 @@ #include <linux/regmap.h> #include <linux/slab.h> #include <linux/iopoll.h> +#include <linux/gpio/consumer.h> #include "fsi-master.h" @@ -21,6 +22,7 @@ struct fsi_master_aspeed { struct device *dev; void __iomem *base; struct clk *clk; + struct gpio_desc *cfam_reset_gpio; }; #define to_fsi_master_aspeed(m) \ @@ -82,7 +84,12 @@ static const u32 fsi_base = 0xa0000000; #define FSI_LINK_ENABLE_SETUP_TIME 10 /* in mS */ -#define DEFAULT_DIVISOR 14 +/* Run the bus at maximum speed by default */ +#define FSI_DIVISOR_DEFAULT 1 +#define FSI_DIVISOR_CABLED 2 +static u16 aspeed_fsi_divisor = FSI_DIVISOR_DEFAULT; +module_param_named(bus_div,aspeed_fsi_divisor, ushort, 0); + #define OPB_POLL_TIMEOUT 10000 static int __opb_write(struct fsi_master_aspeed *aspeed, u32 addr, @@ -241,9 +248,10 @@ static int aspeed_master_read(struct fsi_master *master, int link, struct fsi_master_aspeed *aspeed = to_fsi_master_aspeed(master); int ret; - if (id != 0) + if (id > 0x3) return -EINVAL; + addr |= id << 21; addr += link * FSI_HUB_LINK_SIZE; switch (size) { @@ -273,9 +281,10 @@ static int aspeed_master_write(struct fsi_master *master, int link, struct fsi_master_aspeed *aspeed = to_fsi_master_aspeed(master); int ret; - if (id != 0) + if (id > 0x3) return -EINVAL; + addr |= id << 21; addr += link * FSI_HUB_LINK_SIZE; switch (size) { @@ -299,32 +308,28 @@ static int aspeed_master_write(struct fsi_master *master, int link, return 0; } -static int aspeed_master_link_enable(struct fsi_master *master, int link) +static int aspeed_master_link_enable(struct fsi_master *master, int link, + bool enable) { struct fsi_master_aspeed *aspeed = to_fsi_master_aspeed(master); int idx, bit, ret; - __be32 reg, result; + __be32 reg; idx = link / 32; bit = link % 32; reg = cpu_to_be32(0x80000000 >> bit); + if (!enable) + return opb_writel(aspeed, ctrl_base + FSI_MCENP0 + (4 * idx), + reg); + ret = opb_writel(aspeed, ctrl_base + FSI_MSENP0 + (4 * idx), reg); if (ret) return ret; mdelay(FSI_LINK_ENABLE_SETUP_TIME); - ret = opb_readl(aspeed, ctrl_base + FSI_MENP0 + (4 * idx), &result); - if (ret) - return ret; - - if (result != reg) { - dev_err(aspeed->dev, "%s failed: %08x\n", __func__, result); - return -EIO; - } - return 0; } @@ -386,9 +391,11 @@ static int aspeed_master_init(struct fsi_master_aspeed *aspeed) opb_writel(aspeed, ctrl_base + FSI_MECTRL, reg); reg = cpu_to_be32(FSI_MMODE_ECRC | FSI_MMODE_EPC | FSI_MMODE_RELA - | fsi_mmode_crs0(DEFAULT_DIVISOR) - | fsi_mmode_crs1(DEFAULT_DIVISOR) + | fsi_mmode_crs0(aspeed_fsi_divisor) + | fsi_mmode_crs1(aspeed_fsi_divisor) | FSI_MMODE_P8_TO_LSB); + dev_info(aspeed->dev, "mmode set to %08x (divisor %d)\n", + be32_to_cpu(reg), aspeed_fsi_divisor); opb_writel(aspeed, ctrl_base + FSI_MMODE, reg); reg = cpu_to_be32(0xffff0000); @@ -419,6 +426,90 @@ static int aspeed_master_init(struct fsi_master_aspeed *aspeed) return 0; } +static ssize_t cfam_reset_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct fsi_master_aspeed *aspeed = dev_get_drvdata(dev); + + gpiod_set_value(aspeed->cfam_reset_gpio, 1); + usleep_range(900, 1000); + gpiod_set_value(aspeed->cfam_reset_gpio, 0); + + return count; +} + +static DEVICE_ATTR(cfam_reset, 0200, NULL, cfam_reset_store); + +static int setup_cfam_reset(struct fsi_master_aspeed *aspeed) +{ + struct device *dev = aspeed->dev; + struct gpio_desc *gpio; + int rc; + + gpio = devm_gpiod_get_optional(dev, "cfam-reset", GPIOD_OUT_LOW); + if (IS_ERR(gpio)) + return PTR_ERR(gpio); + if (!gpio) + return 0; + + aspeed->cfam_reset_gpio = gpio; + + rc = device_create_file(dev, &dev_attr_cfam_reset); + if (rc) { + devm_gpiod_put(dev, gpio); + return rc; + } + + return 0; +} + +static int tacoma_cabled_fsi_fixup(struct device *dev) +{ + struct gpio_desc *routing_gpio, *mux_gpio; + int gpio; + + /* + * The routing GPIO is a jumper indicating we should mux for the + * externally connected FSI cable. + */ + routing_gpio = devm_gpiod_get_optional(dev, "fsi-routing", + GPIOD_IN | GPIOD_FLAGS_BIT_NONEXCLUSIVE); + if (IS_ERR(routing_gpio)) + return PTR_ERR(routing_gpio); + if (!routing_gpio) + return 0; + + mux_gpio = devm_gpiod_get_optional(dev, "fsi-mux", GPIOD_ASIS); + if (IS_ERR(mux_gpio)) + return PTR_ERR(mux_gpio); + if (!mux_gpio) + return 0; + + gpio = gpiod_get_value(routing_gpio); + if (gpio < 0) + return gpio; + + /* If the routing GPIO is high we should set the mux to low. */ + if (gpio) { + /* + * Cable signal integrity means we should run the bus + * slightly slower. Do not override if a kernel param + * has already overridden. + */ + if (aspeed_fsi_divisor == FSI_DIVISOR_DEFAULT) + aspeed_fsi_divisor = FSI_DIVISOR_CABLED; + + gpiod_direction_output(mux_gpio, 0); + dev_info(dev, "FSI configured for external cable\n"); + } else { + gpiod_direction_output(mux_gpio, 1); + } + + devm_gpiod_put(dev, routing_gpio); + + return 0; +} + static int fsi_master_aspeed_probe(struct platform_device *pdev) { struct fsi_master_aspeed *aspeed; @@ -426,6 +517,12 @@ static int fsi_master_aspeed_probe(struct platform_device *pdev) int rc, links, reg; __be32 raw; + rc = tacoma_cabled_fsi_fixup(&pdev->dev); + if (rc) { + dev_err(&pdev->dev, "Tacoma FSI cable fixup failed\n"); + return rc; + } + aspeed = devm_kzalloc(&pdev->dev, sizeof(*aspeed), GFP_KERNEL); if (!aspeed) return -ENOMEM; @@ -448,6 +545,11 @@ static int fsi_master_aspeed_probe(struct platform_device *pdev) return rc; } + rc = setup_cfam_reset(aspeed); + if (rc) { + dev_err(&pdev->dev, "CFAM reset GPIO setup failed\n"); + } + writel(0x1, aspeed->base + OPB_CLK_SYNC); writel(OPB1_XFER_ACK_EN | OPB0_XFER_ACK_EN, aspeed->base + OPB_IRQ_MASK); diff --git a/drivers/fsi/fsi-master-ast-cf.c b/drivers/fsi/fsi-master-ast-cf.c index 04d10ea8d343..57a779a89b07 100644 --- a/drivers/fsi/fsi-master-ast-cf.c +++ b/drivers/fsi/fsi-master-ast-cf.c @@ -838,7 +838,7 @@ static int load_copro_firmware(struct fsi_master_acf *master) rc = request_firmware(&fw, FW_FILE_NAME, master->dev); if (rc) { dev_err( - master->dev, "Error %d to load firwmare '%s' !\n", + master->dev, "Error %d to load firmware '%s' !\n", rc, FW_FILE_NAME); return rc; } @@ -1039,7 +1039,8 @@ static void fsi_master_acf_setup_external(struct fsi_master_acf *master) gpiod_direction_input(master->gpio_data); } -static int fsi_master_acf_link_enable(struct fsi_master *_master, int link) +static int fsi_master_acf_link_enable(struct fsi_master *_master, int link, + bool enable) { struct fsi_master_acf *master = to_fsi_master_acf(_master); int rc = -EBUSY; @@ -1049,7 +1050,7 @@ static int fsi_master_acf_link_enable(struct fsi_master *_master, int link) mutex_lock(&master->lock); if (!master->external_mode) { - gpiod_set_value(master->gpio_enable, 1); + gpiod_set_value(master->gpio_enable, enable ? 1 : 0); rc = 0; } mutex_unlock(&master->lock); diff --git a/drivers/fsi/fsi-master-gpio.c b/drivers/fsi/fsi-master-gpio.c index 4dcce17f243f..aa97c4a250cb 100644 --- a/drivers/fsi/fsi-master-gpio.c +++ b/drivers/fsi/fsi-master-gpio.c @@ -678,7 +678,8 @@ static void fsi_master_gpio_init_external(struct fsi_master_gpio *master) gpiod_direction_input(master->gpio_data); } -static int fsi_master_gpio_link_enable(struct fsi_master *_master, int link) +static int fsi_master_gpio_link_enable(struct fsi_master *_master, int link, + bool enable) { struct fsi_master_gpio *master = to_fsi_master_gpio(_master); int rc = -EBUSY; @@ -688,7 +689,7 @@ static int fsi_master_gpio_link_enable(struct fsi_master *_master, int link) mutex_lock(&master->cmd_lock); if (!master->external_mode) { - gpiod_set_value(master->gpio_enable, 1); + gpiod_set_value(master->gpio_enable, enable ? 1 : 0); rc = 0; } mutex_unlock(&master->cmd_lock); diff --git a/drivers/fsi/fsi-master-hub.c b/drivers/fsi/fsi-master-hub.c index def35cf92571..01f0a796111e 100644 --- a/drivers/fsi/fsi-master-hub.c +++ b/drivers/fsi/fsi-master-hub.c @@ -77,7 +77,8 @@ static int hub_master_break(struct fsi_master *master, int link) return hub_master_write(master, link, 0, addr, &cmd, sizeof(cmd)); } -static int hub_master_link_enable(struct fsi_master *master, int link) +static int hub_master_link_enable(struct fsi_master *master, int link, + bool enable) { struct fsi_master_hub *hub = to_fsi_master_hub(master); int idx, bit; @@ -89,13 +90,17 @@ static int hub_master_link_enable(struct fsi_master *master, int link) reg = cpu_to_be32(0x80000000 >> bit); + if (!enable) + return fsi_device_write(hub->upstream, FSI_MCENP0 + (4 * idx), + ®, 4); + rc = fsi_device_write(hub->upstream, FSI_MSENP0 + (4 * idx), ®, 4); + if (rc) + return rc; mdelay(FSI_LINK_ENABLE_SETUP_TIME); - fsi_device_read(hub->upstream, FSI_MENP0 + (4 * idx), ®, 4); - - return rc; + return 0; } static void hub_master_release(struct device *dev) @@ -271,7 +276,7 @@ static int hub_master_remove(struct device *dev) return 0; } -static struct fsi_device_id hub_master_ids[] = { +static const struct fsi_device_id hub_master_ids[] = { { .engine_type = FSI_ENGID_HUB_MASTER, .version = FSI_VERSION_ANY, diff --git a/drivers/fsi/fsi-master.h b/drivers/fsi/fsi-master.h index 6e8d4d4d5149..cd6bee5e12a7 100644 --- a/drivers/fsi/fsi-master.h +++ b/drivers/fsi/fsi-master.h @@ -130,7 +130,8 @@ struct fsi_master { uint32_t addr, const void *val, size_t size); int (*term)(struct fsi_master *, int link, uint8_t id); int (*send_break)(struct fsi_master *, int link); - int (*link_enable)(struct fsi_master *, int link); + int (*link_enable)(struct fsi_master *, int link, + bool enable); int (*link_config)(struct fsi_master *, int link, u8 t_send_delay, u8 t_echo_delay); }; diff --git a/drivers/fsi/fsi-occ.c b/drivers/fsi/fsi-occ.c index 7da9c81759ac..9eeb856c8905 100644 --- a/drivers/fsi/fsi-occ.c +++ b/drivers/fsi/fsi-occ.c @@ -555,7 +555,7 @@ static int occ_probe(struct platform_device *pdev) hwmon_dev_info.id = occ->idx; hwmon_dev = platform_device_register_full(&hwmon_dev_info); - if (!hwmon_dev) + if (IS_ERR(hwmon_dev)) dev_warn(dev, "failed to create hwmon device\n"); return 0; diff --git a/drivers/fsi/fsi-sbefifo.c b/drivers/fsi/fsi-sbefifo.c index f54df9ebc8b3..bfd5e5da8020 100644 --- a/drivers/fsi/fsi-sbefifo.c +++ b/drivers/fsi/fsi-sbefifo.c @@ -1028,7 +1028,7 @@ static int sbefifo_remove(struct device *dev) return 0; } -static struct fsi_device_id sbefifo_ids[] = { +static const struct fsi_device_id sbefifo_ids[] = { { .engine_type = FSI_ENGID_SBE, .version = FSI_VERSION_ANY, diff --git a/drivers/fsi/fsi-scom.c b/drivers/fsi/fsi-scom.c index 004dc03ccf09..b45bfab7b7f5 100644 --- a/drivers/fsi/fsi-scom.c +++ b/drivers/fsi/fsi-scom.c @@ -627,7 +627,7 @@ static int scom_remove(struct device *dev) return 0; } -static struct fsi_device_id scom_ids[] = { +static const struct fsi_device_id scom_ids[] = { { .engine_type = FSI_ENGID_SCOM, .version = FSI_VERSION_ANY, diff --git a/drivers/greybus/interface.c b/drivers/greybus/interface.c index 58ea374d8aaa..9ec949a438ef 100644 --- a/drivers/greybus/interface.c +++ b/drivers/greybus/interface.c @@ -620,7 +620,7 @@ static struct attribute *interface_common_attrs[] = { static umode_t interface_unipro_is_visible(struct kobject *kobj, struct attribute *attr, int n) { - struct device *dev = container_of(kobj, struct device, kobj); + struct device *dev = kobj_to_dev(kobj); struct gb_interface *intf = to_gb_interface(dev); switch (intf->type) { @@ -635,7 +635,7 @@ static umode_t interface_unipro_is_visible(struct kobject *kobj, static umode_t interface_greybus_is_visible(struct kobject *kobj, struct attribute *attr, int n) { - struct device *dev = container_of(kobj, struct device, kobj); + struct device *dev = kobj_to_dev(kobj); struct gb_interface *intf = to_gb_interface(dev); switch (intf->type) { @@ -649,7 +649,7 @@ static umode_t interface_greybus_is_visible(struct kobject *kobj, static umode_t interface_power_is_visible(struct kobject *kobj, struct attribute *attr, int n) { - struct device *dev = container_of(kobj, struct device, kobj); + struct device *dev = kobj_to_dev(kobj); struct gb_interface *intf = to_gb_interface(dev); switch (intf->type) { diff --git a/drivers/hwtracing/coresight/Kconfig b/drivers/hwtracing/coresight/Kconfig index 02dbb5ca3bcf..c1198245461d 100644 --- a/drivers/hwtracing/coresight/Kconfig +++ b/drivers/hwtracing/coresight/Kconfig @@ -3,7 +3,7 @@ # Coresight configuration # menuconfig CORESIGHT - bool "CoreSight Tracing Support" + tristate "CoreSight Tracing Support" depends on ARM || ARM64 depends on OF || ACPI select ARM_AMBA @@ -15,17 +15,24 @@ menuconfig CORESIGHT specification and configure the right series of components when a trace source gets enabled. + To compile this driver as a module, choose M here: the + module will be called coresight. + if CORESIGHT config CORESIGHT_LINKS_AND_SINKS - bool "CoreSight Link and Sink drivers" + tristate "CoreSight Link and Sink drivers" help This enables support for CoreSight link and sink drivers that are responsible for transporting and collecting the trace data respectively. Link and sinks are dynamically aggregated with a trace entity at run time to form a complete trace path. + To compile these drivers as modules, choose M here: the + modules will be called coresight-funnel and coresight-replicator. + config CORESIGHT_LINK_AND_SINK_TMC - bool "Coresight generic TMC driver" + tristate "Coresight generic TMC driver" + depends on CORESIGHT_LINKS_AND_SINKS help This enables support for the Trace Memory Controller driver. @@ -34,8 +41,11 @@ config CORESIGHT_LINK_AND_SINK_TMC complies with the generic implementation of the component without special enhancement or added features. + To compile this driver as a module, choose M here: the + module will be called coresight-tmc. + config CORESIGHT_CATU - bool "Coresight Address Translation Unit (CATU) driver" + tristate "Coresight Address Translation Unit (CATU) driver" depends on CORESIGHT_LINK_AND_SINK_TMC help Enable support for the Coresight Address Translation Unit (CATU). @@ -45,8 +55,11 @@ config CORESIGHT_CATU by looking up the provided table. CATU can also be used in pass-through mode where the address is not translated. + To compile this driver as a module, choose M here: the + module will be called coresight-catu. + config CORESIGHT_SINK_TPIU - bool "Coresight generic TPIU driver" + tristate "Coresight generic TPIU driver" depends on CORESIGHT_LINKS_AND_SINKS help This enables support for the Trace Port Interface Unit driver, @@ -56,16 +69,22 @@ config CORESIGHT_SINK_TPIU connected to an external host for use case capturing more traces than the on-board coresight memory can handle. + To compile this driver as a module, choose M here: the + module will be called coresight-tpiu. + config CORESIGHT_SINK_ETBV10 - bool "Coresight ETBv1.0 driver" + tristate "Coresight ETBv1.0 driver" depends on CORESIGHT_LINKS_AND_SINKS help This enables support for the Embedded Trace Buffer version 1.0 driver that complies with the generic implementation of the component without special enhancement or added features. + To compile this driver as a module, choose M here: the + module will be called coresight-etb10. + config CORESIGHT_SOURCE_ETM3X - bool "CoreSight Embedded Trace Macrocell 3.x driver" + tristate "CoreSight Embedded Trace Macrocell 3.x driver" depends on !ARM64 select CORESIGHT_LINKS_AND_SINKS help @@ -74,8 +93,11 @@ config CORESIGHT_SOURCE_ETM3X This is primarily useful for instruction level tracing. Depending the ETM version data tracing may also be available. + To compile this driver as a module, choose M here: the + module will be called coresight-etm3x. + config CORESIGHT_SOURCE_ETM4X - bool "CoreSight Embedded Trace Macrocell 4.x driver" + tristate "CoreSight Embedded Trace Macrocell 4.x driver" depends on ARM64 select CORESIGHT_LINKS_AND_SINKS select PID_IN_CONTEXTIDR @@ -85,8 +107,11 @@ config CORESIGHT_SOURCE_ETM4X for instruction level tracing. Depending on the implemented version data tracing may also be available. + To compile this driver as a module, choose M here: the + module will be called coresight-etm4x. + config CORESIGHT_STM - bool "CoreSight System Trace Macrocell driver" + tristate "CoreSight System Trace Macrocell driver" depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64 select CORESIGHT_LINKS_AND_SINKS select STM @@ -96,6 +121,9 @@ config CORESIGHT_STM logging useful software events or data coming from various entities in the system, possibly running different OSs + To compile this driver as a module, choose M here: the + module will be called coresight-stm. + config CORESIGHT_CPU_DEBUG tristate "CoreSight CPU Debug driver" depends on ARM || ARM64 @@ -110,8 +138,11 @@ config CORESIGHT_CPU_DEBUG properly, please refer Documentation/trace/coresight/coresight-cpu-debug.rst for detailed description and the example for usage. + To compile this driver as a module, choose M here: the + module will be called coresight-cpu-debug. + config CORESIGHT_CTI - bool "CoreSight Cross Trigger Interface (CTI) driver" + tristate "CoreSight Cross Trigger Interface (CTI) driver" depends on ARM || ARM64 help This driver provides support for CoreSight CTI and CTM components. @@ -122,6 +153,9 @@ config CORESIGHT_CTI halt compared to disabling sources and sinks normally in driver software. + To compile this driver as a module, choose M here: the + module will be called coresight-cti. + config CORESIGHT_CTI_INTEGRATION_REGS bool "Access CTI CoreSight Integration Registers" depends on CORESIGHT_CTI diff --git a/drivers/hwtracing/coresight/Makefile b/drivers/hwtracing/coresight/Makefile index 19497d1d92bf..f20e357758d1 100644 --- a/drivers/hwtracing/coresight/Makefile +++ b/drivers/hwtracing/coresight/Makefile @@ -2,22 +2,24 @@ # # Makefile for CoreSight drivers. # -obj-$(CONFIG_CORESIGHT) += coresight.o coresight-etm-perf.o \ - coresight-platform.o coresight-sysfs.o -obj-$(CONFIG_CORESIGHT_LINK_AND_SINK_TMC) += coresight-tmc.o \ - coresight-tmc-etf.o \ - coresight-tmc-etr.o +obj-$(CONFIG_CORESIGHT) += coresight.o +coresight-y := coresight-core.o coresight-etm-perf.o coresight-platform.o \ + coresight-sysfs.o +obj-$(CONFIG_CORESIGHT_LINK_AND_SINK_TMC) += coresight-tmc.o +coresight-tmc-y := coresight-tmc-core.o coresight-tmc-etf.o \ + coresight-tmc-etr.o obj-$(CONFIG_CORESIGHT_SINK_TPIU) += coresight-tpiu.o obj-$(CONFIG_CORESIGHT_SINK_ETBV10) += coresight-etb10.o obj-$(CONFIG_CORESIGHT_LINKS_AND_SINKS) += coresight-funnel.o \ coresight-replicator.o -obj-$(CONFIG_CORESIGHT_SOURCE_ETM3X) += coresight-etm3x.o coresight-etm-cp14.o \ - coresight-etm3x-sysfs.o -obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o \ - coresight-etm4x-sysfs.o +obj-$(CONFIG_CORESIGHT_SOURCE_ETM3X) += coresight-etm3x.o +coresight-etm3x-y := coresight-etm3x-core.o coresight-etm-cp14.o \ + coresight-etm3x-sysfs.o +obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o +coresight-etm4x-y := coresight-etm4x-core.o coresight-etm4x-sysfs.o obj-$(CONFIG_CORESIGHT_STM) += coresight-stm.o obj-$(CONFIG_CORESIGHT_CPU_DEBUG) += coresight-cpu-debug.o obj-$(CONFIG_CORESIGHT_CATU) += coresight-catu.o -obj-$(CONFIG_CORESIGHT_CTI) += coresight-cti.o \ - coresight-cti-platform.o \ - coresight-cti-sysfs.o +obj-$(CONFIG_CORESIGHT_CTI) += coresight-cti.o +coresight-cti-y := coresight-cti-core.o coresight-cti-platform.o \ + coresight-cti-sysfs.o diff --git a/drivers/hwtracing/coresight/coresight-catu.c b/drivers/hwtracing/coresight/coresight-catu.c index 1801804a7762..99430f6cf5a5 100644 --- a/drivers/hwtracing/coresight/coresight-catu.c +++ b/drivers/hwtracing/coresight/coresight-catu.c @@ -358,7 +358,7 @@ static int catu_alloc_etr_buf(struct tmc_drvdata *tmc_drvdata, return 0; } -const struct etr_buf_operations etr_catu_buf_ops = { +static const struct etr_buf_operations etr_catu_buf_ops = { .alloc = catu_alloc_etr_buf, .free = catu_free_etr_buf, .sync = catu_sync_etr_buf, @@ -567,11 +567,21 @@ out: return ret; } +static int __exit catu_remove(struct amba_device *adev) +{ + struct catu_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + coresight_unregister(drvdata->csdev); + return 0; +} + static struct amba_id catu_ids[] = { CS_AMBA_ID(0x000bb9ee), {}, }; +MODULE_DEVICE_TABLE(amba, catu_ids); + static struct amba_driver catu_driver = { .drv = { .name = "coresight-catu", @@ -579,7 +589,30 @@ static struct amba_driver catu_driver = { .suppress_bind_attrs = true, }, .probe = catu_probe, + .remove = catu_remove, .id_table = catu_ids, }; -builtin_amba_driver(catu_driver); +static int __init catu_init(void) +{ + int ret; + + ret = amba_driver_register(&catu_driver); + if (ret) + pr_info("Error registering catu driver\n"); + tmc_etr_set_catu_ops(&etr_catu_buf_ops); + return ret; +} + +static void __exit catu_exit(void) +{ + tmc_etr_remove_catu_ops(); + amba_driver_unregister(&catu_driver); +} + +module_init(catu_init); +module_exit(catu_exit); + +MODULE_AUTHOR("Suzuki K Poulose <suzuki.poulose@arm.com>"); +MODULE_DESCRIPTION("Arm CoreSight Address Translation Unit (CATU) Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-catu.h b/drivers/hwtracing/coresight/coresight-catu.h index 80ceee3c739c..6160c2d75a56 100644 --- a/drivers/hwtracing/coresight/coresight-catu.h +++ b/drivers/hwtracing/coresight/coresight-catu.h @@ -108,6 +108,4 @@ static inline bool coresight_is_catu_device(struct coresight_device *csdev) return true; } -extern const struct etr_buf_operations etr_catu_buf_ops; - #endif diff --git a/drivers/hwtracing/coresight/coresight.c b/drivers/hwtracing/coresight/coresight-core.c index e9c90f2de34a..6994c1309b2b 100644 --- a/drivers/hwtracing/coresight/coresight.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -53,7 +53,22 @@ static struct list_head *stm_path; * beginning of the data collected in a buffer. That way the decoder knows that * it needs to look for another sync sequence. */ -const u32 barrier_pkt[4] = {0x7fffffff, 0x7fffffff, 0x7fffffff, 0x7fffffff}; +const u32 coresight_barrier_pkt[4] = {0x7fffffff, 0x7fffffff, 0x7fffffff, 0x7fffffff}; +EXPORT_SYMBOL_GPL(coresight_barrier_pkt); + +static const struct cti_assoc_op *cti_assoc_ops; + +void coresight_set_cti_ops(const struct cti_assoc_op *cti_op) +{ + cti_assoc_ops = cti_op; +} +EXPORT_SYMBOL_GPL(coresight_set_cti_ops); + +void coresight_remove_cti_ops(void) +{ + cti_assoc_ops = NULL; +} +EXPORT_SYMBOL_GPL(coresight_remove_cti_ops); static int coresight_id_match(struct device *dev, void *data) { @@ -179,6 +194,7 @@ int coresight_claim_device_unlocked(void __iomem *base) coresight_clear_claim_tags(base); return -EBUSY; } +EXPORT_SYMBOL_GPL(coresight_claim_device_unlocked); int coresight_claim_device(void __iomem *base) { @@ -190,6 +206,7 @@ int coresight_claim_device(void __iomem *base) return rc; } +EXPORT_SYMBOL_GPL(coresight_claim_device); /* * coresight_disclaim_device_unlocked : Clear the claim tags for the device. @@ -208,6 +225,7 @@ void coresight_disclaim_device_unlocked(void __iomem *base) */ WARN_ON_ONCE(1); } +EXPORT_SYMBOL_GPL(coresight_disclaim_device_unlocked); void coresight_disclaim_device(void __iomem *base) { @@ -215,6 +233,7 @@ void coresight_disclaim_device(void __iomem *base) coresight_disclaim_device_unlocked(base); CS_LOCK(base); } +EXPORT_SYMBOL_GPL(coresight_disclaim_device); /* enable or disable an associated CTI device of the supplied CS device */ static int @@ -222,16 +241,32 @@ coresight_control_assoc_ectdev(struct coresight_device *csdev, bool enable) { int ect_ret = 0; struct coresight_device *ect_csdev = csdev->ect_dev; + struct module *mod; if (!ect_csdev) return 0; + if ((!ect_ops(ect_csdev)->enable) || (!ect_ops(ect_csdev)->disable)) + return 0; + mod = ect_csdev->dev.parent->driver->owner; if (enable) { - if (ect_ops(ect_csdev)->enable) + if (try_module_get(mod)) { ect_ret = ect_ops(ect_csdev)->enable(ect_csdev); + if (ect_ret) { + module_put(mod); + } else { + get_device(ect_csdev->dev.parent); + csdev->ect_enabled = true; + } + } else + ect_ret = -ENODEV; } else { - if (ect_ops(ect_csdev)->disable) + if (csdev->ect_enabled) { ect_ret = ect_ops(ect_csdev)->disable(ect_csdev); + put_device(ect_csdev->dev.parent); + module_put(mod); + csdev->ect_enabled = false; + } } /* output warning if ECT enable is preventing trace operation */ @@ -253,6 +288,7 @@ void coresight_set_assoc_ectdev_mutex(struct coresight_device *csdev, csdev->ect_dev = ect_csdev; mutex_unlock(&coresight_mutex); } +EXPORT_SYMBOL_GPL(coresight_set_assoc_ectdev_mutex); static int coresight_enable_sink(struct coresight_device *csdev, u32 mode, void *data) @@ -467,6 +503,7 @@ void coresight_disable_path(struct list_head *path) { coresight_disable_path_from(path, NULL); } +EXPORT_SYMBOL_GPL(coresight_disable_path); int coresight_enable_path(struct list_head *path, u32 mode, void *sink_data) { @@ -540,50 +577,46 @@ struct coresight_device *coresight_get_sink(struct list_head *path) return csdev; } -static int coresight_enabled_sink(struct device *dev, const void *data) +static struct coresight_device * +coresight_find_enabled_sink(struct coresight_device *csdev) { - const bool *reset = data; - struct coresight_device *csdev = to_coresight_device(dev); + int i; + struct coresight_device *sink; if ((csdev->type == CORESIGHT_DEV_TYPE_SINK || csdev->type == CORESIGHT_DEV_TYPE_LINKSINK) && - csdev->activated) { - /* - * Now that we have a handle on the sink for this session, - * disable the sysFS "enable_sink" flag so that possible - * concurrent perf session that wish to use another sink don't - * trip on it. Doing so has no ramification for the current - * session. - */ - if (*reset) - csdev->activated = false; + csdev->activated) + return csdev; - return 1; + /* + * Recursively explore each port found on this element. + */ + for (i = 0; i < csdev->pdata->nr_outport; i++) { + struct coresight_device *child_dev; + + child_dev = csdev->pdata->conns[i].child_dev; + if (child_dev) + sink = coresight_find_enabled_sink(child_dev); + if (sink) + return sink; } - return 0; + return NULL; } /** - * coresight_get_enabled_sink - returns the first enabled sink found on the bus - * @deactivate: Whether the 'enable_sink' flag should be reset - * - * When operated from perf the deactivate parameter should be set to 'true'. - * That way the "enabled_sink" flag of the sink that was selected can be reset, - * allowing for other concurrent perf sessions to choose a different sink. + * coresight_get_enabled_sink - returns the first enabled sink using + * connection based search starting from the source reference * - * When operated from sysFS users have full control and as such the deactivate - * parameter should be set to 'false', hence mandating users to explicitly - * clear the flag. + * @source: Coresight source device reference */ -struct coresight_device *coresight_get_enabled_sink(bool deactivate) +struct coresight_device * +coresight_get_enabled_sink(struct coresight_device *source) { - struct device *dev = NULL; - - dev = bus_find_device(&coresight_bustype, NULL, &deactivate, - coresight_enabled_sink); + if (!source) + return NULL; - return dev ? to_coresight_device(dev) : NULL; + return coresight_find_enabled_sink(source); } static int coresight_sink_by_id(struct device *dev, const void *data) @@ -627,13 +660,45 @@ struct coresight_device *coresight_get_sink_by_id(u32 id) return dev ? to_coresight_device(dev) : NULL; } +/** + * coresight_get_ref- Helper function to increase reference count to module + * and device. + * Return true in successful case and power up the device. + * Return false when failed to get reference of module. + */ +static inline bool coresight_get_ref(struct coresight_device *csdev) +{ + struct device *dev = csdev->dev.parent; + + /* Make sure the driver can't be removed */ + if (!try_module_get(dev->driver->owner)) + return false; + /* Make sure the device can't go away */ + get_device(dev); + pm_runtime_get_sync(dev); + return true; +} + +/** + * coresight_put_ref- Helper function to decrease reference count to module + * and device. Power off the device. + */ +static inline void coresight_put_ref(struct coresight_device *csdev) +{ + struct device *dev = csdev->dev.parent; + + pm_runtime_put(dev); + put_device(dev); + module_put(dev->driver->owner); +} + /* * coresight_grab_device - Power up this device and any of the helper * devices connected to it for trace operation. Since the helper devices * don't appear on the trace path, they should be handled along with the * the master device. */ -static void coresight_grab_device(struct coresight_device *csdev) +static int coresight_grab_device(struct coresight_device *csdev) { int i; @@ -642,9 +707,20 @@ static void coresight_grab_device(struct coresight_device *csdev) child = csdev->pdata->conns[i].child_dev; if (child && child->type == CORESIGHT_DEV_TYPE_HELPER) - pm_runtime_get_sync(child->dev.parent); + if (!coresight_get_ref(child)) + goto err; } - pm_runtime_get_sync(csdev->dev.parent); + if (coresight_get_ref(csdev)) + return 0; +err: + for (i--; i >= 0; i--) { + struct coresight_device *child; + + child = csdev->pdata->conns[i].child_dev; + if (child && child->type == CORESIGHT_DEV_TYPE_HELPER) + coresight_put_ref(child); + } + return -ENODEV; } /* @@ -655,13 +731,13 @@ static void coresight_drop_device(struct coresight_device *csdev) { int i; - pm_runtime_put(csdev->dev.parent); + coresight_put_ref(csdev); for (i = 0; i < csdev->pdata->nr_outport; i++) { struct coresight_device *child; child = csdev->pdata->conns[i].child_dev; if (child && child->type == CORESIGHT_DEV_TYPE_HELPER) - pm_runtime_put(child->dev.parent); + coresight_put_ref(child); } } @@ -680,7 +756,7 @@ static int _coresight_build_path(struct coresight_device *csdev, struct coresight_device *sink, struct list_head *path) { - int i; + int i, ret; bool found = false; struct coresight_node *node; @@ -710,11 +786,14 @@ out: * is tell the PM runtime core we need this element and add a node * for it. */ + ret = coresight_grab_device(csdev); + if (ret) + return ret; + node = kzalloc(sizeof(struct coresight_node), GFP_KERNEL); if (!node) return -ENOMEM; - coresight_grab_device(csdev); node->csdev = csdev; list_add(&node->link, path); @@ -988,11 +1067,7 @@ int coresight_enable(struct coresight_device *csdev) goto out; } - /* - * Search for a valid sink for this session but don't reset the - * "enable_sink" flag in sysFS. Users get to do that explicitly. - */ - sink = coresight_get_enabled_sink(false); + sink = coresight_get_enabled_sink(csdev); if (!sink) { ret = -EINVAL; goto out; @@ -1188,7 +1263,6 @@ static void coresight_device_release(struct device *dev) { struct coresight_device *csdev = to_coresight_device(dev); - cti_remove_assoc_from_csdev(csdev); fwnode_handle_put(csdev->dev.fwnode); kfree(csdev->refcnt); kfree(csdev); @@ -1376,16 +1450,7 @@ int coresight_timeout(void __iomem *addr, u32 offset, int position, int value) return -EAGAIN; } - -struct bus_type coresight_bustype = { - .name = "coresight", -}; - -static int __init coresight_init(void) -{ - return bus_register(&coresight_bustype); -} -postcore_initcall(coresight_init); +EXPORT_SYMBOL_GPL(coresight_timeout); /* * coresight_release_platform_data: Release references to the devices connected @@ -1498,8 +1563,8 @@ struct coresight_device *coresight_register(struct coresight_desc *desc) ret = coresight_fixup_device_conns(csdev); if (!ret) ret = coresight_fixup_orphan_conns(csdev); - if (!ret) - cti_add_assoc_to_csdev(csdev); + if (!ret && cti_assoc_ops && cti_assoc_ops->add) + cti_assoc_ops->add(csdev); mutex_unlock(&coresight_mutex); if (ret) { @@ -1522,6 +1587,8 @@ void coresight_unregister(struct coresight_device *csdev) { etm_perf_del_symlink_sink(csdev); /* Remove references of that device in the topology */ + if (cti_assoc_ops && cti_assoc_ops->remove) + cti_assoc_ops->remove(csdev); coresight_remove_conns(csdev); coresight_clear_default_sink(csdev); coresight_release_platform_data(csdev, csdev->pdata); @@ -1552,6 +1619,7 @@ bool coresight_loses_context_with_cpu(struct device *dev) return fwnode_property_present(dev_fwnode(dev), "arm,coresight-loses-context-with-cpu"); } +EXPORT_SYMBOL_GPL(coresight_loses_context_with_cpu); /* * coresight_alloc_device_name - Get an index for a given device in the @@ -1592,3 +1660,35 @@ done: return name; } EXPORT_SYMBOL_GPL(coresight_alloc_device_name); + +struct bus_type coresight_bustype = { + .name = "coresight", +}; + +static int __init coresight_init(void) +{ + int ret; + + ret = bus_register(&coresight_bustype); + if (ret) + return ret; + + ret = etm_perf_init(); + if (ret) + bus_unregister(&coresight_bustype); + + return ret; +} + +static void __exit coresight_exit(void) +{ + etm_perf_exit(); + bus_unregister(&coresight_bustype); +} + +module_init(coresight_init); +module_exit(coresight_exit); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight tracer driver"); diff --git a/drivers/hwtracing/coresight/coresight-cpu-debug.c b/drivers/hwtracing/coresight/coresight-cpu-debug.c index 7e642fb3ed15..e1d232411d8d 100644 --- a/drivers/hwtracing/coresight/coresight-cpu-debug.c +++ b/drivers/hwtracing/coresight/coresight-cpu-debug.c @@ -665,6 +665,8 @@ static const struct amba_id debug_ids[] = { {}, }; +MODULE_DEVICE_TABLE(amba, debug_ids); + static struct amba_driver debug_driver = { .drv = { .name = "coresight-cpu-debug", diff --git a/drivers/hwtracing/coresight/coresight-cti.c b/drivers/hwtracing/coresight/coresight-cti-core.c index 3ccc703dc940..d28eae93e55c 100644 --- a/drivers/hwtracing/coresight/coresight-cti.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -86,22 +86,16 @@ void cti_write_all_hw_regs(struct cti_drvdata *drvdata) CS_LOCK(drvdata->base); } -static void cti_enable_hw_smp_call(void *info) -{ - struct cti_drvdata *drvdata = info; - - cti_write_all_hw_regs(drvdata); -} - /* write regs to hardware and enable */ static int cti_enable_hw(struct cti_drvdata *drvdata) { struct cti_config *config = &drvdata->config; struct device *dev = &drvdata->csdev->dev; + unsigned long flags; int rc = 0; pm_runtime_get_sync(dev->parent); - spin_lock(&drvdata->spinlock); + spin_lock_irqsave(&drvdata->spinlock, flags); /* no need to do anything if enabled or unpowered*/ if (config->hw_enabled || !config->hw_powered) @@ -112,19 +106,11 @@ static int cti_enable_hw(struct cti_drvdata *drvdata) if (rc) goto cti_err_not_enabled; - if (drvdata->ctidev.cpu >= 0) { - rc = smp_call_function_single(drvdata->ctidev.cpu, - cti_enable_hw_smp_call, - drvdata, 1); - if (rc) - goto cti_err_not_enabled; - } else { - cti_write_all_hw_regs(drvdata); - } + cti_write_all_hw_regs(drvdata); config->hw_enabled = true; atomic_inc(&drvdata->config.enable_req_count); - spin_unlock(&drvdata->spinlock); + spin_unlock_irqrestore(&drvdata->spinlock, flags); return rc; cti_state_unchanged: @@ -132,7 +118,7 @@ cti_state_unchanged: /* cannot enable due to error */ cti_err_not_enabled: - spin_unlock(&drvdata->spinlock); + spin_unlock_irqrestore(&drvdata->spinlock, flags); pm_runtime_put(dev->parent); return rc; } @@ -141,9 +127,7 @@ cti_err_not_enabled: static void cti_cpuhp_enable_hw(struct cti_drvdata *drvdata) { struct cti_config *config = &drvdata->config; - struct device *dev = &drvdata->csdev->dev; - pm_runtime_get_sync(dev->parent); spin_lock(&drvdata->spinlock); config->hw_powered = true; @@ -163,7 +147,6 @@ static void cti_cpuhp_enable_hw(struct cti_drvdata *drvdata) /* did not re-enable due to no claim / no request */ cti_hp_not_enabled: spin_unlock(&drvdata->spinlock); - pm_runtime_put(dev->parent); } /* disable hardware */ @@ -511,12 +494,15 @@ static bool cti_add_sysfs_link(struct cti_drvdata *drvdata, return !link_err; } -static void cti_remove_sysfs_link(struct cti_trig_con *tc) +static void cti_remove_sysfs_link(struct cti_drvdata *drvdata, + struct cti_trig_con *tc) { struct coresight_sysfs_link link_info; + link_info.orig = drvdata->csdev; link_info.orig_name = tc->con_dev_name; link_info.target = tc->con_dev; + link_info.target_name = dev_name(&drvdata->csdev->dev); coresight_remove_sysfs_link(&link_info); } @@ -556,7 +542,7 @@ cti_match_fixup_csdev(struct cti_device *ctidev, const char *node_name, * This will set the association if CTI declared before the CS device. * (called from coresight_register() with coresight_mutex locked). */ -void cti_add_assoc_to_csdev(struct coresight_device *csdev) +static void cti_add_assoc_to_csdev(struct coresight_device *csdev) { struct cti_drvdata *ect_item; struct cti_device *ctidev; @@ -589,13 +575,12 @@ void cti_add_assoc_to_csdev(struct coresight_device *csdev) cti_add_done: mutex_unlock(&ect_mutex); } -EXPORT_SYMBOL_GPL(cti_add_assoc_to_csdev); /* * Removing the associated devices is easier. * A CTI will not have a value for csdev->ect_dev. */ -void cti_remove_assoc_from_csdev(struct coresight_device *csdev) +static void cti_remove_assoc_from_csdev(struct coresight_device *csdev) { struct cti_drvdata *ctidrv; struct cti_trig_con *tc; @@ -606,8 +591,8 @@ void cti_remove_assoc_from_csdev(struct coresight_device *csdev) ctidrv = csdev_to_cti_drvdata(csdev->ect_dev); ctidev = &ctidrv->ctidev; list_for_each_entry(tc, &ctidev->trig_cons, node) { - if (tc->con_dev == csdev->ect_dev) { - cti_remove_sysfs_link(tc); + if (tc->con_dev == csdev) { + cti_remove_sysfs_link(ctidrv, tc); tc->con_dev = NULL; break; } @@ -616,7 +601,15 @@ void cti_remove_assoc_from_csdev(struct coresight_device *csdev) } mutex_unlock(&ect_mutex); } -EXPORT_SYMBOL_GPL(cti_remove_assoc_from_csdev); + +/* + * Operations to add and remove associated CTI. + * Register to coresight core driver as call back function. + */ +static struct cti_assoc_op cti_assoc_ops = { + .add = cti_add_assoc_to_csdev, + .remove = cti_remove_assoc_from_csdev +}; /* * Update the cross references where the associated device was found @@ -651,7 +644,7 @@ static void cti_remove_conn_xrefs(struct cti_drvdata *drvdata) if (tc->con_dev) { coresight_set_assoc_ectdev_mutex(tc->con_dev, NULL); - cti_remove_sysfs_link(tc); + cti_remove_sysfs_link(drvdata, tc); tc->con_dev = NULL; } } @@ -742,7 +735,8 @@ static int cti_dying_cpu(unsigned int cpu) spin_lock(&drvdata->spinlock); drvdata->config.hw_powered = false; - coresight_disclaim_device(drvdata->base); + if (drvdata->config.hw_enabled) + coresight_disclaim_device(drvdata->base); spin_unlock(&drvdata->spinlock); return 0; } @@ -828,7 +822,6 @@ static void cti_device_release(struct device *dev) struct cti_drvdata *ect_item, *ect_tmp; mutex_lock(&ect_mutex); - cti_remove_conn_xrefs(drvdata); cti_pm_release(drvdata); /* remove from the list */ @@ -843,6 +836,18 @@ static void cti_device_release(struct device *dev) if (drvdata->csdev_release) drvdata->csdev_release(dev); } +static int __exit cti_remove(struct amba_device *adev) +{ + struct cti_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + mutex_lock(&ect_mutex); + cti_remove_conn_xrefs(drvdata); + mutex_unlock(&ect_mutex); + + coresight_unregister(drvdata->csdev); + + return 0; +} static int cti_probe(struct amba_device *adev, const struct amba_id *id) { @@ -963,6 +968,8 @@ static const struct amba_id cti_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, cti_ids); + static struct amba_driver cti_driver = { .drv = { .name = "coresight-cti", @@ -970,6 +977,30 @@ static struct amba_driver cti_driver = { .suppress_bind_attrs = true, }, .probe = cti_probe, + .remove = cti_remove, .id_table = cti_ids, }; -builtin_amba_driver(cti_driver); + +static int __init cti_init(void) +{ + int ret; + + ret = amba_driver_register(&cti_driver); + if (ret) + pr_info("Error registering cti driver\n"); + coresight_set_cti_ops(&cti_assoc_ops); + return ret; +} + +static void __exit cti_exit(void) +{ + coresight_remove_cti_ops(); + amba_driver_unregister(&cti_driver); +} + +module_init(cti_init); +module_exit(cti_exit); + +MODULE_AUTHOR("Mike Leach <mike.leach@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight CTI Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-etb10.c b/drivers/hwtracing/coresight/coresight-etb10.c index 03e3f2590191..248cc82c838e 100644 --- a/drivers/hwtracing/coresight/coresight-etb10.c +++ b/drivers/hwtracing/coresight/coresight-etb10.c @@ -525,7 +525,7 @@ static unsigned long etb_update_buffer(struct coresight_device *csdev, cur = buf->cur; offset = buf->offset; - barrier = barrier_pkt; + barrier = coresight_barrier_pkt; for (i = 0; i < to_read; i += 4) { buf_ptr = buf->data_pages[cur] + offset; @@ -801,6 +801,21 @@ err_misc_register: return ret; } +static int __exit etb_remove(struct amba_device *adev) +{ + struct etb_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + /* + * Since misc_open() holds a refcount on the f_ops, which is + * etb fops in this case, device is there until last file + * handler to this device is closed. + */ + misc_deregister(&drvdata->miscdev); + coresight_unregister(drvdata->csdev); + + return 0; +} + #ifdef CONFIG_PM static int etb_runtime_suspend(struct device *dev) { @@ -835,6 +850,8 @@ static const struct amba_id etb_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, etb_ids); + static struct amba_driver etb_driver = { .drv = { .name = "coresight-etb10", @@ -844,6 +861,13 @@ static struct amba_driver etb_driver = { }, .probe = etb_probe, + .remove = etb_remove, .id_table = etb_ids, }; -builtin_amba_driver(etb_driver); + +module_amba_driver(etb_driver); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight Embedded Trace Buffer driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c index 1a3169e69bb1..c2c9b127d074 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.c +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c @@ -126,10 +126,10 @@ static void free_sink_buffer(struct etm_event_data *event_data) cpumask_t *mask = &event_data->mask; struct coresight_device *sink; - if (WARN_ON(cpumask_empty(mask))) + if (!event_data->snk_config) return; - if (!event_data->snk_config) + if (WARN_ON(cpumask_empty(mask))) return; cpu = cpumask_first(mask); @@ -222,8 +222,6 @@ static void *etm_setup_aux(struct perf_event *event, void **pages, if (event->attr.config2) { id = (u32)event->attr.config2; sink = coresight_get_sink_by_id(id); - } else { - sink = coresight_get_enabled_sink(true); } mask = &event_data->mask; @@ -321,6 +319,16 @@ static void etm_event_start(struct perf_event *event, int flags) if (!event_data) goto fail; + /* + * Check if this ETM is allowed to trace, as decided + * at etm_setup_aux(). This could be due to an unreachable + * sink from this ETM. We can't do much in this case if + * the sink was specified or hinted to the driver. For + * now, simply don't record anything on this ETM. + */ + if (!cpumask_test_cpu(cpu, &event_data->mask)) + goto fail_end_stop; + path = etm_event_cpu_path(event_data, cpu); /* We need a sink, no need to continue without one */ sink = coresight_get_sink(path); @@ -517,6 +525,7 @@ int etm_perf_symlink(struct coresight_device *csdev, bool link) return 0; } +EXPORT_SYMBOL_GPL(etm_perf_symlink); static ssize_t etm_perf_sink_name_show(struct device *dev, struct device_attribute *dattr, @@ -590,7 +599,7 @@ void etm_perf_del_symlink_sink(struct coresight_device *csdev) csdev->ea = NULL; } -static int __init etm_perf_init(void) +int __init etm_perf_init(void) { int ret; @@ -617,4 +626,8 @@ static int __init etm_perf_init(void) return ret; } -device_initcall(etm_perf_init); + +void __exit etm_perf_exit(void) +{ + perf_pmu_unregister(&etm_pmu); +} diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.h b/drivers/hwtracing/coresight/coresight-etm-perf.h index 015213abe00a..3e4f2ad5e193 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.h +++ b/drivers/hwtracing/coresight/coresight-etm-perf.h @@ -57,7 +57,7 @@ struct etm_event_data { struct list_head * __percpu *path; }; -#ifdef CONFIG_CORESIGHT +#if IS_ENABLED(CONFIG_CORESIGHT) int etm_perf_symlink(struct coresight_device *csdev, bool link); int etm_perf_add_symlink_sink(struct coresight_device *csdev); void etm_perf_del_symlink_sink(struct coresight_device *csdev); @@ -82,4 +82,7 @@ static inline void *etm_perf_sink_config(struct perf_output_handle *handle) #endif /* CONFIG_CORESIGHT */ +int __init etm_perf_init(void); +void __exit etm_perf_exit(void); + #endif diff --git a/drivers/hwtracing/coresight/coresight-etm3x.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index bf22dcfd3327..47f610b1c2b1 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -40,8 +40,6 @@ static int boot_enable; module_param_named(boot_enable, boot_enable, int, S_IRUGO); -/* The number of ETM/PTM currently registered */ -static int etm_count; static struct etm_drvdata *etmdrvdata[NR_CPUS]; static enum cpuhp_state hp_online; @@ -782,6 +780,42 @@ static void etm_init_trace_id(struct etm_drvdata *drvdata) drvdata->traceid = coresight_get_trace_id(drvdata->cpu); } +static int __init etm_hp_setup(void) +{ + int ret; + + ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ARM_CORESIGHT_STARTING, + "arm/coresight:starting", + etm_starting_cpu, etm_dying_cpu); + + if (ret) + return ret; + + ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ONLINE_DYN, + "arm/coresight:online", + etm_online_cpu, NULL); + + /* HP dyn state ID returned in ret on success */ + if (ret > 0) { + hp_online = ret; + return 0; + } + + /* failed dyn state - remove others */ + cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); + + return ret; +} + +static void etm_hp_clear(void) +{ + cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); + if (hp_online) { + cpuhp_remove_state_nocalls(hp_online); + hp_online = 0; + } +} + static int etm_probe(struct amba_device *adev, const struct amba_id *id) { int ret; @@ -823,39 +857,20 @@ static int etm_probe(struct amba_device *adev, const struct amba_id *id) if (!desc.name) return -ENOMEM; - cpus_read_lock(); - etmdrvdata[drvdata->cpu] = drvdata; - if (smp_call_function_single(drvdata->cpu, etm_init_arch_data, drvdata, 1)) dev_err(dev, "ETM arch init failed\n"); - if (!etm_count++) { - cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ARM_CORESIGHT_STARTING, - "arm/coresight:starting", - etm_starting_cpu, etm_dying_cpu); - ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ONLINE_DYN, - "arm/coresight:online", - etm_online_cpu, NULL); - if (ret < 0) - goto err_arch_supported; - hp_online = ret; - } - cpus_read_unlock(); - - if (etm_arch_supported(drvdata->arch) == false) { - ret = -EINVAL; - goto err_arch_supported; - } + if (etm_arch_supported(drvdata->arch) == false) + return -EINVAL; etm_init_trace_id(drvdata); etm_set_default(&drvdata->config); pdata = coresight_get_platform_data(dev); - if (IS_ERR(pdata)) { - ret = PTR_ERR(pdata); - goto err_arch_supported; - } + if (IS_ERR(pdata)) + return PTR_ERR(pdata); + adev->dev.platform_data = pdata; desc.type = CORESIGHT_DEV_TYPE_SOURCE; @@ -865,17 +880,17 @@ static int etm_probe(struct amba_device *adev, const struct amba_id *id) desc.dev = dev; desc.groups = coresight_etm_groups; drvdata->csdev = coresight_register(&desc); - if (IS_ERR(drvdata->csdev)) { - ret = PTR_ERR(drvdata->csdev); - goto err_arch_supported; - } + if (IS_ERR(drvdata->csdev)) + return PTR_ERR(drvdata->csdev); ret = etm_perf_symlink(drvdata->csdev, true); if (ret) { coresight_unregister(drvdata->csdev); - goto err_arch_supported; + return ret; } + etmdrvdata[drvdata->cpu] = drvdata; + pm_runtime_put(&adev->dev); dev_info(&drvdata->csdev->dev, "%s initialized\n", (char *)coresight_get_uci_data(id)); @@ -885,14 +900,40 @@ static int etm_probe(struct amba_device *adev, const struct amba_id *id) } return 0; +} -err_arch_supported: - if (--etm_count == 0) { - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); - if (hp_online) - cpuhp_remove_state_nocalls(hp_online); - } - return ret; +static void __exit clear_etmdrvdata(void *info) +{ + int cpu = *(int *)info; + + etmdrvdata[cpu] = NULL; +} + +static int __exit etm_remove(struct amba_device *adev) +{ + struct etm_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + etm_perf_symlink(drvdata->csdev, false); + + /* + * Taking hotplug lock here to avoid racing between etm_remove and + * CPU hotplug call backs. + */ + cpus_read_lock(); + /* + * The readers for etmdrvdata[] are CPU hotplug call backs + * and PM notification call backs. Change etmdrvdata[i] on + * CPU i ensures these call backs has consistent view + * inside one call back function. + */ + if (smp_call_function_single(drvdata->cpu, clear_etmdrvdata, &drvdata->cpu, 1)) + etmdrvdata[drvdata->cpu] = NULL; + + cpus_read_unlock(); + + coresight_unregister(drvdata->csdev); + + return 0; } #ifdef CONFIG_PM @@ -937,6 +978,8 @@ static const struct amba_id etm_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, etm_ids); + static struct amba_driver etm_driver = { .drv = { .name = "coresight-etm3x", @@ -945,6 +988,39 @@ static struct amba_driver etm_driver = { .suppress_bind_attrs = true, }, .probe = etm_probe, + .remove = etm_remove, .id_table = etm_ids, }; -builtin_amba_driver(etm_driver); + +static int __init etm_init(void) +{ + int ret; + + ret = etm_hp_setup(); + + /* etm_hp_setup() does its own cleanup - exit on error */ + if (ret) + return ret; + + ret = amba_driver_register(&etm_driver); + if (ret) { + pr_err("Error registering etm3x driver\n"); + etm_hp_clear(); + } + + return ret; +} + +static void __exit etm_exit(void) +{ + amba_driver_unregister(&etm_driver); + etm_hp_clear(); +} + +module_init(etm_init); +module_exit(etm_exit); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight Program Flow Trace driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 96425e818fc2..abd706b216ac 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -48,12 +48,11 @@ module_param(pm_save_enable, int, 0444); MODULE_PARM_DESC(pm_save_enable, "Save/restore state on power down: 1 = never, 2 = self-hosted"); -/* The number of ETMv4 currently registered */ -static int etm4_count; static struct etmv4_drvdata *etmdrvdata[NR_CPUS]; static void etm4_set_default_config(struct etmv4_config *config); static int etm4_set_event_filters(struct etmv4_drvdata *drvdata, struct perf_event *event); +static u64 etm4_get_access_type(struct etmv4_config *config); static enum cpuhp_state hp_online; @@ -743,8 +742,14 @@ static void etm4_init_arch_data(void *info) * The number of resource pairs conveyed by the HW starts at 0, i.e a * value of 0x0 indicate 1 resource pair, 0x1 indicate two and so on. * As such add 1 to the value of NUMRSPAIR for a better representation. + * + * For ETM v4.3 and later, 0x0 means 0, and no pairs are available - + * the default TRUE and FALSE resource selectors are omitted. + * Otherwise for values 0x1 and above the number is N + 1 as per v4.2. */ - drvdata->nr_resource = BMVAL(etmidr4, 16, 19) + 1; + drvdata->nr_resource = BMVAL(etmidr4, 16, 19); + if ((drvdata->arch < ETM4X_ARCH_4V3) || (drvdata->nr_resource > 0)) + drvdata->nr_resource += 1; /* * NUMSSCC, bits[23:20] the number of single-shot * comparator control for tracing. Read any status regs as these @@ -785,6 +790,22 @@ static void etm4_init_arch_data(void *info) CS_LOCK(drvdata->base); } +/* Set ELx trace filter access in the TRCVICTLR register */ +static void etm4_set_victlr_access(struct etmv4_config *config) +{ + u64 access_type; + + config->vinst_ctrl &= ~(ETM_EXLEVEL_S_VICTLR_MASK | ETM_EXLEVEL_NS_VICTLR_MASK); + + /* + * TRCVICTLR::EXLEVEL_NS:EXLEVELS: Set kernel / user filtering + * bits in vinst_ctrl, same bit pattern as TRCACATRn values returned by + * etm4_get_access_type() but with a relative shift in this register. + */ + access_type = etm4_get_access_type(config) << ETM_EXLEVEL_LSHIFT_TRCVICTLR; + config->vinst_ctrl |= (u32)access_type; +} + static void etm4_set_default_config(struct etmv4_config *config) { /* disable all events tracing */ @@ -802,6 +823,9 @@ static void etm4_set_default_config(struct etmv4_config *config) /* TRCVICTLR::EVENT = 0x01, select the always on logic */ config->vinst_ctrl = BIT(0); + + /* TRCVICTLR::EXLEVEL_NS:EXLEVELS: Set kernel / user filtering */ + etm4_set_victlr_access(config); } static u64 etm4_get_ns_access_type(struct etmv4_config *config) @@ -1066,7 +1090,7 @@ out: void etm4_config_trace_mode(struct etmv4_config *config) { - u32 addr_acc, mode; + u32 mode; mode = config->mode; mode &= (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER); @@ -1078,15 +1102,7 @@ void etm4_config_trace_mode(struct etmv4_config *config) if (!(mode & ETM_MODE_EXCL_KERN) && !(mode & ETM_MODE_EXCL_USER)) return; - addr_acc = config->addr_acc[ETM_DEFAULT_ADDR_COMP]; - /* clear default config */ - addr_acc &= ~(ETM_EXLEVEL_NS_APP | ETM_EXLEVEL_NS_OS | - ETM_EXLEVEL_NS_HYP); - - addr_acc |= etm4_get_ns_access_type(config); - - config->addr_acc[ETM_DEFAULT_ADDR_COMP] = addr_acc; - config->addr_acc[ETM_DEFAULT_ADDR_COMP + 1] = addr_acc; + etm4_set_victlr_access(config); } static int etm4_online_cpu(unsigned int cpu) @@ -1183,7 +1199,7 @@ static int etm4_cpu_save(struct etmv4_drvdata *drvdata) state->trcvdsacctlr = readl(drvdata->base + TRCVDSACCTLR); state->trcvdarcctlr = readl(drvdata->base + TRCVDARCCTLR); - for (i = 0; i < drvdata->nrseqstate; i++) + for (i = 0; i < drvdata->nrseqstate - 1; i++) state->trcseqevr[i] = readl(drvdata->base + TRCSEQEVRn(i)); state->trcseqrstevr = readl(drvdata->base + TRCSEQRSTEVR); @@ -1227,7 +1243,7 @@ static int etm4_cpu_save(struct etmv4_drvdata *drvdata) state->trccidcctlr1 = readl(drvdata->base + TRCCIDCCTLR1); state->trcvmidcctlr0 = readl(drvdata->base + TRCVMIDCCTLR0); - state->trcvmidcctlr0 = readl(drvdata->base + TRCVMIDCCTLR1); + state->trcvmidcctlr1 = readl(drvdata->base + TRCVMIDCCTLR1); state->trcclaimset = readl(drvdata->base + TRCCLAIMCLR); @@ -1288,7 +1304,7 @@ static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) writel_relaxed(state->trcvdsacctlr, drvdata->base + TRCVDSACCTLR); writel_relaxed(state->trcvdarcctlr, drvdata->base + TRCVDARCCTLR); - for (i = 0; i < drvdata->nrseqstate; i++) + for (i = 0; i < drvdata->nrseqstate - 1; i++) writel_relaxed(state->trcseqevr[i], drvdata->base + TRCSEQEVRn(i)); @@ -1337,7 +1353,7 @@ static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) writel_relaxed(state->trccidcctlr1, drvdata->base + TRCCIDCCTLR1); writel_relaxed(state->trcvmidcctlr0, drvdata->base + TRCVMIDCCTLR0); - writel_relaxed(state->trcvmidcctlr0, drvdata->base + TRCVMIDCCTLR1); + writel_relaxed(state->trcvmidcctlr1, drvdata->base + TRCVMIDCCTLR1); writel_relaxed(state->trcclaimset, drvdata->base + TRCCLAIMSET); @@ -1397,28 +1413,25 @@ static struct notifier_block etm4_cpu_pm_nb = { .notifier_call = etm4_cpu_pm_notify, }; -/* Setup PM. Called with cpus locked. Deals with error conditions and counts */ -static int etm4_pm_setup_cpuslocked(void) +/* Setup PM. Deals with error conditions and counts */ +static int __init etm4_pm_setup(void) { int ret; - if (etm4_count++) - return 0; - ret = cpu_pm_register_notifier(&etm4_cpu_pm_nb); if (ret) - goto reduce_count; + return ret; - ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ARM_CORESIGHT_STARTING, - "arm/coresight4:starting", - etm4_starting_cpu, etm4_dying_cpu); + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, + "arm/coresight4:starting", + etm4_starting_cpu, etm4_dying_cpu); if (ret) goto unregister_notifier; - ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ONLINE_DYN, - "arm/coresight4:online", - etm4_online_cpu, NULL); + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, + "arm/coresight4:online", + etm4_online_cpu, NULL); /* HP dyn state ID returned in ret on success */ if (ret > 0) { @@ -1427,21 +1440,15 @@ static int etm4_pm_setup_cpuslocked(void) } /* failed dyn state - remove others */ - cpuhp_remove_state_nocalls_cpuslocked(CPUHP_AP_ARM_CORESIGHT_STARTING); + cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); unregister_notifier: cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); - -reduce_count: - --etm4_count; return ret; } static void etm4_pm_clear(void) { - if (--etm4_count != 0) - return; - cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { @@ -1497,35 +1504,20 @@ static int etm4_probe(struct amba_device *adev, const struct amba_id *id) if (!desc.name) return -ENOMEM; - cpus_read_lock(); - etmdrvdata[drvdata->cpu] = drvdata; - if (smp_call_function_single(drvdata->cpu, etm4_init_arch_data, drvdata, 1)) dev_err(dev, "ETM arch init failed\n"); - ret = etm4_pm_setup_cpuslocked(); - cpus_read_unlock(); - - /* etm4_pm_setup_cpuslocked() does its own cleanup - exit on error */ - if (ret) { - etmdrvdata[drvdata->cpu] = NULL; - return ret; - } - - if (etm4_arch_supported(drvdata->arch) == false) { - ret = -EINVAL; - goto err_arch_supported; - } + if (etm4_arch_supported(drvdata->arch) == false) + return -EINVAL; etm4_init_trace_id(drvdata); etm4_set_default(&drvdata->config); pdata = coresight_get_platform_data(dev); - if (IS_ERR(pdata)) { - ret = PTR_ERR(pdata); - goto err_arch_supported; - } + if (IS_ERR(pdata)) + return PTR_ERR(pdata); + adev->dev.platform_data = pdata; desc.type = CORESIGHT_DEV_TYPE_SOURCE; @@ -1535,17 +1527,17 @@ static int etm4_probe(struct amba_device *adev, const struct amba_id *id) desc.dev = dev; desc.groups = coresight_etmv4_groups; drvdata->csdev = coresight_register(&desc); - if (IS_ERR(drvdata->csdev)) { - ret = PTR_ERR(drvdata->csdev); - goto err_arch_supported; - } + if (IS_ERR(drvdata->csdev)) + return PTR_ERR(drvdata->csdev); ret = etm_perf_symlink(drvdata->csdev, true); if (ret) { coresight_unregister(drvdata->csdev); - goto err_arch_supported; + return ret; } + etmdrvdata[drvdata->cpu] = drvdata; + pm_runtime_put(&adev->dev); dev_info(&drvdata->csdev->dev, "CPU%d: ETM v%d.%d initialized\n", drvdata->cpu, drvdata->arch >> 4, drvdata->arch & 0xf); @@ -1556,11 +1548,6 @@ static int etm4_probe(struct amba_device *adev, const struct amba_id *id) } return 0; - -err_arch_supported: - etmdrvdata[drvdata->cpu] = NULL; - etm4_pm_clear(); - return ret; } static struct amba_cs_uci_id uci_id_etm4[] = { @@ -1572,6 +1559,40 @@ static struct amba_cs_uci_id uci_id_etm4[] = { } }; +static void __exit clear_etmdrvdata(void *info) +{ + int cpu = *(int *)info; + + etmdrvdata[cpu] = NULL; +} + +static int __exit etm4_remove(struct amba_device *adev) +{ + struct etmv4_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + etm_perf_symlink(drvdata->csdev, false); + + /* + * Taking hotplug lock here to avoid racing between etm4_remove and + * CPU hotplug call backs. + */ + cpus_read_lock(); + /* + * The readers for etmdrvdata[] are CPU hotplug call backs + * and PM notification call backs. Change etmdrvdata[i] on + * CPU i ensures these call backs has consistent view + * inside one call back function. + */ + if (smp_call_function_single(drvdata->cpu, clear_etmdrvdata, &drvdata->cpu, 1)) + etmdrvdata[drvdata->cpu] = NULL; + + cpus_read_unlock(); + + coresight_unregister(drvdata->csdev); + + return 0; +} + static const struct amba_id etm4_ids[] = { CS_AMBA_ID(0x000bb95d), /* Cortex-A53 */ CS_AMBA_ID(0x000bb95e), /* Cortex-A57 */ @@ -1586,15 +1607,53 @@ static const struct amba_id etm4_ids[] = { CS_AMBA_UCI_ID(0x000bb805, uci_id_etm4),/* Qualcomm Kryo 4XX Cortex-A55 */ CS_AMBA_UCI_ID(0x000bb804, uci_id_etm4),/* Qualcomm Kryo 4XX Cortex-A76 */ CS_AMBA_UCI_ID(0x000cc0af, uci_id_etm4),/* Marvell ThunderX2 */ + CS_AMBA_UCI_ID(0x000b6d01, uci_id_etm4),/* HiSilicon-Hip08 */ + CS_AMBA_UCI_ID(0x000b6d02, uci_id_etm4),/* HiSilicon-Hip09 */ {}, }; +MODULE_DEVICE_TABLE(amba, etm4_ids); + static struct amba_driver etm4x_driver = { .drv = { .name = "coresight-etm4x", + .owner = THIS_MODULE, .suppress_bind_attrs = true, }, .probe = etm4_probe, + .remove = etm4_remove, .id_table = etm4_ids, }; -builtin_amba_driver(etm4x_driver); + +static int __init etm4x_init(void) +{ + int ret; + + ret = etm4_pm_setup(); + + /* etm4_pm_setup() does its own cleanup - exit on error */ + if (ret) + return ret; + + ret = amba_driver_register(&etm4x_driver); + if (ret) { + pr_err("Error registering etm4x driver\n"); + etm4_pm_clear(); + } + + return ret; +} + +static void __exit etm4x_exit(void) +{ + amba_driver_unregister(&etm4x_driver); + etm4_pm_clear(); +} + +module_init(etm4x_init); +module_exit(etm4x_exit); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight Program Flow Trace v4.x driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c index b673e738bc9a..989ce7b8ade7 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c @@ -206,7 +206,7 @@ static ssize_t reset_store(struct device *dev, * each trace run. */ config->vinst_ctrl = BIT(0); - if (drvdata->nr_addr_cmp == true) { + if (drvdata->nr_addr_cmp > 0) { config->mode |= ETM_MODE_VIEWINST_STARTSTOP; /* SSSTATUS, bit[9] */ config->vinst_ctrl |= BIT(9); @@ -236,7 +236,7 @@ static ssize_t reset_store(struct device *dev, } config->res_idx = 0x0; - for (i = 0; i < drvdata->nr_resource; i++) + for (i = 2; i < 2 * drvdata->nr_resource; i++) config->res_ctrl[i] = 0x0; config->ss_idx = 0x0; @@ -1663,8 +1663,11 @@ static ssize_t res_idx_store(struct device *dev, if (kstrtoul(buf, 16, &val)) return -EINVAL; - /* Resource selector pair 0 is always implemented and reserved */ - if ((val == 0) || (val >= drvdata->nr_resource)) + /* + * Resource selector pair 0 is always implemented and reserved, + * namely an idx with 0 and 1 is illegal. + */ + if ((val < 2) || (val >= 2 * drvdata->nr_resource)) return -EINVAL; /* diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index b8283e1d6d88..eefc7371c6c4 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -192,11 +192,17 @@ #define ETM_EXLEVEL_NS_HYP BIT(14) #define ETM_EXLEVEL_NS_NA BIT(15) +/* access level control in TRCVICTLR - same bits as TRCACATRn but shifted */ +#define ETM_EXLEVEL_LSHIFT_TRCVICTLR 8 + /* secure / non secure masks - TRCVICTLR, IDR3 */ #define ETM_EXLEVEL_S_VICTLR_MASK GENMASK(19, 16) /* NS MON (EL3) mode never implemented */ #define ETM_EXLEVEL_NS_VICTLR_MASK GENMASK(22, 20) +/* Interpretation of resource numbers change at ETM v4.3 architecture */ +#define ETM4X_ARCH_4V3 0x43 + /** * struct etmv4_config - configuration information related to an ETMv4 * @mode: Controls various modes supported by this ETM. diff --git a/drivers/hwtracing/coresight/coresight-funnel.c b/drivers/hwtracing/coresight/coresight-funnel.c index 900690a9f7f0..af40814ce560 100644 --- a/drivers/hwtracing/coresight/coresight-funnel.c +++ b/drivers/hwtracing/coresight/coresight-funnel.c @@ -274,6 +274,15 @@ out_disable_clk: return ret; } +static int __exit funnel_remove(struct device *dev) +{ + struct funnel_drvdata *drvdata = dev_get_drvdata(dev); + + coresight_unregister(drvdata->csdev); + + return 0; +} + #ifdef CONFIG_PM static int funnel_runtime_suspend(struct device *dev) { @@ -319,29 +328,41 @@ static int static_funnel_probe(struct platform_device *pdev) return ret; } +static int __exit static_funnel_remove(struct platform_device *pdev) +{ + funnel_remove(&pdev->dev); + pm_runtime_disable(&pdev->dev); + return 0; +} + static const struct of_device_id static_funnel_match[] = { {.compatible = "arm,coresight-static-funnel"}, {} }; +MODULE_DEVICE_TABLE(of, static_funnel_match); + #ifdef CONFIG_ACPI static const struct acpi_device_id static_funnel_ids[] = { {"ARMHC9FE", 0}, {}, }; + +MODULE_DEVICE_TABLE(acpi, static_funnel_ids); #endif static struct platform_driver static_funnel_driver = { .probe = static_funnel_probe, + .remove = static_funnel_remove, .driver = { .name = "coresight-static-funnel", + .owner = THIS_MODULE, .of_match_table = static_funnel_match, .acpi_match_table = ACPI_PTR(static_funnel_ids), .pm = &funnel_dev_pm_ops, .suppress_bind_attrs = true, }, }; -builtin_platform_driver(static_funnel_driver); static int dynamic_funnel_probe(struct amba_device *adev, const struct amba_id *id) @@ -349,6 +370,11 @@ static int dynamic_funnel_probe(struct amba_device *adev, return funnel_probe(&adev->dev, &adev->res); } +static int __exit dynamic_funnel_remove(struct amba_device *adev) +{ + return funnel_remove(&adev->dev); +} + static const struct amba_id dynamic_funnel_ids[] = { { .id = 0x000bb908, @@ -362,6 +388,8 @@ static const struct amba_id dynamic_funnel_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, dynamic_funnel_ids); + static struct amba_driver dynamic_funnel_driver = { .drv = { .name = "coresight-dynamic-funnel", @@ -370,6 +398,39 @@ static struct amba_driver dynamic_funnel_driver = { .suppress_bind_attrs = true, }, .probe = dynamic_funnel_probe, + .remove = dynamic_funnel_remove, .id_table = dynamic_funnel_ids, }; -builtin_amba_driver(dynamic_funnel_driver); + +static int __init funnel_init(void) +{ + int ret; + + ret = platform_driver_register(&static_funnel_driver); + if (ret) { + pr_info("Error registering platform driver\n"); + return ret; + } + + ret = amba_driver_register(&dynamic_funnel_driver); + if (ret) { + pr_info("Error registering amba driver\n"); + platform_driver_unregister(&static_funnel_driver); + } + + return ret; +} + +static void __exit funnel_exit(void) +{ + platform_driver_unregister(&static_funnel_driver); + amba_driver_unregister(&dynamic_funnel_driver); +} + +module_init(funnel_init); +module_exit(funnel_exit); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight Funnel Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-platform.c b/drivers/hwtracing/coresight/coresight-platform.c index bfd44231d7ad..3629b7885aca 100644 --- a/drivers/hwtracing/coresight/coresight-platform.c +++ b/drivers/hwtracing/coresight/coresight-platform.c @@ -75,6 +75,7 @@ coresight_find_csdev_by_fwnode(struct fwnode_handle *r_fwnode) } return csdev; } +EXPORT_SYMBOL_GPL(coresight_find_csdev_by_fwnode); #ifdef CONFIG_OF static inline bool of_coresight_legacy_ep_is_input(struct device_node *ep) @@ -711,11 +712,11 @@ static int acpi_coresight_parse_graph(struct acpi_device *adev, return dir; if (dir == ACPI_CORESIGHT_LINK_MASTER) { - if (ptr->outport > pdata->nr_outport) - pdata->nr_outport = ptr->outport; + if (ptr->outport >= pdata->nr_outport) + pdata->nr_outport = ptr->outport + 1; ptr++; } else { - WARN_ON(pdata->nr_inport == ptr->child_port); + WARN_ON(pdata->nr_inport == ptr->child_port + 1); /* * We do not track input port connections for a device. * However we need the highest port number described, @@ -723,8 +724,8 @@ static int acpi_coresight_parse_graph(struct acpi_device *adev, * record for an output connection. Hence, do not move * the ptr for input connections */ - if (ptr->child_port > pdata->nr_inport) - pdata->nr_inport = ptr->child_port; + if (ptr->child_port >= pdata->nr_inport) + pdata->nr_inport = ptr->child_port + 1; } } diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h index f2dc625ea585..65a29293b6cb 100644 --- a/drivers/hwtracing/coresight/coresight-priv.h +++ b/drivers/hwtracing/coresight/coresight-priv.h @@ -66,8 +66,8 @@ static DEVICE_ATTR_RO(name) #define coresight_simple_reg64(type, name, lo_off, hi_off) \ __coresight_simple_func(type, NULL, name, lo_off, hi_off) -extern const u32 barrier_pkt[4]; -#define CORESIGHT_BARRIER_PKT_SIZE (sizeof(barrier_pkt)) +extern const u32 coresight_barrier_pkt[4]; +#define CORESIGHT_BARRIER_PKT_SIZE (sizeof(coresight_barrier_pkt)) enum etm_addr_type { ETM_ADDR_TYPE_NONE, @@ -104,10 +104,9 @@ struct cs_buffers { static inline void coresight_insert_barrier_packet(void *buf) { if (buf) - memcpy(buf, barrier_pkt, CORESIGHT_BARRIER_PKT_SIZE); + memcpy(buf, coresight_barrier_pkt, CORESIGHT_BARRIER_PKT_SIZE); } - static inline void CS_LOCK(void __iomem *addr) { do { @@ -148,7 +147,8 @@ static inline void coresight_write_reg_pair(void __iomem *addr, u64 val, void coresight_disable_path(struct list_head *path); int coresight_enable_path(struct list_head *path, u32 mode, void *sink_data); struct coresight_device *coresight_get_sink(struct list_head *path); -struct coresight_device *coresight_get_enabled_sink(bool reset); +struct coresight_device * +coresight_get_enabled_sink(struct coresight_device *source); struct coresight_device *coresight_get_sink_by_id(u32 id); struct coresight_device * coresight_find_default_sink(struct coresight_device *csdev); @@ -165,7 +165,7 @@ int coresight_make_links(struct coresight_device *orig, void coresight_remove_links(struct coresight_device *orig, struct coresight_connection *conn); -#ifdef CONFIG_CORESIGHT_SOURCE_ETM3X +#if IS_ENABLED(CONFIG_CORESIGHT_SOURCE_ETM3X) extern int etm_readl_cp14(u32 off, unsigned int *val); extern int etm_writel_cp14(u32 off, u32 val); #else @@ -173,15 +173,13 @@ static inline int etm_readl_cp14(u32 off, unsigned int *val) { return 0; } static inline int etm_writel_cp14(u32 off, u32 val) { return 0; } #endif -#ifdef CONFIG_CORESIGHT_CTI -extern void cti_add_assoc_to_csdev(struct coresight_device *csdev); -extern void cti_remove_assoc_from_csdev(struct coresight_device *csdev); +struct cti_assoc_op { + void (*add)(struct coresight_device *csdev); + void (*remove)(struct coresight_device *csdev); +}; -#else -static inline void cti_add_assoc_to_csdev(struct coresight_device *csdev) {} -static inline void -cti_remove_assoc_from_csdev(struct coresight_device *csdev) {} -#endif +extern void coresight_set_cti_ops(const struct cti_assoc_op *cti_op); +extern void coresight_remove_cti_ops(void); /* * Macros and inline functions to handle CoreSight UCI data and driver diff --git a/drivers/hwtracing/coresight/coresight-replicator.c b/drivers/hwtracing/coresight/coresight-replicator.c index 78acf29c49ca..62afdde0e5ea 100644 --- a/drivers/hwtracing/coresight/coresight-replicator.c +++ b/drivers/hwtracing/coresight/coresight-replicator.c @@ -291,6 +291,14 @@ out_disable_clk: return ret; } +static int __exit replicator_remove(struct device *dev) +{ + struct replicator_drvdata *drvdata = dev_get_drvdata(dev); + + coresight_unregister(drvdata->csdev); + return 0; +} + static int static_replicator_probe(struct platform_device *pdev) { int ret; @@ -310,6 +318,13 @@ static int static_replicator_probe(struct platform_device *pdev) return ret; } +static int __exit static_replicator_remove(struct platform_device *pdev) +{ + replicator_remove(&pdev->dev); + pm_runtime_disable(&pdev->dev); + return 0; +} + #ifdef CONFIG_PM static int replicator_runtime_suspend(struct device *dev) { @@ -343,24 +358,29 @@ static const struct of_device_id static_replicator_match[] = { {} }; +MODULE_DEVICE_TABLE(of, static_replicator_match); + #ifdef CONFIG_ACPI static const struct acpi_device_id static_replicator_acpi_ids[] = { {"ARMHC985", 0}, /* ARM CoreSight Static Replicator */ {} }; + +MODULE_DEVICE_TABLE(acpi, static_replicator_acpi_ids); #endif static struct platform_driver static_replicator_driver = { .probe = static_replicator_probe, + .remove = static_replicator_remove, .driver = { .name = "coresight-static-replicator", + .owner = THIS_MODULE, .of_match_table = of_match_ptr(static_replicator_match), .acpi_match_table = ACPI_PTR(static_replicator_acpi_ids), .pm = &replicator_dev_pm_ops, .suppress_bind_attrs = true, }, }; -builtin_platform_driver(static_replicator_driver); static int dynamic_replicator_probe(struct amba_device *adev, const struct amba_id *id) @@ -368,19 +388,60 @@ static int dynamic_replicator_probe(struct amba_device *adev, return replicator_probe(&adev->dev, &adev->res); } +static int __exit dynamic_replicator_remove(struct amba_device *adev) +{ + return replicator_remove(&adev->dev); +} + static const struct amba_id dynamic_replicator_ids[] = { CS_AMBA_ID(0x000bb909), CS_AMBA_ID(0x000bb9ec), /* Coresight SoC-600 */ {}, }; +MODULE_DEVICE_TABLE(amba, dynamic_replicator_ids); + static struct amba_driver dynamic_replicator_driver = { .drv = { .name = "coresight-dynamic-replicator", .pm = &replicator_dev_pm_ops, + .owner = THIS_MODULE, .suppress_bind_attrs = true, }, .probe = dynamic_replicator_probe, + .remove = dynamic_replicator_remove, .id_table = dynamic_replicator_ids, }; -builtin_amba_driver(dynamic_replicator_driver); + +static int __init replicator_init(void) +{ + int ret; + + ret = platform_driver_register(&static_replicator_driver); + if (ret) { + pr_info("Error registering platform driver\n"); + return ret; + } + + ret = amba_driver_register(&dynamic_replicator_driver); + if (ret) { + pr_info("Error registering amba driver\n"); + platform_driver_unregister(&static_replicator_driver); + } + + return ret; +} + +static void __exit replicator_exit(void) +{ + platform_driver_unregister(&static_replicator_driver); + amba_driver_unregister(&dynamic_replicator_driver); +} + +module_init(replicator_init); +module_exit(replicator_exit); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight Replicator Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-stm.c b/drivers/hwtracing/coresight/coresight-stm.c index 673d2f56ed1e..b0ad912651a9 100644 --- a/drivers/hwtracing/coresight/coresight-stm.c +++ b/drivers/hwtracing/coresight/coresight-stm.c @@ -412,6 +412,7 @@ static ssize_t notrace stm_generic_packet(struct stm_data *stm_data, void __iomem *ch_addr; struct stm_drvdata *drvdata = container_of(stm_data, struct stm_drvdata, stm); + unsigned int stm_flags; if (!(drvdata && local_read(&drvdata->mode))) return -EACCES; @@ -421,8 +422,9 @@ static ssize_t notrace stm_generic_packet(struct stm_data *stm_data, ch_addr = stm_channel_addr(drvdata, channel); - flags = (flags == STP_PACKET_TIMESTAMPED) ? STM_FLAG_TIMESTAMPED : 0; - flags |= test_bit(channel, drvdata->chs.guaranteed) ? + stm_flags = (flags & STP_PACKET_TIMESTAMPED) ? + STM_FLAG_TIMESTAMPED : 0; + stm_flags |= test_bit(channel, drvdata->chs.guaranteed) ? STM_FLAG_GUARANTEED : 0; if (size > drvdata->write_bytes) @@ -432,7 +434,7 @@ static ssize_t notrace stm_generic_packet(struct stm_data *stm_data, switch (packet) { case STP_PACKET_FLAG: - ch_addr += stm_channel_off(STM_PKT_TYPE_FLAG, flags); + ch_addr += stm_channel_off(STM_PKT_TYPE_FLAG, stm_flags); /* * The generic STM core sets a size of '0' on flag packets. @@ -444,7 +446,8 @@ static ssize_t notrace stm_generic_packet(struct stm_data *stm_data, break; case STP_PACKET_DATA: - ch_addr += stm_channel_off(STM_PKT_TYPE_DATA, flags); + stm_flags |= (flags & STP_PACKET_MARKED) ? STM_FLAG_MARKED : 0; + ch_addr += stm_channel_off(STM_PKT_TYPE_DATA, stm_flags); stm_send(ch_addr, payload, size, drvdata->write_bytes); break; @@ -948,6 +951,17 @@ stm_unregister: return ret; } +static int __exit stm_remove(struct amba_device *adev) +{ + struct stm_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + coresight_unregister(drvdata->csdev); + + stm_unregister_device(&drvdata->stm); + + return 0; +} + #ifdef CONFIG_PM static int stm_runtime_suspend(struct device *dev) { @@ -980,6 +994,8 @@ static const struct amba_id stm_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, stm_ids); + static struct amba_driver stm_driver = { .drv = { .name = "coresight-stm", @@ -988,7 +1004,12 @@ static struct amba_driver stm_driver = { .suppress_bind_attrs = true, }, .probe = stm_probe, + .remove = stm_remove, .id_table = stm_ids, }; -builtin_amba_driver(stm_driver); +module_amba_driver(stm_driver); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_DESCRIPTION("Arm CoreSight System Trace Macrocell driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index 82afeaf2ccc4..34d2a2d31d00 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -102,6 +102,7 @@ int coresight_add_sysfs_link(struct coresight_sysfs_link *info) return ret; } +EXPORT_SYMBOL_GPL(coresight_add_sysfs_link); void coresight_remove_sysfs_link(struct coresight_sysfs_link *info) { @@ -122,6 +123,7 @@ void coresight_remove_sysfs_link(struct coresight_sysfs_link *info) info->orig->nr_links--; info->target->nr_links--; } +EXPORT_SYMBOL_GPL(coresight_remove_sysfs_link); /* * coresight_make_links: Make a link for a connection from a @orig diff --git a/drivers/hwtracing/coresight/coresight-tmc.c b/drivers/hwtracing/coresight/coresight-tmc-core.c index 9ca3aaafcfbc..5653e0945c74 100644 --- a/drivers/hwtracing/coresight/coresight-tmc.c +++ b/drivers/hwtracing/coresight/coresight-tmc-core.c @@ -559,6 +559,21 @@ out: spin_unlock_irqrestore(&drvdata->spinlock, flags); } +static int __exit tmc_remove(struct amba_device *adev) +{ + struct tmc_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + /* + * Since misc_open() holds a refcount on the f_ops, which is + * etb fops in this case, device is there until last file + * handler to this device is closed. + */ + misc_deregister(&drvdata->miscdev); + coresight_unregister(drvdata->csdev); + + return 0; +} + static const struct amba_id tmc_ids[] = { CS_AMBA_ID(0x000bb961), /* Coresight SoC 600 TMC-ETR/ETS */ @@ -570,6 +585,8 @@ static const struct amba_id tmc_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, tmc_ids); + static struct amba_driver tmc_driver = { .drv = { .name = "coresight-tmc", @@ -578,6 +595,12 @@ static struct amba_driver tmc_driver = { }, .probe = tmc_probe, .shutdown = tmc_shutdown, + .remove = tmc_remove, .id_table = tmc_ids, }; -builtin_amba_driver(tmc_driver); + +module_amba_driver(tmc_driver); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_DESCRIPTION("Arm CoreSight Trace Memory Controller driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/coresight/coresight-tmc-etf.c b/drivers/hwtracing/coresight/coresight-tmc-etf.c index 6375504ba8b0..44402d413ebb 100644 --- a/drivers/hwtracing/coresight/coresight-tmc-etf.c +++ b/drivers/hwtracing/coresight/coresight-tmc-etf.c @@ -519,7 +519,7 @@ static unsigned long tmc_update_etf_buffer(struct coresight_device *csdev, cur = buf->cur; offset = buf->offset; - barrier = barrier_pkt; + barrier = coresight_barrier_pkt; /* for every byte to read */ for (i = 0; i < to_read; i += 4) { diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c index b29c2db94d96..714f9e867e5f 100644 --- a/drivers/hwtracing/coresight/coresight-tmc-etr.c +++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c @@ -255,6 +255,7 @@ void tmc_free_sg_table(struct tmc_sg_table *sg_table) tmc_free_table_pages(sg_table); tmc_free_data_pages(sg_table); } +EXPORT_SYMBOL_GPL(tmc_free_sg_table); /* * Alloc pages for the table. Since this will be used by the device, @@ -340,6 +341,7 @@ struct tmc_sg_table *tmc_alloc_sg_table(struct device *dev, return sg_table; } +EXPORT_SYMBOL_GPL(tmc_alloc_sg_table); /* * tmc_sg_table_sync_data_range: Sync the data buffer written @@ -360,6 +362,7 @@ void tmc_sg_table_sync_data_range(struct tmc_sg_table *table, PAGE_SIZE, DMA_FROM_DEVICE); } } +EXPORT_SYMBOL_GPL(tmc_sg_table_sync_data_range); /* tmc_sg_sync_table: Sync the page table */ void tmc_sg_table_sync_table(struct tmc_sg_table *sg_table) @@ -372,6 +375,7 @@ void tmc_sg_table_sync_table(struct tmc_sg_table *sg_table) dma_sync_single_for_device(real_dev, table_pages->daddrs[i], PAGE_SIZE, DMA_TO_DEVICE); } +EXPORT_SYMBOL_GPL(tmc_sg_table_sync_table); /* * tmc_sg_table_get_data: Get the buffer pointer for data @offset @@ -401,6 +405,7 @@ ssize_t tmc_sg_table_get_data(struct tmc_sg_table *sg_table, *bufpp = page_address(data_pages->pages[pg_idx]) + pg_offset; return len; } +EXPORT_SYMBOL_GPL(tmc_sg_table_get_data); #ifdef ETR_SG_DEBUG /* Map a dma address to virtual address */ @@ -766,6 +771,7 @@ tmc_etr_get_catu_device(struct tmc_drvdata *drvdata) return NULL; } +EXPORT_SYMBOL_GPL(tmc_etr_get_catu_device); static inline int tmc_etr_enable_catu(struct tmc_drvdata *drvdata, struct etr_buf *etr_buf) @@ -788,10 +794,21 @@ static inline void tmc_etr_disable_catu(struct tmc_drvdata *drvdata) static const struct etr_buf_operations *etr_buf_ops[] = { [ETR_MODE_FLAT] = &etr_flat_buf_ops, [ETR_MODE_ETR_SG] = &etr_sg_buf_ops, - [ETR_MODE_CATU] = IS_ENABLED(CONFIG_CORESIGHT_CATU) - ? &etr_catu_buf_ops : NULL, + [ETR_MODE_CATU] = NULL, }; +void tmc_etr_set_catu_ops(const struct etr_buf_operations *catu) +{ + etr_buf_ops[ETR_MODE_CATU] = catu; +} +EXPORT_SYMBOL_GPL(tmc_etr_set_catu_ops); + +void tmc_etr_remove_catu_ops(void) +{ + etr_buf_ops[ETR_MODE_CATU] = NULL; +} +EXPORT_SYMBOL_GPL(tmc_etr_remove_catu_ops); + static inline int tmc_etr_mode_alloc_buf(int mode, struct tmc_drvdata *drvdata, struct etr_buf *etr_buf, int node, diff --git a/drivers/hwtracing/coresight/coresight-tmc.h b/drivers/hwtracing/coresight/coresight-tmc.h index 6e8d2dc33d17..b91ec7dde7bc 100644 --- a/drivers/hwtracing/coresight/coresight-tmc.h +++ b/drivers/hwtracing/coresight/coresight-tmc.h @@ -326,4 +326,7 @@ tmc_sg_table_buf_size(struct tmc_sg_table *sg_table) struct coresight_device *tmc_etr_get_catu_device(struct tmc_drvdata *drvdata); +void tmc_etr_set_catu_ops(const struct etr_buf_operations *catu); +void tmc_etr_remove_catu_ops(void); + #endif diff --git a/drivers/hwtracing/coresight/coresight-tpiu.c b/drivers/hwtracing/coresight/coresight-tpiu.c index f8583e4032a6..566c57e03596 100644 --- a/drivers/hwtracing/coresight/coresight-tpiu.c +++ b/drivers/hwtracing/coresight/coresight-tpiu.c @@ -173,6 +173,15 @@ static int tpiu_probe(struct amba_device *adev, const struct amba_id *id) return PTR_ERR(drvdata->csdev); } +static int __exit tpiu_remove(struct amba_device *adev) +{ + struct tpiu_drvdata *drvdata = dev_get_drvdata(&adev->dev); + + coresight_unregister(drvdata->csdev); + + return 0; +} + #ifdef CONFIG_PM static int tpiu_runtime_suspend(struct device *dev) { @@ -216,6 +225,8 @@ static const struct amba_id tpiu_ids[] = { { 0, 0}, }; +MODULE_DEVICE_TABLE(amba, tpiu_ids); + static struct amba_driver tpiu_driver = { .drv = { .name = "coresight-tpiu", @@ -224,6 +235,13 @@ static struct amba_driver tpiu_driver = { .suppress_bind_attrs = true, }, .probe = tpiu_probe, + .remove = tpiu_remove, .id_table = tpiu_ids, }; -builtin_amba_driver(tpiu_driver); + +module_amba_driver(tpiu_driver); + +MODULE_AUTHOR("Pratik Patel <pratikp@codeaurora.org>"); +MODULE_AUTHOR("Mathieu Poirier <mathieu.poirier@linaro.org>"); +MODULE_DESCRIPTION("Arm CoreSight TPIU (Trace Port Interface Unit) driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/hwtracing/intel_th/pci.c b/drivers/hwtracing/intel_th/pci.c index 21fdf0b93516..52acd77438ed 100644 --- a/drivers/hwtracing/intel_th/pci.c +++ b/drivers/hwtracing/intel_th/pci.c @@ -263,6 +263,16 @@ static const struct pci_device_id intel_th_pci_id_table[] = { PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x1bcc), .driver_data = (kernel_ulong_t)&intel_th_2x, }, + { + /* Alder Lake */ + PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x7aa6), + .driver_data = (kernel_ulong_t)&intel_th_2x, + }, + { + /* Alder Lake CPU */ + PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x466f), + .driver_data = (kernel_ulong_t)&intel_th_2x, + }, { 0 }, }; diff --git a/drivers/hwtracing/stm/Kconfig b/drivers/hwtracing/stm/Kconfig index d0e92a8a045c..aad594fe79cc 100644 --- a/drivers/hwtracing/stm/Kconfig +++ b/drivers/hwtracing/stm/Kconfig @@ -71,7 +71,7 @@ config STM_SOURCE_HEARTBEAT config STM_SOURCE_FTRACE tristate "Copy the output from kernel Ftrace to STM engine" - depends on FUNCTION_TRACER + depends on TRACING help This option can be used to copy the output from kernel Ftrace to STM engine. Enabling this option will introduce a slight diff --git a/drivers/hwtracing/stm/ftrace.c b/drivers/hwtracing/stm/ftrace.c index ce868e095410..3bb606dfa634 100644 --- a/drivers/hwtracing/stm/ftrace.c +++ b/drivers/hwtracing/stm/ftrace.c @@ -37,8 +37,10 @@ static void notrace stm_ftrace_write(struct trace_export *export, const void *buf, unsigned int len) { struct stm_ftrace *stm = container_of(export, struct stm_ftrace, ftrace); + /* This is called from trace system with preemption disabled */ + unsigned int cpu = smp_processor_id(); - stm_source_write(&stm->data, STM_FTRACE_CHAN, buf, len); + stm_source_write(&stm->data, STM_FTRACE_CHAN + cpu, buf, len); } static int stm_ftrace_link(struct stm_source_data *data) @@ -46,6 +48,8 @@ static int stm_ftrace_link(struct stm_source_data *data) struct stm_ftrace *sf = container_of(data, struct stm_ftrace, data); sf->ftrace.write = stm_ftrace_write; + sf->ftrace.flags = TRACE_EXPORT_FUNCTION | TRACE_EXPORT_EVENT + | TRACE_EXPORT_MARKER; return register_ftrace_export(&sf->ftrace); } @@ -61,6 +65,7 @@ static int __init stm_ftrace_init(void) { int ret; + stm_ftrace.data.nr_chans = roundup_pow_of_two(num_possible_cpus()); ret = stm_source_register_device(NULL, &stm_ftrace.data); if (ret) pr_err("Failed to register stm_source - ftrace.\n"); diff --git a/drivers/interconnect/Makefile b/drivers/interconnect/Makefile index 4825c287ca13..d203520b0a56 100644 --- a/drivers/interconnect/Makefile +++ b/drivers/interconnect/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 CFLAGS_core.o := -I$(src) -icc-core-objs := core.o +icc-core-objs := core.o bulk.o obj-$(CONFIG_INTERCONNECT) += icc-core.o obj-$(CONFIG_INTERCONNECT_IMX) += imx/ diff --git a/drivers/interconnect/bulk.c b/drivers/interconnect/bulk.c new file mode 100644 index 000000000000..73e2c8d0a412 --- /dev/null +++ b/drivers/interconnect/bulk.c @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/interconnect-provider.h> +#include <linux/device.h> +#include <linux/export.h> + +/** + * of_icc_bulk_get() - get interconnect paths + * @dev: the device requesting the path + * @num_paths: the number of icc_bulk_data + * @paths: the table with the paths we want to get + * + * Returns 0 on success or negative errno otherwise. + */ +int __must_check of_icc_bulk_get(struct device *dev, int num_paths, + struct icc_bulk_data *paths) +{ + int ret, i; + + for (i = 0; i < num_paths; i++) { + paths[i].path = of_icc_get(dev, paths[i].name); + if (IS_ERR(paths[i].path)) { + ret = PTR_ERR(paths[i].path); + if (ret != -EPROBE_DEFER) + dev_err(dev, "of_icc_get() failed on path %s (%d)\n", + paths[i].name, ret); + paths[i].path = NULL; + goto err; + } + } + + return 0; + +err: + icc_bulk_put(i, paths); + + return ret; +} +EXPORT_SYMBOL_GPL(of_icc_bulk_get); + +/** + * icc_bulk_put() - put a list of interconnect paths + * @num_paths: the number of icc_bulk_data + * @paths: the icc_bulk_data table with the paths being put + */ +void icc_bulk_put(int num_paths, struct icc_bulk_data *paths) +{ + while (--num_paths >= 0) { + icc_put(paths[num_paths].path); + paths[num_paths].path = NULL; + } +} +EXPORT_SYMBOL_GPL(icc_bulk_put); + +/** + * icc_bulk_set() - set bandwidth to a set of paths + * @num_paths: the number of icc_bulk_data + * @paths: the icc_bulk_data table containing the paths and bandwidth + * + * Returns 0 on success or negative errno otherwise. + */ +int icc_bulk_set_bw(int num_paths, const struct icc_bulk_data *paths) +{ + int ret = 0; + int i; + + for (i = 0; i < num_paths; i++) { + ret = icc_set_bw(paths[i].path, paths[i].avg_bw, paths[i].peak_bw); + if (ret) { + pr_err("icc_set_bw() failed on path %s (%d)\n", paths[i].name, ret); + return ret; + } + } + + return ret; +} +EXPORT_SYMBOL_GPL(icc_bulk_set_bw); + +/** + * icc_bulk_enable() - enable a previously disabled set of paths + * @num_paths: the number of icc_bulk_data + * @paths: the icc_bulk_data table containing the paths and bandwidth + * + * Returns 0 on success or negative errno otherwise. + */ +int icc_bulk_enable(int num_paths, const struct icc_bulk_data *paths) +{ + int ret, i; + + for (i = 0; i < num_paths; i++) { + ret = icc_enable(paths[i].path); + if (ret) { + pr_err("icc_enable() failed on path %s (%d)\n", paths[i].name, ret); + goto err; + } + } + + return 0; + +err: + icc_bulk_disable(i, paths); + + return ret; +} +EXPORT_SYMBOL_GPL(icc_bulk_enable); + +/** + * icc_bulk_disable() - disable a set of interconnect paths + * @num_paths: the number of icc_bulk_data + * @paths: the icc_bulk_data table containing the paths and bandwidth + */ +void icc_bulk_disable(int num_paths, const struct icc_bulk_data *paths) +{ + while (--num_paths >= 0) + icc_disable(paths[num_paths].path); +} +EXPORT_SYMBOL_GPL(icc_bulk_disable); diff --git a/drivers/interconnect/core.c b/drivers/interconnect/core.c index cf07491b7415..eea47b4c84aa 100644 --- a/drivers/interconnect/core.c +++ b/drivers/interconnect/core.c @@ -26,6 +26,8 @@ static DEFINE_IDR(icc_idr); static LIST_HEAD(icc_providers); +static int providers_count; +static bool synced_state; static DEFINE_MUTEX(icc_lock); static struct dentry *icc_debugfs_dir; @@ -267,6 +269,12 @@ static int aggregate_requests(struct icc_node *node) } p->aggregate(node, r->tag, avg_bw, peak_bw, &node->avg_bw, &node->peak_bw); + + /* during boot use the initial bandwidth as a floor value */ + if (!synced_state) { + node->avg_bw = max(node->avg_bw, node->init_avg); + node->peak_bw = max(node->peak_bw, node->init_peak); + } } return 0; @@ -342,12 +350,13 @@ EXPORT_SYMBOL_GPL(of_icc_xlate_onecell); * Looks for interconnect provider under the node specified by @spec and if * found, uses xlate function of the provider to map phandle args to node. * - * Returns a valid pointer to struct icc_node on success or ERR_PTR() + * Returns a valid pointer to struct icc_node_data on success or ERR_PTR() * on failure. */ -struct icc_node *of_icc_get_from_provider(struct of_phandle_args *spec) +struct icc_node_data *of_icc_get_from_provider(struct of_phandle_args *spec) { struct icc_node *node = ERR_PTR(-EPROBE_DEFER); + struct icc_node_data *data = NULL; struct icc_provider *provider; if (!spec) @@ -355,14 +364,33 @@ struct icc_node *of_icc_get_from_provider(struct of_phandle_args *spec) mutex_lock(&icc_lock); list_for_each_entry(provider, &icc_providers, provider_list) { - if (provider->dev->of_node == spec->np) - node = provider->xlate(spec, provider->data); - if (!IS_ERR(node)) - break; + if (provider->dev->of_node == spec->np) { + if (provider->xlate_extended) { + data = provider->xlate_extended(spec, provider->data); + if (!IS_ERR(data)) { + node = data->node; + break; + } + } else { + node = provider->xlate(spec, provider->data); + if (!IS_ERR(node)) + break; + } + } } mutex_unlock(&icc_lock); - return node; + if (IS_ERR(node)) + return ERR_CAST(node); + + if (!data) { + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return ERR_PTR(-ENOMEM); + data->node = node; + } + + return data; } EXPORT_SYMBOL_GPL(of_icc_get_from_provider); @@ -409,7 +437,7 @@ EXPORT_SYMBOL_GPL(devm_of_icc_get); struct icc_path *of_icc_get_by_index(struct device *dev, int idx) { struct icc_path *path; - struct icc_node *src_node, *dst_node; + struct icc_node_data *src_data, *dst_data; struct device_node *np; struct of_phandle_args src_args, dst_args; int ret; @@ -447,39 +475,42 @@ struct icc_path *of_icc_get_by_index(struct device *dev, int idx) of_node_put(dst_args.np); - src_node = of_icc_get_from_provider(&src_args); + src_data = of_icc_get_from_provider(&src_args); - if (IS_ERR(src_node)) { - if (PTR_ERR(src_node) != -EPROBE_DEFER) - dev_err(dev, "error finding src node: %ld\n", - PTR_ERR(src_node)); - return ERR_CAST(src_node); + if (IS_ERR(src_data)) { + dev_err_probe(dev, PTR_ERR(src_data), "error finding src node\n"); + return ERR_CAST(src_data); } - dst_node = of_icc_get_from_provider(&dst_args); + dst_data = of_icc_get_from_provider(&dst_args); - if (IS_ERR(dst_node)) { - if (PTR_ERR(dst_node) != -EPROBE_DEFER) - dev_err(dev, "error finding dst node: %ld\n", - PTR_ERR(dst_node)); - return ERR_CAST(dst_node); + if (IS_ERR(dst_data)) { + dev_err_probe(dev, PTR_ERR(dst_data), "error finding dst node\n"); + kfree(src_data); + return ERR_CAST(dst_data); } mutex_lock(&icc_lock); - path = path_find(dev, src_node, dst_node); + path = path_find(dev, src_data->node, dst_data->node); mutex_unlock(&icc_lock); if (IS_ERR(path)) { dev_err(dev, "%s: invalid path=%ld\n", __func__, PTR_ERR(path)); - return path; + goto free_icc_data; } + if (src_data->tag && src_data->tag == dst_data->tag) + icc_set_tag(path, src_data->tag); + path->name = kasprintf(GFP_KERNEL, "%s-%s", - src_node->name, dst_node->name); + src_data->node->name, dst_data->node->name); if (!path->name) { kfree(path); - return ERR_PTR(-ENOMEM); + path = ERR_PTR(-ENOMEM); } +free_icc_data: + kfree(src_data); + kfree(dst_data); return path; } EXPORT_SYMBOL_GPL(of_icc_get_by_index); @@ -931,6 +962,19 @@ void icc_node_add(struct icc_node *node, struct icc_provider *provider) node->provider = provider; list_add_tail(&node->node_list, &provider->nodes); + /* get the initial bandwidth values and sync them with hardware */ + if (provider->get_bw) { + provider->get_bw(node, &node->init_avg, &node->init_peak); + } else { + node->init_avg = INT_MAX; + node->init_peak = INT_MAX; + } + node->avg_bw = node->init_avg; + node->peak_bw = node->init_peak; + provider->set(node, node); + node->avg_bw = 0; + node->peak_bw = 0; + mutex_unlock(&icc_lock); } EXPORT_SYMBOL_GPL(icc_node_add); @@ -981,7 +1025,7 @@ int icc_provider_add(struct icc_provider *provider) { if (WARN_ON(!provider->set)) return -EINVAL; - if (WARN_ON(!provider->xlate)) + if (WARN_ON(!provider->xlate && !provider->xlate_extended)) return -EINVAL; mutex_lock(&icc_lock); @@ -1026,8 +1070,54 @@ int icc_provider_del(struct icc_provider *provider) } EXPORT_SYMBOL_GPL(icc_provider_del); +static int of_count_icc_providers(struct device_node *np) +{ + struct device_node *child; + int count = 0; + + for_each_available_child_of_node(np, child) { + if (of_property_read_bool(child, "#interconnect-cells")) + count++; + count += of_count_icc_providers(child); + } + of_node_put(np); + + return count; +} + +void icc_sync_state(struct device *dev) +{ + struct icc_provider *p; + struct icc_node *n; + static int count; + + count++; + + if (count < providers_count) + return; + + mutex_lock(&icc_lock); + synced_state = true; + list_for_each_entry(p, &icc_providers, provider_list) { + dev_dbg(p->dev, "interconnect provider is in synced state\n"); + list_for_each_entry(n, &p->nodes, node_list) { + if (n->init_avg || n->init_peak) { + aggregate_requests(n); + p->set(n, n); + } + } + } + mutex_unlock(&icc_lock); +} +EXPORT_SYMBOL_GPL(icc_sync_state); + static int __init icc_init(void) { + struct device_node *root = of_find_node_by_path("/"); + + providers_count = of_count_icc_providers(root); + of_node_put(root); + icc_debugfs_dir = debugfs_create_dir("interconnect", NULL); debugfs_create_file("interconnect_summary", 0444, icc_debugfs_dir, NULL, &icc_summary_fops); diff --git a/drivers/interconnect/imx/imx.c b/drivers/interconnect/imx/imx.c index ac420f86008e..41dba7090c2a 100644 --- a/drivers/interconnect/imx/imx.c +++ b/drivers/interconnect/imx/imx.c @@ -184,10 +184,8 @@ static int imx_icc_register_nodes(struct icc_provider *provider, node = imx_icc_node_add(provider, node_desc); if (IS_ERR(node)) { - ret = PTR_ERR(node); - if (ret != -EPROBE_DEFER) - dev_err(provider->dev, "failed to add %s: %d\n", - node_desc->name, ret); + ret = dev_err_probe(provider->dev, PTR_ERR(node), + "failed to add %s\n", node_desc->name); goto err; } provider_data->nodes[node->id] = node; @@ -269,15 +267,10 @@ EXPORT_SYMBOL_GPL(imx_icc_register); int imx_icc_unregister(struct platform_device *pdev) { struct icc_provider *provider = platform_get_drvdata(pdev); - int ret; imx_icc_unregister_nodes(provider); - ret = icc_provider_del(provider); - if (ret) - return ret; - - return 0; + return icc_provider_del(provider); } EXPORT_SYMBOL_GPL(imx_icc_unregister); diff --git a/drivers/interconnect/qcom/Kconfig b/drivers/interconnect/qcom/Kconfig index a88f2f07bc27..a8f93ba265f8 100644 --- a/drivers/interconnect/qcom/Kconfig +++ b/drivers/interconnect/qcom/Kconfig @@ -65,5 +65,25 @@ config INTERCONNECT_QCOM_SDM845 This is a driver for the Qualcomm Network-on-Chip on sdm845-based platforms. +config INTERCONNECT_QCOM_SM8150 + tristate "Qualcomm SM8150 interconnect driver" + depends on INTERCONNECT_QCOM + depends on (QCOM_RPMH && QCOM_COMMAND_DB && OF) || COMPILE_TEST + select INTERCONNECT_QCOM_RPMH + select INTERCONNECT_QCOM_BCM_VOTER + help + This is a driver for the Qualcomm Network-on-Chip on sm8150-based + platforms. + +config INTERCONNECT_QCOM_SM8250 + tristate "Qualcomm SM8250 interconnect driver" + depends on INTERCONNECT_QCOM + depends on (QCOM_RPMH && QCOM_COMMAND_DB && OF) || COMPILE_TEST + select INTERCONNECT_QCOM_RPMH + select INTERCONNECT_QCOM_BCM_VOTER + help + This is a driver for the Qualcomm Network-on-Chip on sm8250-based + platforms. + config INTERCONNECT_QCOM_SMD_RPM tristate diff --git a/drivers/interconnect/qcom/Makefile b/drivers/interconnect/qcom/Makefile index 3a047fe6e45a..cf628f7990cd 100644 --- a/drivers/interconnect/qcom/Makefile +++ b/drivers/interconnect/qcom/Makefile @@ -8,6 +8,8 @@ qnoc-qcs404-objs := qcs404.o icc-rpmh-obj := icc-rpmh.o qnoc-sc7180-objs := sc7180.o qnoc-sdm845-objs := sdm845.o +qnoc-sm8150-objs := sm8150.o +qnoc-sm8250-objs := sm8250.o icc-smd-rpm-objs := smd-rpm.o obj-$(CONFIG_INTERCONNECT_QCOM_BCM_VOTER) += icc-bcm-voter.o @@ -18,4 +20,6 @@ obj-$(CONFIG_INTERCONNECT_QCOM_QCS404) += qnoc-qcs404.o obj-$(CONFIG_INTERCONNECT_QCOM_RPMH) += icc-rpmh.o obj-$(CONFIG_INTERCONNECT_QCOM_SC7180) += qnoc-sc7180.o obj-$(CONFIG_INTERCONNECT_QCOM_SDM845) += qnoc-sdm845.o +obj-$(CONFIG_INTERCONNECT_QCOM_SM8150) += qnoc-sm8150.o +obj-$(CONFIG_INTERCONNECT_QCOM_SM8250) += qnoc-sm8250.o obj-$(CONFIG_INTERCONNECT_QCOM_SMD_RPM) += icc-smd-rpm.o diff --git a/drivers/interconnect/qcom/bcm-voter.c b/drivers/interconnect/qcom/bcm-voter.c index 609db9c95fd7..887d13721e52 100644 --- a/drivers/interconnect/qcom/bcm-voter.c +++ b/drivers/interconnect/qcom/bcm-voter.c @@ -27,6 +27,7 @@ static DEFINE_MUTEX(bcm_voter_lock); * @commit_list: list containing bcms to be committed to hardware * @ws_list: list containing bcms that have different wake/sleep votes * @voter_node: list of bcm voters + * @tcs_wait: mask for which buckets require TCS completion */ struct bcm_voter { struct device *dev; @@ -35,6 +36,7 @@ struct bcm_voter { struct list_head commit_list; struct list_head ws_list; struct list_head voter_node; + u32 tcs_wait; }; static int cmp_vcd(void *priv, struct list_head *a, struct list_head *b) @@ -83,10 +85,10 @@ static void bcm_aggregate(struct qcom_icc_bcm *bcm) agg_peak[bucket] = max(agg_peak[bucket], temp); } - temp = agg_avg[bucket] * 1000ULL; + temp = agg_avg[bucket] * bcm->vote_scale; bcm->vote_x[bucket] = bcm_div(temp, bcm->aux_data.unit); - temp = agg_peak[bucket] * 1000ULL; + temp = agg_peak[bucket] * bcm->vote_scale; bcm->vote_y[bucket] = bcm_div(temp, bcm->aux_data.unit); } @@ -100,7 +102,7 @@ static void bcm_aggregate(struct qcom_icc_bcm *bcm) } static inline void tcs_cmd_gen(struct tcs_cmd *cmd, u64 vote_x, u64 vote_y, - u32 addr, bool commit) + u32 addr, bool commit, bool wait) { bool valid = true; @@ -125,15 +127,16 @@ static inline void tcs_cmd_gen(struct tcs_cmd *cmd, u64 vote_x, u64 vote_y, * Set the wait for completion flag on command that need to be completed * before the next command. */ - cmd->wait = commit; + cmd->wait = wait; } -static void tcs_list_gen(struct list_head *bcm_list, int bucket, - struct tcs_cmd tcs_list[MAX_BCMS], +static void tcs_list_gen(struct bcm_voter *voter, int bucket, + struct tcs_cmd tcs_list[MAX_VCD], int n[MAX_VCD + 1]) { + struct list_head *bcm_list = &voter->commit_list; struct qcom_icc_bcm *bcm; - bool commit; + bool commit, wait; size_t idx = 0, batch = 0, cur_vcd_size = 0; memset(n, 0, sizeof(int) * (MAX_VCD + 1)); @@ -146,8 +149,11 @@ static void tcs_list_gen(struct list_head *bcm_list, int bucket, commit = true; cur_vcd_size = 0; } + + wait = commit && (voter->tcs_wait & BIT(bucket)); + tcs_cmd_gen(&tcs_list[idx], bcm->vote_x[bucket], - bcm->vote_y[bucket], bcm->addr, commit); + bcm->vote_y[bucket], bcm->addr, commit, wait); idx++; n[batch]++; /* @@ -272,8 +278,7 @@ int qcom_icc_bcm_voter_commit(struct bcm_voter *voter) * Construct the command list based on a pre ordered list of BCMs * based on VCD. */ - tcs_list_gen(&voter->commit_list, QCOM_ICC_BUCKET_AMC, cmds, commit_idx); - + tcs_list_gen(voter, QCOM_ICC_BUCKET_AMC, cmds, commit_idx); if (!commit_idx[0]) goto out; @@ -309,7 +314,7 @@ int qcom_icc_bcm_voter_commit(struct bcm_voter *voter) list_sort(NULL, &voter->commit_list, cmp_vcd); - tcs_list_gen(&voter->commit_list, QCOM_ICC_BUCKET_WAKE, cmds, commit_idx); + tcs_list_gen(voter, QCOM_ICC_BUCKET_WAKE, cmds, commit_idx); ret = rpmh_write_batch(voter->dev, RPMH_WAKE_ONLY_STATE, cmds, commit_idx); if (ret) { @@ -317,7 +322,7 @@ int qcom_icc_bcm_voter_commit(struct bcm_voter *voter) goto out; } - tcs_list_gen(&voter->commit_list, QCOM_ICC_BUCKET_SLEEP, cmds, commit_idx); + tcs_list_gen(voter, QCOM_ICC_BUCKET_SLEEP, cmds, commit_idx); ret = rpmh_write_batch(voter->dev, RPMH_SLEEP_STATE, cmds, commit_idx); if (ret) { @@ -336,6 +341,7 @@ EXPORT_SYMBOL_GPL(qcom_icc_bcm_voter_commit); static int qcom_icc_bcm_voter_probe(struct platform_device *pdev) { + struct device_node *np = pdev->dev.of_node; struct bcm_voter *voter; voter = devm_kzalloc(&pdev->dev, sizeof(*voter), GFP_KERNEL); @@ -343,7 +349,11 @@ static int qcom_icc_bcm_voter_probe(struct platform_device *pdev) return -ENOMEM; voter->dev = &pdev->dev; - voter->np = pdev->dev.of_node; + voter->np = np; + + if (of_property_read_u32(np, "qcom,tcs-wait", &voter->tcs_wait)) + voter->tcs_wait = QCOM_ICC_TAG_ACTIVE_ONLY; + mutex_init(&voter->lock); INIT_LIST_HEAD(&voter->commit_list); INIT_LIST_HEAD(&voter->ws_list); diff --git a/drivers/interconnect/qcom/icc-rpmh.c b/drivers/interconnect/qcom/icc-rpmh.c index 3ac5182c9ab2..cf10a4b9611b 100644 --- a/drivers/interconnect/qcom/icc-rpmh.c +++ b/drivers/interconnect/qcom/icc-rpmh.c @@ -6,6 +6,8 @@ #include <linux/interconnect.h> #include <linux/interconnect-provider.h> #include <linux/module.h> +#include <linux/of.h> +#include <linux/slab.h> #include "bcm-voter.h" #include "icc-rpmh.h" @@ -92,6 +94,31 @@ int qcom_icc_set(struct icc_node *src, struct icc_node *dst) } EXPORT_SYMBOL_GPL(qcom_icc_set); +struct icc_node_data *qcom_icc_xlate_extended(struct of_phandle_args *spec, void *data) +{ + struct icc_node_data *ndata; + struct icc_node *node; + + node = of_icc_xlate_onecell(spec, data); + if (IS_ERR(node)) + return ERR_CAST(node); + + ndata = kzalloc(sizeof(*ndata), GFP_KERNEL); + if (!ndata) + return ERR_PTR(-ENOMEM); + + ndata->node = node; + + if (spec->args_count == 2) + ndata->tag = spec->args[1]; + + if (spec->args_count > 2) + pr_warn("%pOF: Too many arguments, path tag is not parsed\n", spec->np); + + return ndata; +} +EXPORT_SYMBOL_GPL(qcom_icc_xlate_extended); + /** * qcom_icc_bcm_init - populates bcm aux data and connect qnodes * @bcm: bcm to be initialized @@ -136,6 +163,9 @@ int qcom_icc_bcm_init(struct qcom_icc_bcm *bcm, struct device *dev) INIT_LIST_HEAD(&bcm->list); INIT_LIST_HEAD(&bcm->ws_list); + if (!bcm->vote_scale) + bcm->vote_scale = 1000; + /* Link Qnodes to their respective BCMs */ for (i = 0; i < bcm->num_nodes; i++) { qn = bcm->nodes[i]; diff --git a/drivers/interconnect/qcom/icc-rpmh.h b/drivers/interconnect/qcom/icc-rpmh.h index 903d25e61984..e5f61ab989e7 100644 --- a/drivers/interconnect/qcom/icc-rpmh.h +++ b/drivers/interconnect/qcom/icc-rpmh.h @@ -6,6 +6,8 @@ #ifndef __DRIVERS_INTERCONNECT_QCOM_ICC_RPMH_H__ #define __DRIVERS_INTERCONNECT_QCOM_ICC_RPMH_H__ +#include <dt-bindings/interconnect/qcom,icc.h> + #define to_qcom_provider(_provider) \ container_of(_provider, struct qcom_icc_provider, provider) @@ -44,22 +46,6 @@ struct bcm_db { #define MAX_BCM_PER_NODE 3 #define MAX_VCD 10 -/* - * The AMC bucket denotes constraints that are applied to hardware when - * icc_set_bw() completes, whereas the WAKE and SLEEP constraints are applied - * when the execution environment transitions between active and low power mode. - */ -#define QCOM_ICC_BUCKET_AMC 0 -#define QCOM_ICC_BUCKET_WAKE 1 -#define QCOM_ICC_BUCKET_SLEEP 2 -#define QCOM_ICC_NUM_BUCKETS 3 -#define QCOM_ICC_TAG_AMC BIT(QCOM_ICC_BUCKET_AMC) -#define QCOM_ICC_TAG_WAKE BIT(QCOM_ICC_BUCKET_WAKE) -#define QCOM_ICC_TAG_SLEEP BIT(QCOM_ICC_BUCKET_SLEEP) -#define QCOM_ICC_TAG_ACTIVE_ONLY (QCOM_ICC_TAG_AMC | QCOM_ICC_TAG_WAKE) -#define QCOM_ICC_TAG_ALWAYS (QCOM_ICC_TAG_AMC | QCOM_ICC_TAG_WAKE |\ - QCOM_ICC_TAG_SLEEP) - /** * struct qcom_icc_node - Qualcomm specific interconnect nodes * @name: the node name used in debugfs @@ -94,6 +80,7 @@ struct qcom_icc_node { * @addr: address offsets used when voting to RPMH * @vote_x: aggregated threshold values, represents sum_bw when @type is bw bcm * @vote_y: aggregated threshold values, represents peak_bw when @type is bw bcm + * @vote_scale: scaling factor for vote_x and vote_y * @dirty: flag used to indicate whether the bcm needs to be committed * @keepalive: flag used to indicate whether a keepalive is required * @aux_data: auxiliary data used when calculating threshold values and @@ -109,6 +96,7 @@ struct qcom_icc_bcm { u32 addr; u64 vote_x[QCOM_ICC_NUM_BUCKETS]; u64 vote_y[QCOM_ICC_NUM_BUCKETS]; + u64 vote_scale; bool dirty; bool keepalive; struct bcm_db aux_data; @@ -143,6 +131,7 @@ struct qcom_icc_desc { int qcom_icc_aggregate(struct icc_node *node, u32 tag, u32 avg_bw, u32 peak_bw, u32 *agg_avg, u32 *agg_peak); int qcom_icc_set(struct icc_node *src, struct icc_node *dst); +struct icc_node_data *qcom_icc_xlate_extended(struct of_phandle_args *spec, void *data); int qcom_icc_bcm_init(struct qcom_icc_bcm *bcm, struct device *dev); void qcom_icc_pre_aggregate(struct icc_node *node); diff --git a/drivers/interconnect/qcom/osm-l3.c b/drivers/interconnect/qcom/osm-l3.c index 96fb9ff5ff2e..695f28789e98 100644 --- a/drivers/interconnect/qcom/osm-l3.c +++ b/drivers/interconnect/qcom/osm-l3.c @@ -16,17 +16,24 @@ #include "sc7180.h" #include "sdm845.h" +#include "sm8150.h" +#include "sm8250.h" #define LUT_MAX_ENTRIES 40U #define LUT_SRC GENMASK(31, 30) #define LUT_L_VAL GENMASK(7, 0) -#define LUT_ROW_SIZE 32 #define CLK_HW_DIV 2 -/* Register offsets */ +/* OSM Register offsets */ #define REG_ENABLE 0x0 -#define REG_FREQ_LUT 0x110 -#define REG_PERF_STATE 0x920 +#define OSM_LUT_ROW_SIZE 32 +#define OSM_REG_FREQ_LUT 0x110 +#define OSM_REG_PERF_STATE 0x920 + +/* EPSS Register offsets */ +#define EPSS_LUT_ROW_SIZE 4 +#define EPSS_REG_FREQ_LUT 0x100 +#define EPSS_REG_PERF_STATE 0x320 #define OSM_L3_MAX_LINKS 1 @@ -36,6 +43,7 @@ struct qcom_osm_l3_icc_provider { void __iomem *base; unsigned int max_state; + unsigned int reg_perf_state; unsigned long lut_tables[LUT_MAX_ENTRIES]; struct icc_provider provider; }; @@ -57,12 +65,15 @@ struct qcom_icc_node { }; struct qcom_icc_desc { - struct qcom_icc_node **nodes; + const struct qcom_icc_node **nodes; size_t num_nodes; + unsigned int lut_row_size; + unsigned int reg_freq_lut; + unsigned int reg_perf_state; }; #define DEFINE_QNODE(_name, _id, _buswidth, ...) \ - static struct qcom_icc_node _name = { \ + static const struct qcom_icc_node _name = { \ .name = #_name, \ .id = _id, \ .buswidth = _buswidth, \ @@ -73,7 +84,7 @@ struct qcom_icc_desc { DEFINE_QNODE(sdm845_osm_apps_l3, SDM845_MASTER_OSM_L3_APPS, 16, SDM845_SLAVE_OSM_L3); DEFINE_QNODE(sdm845_osm_l3, SDM845_SLAVE_OSM_L3, 16); -static struct qcom_icc_node *sdm845_osm_l3_nodes[] = { +static const struct qcom_icc_node *sdm845_osm_l3_nodes[] = { [MASTER_OSM_L3_APPS] = &sdm845_osm_apps_l3, [SLAVE_OSM_L3] = &sdm845_osm_l3, }; @@ -81,12 +92,15 @@ static struct qcom_icc_node *sdm845_osm_l3_nodes[] = { static const struct qcom_icc_desc sdm845_icc_osm_l3 = { .nodes = sdm845_osm_l3_nodes, .num_nodes = ARRAY_SIZE(sdm845_osm_l3_nodes), + .lut_row_size = OSM_LUT_ROW_SIZE, + .reg_freq_lut = OSM_REG_FREQ_LUT, + .reg_perf_state = OSM_REG_PERF_STATE, }; DEFINE_QNODE(sc7180_osm_apps_l3, SC7180_MASTER_OSM_L3_APPS, 16, SC7180_SLAVE_OSM_L3); DEFINE_QNODE(sc7180_osm_l3, SC7180_SLAVE_OSM_L3, 16); -static struct qcom_icc_node *sc7180_osm_l3_nodes[] = { +static const struct qcom_icc_node *sc7180_osm_l3_nodes[] = { [MASTER_OSM_L3_APPS] = &sc7180_osm_apps_l3, [SLAVE_OSM_L3] = &sc7180_osm_l3, }; @@ -94,13 +108,48 @@ static struct qcom_icc_node *sc7180_osm_l3_nodes[] = { static const struct qcom_icc_desc sc7180_icc_osm_l3 = { .nodes = sc7180_osm_l3_nodes, .num_nodes = ARRAY_SIZE(sc7180_osm_l3_nodes), + .lut_row_size = OSM_LUT_ROW_SIZE, + .reg_freq_lut = OSM_REG_FREQ_LUT, + .reg_perf_state = OSM_REG_PERF_STATE, +}; + +DEFINE_QNODE(sm8150_osm_apps_l3, SM8150_MASTER_OSM_L3_APPS, 32, SM8150_SLAVE_OSM_L3); +DEFINE_QNODE(sm8150_osm_l3, SM8150_SLAVE_OSM_L3, 32); + +static const struct qcom_icc_node *sm8150_osm_l3_nodes[] = { + [MASTER_OSM_L3_APPS] = &sm8150_osm_apps_l3, + [SLAVE_OSM_L3] = &sm8150_osm_l3, +}; + +static const struct qcom_icc_desc sm8150_icc_osm_l3 = { + .nodes = sm8150_osm_l3_nodes, + .num_nodes = ARRAY_SIZE(sm8150_osm_l3_nodes), + .lut_row_size = OSM_LUT_ROW_SIZE, + .reg_freq_lut = OSM_REG_FREQ_LUT, + .reg_perf_state = OSM_REG_PERF_STATE, +}; + +DEFINE_QNODE(sm8250_epss_apps_l3, SM8250_MASTER_EPSS_L3_APPS, 32, SM8250_SLAVE_EPSS_L3); +DEFINE_QNODE(sm8250_epss_l3, SM8250_SLAVE_EPSS_L3, 32); + +static const struct qcom_icc_node *sm8250_epss_l3_nodes[] = { + [MASTER_EPSS_L3_APPS] = &sm8250_epss_apps_l3, + [SLAVE_EPSS_L3_SHARED] = &sm8250_epss_l3, +}; + +static const struct qcom_icc_desc sm8250_icc_epss_l3 = { + .nodes = sm8250_epss_l3_nodes, + .num_nodes = ARRAY_SIZE(sm8250_epss_l3_nodes), + .lut_row_size = EPSS_LUT_ROW_SIZE, + .reg_freq_lut = EPSS_REG_FREQ_LUT, + .reg_perf_state = EPSS_REG_PERF_STATE, }; static int qcom_icc_set(struct icc_node *src, struct icc_node *dst) { struct qcom_osm_l3_icc_provider *qp; struct icc_provider *provider; - struct qcom_icc_node *qn; + const struct qcom_icc_node *qn; struct icc_node *n; unsigned int index; u32 agg_peak = 0; @@ -124,7 +173,7 @@ static int qcom_icc_set(struct icc_node *src, struct icc_node *dst) break; } - writel_relaxed(index, qp->base + REG_PERF_STATE); + writel_relaxed(index, qp->base + qp->reg_perf_state); return 0; } @@ -145,7 +194,7 @@ static int qcom_osm_l3_probe(struct platform_device *pdev) const struct qcom_icc_desc *desc; struct icc_onecell_data *data; struct icc_provider *provider; - struct qcom_icc_node **qnodes; + const struct qcom_icc_node **qnodes; struct icc_node *node; size_t num_nodes; struct clk *clk; @@ -179,9 +228,15 @@ static int qcom_osm_l3_probe(struct platform_device *pdev) return -ENODEV; } + desc = device_get_match_data(&pdev->dev); + if (!desc) + return -EINVAL; + + qp->reg_perf_state = desc->reg_perf_state; + for (i = 0; i < LUT_MAX_ENTRIES; i++) { - info = readl_relaxed(qp->base + REG_FREQ_LUT + - i * LUT_ROW_SIZE); + info = readl_relaxed(qp->base + desc->reg_freq_lut + + i * desc->lut_row_size); src = FIELD_GET(LUT_SRC, info); lval = FIELD_GET(LUT_L_VAL, info); if (src) @@ -200,10 +255,6 @@ static int qcom_osm_l3_probe(struct platform_device *pdev) } qp->max_state = i; - desc = device_get_match_data(&pdev->dev); - if (!desc) - return -EINVAL; - qnodes = desc->nodes; num_nodes = desc->num_nodes; @@ -235,7 +286,8 @@ static int qcom_osm_l3_probe(struct platform_device *pdev) } node->name = qnodes[i]->name; - node->data = qnodes[i]; + /* Cast away const and add it back in qcom_icc_set() */ + node->data = (void *)qnodes[i]; icc_node_add(node, provider); for (j = 0; j < qnodes[i]->num_links; j++) @@ -258,6 +310,8 @@ err: static const struct of_device_id osm_l3_of_match[] = { { .compatible = "qcom,sc7180-osm-l3", .data = &sc7180_icc_osm_l3 }, { .compatible = "qcom,sdm845-osm-l3", .data = &sdm845_icc_osm_l3 }, + { .compatible = "qcom,sm8150-osm-l3", .data = &sm8150_icc_osm_l3 }, + { .compatible = "qcom,sm8250-epss-l3", .data = &sm8250_icc_epss_l3 }, { } }; MODULE_DEVICE_TABLE(of, osm_l3_of_match); @@ -268,6 +322,7 @@ static struct platform_driver osm_l3_driver = { .driver = { .name = "osm-l3", .of_match_table = osm_l3_of_match, + .sync_state = icc_sync_state, }, }; module_platform_driver(osm_l3_driver); diff --git a/drivers/interconnect/qcom/sc7180.c b/drivers/interconnect/qcom/sc7180.c index dcf493d07928..bf11b82ed55c 100644 --- a/drivers/interconnect/qcom/sc7180.c +++ b/drivers/interconnect/qcom/sc7180.c @@ -535,7 +535,7 @@ static int qnoc_probe(struct platform_device *pdev) provider->set = qcom_icc_set; provider->pre_aggregate = qcom_icc_pre_aggregate; provider->aggregate = qcom_icc_aggregate; - provider->xlate = of_icc_xlate_onecell; + provider->xlate_extended = qcom_icc_xlate_extended; INIT_LIST_HEAD(&provider->nodes); provider->data = data; @@ -633,6 +633,7 @@ static struct platform_driver qnoc_driver = { .driver = { .name = "qnoc-sc7180", .of_match_table = qnoc_of_match, + .sync_state = icc_sync_state, }, }; module_platform_driver(qnoc_driver); diff --git a/drivers/interconnect/qcom/sdm845.c b/drivers/interconnect/qcom/sdm845.c index f6c7b969520d..d79e3163e2c3 100644 --- a/drivers/interconnect/qcom/sdm845.c +++ b/drivers/interconnect/qcom/sdm845.c @@ -469,7 +469,7 @@ static int qnoc_probe(struct platform_device *pdev) provider->set = qcom_icc_set; provider->pre_aggregate = qcom_icc_pre_aggregate; provider->aggregate = qcom_icc_aggregate; - provider->xlate = of_icc_xlate_onecell; + provider->xlate_extended = qcom_icc_xlate_extended; INIT_LIST_HEAD(&provider->nodes); provider->data = data; @@ -559,6 +559,7 @@ static struct platform_driver qnoc_driver = { .driver = { .name = "qnoc-sdm845", .of_match_table = qnoc_of_match, + .sync_state = icc_sync_state, }, }; module_platform_driver(qnoc_driver); diff --git a/drivers/interconnect/qcom/sm8150.c b/drivers/interconnect/qcom/sm8150.c new file mode 100644 index 000000000000..9218efed04a0 --- /dev/null +++ b/drivers/interconnect/qcom/sm8150.c @@ -0,0 +1,635 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + * + */ + +#include <linux/device.h> +#include <linux/interconnect.h> +#include <linux/interconnect-provider.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <dt-bindings/interconnect/qcom,sm8150.h> + +#include "bcm-voter.h" +#include "icc-rpmh.h" +#include "sm8150.h" + +DEFINE_QNODE(qhm_a1noc_cfg, SM8150_MASTER_A1NOC_CFG, 1, 4, SM8150_SLAVE_SERVICE_A1NOC); +DEFINE_QNODE(qhm_qup0, SM8150_MASTER_QUP_0, 1, 4, SM8150_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_emac, SM8150_MASTER_EMAC, 1, 8, SM8150_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_ufs_mem, SM8150_MASTER_UFS_MEM, 1, 8, SM8150_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_usb3_0, SM8150_MASTER_USB3, 1, 8, SM8150_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_usb3_1, SM8150_MASTER_USB3_1, 1, 8, SM8150_A1NOC_SNOC_SLV); +DEFINE_QNODE(qhm_a2noc_cfg, SM8150_MASTER_A2NOC_CFG, 1, 4, SM8150_SLAVE_SERVICE_A2NOC); +DEFINE_QNODE(qhm_qdss_bam, SM8150_MASTER_QDSS_BAM, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qspi, SM8150_MASTER_QSPI, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qup1, SM8150_MASTER_QUP_1, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qup2, SM8150_MASTER_QUP_2, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_sensorss_ahb, SM8150_MASTER_SENSORS_AHB, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_tsif, SM8150_MASTER_TSIF, 1, 4, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qnm_cnoc, SM8150_MASTER_CNOC_A2NOC, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qxm_crypto, SM8150_MASTER_CRYPTO_CORE_0, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qxm_ipa, SM8150_MASTER_IPA, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_pcie3_0, SM8150_MASTER_PCIE, 1, 8, SM8150_SLAVE_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(xm_pcie3_1, SM8150_MASTER_PCIE_1, 1, 8, SM8150_SLAVE_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(xm_qdss_etr, SM8150_MASTER_QDSS_ETR, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_sdc2, SM8150_MASTER_SDCC_2, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_sdc4, SM8150_MASTER_SDCC_4, 1, 8, SM8150_A2NOC_SNOC_SLV); +DEFINE_QNODE(qxm_camnoc_hf0_uncomp, SM8150_MASTER_CAMNOC_HF0_UNCOMP, 1, 32, SM8150_SLAVE_CAMNOC_UNCOMP); +DEFINE_QNODE(qxm_camnoc_hf1_uncomp, SM8150_MASTER_CAMNOC_HF1_UNCOMP, 1, 32, SM8150_SLAVE_CAMNOC_UNCOMP); +DEFINE_QNODE(qxm_camnoc_sf_uncomp, SM8150_MASTER_CAMNOC_SF_UNCOMP, 1, 32, SM8150_SLAVE_CAMNOC_UNCOMP); +DEFINE_QNODE(qnm_npu, SM8150_MASTER_NPU, 1, 32, SM8150_SLAVE_CDSP_MEM_NOC); +DEFINE_QNODE(qhm_spdm, SM8150_MASTER_SPDM, 1, 4, SM8150_SLAVE_CNOC_A2NOC); +DEFINE_QNODE(qnm_snoc, SM8150_SNOC_CNOC_MAS, 1, 8, SM8150_SLAVE_TLMM_SOUTH, SM8150_SLAVE_CDSP_CFG, SM8150_SLAVE_SPSS_CFG, SM8150_SLAVE_CAMERA_CFG, SM8150_SLAVE_SDCC_4, SM8150_SLAVE_SDCC_2, SM8150_SLAVE_CNOC_MNOC_CFG, SM8150_SLAVE_EMAC_CFG, SM8150_SLAVE_UFS_MEM_CFG, SM8150_SLAVE_TLMM_EAST, SM8150_SLAVE_SSC_CFG, SM8150_SLAVE_SNOC_CFG, SM8150_SLAVE_NORTH_PHY_CFG, SM8150_SLAVE_QUP_0, SM8150_SLAVE_GLM, SM8150_SLAVE_PCIE_1_CFG, SM8150_SLAVE_A2NOC_CFG, SM8150_SLAVE_QDSS_CFG, SM8150_SLAVE_DISPLAY_CFG, SM8150_SLAVE_TCSR, SM8150_SLAVE_CNOC_DDRSS, SM8150_SLAVE_RBCPR_MMCX_CFG, SM8150_SLAVE_NPU_CFG, SM8150_SLAVE_PCIE_0_CFG, SM8150_SLAVE_GRAPHICS_3D_CFG, SM8150_SLAVE_VENUS_CFG, SM8150_SLAVE_TSIF, SM8150_SLAVE_IPA_CFG, SM8150_SLAVE_CLK_CTL, SM8150_SLAVE_AOP, SM8150_SLAVE_QUP_1, SM8150_SLAVE_AHB2PHY_SOUTH, SM8150_SLAVE_USB3_1, SM8150_SLAVE_SERVICE_CNOC, SM8150_SLAVE_UFS_CARD_CFG, SM8150_SLAVE_QUP_2, SM8150_SLAVE_RBCPR_CX_CFG, SM8150_SLAVE_TLMM_WEST, SM8150_SLAVE_A1NOC_CFG, SM8150_SLAVE_AOSS, SM8150_SLAVE_PRNG, SM8150_SLAVE_VSENSE_CTRL_CFG, SM8150_SLAVE_QSPI, SM8150_SLAVE_USB3, SM8150_SLAVE_SPDM_WRAPPER, SM8150_SLAVE_CRYPTO_0_CFG, SM8150_SLAVE_PIMEM_CFG, SM8150_SLAVE_TLMM_NORTH, SM8150_SLAVE_RBCPR_MX_CFG, SM8150_SLAVE_IMEM_CFG); +DEFINE_QNODE(xm_qdss_dap, SM8150_MASTER_QDSS_DAP, 1, 8, SM8150_SLAVE_TLMM_SOUTH, SM8150_SLAVE_CDSP_CFG, SM8150_SLAVE_SPSS_CFG, SM8150_SLAVE_CAMERA_CFG, SM8150_SLAVE_SDCC_4, SM8150_SLAVE_SDCC_2, SM8150_SLAVE_CNOC_MNOC_CFG, SM8150_SLAVE_EMAC_CFG, SM8150_SLAVE_UFS_MEM_CFG, SM8150_SLAVE_TLMM_EAST, SM8150_SLAVE_SSC_CFG, SM8150_SLAVE_SNOC_CFG, SM8150_SLAVE_NORTH_PHY_CFG, SM8150_SLAVE_QUP_0, SM8150_SLAVE_GLM, SM8150_SLAVE_PCIE_1_CFG, SM8150_SLAVE_A2NOC_CFG, SM8150_SLAVE_QDSS_CFG, SM8150_SLAVE_DISPLAY_CFG, SM8150_SLAVE_TCSR, SM8150_SLAVE_CNOC_DDRSS, SM8150_SLAVE_CNOC_A2NOC, SM8150_SLAVE_RBCPR_MMCX_CFG, SM8150_SLAVE_NPU_CFG, SM8150_SLAVE_PCIE_0_CFG, SM8150_SLAVE_GRAPHICS_3D_CFG, SM8150_SLAVE_VENUS_CFG, SM8150_SLAVE_TSIF, SM8150_SLAVE_IPA_CFG, SM8150_SLAVE_CLK_CTL, SM8150_SLAVE_AOP, SM8150_SLAVE_QUP_1, SM8150_SLAVE_AHB2PHY_SOUTH, SM8150_SLAVE_USB3_1, SM8150_SLAVE_SERVICE_CNOC, SM8150_SLAVE_UFS_CARD_CFG, SM8150_SLAVE_QUP_2, SM8150_SLAVE_RBCPR_CX_CFG, SM8150_SLAVE_TLMM_WEST, SM8150_SLAVE_A1NOC_CFG, SM8150_SLAVE_AOSS, SM8150_SLAVE_PRNG, SM8150_SLAVE_VSENSE_CTRL_CFG, SM8150_SLAVE_QSPI, SM8150_SLAVE_USB3, SM8150_SLAVE_SPDM_WRAPPER, SM8150_SLAVE_CRYPTO_0_CFG, SM8150_SLAVE_PIMEM_CFG, SM8150_SLAVE_TLMM_NORTH, SM8150_SLAVE_RBCPR_MX_CFG, SM8150_SLAVE_IMEM_CFG); +DEFINE_QNODE(qhm_cnoc_dc_noc, SM8150_MASTER_CNOC_DC_NOC, 1, 4, SM8150_SLAVE_GEM_NOC_CFG, SM8150_SLAVE_LLCC_CFG); +DEFINE_QNODE(acm_apps, SM8150_MASTER_AMPSS_M0, 2, 32, SM8150_SLAVE_ECC, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(acm_gpu_tcu, SM8150_MASTER_GPU_TCU, 1, 8, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(acm_sys_tcu, SM8150_MASTER_SYS_TCU, 1, 8, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qhm_gemnoc_cfg, SM8150_MASTER_GEM_NOC_CFG, 1, 4, SM8150_SLAVE_SERVICE_GEM_NOC, SM8150_SLAVE_MSS_PROC_MS_MPU_CFG); +DEFINE_QNODE(qnm_cmpnoc, SM8150_MASTER_COMPUTE_NOC, 2, 32, SM8150_SLAVE_ECC, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_gpu, SM8150_MASTER_GRAPHICS_3D, 2, 32, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_mnoc_hf, SM8150_MASTER_MNOC_HF_MEM_NOC, 2, 32, SM8150_SLAVE_LLCC); +DEFINE_QNODE(qnm_mnoc_sf, SM8150_MASTER_MNOC_SF_MEM_NOC, 1, 32, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_pcie, SM8150_MASTER_GEM_NOC_PCIE_SNOC, 1, 16, SM8150_SLAVE_LLCC, SM8150_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_snoc_gc, SM8150_MASTER_SNOC_GC_MEM_NOC, 1, 8, SM8150_SLAVE_LLCC); +DEFINE_QNODE(qnm_snoc_sf, SM8150_MASTER_SNOC_SF_MEM_NOC, 1, 16, SM8150_SLAVE_LLCC); +DEFINE_QNODE(qxm_ecc, SM8150_MASTER_ECC, 2, 32, SM8150_SLAVE_LLCC); +DEFINE_QNODE(ipa_core_master, SM8150_MASTER_IPA_CORE, 1, 8, SM8150_SLAVE_IPA_CORE); +DEFINE_QNODE(llcc_mc, SM8150_MASTER_LLCC, 4, 4, SM8150_SLAVE_EBI_CH0); +DEFINE_QNODE(qhm_mnoc_cfg, SM8150_MASTER_CNOC_MNOC_CFG, 1, 4, SM8150_SLAVE_SERVICE_MNOC); +DEFINE_QNODE(qxm_camnoc_hf0, SM8150_MASTER_CAMNOC_HF0, 1, 32, SM8150_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_camnoc_hf1, SM8150_MASTER_CAMNOC_HF1, 1, 32, SM8150_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_camnoc_sf, SM8150_MASTER_CAMNOC_SF, 1, 32, SM8150_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qxm_mdp0, SM8150_MASTER_MDP_PORT0, 1, 32, SM8150_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_mdp1, SM8150_MASTER_MDP_PORT1, 1, 32, SM8150_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_rot, SM8150_MASTER_ROTATOR, 1, 32, SM8150_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qxm_venus0, SM8150_MASTER_VIDEO_P0, 1, 32, SM8150_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qxm_venus1, SM8150_MASTER_VIDEO_P1, 1, 32, SM8150_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qxm_venus_arm9, SM8150_MASTER_VIDEO_PROC, 1, 8, SM8150_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qhm_snoc_cfg, SM8150_MASTER_SNOC_CFG, 1, 4, SM8150_SLAVE_SERVICE_SNOC); +DEFINE_QNODE(qnm_aggre1_noc, SM8150_A1NOC_SNOC_MAS, 1, 16, SM8150_SLAVE_SNOC_GEM_NOC_SF, SM8150_SLAVE_PIMEM, SM8150_SLAVE_OCIMEM, SM8150_SLAVE_APPSS, SM8150_SNOC_CNOC_SLV, SM8150_SLAVE_QDSS_STM); +DEFINE_QNODE(qnm_aggre2_noc, SM8150_A2NOC_SNOC_MAS, 1, 16, SM8150_SLAVE_SNOC_GEM_NOC_SF, SM8150_SLAVE_PIMEM, SM8150_SLAVE_OCIMEM, SM8150_SLAVE_APPSS, SM8150_SNOC_CNOC_SLV, SM8150_SLAVE_PCIE_0, SM8150_SLAVE_PCIE_1, SM8150_SLAVE_TCU, SM8150_SLAVE_QDSS_STM); +DEFINE_QNODE(qnm_gemnoc, SM8150_MASTER_GEM_NOC_SNOC, 1, 8, SM8150_SLAVE_PIMEM, SM8150_SLAVE_OCIMEM, SM8150_SLAVE_APPSS, SM8150_SNOC_CNOC_SLV, SM8150_SLAVE_TCU, SM8150_SLAVE_QDSS_STM); +DEFINE_QNODE(qxm_pimem, SM8150_MASTER_PIMEM, 1, 8, SM8150_SLAVE_SNOC_GEM_NOC_GC, SM8150_SLAVE_OCIMEM); +DEFINE_QNODE(xm_gic, SM8150_MASTER_GIC, 1, 8, SM8150_SLAVE_SNOC_GEM_NOC_GC, SM8150_SLAVE_OCIMEM); +DEFINE_QNODE(qns_a1noc_snoc, SM8150_A1NOC_SNOC_SLV, 1, 16, SM8150_A1NOC_SNOC_MAS); +DEFINE_QNODE(srvc_aggre1_noc, SM8150_SLAVE_SERVICE_A1NOC, 1, 4); +DEFINE_QNODE(qns_a2noc_snoc, SM8150_A2NOC_SNOC_SLV, 1, 16, SM8150_A2NOC_SNOC_MAS); +DEFINE_QNODE(qns_pcie_mem_noc, SM8150_SLAVE_ANOC_PCIE_GEM_NOC, 1, 16, SM8150_MASTER_GEM_NOC_PCIE_SNOC); +DEFINE_QNODE(srvc_aggre2_noc, SM8150_SLAVE_SERVICE_A2NOC, 1, 4); +DEFINE_QNODE(qns_camnoc_uncomp, SM8150_SLAVE_CAMNOC_UNCOMP, 1, 32); +DEFINE_QNODE(qns_cdsp_mem_noc, SM8150_SLAVE_CDSP_MEM_NOC, 2, 32, SM8150_MASTER_COMPUTE_NOC); +DEFINE_QNODE(qhs_a1_noc_cfg, SM8150_SLAVE_A1NOC_CFG, 1, 4, SM8150_MASTER_A1NOC_CFG); +DEFINE_QNODE(qhs_a2_noc_cfg, SM8150_SLAVE_A2NOC_CFG, 1, 4, SM8150_MASTER_A2NOC_CFG); +DEFINE_QNODE(qhs_ahb2phy_south, SM8150_SLAVE_AHB2PHY_SOUTH, 1, 4); +DEFINE_QNODE(qhs_aop, SM8150_SLAVE_AOP, 1, 4); +DEFINE_QNODE(qhs_aoss, SM8150_SLAVE_AOSS, 1, 4); +DEFINE_QNODE(qhs_camera_cfg, SM8150_SLAVE_CAMERA_CFG, 1, 4); +DEFINE_QNODE(qhs_clk_ctl, SM8150_SLAVE_CLK_CTL, 1, 4); +DEFINE_QNODE(qhs_compute_dsp, SM8150_SLAVE_CDSP_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_cx, SM8150_SLAVE_RBCPR_CX_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_mmcx, SM8150_SLAVE_RBCPR_MMCX_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_mx, SM8150_SLAVE_RBCPR_MX_CFG, 1, 4); +DEFINE_QNODE(qhs_crypto0_cfg, SM8150_SLAVE_CRYPTO_0_CFG, 1, 4); +DEFINE_QNODE(qhs_ddrss_cfg, SM8150_SLAVE_CNOC_DDRSS, 1, 4, SM8150_MASTER_CNOC_DC_NOC); +DEFINE_QNODE(qhs_display_cfg, SM8150_SLAVE_DISPLAY_CFG, 1, 4); +DEFINE_QNODE(qhs_emac_cfg, SM8150_SLAVE_EMAC_CFG, 1, 4); +DEFINE_QNODE(qhs_glm, SM8150_SLAVE_GLM, 1, 4); +DEFINE_QNODE(qhs_gpuss_cfg, SM8150_SLAVE_GRAPHICS_3D_CFG, 1, 8); +DEFINE_QNODE(qhs_imem_cfg, SM8150_SLAVE_IMEM_CFG, 1, 4); +DEFINE_QNODE(qhs_ipa, SM8150_SLAVE_IPA_CFG, 1, 4); +DEFINE_QNODE(qhs_mnoc_cfg, SM8150_SLAVE_CNOC_MNOC_CFG, 1, 4, SM8150_MASTER_CNOC_MNOC_CFG); +DEFINE_QNODE(qhs_npu_cfg, SM8150_SLAVE_NPU_CFG, 1, 4); +DEFINE_QNODE(qhs_pcie0_cfg, SM8150_SLAVE_PCIE_0_CFG, 1, 4); +DEFINE_QNODE(qhs_pcie1_cfg, SM8150_SLAVE_PCIE_1_CFG, 1, 4); +DEFINE_QNODE(qhs_phy_refgen_north, SM8150_SLAVE_NORTH_PHY_CFG, 1, 4); +DEFINE_QNODE(qhs_pimem_cfg, SM8150_SLAVE_PIMEM_CFG, 1, 4); +DEFINE_QNODE(qhs_prng, SM8150_SLAVE_PRNG, 1, 4); +DEFINE_QNODE(qhs_qdss_cfg, SM8150_SLAVE_QDSS_CFG, 1, 4); +DEFINE_QNODE(qhs_qspi, SM8150_SLAVE_QSPI, 1, 4); +DEFINE_QNODE(qhs_qupv3_east, SM8150_SLAVE_QUP_2, 1, 4); +DEFINE_QNODE(qhs_qupv3_north, SM8150_SLAVE_QUP_1, 1, 4); +DEFINE_QNODE(qhs_qupv3_south, SM8150_SLAVE_QUP_0, 1, 4); +DEFINE_QNODE(qhs_sdc2, SM8150_SLAVE_SDCC_2, 1, 4); +DEFINE_QNODE(qhs_sdc4, SM8150_SLAVE_SDCC_4, 1, 4); +DEFINE_QNODE(qhs_snoc_cfg, SM8150_SLAVE_SNOC_CFG, 1, 4, SM8150_MASTER_SNOC_CFG); +DEFINE_QNODE(qhs_spdm, SM8150_SLAVE_SPDM_WRAPPER, 1, 4); +DEFINE_QNODE(qhs_spss_cfg, SM8150_SLAVE_SPSS_CFG, 1, 4); +DEFINE_QNODE(qhs_ssc_cfg, SM8150_SLAVE_SSC_CFG, 1, 4); +DEFINE_QNODE(qhs_tcsr, SM8150_SLAVE_TCSR, 1, 4); +DEFINE_QNODE(qhs_tlmm_east, SM8150_SLAVE_TLMM_EAST, 1, 4); +DEFINE_QNODE(qhs_tlmm_north, SM8150_SLAVE_TLMM_NORTH, 1, 4); +DEFINE_QNODE(qhs_tlmm_south, SM8150_SLAVE_TLMM_SOUTH, 1, 4); +DEFINE_QNODE(qhs_tlmm_west, SM8150_SLAVE_TLMM_WEST, 1, 4); +DEFINE_QNODE(qhs_tsif, SM8150_SLAVE_TSIF, 1, 4); +DEFINE_QNODE(qhs_ufs_card_cfg, SM8150_SLAVE_UFS_CARD_CFG, 1, 4); +DEFINE_QNODE(qhs_ufs_mem_cfg, SM8150_SLAVE_UFS_MEM_CFG, 1, 4); +DEFINE_QNODE(qhs_usb3_0, SM8150_SLAVE_USB3, 1, 4); +DEFINE_QNODE(qhs_usb3_1, SM8150_SLAVE_USB3_1, 1, 4); +DEFINE_QNODE(qhs_venus_cfg, SM8150_SLAVE_VENUS_CFG, 1, 4); +DEFINE_QNODE(qhs_vsense_ctrl_cfg, SM8150_SLAVE_VSENSE_CTRL_CFG, 1, 4); +DEFINE_QNODE(qns_cnoc_a2noc, SM8150_SLAVE_CNOC_A2NOC, 1, 8, SM8150_MASTER_CNOC_A2NOC); +DEFINE_QNODE(srvc_cnoc, SM8150_SLAVE_SERVICE_CNOC, 1, 4); +DEFINE_QNODE(qhs_llcc, SM8150_SLAVE_LLCC_CFG, 1, 4); +DEFINE_QNODE(qhs_memnoc, SM8150_SLAVE_GEM_NOC_CFG, 1, 4, SM8150_MASTER_GEM_NOC_CFG); +DEFINE_QNODE(qhs_mdsp_ms_mpu_cfg, SM8150_SLAVE_MSS_PROC_MS_MPU_CFG, 1, 4); +DEFINE_QNODE(qns_ecc, SM8150_SLAVE_ECC, 1, 32); +DEFINE_QNODE(qns_gem_noc_snoc, SM8150_SLAVE_GEM_NOC_SNOC, 1, 8, SM8150_MASTER_GEM_NOC_SNOC); +DEFINE_QNODE(qns_llcc, SM8150_SLAVE_LLCC, 4, 16, SM8150_MASTER_LLCC); +DEFINE_QNODE(srvc_gemnoc, SM8150_SLAVE_SERVICE_GEM_NOC, 1, 4); +DEFINE_QNODE(ipa_core_slave, SM8150_SLAVE_IPA_CORE, 1, 8); +DEFINE_QNODE(ebi, SM8150_SLAVE_EBI_CH0, 4, 4); +DEFINE_QNODE(qns2_mem_noc, SM8150_SLAVE_MNOC_SF_MEM_NOC, 1, 32, SM8150_MASTER_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qns_mem_noc_hf, SM8150_SLAVE_MNOC_HF_MEM_NOC, 2, 32, SM8150_MASTER_MNOC_HF_MEM_NOC); +DEFINE_QNODE(srvc_mnoc, SM8150_SLAVE_SERVICE_MNOC, 1, 4); +DEFINE_QNODE(qhs_apss, SM8150_SLAVE_APPSS, 1, 8); +DEFINE_QNODE(qns_cnoc, SM8150_SNOC_CNOC_SLV, 1, 8, SM8150_SNOC_CNOC_MAS); +DEFINE_QNODE(qns_gemnoc_gc, SM8150_SLAVE_SNOC_GEM_NOC_GC, 1, 8, SM8150_MASTER_SNOC_GC_MEM_NOC); +DEFINE_QNODE(qns_gemnoc_sf, SM8150_SLAVE_SNOC_GEM_NOC_SF, 1, 16, SM8150_MASTER_SNOC_SF_MEM_NOC); +DEFINE_QNODE(qxs_imem, SM8150_SLAVE_OCIMEM, 1, 8); +DEFINE_QNODE(qxs_pimem, SM8150_SLAVE_PIMEM, 1, 8); +DEFINE_QNODE(srvc_snoc, SM8150_SLAVE_SERVICE_SNOC, 1, 4); +DEFINE_QNODE(xs_pcie_0, SM8150_SLAVE_PCIE_0, 1, 8); +DEFINE_QNODE(xs_pcie_1, SM8150_SLAVE_PCIE_1, 1, 8); +DEFINE_QNODE(xs_qdss_stm, SM8150_SLAVE_QDSS_STM, 1, 4); +DEFINE_QNODE(xs_sys_tcu_cfg, SM8150_SLAVE_TCU, 1, 8); + +DEFINE_QBCM(bcm_acv, "ACV", false, &ebi); +DEFINE_QBCM(bcm_mc0, "MC0", true, &ebi); +DEFINE_QBCM(bcm_sh0, "SH0", true, &qns_llcc); +DEFINE_QBCM(bcm_mm0, "MM0", true, &qns_mem_noc_hf); +DEFINE_QBCM(bcm_mm1, "MM1", false, &qxm_camnoc_hf0_uncomp, &qxm_camnoc_hf1_uncomp, &qxm_camnoc_sf_uncomp, &qxm_camnoc_hf0, &qxm_camnoc_hf1, &qxm_mdp0, &qxm_mdp1); +DEFINE_QBCM(bcm_sh2, "SH2", false, &qns_gem_noc_snoc); +DEFINE_QBCM(bcm_mm2, "MM2", false, &qxm_camnoc_sf, &qns2_mem_noc); +DEFINE_QBCM(bcm_sh3, "SH3", false, &acm_gpu_tcu, &acm_sys_tcu); +DEFINE_QBCM(bcm_mm3, "MM3", false, &qxm_rot, &qxm_venus0, &qxm_venus1, &qxm_venus_arm9); +DEFINE_QBCM(bcm_sh4, "SH4", false, &qnm_cmpnoc); +DEFINE_QBCM(bcm_sh5, "SH5", false, &acm_apps); +DEFINE_QBCM(bcm_sn0, "SN0", true, &qns_gemnoc_sf); +DEFINE_QBCM(bcm_co0, "CO0", false, &qns_cdsp_mem_noc); +DEFINE_QBCM(bcm_ce0, "CE0", false, &qxm_crypto); +DEFINE_QBCM(bcm_sn1, "SN1", false, &qxs_imem); +DEFINE_QBCM(bcm_co1, "CO1", false, &qnm_npu); +DEFINE_QBCM(bcm_ip0, "IP0", false, &ipa_core_slave); +DEFINE_QBCM(bcm_cn0, "CN0", true, &qhm_spdm, &qnm_snoc, &qhs_a1_noc_cfg, &qhs_a2_noc_cfg, &qhs_ahb2phy_south, &qhs_aop, &qhs_aoss, &qhs_camera_cfg, &qhs_clk_ctl, &qhs_compute_dsp, &qhs_cpr_cx, &qhs_cpr_mmcx, &qhs_cpr_mx, &qhs_crypto0_cfg, &qhs_ddrss_cfg, &qhs_display_cfg, &qhs_emac_cfg, &qhs_glm, &qhs_gpuss_cfg, &qhs_imem_cfg, &qhs_ipa, &qhs_mnoc_cfg, &qhs_npu_cfg, &qhs_pcie0_cfg, &qhs_pcie1_cfg, &qhs_phy_refgen_north, &qhs_pimem_cfg, &qhs_prng, &qhs_qdss_cfg, &qhs_qspi, &qhs_qupv3_east, &qhs_qupv3_north, &qhs_qupv3_south, &qhs_sdc2, &qhs_sdc4, &qhs_snoc_cfg, &qhs_spdm, &qhs_spss_cfg, &qhs_ssc_cfg, &qhs_tcsr, &qhs_tlmm_east, &qhs_tlmm_north, &qhs_tlmm_south, &qhs_tlmm_west, &qhs_tsif, &qhs_ufs_card_cfg, &qhs_ufs_mem_cfg, &qhs_usb3_0, &qhs_usb3_1, &qhs_venus_cfg, &qhs_vsense_ctrl_cfg, &qns_cnoc_a2noc, &srvc_cnoc); +DEFINE_QBCM(bcm_qup0, "QUP0", false, &qhm_qup0, &qhm_qup1, &qhm_qup2); +DEFINE_QBCM(bcm_sn2, "SN2", false, &qns_gemnoc_gc); +DEFINE_QBCM(bcm_sn3, "SN3", false, &srvc_aggre1_noc, &srvc_aggre2_noc, &qns_cnoc); +DEFINE_QBCM(bcm_sn4, "SN4", false, &qxs_pimem); +DEFINE_QBCM(bcm_sn5, "SN5", false, &xs_qdss_stm); +DEFINE_QBCM(bcm_sn8, "SN8", false, &xs_pcie_0, &xs_pcie_1); +DEFINE_QBCM(bcm_sn9, "SN9", false, &qnm_aggre1_noc); +DEFINE_QBCM(bcm_sn11, "SN11", false, &qnm_aggre2_noc); +DEFINE_QBCM(bcm_sn12, "SN12", false, &qxm_pimem, &xm_gic); +DEFINE_QBCM(bcm_sn14, "SN14", false, &qns_pcie_mem_noc); +DEFINE_QBCM(bcm_sn15, "SN15", false, &qnm_gemnoc); + +static struct qcom_icc_bcm *aggre1_noc_bcms[] = { + &bcm_qup0, + &bcm_sn3, +}; + +static struct qcom_icc_node *aggre1_noc_nodes[] = { + [MASTER_A1NOC_CFG] = &qhm_a1noc_cfg, + [MASTER_QUP_0] = &qhm_qup0, + [MASTER_EMAC] = &xm_emac, + [MASTER_UFS_MEM] = &xm_ufs_mem, + [MASTER_USB3] = &xm_usb3_0, + [MASTER_USB3_1] = &xm_usb3_1, + [A1NOC_SNOC_SLV] = &qns_a1noc_snoc, + [SLAVE_SERVICE_A1NOC] = &srvc_aggre1_noc, +}; + +static struct qcom_icc_desc sm8150_aggre1_noc = { + .nodes = aggre1_noc_nodes, + .num_nodes = ARRAY_SIZE(aggre1_noc_nodes), + .bcms = aggre1_noc_bcms, + .num_bcms = ARRAY_SIZE(aggre1_noc_bcms), +}; + +static struct qcom_icc_bcm *aggre2_noc_bcms[] = { + &bcm_ce0, + &bcm_qup0, + &bcm_sn14, + &bcm_sn3, +}; + +static struct qcom_icc_node *aggre2_noc_nodes[] = { + [MASTER_A2NOC_CFG] = &qhm_a2noc_cfg, + [MASTER_QDSS_BAM] = &qhm_qdss_bam, + [MASTER_QSPI] = &qhm_qspi, + [MASTER_QUP_1] = &qhm_qup1, + [MASTER_QUP_2] = &qhm_qup2, + [MASTER_SENSORS_AHB] = &qhm_sensorss_ahb, + [MASTER_TSIF] = &qhm_tsif, + [MASTER_CNOC_A2NOC] = &qnm_cnoc, + [MASTER_CRYPTO_CORE_0] = &qxm_crypto, + [MASTER_IPA] = &qxm_ipa, + [MASTER_PCIE] = &xm_pcie3_0, + [MASTER_PCIE_1] = &xm_pcie3_1, + [MASTER_QDSS_ETR] = &xm_qdss_etr, + [MASTER_SDCC_2] = &xm_sdc2, + [MASTER_SDCC_4] = &xm_sdc4, + [A2NOC_SNOC_SLV] = &qns_a2noc_snoc, + [SLAVE_ANOC_PCIE_GEM_NOC] = &qns_pcie_mem_noc, + [SLAVE_SERVICE_A2NOC] = &srvc_aggre2_noc, +}; + +static struct qcom_icc_desc sm8150_aggre2_noc = { + .nodes = aggre2_noc_nodes, + .num_nodes = ARRAY_SIZE(aggre2_noc_nodes), + .bcms = aggre2_noc_bcms, + .num_bcms = ARRAY_SIZE(aggre2_noc_bcms), +}; + +static struct qcom_icc_bcm *camnoc_virt_bcms[] = { + &bcm_mm1, +}; + +static struct qcom_icc_node *camnoc_virt_nodes[] = { + [MASTER_CAMNOC_HF0_UNCOMP] = &qxm_camnoc_hf0_uncomp, + [MASTER_CAMNOC_HF1_UNCOMP] = &qxm_camnoc_hf1_uncomp, + [MASTER_CAMNOC_SF_UNCOMP] = &qxm_camnoc_sf_uncomp, + [SLAVE_CAMNOC_UNCOMP] = &qns_camnoc_uncomp, +}; + +static struct qcom_icc_desc sm8150_camnoc_virt = { + .nodes = camnoc_virt_nodes, + .num_nodes = ARRAY_SIZE(camnoc_virt_nodes), + .bcms = camnoc_virt_bcms, + .num_bcms = ARRAY_SIZE(camnoc_virt_bcms), +}; + +static struct qcom_icc_bcm *compute_noc_bcms[] = { + &bcm_co0, + &bcm_co1, +}; + +static struct qcom_icc_node *compute_noc_nodes[] = { + [MASTER_NPU] = &qnm_npu, + [SLAVE_CDSP_MEM_NOC] = &qns_cdsp_mem_noc, +}; + +static struct qcom_icc_desc sm8150_compute_noc = { + .nodes = compute_noc_nodes, + .num_nodes = ARRAY_SIZE(compute_noc_nodes), + .bcms = compute_noc_bcms, + .num_bcms = ARRAY_SIZE(compute_noc_bcms), +}; + +static struct qcom_icc_bcm *config_noc_bcms[] = { + &bcm_cn0, +}; + +static struct qcom_icc_node *config_noc_nodes[] = { + [MASTER_SPDM] = &qhm_spdm, + [SNOC_CNOC_MAS] = &qnm_snoc, + [MASTER_QDSS_DAP] = &xm_qdss_dap, + [SLAVE_A1NOC_CFG] = &qhs_a1_noc_cfg, + [SLAVE_A2NOC_CFG] = &qhs_a2_noc_cfg, + [SLAVE_AHB2PHY_SOUTH] = &qhs_ahb2phy_south, + [SLAVE_AOP] = &qhs_aop, + [SLAVE_AOSS] = &qhs_aoss, + [SLAVE_CAMERA_CFG] = &qhs_camera_cfg, + [SLAVE_CLK_CTL] = &qhs_clk_ctl, + [SLAVE_CDSP_CFG] = &qhs_compute_dsp, + [SLAVE_RBCPR_CX_CFG] = &qhs_cpr_cx, + [SLAVE_RBCPR_MMCX_CFG] = &qhs_cpr_mmcx, + [SLAVE_RBCPR_MX_CFG] = &qhs_cpr_mx, + [SLAVE_CRYPTO_0_CFG] = &qhs_crypto0_cfg, + [SLAVE_CNOC_DDRSS] = &qhs_ddrss_cfg, + [SLAVE_DISPLAY_CFG] = &qhs_display_cfg, + [SLAVE_EMAC_CFG] = &qhs_emac_cfg, + [SLAVE_GLM] = &qhs_glm, + [SLAVE_GRAPHICS_3D_CFG] = &qhs_gpuss_cfg, + [SLAVE_IMEM_CFG] = &qhs_imem_cfg, + [SLAVE_IPA_CFG] = &qhs_ipa, + [SLAVE_CNOC_MNOC_CFG] = &qhs_mnoc_cfg, + [SLAVE_NPU_CFG] = &qhs_npu_cfg, + [SLAVE_PCIE_0_CFG] = &qhs_pcie0_cfg, + [SLAVE_PCIE_1_CFG] = &qhs_pcie1_cfg, + [SLAVE_NORTH_PHY_CFG] = &qhs_phy_refgen_north, + [SLAVE_PIMEM_CFG] = &qhs_pimem_cfg, + [SLAVE_PRNG] = &qhs_prng, + [SLAVE_QDSS_CFG] = &qhs_qdss_cfg, + [SLAVE_QSPI] = &qhs_qspi, + [SLAVE_QUP_2] = &qhs_qupv3_east, + [SLAVE_QUP_1] = &qhs_qupv3_north, + [SLAVE_QUP_0] = &qhs_qupv3_south, + [SLAVE_SDCC_2] = &qhs_sdc2, + [SLAVE_SDCC_4] = &qhs_sdc4, + [SLAVE_SNOC_CFG] = &qhs_snoc_cfg, + [SLAVE_SPDM_WRAPPER] = &qhs_spdm, + [SLAVE_SPSS_CFG] = &qhs_spss_cfg, + [SLAVE_SSC_CFG] = &qhs_ssc_cfg, + [SLAVE_TCSR] = &qhs_tcsr, + [SLAVE_TLMM_EAST] = &qhs_tlmm_east, + [SLAVE_TLMM_NORTH] = &qhs_tlmm_north, + [SLAVE_TLMM_SOUTH] = &qhs_tlmm_south, + [SLAVE_TLMM_WEST] = &qhs_tlmm_west, + [SLAVE_TSIF] = &qhs_tsif, + [SLAVE_UFS_CARD_CFG] = &qhs_ufs_card_cfg, + [SLAVE_UFS_MEM_CFG] = &qhs_ufs_mem_cfg, + [SLAVE_USB3] = &qhs_usb3_0, + [SLAVE_USB3_1] = &qhs_usb3_1, + [SLAVE_VENUS_CFG] = &qhs_venus_cfg, + [SLAVE_VSENSE_CTRL_CFG] = &qhs_vsense_ctrl_cfg, + [SLAVE_CNOC_A2NOC] = &qns_cnoc_a2noc, + [SLAVE_SERVICE_CNOC] = &srvc_cnoc, +}; + +static struct qcom_icc_desc sm8150_config_noc = { + .nodes = config_noc_nodes, + .num_nodes = ARRAY_SIZE(config_noc_nodes), + .bcms = config_noc_bcms, + .num_bcms = ARRAY_SIZE(config_noc_bcms), +}; + +static struct qcom_icc_bcm *dc_noc_bcms[] = { +}; + +static struct qcom_icc_node *dc_noc_nodes[] = { + [MASTER_CNOC_DC_NOC] = &qhm_cnoc_dc_noc, + [SLAVE_LLCC_CFG] = &qhs_llcc, + [SLAVE_GEM_NOC_CFG] = &qhs_memnoc, +}; + +static struct qcom_icc_desc sm8150_dc_noc = { + .nodes = dc_noc_nodes, + .num_nodes = ARRAY_SIZE(dc_noc_nodes), + .bcms = dc_noc_bcms, + .num_bcms = ARRAY_SIZE(dc_noc_bcms), +}; + +static struct qcom_icc_bcm *gem_noc_bcms[] = { + &bcm_sh0, + &bcm_sh2, + &bcm_sh3, + &bcm_sh4, + &bcm_sh5, +}; + +static struct qcom_icc_node *gem_noc_nodes[] = { + [MASTER_AMPSS_M0] = &acm_apps, + [MASTER_GPU_TCU] = &acm_gpu_tcu, + [MASTER_SYS_TCU] = &acm_sys_tcu, + [MASTER_GEM_NOC_CFG] = &qhm_gemnoc_cfg, + [MASTER_COMPUTE_NOC] = &qnm_cmpnoc, + [MASTER_GRAPHICS_3D] = &qnm_gpu, + [MASTER_MNOC_HF_MEM_NOC] = &qnm_mnoc_hf, + [MASTER_MNOC_SF_MEM_NOC] = &qnm_mnoc_sf, + [MASTER_GEM_NOC_PCIE_SNOC] = &qnm_pcie, + [MASTER_SNOC_GC_MEM_NOC] = &qnm_snoc_gc, + [MASTER_SNOC_SF_MEM_NOC] = &qnm_snoc_sf, + [MASTER_ECC] = &qxm_ecc, + [SLAVE_MSS_PROC_MS_MPU_CFG] = &qhs_mdsp_ms_mpu_cfg, + [SLAVE_ECC] = &qns_ecc, + [SLAVE_GEM_NOC_SNOC] = &qns_gem_noc_snoc, + [SLAVE_LLCC] = &qns_llcc, + [SLAVE_SERVICE_GEM_NOC] = &srvc_gemnoc, +}; + +static struct qcom_icc_desc sm8150_gem_noc = { + .nodes = gem_noc_nodes, + .num_nodes = ARRAY_SIZE(gem_noc_nodes), + .bcms = gem_noc_bcms, + .num_bcms = ARRAY_SIZE(gem_noc_bcms), +}; + +static struct qcom_icc_bcm *ipa_virt_bcms[] = { + &bcm_ip0, +}; + +static struct qcom_icc_node *ipa_virt_nodes[] = { + [MASTER_IPA_CORE] = &ipa_core_master, + [SLAVE_IPA_CORE] = &ipa_core_slave, +}; + +static struct qcom_icc_desc sm8150_ipa_virt = { + .nodes = ipa_virt_nodes, + .num_nodes = ARRAY_SIZE(ipa_virt_nodes), + .bcms = ipa_virt_bcms, + .num_bcms = ARRAY_SIZE(ipa_virt_bcms), +}; + +static struct qcom_icc_bcm *mc_virt_bcms[] = { + &bcm_acv, + &bcm_mc0, +}; + +static struct qcom_icc_node *mc_virt_nodes[] = { + [MASTER_LLCC] = &llcc_mc, + [SLAVE_EBI_CH0] = &ebi, +}; + +static struct qcom_icc_desc sm8150_mc_virt = { + .nodes = mc_virt_nodes, + .num_nodes = ARRAY_SIZE(mc_virt_nodes), + .bcms = mc_virt_bcms, + .num_bcms = ARRAY_SIZE(mc_virt_bcms), +}; + +static struct qcom_icc_bcm *mmss_noc_bcms[] = { + &bcm_mm0, + &bcm_mm1, + &bcm_mm2, + &bcm_mm3, +}; + +static struct qcom_icc_node *mmss_noc_nodes[] = { + [MASTER_CNOC_MNOC_CFG] = &qhm_mnoc_cfg, + [MASTER_CAMNOC_HF0] = &qxm_camnoc_hf0, + [MASTER_CAMNOC_HF1] = &qxm_camnoc_hf1, + [MASTER_CAMNOC_SF] = &qxm_camnoc_sf, + [MASTER_MDP_PORT0] = &qxm_mdp0, + [MASTER_MDP_PORT1] = &qxm_mdp1, + [MASTER_ROTATOR] = &qxm_rot, + [MASTER_VIDEO_P0] = &qxm_venus0, + [MASTER_VIDEO_P1] = &qxm_venus1, + [MASTER_VIDEO_PROC] = &qxm_venus_arm9, + [SLAVE_MNOC_SF_MEM_NOC] = &qns2_mem_noc, + [SLAVE_MNOC_HF_MEM_NOC] = &qns_mem_noc_hf, + [SLAVE_SERVICE_MNOC] = &srvc_mnoc, +}; + +static struct qcom_icc_desc sm8150_mmss_noc = { + .nodes = mmss_noc_nodes, + .num_nodes = ARRAY_SIZE(mmss_noc_nodes), + .bcms = mmss_noc_bcms, + .num_bcms = ARRAY_SIZE(mmss_noc_bcms), +}; + +static struct qcom_icc_bcm *system_noc_bcms[] = { + &bcm_sn0, + &bcm_sn1, + &bcm_sn11, + &bcm_sn12, + &bcm_sn15, + &bcm_sn2, + &bcm_sn3, + &bcm_sn4, + &bcm_sn5, + &bcm_sn8, + &bcm_sn9, +}; + +static struct qcom_icc_node *system_noc_nodes[] = { + [MASTER_SNOC_CFG] = &qhm_snoc_cfg, + [A1NOC_SNOC_MAS] = &qnm_aggre1_noc, + [A2NOC_SNOC_MAS] = &qnm_aggre2_noc, + [MASTER_GEM_NOC_SNOC] = &qnm_gemnoc, + [MASTER_PIMEM] = &qxm_pimem, + [MASTER_GIC] = &xm_gic, + [SLAVE_APPSS] = &qhs_apss, + [SNOC_CNOC_SLV] = &qns_cnoc, + [SLAVE_SNOC_GEM_NOC_GC] = &qns_gemnoc_gc, + [SLAVE_SNOC_GEM_NOC_SF] = &qns_gemnoc_sf, + [SLAVE_OCIMEM] = &qxs_imem, + [SLAVE_PIMEM] = &qxs_pimem, + [SLAVE_SERVICE_SNOC] = &srvc_snoc, + [SLAVE_PCIE_0] = &xs_pcie_0, + [SLAVE_PCIE_1] = &xs_pcie_1, + [SLAVE_QDSS_STM] = &xs_qdss_stm, + [SLAVE_TCU] = &xs_sys_tcu_cfg, +}; + +static struct qcom_icc_desc sm8150_system_noc = { + .nodes = system_noc_nodes, + .num_nodes = ARRAY_SIZE(system_noc_nodes), + .bcms = system_noc_bcms, + .num_bcms = ARRAY_SIZE(system_noc_bcms), +}; + +static int qnoc_probe(struct platform_device *pdev) +{ + const struct qcom_icc_desc *desc; + struct icc_onecell_data *data; + struct icc_provider *provider; + struct qcom_icc_node **qnodes; + struct qcom_icc_provider *qp; + struct icc_node *node; + size_t num_nodes, i; + int ret; + + desc = device_get_match_data(&pdev->dev); + if (!desc) + return -EINVAL; + + qnodes = desc->nodes; + num_nodes = desc->num_nodes; + + qp = devm_kzalloc(&pdev->dev, sizeof(*qp), GFP_KERNEL); + if (!qp) + return -ENOMEM; + + data = devm_kcalloc(&pdev->dev, num_nodes, sizeof(*node), GFP_KERNEL); + if (!data) + return -ENOMEM; + + provider = &qp->provider; + provider->dev = &pdev->dev; + provider->set = qcom_icc_set; + provider->pre_aggregate = qcom_icc_pre_aggregate; + provider->aggregate = qcom_icc_aggregate; + provider->xlate = of_icc_xlate_onecell; + INIT_LIST_HEAD(&provider->nodes); + provider->data = data; + + qp->dev = &pdev->dev; + qp->bcms = desc->bcms; + qp->num_bcms = desc->num_bcms; + + qp->voter = of_bcm_voter_get(qp->dev, NULL); + if (IS_ERR(qp->voter)) + return PTR_ERR(qp->voter); + + ret = icc_provider_add(provider); + if (ret) { + dev_err(&pdev->dev, "error adding interconnect provider\n"); + return ret; + } + + for (i = 0; i < num_nodes; i++) { + size_t j; + + if (!qnodes[i]) + continue; + + node = icc_node_create(qnodes[i]->id); + if (IS_ERR(node)) { + ret = PTR_ERR(node); + goto err; + } + + node->name = qnodes[i]->name; + node->data = qnodes[i]; + icc_node_add(node, provider); + + for (j = 0; j < qnodes[i]->num_links; j++) + icc_link_create(node, qnodes[i]->links[j]); + + data->nodes[i] = node; + } + data->num_nodes = num_nodes; + + for (i = 0; i < qp->num_bcms; i++) + qcom_icc_bcm_init(qp->bcms[i], &pdev->dev); + + platform_set_drvdata(pdev, qp); + + return 0; +err: + icc_nodes_remove(provider); + icc_provider_del(provider); + return ret; +} + +static int qnoc_remove(struct platform_device *pdev) +{ + struct qcom_icc_provider *qp = platform_get_drvdata(pdev); + + icc_nodes_remove(&qp->provider); + return icc_provider_del(&qp->provider); +} + +static const struct of_device_id qnoc_of_match[] = { + { .compatible = "qcom,sm8150-aggre1-noc", + .data = &sm8150_aggre1_noc}, + { .compatible = "qcom,sm8150-aggre2-noc", + .data = &sm8150_aggre2_noc}, + { .compatible = "qcom,sm8150-camnoc-virt", + .data = &sm8150_camnoc_virt}, + { .compatible = "qcom,sm8150-compute-noc", + .data = &sm8150_compute_noc}, + { .compatible = "qcom,sm8150-config-noc", + .data = &sm8150_config_noc}, + { .compatible = "qcom,sm8150-dc-noc", + .data = &sm8150_dc_noc}, + { .compatible = "qcom,sm8150-gem-noc", + .data = &sm8150_gem_noc}, + { .compatible = "qcom,sm8150-ipa-virt", + .data = &sm8150_ipa_virt}, + { .compatible = "qcom,sm8150-mc-virt", + .data = &sm8150_mc_virt}, + { .compatible = "qcom,sm8150-mmss-noc", + .data = &sm8150_mmss_noc}, + { .compatible = "qcom,sm8150-system-noc", + .data = &sm8150_system_noc}, + { } +}; +MODULE_DEVICE_TABLE(of, qnoc_of_match); + +static struct platform_driver qnoc_driver = { + .probe = qnoc_probe, + .remove = qnoc_remove, + .driver = { + .name = "qnoc-sm8150", + .of_match_table = qnoc_of_match, + }, +}; +module_platform_driver(qnoc_driver); + +MODULE_DESCRIPTION("Qualcomm SM8150 NoC driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/interconnect/qcom/sm8150.h b/drivers/interconnect/qcom/sm8150.h new file mode 100644 index 000000000000..97996f64d799 --- /dev/null +++ b/drivers/interconnect/qcom/sm8150.h @@ -0,0 +1,154 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Qualcomm #define SM8250 interconnect IDs + * + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + */ + +#ifndef __DRIVERS_INTERCONNECT_QCOM_SM8150_H +#define __DRIVERS_INTERCONNECT_QCOM_SM8150_H + +#define SM8150_A1NOC_SNOC_MAS 0 +#define SM8150_A1NOC_SNOC_SLV 1 +#define SM8150_A2NOC_SNOC_MAS 2 +#define SM8150_A2NOC_SNOC_SLV 3 +#define SM8150_MASTER_A1NOC_CFG 4 +#define SM8150_MASTER_A2NOC_CFG 5 +#define SM8150_MASTER_AMPSS_M0 6 +#define SM8150_MASTER_CAMNOC_HF0 7 +#define SM8150_MASTER_CAMNOC_HF0_UNCOMP 8 +#define SM8150_MASTER_CAMNOC_HF1 9 +#define SM8150_MASTER_CAMNOC_HF1_UNCOMP 10 +#define SM8150_MASTER_CAMNOC_SF 11 +#define SM8150_MASTER_CAMNOC_SF_UNCOMP 12 +#define SM8150_MASTER_CNOC_A2NOC 13 +#define SM8150_MASTER_CNOC_DC_NOC 14 +#define SM8150_MASTER_CNOC_MNOC_CFG 15 +#define SM8150_MASTER_COMPUTE_NOC 16 +#define SM8150_MASTER_CRYPTO_CORE_0 17 +#define SM8150_MASTER_ECC 18 +#define SM8150_MASTER_EMAC 19 +#define SM8150_MASTER_GEM_NOC_CFG 20 +#define SM8150_MASTER_GEM_NOC_PCIE_SNOC 21 +#define SM8150_MASTER_GEM_NOC_SNOC 22 +#define SM8150_MASTER_GIC 23 +#define SM8150_MASTER_GPU_TCU 24 +#define SM8150_MASTER_GRAPHICS_3D 25 +#define SM8150_MASTER_IPA 26 +#define SM8150_MASTER_IPA_CORE 27 +#define SM8150_MASTER_LLCC 28 +#define SM8150_MASTER_MDP_PORT0 29 +#define SM8150_MASTER_MDP_PORT1 30 +#define SM8150_MASTER_MNOC_HF_MEM_NOC 31 +#define SM8150_MASTER_MNOC_SF_MEM_NOC 32 +#define SM8150_MASTER_NPU 33 +#define SM8150_MASTER_PCIE 34 +#define SM8150_MASTER_PCIE_1 35 +#define SM8150_MASTER_PIMEM 36 +#define SM8150_MASTER_QDSS_BAM 37 +#define SM8150_MASTER_QDSS_DAP 38 +#define SM8150_MASTER_QDSS_ETR 39 +#define SM8150_MASTER_QSPI 40 +#define SM8150_MASTER_QUP_0 41 +#define SM8150_MASTER_QUP_1 42 +#define SM8150_MASTER_QUP_2 43 +#define SM8150_MASTER_ROTATOR 44 +#define SM8150_MASTER_SDCC_2 45 +#define SM8150_MASTER_SDCC_4 46 +#define SM8150_MASTER_SENSORS_AHB 47 +#define SM8150_MASTER_SNOC_CFG 48 +#define SM8150_MASTER_SNOC_GC_MEM_NOC 49 +#define SM8150_MASTER_SNOC_SF_MEM_NOC 50 +#define SM8150_MASTER_SPDM 51 +#define SM8150_MASTER_SYS_TCU 52 +#define SM8150_MASTER_TSIF 53 +#define SM8150_MASTER_UFS_MEM 54 +#define SM8150_MASTER_USB3 55 +#define SM8150_MASTER_USB3_1 56 +#define SM8150_MASTER_VIDEO_P0 57 +#define SM8150_MASTER_VIDEO_P1 58 +#define SM8150_MASTER_VIDEO_PROC 59 +#define SM8150_SLAVE_A1NOC_CFG 60 +#define SM8150_SLAVE_A2NOC_CFG 61 +#define SM8150_SLAVE_AHB2PHY_SOUTH 62 +#define SM8150_SLAVE_ANOC_PCIE_GEM_NOC 63 +#define SM8150_SLAVE_AOP 64 +#define SM8150_SLAVE_AOSS 65 +#define SM8150_SLAVE_APPSS 66 +#define SM8150_SLAVE_CAMERA_CFG 67 +#define SM8150_SLAVE_CAMNOC_UNCOMP 68 +#define SM8150_SLAVE_CDSP_CFG 69 +#define SM8150_SLAVE_CDSP_MEM_NOC 70 +#define SM8150_SLAVE_CLK_CTL 71 +#define SM8150_SLAVE_CNOC_A2NOC 72 +#define SM8150_SLAVE_CNOC_DDRSS 73 +#define SM8150_SLAVE_CNOC_MNOC_CFG 74 +#define SM8150_SLAVE_CRYPTO_0_CFG 75 +#define SM8150_SLAVE_DISPLAY_CFG 76 +#define SM8150_SLAVE_EBI_CH0 77 +#define SM8150_SLAVE_ECC 78 +#define SM8150_SLAVE_EMAC_CFG 79 +#define SM8150_SLAVE_GEM_NOC_CFG 80 +#define SM8150_SLAVE_GEM_NOC_SNOC 81 +#define SM8150_SLAVE_GLM 82 +#define SM8150_SLAVE_GRAPHICS_3D_CFG 83 +#define SM8150_SLAVE_IMEM_CFG 84 +#define SM8150_SLAVE_IPA_CFG 85 +#define SM8150_SLAVE_IPA_CORE 86 +#define SM8150_SLAVE_LLCC 87 +#define SM8150_SLAVE_LLCC_CFG 88 +#define SM8150_SLAVE_MNOC_HF_MEM_NOC 89 +#define SM8150_SLAVE_MNOC_SF_MEM_NOC 90 +#define SM8150_SLAVE_MSS_PROC_MS_MPU_CFG 91 +#define SM8150_SLAVE_NORTH_PHY_CFG 92 +#define SM8150_SLAVE_NPU_CFG 93 +#define SM8150_SLAVE_OCIMEM 94 +#define SM8150_SLAVE_PCIE_0 95 +#define SM8150_SLAVE_PCIE_0_CFG 96 +#define SM8150_SLAVE_PCIE_1 97 +#define SM8150_SLAVE_PCIE_1_CFG 98 +#define SM8150_SLAVE_PIMEM 99 +#define SM8150_SLAVE_PIMEM_CFG 100 +#define SM8150_SLAVE_PRNG 101 +#define SM8150_SLAVE_QDSS_CFG 102 +#define SM8150_SLAVE_QDSS_STM 103 +#define SM8150_SLAVE_QSPI 104 +#define SM8150_SLAVE_QUP_0 105 +#define SM8150_SLAVE_QUP_1 106 +#define SM8150_SLAVE_QUP_2 107 +#define SM8150_SLAVE_RBCPR_CX_CFG 108 +#define SM8150_SLAVE_RBCPR_MMCX_CFG 109 +#define SM8150_SLAVE_RBCPR_MX_CFG 110 +#define SM8150_SLAVE_SDCC_2 111 +#define SM8150_SLAVE_SDCC_4 112 +#define SM8150_SLAVE_SERVICE_A1NOC 113 +#define SM8150_SLAVE_SERVICE_A2NOC 114 +#define SM8150_SLAVE_SERVICE_CNOC 115 +#define SM8150_SLAVE_SERVICE_GEM_NOC 116 +#define SM8150_SLAVE_SERVICE_MNOC 117 +#define SM8150_SLAVE_SERVICE_SNOC 118 +#define SM8150_SLAVE_SNOC_CFG 119 +#define SM8150_SLAVE_SNOC_GEM_NOC_GC 120 +#define SM8150_SLAVE_SNOC_GEM_NOC_SF 121 +#define SM8150_SLAVE_SPDM_WRAPPER 122 +#define SM8150_SLAVE_SPSS_CFG 123 +#define SM8150_SLAVE_SSC_CFG 124 +#define SM8150_SLAVE_TCSR 125 +#define SM8150_SLAVE_TCU 126 +#define SM8150_SLAVE_TLMM_EAST 127 +#define SM8150_SLAVE_TLMM_NORTH 128 +#define SM8150_SLAVE_TLMM_SOUTH 129 +#define SM8150_SLAVE_TLMM_WEST 130 +#define SM8150_SLAVE_TSIF 131 +#define SM8150_SLAVE_UFS_CARD_CFG 132 +#define SM8150_SLAVE_UFS_MEM_CFG 133 +#define SM8150_SLAVE_USB3 134 +#define SM8150_SLAVE_USB3_1 135 +#define SM8150_SLAVE_VENUS_CFG 136 +#define SM8150_SLAVE_VSENSE_CTRL_CFG 137 +#define SM8150_SNOC_CNOC_MAS 138 +#define SM8150_SNOC_CNOC_SLV 139 +#define SM8150_MASTER_OSM_L3_APPS 140 +#define SM8150_SLAVE_OSM_L3 141 + +#endif diff --git a/drivers/interconnect/qcom/sm8250.c b/drivers/interconnect/qcom/sm8250.c new file mode 100644 index 000000000000..9b58946f7898 --- /dev/null +++ b/drivers/interconnect/qcom/sm8250.c @@ -0,0 +1,651 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + * + */ + +#include <linux/device.h> +#include <linux/interconnect.h> +#include <linux/interconnect-provider.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <dt-bindings/interconnect/qcom,sm8250.h> + +#include "bcm-voter.h" +#include "icc-rpmh.h" +#include "sm8250.h" + +DEFINE_QNODE(qhm_a1noc_cfg, SM8250_MASTER_A1NOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_A1NOC); +DEFINE_QNODE(qhm_qspi, SM8250_MASTER_QSPI_0, 1, 4, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qup1, SM8250_MASTER_QUP_1, 1, 4, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qup2, SM8250_MASTER_QUP_2, 1, 4, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(qhm_tsif, SM8250_MASTER_TSIF, 1, 4, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_pcie3_modem, SM8250_MASTER_PCIE_2, 1, 8, SM8250_SLAVE_ANOC_PCIE_GEM_NOC_1); +DEFINE_QNODE(xm_sdc4, SM8250_MASTER_SDCC_4, 1, 8, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_ufs_mem, SM8250_MASTER_UFS_MEM, 1, 8, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_usb3_0, SM8250_MASTER_USB3, 1, 8, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(xm_usb3_1, SM8250_MASTER_USB3_1, 1, 8, SM8250_A1NOC_SNOC_SLV); +DEFINE_QNODE(qhm_a2noc_cfg, SM8250_MASTER_A2NOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_A2NOC); +DEFINE_QNODE(qhm_qdss_bam, SM8250_MASTER_QDSS_BAM, 1, 4, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(qhm_qup0, SM8250_MASTER_QUP_0, 1, 4, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(qnm_cnoc, SM8250_MASTER_CNOC_A2NOC, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(qxm_crypto, SM8250_MASTER_CRYPTO_CORE_0, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(qxm_ipa, SM8250_MASTER_IPA, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_pcie3_0, SM8250_MASTER_PCIE, 1, 8, SM8250_SLAVE_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(xm_pcie3_1, SM8250_MASTER_PCIE_1, 1, 8, SM8250_SLAVE_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(xm_qdss_etr, SM8250_MASTER_QDSS_ETR, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_sdc2, SM8250_MASTER_SDCC_2, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(xm_ufs_card, SM8250_MASTER_UFS_CARD, 1, 8, SM8250_A2NOC_SNOC_SLV); +DEFINE_QNODE(qnm_npu, SM8250_MASTER_NPU, 2, 32, SM8250_SLAVE_CDSP_MEM_NOC); +DEFINE_QNODE(qnm_snoc, SM8250_SNOC_CNOC_MAS, 1, 8, SM8250_SLAVE_CDSP_CFG, SM8250_SLAVE_CAMERA_CFG, SM8250_SLAVE_TLMM_SOUTH, SM8250_SLAVE_TLMM_NORTH, SM8250_SLAVE_SDCC_4, SM8250_SLAVE_TLMM_WEST, SM8250_SLAVE_SDCC_2, SM8250_SLAVE_CNOC_MNOC_CFG, SM8250_SLAVE_UFS_MEM_CFG, SM8250_SLAVE_SNOC_CFG, SM8250_SLAVE_PDM, SM8250_SLAVE_CX_RDPM, SM8250_SLAVE_PCIE_1_CFG, SM8250_SLAVE_A2NOC_CFG, SM8250_SLAVE_QDSS_CFG, SM8250_SLAVE_DISPLAY_CFG, SM8250_SLAVE_PCIE_2_CFG, SM8250_SLAVE_TCSR, SM8250_SLAVE_DCC_CFG, SM8250_SLAVE_CNOC_DDRSS, SM8250_SLAVE_IPC_ROUTER_CFG, SM8250_SLAVE_PCIE_0_CFG, SM8250_SLAVE_RBCPR_MMCX_CFG, SM8250_SLAVE_NPU_CFG, SM8250_SLAVE_AHB2PHY_SOUTH, SM8250_SLAVE_AHB2PHY_NORTH, SM8250_SLAVE_GRAPHICS_3D_CFG, SM8250_SLAVE_VENUS_CFG, SM8250_SLAVE_TSIF, SM8250_SLAVE_IPA_CFG, SM8250_SLAVE_IMEM_CFG, SM8250_SLAVE_USB3, SM8250_SLAVE_SERVICE_CNOC, SM8250_SLAVE_UFS_CARD_CFG, SM8250_SLAVE_USB3_1, SM8250_SLAVE_LPASS, SM8250_SLAVE_RBCPR_CX_CFG, SM8250_SLAVE_A1NOC_CFG, SM8250_SLAVE_AOSS, SM8250_SLAVE_PRNG, SM8250_SLAVE_VSENSE_CTRL_CFG, SM8250_SLAVE_QSPI_0, SM8250_SLAVE_CRYPTO_0_CFG, SM8250_SLAVE_PIMEM_CFG, SM8250_SLAVE_RBCPR_MX_CFG, SM8250_SLAVE_QUP_0, SM8250_SLAVE_QUP_1, SM8250_SLAVE_QUP_2, SM8250_SLAVE_CLK_CTL); +DEFINE_QNODE(xm_qdss_dap, SM8250_MASTER_QDSS_DAP, 1, 8, SM8250_SLAVE_CDSP_CFG, SM8250_SLAVE_CAMERA_CFG, SM8250_SLAVE_TLMM_SOUTH, SM8250_SLAVE_TLMM_NORTH, SM8250_SLAVE_SDCC_4, SM8250_SLAVE_TLMM_WEST, SM8250_SLAVE_SDCC_2, SM8250_SLAVE_CNOC_MNOC_CFG, SM8250_SLAVE_UFS_MEM_CFG, SM8250_SLAVE_SNOC_CFG, SM8250_SLAVE_PDM, SM8250_SLAVE_CX_RDPM, SM8250_SLAVE_PCIE_1_CFG, SM8250_SLAVE_A2NOC_CFG, SM8250_SLAVE_QDSS_CFG, SM8250_SLAVE_DISPLAY_CFG, SM8250_SLAVE_PCIE_2_CFG, SM8250_SLAVE_TCSR, SM8250_SLAVE_DCC_CFG, SM8250_SLAVE_CNOC_DDRSS, SM8250_SLAVE_IPC_ROUTER_CFG, SM8250_SLAVE_CNOC_A2NOC, SM8250_SLAVE_PCIE_0_CFG, SM8250_SLAVE_RBCPR_MMCX_CFG, SM8250_SLAVE_NPU_CFG, SM8250_SLAVE_AHB2PHY_SOUTH, SM8250_SLAVE_AHB2PHY_NORTH, SM8250_SLAVE_GRAPHICS_3D_CFG, SM8250_SLAVE_VENUS_CFG, SM8250_SLAVE_TSIF, SM8250_SLAVE_IPA_CFG, SM8250_SLAVE_IMEM_CFG, SM8250_SLAVE_USB3, SM8250_SLAVE_SERVICE_CNOC, SM8250_SLAVE_UFS_CARD_CFG, SM8250_SLAVE_USB3_1, SM8250_SLAVE_LPASS, SM8250_SLAVE_RBCPR_CX_CFG, SM8250_SLAVE_A1NOC_CFG, SM8250_SLAVE_AOSS, SM8250_SLAVE_PRNG, SM8250_SLAVE_VSENSE_CTRL_CFG, SM8250_SLAVE_QSPI_0, SM8250_SLAVE_CRYPTO_0_CFG, SM8250_SLAVE_PIMEM_CFG, SM8250_SLAVE_RBCPR_MX_CFG, SM8250_SLAVE_QUP_0, SM8250_SLAVE_QUP_1, SM8250_SLAVE_QUP_2, SM8250_SLAVE_CLK_CTL); +DEFINE_QNODE(qhm_cnoc_dc_noc, SM8250_MASTER_CNOC_DC_NOC, 1, 4, SM8250_SLAVE_GEM_NOC_CFG, SM8250_SLAVE_LLCC_CFG); +DEFINE_QNODE(alm_gpu_tcu, SM8250_MASTER_GPU_TCU, 1, 8, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(alm_sys_tcu, SM8250_MASTER_SYS_TCU, 1, 8, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(chm_apps, SM8250_MASTER_AMPSS_M0, 2, 32, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC, SM8250_SLAVE_MEM_NOC_PCIE_SNOC); +DEFINE_QNODE(qhm_gemnoc_cfg, SM8250_MASTER_GEM_NOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_GEM_NOC_2, SM8250_SLAVE_SERVICE_GEM_NOC_1, SM8250_SLAVE_SERVICE_GEM_NOC); +DEFINE_QNODE(qnm_cmpnoc, SM8250_MASTER_COMPUTE_NOC, 2, 32, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_gpu, SM8250_MASTER_GRAPHICS_3D, 2, 32, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_mnoc_hf, SM8250_MASTER_MNOC_HF_MEM_NOC, 2, 32, SM8250_SLAVE_LLCC); +DEFINE_QNODE(qnm_mnoc_sf, SM8250_MASTER_MNOC_SF_MEM_NOC, 2, 32, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_pcie, SM8250_MASTER_ANOC_PCIE_GEM_NOC, 1, 16, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC); +DEFINE_QNODE(qnm_snoc_gc, SM8250_MASTER_SNOC_GC_MEM_NOC, 1, 8, SM8250_SLAVE_LLCC); +DEFINE_QNODE(qnm_snoc_sf, SM8250_MASTER_SNOC_SF_MEM_NOC, 1, 16, SM8250_SLAVE_LLCC, SM8250_SLAVE_GEM_NOC_SNOC, SM8250_SLAVE_MEM_NOC_PCIE_SNOC); +DEFINE_QNODE(ipa_core_master, SM8250_MASTER_IPA_CORE, 1, 8, SM8250_SLAVE_IPA_CORE); +DEFINE_QNODE(llcc_mc, SM8250_MASTER_LLCC, 4, 4, SM8250_SLAVE_EBI_CH0); +DEFINE_QNODE(qhm_mnoc_cfg, SM8250_MASTER_CNOC_MNOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_MNOC); +DEFINE_QNODE(qnm_camnoc_hf, SM8250_MASTER_CAMNOC_HF, 2, 32, SM8250_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qnm_camnoc_icp, SM8250_MASTER_CAMNOC_ICP, 1, 8, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qnm_camnoc_sf, SM8250_MASTER_CAMNOC_SF, 2, 32, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qnm_video0, SM8250_MASTER_VIDEO_P0, 1, 32, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qnm_video1, SM8250_MASTER_VIDEO_P1, 1, 32, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qnm_video_cvp, SM8250_MASTER_VIDEO_PROC, 1, 32, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(qxm_mdp0, SM8250_MASTER_MDP_PORT0, 1, 32, SM8250_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_mdp1, SM8250_MASTER_MDP_PORT1, 1, 32, SM8250_SLAVE_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qxm_rot, SM8250_MASTER_ROTATOR, 1, 32, SM8250_SLAVE_MNOC_SF_MEM_NOC); +DEFINE_QNODE(amm_npu_sys, SM8250_MASTER_NPU_SYS, 4, 32, SM8250_SLAVE_NPU_COMPUTE_NOC); +DEFINE_QNODE(amm_npu_sys_cdp_w, SM8250_MASTER_NPU_CDP, 2, 16, SM8250_SLAVE_NPU_COMPUTE_NOC); +DEFINE_QNODE(qhm_cfg, SM8250_MASTER_NPU_NOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_NPU_NOC, SM8250_SLAVE_ISENSE_CFG, SM8250_SLAVE_NPU_LLM_CFG, SM8250_SLAVE_NPU_INT_DMA_BWMON_CFG, SM8250_SLAVE_NPU_CP, SM8250_SLAVE_NPU_TCM, SM8250_SLAVE_NPU_CAL_DP0, SM8250_SLAVE_NPU_CAL_DP1, SM8250_SLAVE_NPU_DPM); +DEFINE_QNODE(qhm_snoc_cfg, SM8250_MASTER_SNOC_CFG, 1, 4, SM8250_SLAVE_SERVICE_SNOC); +DEFINE_QNODE(qnm_aggre1_noc, SM8250_A1NOC_SNOC_MAS, 1, 16, SM8250_SLAVE_SNOC_GEM_NOC_SF); +DEFINE_QNODE(qnm_aggre2_noc, SM8250_A2NOC_SNOC_MAS, 1, 16, SM8250_SLAVE_SNOC_GEM_NOC_SF); +DEFINE_QNODE(qnm_gemnoc, SM8250_MASTER_GEM_NOC_SNOC, 1, 16, SM8250_SLAVE_PIMEM, SM8250_SLAVE_OCIMEM, SM8250_SLAVE_APPSS, SM8250_SNOC_CNOC_SLV, SM8250_SLAVE_TCU, SM8250_SLAVE_QDSS_STM); +DEFINE_QNODE(qnm_gemnoc_pcie, SM8250_MASTER_GEM_NOC_PCIE_SNOC, 1, 8, SM8250_SLAVE_PCIE_2, SM8250_SLAVE_PCIE_0, SM8250_SLAVE_PCIE_1); +DEFINE_QNODE(qxm_pimem, SM8250_MASTER_PIMEM, 1, 8, SM8250_SLAVE_SNOC_GEM_NOC_GC); +DEFINE_QNODE(xm_gic, SM8250_MASTER_GIC, 1, 8, SM8250_SLAVE_SNOC_GEM_NOC_GC); +DEFINE_QNODE(qns_a1noc_snoc, SM8250_A1NOC_SNOC_SLV, 1, 16, SM8250_A1NOC_SNOC_MAS); +DEFINE_QNODE(qns_pcie_modem_mem_noc, SM8250_SLAVE_ANOC_PCIE_GEM_NOC_1, 1, 16, SM8250_MASTER_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(srvc_aggre1_noc, SM8250_SLAVE_SERVICE_A1NOC, 1, 4); +DEFINE_QNODE(qns_a2noc_snoc, SM8250_A2NOC_SNOC_SLV, 1, 16, SM8250_A2NOC_SNOC_MAS); +DEFINE_QNODE(qns_pcie_mem_noc, SM8250_SLAVE_ANOC_PCIE_GEM_NOC, 1, 16, SM8250_MASTER_ANOC_PCIE_GEM_NOC); +DEFINE_QNODE(srvc_aggre2_noc, SM8250_SLAVE_SERVICE_A2NOC, 1, 4); +DEFINE_QNODE(qns_cdsp_mem_noc, SM8250_SLAVE_CDSP_MEM_NOC, 2, 32, SM8250_MASTER_COMPUTE_NOC); +DEFINE_QNODE(qhs_a1_noc_cfg, SM8250_SLAVE_A1NOC_CFG, 1, 4, SM8250_MASTER_A1NOC_CFG); +DEFINE_QNODE(qhs_a2_noc_cfg, SM8250_SLAVE_A2NOC_CFG, 1, 4, SM8250_MASTER_A2NOC_CFG); +DEFINE_QNODE(qhs_ahb2phy0, SM8250_SLAVE_AHB2PHY_SOUTH, 1, 4); +DEFINE_QNODE(qhs_ahb2phy1, SM8250_SLAVE_AHB2PHY_NORTH, 1, 4); +DEFINE_QNODE(qhs_aoss, SM8250_SLAVE_AOSS, 1, 4); +DEFINE_QNODE(qhs_camera_cfg, SM8250_SLAVE_CAMERA_CFG, 1, 4); +DEFINE_QNODE(qhs_clk_ctl, SM8250_SLAVE_CLK_CTL, 1, 4); +DEFINE_QNODE(qhs_compute_dsp, SM8250_SLAVE_CDSP_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_cx, SM8250_SLAVE_RBCPR_CX_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_mmcx, SM8250_SLAVE_RBCPR_MMCX_CFG, 1, 4); +DEFINE_QNODE(qhs_cpr_mx, SM8250_SLAVE_RBCPR_MX_CFG, 1, 4); +DEFINE_QNODE(qhs_crypto0_cfg, SM8250_SLAVE_CRYPTO_0_CFG, 1, 4); +DEFINE_QNODE(qhs_cx_rdpm, SM8250_SLAVE_CX_RDPM, 1, 4); +DEFINE_QNODE(qhs_dcc_cfg, SM8250_SLAVE_DCC_CFG, 1, 4); +DEFINE_QNODE(qhs_ddrss_cfg, SM8250_SLAVE_CNOC_DDRSS, 1, 4, SM8250_MASTER_CNOC_DC_NOC); +DEFINE_QNODE(qhs_display_cfg, SM8250_SLAVE_DISPLAY_CFG, 1, 4); +DEFINE_QNODE(qhs_gpuss_cfg, SM8250_SLAVE_GRAPHICS_3D_CFG, 1, 8); +DEFINE_QNODE(qhs_imem_cfg, SM8250_SLAVE_IMEM_CFG, 1, 4); +DEFINE_QNODE(qhs_ipa, SM8250_SLAVE_IPA_CFG, 1, 4); +DEFINE_QNODE(qhs_ipc_router, SM8250_SLAVE_IPC_ROUTER_CFG, 1, 4); +DEFINE_QNODE(qhs_lpass_cfg, SM8250_SLAVE_LPASS, 1, 4); +DEFINE_QNODE(qhs_mnoc_cfg, SM8250_SLAVE_CNOC_MNOC_CFG, 1, 4, SM8250_MASTER_CNOC_MNOC_CFG); +DEFINE_QNODE(qhs_npu_cfg, SM8250_SLAVE_NPU_CFG, 1, 4, SM8250_MASTER_NPU_NOC_CFG); +DEFINE_QNODE(qhs_pcie0_cfg, SM8250_SLAVE_PCIE_0_CFG, 1, 4); +DEFINE_QNODE(qhs_pcie1_cfg, SM8250_SLAVE_PCIE_1_CFG, 1, 4); +DEFINE_QNODE(qhs_pcie_modem_cfg, SM8250_SLAVE_PCIE_2_CFG, 1, 4); +DEFINE_QNODE(qhs_pdm, SM8250_SLAVE_PDM, 1, 4); +DEFINE_QNODE(qhs_pimem_cfg, SM8250_SLAVE_PIMEM_CFG, 1, 4); +DEFINE_QNODE(qhs_prng, SM8250_SLAVE_PRNG, 1, 4); +DEFINE_QNODE(qhs_qdss_cfg, SM8250_SLAVE_QDSS_CFG, 1, 4); +DEFINE_QNODE(qhs_qspi, SM8250_SLAVE_QSPI_0, 1, 4); +DEFINE_QNODE(qhs_qup0, SM8250_SLAVE_QUP_0, 1, 4); +DEFINE_QNODE(qhs_qup1, SM8250_SLAVE_QUP_1, 1, 4); +DEFINE_QNODE(qhs_qup2, SM8250_SLAVE_QUP_2, 1, 4); +DEFINE_QNODE(qhs_sdc2, SM8250_SLAVE_SDCC_2, 1, 4); +DEFINE_QNODE(qhs_sdc4, SM8250_SLAVE_SDCC_4, 1, 4); +DEFINE_QNODE(qhs_snoc_cfg, SM8250_SLAVE_SNOC_CFG, 1, 4, SM8250_MASTER_SNOC_CFG); +DEFINE_QNODE(qhs_tcsr, SM8250_SLAVE_TCSR, 1, 4); +DEFINE_QNODE(qhs_tlmm0, SM8250_SLAVE_TLMM_NORTH, 1, 4); +DEFINE_QNODE(qhs_tlmm1, SM8250_SLAVE_TLMM_SOUTH, 1, 4); +DEFINE_QNODE(qhs_tlmm2, SM8250_SLAVE_TLMM_WEST, 1, 4); +DEFINE_QNODE(qhs_tsif, SM8250_SLAVE_TSIF, 1, 4); +DEFINE_QNODE(qhs_ufs_card_cfg, SM8250_SLAVE_UFS_CARD_CFG, 1, 4); +DEFINE_QNODE(qhs_ufs_mem_cfg, SM8250_SLAVE_UFS_MEM_CFG, 1, 4); +DEFINE_QNODE(qhs_usb3_0, SM8250_SLAVE_USB3, 1, 4); +DEFINE_QNODE(qhs_usb3_1, SM8250_SLAVE_USB3_1, 1, 4); +DEFINE_QNODE(qhs_venus_cfg, SM8250_SLAVE_VENUS_CFG, 1, 4); +DEFINE_QNODE(qhs_vsense_ctrl_cfg, SM8250_SLAVE_VSENSE_CTRL_CFG, 1, 4); +DEFINE_QNODE(qns_cnoc_a2noc, SM8250_SLAVE_CNOC_A2NOC, 1, 8, SM8250_MASTER_CNOC_A2NOC); +DEFINE_QNODE(srvc_cnoc, SM8250_SLAVE_SERVICE_CNOC, 1, 4); +DEFINE_QNODE(qhs_llcc, SM8250_SLAVE_LLCC_CFG, 1, 4); +DEFINE_QNODE(qhs_memnoc, SM8250_SLAVE_GEM_NOC_CFG, 1, 4, SM8250_MASTER_GEM_NOC_CFG); +DEFINE_QNODE(qns_gem_noc_snoc, SM8250_SLAVE_GEM_NOC_SNOC, 1, 16, SM8250_MASTER_GEM_NOC_SNOC); +DEFINE_QNODE(qns_llcc, SM8250_SLAVE_LLCC, 4, 16, SM8250_MASTER_LLCC); +DEFINE_QNODE(qns_sys_pcie, SM8250_SLAVE_MEM_NOC_PCIE_SNOC, 1, 8, SM8250_MASTER_GEM_NOC_PCIE_SNOC); +DEFINE_QNODE(srvc_even_gemnoc, SM8250_SLAVE_SERVICE_GEM_NOC_1, 1, 4); +DEFINE_QNODE(srvc_odd_gemnoc, SM8250_SLAVE_SERVICE_GEM_NOC_2, 1, 4); +DEFINE_QNODE(srvc_sys_gemnoc, SM8250_SLAVE_SERVICE_GEM_NOC, 1, 4); +DEFINE_QNODE(ipa_core_slave, SM8250_SLAVE_IPA_CORE, 1, 8); +DEFINE_QNODE(ebi, SM8250_SLAVE_EBI_CH0, 4, 4); +DEFINE_QNODE(qns_mem_noc_hf, SM8250_SLAVE_MNOC_HF_MEM_NOC, 2, 32, SM8250_MASTER_MNOC_HF_MEM_NOC); +DEFINE_QNODE(qns_mem_noc_sf, SM8250_SLAVE_MNOC_SF_MEM_NOC, 2, 32, SM8250_MASTER_MNOC_SF_MEM_NOC); +DEFINE_QNODE(srvc_mnoc, SM8250_SLAVE_SERVICE_MNOC, 1, 4); +DEFINE_QNODE(qhs_cal_dp0, SM8250_SLAVE_NPU_CAL_DP0, 1, 4); +DEFINE_QNODE(qhs_cal_dp1, SM8250_SLAVE_NPU_CAL_DP1, 1, 4); +DEFINE_QNODE(qhs_cp, SM8250_SLAVE_NPU_CP, 1, 4); +DEFINE_QNODE(qhs_dma_bwmon, SM8250_SLAVE_NPU_INT_DMA_BWMON_CFG, 1, 4); +DEFINE_QNODE(qhs_dpm, SM8250_SLAVE_NPU_DPM, 1, 4); +DEFINE_QNODE(qhs_isense, SM8250_SLAVE_ISENSE_CFG, 1, 4); +DEFINE_QNODE(qhs_llm, SM8250_SLAVE_NPU_LLM_CFG, 1, 4); +DEFINE_QNODE(qhs_tcm, SM8250_SLAVE_NPU_TCM, 1, 4); +DEFINE_QNODE(qns_npu_sys, SM8250_SLAVE_NPU_COMPUTE_NOC, 2, 32); +DEFINE_QNODE(srvc_noc, SM8250_SLAVE_SERVICE_NPU_NOC, 1, 4); +DEFINE_QNODE(qhs_apss, SM8250_SLAVE_APPSS, 1, 8); +DEFINE_QNODE(qns_cnoc, SM8250_SNOC_CNOC_SLV, 1, 8, SM8250_SNOC_CNOC_MAS); +DEFINE_QNODE(qns_gemnoc_gc, SM8250_SLAVE_SNOC_GEM_NOC_GC, 1, 8, SM8250_MASTER_SNOC_GC_MEM_NOC); +DEFINE_QNODE(qns_gemnoc_sf, SM8250_SLAVE_SNOC_GEM_NOC_SF, 1, 16, SM8250_MASTER_SNOC_SF_MEM_NOC); +DEFINE_QNODE(qxs_imem, SM8250_SLAVE_OCIMEM, 1, 8); +DEFINE_QNODE(qxs_pimem, SM8250_SLAVE_PIMEM, 1, 8); +DEFINE_QNODE(srvc_snoc, SM8250_SLAVE_SERVICE_SNOC, 1, 4); +DEFINE_QNODE(xs_pcie_0, SM8250_SLAVE_PCIE_0, 1, 8); +DEFINE_QNODE(xs_pcie_1, SM8250_SLAVE_PCIE_1, 1, 8); +DEFINE_QNODE(xs_pcie_modem, SM8250_SLAVE_PCIE_2, 1, 8); +DEFINE_QNODE(xs_qdss_stm, SM8250_SLAVE_QDSS_STM, 1, 4); +DEFINE_QNODE(xs_sys_tcu_cfg, SM8250_SLAVE_TCU, 1, 8); + +DEFINE_QBCM(bcm_acv, "ACV", false, &ebi); +DEFINE_QBCM(bcm_mc0, "MC0", true, &ebi); +DEFINE_QBCM(bcm_sh0, "SH0", true, &qns_llcc); +DEFINE_QBCM(bcm_mm0, "MM0", true, &qns_mem_noc_hf); +DEFINE_QBCM(bcm_ce0, "CE0", false, &qxm_crypto); +DEFINE_QBCM(bcm_ip0, "IP0", false, &ipa_core_slave); +DEFINE_QBCM(bcm_mm1, "MM1", false, &qnm_camnoc_hf, &qxm_mdp0, &qxm_mdp1); +DEFINE_QBCM(bcm_sh2, "SH2", false, &alm_gpu_tcu, &alm_sys_tcu); +DEFINE_QBCM(bcm_mm2, "MM2", false, &qns_mem_noc_sf); +DEFINE_QBCM(bcm_qup0, "QUP0", false, &qhm_qup1, &qhm_qup2, &qhm_qup0); +DEFINE_QBCM(bcm_sh3, "SH3", false, &qnm_cmpnoc); +DEFINE_QBCM(bcm_mm3, "MM3", false, &qnm_camnoc_icp, &qnm_camnoc_sf, &qnm_video0, &qnm_video1, &qnm_video_cvp); +DEFINE_QBCM(bcm_sh4, "SH4", false, &chm_apps); +DEFINE_QBCM(bcm_sn0, "SN0", true, &qns_gemnoc_sf); +DEFINE_QBCM(bcm_co0, "CO0", false, &qns_cdsp_mem_noc); +DEFINE_QBCM(bcm_cn0, "CN0", true, &qnm_snoc, &xm_qdss_dap, &qhs_a1_noc_cfg, &qhs_a2_noc_cfg, &qhs_ahb2phy0, &qhs_ahb2phy1, &qhs_aoss, &qhs_camera_cfg, &qhs_clk_ctl, &qhs_compute_dsp, &qhs_cpr_cx, &qhs_cpr_mmcx, &qhs_cpr_mx, &qhs_crypto0_cfg, &qhs_cx_rdpm, &qhs_dcc_cfg, &qhs_ddrss_cfg, &qhs_display_cfg, &qhs_gpuss_cfg, &qhs_imem_cfg, &qhs_ipa, &qhs_ipc_router, &qhs_lpass_cfg, &qhs_mnoc_cfg, &qhs_npu_cfg, &qhs_pcie0_cfg, &qhs_pcie1_cfg, &qhs_pcie_modem_cfg, &qhs_pdm, &qhs_pimem_cfg, &qhs_prng, &qhs_qdss_cfg, &qhs_qspi, &qhs_qup0, &qhs_qup1, &qhs_qup2, &qhs_sdc2, &qhs_sdc4, &qhs_snoc_cfg, &qhs_tcsr, &qhs_tlmm0, &qhs_tlmm1, &qhs_tlmm2, &qhs_tsif, &qhs_ufs_card_cfg, &qhs_ufs_mem_cfg, &qhs_usb3_0, &qhs_usb3_1, &qhs_venus_cfg, &qhs_vsense_ctrl_cfg, &qns_cnoc_a2noc, &srvc_cnoc); +DEFINE_QBCM(bcm_sn1, "SN1", false, &qxs_imem); +DEFINE_QBCM(bcm_sn2, "SN2", false, &qns_gemnoc_gc); +DEFINE_QBCM(bcm_co2, "CO2", false, &qnm_npu); +DEFINE_QBCM(bcm_sn3, "SN3", false, &qxs_pimem); +DEFINE_QBCM(bcm_sn4, "SN4", false, &xs_qdss_stm); +DEFINE_QBCM(bcm_sn5, "SN5", false, &xs_pcie_modem); +DEFINE_QBCM(bcm_sn6, "SN6", false, &xs_pcie_0, &xs_pcie_1); +DEFINE_QBCM(bcm_sn7, "SN7", false, &qnm_aggre1_noc); +DEFINE_QBCM(bcm_sn8, "SN8", false, &qnm_aggre2_noc); +DEFINE_QBCM(bcm_sn9, "SN9", false, &qnm_gemnoc_pcie); +DEFINE_QBCM(bcm_sn11, "SN11", false, &qnm_gemnoc); +DEFINE_QBCM(bcm_sn12, "SN12", false, &qns_pcie_modem_mem_noc, &qns_pcie_mem_noc); + +static struct qcom_icc_bcm *aggre1_noc_bcms[] = { + &bcm_qup0, + &bcm_sn12, +}; + +static struct qcom_icc_node *aggre1_noc_nodes[] = { + [MASTER_A1NOC_CFG] = &qhm_a1noc_cfg, + [MASTER_QSPI_0] = &qhm_qspi, + [MASTER_QUP_1] = &qhm_qup1, + [MASTER_QUP_2] = &qhm_qup2, + [MASTER_TSIF] = &qhm_tsif, + [MASTER_PCIE_2] = &xm_pcie3_modem, + [MASTER_SDCC_4] = &xm_sdc4, + [MASTER_UFS_MEM] = &xm_ufs_mem, + [MASTER_USB3] = &xm_usb3_0, + [MASTER_USB3_1] = &xm_usb3_1, + [A1NOC_SNOC_SLV] = &qns_a1noc_snoc, + [SLAVE_ANOC_PCIE_GEM_NOC_1] = &qns_pcie_modem_mem_noc, + [SLAVE_SERVICE_A1NOC] = &srvc_aggre1_noc, +}; + +static struct qcom_icc_desc sm8250_aggre1_noc = { + .nodes = aggre1_noc_nodes, + .num_nodes = ARRAY_SIZE(aggre1_noc_nodes), + .bcms = aggre1_noc_bcms, + .num_bcms = ARRAY_SIZE(aggre1_noc_bcms), +}; + +static struct qcom_icc_bcm *aggre2_noc_bcms[] = { + &bcm_ce0, + &bcm_qup0, + &bcm_sn12, +}; + +static struct qcom_icc_node *aggre2_noc_nodes[] = { + [MASTER_A2NOC_CFG] = &qhm_a2noc_cfg, + [MASTER_QDSS_BAM] = &qhm_qdss_bam, + [MASTER_QUP_0] = &qhm_qup0, + [MASTER_CNOC_A2NOC] = &qnm_cnoc, + [MASTER_CRYPTO_CORE_0] = &qxm_crypto, + [MASTER_IPA] = &qxm_ipa, + [MASTER_PCIE] = &xm_pcie3_0, + [MASTER_PCIE_1] = &xm_pcie3_1, + [MASTER_QDSS_ETR] = &xm_qdss_etr, + [MASTER_SDCC_2] = &xm_sdc2, + [MASTER_UFS_CARD] = &xm_ufs_card, + [A2NOC_SNOC_SLV] = &qns_a2noc_snoc, + [SLAVE_ANOC_PCIE_GEM_NOC] = &qns_pcie_mem_noc, + [SLAVE_SERVICE_A2NOC] = &srvc_aggre2_noc, +}; + +static struct qcom_icc_desc sm8250_aggre2_noc = { + .nodes = aggre2_noc_nodes, + .num_nodes = ARRAY_SIZE(aggre2_noc_nodes), + .bcms = aggre2_noc_bcms, + .num_bcms = ARRAY_SIZE(aggre2_noc_bcms), +}; + +static struct qcom_icc_bcm *compute_noc_bcms[] = { + &bcm_co0, + &bcm_co2, +}; + +static struct qcom_icc_node *compute_noc_nodes[] = { + [MASTER_NPU] = &qnm_npu, + [SLAVE_CDSP_MEM_NOC] = &qns_cdsp_mem_noc, +}; + +static struct qcom_icc_desc sm8250_compute_noc = { + .nodes = compute_noc_nodes, + .num_nodes = ARRAY_SIZE(compute_noc_nodes), + .bcms = compute_noc_bcms, + .num_bcms = ARRAY_SIZE(compute_noc_bcms), +}; + +static struct qcom_icc_bcm *config_noc_bcms[] = { + &bcm_cn0, +}; + +static struct qcom_icc_node *config_noc_nodes[] = { + [SNOC_CNOC_MAS] = &qnm_snoc, + [MASTER_QDSS_DAP] = &xm_qdss_dap, + [SLAVE_A1NOC_CFG] = &qhs_a1_noc_cfg, + [SLAVE_A2NOC_CFG] = &qhs_a2_noc_cfg, + [SLAVE_AHB2PHY_SOUTH] = &qhs_ahb2phy0, + [SLAVE_AHB2PHY_NORTH] = &qhs_ahb2phy1, + [SLAVE_AOSS] = &qhs_aoss, + [SLAVE_CAMERA_CFG] = &qhs_camera_cfg, + [SLAVE_CLK_CTL] = &qhs_clk_ctl, + [SLAVE_CDSP_CFG] = &qhs_compute_dsp, + [SLAVE_RBCPR_CX_CFG] = &qhs_cpr_cx, + [SLAVE_RBCPR_MMCX_CFG] = &qhs_cpr_mmcx, + [SLAVE_RBCPR_MX_CFG] = &qhs_cpr_mx, + [SLAVE_CRYPTO_0_CFG] = &qhs_crypto0_cfg, + [SLAVE_CX_RDPM] = &qhs_cx_rdpm, + [SLAVE_DCC_CFG] = &qhs_dcc_cfg, + [SLAVE_CNOC_DDRSS] = &qhs_ddrss_cfg, + [SLAVE_DISPLAY_CFG] = &qhs_display_cfg, + [SLAVE_GRAPHICS_3D_CFG] = &qhs_gpuss_cfg, + [SLAVE_IMEM_CFG] = &qhs_imem_cfg, + [SLAVE_IPA_CFG] = &qhs_ipa, + [SLAVE_IPC_ROUTER_CFG] = &qhs_ipc_router, + [SLAVE_LPASS] = &qhs_lpass_cfg, + [SLAVE_CNOC_MNOC_CFG] = &qhs_mnoc_cfg, + [SLAVE_NPU_CFG] = &qhs_npu_cfg, + [SLAVE_PCIE_0_CFG] = &qhs_pcie0_cfg, + [SLAVE_PCIE_1_CFG] = &qhs_pcie1_cfg, + [SLAVE_PCIE_2_CFG] = &qhs_pcie_modem_cfg, + [SLAVE_PDM] = &qhs_pdm, + [SLAVE_PIMEM_CFG] = &qhs_pimem_cfg, + [SLAVE_PRNG] = &qhs_prng, + [SLAVE_QDSS_CFG] = &qhs_qdss_cfg, + [SLAVE_QSPI_0] = &qhs_qspi, + [SLAVE_QUP_0] = &qhs_qup0, + [SLAVE_QUP_1] = &qhs_qup1, + [SLAVE_QUP_2] = &qhs_qup2, + [SLAVE_SDCC_2] = &qhs_sdc2, + [SLAVE_SDCC_4] = &qhs_sdc4, + [SLAVE_SNOC_CFG] = &qhs_snoc_cfg, + [SLAVE_TCSR] = &qhs_tcsr, + [SLAVE_TLMM_NORTH] = &qhs_tlmm0, + [SLAVE_TLMM_SOUTH] = &qhs_tlmm1, + [SLAVE_TLMM_WEST] = &qhs_tlmm2, + [SLAVE_TSIF] = &qhs_tsif, + [SLAVE_UFS_CARD_CFG] = &qhs_ufs_card_cfg, + [SLAVE_UFS_MEM_CFG] = &qhs_ufs_mem_cfg, + [SLAVE_USB3] = &qhs_usb3_0, + [SLAVE_USB3_1] = &qhs_usb3_1, + [SLAVE_VENUS_CFG] = &qhs_venus_cfg, + [SLAVE_VSENSE_CTRL_CFG] = &qhs_vsense_ctrl_cfg, + [SLAVE_CNOC_A2NOC] = &qns_cnoc_a2noc, + [SLAVE_SERVICE_CNOC] = &srvc_cnoc, +}; + +static struct qcom_icc_desc sm8250_config_noc = { + .nodes = config_noc_nodes, + .num_nodes = ARRAY_SIZE(config_noc_nodes), + .bcms = config_noc_bcms, + .num_bcms = ARRAY_SIZE(config_noc_bcms), +}; + +static struct qcom_icc_bcm *dc_noc_bcms[] = { +}; + +static struct qcom_icc_node *dc_noc_nodes[] = { + [MASTER_CNOC_DC_NOC] = &qhm_cnoc_dc_noc, + [SLAVE_LLCC_CFG] = &qhs_llcc, + [SLAVE_GEM_NOC_CFG] = &qhs_memnoc, +}; + +static struct qcom_icc_desc sm8250_dc_noc = { + .nodes = dc_noc_nodes, + .num_nodes = ARRAY_SIZE(dc_noc_nodes), + .bcms = dc_noc_bcms, + .num_bcms = ARRAY_SIZE(dc_noc_bcms), +}; + +static struct qcom_icc_bcm *gem_noc_bcms[] = { + &bcm_sh0, + &bcm_sh2, + &bcm_sh3, + &bcm_sh4, +}; + +static struct qcom_icc_node *gem_noc_nodes[] = { + [MASTER_GPU_TCU] = &alm_gpu_tcu, + [MASTER_SYS_TCU] = &alm_sys_tcu, + [MASTER_AMPSS_M0] = &chm_apps, + [MASTER_GEM_NOC_CFG] = &qhm_gemnoc_cfg, + [MASTER_COMPUTE_NOC] = &qnm_cmpnoc, + [MASTER_GRAPHICS_3D] = &qnm_gpu, + [MASTER_MNOC_HF_MEM_NOC] = &qnm_mnoc_hf, + [MASTER_MNOC_SF_MEM_NOC] = &qnm_mnoc_sf, + [MASTER_ANOC_PCIE_GEM_NOC] = &qnm_pcie, + [MASTER_SNOC_GC_MEM_NOC] = &qnm_snoc_gc, + [MASTER_SNOC_SF_MEM_NOC] = &qnm_snoc_sf, + [SLAVE_GEM_NOC_SNOC] = &qns_gem_noc_snoc, + [SLAVE_LLCC] = &qns_llcc, + [SLAVE_MEM_NOC_PCIE_SNOC] = &qns_sys_pcie, + [SLAVE_SERVICE_GEM_NOC_1] = &srvc_even_gemnoc, + [SLAVE_SERVICE_GEM_NOC_2] = &srvc_odd_gemnoc, + [SLAVE_SERVICE_GEM_NOC] = &srvc_sys_gemnoc, +}; + +static struct qcom_icc_desc sm8250_gem_noc = { + .nodes = gem_noc_nodes, + .num_nodes = ARRAY_SIZE(gem_noc_nodes), + .bcms = gem_noc_bcms, + .num_bcms = ARRAY_SIZE(gem_noc_bcms), +}; + +static struct qcom_icc_bcm *ipa_virt_bcms[] = { + &bcm_ip0, +}; + +static struct qcom_icc_node *ipa_virt_nodes[] = { + [MASTER_IPA_CORE] = &ipa_core_master, + [SLAVE_IPA_CORE] = &ipa_core_slave, +}; + +static struct qcom_icc_desc sm8250_ipa_virt = { + .nodes = ipa_virt_nodes, + .num_nodes = ARRAY_SIZE(ipa_virt_nodes), + .bcms = ipa_virt_bcms, + .num_bcms = ARRAY_SIZE(ipa_virt_bcms), +}; + +static struct qcom_icc_bcm *mc_virt_bcms[] = { + &bcm_acv, + &bcm_mc0, +}; + +static struct qcom_icc_node *mc_virt_nodes[] = { + [MASTER_LLCC] = &llcc_mc, + [SLAVE_EBI_CH0] = &ebi, +}; + +static struct qcom_icc_desc sm8250_mc_virt = { + .nodes = mc_virt_nodes, + .num_nodes = ARRAY_SIZE(mc_virt_nodes), + .bcms = mc_virt_bcms, + .num_bcms = ARRAY_SIZE(mc_virt_bcms), +}; + +static struct qcom_icc_bcm *mmss_noc_bcms[] = { + &bcm_mm0, + &bcm_mm1, + &bcm_mm2, + &bcm_mm3, +}; + +static struct qcom_icc_node *mmss_noc_nodes[] = { + [MASTER_CNOC_MNOC_CFG] = &qhm_mnoc_cfg, + [MASTER_CAMNOC_HF] = &qnm_camnoc_hf, + [MASTER_CAMNOC_ICP] = &qnm_camnoc_icp, + [MASTER_CAMNOC_SF] = &qnm_camnoc_sf, + [MASTER_VIDEO_P0] = &qnm_video0, + [MASTER_VIDEO_P1] = &qnm_video1, + [MASTER_VIDEO_PROC] = &qnm_video_cvp, + [MASTER_MDP_PORT0] = &qxm_mdp0, + [MASTER_MDP_PORT1] = &qxm_mdp1, + [MASTER_ROTATOR] = &qxm_rot, + [SLAVE_MNOC_HF_MEM_NOC] = &qns_mem_noc_hf, + [SLAVE_MNOC_SF_MEM_NOC] = &qns_mem_noc_sf, + [SLAVE_SERVICE_MNOC] = &srvc_mnoc, +}; + +static struct qcom_icc_desc sm8250_mmss_noc = { + .nodes = mmss_noc_nodes, + .num_nodes = ARRAY_SIZE(mmss_noc_nodes), + .bcms = mmss_noc_bcms, + .num_bcms = ARRAY_SIZE(mmss_noc_bcms), +}; + +static struct qcom_icc_bcm *npu_noc_bcms[] = { +}; + +static struct qcom_icc_node *npu_noc_nodes[] = { + [MASTER_NPU_SYS] = &amm_npu_sys, + [MASTER_NPU_CDP] = &amm_npu_sys_cdp_w, + [MASTER_NPU_NOC_CFG] = &qhm_cfg, + [SLAVE_NPU_CAL_DP0] = &qhs_cal_dp0, + [SLAVE_NPU_CAL_DP1] = &qhs_cal_dp1, + [SLAVE_NPU_CP] = &qhs_cp, + [SLAVE_NPU_INT_DMA_BWMON_CFG] = &qhs_dma_bwmon, + [SLAVE_NPU_DPM] = &qhs_dpm, + [SLAVE_ISENSE_CFG] = &qhs_isense, + [SLAVE_NPU_LLM_CFG] = &qhs_llm, + [SLAVE_NPU_TCM] = &qhs_tcm, + [SLAVE_NPU_COMPUTE_NOC] = &qns_npu_sys, + [SLAVE_SERVICE_NPU_NOC] = &srvc_noc, +}; + +static struct qcom_icc_desc sm8250_npu_noc = { + .nodes = npu_noc_nodes, + .num_nodes = ARRAY_SIZE(npu_noc_nodes), + .bcms = npu_noc_bcms, + .num_bcms = ARRAY_SIZE(npu_noc_bcms), +}; + +static struct qcom_icc_bcm *system_noc_bcms[] = { + &bcm_sn0, + &bcm_sn1, + &bcm_sn11, + &bcm_sn2, + &bcm_sn3, + &bcm_sn4, + &bcm_sn5, + &bcm_sn6, + &bcm_sn7, + &bcm_sn8, + &bcm_sn9, +}; + +static struct qcom_icc_node *system_noc_nodes[] = { + [MASTER_SNOC_CFG] = &qhm_snoc_cfg, + [A1NOC_SNOC_MAS] = &qnm_aggre1_noc, + [A2NOC_SNOC_MAS] = &qnm_aggre2_noc, + [MASTER_GEM_NOC_SNOC] = &qnm_gemnoc, + [MASTER_GEM_NOC_PCIE_SNOC] = &qnm_gemnoc_pcie, + [MASTER_PIMEM] = &qxm_pimem, + [MASTER_GIC] = &xm_gic, + [SLAVE_APPSS] = &qhs_apss, + [SNOC_CNOC_SLV] = &qns_cnoc, + [SLAVE_SNOC_GEM_NOC_GC] = &qns_gemnoc_gc, + [SLAVE_SNOC_GEM_NOC_SF] = &qns_gemnoc_sf, + [SLAVE_OCIMEM] = &qxs_imem, + [SLAVE_PIMEM] = &qxs_pimem, + [SLAVE_SERVICE_SNOC] = &srvc_snoc, + [SLAVE_PCIE_0] = &xs_pcie_0, + [SLAVE_PCIE_1] = &xs_pcie_1, + [SLAVE_PCIE_2] = &xs_pcie_modem, + [SLAVE_QDSS_STM] = &xs_qdss_stm, + [SLAVE_TCU] = &xs_sys_tcu_cfg, +}; + +static struct qcom_icc_desc sm8250_system_noc = { + .nodes = system_noc_nodes, + .num_nodes = ARRAY_SIZE(system_noc_nodes), + .bcms = system_noc_bcms, + .num_bcms = ARRAY_SIZE(system_noc_bcms), +}; + +static int qnoc_probe(struct platform_device *pdev) +{ + const struct qcom_icc_desc *desc; + struct icc_onecell_data *data; + struct icc_provider *provider; + struct qcom_icc_node **qnodes; + struct qcom_icc_provider *qp; + struct icc_node *node; + size_t num_nodes, i; + int ret; + + desc = device_get_match_data(&pdev->dev); + if (!desc) + return -EINVAL; + + qnodes = desc->nodes; + num_nodes = desc->num_nodes; + + qp = devm_kzalloc(&pdev->dev, sizeof(*qp), GFP_KERNEL); + if (!qp) + return -ENOMEM; + + data = devm_kcalloc(&pdev->dev, num_nodes, sizeof(*node), GFP_KERNEL); + if (!data) + return -ENOMEM; + + provider = &qp->provider; + provider->dev = &pdev->dev; + provider->set = qcom_icc_set; + provider->pre_aggregate = qcom_icc_pre_aggregate; + provider->aggregate = qcom_icc_aggregate; + provider->xlate = of_icc_xlate_onecell; + INIT_LIST_HEAD(&provider->nodes); + provider->data = data; + + qp->dev = &pdev->dev; + qp->bcms = desc->bcms; + qp->num_bcms = desc->num_bcms; + + qp->voter = of_bcm_voter_get(qp->dev, NULL); + if (IS_ERR(qp->voter)) + return PTR_ERR(qp->voter); + + ret = icc_provider_add(provider); + if (ret) { + dev_err(&pdev->dev, "error adding interconnect provider\n"); + return ret; + } + + for (i = 0; i < num_nodes; i++) { + size_t j; + + if (!qnodes[i]) + continue; + + node = icc_node_create(qnodes[i]->id); + if (IS_ERR(node)) { + ret = PTR_ERR(node); + goto err; + } + + node->name = qnodes[i]->name; + node->data = qnodes[i]; + icc_node_add(node, provider); + + for (j = 0; j < qnodes[i]->num_links; j++) + icc_link_create(node, qnodes[i]->links[j]); + + data->nodes[i] = node; + } + data->num_nodes = num_nodes; + + for (i = 0; i < qp->num_bcms; i++) + qcom_icc_bcm_init(qp->bcms[i], &pdev->dev); + + platform_set_drvdata(pdev, qp); + + return 0; +err: + icc_nodes_remove(provider); + icc_provider_del(provider); + return ret; +} + +static int qnoc_remove(struct platform_device *pdev) +{ + struct qcom_icc_provider *qp = platform_get_drvdata(pdev); + + icc_nodes_remove(&qp->provider); + return icc_provider_del(&qp->provider); +} + +static const struct of_device_id qnoc_of_match[] = { + { .compatible = "qcom,sm8250-aggre1-noc", + .data = &sm8250_aggre1_noc}, + { .compatible = "qcom,sm8250-aggre2-noc", + .data = &sm8250_aggre2_noc}, + { .compatible = "qcom,sm8250-compute-noc", + .data = &sm8250_compute_noc}, + { .compatible = "qcom,sm8250-config-noc", + .data = &sm8250_config_noc}, + { .compatible = "qcom,sm8250-dc-noc", + .data = &sm8250_dc_noc}, + { .compatible = "qcom,sm8250-gem-noc", + .data = &sm8250_gem_noc}, + { .compatible = "qcom,sm8250-ipa-virt", + .data = &sm8250_ipa_virt}, + { .compatible = "qcom,sm8250-mc-virt", + .data = &sm8250_mc_virt}, + { .compatible = "qcom,sm8250-mmss-noc", + .data = &sm8250_mmss_noc}, + { .compatible = "qcom,sm8250-npu-noc", + .data = &sm8250_npu_noc}, + { .compatible = "qcom,sm8250-system-noc", + .data = &sm8250_system_noc}, + { } +}; +MODULE_DEVICE_TABLE(of, qnoc_of_match); + +static struct platform_driver qnoc_driver = { + .probe = qnoc_probe, + .remove = qnoc_remove, + .driver = { + .name = "qnoc-sm8250", + .of_match_table = qnoc_of_match, + }, +}; +module_platform_driver(qnoc_driver); + +MODULE_DESCRIPTION("Qualcomm SM8250 NoC driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/interconnect/qcom/sm8250.h b/drivers/interconnect/qcom/sm8250.h new file mode 100644 index 000000000000..b31fb431a20f --- /dev/null +++ b/drivers/interconnect/qcom/sm8250.h @@ -0,0 +1,164 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Qualcomm #define SM8250 interconnect IDs + * + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + */ + +#ifndef __DRIVERS_INTERCONNECT_QCOM_SM8250_H +#define __DRIVERS_INTERCONNECT_QCOM_SM8250_H + +#define SM8250_A1NOC_SNOC_MAS 0 +#define SM8250_A1NOC_SNOC_SLV 1 +#define SM8250_A2NOC_SNOC_MAS 2 +#define SM8250_A2NOC_SNOC_SLV 3 +#define SM8250_MASTER_A1NOC_CFG 4 +#define SM8250_MASTER_A2NOC_CFG 5 +#define SM8250_MASTER_AMPSS_M0 6 +#define SM8250_MASTER_ANOC_PCIE_GEM_NOC 7 +#define SM8250_MASTER_CAMNOC_HF 8 +#define SM8250_MASTER_CAMNOC_ICP 9 +#define SM8250_MASTER_CAMNOC_SF 10 +#define SM8250_MASTER_CNOC_A2NOC 11 +#define SM8250_MASTER_CNOC_DC_NOC 12 +#define SM8250_MASTER_CNOC_MNOC_CFG 13 +#define SM8250_MASTER_COMPUTE_NOC 14 +#define SM8250_MASTER_CRYPTO_CORE_0 15 +#define SM8250_MASTER_GEM_NOC_CFG 16 +#define SM8250_MASTER_GEM_NOC_PCIE_SNOC 17 +#define SM8250_MASTER_GEM_NOC_SNOC 18 +#define SM8250_MASTER_GIC 19 +#define SM8250_MASTER_GPU_TCU 20 +#define SM8250_MASTER_GRAPHICS_3D 21 +#define SM8250_MASTER_IPA 22 +#define SM8250_MASTER_IPA_CORE 23 +#define SM8250_MASTER_LLCC 24 +#define SM8250_MASTER_MDP_PORT0 25 +#define SM8250_MASTER_MDP_PORT1 26 +#define SM8250_MASTER_MNOC_HF_MEM_NOC 27 +#define SM8250_MASTER_MNOC_SF_MEM_NOC 28 +#define SM8250_MASTER_NPU 29 +#define SM8250_MASTER_NPU_CDP 30 +#define SM8250_MASTER_NPU_NOC_CFG 31 +#define SM8250_MASTER_NPU_SYS 32 +#define SM8250_MASTER_PCIE 33 +#define SM8250_MASTER_PCIE_1 34 +#define SM8250_MASTER_PCIE_2 35 +#define SM8250_MASTER_PIMEM 36 +#define SM8250_MASTER_QDSS_BAM 37 +#define SM8250_MASTER_QDSS_DAP 38 +#define SM8250_MASTER_QDSS_ETR 39 +#define SM8250_MASTER_QSPI_0 40 +#define SM8250_MASTER_QUP_0 41 +#define SM8250_MASTER_QUP_1 42 +#define SM8250_MASTER_QUP_2 43 +#define SM8250_MASTER_ROTATOR 44 +#define SM8250_MASTER_SDCC_2 45 +#define SM8250_MASTER_SDCC_4 46 +#define SM8250_MASTER_SNOC_CFG 47 +#define SM8250_MASTER_SNOC_GC_MEM_NOC 48 +#define SM8250_MASTER_SNOC_SF_MEM_NOC 49 +#define SM8250_MASTER_SYS_TCU 50 +#define SM8250_MASTER_TSIF 51 +#define SM8250_MASTER_UFS_CARD 52 +#define SM8250_MASTER_UFS_MEM 53 +#define SM8250_MASTER_USB3 54 +#define SM8250_MASTER_USB3_1 55 +#define SM8250_MASTER_VIDEO_P0 56 +#define SM8250_MASTER_VIDEO_P1 57 +#define SM8250_MASTER_VIDEO_PROC 58 +#define SM8250_SLAVE_A1NOC_CFG 59 +#define SM8250_SLAVE_A2NOC_CFG 60 +#define SM8250_SLAVE_AHB2PHY_NORTH 61 +#define SM8250_SLAVE_AHB2PHY_SOUTH 62 +#define SM8250_SLAVE_ANOC_PCIE_GEM_NOC 63 +#define SM8250_SLAVE_ANOC_PCIE_GEM_NOC_1 64 +#define SM8250_SLAVE_AOSS 65 +#define SM8250_SLAVE_APPSS 66 +#define SM8250_SLAVE_CAMERA_CFG 67 +#define SM8250_SLAVE_CDSP_CFG 68 +#define SM8250_SLAVE_CDSP_MEM_NOC 69 +#define SM8250_SLAVE_CLK_CTL 70 +#define SM8250_SLAVE_CNOC_A2NOC 71 +#define SM8250_SLAVE_CNOC_DDRSS 72 +#define SM8250_SLAVE_CNOC_MNOC_CFG 73 +#define SM8250_SLAVE_CRYPTO_0_CFG 74 +#define SM8250_SLAVE_CX_RDPM 75 +#define SM8250_SLAVE_DCC_CFG 76 +#define SM8250_SLAVE_DISPLAY_CFG 77 +#define SM8250_SLAVE_EBI_CH0 78 +#define SM8250_SLAVE_GEM_NOC_CFG 79 +#define SM8250_SLAVE_GEM_NOC_SNOC 80 +#define SM8250_SLAVE_GRAPHICS_3D_CFG 81 +#define SM8250_SLAVE_IMEM_CFG 82 +#define SM8250_SLAVE_IPA_CFG 83 +#define SM8250_SLAVE_IPA_CORE 84 +#define SM8250_SLAVE_IPC_ROUTER_CFG 85 +#define SM8250_SLAVE_ISENSE_CFG 86 +#define SM8250_SLAVE_LLCC 87 +#define SM8250_SLAVE_LLCC_CFG 88 +#define SM8250_SLAVE_LPASS 89 +#define SM8250_SLAVE_MEM_NOC_PCIE_SNOC 90 +#define SM8250_SLAVE_MNOC_HF_MEM_NOC 91 +#define SM8250_SLAVE_MNOC_SF_MEM_NOC 92 +#define SM8250_SLAVE_NPU_CAL_DP0 93 +#define SM8250_SLAVE_NPU_CAL_DP1 94 +#define SM8250_SLAVE_NPU_CFG 95 +#define SM8250_SLAVE_NPU_COMPUTE_NOC 96 +#define SM8250_SLAVE_NPU_CP 97 +#define SM8250_SLAVE_NPU_DPM 98 +#define SM8250_SLAVE_NPU_INT_DMA_BWMON_CFG 99 +#define SM8250_SLAVE_NPU_LLM_CFG 100 +#define SM8250_SLAVE_NPU_TCM 101 +#define SM8250_SLAVE_OCIMEM 102 +#define SM8250_SLAVE_PCIE_0 103 +#define SM8250_SLAVE_PCIE_0_CFG 104 +#define SM8250_SLAVE_PCIE_1 105 +#define SM8250_SLAVE_PCIE_1_CFG 106 +#define SM8250_SLAVE_PCIE_2 107 +#define SM8250_SLAVE_PCIE_2_CFG 108 +#define SM8250_SLAVE_PDM 109 +#define SM8250_SLAVE_PIMEM 110 +#define SM8250_SLAVE_PIMEM_CFG 111 +#define SM8250_SLAVE_PRNG 112 +#define SM8250_SLAVE_QDSS_CFG 113 +#define SM8250_SLAVE_QDSS_STM 114 +#define SM8250_SLAVE_QSPI_0 115 +#define SM8250_SLAVE_QUP_0 116 +#define SM8250_SLAVE_QUP_1 117 +#define SM8250_SLAVE_QUP_2 118 +#define SM8250_SLAVE_RBCPR_CX_CFG 119 +#define SM8250_SLAVE_RBCPR_MMCX_CFG 120 +#define SM8250_SLAVE_RBCPR_MX_CFG 121 +#define SM8250_SLAVE_SDCC_2 122 +#define SM8250_SLAVE_SDCC_4 123 +#define SM8250_SLAVE_SERVICE_A1NOC 124 +#define SM8250_SLAVE_SERVICE_A2NOC 125 +#define SM8250_SLAVE_SERVICE_CNOC 126 +#define SM8250_SLAVE_SERVICE_GEM_NOC 127 +#define SM8250_SLAVE_SERVICE_GEM_NOC_1 128 +#define SM8250_SLAVE_SERVICE_GEM_NOC_2 129 +#define SM8250_SLAVE_SERVICE_MNOC 130 +#define SM8250_SLAVE_SERVICE_NPU_NOC 131 +#define SM8250_SLAVE_SERVICE_SNOC 132 +#define SM8250_SLAVE_SNOC_CFG 133 +#define SM8250_SLAVE_SNOC_GEM_NOC_GC 134 +#define SM8250_SLAVE_SNOC_GEM_NOC_SF 135 +#define SM8250_SLAVE_TCSR 136 +#define SM8250_SLAVE_TCU 137 +#define SM8250_SLAVE_TLMM_NORTH 138 +#define SM8250_SLAVE_TLMM_SOUTH 139 +#define SM8250_SLAVE_TLMM_WEST 140 +#define SM8250_SLAVE_TSIF 141 +#define SM8250_SLAVE_UFS_CARD_CFG 142 +#define SM8250_SLAVE_UFS_MEM_CFG 143 +#define SM8250_SLAVE_USB3 144 +#define SM8250_SLAVE_USB3_1 145 +#define SM8250_SLAVE_VENUS_CFG 146 +#define SM8250_SLAVE_VSENSE_CTRL_CFG 147 +#define SM8250_SNOC_CNOC_MAS 148 +#define SM8250_SNOC_CNOC_SLV 149 +#define SM8250_MASTER_EPSS_L3_APPS 150 +#define SM8250_SLAVE_EPSS_L3 151 + +#endif diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index ce136d685d14..d5ce8082b0a0 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -456,6 +456,16 @@ config PVPANIC a paravirtualized device provided by QEMU; it lets a virtual machine (guest) communicate panic events to the host. +config HISI_HIKEY_USB + tristate "USB GPIO Hub on HiSilicon Hikey 960/970 Platform" + depends on (OF && GPIOLIB) || COMPILE_TEST + depends on USB_ROLE_SWITCH + help + If you say yes here this adds support for the on-board USB GPIO hub + found on HiKey 960/970 boards, which is necessary to support + switching between the dual-role USB-C port and the USB-A host ports + using only one USB controller. + source "drivers/misc/c2port/Kconfig" source "drivers/misc/eeprom/Kconfig" source "drivers/misc/cb710/Kconfig" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index c7bd01ac6291..2521359e8ef7 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -57,3 +57,4 @@ obj-$(CONFIG_PVPANIC) += pvpanic.o obj-$(CONFIG_HABANA_AI) += habanalabs/ obj-$(CONFIG_UACCE) += uacce/ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o +obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o diff --git a/drivers/misc/cardreader/rts5227.c b/drivers/misc/cardreader/rts5227.c index f5f392ddf3d6..8859011672cb 100644 --- a/drivers/misc/cardreader/rts5227.c +++ b/drivers/misc/cardreader/rts5227.c @@ -72,28 +72,80 @@ static void rts5227_fetch_vendor_settings(struct rtsx_pcr *pcr) pci_read_config_dword(pdev, PCR_SETTING_REG2, ®); pcr_dbg(pcr, "Cfg 0x%x: 0x%x\n", PCR_SETTING_REG2, reg); + if (rtsx_check_mmc_support(reg)) + pcr->extra_caps |= EXTRA_CAPS_NO_MMC; pcr->sd30_drive_sel_3v3 = rtsx_reg_to_sd30_drive_sel_3v3(reg); if (rtsx_reg_check_reverse_socket(reg)) pcr->flags |= PCR_REVERSE_SOCKET; } -static void rts5227_force_power_down(struct rtsx_pcr *pcr, u8 pm_state) +static void rts5227_init_from_cfg(struct rtsx_pcr *pcr) { - /* Set relink_time to 0 */ - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 1, 0xFF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 2, 0xFF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 3, 0x01, 0); + struct pci_dev *pdev = pcr->pci; + int l1ss; + u32 lval; + struct rtsx_cr_option *option = &pcr->option; + + l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); + if (!l1ss) + return; + + pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); - if (pm_state == HOST_ENTER_S3) - rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, 0x10, 0x10); + if (CHK_PCI_PID(pcr, 0x522A)) { + if (0 == (lval & 0x0F)) + rtsx_pci_enable_oobs_polling(pcr); + else + rtsx_pci_disable_oobs_polling(pcr); + } + + if (lval & PCI_L1SS_CTL1_ASPM_L1_1) + rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); + else + rtsx_clear_dev_flag(pcr, ASPM_L1_1_EN); + + if (lval & PCI_L1SS_CTL1_ASPM_L1_2) + rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); + else + rtsx_clear_dev_flag(pcr, ASPM_L1_2_EN); + + if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) + rtsx_set_dev_flag(pcr, PM_L1_1_EN); + else + rtsx_clear_dev_flag(pcr, PM_L1_1_EN); + + if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) + rtsx_set_dev_flag(pcr, PM_L1_2_EN); + else + rtsx_clear_dev_flag(pcr, PM_L1_2_EN); + + if (option->ltr_en) { + u16 val; + + pcie_capability_read_word(pcr->pci, PCI_EXP_DEVCTL2, &val); + if (val & PCI_EXP_DEVCTL2_LTR_EN) { + option->ltr_enabled = true; + option->ltr_active = true; + rtsx_set_ltr_latency(pcr, option->ltr_active_latency); + } else { + option->ltr_enabled = false; + } + } + + if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN + | PM_L1_1_EN | PM_L1_2_EN)) + option->force_clkreq_0 = false; + else + option->force_clkreq_0 = true; - rtsx_pci_write_register(pcr, FPDCTL, 0x03, 0x03); } static int rts5227_extra_init_hw(struct rtsx_pcr *pcr) { u16 cap; + struct rtsx_cr_option *option = &pcr->option; + rts5227_init_from_cfg(pcr); rtsx_pci_init_cmd(pcr); /* Configure GPIO as output */ @@ -115,9 +167,17 @@ static int rts5227_extra_init_hw(struct rtsx_pcr *pcr) rts5227_fill_driving(pcr, OUTPUT_3V3); /* Configure force_clock_req */ if (pcr->flags & PCR_REVERSE_SOCKET) - rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, 0xB8, 0xB8); + rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, 0x30, 0x30); + else + rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, 0x30, 0x00); + + if (option->force_clkreq_0) + rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); else - rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, 0xB8, 0x88); + rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); + rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, pcr->reg_pm_ctrl3, 0x10, 0x00); return rtsx_pci_send_cmd(pcr, 100); @@ -239,7 +299,6 @@ static const struct pcr_ops rts5227_pcr_ops = { .switch_output_voltage = rts5227_switch_output_voltage, .cd_deglitch = NULL, .conv_clk_and_div_n = NULL, - .force_power_down = rts5227_force_power_down, }; /* SD Pull Control Enable: @@ -373,6 +432,27 @@ static int rts522a_switch_output_voltage(struct rtsx_pcr *pcr, u8 voltage) return rtsx_pci_send_cmd(pcr, 100); } +static void rts522a_set_l1off_cfg_sub_d0(struct rtsx_pcr *pcr, int active) +{ + struct rtsx_cr_option *option = &pcr->option; + int aspm_L1_1, aspm_L1_2; + u8 val = 0; + + aspm_L1_1 = rtsx_check_dev_flag(pcr, ASPM_L1_1_EN); + aspm_L1_2 = rtsx_check_dev_flag(pcr, ASPM_L1_2_EN); + + if (active) { + /* run, latency: 60us */ + if (aspm_L1_1) + val = option->ltr_l1off_snooze_sspwrgate; + } else { + /* l1off, latency: 300us */ + if (aspm_L1_2) + val = option->ltr_l1off_sspwrgate; + } + + rtsx_set_l1off_sub(pcr, val); +} /* rts522a operations mainly derived from rts5227, except phy/hw init setting. */ @@ -389,16 +469,29 @@ static const struct pcr_ops rts522a_pcr_ops = { .switch_output_voltage = rts522a_switch_output_voltage, .cd_deglitch = NULL, .conv_clk_and_div_n = NULL, - .force_power_down = rts5227_force_power_down, + .set_l1off_cfg_sub_d0 = rts522a_set_l1off_cfg_sub_d0, }; void rts522a_init_params(struct rtsx_pcr *pcr) { + struct rtsx_cr_option *option = &pcr->option; + rts5227_init_params(pcr); pcr->ops = &rts522a_pcr_ops; pcr->tx_initial_phase = SET_CLOCK_PHASE(20, 20, 11); pcr->reg_pm_ctrl3 = RTS522A_PM_CTRL3; + option->dev_flags = LTR_L1SS_PWR_GATE_EN; + option->ltr_en = true; + + /* init latency of active, idle, L1OFF to 60us, 300us, 3ms */ + option->ltr_active_latency = LTR_ACTIVE_LATENCY_DEF; + option->ltr_idle_latency = LTR_IDLE_LATENCY_DEF; + option->ltr_l1off_latency = LTR_L1OFF_LATENCY_DEF; + option->l1_snooze_delay = L1_SNOOZE_DELAY_DEF; + option->ltr_l1off_sspwrgate = 0x7F; + option->ltr_l1off_snooze_sspwrgate = 0x78; + pcr->option.ocp_en = 1; if (pcr->option.ocp_en) pcr->hw_param.interrupt_en |= SD_OC_INT_EN; diff --git a/drivers/misc/cardreader/rts5228.c b/drivers/misc/cardreader/rts5228.c index 28feab1449ab..781a86def59a 100644 --- a/drivers/misc/cardreader/rts5228.c +++ b/drivers/misc/cardreader/rts5228.c @@ -99,9 +99,8 @@ static void rts5228_force_power_down(struct rtsx_pcr *pcr, u8 pm_state) rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 3, RELINK_TIME_MASK, 0); - if (pm_state == HOST_ENTER_S3) - rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, - D3_DELINK_MODE_EN, D3_DELINK_MODE_EN); + rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, + D3_DELINK_MODE_EN, D3_DELINK_MODE_EN); rtsx_pci_write_register(pcr, FPDCTL, SSC_POWER_DOWN, SSC_POWER_DOWN); diff --git a/drivers/misc/cardreader/rts5249.c b/drivers/misc/cardreader/rts5249.c index 941b3d77f1e9..b85279f1fc5e 100644 --- a/drivers/misc/cardreader/rts5249.c +++ b/drivers/misc/cardreader/rts5249.c @@ -73,25 +73,13 @@ static void rtsx_base_fetch_vendor_settings(struct rtsx_pcr *pcr) pci_read_config_dword(pdev, PCR_SETTING_REG2, ®); pcr_dbg(pcr, "Cfg 0x%x: 0x%x\n", PCR_SETTING_REG2, reg); + if (rtsx_check_mmc_support(reg)) + pcr->extra_caps |= EXTRA_CAPS_NO_MMC; pcr->sd30_drive_sel_3v3 = rtsx_reg_to_sd30_drive_sel_3v3(reg); if (rtsx_reg_check_reverse_socket(reg)) pcr->flags |= PCR_REVERSE_SOCKET; } -static void rtsx_base_force_power_down(struct rtsx_pcr *pcr, u8 pm_state) -{ - /* Set relink_time to 0 */ - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 1, 0xFF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 2, 0xFF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 3, 0x01, 0); - - if (pm_state == HOST_ENTER_S3) - rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, - D3_DELINK_MODE_EN, D3_DELINK_MODE_EN); - - rtsx_pci_write_register(pcr, FPDCTL, 0x03, 0x03); -} - static void rts5249_init_from_cfg(struct rtsx_pcr *pcr) { struct pci_dev *pdev = pcr->pci; @@ -105,6 +93,14 @@ static void rts5249_init_from_cfg(struct rtsx_pcr *pcr) pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); + if (CHK_PCI_PID(pcr, PID_524A) || CHK_PCI_PID(pcr, PID_525A)) { + if (0 == (lval & 0x0F)) + rtsx_pci_enable_oobs_polling(pcr); + else + rtsx_pci_disable_oobs_polling(pcr); + } + + if (lval & PCI_L1SS_CTL1_ASPM_L1_1) rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); @@ -144,6 +140,112 @@ static int rts5249_init_from_hw(struct rtsx_pcr *pcr) return 0; } +static void rts52xa_save_content_from_efuse(struct rtsx_pcr *pcr) +{ + u8 cnt, sv; + u16 j = 0; + u8 tmp; + u8 val; + int i; + + rtsx_pci_write_register(pcr, RTS524A_PME_FORCE_CTL, + REG_EFUSE_BYPASS | REG_EFUSE_POR, REG_EFUSE_POR); + udelay(1); + + pcr_dbg(pcr, "Enable efuse por!"); + pcr_dbg(pcr, "save efuse to autoload"); + + rtsx_pci_write_register(pcr, RTS525A_EFUSE_ADD, REG_EFUSE_ADD_MASK, 0x00); + rtsx_pci_write_register(pcr, RTS525A_EFUSE_CTL, + REG_EFUSE_ENABLE | REG_EFUSE_MODE, REG_EFUSE_ENABLE); + /* Wait transfer end */ + for (j = 0; j < 1024; j++) { + rtsx_pci_read_register(pcr, RTS525A_EFUSE_CTL, &tmp); + if ((tmp & 0x80) == 0) + break; + } + rtsx_pci_read_register(pcr, RTS525A_EFUSE_DATA, &val); + cnt = val & 0x0F; + sv = val & 0x10; + + if (sv) { + for (i = 0; i < 4; i++) { + rtsx_pci_write_register(pcr, RTS525A_EFUSE_ADD, + REG_EFUSE_ADD_MASK, 0x04 + i); + rtsx_pci_write_register(pcr, RTS525A_EFUSE_CTL, + REG_EFUSE_ENABLE | REG_EFUSE_MODE, REG_EFUSE_ENABLE); + /* Wait transfer end */ + for (j = 0; j < 1024; j++) { + rtsx_pci_read_register(pcr, RTS525A_EFUSE_CTL, &tmp); + if ((tmp & 0x80) == 0) + break; + } + rtsx_pci_read_register(pcr, RTS525A_EFUSE_DATA, &val); + rtsx_pci_write_register(pcr, 0xFF04 + i, 0xFF, val); + } + } else { + rtsx_pci_write_register(pcr, 0xFF04, 0xFF, (u8)PCI_VID(pcr)); + rtsx_pci_write_register(pcr, 0xFF05, 0xFF, (u8)(PCI_VID(pcr) >> 8)); + rtsx_pci_write_register(pcr, 0xFF06, 0xFF, (u8)PCI_PID(pcr)); + rtsx_pci_write_register(pcr, 0xFF07, 0xFF, (u8)(PCI_PID(pcr) >> 8)); + } + + for (i = 0; i < cnt * 4; i++) { + if (sv) + rtsx_pci_write_register(pcr, RTS525A_EFUSE_ADD, + REG_EFUSE_ADD_MASK, 0x08 + i); + else + rtsx_pci_write_register(pcr, RTS525A_EFUSE_ADD, + REG_EFUSE_ADD_MASK, 0x04 + i); + rtsx_pci_write_register(pcr, RTS525A_EFUSE_CTL, + REG_EFUSE_ENABLE | REG_EFUSE_MODE, REG_EFUSE_ENABLE); + /* Wait transfer end */ + for (j = 0; j < 1024; j++) { + rtsx_pci_read_register(pcr, RTS525A_EFUSE_CTL, &tmp); + if ((tmp & 0x80) == 0) + break; + } + rtsx_pci_read_register(pcr, RTS525A_EFUSE_DATA, &val); + rtsx_pci_write_register(pcr, 0xFF08 + i, 0xFF, val); + } + rtsx_pci_write_register(pcr, 0xFF00, 0xFF, (cnt & 0x7F) | 0x80); + rtsx_pci_write_register(pcr, RTS524A_PME_FORCE_CTL, + REG_EFUSE_BYPASS | REG_EFUSE_POR, REG_EFUSE_BYPASS); + pcr_dbg(pcr, "Disable efuse por!"); +} + +static void rts52xa_save_content_to_autoload_space(struct rtsx_pcr *pcr) +{ + u8 val; + + rtsx_pci_read_register(pcr, RESET_LOAD_REG, &val); + if (val & 0x02) { + rtsx_pci_read_register(pcr, RTS525A_BIOS_CFG, &val); + if (val & RTS525A_LOAD_BIOS_FLAG) { + rtsx_pci_write_register(pcr, RTS525A_BIOS_CFG, + RTS525A_LOAD_BIOS_FLAG, RTS525A_CLEAR_BIOS_FLAG); + + rtsx_pci_write_register(pcr, RTS524A_PME_FORCE_CTL, + REG_EFUSE_POWER_MASK, REG_EFUSE_POWERON); + + pcr_dbg(pcr, "Power ON efuse!"); + mdelay(1); + rts52xa_save_content_from_efuse(pcr); + } else { + rtsx_pci_read_register(pcr, RTS524A_PME_FORCE_CTL, &val); + if (!(val & 0x08)) + rts52xa_save_content_from_efuse(pcr); + } + } else { + pcr_dbg(pcr, "Load from autoload"); + rtsx_pci_write_register(pcr, 0xFF00, 0xFF, 0x80); + rtsx_pci_write_register(pcr, 0xFF04, 0xFF, (u8)PCI_VID(pcr)); + rtsx_pci_write_register(pcr, 0xFF05, 0xFF, (u8)(PCI_VID(pcr) >> 8)); + rtsx_pci_write_register(pcr, 0xFF06, 0xFF, (u8)PCI_PID(pcr)); + rtsx_pci_write_register(pcr, 0xFF07, 0xFF, (u8)(PCI_PID(pcr) >> 8)); + } +} + static int rts5249_extra_init_hw(struct rtsx_pcr *pcr) { struct rtsx_cr_option *option = &(pcr->option); @@ -153,6 +255,9 @@ static int rts5249_extra_init_hw(struct rtsx_pcr *pcr) rtsx_pci_init_cmd(pcr); + if (CHK_PCI_PID(pcr, PID_524A) || CHK_PCI_PID(pcr, PID_525A)) + rts52xa_save_content_to_autoload_space(pcr); + /* Rest L1SUB Config */ rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, L1SUB_CONFIG3, 0xFF, 0x00); /* Configure GPIO as output */ @@ -171,18 +276,36 @@ static int rts5249_extra_init_hw(struct rtsx_pcr *pcr) else rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, 0xB0, 0x80); + rtsx_pci_send_cmd(pcr, CMD_TIMEOUT_DEF); + + if (CHK_PCI_PID(pcr, PID_524A) || CHK_PCI_PID(pcr, PID_525A)) { + rtsx_pci_write_register(pcr, REG_VREF, PWD_SUSPND_EN, PWD_SUSPND_EN); + rtsx_pci_write_register(pcr, RTS524A_PM_CTRL3, 0x01, 0x00); + rtsx_pci_write_register(pcr, RTS524A_PME_FORCE_CTL, 0x30, 0x20); + } else { + rtsx_pci_write_register(pcr, PME_FORCE_CTL, 0xFF, 0x30); + rtsx_pci_write_register(pcr, PM_CTRL3, 0x01, 0x00); + } + /* * If u_force_clkreq_0 is enabled, CLKREQ# PIN will be forced * to drive low, and we forcibly request clock. */ if (option->force_clkreq_0) - rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, + rtsx_pci_write_register(pcr, PETXCFG, FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); else - rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, + rtsx_pci_write_register(pcr, PETXCFG, FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); - return rtsx_pci_send_cmd(pcr, CMD_TIMEOUT_DEF); + rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, 0x10, 0x00); + if (CHK_PCI_PID(pcr, PID_524A) || CHK_PCI_PID(pcr, PID_525A)) { + rtsx_pci_write_register(pcr, RTS524A_PME_FORCE_CTL, + REG_EFUSE_POWER_MASK, REG_EFUSE_POWEROFF); + pcr_dbg(pcr, "Power OFF efuse!"); + } + + return 0; } static int rts5249_optimize_phy(struct rtsx_pcr *pcr) @@ -360,7 +483,6 @@ static const struct pcr_ops rts5249_pcr_ops = { .card_power_on = rtsx_base_card_power_on, .card_power_off = rtsx_base_card_power_off, .switch_output_voltage = rtsx_base_switch_output_voltage, - .force_power_down = rtsx_base_force_power_down, }; /* SD Pull Control Enable: @@ -585,7 +707,6 @@ static const struct pcr_ops rts524a_pcr_ops = { .card_power_on = rtsx_base_card_power_on, .card_power_off = rtsx_base_card_power_off, .switch_output_voltage = rtsx_base_switch_output_voltage, - .force_power_down = rtsx_base_force_power_down, .set_l1off_cfg_sub_d0 = rts5250_set_l1off_cfg_sub_d0, }; @@ -668,6 +789,8 @@ static int rts525a_extra_init_hw(struct rtsx_pcr *pcr) { rts5249_extra_init_hw(pcr); + rtsx_pci_write_register(pcr, RTS5250_CLK_CFG3, RTS525A_CFG_MEM_PD, RTS525A_CFG_MEM_PD); + rtsx_pci_write_register(pcr, PCLK_CTL, PCLK_MODE_SEL, PCLK_MODE_SEL); if (is_version(pcr, 0x525A, IC_VER_A)) { rtsx_pci_write_register(pcr, L1SUB_CONFIG2, @@ -700,7 +823,6 @@ static const struct pcr_ops rts525a_pcr_ops = { .card_power_on = rts525a_card_power_on, .card_power_off = rtsx_base_card_power_off, .switch_output_voltage = rts525a_switch_output_voltage, - .force_power_down = rtsx_base_force_power_down, .set_l1off_cfg_sub_d0 = rts5250_set_l1off_cfg_sub_d0, }; diff --git a/drivers/misc/cardreader/rts5260.c b/drivers/misc/cardreader/rts5260.c index b9f66b1384a6..080a7d67a8e1 100644 --- a/drivers/misc/cardreader/rts5260.c +++ b/drivers/misc/cardreader/rts5260.c @@ -26,21 +26,17 @@ static u8 rts5260_get_ic_version(struct rtsx_pcr *pcr) static void rts5260_fill_driving(struct rtsx_pcr *pcr, u8 voltage) { - u8 driving_3v3[6][3] = { - {0x94, 0x94, 0x94}, - {0x11, 0x11, 0x18}, - {0x55, 0x55, 0x5C}, - {0x94, 0x94, 0x94}, - {0x94, 0x94, 0x94}, - {0xFF, 0xFF, 0xFF}, + u8 driving_3v3[4][3] = { + {0x11, 0x11, 0x11}, + {0x22, 0x22, 0x22}, + {0x55, 0x55, 0x55}, + {0x33, 0x33, 0x33}, }; - u8 driving_1v8[6][3] = { - {0x9A, 0x89, 0x89}, - {0xC4, 0xC4, 0xC4}, - {0x3C, 0x3C, 0x3C}, + u8 driving_1v8[4][3] = { + {0x35, 0x33, 0x33}, + {0x8A, 0x88, 0x88}, + {0xBD, 0xBB, 0xBB}, {0x9B, 0x99, 0x99}, - {0x9A, 0x89, 0x89}, - {0xFE, 0xFE, 0xFE}, }; u8 (*driving)[3], drive_sel; @@ -58,7 +54,7 @@ static void rts5260_fill_driving(struct rtsx_pcr *pcr, u8 voltage) rtsx_pci_write_register(pcr, SD30_CMD_DRIVE_SEL, 0xFF, driving[drive_sel][1]); - rtsx_pci_write_register(pcr, SD30_CMD_DRIVE_SEL, + rtsx_pci_write_register(pcr, SD30_DAT_DRIVE_SEL, 0xFF, driving[drive_sel][2]); } @@ -82,26 +78,13 @@ static void rtsx_base_fetch_vendor_settings(struct rtsx_pcr *pcr) pci_read_config_dword(pdev, PCR_SETTING_REG2, ®); pcr_dbg(pcr, "Cfg 0x%x: 0x%x\n", PCR_SETTING_REG2, reg); + if (rtsx_check_mmc_support(reg)) + pcr->extra_caps |= EXTRA_CAPS_NO_MMC; pcr->sd30_drive_sel_3v3 = rtsx_reg_to_sd30_drive_sel_3v3(reg); if (rtsx_reg_check_reverse_socket(reg)) pcr->flags |= PCR_REVERSE_SOCKET; } -static void rtsx_base_force_power_down(struct rtsx_pcr *pcr, u8 pm_state) -{ - /* Set relink_time to 0 */ - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 1, MASK_8_BIT_DEF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 2, MASK_8_BIT_DEF, 0); - rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 3, - RELINK_TIME_MASK, 0); - - if (pm_state == HOST_ENTER_S3) - rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, - D3_DELINK_MODE_EN, D3_DELINK_MODE_EN); - - rtsx_pci_write_register(pcr, FPDCTL, ALL_POWER_DOWN, ALL_POWER_DOWN); -} - static int rtsx_base_enable_auto_blink(struct rtsx_pcr *pcr) { return rtsx_pci_write_register(pcr, OLT_LED_CTL, @@ -574,6 +557,8 @@ static int rts5260_extra_init_hw(struct rtsx_pcr *pcr) rtsx_pci_write_register(pcr, PETXCFG, FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); + rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, 0x10, 0x00); + return 0; } @@ -620,7 +605,6 @@ static const struct pcr_ops rts5260_pcr_ops = { .card_power_on = rts5260_card_power_on, .card_power_off = rts5260_card_power_off, .switch_output_voltage = rts5260_switch_output_voltage, - .force_power_down = rtsx_base_force_power_down, .stop_cmd = rts5260_stop_cmd, .set_l1off_cfg_sub_d0 = rts5260_set_l1off_cfg_sub_d0, .enable_ocp = rts5260_enable_ocp, diff --git a/drivers/misc/cardreader/rtsx_pcr.c b/drivers/misc/cardreader/rtsx_pcr.c index 37ccc67f4914..5d15607027e9 100644 --- a/drivers/misc/cardreader/rtsx_pcr.c +++ b/drivers/misc/cardreader/rtsx_pcr.c @@ -1096,6 +1096,20 @@ static void rtsx_pci_idle_work(struct work_struct *work) mutex_unlock(&pcr->pcr_mutex); } +static void rtsx_base_force_power_down(struct rtsx_pcr *pcr, u8 pm_state) +{ + /* Set relink_time to 0 */ + rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 1, MASK_8_BIT_DEF, 0); + rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 2, MASK_8_BIT_DEF, 0); + rtsx_pci_write_register(pcr, AUTOLOAD_CFG_BASE + 3, + RELINK_TIME_MASK, 0); + + rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, + D3_DELINK_MODE_EN, D3_DELINK_MODE_EN); + + rtsx_pci_write_register(pcr, FPDCTL, ALL_POWER_DOWN, ALL_POWER_DOWN); +} + static void __maybe_unused rtsx_pci_power_off(struct rtsx_pcr *pcr, u8 pm_state) { if (pcr->ops->turn_off_led) @@ -1109,6 +1123,8 @@ static void __maybe_unused rtsx_pci_power_off(struct rtsx_pcr *pcr, u8 pm_state) if (pcr->ops->force_power_down) pcr->ops->force_power_down(pcr, pm_state); + else + rtsx_base_force_power_down(pcr, pm_state); } void rtsx_pci_enable_ocp(struct rtsx_pcr *pcr) @@ -1155,10 +1171,6 @@ void rtsx_pci_init_ocp(struct rtsx_pcr *pcr) rtsx_pci_write_register(pcr, REG_OCPGLITCH, SD_OCP_GLITCH_MASK, pcr->hw_param.ocp_glitch); rtsx_pci_enable_ocp(pcr); - } else { - /* OC power down */ - rtsx_pci_write_register(pcr, FPDCTL, OC_POWER_DOWN, - OC_POWER_DOWN); } } } @@ -1562,12 +1574,14 @@ static int rtsx_pci_probe(struct pci_dev *pcidev, ret = mfd_add_devices(&pcidev->dev, pcr->id, rtsx_pcr_cells, ARRAY_SIZE(rtsx_pcr_cells), NULL, 0, NULL); if (ret < 0) - goto disable_irq; + goto free_slots; schedule_delayed_work(&pcr->idle_work, msecs_to_jiffies(200)); return 0; +free_slots: + kfree(pcr->slots); disable_irq: free_irq(pcr->irq, (void *)pcr); disable_msi: diff --git a/drivers/misc/cardreader/rtsx_pcr.h b/drivers/misc/cardreader/rtsx_pcr.h index 6b322db8738e..fe5f4ca0f937 100644 --- a/drivers/misc/cardreader/rtsx_pcr.h +++ b/drivers/misc/cardreader/rtsx_pcr.h @@ -18,7 +18,24 @@ #define RTS522A_PM_CTRL3 0xFF7E #define RTS524A_PME_FORCE_CTL 0xFF78 +#define REG_EFUSE_BYPASS 0x08 +#define REG_EFUSE_POR 0x04 +#define REG_EFUSE_POWER_MASK 0x03 +#define REG_EFUSE_POWERON 0x03 +#define REG_EFUSE_POWEROFF 0x00 +#define RTS5250_CLK_CFG3 0xFF79 +#define RTS525A_CFG_MEM_PD 0xF0 #define RTS524A_PM_CTRL3 0xFF7E +#define RTS525A_BIOS_CFG 0xFF2D +#define RTS525A_LOAD_BIOS_FLAG 0x01 +#define RTS525A_CLEAR_BIOS_FLAG 0x00 + +#define RTS525A_EFUSE_CTL 0xFC32 +#define REG_EFUSE_ENABLE 0x80 +#define REG_EFUSE_MODE 0x40 +#define RTS525A_EFUSE_ADD 0xFC33 +#define REG_EFUSE_ADD_MASK 0x3F +#define RTS525A_EFUSE_DATA 0xFC35 #define LTR_ACTIVE_LATENCY_DEF 0x883C #define LTR_IDLE_LATENCY_DEF 0x892C diff --git a/drivers/misc/eeprom/at25.c b/drivers/misc/eeprom/at25.c index ed8d38b09925..3b7d8b7584f4 100644 --- a/drivers/misc/eeprom/at25.c +++ b/drivers/misc/eeprom/at25.c @@ -261,7 +261,7 @@ static int at25_fw_to_chip(struct device *dev, struct spi_eeprom *chip) if (device_property_read_u32(dev, "pagesize", &val) == 0 || device_property_read_u32(dev, "at25,page-size", &val) == 0) { - chip->page_size = (u16)val; + chip->page_size = val; } else { dev_err(dev, "Error: missing \"pagesize\" property\n"); return -ENODEV; @@ -348,6 +348,7 @@ static int at25_probe(struct spi_device *spi) spi_set_drvdata(spi, at25); at25->addrlen = addrlen; + at25->nvmem_config.type = NVMEM_TYPE_EEPROM; at25->nvmem_config.name = dev_name(&spi->dev); at25->nvmem_config.dev = &spi->dev; at25->nvmem_config.read_only = chip.flags & EE_READONLY; @@ -358,7 +359,7 @@ static int at25_probe(struct spi_device *spi) at25->nvmem_config.reg_read = at25_ee_read; at25->nvmem_config.reg_write = at25_ee_write; at25->nvmem_config.priv = at25; - at25->nvmem_config.stride = 4; + at25->nvmem_config.stride = 1; at25->nvmem_config.word_size = 1; at25->nvmem_config.size = chip.byte_len; diff --git a/drivers/misc/eeprom/ee1004.c b/drivers/misc/eeprom/ee1004.c index b081c67416d7..252e15ba65e1 100644 --- a/drivers/misc/eeprom/ee1004.c +++ b/drivers/misc/eeprom/ee1004.c @@ -280,18 +280,7 @@ static struct i2c_driver ee1004_driver = { .remove = ee1004_remove, .id_table = ee1004_ids, }; - -static int __init ee1004_init(void) -{ - return i2c_add_driver(&ee1004_driver); -} -module_init(ee1004_init); - -static void __exit ee1004_exit(void) -{ - i2c_del_driver(&ee1004_driver); -} -module_exit(ee1004_exit); +module_i2c_driver(ee1004_driver); MODULE_DESCRIPTION("Driver for EE1004-compliant DDR4 SPD EEPROMs"); MODULE_AUTHOR("Jean Delvare"); diff --git a/drivers/misc/eeprom/eeprom_93xx46.c b/drivers/misc/eeprom/eeprom_93xx46.c index 94cfb675fe4e..7c45f82b4302 100644 --- a/drivers/misc/eeprom/eeprom_93xx46.c +++ b/drivers/misc/eeprom/eeprom_93xx46.c @@ -455,6 +455,7 @@ static int eeprom_93xx46_probe(struct spi_device *spi) edev->pdata = pd; edev->size = 128; + edev->nvmem_config.type = NVMEM_TYPE_EEPROM; edev->nvmem_config.name = dev_name(&spi->dev); edev->nvmem_config.dev = &spi->dev; edev->nvmem_config.read_only = pd->flags & EE_READONLY; diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 7939c55daceb..994ab67bc2dc 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -73,6 +73,11 @@ #define FASTRPC_RMID_INIT_CREATE_ATTR 7 #define FASTRPC_RMID_INIT_CREATE_STATIC 8 +/* Protection Domain(PD) ids */ +#define AUDIO_PD (0) /* also GUEST_OS PD? */ +#define USER_PD (1) +#define SENSORS_PD (2) + #define miscdev_to_cctx(d) container_of(d, struct fastrpc_channel_ctx, miscdev) static const char *domains[FASTRPC_DEV_MAX] = { "adsp", "mdsp", @@ -518,7 +523,7 @@ fastrpc_map_dma_buf(struct dma_buf_attachment *attachment, table = &a->sgt; - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, dir)) + if (!dma_map_sgtable(attachment->dev, table, dir, 0)) return ERR_PTR(-ENOMEM); return table; @@ -528,7 +533,7 @@ static void fastrpc_unmap_dma_buf(struct dma_buf_attachment *attach, struct sg_table *table, enum dma_data_direction dir) { - dma_unmap_sg(attach->dev, table->sgl, table->nents, dir); + dma_unmap_sgtable(attach->dev, table, dir, 0); } static void fastrpc_release(struct dma_buf *dmabuf) @@ -1037,7 +1042,7 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl, inbuf.pageslen = 1; inbuf.attrs = init.attrs; inbuf.siglen = init.siglen; - fl->pd = 1; + fl->pd = USER_PD; if (init.filelen && init.filefd) { err = fastrpc_map_create(fl, init.filefd, init.filelen, &map); @@ -1276,7 +1281,7 @@ static int fastrpc_dmabuf_alloc(struct fastrpc_user *fl, char __user *argp) return 0; } -static int fastrpc_init_attach(struct fastrpc_user *fl) +static int fastrpc_init_attach(struct fastrpc_user *fl, int pd) { struct fastrpc_invoke_args args[1]; int tgid = fl->tgid; @@ -1287,7 +1292,7 @@ static int fastrpc_init_attach(struct fastrpc_user *fl) args[0].fd = -1; args[0].reserved = 0; sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_ATTACH, 1, 0); - fl->pd = 0; + fl->pd = pd; return fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, sc, &args[0]); @@ -1477,7 +1482,10 @@ static long fastrpc_device_ioctl(struct file *file, unsigned int cmd, err = fastrpc_invoke(fl, argp); break; case FASTRPC_IOCTL_INIT_ATTACH: - err = fastrpc_init_attach(fl); + err = fastrpc_init_attach(fl, AUDIO_PD); + break; + case FASTRPC_IOCTL_INIT_ATTACH_SNS: + err = fastrpc_init_attach(fl, SENSORS_PD); break; case FASTRPC_IOCTL_INIT_CREATE: err = fastrpc_init_create_process(fl, argp); diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig index 8eb5d38c618e..1640340d3e62 100644 --- a/drivers/misc/habanalabs/Kconfig +++ b/drivers/misc/habanalabs/Kconfig @@ -7,7 +7,6 @@ config HABANA_AI tristate "HabanaAI accelerators (habanalabs)" depends on PCI && HAS_IOMEM select FRAME_VECTOR - select DMA_SHARED_BUFFER select GENERIC_ALLOCATOR select HWMON help diff --git a/drivers/misc/habanalabs/common/Makefile b/drivers/misc/habanalabs/common/Makefile index b984bfa4face..eccd8c7dc62d 100644 --- a/drivers/misc/habanalabs/common/Makefile +++ b/drivers/misc/habanalabs/common/Makefile @@ -3,5 +3,5 @@ HL_COMMON_FILES := common/habanalabs_drv.o common/device.o common/context.o \ common/asid.o common/habanalabs_ioctl.o \ common/command_buffer.o common/hw_queue.o common/irq.o \ common/sysfs.o common/hwmon.o common/memory.o \ - common/command_submission.o common/mmu.o common/firmware_if.o \ - common/pci.o + common/command_submission.o common/mmu.o common/mmu_v1.o \ + common/firmware_if.o common/pci.o diff --git a/drivers/misc/habanalabs/common/command_buffer.c b/drivers/misc/habanalabs/common/command_buffer.c index a8004911c977..901e213daf40 100644 --- a/drivers/misc/habanalabs/common/command_buffer.c +++ b/drivers/misc/habanalabs/common/command_buffer.c @@ -13,6 +13,131 @@ #include <linux/uaccess.h> #include <linux/genalloc.h> +static int cb_map_mem(struct hl_ctx *ctx, struct hl_cb *cb) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_vm_va_block *va_block, *tmp; + dma_addr_t bus_addr; + u64 virt_addr; + u32 page_size = prop->pmmu.page_size; + s32 offset; + int rc; + + if (!hdev->supports_cb_mapping) { + dev_err_ratelimited(hdev->dev, + "Cannot map CB because no VA range is allocated for CB mapping\n"); + return -EINVAL; + } + + if (!hdev->mmu_enable) { + dev_err_ratelimited(hdev->dev, + "Cannot map CB because MMU is disabled\n"); + return -EINVAL; + } + + INIT_LIST_HEAD(&cb->va_block_list); + + for (bus_addr = cb->bus_address; + bus_addr < cb->bus_address + cb->size; + bus_addr += page_size) { + + virt_addr = (u64) gen_pool_alloc(ctx->cb_va_pool, page_size); + if (!virt_addr) { + dev_err(hdev->dev, + "Failed to allocate device virtual address for CB\n"); + rc = -ENOMEM; + goto err_va_pool_free; + } + + va_block = kzalloc(sizeof(*va_block), GFP_KERNEL); + if (!va_block) { + rc = -ENOMEM; + gen_pool_free(ctx->cb_va_pool, virt_addr, page_size); + goto err_va_pool_free; + } + + va_block->start = virt_addr; + va_block->end = virt_addr + page_size; + va_block->size = page_size; + list_add_tail(&va_block->node, &cb->va_block_list); + } + + mutex_lock(&ctx->mmu_lock); + + bus_addr = cb->bus_address; + offset = 0; + list_for_each_entry(va_block, &cb->va_block_list, node) { + rc = hl_mmu_map(ctx, va_block->start, bus_addr, va_block->size, + list_is_last(&va_block->node, + &cb->va_block_list)); + if (rc) { + dev_err(hdev->dev, "Failed to map VA %#llx to CB\n", + va_block->start); + goto err_va_umap; + } + + bus_addr += va_block->size; + offset += va_block->size; + } + + hdev->asic_funcs->mmu_invalidate_cache(hdev, false, VM_TYPE_USERPTR); + + mutex_unlock(&ctx->mmu_lock); + + cb->is_mmu_mapped = true; + + return 0; + +err_va_umap: + list_for_each_entry(va_block, &cb->va_block_list, node) { + if (offset <= 0) + break; + hl_mmu_unmap(ctx, va_block->start, va_block->size, + offset <= va_block->size); + offset -= va_block->size; + } + + hdev->asic_funcs->mmu_invalidate_cache(hdev, true, VM_TYPE_USERPTR); + + mutex_unlock(&ctx->mmu_lock); + +err_va_pool_free: + list_for_each_entry_safe(va_block, tmp, &cb->va_block_list, node) { + gen_pool_free(ctx->cb_va_pool, va_block->start, va_block->size); + list_del(&va_block->node); + kfree(va_block); + } + + return rc; +} + +static void cb_unmap_mem(struct hl_ctx *ctx, struct hl_cb *cb) +{ + struct hl_device *hdev = ctx->hdev; + struct hl_vm_va_block *va_block, *tmp; + + mutex_lock(&ctx->mmu_lock); + + list_for_each_entry(va_block, &cb->va_block_list, node) + if (hl_mmu_unmap(ctx, va_block->start, va_block->size, + list_is_last(&va_block->node, + &cb->va_block_list))) + dev_warn_ratelimited(hdev->dev, + "Failed to unmap CB's va 0x%llx\n", + va_block->start); + + hdev->asic_funcs->mmu_invalidate_cache(hdev, true, VM_TYPE_USERPTR); + + mutex_unlock(&ctx->mmu_lock); + + list_for_each_entry_safe(va_block, tmp, &cb->va_block_list, node) { + gen_pool_free(ctx->cb_va_pool, va_block->start, va_block->size); + list_del(&va_block->node); + kfree(va_block); + } +} + static void cb_fini(struct hl_device *hdev, struct hl_cb *cb) { if (cb->is_internal) @@ -47,6 +172,11 @@ static void cb_release(struct kref *ref) hl_debugfs_remove_cb(cb); + if (cb->is_mmu_mapped) + cb_unmap_mem(cb->ctx, cb); + + hl_ctx_put(cb->ctx); + cb_do_release(hdev, cb); } @@ -107,11 +237,12 @@ static struct hl_cb *hl_cb_alloc(struct hl_device *hdev, u32 cb_size, } int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, - u32 cb_size, u64 *handle, int ctx_id, bool internal_cb) + struct hl_ctx *ctx, u32 cb_size, bool internal_cb, + bool map_cb, u64 *handle) { struct hl_cb *cb; bool alloc_new_cb = true; - int rc; + int rc, ctx_id = ctx->asid; /* * Can't use generic function to check this because of special case @@ -163,7 +294,21 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, } cb->hdev = hdev; - cb->ctx_id = ctx_id; + cb->ctx = ctx; + hl_ctx_get(hdev, cb->ctx); + + if (map_cb) { + if (ctx_id == HL_KERNEL_ASID_ID) { + dev_err(hdev->dev, + "CB mapping is not supported for kernel context\n"); + rc = -EINVAL; + goto release_cb; + } + + rc = cb_map_mem(ctx, cb); + if (rc) + goto release_cb; + } spin_lock(&mgr->cb_lock); rc = idr_alloc(&mgr->cb_handles, cb, 1, 0, GFP_ATOMIC); @@ -171,10 +316,10 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, if (rc < 0) { dev_err(hdev->dev, "Failed to allocate IDR for a new CB\n"); - goto release_cb; + goto unmap_mem; } - cb->id = rc; + cb->id = (u64) rc; kref_init(&cb->refcount); spin_lock_init(&cb->lock); @@ -183,14 +328,18 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, * idr is 32-bit so we can safely OR it with a mask that is above * 32 bit */ - *handle = cb->id | HL_MMAP_CB_MASK; + *handle = cb->id | HL_MMAP_TYPE_CB; *handle <<= PAGE_SHIFT; hl_debugfs_add_cb(cb); return 0; +unmap_mem: + if (cb->is_mmu_mapped) + cb_unmap_mem(cb->ctx, cb); release_cb: + hl_ctx_put(cb->ctx); cb_do_release(hdev, cb); out_err: *handle = 0; @@ -250,9 +399,10 @@ int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data) args->in.cb_size, HL_MAX_CB_SIZE); rc = -EINVAL; } else { - rc = hl_cb_create(hdev, &hpriv->cb_mgr, - args->in.cb_size, &handle, - hpriv->ctx->asid, false); + rc = hl_cb_create(hdev, &hpriv->cb_mgr, hpriv->ctx, + args->in.cb_size, false, + !!(args->in.flags & HL_CB_FLAGS_MAP), + &handle); } memset(args, 0, sizeof(*args)); @@ -300,11 +450,14 @@ int hl_cb_mmap(struct hl_fpriv *hpriv, struct vm_area_struct *vma) { struct hl_device *hdev = hpriv->hdev; struct hl_cb *cb; - phys_addr_t address; u32 handle, user_cb_size; int rc; + /* We use the page offset to hold the idr and thus we need to clear + * it before doing the mmap itself + */ handle = vma->vm_pgoff; + vma->vm_pgoff = 0; /* reference was taken here */ cb = hl_cb_get(hdev, &hpriv->cb_mgr, handle); @@ -356,12 +509,8 @@ int hl_cb_mmap(struct hl_fpriv *hpriv, struct vm_area_struct *vma) vma->vm_private_data = cb; - /* Calculate address for CB */ - address = virt_to_phys((void *) (uintptr_t) cb->kernel_address); - - rc = hdev->asic_funcs->cb_mmap(hdev, vma, cb->kernel_address, - address, cb->size); - + rc = hdev->asic_funcs->cb_mmap(hdev, vma, (void *) cb->kernel_address, + cb->bus_address, cb->size); if (rc) { spin_lock(&cb->lock); cb->mmap = false; @@ -425,7 +574,7 @@ void hl_cb_mgr_fini(struct hl_device *hdev, struct hl_cb_mgr *mgr) if (kref_put(&cb->refcount, cb_release) != 1) dev_err(hdev->dev, "CB %d for CTX ID %d is still alive\n", - id, cb->ctx_id); + id, cb->ctx->asid); } idr_destroy(&mgr->cb_handles); @@ -438,8 +587,8 @@ struct hl_cb *hl_cb_kernel_create(struct hl_device *hdev, u32 cb_size, struct hl_cb *cb; int rc; - rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, cb_size, &cb_handle, - HL_KERNEL_ASID_ID, internal_cb); + rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, hdev->kernel_ctx, cb_size, + internal_cb, false, &cb_handle); if (rc) { dev_err(hdev->dev, "Failed to allocate CB for the kernel driver %d\n", rc); @@ -495,3 +644,45 @@ int hl_cb_pool_fini(struct hl_device *hdev) return 0; } + +int hl_cb_va_pool_init(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + int rc; + + if (!hdev->supports_cb_mapping) + return 0; + + ctx->cb_va_pool = gen_pool_create(__ffs(prop->pmmu.page_size), -1); + if (!ctx->cb_va_pool) { + dev_err(hdev->dev, + "Failed to create VA gen pool for CB mapping\n"); + return -ENOMEM; + } + + rc = gen_pool_add(ctx->cb_va_pool, prop->cb_va_start_addr, + prop->cb_va_end_addr - prop->cb_va_start_addr, -1); + if (rc) { + dev_err(hdev->dev, + "Failed to add memory to VA gen pool for CB mapping\n"); + goto err_pool_destroy; + } + + return 0; + +err_pool_destroy: + gen_pool_destroy(ctx->cb_va_pool); + + return rc; +} + +void hl_cb_va_pool_fini(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + + if (!hdev->supports_cb_mapping) + return; + + gen_pool_destroy(ctx->cb_va_pool); +} diff --git a/drivers/misc/habanalabs/common/command_submission.c b/drivers/misc/habanalabs/common/command_submission.c index 2e3fcbc794db..b2b974ecc431 100644 --- a/drivers/misc/habanalabs/common/command_submission.c +++ b/drivers/misc/habanalabs/common/command_submission.c @@ -38,26 +38,10 @@ void hl_sob_reset_error(struct kref *ref) hw_sob->q_idx, hw_sob->sob_id); } -static const char *hl_fence_get_driver_name(struct dma_fence *fence) -{ - return "HabanaLabs"; -} - -static const char *hl_fence_get_timeline_name(struct dma_fence *fence) -{ - struct hl_cs_compl *hl_cs_compl = - container_of(fence, struct hl_cs_compl, base_fence); - - return dev_name(hl_cs_compl->hdev->dev); -} - -static bool hl_fence_enable_signaling(struct dma_fence *fence) -{ - return true; -} - -static void hl_fence_release(struct dma_fence *fence) +static void hl_fence_release(struct kref *kref) { + struct hl_fence *fence = + container_of(kref, struct hl_fence, refcount); struct hl_cs_compl *hl_cs_cmpl = container_of(fence, struct hl_cs_compl, base_fence); struct hl_device *hdev = hl_cs_cmpl->hdev; @@ -99,15 +83,27 @@ static void hl_fence_release(struct dma_fence *fence) } free: - kfree_rcu(hl_cs_cmpl, base_fence.rcu); + kfree(hl_cs_cmpl); } -static const struct dma_fence_ops hl_fence_ops = { - .get_driver_name = hl_fence_get_driver_name, - .get_timeline_name = hl_fence_get_timeline_name, - .enable_signaling = hl_fence_enable_signaling, - .release = hl_fence_release -}; +void hl_fence_put(struct hl_fence *fence) +{ + if (fence) + kref_put(&fence->refcount, hl_fence_release); +} + +void hl_fence_get(struct hl_fence *fence) +{ + if (fence) + kref_get(&fence->refcount); +} + +static void hl_fence_init(struct hl_fence *fence) +{ + kref_init(&fence->refcount); + fence->error = 0; + init_completion(&fence->completion); +} static void cs_get(struct hl_cs *cs) { @@ -256,6 +252,8 @@ static void cs_counters_aggregate(struct hl_device *hdev, struct hl_ctx *ctx) ctx->cs_counters.parsing_drop_cnt; hdev->aggregated_cs_counters.queue_full_drop_cnt += ctx->cs_counters.queue_full_drop_cnt; + hdev->aggregated_cs_counters.max_cs_in_flight_drop_cnt += + ctx->cs_counters.max_cs_in_flight_drop_cnt; } static void cs_do_release(struct kref *ref) @@ -336,7 +334,7 @@ static void cs_do_release(struct kref *ref) * In case the wait for signal CS was submitted, the put occurs * in init_signal_wait_cs() right before hanging on the PQ. */ - dma_fence_put(cs->signal_fence); + hl_fence_put(cs->signal_fence); } /* @@ -348,19 +346,18 @@ static void cs_do_release(struct kref *ref) hl_ctx_put(cs->ctx); /* We need to mark an error for not submitted because in that case - * the dma fence release flow is different. Mainly, we don't need + * the hl fence release flow is different. Mainly, we don't need * to handle hw_sob for signal/wait */ if (cs->timedout) - dma_fence_set_error(cs->fence, -ETIMEDOUT); + cs->fence->error = -ETIMEDOUT; else if (cs->aborted) - dma_fence_set_error(cs->fence, -EIO); + cs->fence->error = -EIO; else if (!cs->submitted) - dma_fence_set_error(cs->fence, -EBUSY); - - dma_fence_signal(cs->fence); - dma_fence_put(cs->fence); + cs->fence->error = -EBUSY; + complete_all(&cs->fence->completion); + hl_fence_put(cs->fence); cs_counters_aggregate(hdev, cs->ctx); kfree(cs->jobs_in_queue_cnt); @@ -401,7 +398,7 @@ static int allocate_cs(struct hl_device *hdev, struct hl_ctx *ctx, enum hl_cs_type cs_type, struct hl_cs **cs_new) { struct hl_cs_compl *cs_cmpl; - struct dma_fence *other = NULL; + struct hl_fence *other = NULL; struct hl_cs *cs; int rc; @@ -434,9 +431,11 @@ static int allocate_cs(struct hl_device *hdev, struct hl_ctx *ctx, cs_cmpl->cs_seq = ctx->cs_sequence; other = ctx->cs_pending[cs_cmpl->cs_seq & (hdev->asic_prop.max_pending_cs - 1)]; - if ((other) && (!dma_fence_is_signaled(other))) { - dev_dbg(hdev->dev, + + if (other && !completion_done(&other->completion)) { + dev_dbg_ratelimited(hdev->dev, "Rejecting CS because of too many in-flights CS\n"); + ctx->cs_counters.max_cs_in_flight_drop_cnt++; rc = -EAGAIN; goto free_fence; } @@ -448,8 +447,8 @@ static int allocate_cs(struct hl_device *hdev, struct hl_ctx *ctx, goto free_fence; } - dma_fence_init(&cs_cmpl->base_fence, &hl_fence_ops, &cs_cmpl->lock, - ctx->asid, ctx->cs_sequence); + /* init hl_fence */ + hl_fence_init(&cs_cmpl->base_fence); cs->sequence = cs_cmpl->cs_seq; @@ -458,9 +457,9 @@ static int allocate_cs(struct hl_device *hdev, struct hl_ctx *ctx, &cs_cmpl->base_fence; ctx->cs_sequence++; - dma_fence_get(&cs_cmpl->base_fence); + hl_fence_get(&cs_cmpl->base_fence); - dma_fence_put(other); + hl_fence_put(other); spin_unlock(&ctx->cs_lock); @@ -690,8 +689,8 @@ static int cs_ioctl_default(struct hl_fpriv *hpriv, void __user *chunks, rc = -ENOMEM; if (is_kernel_allocated_cb) goto release_cb; - else - goto free_cs_object; + + goto free_cs_object; } job->id = i + 1; @@ -773,7 +772,7 @@ static int cs_ioctl_signal_wait(struct hl_fpriv *hpriv, enum hl_cs_type cs_type, struct hl_ctx *ctx = hpriv->ctx; struct hl_cs_chunk *cs_chunk_array, *chunk; struct hw_queue_properties *hw_queue_prop; - struct dma_fence *sig_fence = NULL; + struct hl_fence *sig_fence = NULL; struct hl_cs_job *job; struct hl_cs *cs; struct hl_cb *cb; @@ -883,14 +882,14 @@ static int cs_ioctl_signal_wait(struct hl_fpriv *hpriv, enum hl_cs_type cs_type, dev_err(hdev->dev, "CS seq 0x%llx is not of a signal CS\n", signal_seq); - dma_fence_put(sig_fence); + hl_fence_put(sig_fence); rc = -EINVAL; goto free_signal_seq_array; } - if (dma_fence_is_signaled(sig_fence)) { + if (completion_done(&sig_fence->completion)) { /* signal CS already finished */ - dma_fence_put(sig_fence); + hl_fence_put(sig_fence); rc = 0; goto free_signal_seq_array; } @@ -902,7 +901,7 @@ static int cs_ioctl_signal_wait(struct hl_fpriv *hpriv, enum hl_cs_type cs_type, rc = allocate_cs(hdev, ctx, cs_type, &cs); if (rc) { if (cs_type == CS_TYPE_WAIT) - dma_fence_put(sig_fence); + hl_fence_put(sig_fence); hl_ctx_put(ctx); goto free_signal_seq_array; } @@ -1162,7 +1161,7 @@ out: static long _hl_cs_wait_ioctl(struct hl_device *hdev, struct hl_ctx *ctx, u64 timeout_us, u64 seq) { - struct dma_fence *fence; + struct hl_fence *fence; unsigned long timeout; long rc; @@ -1181,12 +1180,18 @@ static long _hl_cs_wait_ioctl(struct hl_device *hdev, "Can't wait on CS %llu because current CS is at seq %llu\n", seq, ctx->cs_sequence); } else if (fence) { - rc = dma_fence_wait_timeout(fence, true, timeout); + if (!timeout_us) + rc = completion_done(&fence->completion); + else + rc = wait_for_completion_interruptible_timeout( + &fence->completion, timeout); + if (fence->error == -ETIMEDOUT) rc = -ETIMEDOUT; else if (fence->error == -EIO) rc = -EIO; - dma_fence_put(fence); + + hl_fence_put(fence); } else { dev_dbg(hdev->dev, "Can't wait on seq %llu because current CS is at seq %llu (Fence is gone)\n", diff --git a/drivers/misc/habanalabs/common/context.c b/drivers/misc/habanalabs/common/context.c index 3e375958e73b..7a59dd7c6450 100644 --- a/drivers/misc/habanalabs/common/context.c +++ b/drivers/misc/habanalabs/common/context.c @@ -12,6 +12,7 @@ static void hl_ctx_fini(struct hl_ctx *ctx) { struct hl_device *hdev = ctx->hdev; + u64 idle_mask = 0; int i; /* @@ -23,11 +24,13 @@ static void hl_ctx_fini(struct hl_ctx *ctx) */ for (i = 0 ; i < hdev->asic_prop.max_pending_cs ; i++) - dma_fence_put(ctx->cs_pending[i]); + hl_fence_put(ctx->cs_pending[i]); kfree(ctx->cs_pending); if (ctx->asid != HL_KERNEL_ASID_ID) { + dev_dbg(hdev->dev, "closing user context %d\n", ctx->asid); + /* The engines are stopped as there is no executing CS, but the * Coresight might be still working by accessing addresses * related to the stopped engines. Hence stop it explicitly. @@ -37,9 +40,18 @@ static void hl_ctx_fini(struct hl_ctx *ctx) if ((hdev->in_debug) && (hdev->compute_ctx == ctx)) hl_device_set_debug_mode(hdev, false); + hl_cb_va_pool_fini(ctx); hl_vm_ctx_fini(ctx); hl_asid_free(hdev, ctx->asid); + + if ((!hdev->pldm) && (hdev->pdev) && + (!hdev->asic_funcs->is_device_idle(hdev, + &idle_mask, NULL))) + dev_notice(hdev->dev, + "device not idle after user context is closed (0x%llx)\n", + idle_mask); } else { + dev_dbg(hdev->dev, "closing kernel context\n"); hl_mmu_ctx_fini(ctx); } } @@ -128,7 +140,7 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx) atomic_set(&ctx->thread_ctx_switch_token, 1); ctx->thread_ctx_switch_wait_token = 0; ctx->cs_pending = kcalloc(hdev->asic_prop.max_pending_cs, - sizeof(struct dma_fence *), + sizeof(struct hl_fence *), GFP_KERNEL); if (!ctx->cs_pending) return -ENOMEM; @@ -155,15 +167,26 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx) goto err_asid_free; } + rc = hl_cb_va_pool_init(ctx); + if (rc) { + dev_err(hdev->dev, + "Failed to init VA pool for mapped CB\n"); + goto err_vm_ctx_fini; + } + rc = hdev->asic_funcs->ctx_init(ctx); if (rc) { dev_err(hdev->dev, "ctx_init failed\n"); - goto err_vm_ctx_fini; + goto err_cb_va_pool_fini; } + + dev_dbg(hdev->dev, "create user context %d\n", ctx->asid); } return 0; +err_cb_va_pool_fini: + hl_cb_va_pool_fini(ctx); err_vm_ctx_fini: hl_vm_ctx_fini(ctx); err_asid_free: @@ -184,10 +207,10 @@ int hl_ctx_put(struct hl_ctx *ctx) return kref_put(&ctx->refcount, hl_ctx_do_release); } -struct dma_fence *hl_ctx_get_fence(struct hl_ctx *ctx, u64 seq) +struct hl_fence *hl_ctx_get_fence(struct hl_ctx *ctx, u64 seq) { struct asic_fixed_properties *asic_prop = &ctx->hdev->asic_prop; - struct dma_fence *fence; + struct hl_fence *fence; spin_lock(&ctx->cs_lock); @@ -201,8 +224,9 @@ struct dma_fence *hl_ctx_get_fence(struct hl_ctx *ctx, u64 seq) return NULL; } - fence = dma_fence_get( - ctx->cs_pending[seq & (asic_prop->max_pending_cs - 1)]); + fence = ctx->cs_pending[seq & (asic_prop->max_pending_cs - 1)]; + hl_fence_get(fence); + spin_unlock(&ctx->cs_lock); return fence; diff --git a/drivers/misc/habanalabs/common/debugfs.c b/drivers/misc/habanalabs/common/debugfs.c index aa77771635d3..912ddfa360b1 100644 --- a/drivers/misc/habanalabs/common/debugfs.c +++ b/drivers/misc/habanalabs/common/debugfs.c @@ -21,7 +21,7 @@ static struct dentry *hl_debug_root; static int hl_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, u8 i2c_reg, long *val) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; if (hl_device_disabled_or_in_reset(hdev)) @@ -29,8 +29,8 @@ static int hl_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_I2C_RD << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_I2C_RD << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.i2c_bus = i2c_bus; pkt.i2c_addr = i2c_addr; pkt.i2c_reg = i2c_reg; @@ -47,7 +47,7 @@ static int hl_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, static int hl_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, u8 i2c_reg, u32 val) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; if (hl_device_disabled_or_in_reset(hdev)) @@ -55,8 +55,8 @@ static int hl_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_I2C_WR << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_I2C_WR << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.i2c_bus = i2c_bus; pkt.i2c_addr = i2c_addr; pkt.i2c_reg = i2c_reg; @@ -73,7 +73,7 @@ static int hl_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr, static void hl_debugfs_led_set(struct hl_device *hdev, u8 led, u8 state) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; if (hl_device_disabled_or_in_reset(hdev)) @@ -81,8 +81,8 @@ static void hl_debugfs_led_set(struct hl_device *hdev, u8 led, u8 state) memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_LED_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_LED_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.led_index = cpu_to_le32(led); pkt.value = cpu_to_le64(state); @@ -110,8 +110,8 @@ static int command_buffers_show(struct seq_file *s, void *data) seq_puts(s, "---------------------------------------------------------------\n"); } seq_printf(s, - " %03d %d 0x%08x %d %d %d\n", - cb->id, cb->ctx_id, cb->size, + " %03llu %d 0x%08x %d %d %d\n", + cb->id, cb->ctx->asid, cb->size, kref_read(&cb->refcount), cb->mmap, cb->cs_cnt); } @@ -354,6 +354,14 @@ static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, mmu_specs->hop4_shift); } +static inline u64 get_hop5_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_specs, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_specs->hop5_mask, + mmu_specs->hop5_shift); +} + static inline u64 get_next_hop_addr(u64 curr_pte) { if (curr_pte & PAGE_PRESENT_MASK) @@ -377,6 +385,7 @@ static int mmu_show(struct seq_file *s, void *data) hop2_addr = 0, hop2_pte_addr = 0, hop2_pte = 0, hop3_addr = 0, hop3_pte_addr = 0, hop3_pte = 0, hop4_addr = 0, hop4_pte_addr = 0, hop4_pte = 0, + hop5_addr = 0, hop5_pte_addr = 0, hop5_pte = 0, virt_addr = dev_entry->mmu_addr; if (!hdev->mmu_enable) @@ -428,20 +437,49 @@ static int mmu_show(struct seq_file *s, void *data) hop3_pte_addr = get_hop3_pte_addr(ctx, mmu_prop, hop3_addr, virt_addr); hop3_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr); - if (!(hop3_pte & LAST_MASK)) { + if (mmu_prop->num_hops == MMU_ARCH_5_HOPS) { + if (!(hop3_pte & LAST_MASK)) { + hop4_addr = get_next_hop_addr(hop3_pte); + + if (hop4_addr == ULLONG_MAX) + goto not_mapped; + + hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, + hop4_addr, virt_addr); + hop4_pte = hdev->asic_funcs->read_pte(hdev, + hop4_pte_addr); + if (!(hop4_pte & PAGE_PRESENT_MASK)) + goto not_mapped; + } else { + if (!(hop3_pte & PAGE_PRESENT_MASK)) + goto not_mapped; + } + } else { hop4_addr = get_next_hop_addr(hop3_pte); if (hop4_addr == ULLONG_MAX) goto not_mapped; - hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, hop4_addr, - virt_addr); - hop4_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr); - if (!(hop4_pte & PAGE_PRESENT_MASK)) - goto not_mapped; - } else { - if (!(hop3_pte & PAGE_PRESENT_MASK)) - goto not_mapped; + hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, + hop4_addr, virt_addr); + hop4_pte = hdev->asic_funcs->read_pte(hdev, + hop4_pte_addr); + if (!(hop4_pte & LAST_MASK)) { + hop5_addr = get_next_hop_addr(hop4_pte); + + if (hop5_addr == ULLONG_MAX) + goto not_mapped; + + hop5_pte_addr = get_hop5_pte_addr(ctx, mmu_prop, + hop5_addr, virt_addr); + hop5_pte = hdev->asic_funcs->read_pte(hdev, + hop5_pte_addr); + if (!(hop5_pte & PAGE_PRESENT_MASK)) + goto not_mapped; + } else { + if (!(hop4_pte & PAGE_PRESENT_MASK)) + goto not_mapped; + } } seq_printf(s, "asid: %u, virt_addr: 0x%llx\n", @@ -463,10 +501,22 @@ static int mmu_show(struct seq_file *s, void *data) seq_printf(s, "hop3_pte_addr: 0x%llx\n", hop3_pte_addr); seq_printf(s, "hop3_pte: 0x%llx\n", hop3_pte); - if (!(hop3_pte & LAST_MASK)) { + if (mmu_prop->num_hops == MMU_ARCH_5_HOPS) { + if (!(hop3_pte & LAST_MASK)) { + seq_printf(s, "hop4_addr: 0x%llx\n", hop4_addr); + seq_printf(s, "hop4_pte_addr: 0x%llx\n", hop4_pte_addr); + seq_printf(s, "hop4_pte: 0x%llx\n", hop4_pte); + } + } else { seq_printf(s, "hop4_addr: 0x%llx\n", hop4_addr); seq_printf(s, "hop4_pte_addr: 0x%llx\n", hop4_pte_addr); seq_printf(s, "hop4_pte: 0x%llx\n", hop4_pte); + + if (!(hop4_pte & LAST_MASK)) { + seq_printf(s, "hop5_addr: 0x%llx\n", hop5_addr); + seq_printf(s, "hop5_pte_addr: 0x%llx\n", hop5_pte_addr); + seq_printf(s, "hop5_pte: 0x%llx\n", hop5_pte); + } } goto out; diff --git a/drivers/misc/habanalabs/common/device.c b/drivers/misc/habanalabs/common/device.c index 24b01cce0a38..20572224099a 100644 --- a/drivers/misc/habanalabs/common/device.c +++ b/drivers/misc/habanalabs/common/device.c @@ -123,9 +123,13 @@ static int hl_device_release_ctrl(struct inode *inode, struct file *filp) static int hl_mmap(struct file *filp, struct vm_area_struct *vma) { struct hl_fpriv *hpriv = filp->private_data; + unsigned long vm_pgoff; - if ((vma->vm_pgoff & HL_MMAP_CB_MASK) == HL_MMAP_CB_MASK) { - vma->vm_pgoff ^= HL_MMAP_CB_MASK; + vm_pgoff = vma->vm_pgoff; + vma->vm_pgoff = HL_MMAP_OFFSET_VALUE_GET(vm_pgoff); + + switch (vm_pgoff & HL_MMAP_TYPE_MASK) { + case HL_MMAP_TYPE_CB: return hl_cb_mmap(hpriv, vma); } @@ -286,7 +290,7 @@ static int device_early_init(struct hl_device *hdev) } for (i = 0 ; i < hdev->asic_prop.completion_queues_count ; i++) { - snprintf(workq_name, 32, "hl-free-jobs-%u", i); + snprintf(workq_name, 32, "hl-free-jobs-%u", (u32) i); hdev->cq_wq[i] = create_singlethread_workqueue(workq_name); if (hdev->cq_wq[i] == NULL) { dev_err(hdev->dev, "Failed to allocate CQ workqueue\n"); @@ -317,6 +321,10 @@ static int device_early_init(struct hl_device *hdev) goto free_chip_info; } + rc = hl_mmu_if_set_funcs(hdev); + if (rc) + goto free_idle_busy_ts_arr; + hl_cb_mgr_init(&hdev->kernel_cb_mgr); mutex_init(&hdev->send_cpu_message_lock); @@ -330,6 +338,8 @@ static int device_early_init(struct hl_device *hdev) return 0; +free_idle_busy_ts_arr: + kfree(hdev->idle_busy_ts_arr); free_chip_info: kfree(hdev->hl_chip_info); free_eq_wq: @@ -871,7 +881,7 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset, * so this message won't be sent */ if (hl_fw_send_pci_access_msg(hdev, - ARMCP_PACKET_DISABLE_PCI_ACCESS)) + CPUCP_PACKET_DISABLE_PCI_ACCESS)) dev_warn(hdev->dev, "Failed to disable PCI access by F/W\n"); } @@ -957,14 +967,13 @@ again: flush_workqueue(hdev->eq_wq); } - /* Release kernel context */ - if ((hard_reset) && (hl_ctx_put(hdev->kernel_ctx) == 1)) - hdev->kernel_ctx = NULL; - /* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini(hdev, hard_reset); if (hard_reset) { + /* Release kernel context */ + if (hl_ctx_put(hdev->kernel_ctx) == 1) + hdev->kernel_ctx = NULL; hl_vm_fini(hdev); hl_mmu_fini(hdev); hl_eq_reset(hdev, &hdev->event_queue); @@ -1455,13 +1464,13 @@ void hl_device_fini(struct hl_device *hdev) hl_cb_pool_fini(hdev); + /* Reset the H/W. It will be in idle state after this returns */ + hdev->asic_funcs->hw_fini(hdev, true); + /* Release kernel context */ if ((hdev->kernel_ctx) && (hl_ctx_put(hdev->kernel_ctx) != 1)) dev_err(hdev->dev, "kernel ctx is still alive\n"); - /* Reset the H/W. It will be in idle state after this returns */ - hdev->asic_funcs->hw_fini(hdev, true); - hl_vm_fini(hdev); hl_mmu_fini(hdev); diff --git a/drivers/misc/habanalabs/common/firmware_if.c b/drivers/misc/habanalabs/common/firmware_if.c index f52bc690dfc5..cd41c7ceb0e7 100644 --- a/drivers/misc/habanalabs/common/firmware_if.c +++ b/drivers/misc/habanalabs/common/firmware_if.c @@ -68,9 +68,9 @@ out: int hl_fw_send_pci_access_msg(struct hl_device *hdev, u32 opcode) { - struct armcp_packet pkt = {}; + struct cpucp_packet pkt = {}; - pkt.ctl = cpu_to_le32(opcode << ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(opcode << CPUCP_PKT_CTL_OPCODE_SHIFT); return hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), 0, NULL); @@ -79,7 +79,7 @@ int hl_fw_send_pci_access_msg(struct hl_device *hdev, u32 opcode) int hl_fw_send_cpu_message(struct hl_device *hdev, u32 hw_queue_id, u32 *msg, u16 len, u32 timeout, long *result) { - struct armcp_packet *pkt; + struct cpucp_packet *pkt; dma_addr_t pkt_dma_addr; u32 tmp; int rc = 0; @@ -111,7 +111,7 @@ int hl_fw_send_cpu_message(struct hl_device *hdev, u32 hw_queue_id, u32 *msg, } rc = hl_poll_timeout_memory(hdev, &pkt->fence, tmp, - (tmp == ARMCP_PACKET_FENCE_VAL), 1000, + (tmp == CPUCP_PACKET_FENCE_VAL), 1000, timeout, true); hl_hw_queue_inc_ci_kernel(hdev, hw_queue_id); @@ -124,12 +124,12 @@ int hl_fw_send_cpu_message(struct hl_device *hdev, u32 hw_queue_id, u32 *msg, tmp = le32_to_cpu(pkt->ctl); - rc = (tmp & ARMCP_PKT_CTL_RC_MASK) >> ARMCP_PKT_CTL_RC_SHIFT; + rc = (tmp & CPUCP_PKT_CTL_RC_MASK) >> CPUCP_PKT_CTL_RC_SHIFT; if (rc) { dev_err(hdev->dev, "F/W ERROR %d for CPU packet %d\n", rc, - (tmp & ARMCP_PKT_CTL_OPCODE_MASK) - >> ARMCP_PKT_CTL_OPCODE_SHIFT); + (tmp & CPUCP_PKT_CTL_OPCODE_MASK) + >> CPUCP_PKT_CTL_OPCODE_SHIFT); rc = -EIO; } else if (result) { *result = (long) le64_to_cpu(pkt->result); @@ -145,14 +145,14 @@ out: int hl_fw_unmask_irq(struct hl_device *hdev, u16 event_type) { - struct armcp_packet pkt; + struct cpucp_packet pkt; long result; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_UNMASK_RAZWI_IRQ << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_UNMASK_RAZWI_IRQ << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.value = cpu_to_le64(event_type); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), @@ -167,15 +167,15 @@ int hl_fw_unmask_irq(struct hl_device *hdev, u16 event_type) int hl_fw_unmask_irq_arr(struct hl_device *hdev, const u32 *irq_arr, size_t irq_arr_size) { - struct armcp_unmask_irq_arr_packet *pkt; + struct cpucp_unmask_irq_arr_packet *pkt; size_t total_pkt_size; long result; int rc; - total_pkt_size = sizeof(struct armcp_unmask_irq_arr_packet) + + total_pkt_size = sizeof(struct cpucp_unmask_irq_arr_packet) + irq_arr_size; - /* data should be aligned to 8 bytes in order to ArmCP to copy it */ + /* data should be aligned to 8 bytes in order to CPU-CP to copy it */ total_pkt_size = (total_pkt_size + 0x7) & ~0x7; /* total_pkt_size is casted to u16 later on */ @@ -191,8 +191,8 @@ int hl_fw_unmask_irq_arr(struct hl_device *hdev, const u32 *irq_arr, pkt->length = cpu_to_le32(irq_arr_size / sizeof(irq_arr[0])); memcpy(&pkt->irqs, irq_arr, irq_arr_size); - pkt->armcp_pkt.ctl = cpu_to_le32(ARMCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt->cpucp_pkt.ctl = cpu_to_le32(CPUCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY << + CPUCP_PKT_CTL_OPCODE_SHIFT); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) pkt, total_pkt_size, 0, &result); @@ -207,19 +207,19 @@ int hl_fw_unmask_irq_arr(struct hl_device *hdev, const u32 *irq_arr, int hl_fw_test_cpu_queue(struct hl_device *hdev) { - struct armcp_packet test_pkt = {}; + struct cpucp_packet test_pkt = {}; long result; int rc; - test_pkt.ctl = cpu_to_le32(ARMCP_PACKET_TEST << - ARMCP_PKT_CTL_OPCODE_SHIFT); - test_pkt.value = cpu_to_le64(ARMCP_PACKET_FENCE_VAL); + test_pkt.ctl = cpu_to_le32(CPUCP_PACKET_TEST << + CPUCP_PKT_CTL_OPCODE_SHIFT); + test_pkt.value = cpu_to_le64(CPUCP_PACKET_FENCE_VAL); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &test_pkt, sizeof(test_pkt), 0, &result); if (!rc) { - if (result != ARMCP_PACKET_FENCE_VAL) + if (result != CPUCP_PACKET_FENCE_VAL) dev_err(hdev->dev, "CPU queue test failed (0x%08lX)\n", result); } else { @@ -251,61 +251,61 @@ void hl_fw_cpu_accessible_dma_pool_free(struct hl_device *hdev, size_t size, int hl_fw_send_heartbeat(struct hl_device *hdev) { - struct armcp_packet hb_pkt = {}; + struct cpucp_packet hb_pkt = {}; long result; int rc; - hb_pkt.ctl = cpu_to_le32(ARMCP_PACKET_TEST << - ARMCP_PKT_CTL_OPCODE_SHIFT); - hb_pkt.value = cpu_to_le64(ARMCP_PACKET_FENCE_VAL); + hb_pkt.ctl = cpu_to_le32(CPUCP_PACKET_TEST << + CPUCP_PKT_CTL_OPCODE_SHIFT); + hb_pkt.value = cpu_to_le64(CPUCP_PACKET_FENCE_VAL); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &hb_pkt, sizeof(hb_pkt), 0, &result); - if ((rc) || (result != ARMCP_PACKET_FENCE_VAL)) + if ((rc) || (result != CPUCP_PACKET_FENCE_VAL)) rc = -EIO; return rc; } -int hl_fw_armcp_info_get(struct hl_device *hdev) +int hl_fw_cpucp_info_get(struct hl_device *hdev) { struct asic_fixed_properties *prop = &hdev->asic_prop; - struct armcp_packet pkt = {}; - void *armcp_info_cpu_addr; - dma_addr_t armcp_info_dma_addr; + struct cpucp_packet pkt = {}; + void *cpucp_info_cpu_addr; + dma_addr_t cpucp_info_dma_addr; long result; int rc; - armcp_info_cpu_addr = + cpucp_info_cpu_addr = hdev->asic_funcs->cpu_accessible_dma_pool_alloc(hdev, - sizeof(struct armcp_info), - &armcp_info_dma_addr); - if (!armcp_info_cpu_addr) { + sizeof(struct cpucp_info), + &cpucp_info_dma_addr); + if (!cpucp_info_cpu_addr) { dev_err(hdev->dev, - "Failed to allocate DMA memory for ArmCP info packet\n"); + "Failed to allocate DMA memory for CPU-CP info packet\n"); return -ENOMEM; } - memset(armcp_info_cpu_addr, 0, sizeof(struct armcp_info)); + memset(cpucp_info_cpu_addr, 0, sizeof(struct cpucp_info)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_INFO_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); - pkt.addr = cpu_to_le64(armcp_info_dma_addr); - pkt.data_max_size = cpu_to_le32(sizeof(struct armcp_info)); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_INFO_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); + pkt.addr = cpu_to_le64(cpucp_info_dma_addr); + pkt.data_max_size = cpu_to_le32(sizeof(struct cpucp_info)); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), - HL_ARMCP_INFO_TIMEOUT_USEC, &result); + HL_CPUCP_INFO_TIMEOUT_USEC, &result); if (rc) { dev_err(hdev->dev, - "Failed to handle ArmCP info pkt, error %d\n", rc); + "Failed to handle CPU-CP info pkt, error %d\n", rc); goto out; } - memcpy(&prop->armcp_info, armcp_info_cpu_addr, - sizeof(prop->armcp_info)); + memcpy(&prop->cpucp_info, cpucp_info_cpu_addr, + sizeof(prop->cpucp_info)); - rc = hl_build_hwmon_channel_info(hdev, prop->armcp_info.sensors); + rc = hl_build_hwmon_channel_info(hdev, prop->cpucp_info.sensors); if (rc) { dev_err(hdev->dev, "Failed to build hwmon channel info, error %d\n", rc); @@ -315,14 +315,14 @@ int hl_fw_armcp_info_get(struct hl_device *hdev) out: hdev->asic_funcs->cpu_accessible_dma_pool_free(hdev, - sizeof(struct armcp_info), armcp_info_cpu_addr); + sizeof(struct cpucp_info), cpucp_info_cpu_addr); return rc; } int hl_fw_get_eeprom_data(struct hl_device *hdev, void *data, size_t max_size) { - struct armcp_packet pkt = {}; + struct cpucp_packet pkt = {}; void *eeprom_info_cpu_addr; dma_addr_t eeprom_info_dma_addr; long result; @@ -333,23 +333,24 @@ int hl_fw_get_eeprom_data(struct hl_device *hdev, void *data, size_t max_size) max_size, &eeprom_info_dma_addr); if (!eeprom_info_cpu_addr) { dev_err(hdev->dev, - "Failed to allocate DMA memory for ArmCP EEPROM packet\n"); + "Failed to allocate DMA memory for CPU-CP EEPROM packet\n"); return -ENOMEM; } memset(eeprom_info_cpu_addr, 0, max_size); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_EEPROM_DATA_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_EEPROM_DATA_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.addr = cpu_to_le64(eeprom_info_dma_addr); pkt.data_max_size = cpu_to_le32(max_size); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), - HL_ARMCP_EEPROM_TIMEOUT_USEC, &result); + HL_CPUCP_EEPROM_TIMEOUT_USEC, &result); if (rc) { dev_err(hdev->dev, - "Failed to handle ArmCP EEPROM packet, error %d\n", rc); + "Failed to handle CPU-CP EEPROM packet, error %d\n", + rc); goto out; } @@ -363,6 +364,77 @@ out: return rc; } +int hl_fw_cpucp_pci_counters_get(struct hl_device *hdev, + struct hl_info_pci_counters *counters) +{ + struct cpucp_packet pkt = {}; + long result; + int rc; + + pkt.ctl = cpu_to_le32(CPUCP_PACKET_PCIE_THROUGHPUT_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); + + /* Fetch PCI rx counter */ + pkt.index = cpu_to_le32(cpucp_pcie_throughput_rx); + rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), + HL_CPUCP_INFO_TIMEOUT_USEC, &result); + if (rc) { + dev_err(hdev->dev, + "Failed to handle CPU-CP PCI info pkt, error %d\n", rc); + return rc; + } + counters->rx_throughput = result; + + /* Fetch PCI tx counter */ + pkt.index = cpu_to_le32(cpucp_pcie_throughput_tx); + rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), + HL_CPUCP_INFO_TIMEOUT_USEC, &result); + if (rc) { + dev_err(hdev->dev, + "Failed to handle CPU-CP PCI info pkt, error %d\n", rc); + return rc; + } + counters->tx_throughput = result; + + /* Fetch PCI replay counter */ + pkt.ctl = cpu_to_le32(CPUCP_PACKET_PCIE_REPLAY_CNT_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); + + rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), + HL_CPUCP_INFO_TIMEOUT_USEC, &result); + if (rc) { + dev_err(hdev->dev, + "Failed to handle CPU-CP PCI info pkt, error %d\n", rc); + return rc; + } + counters->replay_cnt = (u32) result; + + return rc; +} + +int hl_fw_cpucp_total_energy_get(struct hl_device *hdev, u64 *total_energy) +{ + struct cpucp_packet pkt = {}; + long result; + int rc; + + pkt.ctl = cpu_to_le32(CPUCP_PACKET_TOTAL_ENERGY_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); + + rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), + HL_CPUCP_INFO_TIMEOUT_USEC, &result); + if (rc) { + dev_err(hdev->dev, + "Failed to handle CpuCP total energy pkt, error %d\n", + rc); + return rc; + } + + *total_energy = result; + + return rc; +} + static void fw_read_errors(struct hl_device *hdev, u32 boot_err0_reg) { u32 err_val; @@ -402,8 +474,11 @@ static void fw_read_errors(struct hl_device *hdev, u32 boot_err0_reg) "Device boot error - NIC F/W initialization failed\n"); } -static void hl_detect_cpu_boot_status(struct hl_device *hdev, u32 status) +static void detect_cpu_boot_status(struct hl_device *hdev, u32 status) { + /* Some of the status codes below are deprecated in newer f/w + * versions but we keep them here for backward compatibility + */ switch (status) { case CPU_BOOT_STATUS_NA: dev_err(hdev->dev, @@ -449,6 +524,48 @@ static void hl_detect_cpu_boot_status(struct hl_device *hdev, u32 status) } } +int hl_fw_read_preboot_ver(struct hl_device *hdev, u32 cpu_boot_status_reg, + u32 boot_err0_reg, u32 timeout) +{ + u32 status; + int rc; + + if (!hdev->cpu_enable) + return 0; + + /* Need to check two possible scenarios: + * + * CPU_BOOT_STATUS_WAITING_FOR_BOOT_FIT - for newer firmwares where + * the preboot is waiting for the boot fit + * + * All other status values - for older firmwares where the uboot was + * loaded from the FLASH + */ + rc = hl_poll_timeout( + hdev, + cpu_boot_status_reg, + status, + (status == CPU_BOOT_STATUS_IN_UBOOT) || + (status == CPU_BOOT_STATUS_DRAM_RDY) || + (status == CPU_BOOT_STATUS_NIC_FW_RDY) || + (status == CPU_BOOT_STATUS_READY_TO_BOOT) || + (status == CPU_BOOT_STATUS_SRAM_AVAIL) || + (status == CPU_BOOT_STATUS_WAITING_FOR_BOOT_FIT), + 10000, + timeout); + + if (rc) { + dev_err(hdev->dev, "Failed to read preboot version\n"); + detect_cpu_boot_status(hdev, status); + fw_read_errors(hdev, boot_err0_reg); + return -EIO; + } + + hdev->asic_funcs->read_device_fw_version(hdev, FW_COMP_PREBOOT); + + return 0; +} + int hl_fw_init_cpu(struct hl_device *hdev, u32 cpu_boot_status_reg, u32 msg_to_cpu_reg, u32 cpu_msg_status_reg, u32 boot_err0_reg, bool skip_bmc, @@ -514,15 +631,11 @@ int hl_fw_init_cpu(struct hl_device *hdev, u32 cpu_boot_status_reg, 10000, cpu_timeout); - /* Read U-Boot, preboot versions now in case we will later fail */ + /* Read U-Boot version now in case we will later fail */ hdev->asic_funcs->read_device_fw_version(hdev, FW_COMP_UBOOT); - hdev->asic_funcs->read_device_fw_version(hdev, FW_COMP_PREBOOT); - /* Some of the status codes below are deprecated in newer f/w - * versions but we keep them here for backward compatibility - */ if (rc) { - hl_detect_cpu_boot_status(hdev, status); + detect_cpu_boot_status(hdev, status); rc = -EIO; goto out; } diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index edbd627b29d2..80d4d7385ffe 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -8,21 +8,33 @@ #ifndef HABANALABSP_H_ #define HABANALABSP_H_ -#include "../include/common/armcp_if.h" +#include "../include/common/cpucp_if.h" #include "../include/common/qman_if.h" #include <uapi/misc/habanalabs.h> #include <linux/cdev.h> #include <linux/iopoll.h> #include <linux/irqreturn.h> -#include <linux/dma-fence.h> #include <linux/dma-direction.h> #include <linux/scatterlist.h> #include <linux/hashtable.h> +#include <linux/bitfield.h> #define HL_NAME "habanalabs" -#define HL_MMAP_CB_MASK (0x8000000000000000ull >> PAGE_SHIFT) +/* Use upper bits of mmap offset to store habana driver specific information. + * bits[63:62] - Encode mmap type + * bits[45:0] - mmap offset value + * + * NOTE: struct vm_area_struct.vm_pgoff uses offset in pages. Hence, these + * defines are w.r.t to PAGE_SIZE + */ +#define HL_MMAP_TYPE_SHIFT (62 - PAGE_SHIFT) +#define HL_MMAP_TYPE_MASK (0x3ull << HL_MMAP_TYPE_SHIFT) +#define HL_MMAP_TYPE_CB (0x2ull << HL_MMAP_TYPE_SHIFT) + +#define HL_MMAP_OFFSET_VALUE_MASK (0x3FFFFFFFFFFFull >> PAGE_SHIFT) +#define HL_MMAP_OFFSET_VALUE_GET(off) (off & HL_MMAP_OFFSET_VALUE_MASK) #define HL_PENDING_RESET_PER_SEC 30 @@ -34,8 +46,8 @@ #define HL_PLL_LOW_JOB_FREQ_USEC 5000000 /* 5 s */ -#define HL_ARMCP_INFO_TIMEOUT_USEC 10000000 /* 10s */ -#define HL_ARMCP_EEPROM_TIMEOUT_USEC 10000000 /* 10s */ +#define HL_CPUCP_INFO_TIMEOUT_USEC 10000000 /* 10s */ +#define HL_CPUCP_EEPROM_TIMEOUT_USEC 10000000 /* 10s */ #define HL_PCI_ELBI_TIMEOUT_MSEC 10 /* 10ms */ @@ -66,6 +78,8 @@ #define HL_PCI_NUM_BARS 6 +#define HL_MAX_DCORES 4 + /** * struct pgt_info - MMU hop page info. * @node: hash linked-list node for the pgts shadow hash of pgts. @@ -222,12 +236,15 @@ enum hl_device_hw_state { * @hop2_shift: shift of hop 2 mask. * @hop3_shift: shift of hop 3 mask. * @hop4_shift: shift of hop 4 mask. + * @hop5_shift: shift of hop 5 mask. * @hop0_mask: mask to get the PTE address in hop 0. * @hop1_mask: mask to get the PTE address in hop 1. * @hop2_mask: mask to get the PTE address in hop 2. * @hop3_mask: mask to get the PTE address in hop 3. * @hop4_mask: mask to get the PTE address in hop 4. + * @hop5_mask: mask to get the PTE address in hop 5. * @page_size: default page size used to allocate memory. + * @num_hops: The amount of hops supported by the translation table. */ struct hl_mmu_properties { u64 start_addr; @@ -237,18 +254,21 @@ struct hl_mmu_properties { u64 hop2_shift; u64 hop3_shift; u64 hop4_shift; + u64 hop5_shift; u64 hop0_mask; u64 hop1_mask; u64 hop2_mask; u64 hop3_mask; u64 hop4_mask; + u64 hop5_mask; u32 page_size; + u32 num_hops; }; /** * struct asic_fixed_properties - ASIC specific immutable properties. * @hw_queues_props: H/W queues properties. - * @armcp_info: received various information from ArmCP regarding the H/W, e.g. + * @cpucp_info: received various information from CPU-CP regarding the H/W, e.g. * available sensors. * @uboot_ver: F/W U-boot version. * @preboot_ver: F/W Preboot version. @@ -271,6 +291,10 @@ struct hl_mmu_properties { * @pcie_aux_dbi_reg_addr: Address of the PCIE_AUX DBI register. * @mmu_pgt_addr: base physical address in DRAM of MMU page tables. * @mmu_dram_default_page_addr: DRAM default page physical address. + * @cb_va_start_addr: virtual start address of command buffers which are mapped + * to the device's MMU. + * @cb_va_end_addr: virtual end address of command buffers which are mapped to + * the device's MMU. * @mmu_pgt_size: MMU page tables total size. * @mmu_pte_size: PTE size in MMU page tables. * @mmu_hop_table_size: MMU hop table size. @@ -292,12 +316,16 @@ struct hl_mmu_properties { * @max_queues: maximum amount of queues in the system * @sync_stream_first_sob: first sync object available for sync stream use * @sync_stream_first_mon: first monitor available for sync stream use + * @first_available_user_sob: first sob available for the user + * @first_available_user_mon: first monitor available for the user * @tpc_enabled_mask: which TPCs are enabled. * @completion_queues_count: number of completion queues. + * @fw_security_disabled: true if security measures are disabled in firmware, + * false otherwise */ struct asic_fixed_properties { struct hw_queue_properties *hw_queues_props; - struct armcp_info armcp_info; + struct cpucp_info cpucp_info; char uboot_ver[VERSION_MAX_LEN]; char preboot_ver[VERSION_MAX_LEN]; struct hl_mmu_properties dmmu; @@ -317,6 +345,8 @@ struct asic_fixed_properties { u64 pcie_aux_dbi_reg_addr; u64 mmu_pgt_addr; u64 mmu_dram_default_page_addr; + u64 cb_va_start_addr; + u64 cb_va_end_addr; u32 mmu_pgt_size; u32 mmu_pte_size; u32 mmu_hop_table_size; @@ -338,13 +368,29 @@ struct asic_fixed_properties { u32 max_queues; u16 sync_stream_first_sob; u16 sync_stream_first_mon; + u16 first_available_user_sob[HL_MAX_DCORES]; + u16 first_available_user_mon[HL_MAX_DCORES]; u8 tpc_enabled_mask; u8 completion_queues_count; + u8 fw_security_disabled; +}; + +/** + * struct hl_fence - software synchronization primitive + * @completion: fence is implemented using completion + * @refcount: refcount for this fence + * @error: mark this fence with error + * + */ +struct hl_fence { + struct completion completion; + struct kref refcount; + int error; }; /** * struct hl_cs_compl - command submission completion object. - * @base_fence: kernel fence object. + * @base_fence: hl fence object. * @lock: spinlock to protect fence. * @hdev: habanalabs device structure. * @hw_sob: the H/W SOB used in this signal/wait CS. @@ -353,7 +399,7 @@ struct asic_fixed_properties { * @sob_val: the SOB value that is used in this signal/wait CS. */ struct hl_cs_compl { - struct dma_fence base_fence; + struct hl_fence base_fence; spinlock_t lock; struct hl_device *hdev; struct hl_hw_sob *hw_sob; @@ -380,36 +426,41 @@ struct hl_cb_mgr { * struct hl_cb - describes a Command Buffer. * @refcount: reference counter for usage of the CB. * @hdev: pointer to device this CB belongs to. + * @ctx: pointer to the CB owner's context. * @lock: spinlock to protect mmap/cs flows. * @debugfs_list: node in debugfs list of command buffers. * @pool_list: node in pool list of command buffers. + * @va_block_list: list of virtual addresses blocks of the CB if it is mapped to + * the device's MMU. + * @id: the CB's ID. * @kernel_address: Holds the CB's kernel virtual address. * @bus_address: Holds the CB's DMA address. * @mmap_size: Holds the CB's size that was mmaped. * @size: holds the CB's size. - * @id: the CB's ID. * @cs_cnt: holds number of CS that this CB participates in. - * @ctx_id: holds the ID of the owner's context. * @mmap: true if the CB is currently mmaped to user. * @is_pool: true if CB was acquired from the pool, false otherwise. * @is_internal: internaly allocated + * @is_mmu_mapped: true if the CB is mapped to the device's MMU. */ struct hl_cb { struct kref refcount; struct hl_device *hdev; + struct hl_ctx *ctx; spinlock_t lock; struct list_head debugfs_list; struct list_head pool_list; + struct list_head va_block_list; + u64 id; u64 kernel_address; dma_addr_t bus_address; u32 mmap_size; u32 size; - u32 id; u32 cs_cnt; - u32 ctx_id; u8 mmap; u8 is_pool; u8 is_internal; + u8 is_mmu_mapped; }; @@ -435,7 +486,7 @@ struct hl_cs_job; #define HL_EQ_LENGTH 64 #define HL_EQ_SIZE_IN_BYTES (HL_EQ_LENGTH * HL_EQ_ENTRY_SIZE) -/* Host <-> ArmCP shared memory size */ +/* Host <-> CPU-CP shared memory size */ #define HL_CPU_ACCESSIBLE_MEM_SIZE SZ_2M /** @@ -617,7 +668,7 @@ enum div_select_defs { * @debugfs_read32: debug interface for reading u32 from DRAM/SRAM. * @debugfs_write32: debug interface for writing u32 to DRAM/SRAM. * @add_device_attr: add ASIC specific device attributes. - * @handle_eqe: handle event queue entry (IRQ) from ArmCP. + * @handle_eqe: handle event queue entry (IRQ) from CPU-CP. * @set_pll_profile: change PLL profile (manual/automatic). * @get_events_stat: retrieve event queue entries histogram. * @read_pte: read MMU page table entry from DRAM. @@ -626,7 +677,7 @@ enum div_select_defs { * (L1 only) or hard (L0 & L1) flush. * @mmu_invalidate_cache_range: flush specific MMU STLB cache lines with * ASID-VA-size mask. - * @send_heartbeat: send is-alive packet to ArmCP and verify response. + * @send_heartbeat: send is-alive packet to CPU-CP and verify response. * @set_clock_gating: enable/disable clock gating per engine according to * clock gating mask in hdev * @disable_clock_gating: disable clock gating completely @@ -644,8 +695,6 @@ enum div_select_defs { * ASIC * @get_hw_state: retrieve the H/W state * @pci_bars_map: Map PCI BARs. - * @set_dram_bar_base: Set DRAM BAR to map specific device address. Returns - * old address the bar pointed to or U64_MAX for failure * @init_iatu: Initialize the iATU unit inside the PCI controller. * @rreg: Read a register. Needed for simulator support. * @wreg: Write a register. Needed for simulator support. @@ -679,7 +728,7 @@ struct hl_asic_funcs { int (*suspend)(struct hl_device *hdev); int (*resume)(struct hl_device *hdev); int (*cb_mmap)(struct hl_device *hdev, struct vm_area_struct *vma, - u64 kaddress, phys_addr_t paddress, u32 size); + void *cpu_addr, dma_addr_t dma_addr, size_t size); void (*ring_doorbell)(struct hl_device *hdev, u32 hw_queue_id, u32 pi); void (*pqe_write)(struct hl_device *hdev, __le64 *pqe, struct hl_bd *bd); @@ -736,7 +785,7 @@ struct hl_asic_funcs { void (*set_clock_gating)(struct hl_device *hdev); void (*disable_clock_gating)(struct hl_device *hdev); int (*debug_coresight)(struct hl_device *hdev, void *data); - bool (*is_device_idle)(struct hl_device *hdev, u32 *mask, + bool (*is_device_idle)(struct hl_device *hdev, u64 *mask, struct seq_file *s); int (*soft_reset_late_init)(struct hl_device *hdev); void (*hw_queues_lock)(struct hl_device *hdev); @@ -748,7 +797,6 @@ struct hl_asic_funcs { u16 len, u32 timeout, long *result); enum hl_device_hw_state (*get_hw_state)(struct hl_device *hdev); int (*pci_bars_map)(struct hl_device *hdev); - u64 (*set_dram_bar_base)(struct hl_device *hdev, u64 addr); int (*init_iatu)(struct hl_device *hdev); u32 (*rreg)(struct hl_device *hdev, u32 reg); void (*wreg)(struct hl_device *hdev, u32 reg, u32 val); @@ -800,7 +848,7 @@ struct hl_va_range { * @hdev: pointer to the device structure. * @refcount: reference counter for the context. Context is released only when * this hits 0l. It is incremented on CS and CS_WAIT. - * @cs_pending: array of DMA fence objects representing pending CS. + * @cs_pending: array of hl fence objects representing pending CS. * @host_va_range: holds available virtual addresses for host mappings. * @host_huge_va_range: holds available virtual addresses for host mappings * with huge pages. @@ -809,6 +857,8 @@ struct hl_va_range { * @mmu_lock: protects the MMU page tables. Any change to the PGT, modifying the * MMU hash or walking the PGT requires talking this lock. * @debugfs_list: node in debugfs list of contexts. + * @cb_va_pool: device VA pool for command buffers which are mapped to the + * device's MMU. * @cs_sequence: sequence number for CS. Value is assigned to a CS and passed * to user so user could inquire about CS. It is used as * index to cs_pending array. @@ -832,7 +882,7 @@ struct hl_ctx { struct hl_fpriv *hpriv; struct hl_device *hdev; struct kref refcount; - struct dma_fence **cs_pending; + struct hl_fence **cs_pending; struct hl_va_range *host_va_range; struct hl_va_range *host_huge_va_range; struct hl_va_range *dram_va_range; @@ -840,6 +890,7 @@ struct hl_ctx { struct mutex mmu_lock; struct list_head debugfs_list; struct hl_cs_counters cs_counters; + struct gen_pool *cb_va_pool; u64 cs_sequence; u64 *dram_default_hops; spinlock_t cs_lock; @@ -919,8 +970,8 @@ struct hl_cs { struct list_head job_list; spinlock_t job_lock; struct kref refcount; - struct dma_fence *fence; - struct dma_fence *signal_fence; + struct hl_fence *fence; + struct hl_fence *signal_fence; struct work_struct finish_work; struct delayed_work work_tdr; struct list_head mirror_node; @@ -1395,6 +1446,44 @@ struct hl_device_idle_busy_ts { ktime_t busy_to_idle_ts; }; + +/** + * struct hl_mmu_priv - used for holding per-device mmu internal information. + * @mmu_pgt_pool: pool of page tables used by MMU for allocating hops. + * @mmu_shadow_hop0: shadow array of hop0 tables. + */ +struct hl_mmu_priv { + struct gen_pool *mmu_pgt_pool; + void *mmu_shadow_hop0; +}; + +/** + * struct hl_mmu_funcs - Device related MMU functions. + * @init: initialize the MMU module. + * @fini: release the MMU module. + * @ctx_init: Initialize a context for using the MMU module. + * @ctx_fini: disable a ctx from using the mmu module. + * @map: maps a virtual address to physical address for a context. + * @unmap: unmap a virtual address of a context. + * @flush: flush all writes from all cores to reach device MMU. + * @swap_out: marks all mapping of the given context as swapped out. + * @swap_in: marks all mapping of the given context as swapped in. + */ +struct hl_mmu_funcs { + int (*init)(struct hl_device *hdev); + void (*fini)(struct hl_device *hdev); + int (*ctx_init)(struct hl_ctx *ctx); + void (*ctx_fini)(struct hl_ctx *ctx); + int (*map)(struct hl_ctx *ctx, + u64 virt_addr, u64 phys_addr, u32 page_size, + bool is_dram_addr); + int (*unmap)(struct hl_ctx *ctx, + u64 virt_addr, bool is_dram_addr); + void (*flush)(struct hl_ctx *ctx); + void (*swap_out)(struct hl_ctx *ctx); + void (*swap_in)(struct hl_ctx *ctx); +}; + /** * struct hl_device - habanalabs device structure. * @pdev: pointer to PCI device, can be NULL in case of simulator device. @@ -1407,8 +1496,8 @@ struct hl_device_idle_busy_ts { * @dev: related kernel basic device structure. * @dev_ctrl: related kernel device structure for the control device * @work_freq: delayed work to lower device frequency if possible. - * @work_heartbeat: delayed work for ArmCP is-alive check. - * @asic_name: ASIC specific nmae. + * @work_heartbeat: delayed work for CPU-CP is-alive check. + * @asic_name: ASIC specific name. * @asic_type: ASIC specific type. * @completion_queue: array of hl_cq. * @cq_wq: work queues of completion queues for executing work in process @@ -1419,22 +1508,20 @@ struct hl_device_idle_busy_ts { * @hw_queues_mirror_list: CS mirror list for TDR. * @hw_queues_mirror_lock: protects hw_queues_mirror_list. * @kernel_cb_mgr: command buffer manager for creating/destroying/handling CGs. - * @event_queue: event queue for IRQ from ArmCP. + * @event_queue: event queue for IRQ from CPU-CP. * @dma_pool: DMA pool for small allocations. - * @cpu_accessible_dma_mem: Host <-> ArmCP shared memory CPU address. - * @cpu_accessible_dma_address: Host <-> ArmCP shared memory DMA address. - * @cpu_accessible_dma_pool: Host <-> ArmCP shared memory pool. + * @cpu_accessible_dma_mem: Host <-> CPU-CP shared memory CPU address. + * @cpu_accessible_dma_address: Host <-> CPU-CP shared memory DMA address. + * @cpu_accessible_dma_pool: Host <-> CPU-CP shared memory pool. * @asid_bitmap: holds used/available ASIDs. * @asid_mutex: protects asid_bitmap. - * @send_cpu_message_lock: enforces only one message in Host <-> ArmCP queue. + * @send_cpu_message_lock: enforces only one message in Host <-> CPU-CP queue. * @debug_lock: protects critical section of setting debug mode for device * @asic_prop: ASIC specific immutable properties. * @asic_funcs: ASIC specific functions. * @asic_specific: ASIC specific information to use only from ASIC files. - * @mmu_pgt_pool: pool of available MMU hops. * @vm: virtual memory manager for MMU. * @mmu_cache_lock: protects MMU cache invalidation as it can serve one context. - * @mmu_shadow_hop0: shadow mapping of the MMU hop 0 zone. * @hwmon_dev: H/W monitor device. * @pm_mng_profile: current power management profile. * @hl_chip_info: ASIC's sensors information. @@ -1452,6 +1539,8 @@ struct hl_device_idle_busy_ts { * @idle_busy_ts_arr: array to hold time stamps of transitions from idle to busy * and vice-versa * @aggregated_cs_counters: aggregated cs counters among all contexts + * @mmu_priv: device-specific MMU data. + * @mmu_func: device-related MMU functions. * @dram_used_mem: current DRAM memory consumption. * @timeout_jiffies: device CS timeout value. * @max_power: the max power of the device, as configured by the sysadmin. This @@ -1471,6 +1560,7 @@ struct hl_device_idle_busy_ts { * @soft_reset_cnt: number of soft reset since the driver was loaded. * @hard_reset_cnt: number of hard reset since the driver was loaded. * @idle_busy_ts_idx: index of current entry in idle_busy_ts_arr + * @clk_throttling_reason: bitmask represents the current clk throttling reasons * @id: device minor. * @id_control: minor of the control device * @cpu_pci_msb_addr: 50-bit extension bits for the device CPU's 40-bit @@ -1479,7 +1569,7 @@ struct hl_device_idle_busy_ts { * @late_init_done: is late init stage was done during initialization. * @hwmon_initialized: is H/W monitor sensors was initialized. * @hard_reset_pending: is there a hard reset work pending. - * @heartbeat: is heartbeat sanity check towards ArmCP enabled. + * @heartbeat: is heartbeat sanity check towards CPU-CP enabled. * @reset_on_lockup: true if a reset should be done in case of stuck CS, false * otherwise. * @dram_supports_virtual_memory: is MMU enabled towards DRAM. @@ -1501,6 +1591,7 @@ struct hl_device_idle_busy_ts { * @sync_stream_queue_idx: helper index for sync stream queues initialization. * @supports_coresight: is CoreSight supported. * @supports_soft_reset: is soft reset supported. + * @supports_cb_mapping: is mapping a CB to the device's MMU supported. */ struct hl_device { struct pci_dev *pdev; @@ -1513,7 +1604,7 @@ struct hl_device { struct device *dev_ctrl; struct delayed_work work_freq; struct delayed_work work_heartbeat; - char asic_name[16]; + char asic_name[32]; enum hl_asic_type asic_type; struct hl_cq *completion_queue; struct workqueue_struct **cq_wq; @@ -1535,10 +1626,8 @@ struct hl_device { struct asic_fixed_properties asic_prop; const struct hl_asic_funcs *asic_funcs; void *asic_specific; - struct gen_pool *mmu_pgt_pool; struct hl_vm vm; struct mutex mmu_cache_lock; - void *mmu_shadow_hop0; struct device *hwmon_dev; enum hl_pm_mng_profile pm_mng_profile; struct hwmon_chip_info *hl_chip_info; @@ -1562,19 +1651,23 @@ struct hl_device { struct hl_cs_counters aggregated_cs_counters; + struct hl_mmu_priv mmu_priv; + struct hl_mmu_funcs mmu_func; + atomic64_t dram_used_mem; u64 timeout_jiffies; u64 max_power; u64 clock_gating_mask; atomic_t in_reset; enum hl_pll_frequency curr_pll_profile; - enum armcp_card_types card_type; + enum cpucp_card_types card_type; int cs_active_cnt; u32 major; u32 high_pll; u32 soft_reset_cnt; u32 hard_reset_cnt; u32 idle_busy_ts_idx; + u32 clk_throttling_reason; u16 id; u16 id_control; u16 cpu_pci_msb_addr; @@ -1598,6 +1691,7 @@ struct hl_device { u8 sync_stream_queue_idx; u8 supports_coresight; u8 supports_soft_reset; + u8 supports_cb_mapping; /* Parameters for bring-up */ u8 mmu_enable; @@ -1739,7 +1833,7 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx); void hl_ctx_do_release(struct kref *ref); void hl_ctx_get(struct hl_device *hdev, struct hl_ctx *ctx); int hl_ctx_put(struct hl_ctx *ctx); -struct dma_fence *hl_ctx_get_fence(struct hl_ctx *ctx, u64 seq); +struct hl_fence *hl_ctx_get_fence(struct hl_ctx *ctx, u64 seq); void hl_ctx_mgr_init(struct hl_ctx_mgr *mgr); void hl_ctx_mgr_fini(struct hl_device *hdev, struct hl_ctx_mgr *mgr); @@ -1755,7 +1849,7 @@ int hl_device_set_frequency(struct hl_device *hdev, enum hl_pll_frequency freq); uint32_t hl_device_utilization(struct hl_device *hdev, uint32_t period_ms); int hl_build_hwmon_channel_info(struct hl_device *hdev, - struct armcp_sensor *sensors_arr); + struct cpucp_sensor *sensors_arr); int hl_sysfs_init(struct hl_device *hdev); void hl_sysfs_fini(struct hl_device *hdev); @@ -1763,8 +1857,9 @@ void hl_sysfs_fini(struct hl_device *hdev); int hl_hwmon_init(struct hl_device *hdev); void hl_hwmon_fini(struct hl_device *hdev); -int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, u32 cb_size, - u64 *handle, int ctx_id, bool internal_cb); +int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, + struct hl_ctx *ctx, u32 cb_size, bool internal_cb, + bool map_cb, u64 *handle); int hl_cb_destroy(struct hl_device *hdev, struct hl_cb_mgr *mgr, u64 cb_handle); int hl_cb_mmap(struct hl_fpriv *hpriv, struct vm_area_struct *vma); struct hl_cb *hl_cb_get(struct hl_device *hdev, struct hl_cb_mgr *mgr, @@ -1776,11 +1871,15 @@ struct hl_cb *hl_cb_kernel_create(struct hl_device *hdev, u32 cb_size, bool internal_cb); int hl_cb_pool_init(struct hl_device *hdev); int hl_cb_pool_fini(struct hl_device *hdev); +int hl_cb_va_pool_init(struct hl_ctx *ctx); +void hl_cb_va_pool_fini(struct hl_ctx *ctx); void hl_cs_rollback_all(struct hl_device *hdev); struct hl_cs_job *hl_cs_allocate_job(struct hl_device *hdev, enum hl_queue_type queue_type, bool is_kernel_allocated_cb); void hl_sob_reset_error(struct kref *ref); +void hl_fence_put(struct hl_fence *fence); +void hl_fence_get(struct hl_fence *fence); void goya_set_asic_funcs(struct hl_device *hdev); void gaudi_set_asic_funcs(struct hl_device *hdev); @@ -1810,6 +1909,8 @@ int hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr, u32 page_size, bool flush_pte); void hl_mmu_swap_out(struct hl_ctx *ctx); void hl_mmu_swap_in(struct hl_ctx *ctx); +int hl_mmu_if_set_funcs(struct hl_device *hdev); +void hl_mmu_v1_set_funcs(struct hl_device *hdev); int hl_fw_load_fw_to_device(struct hl_device *hdev, const char *fw_name, void __iomem *dst); @@ -1825,23 +1926,28 @@ void *hl_fw_cpu_accessible_dma_pool_alloc(struct hl_device *hdev, size_t size, void hl_fw_cpu_accessible_dma_pool_free(struct hl_device *hdev, size_t size, void *vaddr); int hl_fw_send_heartbeat(struct hl_device *hdev); -int hl_fw_armcp_info_get(struct hl_device *hdev); +int hl_fw_cpucp_info_get(struct hl_device *hdev); int hl_fw_get_eeprom_data(struct hl_device *hdev, void *data, size_t max_size); +int hl_fw_cpucp_pci_counters_get(struct hl_device *hdev, + struct hl_info_pci_counters *counters); +int hl_fw_cpucp_total_energy_get(struct hl_device *hdev, + u64 *total_energy); int hl_fw_init_cpu(struct hl_device *hdev, u32 cpu_boot_status_reg, u32 msg_to_cpu_reg, u32 cpu_msg_status_reg, u32 boot_err0_reg, bool skip_bmc, u32 cpu_timeout, u32 boot_fit_timeout); +int hl_fw_read_preboot_ver(struct hl_device *hdev, u32 cpu_boot_status_reg, + u32 boot_err0_reg, u32 timeout); int hl_pci_bars_map(struct hl_device *hdev, const char * const name[3], bool is_wc[3]); int hl_pci_iatu_write(struct hl_device *hdev, u32 addr, u32 data); -int hl_pci_set_dram_bar_base(struct hl_device *hdev, u8 inbound_region, u8 bar, - u64 addr); int hl_pci_set_inbound_region(struct hl_device *hdev, u8 region, struct hl_inbound_pci_region *pci_region); int hl_pci_set_outbound_region(struct hl_device *hdev, struct hl_outbound_pci_region *pci_region); -int hl_pci_init(struct hl_device *hdev); +int hl_pci_init(struct hl_device *hdev, u32 cpu_boot_status_reg, + u32 boot_err0_reg, u32 preboot_ver_timeout); void hl_pci_fini(struct hl_device *hdev); long hl_get_frequency(struct hl_device *hdev, u32 pll_index, bool curr); diff --git a/drivers/misc/habanalabs/common/habanalabs_drv.c b/drivers/misc/habanalabs/common/habanalabs_drv.c index c6b31e93fb5e..f9067d3ef437 100644 --- a/drivers/misc/habanalabs/common/habanalabs_drv.c +++ b/drivers/misc/habanalabs/common/habanalabs_drv.c @@ -11,6 +11,7 @@ #include "habanalabs.h" #include <linux/pci.h> +#include <linux/aer.h> #include <linux/module.h> #define HL_DRIVER_AUTHOR "HabanaLabs Kernel Driver Team" @@ -408,6 +409,8 @@ static int hl_pci_probe(struct pci_dev *pdev, pci_set_drvdata(pdev, hdev); + pci_enable_pcie_error_reporting(pdev); + rc = hl_device_init(hdev, hl_class); if (rc) { dev_err(&pdev->dev, "Fatal error during habanalabs device init\n"); @@ -440,22 +443,93 @@ static void hl_pci_remove(struct pci_dev *pdev) return; hl_device_fini(hdev); + pci_disable_pcie_error_reporting(pdev); pci_set_drvdata(pdev, NULL); - destroy_hdev(hdev); } +/** + * hl_pci_err_detected - a PCI bus error detected on this device + * + * @pdev: pointer to pci device + * @state: PCI error type + * + * Called by the PCI subsystem whenever a non-correctable + * PCI bus error is detected + */ +static pci_ers_result_t +hl_pci_err_detected(struct pci_dev *pdev, pci_channel_state_t state) +{ + struct hl_device *hdev = pci_get_drvdata(pdev); + enum pci_ers_result result; + + switch (state) { + case pci_channel_io_normal: + return PCI_ERS_RESULT_CAN_RECOVER; + + case pci_channel_io_frozen: + dev_warn(hdev->dev, "frozen state error detected\n"); + result = PCI_ERS_RESULT_NEED_RESET; + break; + + case pci_channel_io_perm_failure: + dev_warn(hdev->dev, "failure state error detected\n"); + result = PCI_ERS_RESULT_DISCONNECT; + break; + + default: + result = PCI_ERS_RESULT_NONE; + } + + hdev->asic_funcs->halt_engines(hdev, true); + + return result; +} + +/** + * hl_pci_err_resume - resume after a PCI slot reset + * + * @pdev: pointer to pci device + * + */ +static void hl_pci_err_resume(struct pci_dev *pdev) +{ + struct hl_device *hdev = pci_get_drvdata(pdev); + + dev_warn(hdev->dev, "Resuming device after PCI slot reset\n"); + hl_device_resume(hdev); +} + +/** + * hl_pci_err_slot_reset - a PCI slot reset has just happened + * + * @pdev: pointer to pci device + * + * Determine if the driver can recover from the PCI slot reset + */ +static pci_ers_result_t hl_pci_err_slot_reset(struct pci_dev *pdev) +{ + return PCI_ERS_RESULT_RECOVERED; +} + static const struct dev_pm_ops hl_pm_ops = { .suspend = hl_pmops_suspend, .resume = hl_pmops_resume, }; +static const struct pci_error_handlers hl_pci_err_handler = { + .error_detected = hl_pci_err_detected, + .slot_reset = hl_pci_err_slot_reset, + .resume = hl_pci_err_resume, +}; + static struct pci_driver hl_pci_driver = { .name = HL_NAME, .id_table = ids, .probe = hl_pci_probe, .remove = hl_pci_remove, .driver.pm = &hl_pm_ops, + .err_handler = &hl_pci_err_handler, }; /* diff --git a/drivers/misc/habanalabs/common/habanalabs_ioctl.c b/drivers/misc/habanalabs/common/habanalabs_ioctl.c index 5af1c03da473..07317ea49129 100644 --- a/drivers/misc/habanalabs/common/habanalabs_ioctl.c +++ b/drivers/misc/habanalabs/common/habanalabs_ioctl.c @@ -8,6 +8,7 @@ #include <uapi/misc/habanalabs.h> #include "habanalabs.h" +#include <linux/kernel.h> #include <linux/fs.h> #include <linux/uaccess.h> #include <linux/slab.h> @@ -64,14 +65,14 @@ static int hw_ip_info(struct hl_device *hdev, struct hl_info_args *args) hw_ip.dram_enabled = 1; hw_ip.num_of_events = prop->num_of_events; - memcpy(hw_ip.armcp_version, prop->armcp_info.armcp_version, + memcpy(hw_ip.cpucp_version, prop->cpucp_info.cpucp_version, min(VERSION_MAX_LEN, HL_INFO_VERSION_MAX_LEN)); - memcpy(hw_ip.card_name, prop->armcp_info.card_name, + memcpy(hw_ip.card_name, prop->cpucp_info.card_name, min(CARD_NAME_MAX_LEN, HL_INFO_CARD_NAME_MAX_LEN)); - hw_ip.armcp_cpld_version = le32_to_cpu(prop->armcp_info.cpld_version); - hw_ip.module_id = le32_to_cpu(prop->armcp_info.card_location); + hw_ip.cpld_version = le32_to_cpu(prop->cpucp_info.cpld_version); + hw_ip.module_id = le32_to_cpu(prop->cpucp_info.card_location); hw_ip.psoc_pci_pll_nr = prop->psoc_pci_pll_nr; hw_ip.psoc_pci_pll_nf = prop->psoc_pci_pll_nf; @@ -131,7 +132,7 @@ static int hw_idle(struct hl_device *hdev, struct hl_info_args *args) return -EINVAL; hw_idle.is_idle = hdev->asic_funcs->is_device_idle(hdev, - &hw_idle.busy_engines_mask, NULL); + &hw_idle.busy_engines_mask_ext, NULL); return copy_to_user(out, &hw_idle, min((size_t) max_size, sizeof(hw_idle))) ? -EFAULT : 0; @@ -276,10 +277,45 @@ static int time_sync_info(struct hl_device *hdev, struct hl_info_args *args) min((size_t) max_size, sizeof(time_sync))) ? -EFAULT : 0; } +static int pci_counters_info(struct hl_fpriv *hpriv, struct hl_info_args *args) +{ + struct hl_device *hdev = hpriv->hdev; + struct hl_info_pci_counters pci_counters = {0}; + u32 max_size = args->return_size; + void __user *out = (void __user *) (uintptr_t) args->return_pointer; + int rc; + + if ((!max_size) || (!out)) + return -EINVAL; + + rc = hl_fw_cpucp_pci_counters_get(hdev, &pci_counters); + if (rc) + return rc; + + return copy_to_user(out, &pci_counters, + min((size_t) max_size, sizeof(pci_counters))) ? -EFAULT : 0; +} + +static int clk_throttle_info(struct hl_fpriv *hpriv, struct hl_info_args *args) +{ + struct hl_device *hdev = hpriv->hdev; + struct hl_info_clk_throttle clk_throttle = {0}; + u32 max_size = args->return_size; + void __user *out = (void __user *) (uintptr_t) args->return_pointer; + + if ((!max_size) || (!out)) + return -EINVAL; + + clk_throttle.clk_throttling_reason = hdev->clk_throttling_reason; + + return copy_to_user(out, &clk_throttle, + min((size_t) max_size, sizeof(clk_throttle))) ? -EFAULT : 0; +} + static int cs_counters_info(struct hl_fpriv *hpriv, struct hl_info_args *args) { struct hl_device *hdev = hpriv->hdev; - struct hl_info_cs_counters cs_counters = {0}; + struct hl_info_cs_counters cs_counters = { {0} }; u32 max_size = args->return_size; void __user *out = (void __user *) (uintptr_t) args->return_pointer; @@ -297,6 +333,51 @@ static int cs_counters_info(struct hl_fpriv *hpriv, struct hl_info_args *args) min((size_t) max_size, sizeof(cs_counters))) ? -EFAULT : 0; } +static int sync_manager_info(struct hl_fpriv *hpriv, struct hl_info_args *args) +{ + struct hl_device *hdev = hpriv->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_info_sync_manager sm_info = {0}; + u32 max_size = args->return_size; + void __user *out = (void __user *) (uintptr_t) args->return_pointer; + + if ((!max_size) || (!out)) + return -EINVAL; + + if (args->dcore_id >= HL_MAX_DCORES) + return -EINVAL; + + sm_info.first_available_sync_object = + prop->first_available_user_sob[args->dcore_id]; + sm_info.first_available_monitor = + prop->first_available_user_mon[args->dcore_id]; + + + return copy_to_user(out, &sm_info, min_t(size_t, (size_t) max_size, + sizeof(sm_info))) ? -EFAULT : 0; +} + +static int total_energy_consumption_info(struct hl_fpriv *hpriv, + struct hl_info_args *args) +{ + struct hl_device *hdev = hpriv->hdev; + struct hl_info_energy total_energy = {0}; + u32 max_size = args->return_size; + void __user *out = (void __user *) (uintptr_t) args->return_pointer; + int rc; + + if ((!max_size) || (!out)) + return -EINVAL; + + rc = hl_fw_cpucp_total_energy_get(hdev, + &total_energy.total_energy_consumption); + if (rc) + return rc; + + return copy_to_user(out, &total_energy, + min((size_t) max_size, sizeof(total_energy))) ? -EFAULT : 0; +} + static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, struct device *dev) { @@ -360,6 +441,18 @@ static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, case HL_INFO_CS_COUNTERS: return cs_counters_info(hpriv, args); + case HL_INFO_PCI_COUNTERS: + return pci_counters_info(hpriv, args); + + case HL_INFO_CLK_THROTTLE_REASON: + return clk_throttle_info(hpriv, args); + + case HL_INFO_SYNC_MANAGER: + return sync_manager_info(hpriv, args); + + case HL_INFO_TOTAL_ENERGY: + return total_energy_consumption_info(hpriv, args); + default: dev_err(dev, "Invalid request %d\n", args->op); rc = -ENOTTY; diff --git a/drivers/misc/habanalabs/common/hw_queue.c b/drivers/misc/habanalabs/common/hw_queue.c index 287681646071..5e66c98fb0d3 100644 --- a/drivers/misc/habanalabs/common/hw_queue.c +++ b/drivers/misc/habanalabs/common/hw_queue.c @@ -288,10 +288,10 @@ static void ext_queue_schedule_job(struct hl_cs_job *job) ptr = cb->bus_address; cq_pkt.data = cpu_to_le32( - ((q->pi << CQ_ENTRY_SHADOW_INDEX_SHIFT) - & CQ_ENTRY_SHADOW_INDEX_MASK) | - (1 << CQ_ENTRY_SHADOW_INDEX_VALID_SHIFT) | - (1 << CQ_ENTRY_READY_SHIFT)); + ((q->pi << CQ_ENTRY_SHADOW_INDEX_SHIFT) + & CQ_ENTRY_SHADOW_INDEX_MASK) | + FIELD_PREP(CQ_ENTRY_SHADOW_INDEX_VALID_MASK, 1) | + FIELD_PREP(CQ_ENTRY_READY_MASK, 1)); /* * No need to protect pi_offset because scheduling to the @@ -474,7 +474,7 @@ static void init_signal_wait_cs(struct hl_cs *cs) * wait CS was submitted. */ mb(); - dma_fence_put(cs->signal_fence); + hl_fence_put(cs->signal_fence); cs->signal_fence = NULL; } } diff --git a/drivers/misc/habanalabs/common/hwmon.c b/drivers/misc/habanalabs/common/hwmon.c index b997336fa75f..2ac29cb2fe61 100644 --- a/drivers/misc/habanalabs/common/hwmon.c +++ b/drivers/misc/habanalabs/common/hwmon.c @@ -13,7 +13,7 @@ #define HWMON_NR_SENSOR_TYPES (hwmon_pwm + 1) int hl_build_hwmon_channel_info(struct hl_device *hdev, - struct armcp_sensor *sensors_arr) + struct cpucp_sensor *sensors_arr) { u32 counts[HWMON_NR_SENSOR_TYPES] = {0}; u32 *sensors_by_type[HWMON_NR_SENSOR_TYPES] = {NULL}; @@ -24,7 +24,7 @@ int hl_build_hwmon_channel_info(struct hl_device *hdev, enum hwmon_sensor_types type; int rc, i, j; - for (i = 0 ; i < ARMCP_MAX_SENSORS ; i++) { + for (i = 0 ; i < CPUCP_MAX_SENSORS ; i++) { type = le32_to_cpu(sensors_arr[i].type); if ((type == 0) && (sensors_arr[i].flags == 0)) @@ -311,13 +311,13 @@ static const struct hwmon_ops hl_hwmon_ops = { int hl_get_temperature(struct hl_device *hdev, int sensor_index, u32 attr, long *value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_TEMPERATURE_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_TEMPERATURE_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); @@ -337,13 +337,13 @@ int hl_get_temperature(struct hl_device *hdev, int hl_set_temperature(struct hl_device *hdev, int sensor_index, u32 attr, long value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_TEMPERATURE_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_TEMPERATURE_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); pkt.value = __cpu_to_le64(value); @@ -362,13 +362,13 @@ int hl_set_temperature(struct hl_device *hdev, int hl_get_voltage(struct hl_device *hdev, int sensor_index, u32 attr, long *value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_VOLTAGE_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_VOLTAGE_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); @@ -388,13 +388,13 @@ int hl_get_voltage(struct hl_device *hdev, int hl_get_current(struct hl_device *hdev, int sensor_index, u32 attr, long *value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_CURRENT_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_CURRENT_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); @@ -414,13 +414,13 @@ int hl_get_current(struct hl_device *hdev, int hl_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr, long *value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_FAN_SPEED_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_FAN_SPEED_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); @@ -440,13 +440,13 @@ int hl_get_fan_speed(struct hl_device *hdev, int hl_get_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr, long *value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_PWM_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_PWM_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); @@ -466,13 +466,13 @@ int hl_get_pwm_info(struct hl_device *hdev, void hl_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr, long value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_PWM_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_PWM_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); pkt.value = cpu_to_le64(value); @@ -489,13 +489,13 @@ void hl_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr, int hl_set_voltage(struct hl_device *hdev, int sensor_index, u32 attr, long value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_VOLTAGE_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_VOLTAGE_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); pkt.value = __cpu_to_le64(value); @@ -514,13 +514,13 @@ int hl_set_voltage(struct hl_device *hdev, int hl_set_current(struct hl_device *hdev, int sensor_index, u32 attr, long value) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_CURRENT_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_CURRENT_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.sensor_index = __cpu_to_le16(sensor_index); pkt.type = __cpu_to_le16(attr); pkt.value = __cpu_to_le64(value); @@ -549,7 +549,7 @@ int hl_hwmon_init(struct hl_device *hdev) hdev->hl_chip_info->ops = &hl_hwmon_ops; hdev->hwmon_dev = hwmon_device_register_with_info(dev, - prop->armcp_info.card_name, hdev, + prop->cpucp_info.card_name, hdev, hdev->hl_chip_info, NULL); if (IS_ERR(hdev->hwmon_dev)) { rc = PTR_ERR(hdev->hwmon_dev); diff --git a/drivers/misc/habanalabs/common/irq.c b/drivers/misc/habanalabs/common/irq.c index c8db717023f5..d20e40a53d70 100644 --- a/drivers/misc/habanalabs/common/irq.c +++ b/drivers/misc/habanalabs/common/irq.c @@ -11,7 +11,7 @@ /** * struct hl_eqe_work - This structure is used to schedule work of EQ - * entry and armcp_reset event + * entry and cpucp_reset event * * @eq_work: workqueue object to run when EQ entry is received * @hdev: pointer to device structure diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c index 5ff4688683fd..84227819e4d1 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/misc/habanalabs/common/memory.c @@ -77,8 +77,8 @@ static int alloc_device_memory(struct hl_ctx *ctx, struct hl_mem_in *args, paddr = (u64) gen_pool_alloc(vm->dram_pg_pool, total_size); if (!paddr) { dev_err(hdev->dev, - "failed to allocate %llu huge contiguous pages\n", - num_pgs); + "failed to allocate %llu contiguous pages with total size of %llu\n", + num_pgs, total_size); return -ENOMEM; } } @@ -505,41 +505,32 @@ static inline int add_va_block(struct hl_device *hdev, } /* - * get_va_block - get a virtual block with the requested size - * - * @hdev : pointer to the habanalabs device structure - * @va_range : pointer to the virtual addresses range - * @size : requested block size - * @hint_addr : hint for request address by the user - * @is_userptr : is host or DRAM memory + * get_va_block() - get a virtual block for the given size and alignment. + * @hdev: pointer to the habanalabs device structure. + * @va_range: pointer to the virtual addresses range. + * @size: requested block size. + * @hint_addr: hint for requested address by the user. + * @va_block_align: required alignment of the virtual block start address. * * This function does the following: * - Iterate on the virtual block list to find a suitable virtual block for the - * requested size - * - Reserve the requested block and update the list - * - Return the start address of the virtual block + * given size and alignment. + * - Reserve the requested block and update the list. + * - Return the start address of the virtual block. */ -static u64 get_va_block(struct hl_device *hdev, - struct hl_va_range *va_range, u64 size, u64 hint_addr, - bool is_userptr) +static u64 get_va_block(struct hl_device *hdev, struct hl_va_range *va_range, + u64 size, u64 hint_addr, u32 va_block_align) { struct hl_vm_va_block *va_block, *new_va_block = NULL; - u64 valid_start, valid_size, prev_start, prev_end, page_mask, + u64 valid_start, valid_size, prev_start, prev_end, align_mask, res_valid_start = 0, res_valid_size = 0; - u32 page_size; bool add_prev = false; - if (is_userptr) - /* - * We cannot know if the user allocated memory with huge pages - * or not, hence we continue with the biggest possible - * granularity. - */ - page_size = hdev->asic_prop.pmmu_huge.page_size; - else - page_size = hdev->asic_prop.dmmu.page_size; + align_mask = ~((u64)va_block_align - 1); - page_mask = ~((u64)page_size - 1); + /* check if hint_addr is aligned */ + if (hint_addr & (va_block_align - 1)) + hint_addr = 0; mutex_lock(&va_range->lock); @@ -549,9 +540,9 @@ static u64 get_va_block(struct hl_device *hdev, /* calc the first possible aligned addr */ valid_start = va_block->start; - if (valid_start & (page_size - 1)) { - valid_start &= page_mask; - valid_start += page_size; + if (valid_start & (va_block_align - 1)) { + valid_start &= align_mask; + valid_start += va_block_align; if (valid_start > va_block->end) continue; } @@ -863,7 +854,7 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, struct hl_va_range *va_range; enum vm_type_t *vm_type; u64 ret_vaddr, hint_addr; - u32 handle = 0; + u32 handle = 0, va_block_align; int rc; bool is_userptr = args->flags & HL_MEM_USERPTR; @@ -873,6 +864,8 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, if (is_userptr) { u64 addr = args->map_host.host_virt_addr, size = args->map_host.mem_size; + u32 page_size = hdev->asic_prop.pmmu.page_size, + huge_page_size = hdev->asic_prop.pmmu_huge.page_size; rc = dma_map_host_va(hdev, addr, size, &userptr); if (rc) { @@ -892,6 +885,27 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, vm_type = (enum vm_type_t *) userptr; hint_addr = args->map_host.hint_addr; handle = phys_pg_pack->handle; + + /* get required alignment */ + if (phys_pg_pack->page_size == page_size) { + va_range = ctx->host_va_range; + + /* + * huge page alignment may be needed in case of regular + * page mapping, depending on the host VA alignment + */ + if (addr & (huge_page_size - 1)) + va_block_align = page_size; + else + va_block_align = huge_page_size; + } else { + /* + * huge page alignment is needed in case of huge page + * mapping + */ + va_range = ctx->host_huge_va_range; + va_block_align = huge_page_size; + } } else { handle = lower_32_bits(args->map_device.handle); @@ -912,6 +926,10 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, vm_type = (enum vm_type_t *) phys_pg_pack; hint_addr = args->map_device.hint_addr; + + /* DRAM VA alignment is the same as the DRAM page size */ + va_range = ctx->dram_va_range; + va_block_align = hdev->asic_prop.dmmu.page_size; } /* @@ -933,16 +951,8 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, goto hnode_err; } - if (is_userptr) - if (phys_pg_pack->page_size == hdev->asic_prop.pmmu.page_size) - va_range = ctx->host_va_range; - else - va_range = ctx->host_huge_va_range; - else - va_range = ctx->dram_va_range; - ret_vaddr = get_va_block(hdev, va_range, phys_pg_pack->total_size, - hint_addr, is_userptr); + hint_addr, va_block_align); if (!ret_vaddr) { dev_err(hdev->dev, "no available va block for handle %u\n", handle); diff --git a/drivers/misc/habanalabs/common/mmu.c b/drivers/misc/habanalabs/common/mmu.c index 3fc0f497fab3..b5058798aeb9 100644 --- a/drivers/misc/habanalabs/common/mmu.c +++ b/drivers/misc/habanalabs/common/mmu.c @@ -1,258 +1,13 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Copyright 2016-2019 HabanaLabs, Ltd. + * Copyright 2016-2020 HabanaLabs, Ltd. * All Rights Reserved. */ -#include "habanalabs.h" -#include "../include/hw_ip/mmu/mmu_general.h" - -#include <linux/genalloc.h> #include <linux/slab.h> -static inline u64 get_phys_addr(struct hl_ctx *ctx, u64 shadow_addr); - -static struct pgt_info *get_pgt_info(struct hl_ctx *ctx, u64 hop_addr) -{ - struct pgt_info *pgt_info = NULL; - - hash_for_each_possible(ctx->mmu_shadow_hash, pgt_info, node, - (unsigned long) hop_addr) - if (hop_addr == pgt_info->shadow_addr) - break; - - return pgt_info; -} - -static void _free_hop(struct hl_ctx *ctx, struct pgt_info *pgt_info) -{ - struct hl_device *hdev = ctx->hdev; - - gen_pool_free(hdev->mmu_pgt_pool, pgt_info->phys_addr, - hdev->asic_prop.mmu_hop_table_size); - hash_del(&pgt_info->node); - kfree((u64 *) (uintptr_t) pgt_info->shadow_addr); - kfree(pgt_info); -} - -static void free_hop(struct hl_ctx *ctx, u64 hop_addr) -{ - struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr); - - _free_hop(ctx, pgt_info); -} - -static u64 alloc_hop(struct hl_ctx *ctx) -{ - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop = &hdev->asic_prop; - struct pgt_info *pgt_info; - u64 phys_addr, shadow_addr; - - pgt_info = kmalloc(sizeof(*pgt_info), GFP_KERNEL); - if (!pgt_info) - return ULLONG_MAX; - - phys_addr = (u64) gen_pool_alloc(hdev->mmu_pgt_pool, - prop->mmu_hop_table_size); - if (!phys_addr) { - dev_err(hdev->dev, "failed to allocate page\n"); - goto pool_add_err; - } - - shadow_addr = (u64) (uintptr_t) kzalloc(prop->mmu_hop_table_size, - GFP_KERNEL); - if (!shadow_addr) - goto shadow_err; - - pgt_info->phys_addr = phys_addr; - pgt_info->shadow_addr = shadow_addr; - pgt_info->ctx = ctx; - pgt_info->num_of_ptes = 0; - hash_add(ctx->mmu_shadow_hash, &pgt_info->node, shadow_addr); - - return shadow_addr; - -shadow_err: - gen_pool_free(hdev->mmu_pgt_pool, phys_addr, prop->mmu_hop_table_size); -pool_add_err: - kfree(pgt_info); - - return ULLONG_MAX; -} - -static inline u64 get_phys_hop0_addr(struct hl_ctx *ctx) -{ - return ctx->hdev->asic_prop.mmu_pgt_addr + - (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); -} - -static inline u64 get_hop0_addr(struct hl_ctx *ctx) -{ - return (u64) (uintptr_t) ctx->hdev->mmu_shadow_hop0 + - (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); -} - -static inline void flush(struct hl_ctx *ctx) -{ - /* flush all writes from all cores to reach PCI */ - mb(); - ctx->hdev->asic_funcs->read_pte(ctx->hdev, get_phys_hop0_addr(ctx)); -} - -/* transform the value to physical address when writing to H/W */ -static inline void write_pte(struct hl_ctx *ctx, u64 shadow_pte_addr, u64 val) -{ - /* - * The value to write is actually the address of the next shadow hop + - * flags at the 12 LSBs. - * Hence in order to get the value to write to the physical PTE, we - * clear the 12 LSBs and translate the shadow hop to its associated - * physical hop, and add back the original 12 LSBs. - */ - u64 phys_val = get_phys_addr(ctx, val & HOP_PHYS_ADDR_MASK) | - (val & FLAGS_MASK); - - ctx->hdev->asic_funcs->write_pte(ctx->hdev, - get_phys_addr(ctx, shadow_pte_addr), - phys_val); - - *(u64 *) (uintptr_t) shadow_pte_addr = val; -} - -/* do not transform the value to physical address when writing to H/W */ -static inline void write_final_pte(struct hl_ctx *ctx, u64 shadow_pte_addr, - u64 val) -{ - ctx->hdev->asic_funcs->write_pte(ctx->hdev, - get_phys_addr(ctx, shadow_pte_addr), - val); - *(u64 *) (uintptr_t) shadow_pte_addr = val; -} - -/* clear the last and present bits */ -static inline void clear_pte(struct hl_ctx *ctx, u64 pte_addr) -{ - /* no need to transform the value to physical address */ - write_final_pte(ctx, pte_addr, 0); -} - -static inline void get_pte(struct hl_ctx *ctx, u64 hop_addr) -{ - get_pgt_info(ctx, hop_addr)->num_of_ptes++; -} - -/* - * put_pte - decrement the num of ptes and free the hop if possible - * - * @ctx: pointer to the context structure - * @hop_addr: addr of the hop - * - * This function returns the number of ptes left on this hop. If the number is - * 0, it means the pte was freed. - */ -static inline int put_pte(struct hl_ctx *ctx, u64 hop_addr) -{ - struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr); - int num_of_ptes_left; - - pgt_info->num_of_ptes--; - - /* - * Need to save the number of ptes left because free_hop might free - * the pgt_info - */ - num_of_ptes_left = pgt_info->num_of_ptes; - if (!num_of_ptes_left) - _free_hop(ctx, pgt_info); - - return num_of_ptes_left; -} - -static inline u64 get_hopN_pte_addr(struct hl_ctx *ctx, u64 hop_addr, - u64 virt_addr, u64 mask, u64 shift) -{ - return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * - ((virt_addr & mask) >> shift); -} - -static inline u64 get_hop0_pte_addr(struct hl_ctx *ctx, - struct hl_mmu_properties *mmu_prop, - u64 hop_addr, u64 vaddr) -{ - return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop0_mask, - mmu_prop->hop0_shift); -} - -static inline u64 get_hop1_pte_addr(struct hl_ctx *ctx, - struct hl_mmu_properties *mmu_prop, - u64 hop_addr, u64 vaddr) -{ - return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop1_mask, - mmu_prop->hop1_shift); -} - -static inline u64 get_hop2_pte_addr(struct hl_ctx *ctx, - struct hl_mmu_properties *mmu_prop, - u64 hop_addr, u64 vaddr) -{ - return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop2_mask, - mmu_prop->hop2_shift); -} - -static inline u64 get_hop3_pte_addr(struct hl_ctx *ctx, - struct hl_mmu_properties *mmu_prop, - u64 hop_addr, u64 vaddr) -{ - return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop3_mask, - mmu_prop->hop3_shift); -} - -static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, - struct hl_mmu_properties *mmu_prop, - u64 hop_addr, u64 vaddr) -{ - return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop4_mask, - mmu_prop->hop4_shift); -} - -static inline u64 get_next_hop_addr(struct hl_ctx *ctx, u64 curr_pte) -{ - if (curr_pte & PAGE_PRESENT_MASK) - return curr_pte & HOP_PHYS_ADDR_MASK; - else - return ULLONG_MAX; -} - -static inline u64 get_alloc_next_hop_addr(struct hl_ctx *ctx, u64 curr_pte, - bool *is_new_hop) -{ - u64 hop_addr = get_next_hop_addr(ctx, curr_pte); - - if (hop_addr == ULLONG_MAX) { - hop_addr = alloc_hop(ctx); - *is_new_hop = (hop_addr != ULLONG_MAX); - } - - return hop_addr; -} - -/* translates shadow address inside hop to a physical address */ -static inline u64 get_phys_addr(struct hl_ctx *ctx, u64 shadow_addr) -{ - u64 page_mask = (ctx->hdev->asic_prop.mmu_hop_table_size - 1); - u64 shadow_hop_addr = shadow_addr & ~page_mask; - u64 pte_offset = shadow_addr & page_mask; - u64 phys_hop_addr; - - if (shadow_hop_addr != get_hop0_addr(ctx)) - phys_hop_addr = get_pgt_info(ctx, shadow_hop_addr)->phys_addr; - else - phys_hop_addr = get_phys_hop0_addr(ctx); - - return phys_hop_addr + pte_offset; -} +#include "habanalabs.h" static bool is_dram_va(struct hl_device *hdev, u64 virt_addr) { @@ -263,155 +18,6 @@ static bool is_dram_va(struct hl_device *hdev, u64 virt_addr) prop->dmmu.end_addr); } -static int dram_default_mapping_init(struct hl_ctx *ctx) -{ - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop = &hdev->asic_prop; - u64 num_of_hop3, total_hops, hop0_addr, hop1_addr, hop2_addr, - hop2_pte_addr, hop3_pte_addr, pte_val; - int rc, i, j, hop3_allocated = 0; - - if ((!hdev->dram_supports_virtual_memory) || - (!hdev->dram_default_page_mapping) || - (ctx->asid == HL_KERNEL_ASID_ID)) - return 0; - - num_of_hop3 = prop->dram_size_for_default_page_mapping; - do_div(num_of_hop3, prop->dram_page_size); - do_div(num_of_hop3, PTE_ENTRIES_IN_HOP); - - /* add hop1 and hop2 */ - total_hops = num_of_hop3 + 2; - - ctx->dram_default_hops = kzalloc(HL_PTE_SIZE * total_hops, GFP_KERNEL); - if (!ctx->dram_default_hops) - return -ENOMEM; - - hop0_addr = get_hop0_addr(ctx); - - hop1_addr = alloc_hop(ctx); - if (hop1_addr == ULLONG_MAX) { - dev_err(hdev->dev, "failed to alloc hop 1\n"); - rc = -ENOMEM; - goto hop1_err; - } - - ctx->dram_default_hops[total_hops - 1] = hop1_addr; - - hop2_addr = alloc_hop(ctx); - if (hop2_addr == ULLONG_MAX) { - dev_err(hdev->dev, "failed to alloc hop 2\n"); - rc = -ENOMEM; - goto hop2_err; - } - - ctx->dram_default_hops[total_hops - 2] = hop2_addr; - - for (i = 0 ; i < num_of_hop3 ; i++) { - ctx->dram_default_hops[i] = alloc_hop(ctx); - if (ctx->dram_default_hops[i] == ULLONG_MAX) { - dev_err(hdev->dev, "failed to alloc hop 3, i: %d\n", i); - rc = -ENOMEM; - goto hop3_err; - } - hop3_allocated++; - } - - /* need only pte 0 in hops 0 and 1 */ - pte_val = (hop1_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; - write_pte(ctx, hop0_addr, pte_val); - - pte_val = (hop2_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; - write_pte(ctx, hop1_addr, pte_val); - get_pte(ctx, hop1_addr); - - hop2_pte_addr = hop2_addr; - for (i = 0 ; i < num_of_hop3 ; i++) { - pte_val = (ctx->dram_default_hops[i] & HOP_PHYS_ADDR_MASK) | - PAGE_PRESENT_MASK; - write_pte(ctx, hop2_pte_addr, pte_val); - get_pte(ctx, hop2_addr); - hop2_pte_addr += HL_PTE_SIZE; - } - - pte_val = (prop->mmu_dram_default_page_addr & HOP_PHYS_ADDR_MASK) | - LAST_MASK | PAGE_PRESENT_MASK; - - for (i = 0 ; i < num_of_hop3 ; i++) { - hop3_pte_addr = ctx->dram_default_hops[i]; - for (j = 0 ; j < PTE_ENTRIES_IN_HOP ; j++) { - write_final_pte(ctx, hop3_pte_addr, pte_val); - get_pte(ctx, ctx->dram_default_hops[i]); - hop3_pte_addr += HL_PTE_SIZE; - } - } - - flush(ctx); - - return 0; - -hop3_err: - for (i = 0 ; i < hop3_allocated ; i++) - free_hop(ctx, ctx->dram_default_hops[i]); - - free_hop(ctx, hop2_addr); -hop2_err: - free_hop(ctx, hop1_addr); -hop1_err: - kfree(ctx->dram_default_hops); - - return rc; -} - -static void dram_default_mapping_fini(struct hl_ctx *ctx) -{ - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop = &hdev->asic_prop; - u64 num_of_hop3, total_hops, hop0_addr, hop1_addr, hop2_addr, - hop2_pte_addr, hop3_pte_addr; - int i, j; - - if ((!hdev->dram_supports_virtual_memory) || - (!hdev->dram_default_page_mapping) || - (ctx->asid == HL_KERNEL_ASID_ID)) - return; - - num_of_hop3 = prop->dram_size_for_default_page_mapping; - do_div(num_of_hop3, prop->dram_page_size); - do_div(num_of_hop3, PTE_ENTRIES_IN_HOP); - - hop0_addr = get_hop0_addr(ctx); - /* add hop1 and hop2 */ - total_hops = num_of_hop3 + 2; - hop1_addr = ctx->dram_default_hops[total_hops - 1]; - hop2_addr = ctx->dram_default_hops[total_hops - 2]; - - for (i = 0 ; i < num_of_hop3 ; i++) { - hop3_pte_addr = ctx->dram_default_hops[i]; - for (j = 0 ; j < PTE_ENTRIES_IN_HOP ; j++) { - clear_pte(ctx, hop3_pte_addr); - put_pte(ctx, ctx->dram_default_hops[i]); - hop3_pte_addr += HL_PTE_SIZE; - } - } - - hop2_pte_addr = hop2_addr; - hop2_pte_addr = hop2_addr; - for (i = 0 ; i < num_of_hop3 ; i++) { - clear_pte(ctx, hop2_pte_addr); - put_pte(ctx, hop2_addr); - hop2_pte_addr += HL_PTE_SIZE; - } - - clear_pte(ctx, hop1_addr); - put_pte(ctx, hop1_addr); - clear_pte(ctx, hop0_addr); - - kfree(ctx->dram_default_hops); - - flush(ctx); -} - /** * hl_mmu_init() - initialize the MMU module. * @hdev: habanalabs device structure. @@ -424,45 +30,10 @@ static void dram_default_mapping_fini(struct hl_ctx *ctx) */ int hl_mmu_init(struct hl_device *hdev) { - struct asic_fixed_properties *prop = &hdev->asic_prop; - int rc; - - if (!hdev->mmu_enable) - return 0; - - hdev->mmu_pgt_pool = - gen_pool_create(__ffs(prop->mmu_hop_table_size), -1); - - if (!hdev->mmu_pgt_pool) { - dev_err(hdev->dev, "Failed to create page gen pool\n"); - return -ENOMEM; - } - - rc = gen_pool_add(hdev->mmu_pgt_pool, prop->mmu_pgt_addr + - prop->mmu_hop0_tables_total_size, - prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size, - -1); - if (rc) { - dev_err(hdev->dev, "Failed to add memory to page gen pool\n"); - goto err_pool_add; - } - - hdev->mmu_shadow_hop0 = kvmalloc_array(prop->max_asid, - prop->mmu_hop_table_size, - GFP_KERNEL | __GFP_ZERO); - if (ZERO_OR_NULL_PTR(hdev->mmu_shadow_hop0)) { - rc = -ENOMEM; - goto err_pool_add; - } - - /* MMU H/W init will be done in device hw_init() */ + if (hdev->mmu_enable) + return hdev->mmu_func.init(hdev); return 0; - -err_pool_add: - gen_pool_destroy(hdev->mmu_pgt_pool); - - return rc; } /** @@ -477,13 +48,8 @@ err_pool_add: */ void hl_mmu_fini(struct hl_device *hdev) { - if (!hdev->mmu_enable) - return; - - /* MMU H/W fini was already done in device hw_fini() */ - - kvfree(hdev->mmu_shadow_hop0); - gen_pool_destroy(hdev->mmu_pgt_pool); + if (hdev->mmu_enable) + hdev->mmu_func.fini(hdev); } /** @@ -498,13 +64,10 @@ int hl_mmu_ctx_init(struct hl_ctx *ctx) { struct hl_device *hdev = ctx->hdev; - if (!hdev->mmu_enable) - return 0; + if (hdev->mmu_enable) + return hdev->mmu_func.ctx_init(ctx); - mutex_init(&ctx->mmu_lock); - hash_init(ctx->mmu_shadow_hash); - - return dram_default_mapping_init(ctx); + return 0; } /* @@ -520,160 +83,9 @@ int hl_mmu_ctx_init(struct hl_ctx *ctx) void hl_mmu_ctx_fini(struct hl_ctx *ctx) { struct hl_device *hdev = ctx->hdev; - struct pgt_info *pgt_info; - struct hlist_node *tmp; - int i; - - if (!hdev->mmu_enable) - return; - - dram_default_mapping_fini(ctx); - - if (!hash_empty(ctx->mmu_shadow_hash)) - dev_err(hdev->dev, "ctx %d is freed while it has pgts in use\n", - ctx->asid); - - hash_for_each_safe(ctx->mmu_shadow_hash, i, tmp, pgt_info, node) { - dev_err_ratelimited(hdev->dev, - "pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n", - pgt_info->phys_addr, ctx->asid, pgt_info->num_of_ptes); - _free_hop(ctx, pgt_info); - } - - mutex_destroy(&ctx->mmu_lock); -} - -static int _hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr, bool is_dram_addr) -{ - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop = &hdev->asic_prop; - struct hl_mmu_properties *mmu_prop; - u64 hop0_addr = 0, hop0_pte_addr = 0, - hop1_addr = 0, hop1_pte_addr = 0, - hop2_addr = 0, hop2_pte_addr = 0, - hop3_addr = 0, hop3_pte_addr = 0, - hop4_addr = 0, hop4_pte_addr = 0, - curr_pte; - bool is_huge, clear_hop3 = true; - - /* shifts and masks are the same in PMMU and HPMMU, use one of them */ - mmu_prop = is_dram_addr ? &prop->dmmu : &prop->pmmu; - - hop0_addr = get_hop0_addr(ctx); - hop0_pte_addr = get_hop0_pte_addr(ctx, mmu_prop, hop0_addr, virt_addr); - - curr_pte = *(u64 *) (uintptr_t) hop0_pte_addr; - - hop1_addr = get_next_hop_addr(ctx, curr_pte); - - if (hop1_addr == ULLONG_MAX) - goto not_mapped; - - hop1_pte_addr = get_hop1_pte_addr(ctx, mmu_prop, hop1_addr, virt_addr); - - curr_pte = *(u64 *) (uintptr_t) hop1_pte_addr; - - hop2_addr = get_next_hop_addr(ctx, curr_pte); - - if (hop2_addr == ULLONG_MAX) - goto not_mapped; - - hop2_pte_addr = get_hop2_pte_addr(ctx, mmu_prop, hop2_addr, virt_addr); - - curr_pte = *(u64 *) (uintptr_t) hop2_pte_addr; - - hop3_addr = get_next_hop_addr(ctx, curr_pte); - - if (hop3_addr == ULLONG_MAX) - goto not_mapped; - - hop3_pte_addr = get_hop3_pte_addr(ctx, mmu_prop, hop3_addr, virt_addr); - - curr_pte = *(u64 *) (uintptr_t) hop3_pte_addr; - - is_huge = curr_pte & LAST_MASK; - - if (is_dram_addr && !is_huge) { - dev_err(hdev->dev, - "DRAM unmapping should use huge pages only\n"); - return -EFAULT; - } - - if (!is_huge) { - hop4_addr = get_next_hop_addr(ctx, curr_pte); - - if (hop4_addr == ULLONG_MAX) - goto not_mapped; - - hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, hop4_addr, - virt_addr); - - curr_pte = *(u64 *) (uintptr_t) hop4_pte_addr; - - clear_hop3 = false; - } - - if (hdev->dram_default_page_mapping && is_dram_addr) { - u64 default_pte = (prop->mmu_dram_default_page_addr & - HOP_PHYS_ADDR_MASK) | LAST_MASK | - PAGE_PRESENT_MASK; - if (curr_pte == default_pte) { - dev_err(hdev->dev, - "DRAM: hop3 PTE points to zero page, can't unmap, va: 0x%llx\n", - virt_addr); - goto not_mapped; - } - - if (!(curr_pte & PAGE_PRESENT_MASK)) { - dev_err(hdev->dev, - "DRAM: hop3 PTE is cleared! can't unmap, va: 0x%llx\n", - virt_addr); - goto not_mapped; - } - - write_final_pte(ctx, hop3_pte_addr, default_pte); - put_pte(ctx, hop3_addr); - } else { - if (!(curr_pte & PAGE_PRESENT_MASK)) - goto not_mapped; - - if (hop4_addr) - clear_pte(ctx, hop4_pte_addr); - else - clear_pte(ctx, hop3_pte_addr); - - if (hop4_addr && !put_pte(ctx, hop4_addr)) - clear_hop3 = true; - - if (!clear_hop3) - goto mapped; - - clear_pte(ctx, hop3_pte_addr); - if (put_pte(ctx, hop3_addr)) - goto mapped; - - clear_pte(ctx, hop2_pte_addr); - - if (put_pte(ctx, hop2_addr)) - goto mapped; - - clear_pte(ctx, hop1_pte_addr); - - if (put_pte(ctx, hop1_addr)) - goto mapped; - - clear_pte(ctx, hop0_pte_addr); - } - -mapped: - return 0; - -not_mapped: - dev_err(hdev->dev, "virt addr 0x%llx is not mapped to phys addr\n", - virt_addr); - - return -EINVAL; + if (hdev->mmu_enable) + hdev->mmu_func.ctx_fini(ctx); } /* @@ -738,7 +150,7 @@ int hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr, u32 page_size, real_virt_addr = virt_addr; for (i = 0 ; i < npages ; i++) { - rc = _hl_mmu_unmap(ctx, real_virt_addr, is_dram_addr); + rc = hdev->mmu_func.unmap(ctx, real_virt_addr, is_dram_addr); if (rc) break; @@ -746,172 +158,7 @@ int hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr, u32 page_size, } if (flush_pte) - flush(ctx); - - return rc; -} - -static int _hl_mmu_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, - u32 page_size, bool is_dram_addr) -{ - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop = &hdev->asic_prop; - struct hl_mmu_properties *mmu_prop; - u64 hop0_addr = 0, hop0_pte_addr = 0, - hop1_addr = 0, hop1_pte_addr = 0, - hop2_addr = 0, hop2_pte_addr = 0, - hop3_addr = 0, hop3_pte_addr = 0, - hop4_addr = 0, hop4_pte_addr = 0, - curr_pte = 0; - bool hop1_new = false, hop2_new = false, hop3_new = false, - hop4_new = false, is_huge; - int rc = -ENOMEM; - - /* - * This mapping function can map a page or a huge page. For huge page - * there are only 3 hops rather than 4. Currently the DRAM allocation - * uses huge pages only but user memory could have been allocated with - * one of the two page sizes. Since this is a common code for all the - * three cases, we need this hugs page check. - */ - if (is_dram_addr) { - mmu_prop = &prop->dmmu; - is_huge = true; - } else if (page_size == prop->pmmu_huge.page_size) { - mmu_prop = &prop->pmmu_huge; - is_huge = true; - } else { - mmu_prop = &prop->pmmu; - is_huge = false; - } - - hop0_addr = get_hop0_addr(ctx); - hop0_pte_addr = get_hop0_pte_addr(ctx, mmu_prop, hop0_addr, virt_addr); - curr_pte = *(u64 *) (uintptr_t) hop0_pte_addr; - - hop1_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop1_new); - if (hop1_addr == ULLONG_MAX) - goto err; - - hop1_pte_addr = get_hop1_pte_addr(ctx, mmu_prop, hop1_addr, virt_addr); - curr_pte = *(u64 *) (uintptr_t) hop1_pte_addr; - - hop2_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop2_new); - if (hop2_addr == ULLONG_MAX) - goto err; - - hop2_pte_addr = get_hop2_pte_addr(ctx, mmu_prop, hop2_addr, virt_addr); - curr_pte = *(u64 *) (uintptr_t) hop2_pte_addr; - - hop3_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop3_new); - if (hop3_addr == ULLONG_MAX) - goto err; - - hop3_pte_addr = get_hop3_pte_addr(ctx, mmu_prop, hop3_addr, virt_addr); - curr_pte = *(u64 *) (uintptr_t) hop3_pte_addr; - - if (!is_huge) { - hop4_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop4_new); - if (hop4_addr == ULLONG_MAX) - goto err; - - hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, hop4_addr, - virt_addr); - curr_pte = *(u64 *) (uintptr_t) hop4_pte_addr; - } - - if (hdev->dram_default_page_mapping && is_dram_addr) { - u64 default_pte = (prop->mmu_dram_default_page_addr & - HOP_PHYS_ADDR_MASK) | LAST_MASK | - PAGE_PRESENT_MASK; - - if (curr_pte != default_pte) { - dev_err(hdev->dev, - "DRAM: mapping already exists for virt_addr 0x%llx\n", - virt_addr); - rc = -EINVAL; - goto err; - } - - if (hop1_new || hop2_new || hop3_new || hop4_new) { - dev_err(hdev->dev, - "DRAM mapping should not allocate more hops\n"); - rc = -EFAULT; - goto err; - } - } else if (curr_pte & PAGE_PRESENT_MASK) { - dev_err(hdev->dev, - "mapping already exists for virt_addr 0x%llx\n", - virt_addr); - - dev_dbg(hdev->dev, "hop0 pte: 0x%llx (0x%llx)\n", - *(u64 *) (uintptr_t) hop0_pte_addr, hop0_pte_addr); - dev_dbg(hdev->dev, "hop1 pte: 0x%llx (0x%llx)\n", - *(u64 *) (uintptr_t) hop1_pte_addr, hop1_pte_addr); - dev_dbg(hdev->dev, "hop2 pte: 0x%llx (0x%llx)\n", - *(u64 *) (uintptr_t) hop2_pte_addr, hop2_pte_addr); - dev_dbg(hdev->dev, "hop3 pte: 0x%llx (0x%llx)\n", - *(u64 *) (uintptr_t) hop3_pte_addr, hop3_pte_addr); - - if (!is_huge) - dev_dbg(hdev->dev, "hop4 pte: 0x%llx (0x%llx)\n", - *(u64 *) (uintptr_t) hop4_pte_addr, - hop4_pte_addr); - - rc = -EINVAL; - goto err; - } - - curr_pte = (phys_addr & HOP_PHYS_ADDR_MASK) | LAST_MASK - | PAGE_PRESENT_MASK; - - if (is_huge) - write_final_pte(ctx, hop3_pte_addr, curr_pte); - else - write_final_pte(ctx, hop4_pte_addr, curr_pte); - - if (hop1_new) { - curr_pte = - (hop1_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; - write_pte(ctx, hop0_pte_addr, curr_pte); - } - if (hop2_new) { - curr_pte = - (hop2_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; - write_pte(ctx, hop1_pte_addr, curr_pte); - get_pte(ctx, hop1_addr); - } - if (hop3_new) { - curr_pte = - (hop3_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; - write_pte(ctx, hop2_pte_addr, curr_pte); - get_pte(ctx, hop2_addr); - } - - if (!is_huge) { - if (hop4_new) { - curr_pte = (hop4_addr & HOP_PHYS_ADDR_MASK) | - PAGE_PRESENT_MASK; - write_pte(ctx, hop3_pte_addr, curr_pte); - get_pte(ctx, hop3_addr); - } - - get_pte(ctx, hop4_addr); - } else { - get_pte(ctx, hop3_addr); - } - - return 0; - -err: - if (hop4_new) - free_hop(ctx, hop4_addr); - if (hop3_new) - free_hop(ctx, hop3_addr); - if (hop2_new) - free_hop(ctx, hop2_addr); - if (hop1_new) - free_hop(ctx, hop1_addr); + hdev->mmu_func.flush(ctx); return rc; } @@ -984,7 +231,7 @@ int hl_mmu_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, u32 page_size, real_phys_addr = phys_addr; for (i = 0 ; i < npages ; i++) { - rc = _hl_mmu_map(ctx, real_virt_addr, real_phys_addr, + rc = hdev->mmu_func.map(ctx, real_virt_addr, real_phys_addr, real_page_size, is_dram_addr); if (rc) goto err; @@ -995,21 +242,21 @@ int hl_mmu_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, u32 page_size, } if (flush_pte) - flush(ctx); + hdev->mmu_func.flush(ctx); return 0; err: real_virt_addr = virt_addr; for (i = 0 ; i < mapped_cnt ; i++) { - if (_hl_mmu_unmap(ctx, real_virt_addr, is_dram_addr)) + if (hdev->mmu_func.unmap(ctx, real_virt_addr, is_dram_addr)) dev_warn_ratelimited(hdev->dev, "failed to unmap va: 0x%llx\n", real_virt_addr); real_virt_addr += real_page_size; } - flush(ctx); + hdev->mmu_func.flush(ctx); return rc; } @@ -1022,7 +269,10 @@ err: */ void hl_mmu_swap_out(struct hl_ctx *ctx) { + struct hl_device *hdev = ctx->hdev; + if (hdev->mmu_enable) + hdev->mmu_func.swap_out(ctx); } /* @@ -1033,5 +283,27 @@ void hl_mmu_swap_out(struct hl_ctx *ctx) */ void hl_mmu_swap_in(struct hl_ctx *ctx) { + struct hl_device *hdev = ctx->hdev; + + if (hdev->mmu_enable) + hdev->mmu_func.swap_in(ctx); +} + +int hl_mmu_if_set_funcs(struct hl_device *hdev) +{ + if (!hdev->mmu_enable) + return 0; + + switch (hdev->asic_type) { + case ASIC_GOYA: + case ASIC_GAUDI: + hl_mmu_v1_set_funcs(hdev); + break; + default: + dev_err(hdev->dev, "Unrecognized ASIC type %d\n", + hdev->asic_type); + return -EOPNOTSUPP; + } + return 0; } diff --git a/drivers/misc/habanalabs/common/mmu_v1.c b/drivers/misc/habanalabs/common/mmu_v1.c new file mode 100644 index 000000000000..8d1eb5265419 --- /dev/null +++ b/drivers/misc/habanalabs/common/mmu_v1.c @@ -0,0 +1,863 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright 2016-2019 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +#include "habanalabs.h" +#include "../include/hw_ip/mmu/mmu_general.h" + +#include <linux/genalloc.h> +#include <linux/slab.h> + +static inline u64 get_phys_addr(struct hl_ctx *ctx, u64 shadow_addr); + +static struct pgt_info *get_pgt_info(struct hl_ctx *ctx, u64 hop_addr) +{ + struct pgt_info *pgt_info = NULL; + + hash_for_each_possible(ctx->mmu_shadow_hash, pgt_info, node, + (unsigned long) hop_addr) + if (hop_addr == pgt_info->shadow_addr) + break; + + return pgt_info; +} + +static void _free_hop(struct hl_ctx *ctx, struct pgt_info *pgt_info) +{ + struct hl_device *hdev = ctx->hdev; + + gen_pool_free(hdev->mmu_priv.mmu_pgt_pool, pgt_info->phys_addr, + hdev->asic_prop.mmu_hop_table_size); + hash_del(&pgt_info->node); + kfree((u64 *) (uintptr_t) pgt_info->shadow_addr); + kfree(pgt_info); +} + +static void free_hop(struct hl_ctx *ctx, u64 hop_addr) +{ + struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr); + + _free_hop(ctx, pgt_info); +} + +static u64 alloc_hop(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct pgt_info *pgt_info; + u64 phys_addr, shadow_addr; + + pgt_info = kmalloc(sizeof(*pgt_info), GFP_KERNEL); + if (!pgt_info) + return ULLONG_MAX; + + phys_addr = (u64) gen_pool_alloc(hdev->mmu_priv.mmu_pgt_pool, + prop->mmu_hop_table_size); + if (!phys_addr) { + dev_err(hdev->dev, "failed to allocate page\n"); + goto pool_add_err; + } + + shadow_addr = (u64) (uintptr_t) kzalloc(prop->mmu_hop_table_size, + GFP_KERNEL); + if (!shadow_addr) + goto shadow_err; + + pgt_info->phys_addr = phys_addr; + pgt_info->shadow_addr = shadow_addr; + pgt_info->ctx = ctx; + pgt_info->num_of_ptes = 0; + hash_add(ctx->mmu_shadow_hash, &pgt_info->node, shadow_addr); + + return shadow_addr; + +shadow_err: + gen_pool_free(hdev->mmu_priv.mmu_pgt_pool, phys_addr, + prop->mmu_hop_table_size); +pool_add_err: + kfree(pgt_info); + + return ULLONG_MAX; +} + +static inline u64 get_phys_hop0_addr(struct hl_ctx *ctx) +{ + return ctx->hdev->asic_prop.mmu_pgt_addr + + (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); +} + +static inline u64 get_hop0_addr(struct hl_ctx *ctx) +{ + return (u64) (uintptr_t) ctx->hdev->mmu_priv.mmu_shadow_hop0 + + (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); +} + +static void flush(struct hl_ctx *ctx) +{ + /* flush all writes from all cores to reach PCI */ + mb(); + ctx->hdev->asic_funcs->read_pte(ctx->hdev, get_phys_hop0_addr(ctx)); +} + +/* transform the value to physical address when writing to H/W */ +static inline void write_pte(struct hl_ctx *ctx, u64 shadow_pte_addr, u64 val) +{ + /* + * The value to write is actually the address of the next shadow hop + + * flags at the 12 LSBs. + * Hence in order to get the value to write to the physical PTE, we + * clear the 12 LSBs and translate the shadow hop to its associated + * physical hop, and add back the original 12 LSBs. + */ + u64 phys_val = get_phys_addr(ctx, val & HOP_PHYS_ADDR_MASK) | + (val & FLAGS_MASK); + + ctx->hdev->asic_funcs->write_pte(ctx->hdev, + get_phys_addr(ctx, shadow_pte_addr), + phys_val); + + *(u64 *) (uintptr_t) shadow_pte_addr = val; +} + +/* do not transform the value to physical address when writing to H/W */ +static inline void write_final_pte(struct hl_ctx *ctx, u64 shadow_pte_addr, + u64 val) +{ + ctx->hdev->asic_funcs->write_pte(ctx->hdev, + get_phys_addr(ctx, shadow_pte_addr), + val); + *(u64 *) (uintptr_t) shadow_pte_addr = val; +} + +/* clear the last and present bits */ +static inline void clear_pte(struct hl_ctx *ctx, u64 pte_addr) +{ + /* no need to transform the value to physical address */ + write_final_pte(ctx, pte_addr, 0); +} + +static inline void get_pte(struct hl_ctx *ctx, u64 hop_addr) +{ + get_pgt_info(ctx, hop_addr)->num_of_ptes++; +} + +/* + * put_pte - decrement the num of ptes and free the hop if possible + * + * @ctx: pointer to the context structure + * @hop_addr: addr of the hop + * + * This function returns the number of ptes left on this hop. If the number is + * 0, it means the pte was freed. + */ +static inline int put_pte(struct hl_ctx *ctx, u64 hop_addr) +{ + struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr); + int num_of_ptes_left; + + pgt_info->num_of_ptes--; + + /* + * Need to save the number of ptes left because free_hop might free + * the pgt_info + */ + num_of_ptes_left = pgt_info->num_of_ptes; + if (!num_of_ptes_left) + _free_hop(ctx, pgt_info); + + return num_of_ptes_left; +} + +static inline u64 get_hopN_pte_addr(struct hl_ctx *ctx, u64 hop_addr, + u64 virt_addr, u64 mask, u64 shift) +{ + return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr & mask) >> shift); +} + +static inline u64 get_hop0_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_prop, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop0_mask, + mmu_prop->hop0_shift); +} + +static inline u64 get_hop1_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_prop, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop1_mask, + mmu_prop->hop1_shift); +} + +static inline u64 get_hop2_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_prop, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop2_mask, + mmu_prop->hop2_shift); +} + +static inline u64 get_hop3_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_prop, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop3_mask, + mmu_prop->hop3_shift); +} + +static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, + struct hl_mmu_properties *mmu_prop, + u64 hop_addr, u64 vaddr) +{ + return get_hopN_pte_addr(ctx, hop_addr, vaddr, mmu_prop->hop4_mask, + mmu_prop->hop4_shift); +} + +static inline u64 get_next_hop_addr(struct hl_ctx *ctx, u64 curr_pte) +{ + if (curr_pte & PAGE_PRESENT_MASK) + return curr_pte & HOP_PHYS_ADDR_MASK; + else + return ULLONG_MAX; +} + +static inline u64 get_alloc_next_hop_addr(struct hl_ctx *ctx, u64 curr_pte, + bool *is_new_hop) +{ + u64 hop_addr = get_next_hop_addr(ctx, curr_pte); + + if (hop_addr == ULLONG_MAX) { + hop_addr = alloc_hop(ctx); + *is_new_hop = (hop_addr != ULLONG_MAX); + } + + return hop_addr; +} + +/* translates shadow address inside hop to a physical address */ +static inline u64 get_phys_addr(struct hl_ctx *ctx, u64 shadow_addr) +{ + u64 page_mask = (ctx->hdev->asic_prop.mmu_hop_table_size - 1); + u64 shadow_hop_addr = shadow_addr & ~page_mask; + u64 pte_offset = shadow_addr & page_mask; + u64 phys_hop_addr; + + if (shadow_hop_addr != get_hop0_addr(ctx)) + phys_hop_addr = get_pgt_info(ctx, shadow_hop_addr)->phys_addr; + else + phys_hop_addr = get_phys_hop0_addr(ctx); + + return phys_hop_addr + pte_offset; +} + +static int dram_default_mapping_init(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + u64 num_of_hop3, total_hops, hop0_addr, hop1_addr, hop2_addr, + hop2_pte_addr, hop3_pte_addr, pte_val; + int rc, i, j, hop3_allocated = 0; + + if ((!hdev->dram_supports_virtual_memory) || + (!hdev->dram_default_page_mapping) || + (ctx->asid == HL_KERNEL_ASID_ID)) + return 0; + + num_of_hop3 = prop->dram_size_for_default_page_mapping; + do_div(num_of_hop3, prop->dram_page_size); + do_div(num_of_hop3, PTE_ENTRIES_IN_HOP); + + /* add hop1 and hop2 */ + total_hops = num_of_hop3 + 2; + + ctx->dram_default_hops = kzalloc(HL_PTE_SIZE * total_hops, GFP_KERNEL); + if (!ctx->dram_default_hops) + return -ENOMEM; + + hop0_addr = get_hop0_addr(ctx); + + hop1_addr = alloc_hop(ctx); + if (hop1_addr == ULLONG_MAX) { + dev_err(hdev->dev, "failed to alloc hop 1\n"); + rc = -ENOMEM; + goto hop1_err; + } + + ctx->dram_default_hops[total_hops - 1] = hop1_addr; + + hop2_addr = alloc_hop(ctx); + if (hop2_addr == ULLONG_MAX) { + dev_err(hdev->dev, "failed to alloc hop 2\n"); + rc = -ENOMEM; + goto hop2_err; + } + + ctx->dram_default_hops[total_hops - 2] = hop2_addr; + + for (i = 0 ; i < num_of_hop3 ; i++) { + ctx->dram_default_hops[i] = alloc_hop(ctx); + if (ctx->dram_default_hops[i] == ULLONG_MAX) { + dev_err(hdev->dev, "failed to alloc hop 3, i: %d\n", i); + rc = -ENOMEM; + goto hop3_err; + } + hop3_allocated++; + } + + /* need only pte 0 in hops 0 and 1 */ + pte_val = (hop1_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; + write_pte(ctx, hop0_addr, pte_val); + + pte_val = (hop2_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; + write_pte(ctx, hop1_addr, pte_val); + get_pte(ctx, hop1_addr); + + hop2_pte_addr = hop2_addr; + for (i = 0 ; i < num_of_hop3 ; i++) { + pte_val = (ctx->dram_default_hops[i] & HOP_PHYS_ADDR_MASK) | + PAGE_PRESENT_MASK; + write_pte(ctx, hop2_pte_addr, pte_val); + get_pte(ctx, hop2_addr); + hop2_pte_addr += HL_PTE_SIZE; + } + + pte_val = (prop->mmu_dram_default_page_addr & HOP_PHYS_ADDR_MASK) | + LAST_MASK | PAGE_PRESENT_MASK; + + for (i = 0 ; i < num_of_hop3 ; i++) { + hop3_pte_addr = ctx->dram_default_hops[i]; + for (j = 0 ; j < PTE_ENTRIES_IN_HOP ; j++) { + write_final_pte(ctx, hop3_pte_addr, pte_val); + get_pte(ctx, ctx->dram_default_hops[i]); + hop3_pte_addr += HL_PTE_SIZE; + } + } + + flush(ctx); + + return 0; + +hop3_err: + for (i = 0 ; i < hop3_allocated ; i++) + free_hop(ctx, ctx->dram_default_hops[i]); + + free_hop(ctx, hop2_addr); +hop2_err: + free_hop(ctx, hop1_addr); +hop1_err: + kfree(ctx->dram_default_hops); + + return rc; +} + +static void dram_default_mapping_fini(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + u64 num_of_hop3, total_hops, hop0_addr, hop1_addr, hop2_addr, + hop2_pte_addr, hop3_pte_addr; + int i, j; + + if ((!hdev->dram_supports_virtual_memory) || + (!hdev->dram_default_page_mapping) || + (ctx->asid == HL_KERNEL_ASID_ID)) + return; + + num_of_hop3 = prop->dram_size_for_default_page_mapping; + do_div(num_of_hop3, prop->dram_page_size); + do_div(num_of_hop3, PTE_ENTRIES_IN_HOP); + + hop0_addr = get_hop0_addr(ctx); + /* add hop1 and hop2 */ + total_hops = num_of_hop3 + 2; + hop1_addr = ctx->dram_default_hops[total_hops - 1]; + hop2_addr = ctx->dram_default_hops[total_hops - 2]; + + for (i = 0 ; i < num_of_hop3 ; i++) { + hop3_pte_addr = ctx->dram_default_hops[i]; + for (j = 0 ; j < PTE_ENTRIES_IN_HOP ; j++) { + clear_pte(ctx, hop3_pte_addr); + put_pte(ctx, ctx->dram_default_hops[i]); + hop3_pte_addr += HL_PTE_SIZE; + } + } + + hop2_pte_addr = hop2_addr; + hop2_pte_addr = hop2_addr; + for (i = 0 ; i < num_of_hop3 ; i++) { + clear_pte(ctx, hop2_pte_addr); + put_pte(ctx, hop2_addr); + hop2_pte_addr += HL_PTE_SIZE; + } + + clear_pte(ctx, hop1_addr); + put_pte(ctx, hop1_addr); + clear_pte(ctx, hop0_addr); + + kfree(ctx->dram_default_hops); + + flush(ctx); +} + +/** + * hl_mmu_v1_init() - initialize the MMU module. + * @hdev: habanalabs device structure. + * + * This function does the following: + * - Create a pool of pages for pgt_infos. + * - Create a shadow table for pgt + * + * Return: 0 for success, non-zero for failure. + */ +static int hl_mmu_v1_init(struct hl_device *hdev) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + int rc; + + hdev->mmu_priv.mmu_pgt_pool = + gen_pool_create(__ffs(prop->mmu_hop_table_size), -1); + + if (!hdev->mmu_priv.mmu_pgt_pool) { + dev_err(hdev->dev, "Failed to create page gen pool\n"); + return -ENOMEM; + } + + rc = gen_pool_add(hdev->mmu_priv.mmu_pgt_pool, prop->mmu_pgt_addr + + prop->mmu_hop0_tables_total_size, + prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size, + -1); + if (rc) { + dev_err(hdev->dev, "Failed to add memory to page gen pool\n"); + goto err_pool_add; + } + + hdev->mmu_priv.mmu_shadow_hop0 = kvmalloc_array(prop->max_asid, + prop->mmu_hop_table_size, + GFP_KERNEL | __GFP_ZERO); + if (ZERO_OR_NULL_PTR(hdev->mmu_priv.mmu_shadow_hop0)) { + rc = -ENOMEM; + goto err_pool_add; + } + + /* MMU H/W init will be done in device hw_init() */ + + return 0; + +err_pool_add: + gen_pool_destroy(hdev->mmu_priv.mmu_pgt_pool); + + return rc; +} + +/** + * hl_mmu_fini() - release the MMU module. + * @hdev: habanalabs device structure. + * + * This function does the following: + * - Disable MMU in H/W. + * - Free the pgt_infos pool. + * + * All contexts should be freed before calling this function. + */ +static void hl_mmu_v1_fini(struct hl_device *hdev) +{ + /* MMU H/W fini was already done in device hw_fini() */ + + kvfree(hdev->mmu_priv.mmu_shadow_hop0); + gen_pool_destroy(hdev->mmu_priv.mmu_pgt_pool); +} + +/** + * hl_mmu_ctx_init() - initialize a context for using the MMU module. + * @ctx: pointer to the context structure to initialize. + * + * Initialize a mutex to protect the concurrent mapping flow, a hash to hold all + * page tables hops related to this context. + * Return: 0 on success, non-zero otherwise. + */ +static int hl_mmu_v1_ctx_init(struct hl_ctx *ctx) +{ + mutex_init(&ctx->mmu_lock); + hash_init(ctx->mmu_shadow_hash); + + return dram_default_mapping_init(ctx); +} + +/* + * hl_mmu_ctx_fini - disable a ctx from using the mmu module + * + * @ctx: pointer to the context structure + * + * This function does the following: + * - Free any pgts which were not freed yet + * - Free the mutex + * - Free DRAM default page mapping hops + */ +static void hl_mmu_v1_ctx_fini(struct hl_ctx *ctx) +{ + struct hl_device *hdev = ctx->hdev; + struct pgt_info *pgt_info; + struct hlist_node *tmp; + int i; + + dram_default_mapping_fini(ctx); + + if (!hash_empty(ctx->mmu_shadow_hash)) + dev_err(hdev->dev, "ctx %d is freed while it has pgts in use\n", + ctx->asid); + + hash_for_each_safe(ctx->mmu_shadow_hash, i, tmp, pgt_info, node) { + dev_err_ratelimited(hdev->dev, + "pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n", + pgt_info->phys_addr, ctx->asid, pgt_info->num_of_ptes); + _free_hop(ctx, pgt_info); + } + + mutex_destroy(&ctx->mmu_lock); +} + +static int _hl_mmu_v1_unmap(struct hl_ctx *ctx, + u64 virt_addr, bool is_dram_addr) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_mmu_properties *mmu_prop; + u64 hop0_addr = 0, hop0_pte_addr = 0, + hop1_addr = 0, hop1_pte_addr = 0, + hop2_addr = 0, hop2_pte_addr = 0, + hop3_addr = 0, hop3_pte_addr = 0, + hop4_addr = 0, hop4_pte_addr = 0, + curr_pte; + bool is_huge, clear_hop3 = true; + + /* shifts and masks are the same in PMMU and HPMMU, use one of them */ + mmu_prop = is_dram_addr ? &prop->dmmu : &prop->pmmu; + + hop0_addr = get_hop0_addr(ctx); + hop0_pte_addr = get_hop0_pte_addr(ctx, mmu_prop, hop0_addr, virt_addr); + + curr_pte = *(u64 *) (uintptr_t) hop0_pte_addr; + + hop1_addr = get_next_hop_addr(ctx, curr_pte); + + if (hop1_addr == ULLONG_MAX) + goto not_mapped; + + hop1_pte_addr = get_hop1_pte_addr(ctx, mmu_prop, hop1_addr, virt_addr); + + curr_pte = *(u64 *) (uintptr_t) hop1_pte_addr; + + hop2_addr = get_next_hop_addr(ctx, curr_pte); + + if (hop2_addr == ULLONG_MAX) + goto not_mapped; + + hop2_pte_addr = get_hop2_pte_addr(ctx, mmu_prop, hop2_addr, virt_addr); + + curr_pte = *(u64 *) (uintptr_t) hop2_pte_addr; + + hop3_addr = get_next_hop_addr(ctx, curr_pte); + + if (hop3_addr == ULLONG_MAX) + goto not_mapped; + + hop3_pte_addr = get_hop3_pte_addr(ctx, mmu_prop, hop3_addr, virt_addr); + + curr_pte = *(u64 *) (uintptr_t) hop3_pte_addr; + + is_huge = curr_pte & LAST_MASK; + + if (is_dram_addr && !is_huge) { + dev_err(hdev->dev, + "DRAM unmapping should use huge pages only\n"); + return -EFAULT; + } + + if (!is_huge) { + hop4_addr = get_next_hop_addr(ctx, curr_pte); + + if (hop4_addr == ULLONG_MAX) + goto not_mapped; + + hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, hop4_addr, + virt_addr); + + curr_pte = *(u64 *) (uintptr_t) hop4_pte_addr; + + clear_hop3 = false; + } + + if (hdev->dram_default_page_mapping && is_dram_addr) { + u64 default_pte = (prop->mmu_dram_default_page_addr & + HOP_PHYS_ADDR_MASK) | LAST_MASK | + PAGE_PRESENT_MASK; + if (curr_pte == default_pte) { + dev_err(hdev->dev, + "DRAM: hop3 PTE points to zero page, can't unmap, va: 0x%llx\n", + virt_addr); + goto not_mapped; + } + + if (!(curr_pte & PAGE_PRESENT_MASK)) { + dev_err(hdev->dev, + "DRAM: hop3 PTE is cleared! can't unmap, va: 0x%llx\n", + virt_addr); + goto not_mapped; + } + + write_final_pte(ctx, hop3_pte_addr, default_pte); + put_pte(ctx, hop3_addr); + } else { + if (!(curr_pte & PAGE_PRESENT_MASK)) + goto not_mapped; + + if (hop4_addr) + clear_pte(ctx, hop4_pte_addr); + else + clear_pte(ctx, hop3_pte_addr); + + if (hop4_addr && !put_pte(ctx, hop4_addr)) + clear_hop3 = true; + + if (!clear_hop3) + goto mapped; + + clear_pte(ctx, hop3_pte_addr); + + if (put_pte(ctx, hop3_addr)) + goto mapped; + + clear_pte(ctx, hop2_pte_addr); + + if (put_pte(ctx, hop2_addr)) + goto mapped; + + clear_pte(ctx, hop1_pte_addr); + + if (put_pte(ctx, hop1_addr)) + goto mapped; + + clear_pte(ctx, hop0_pte_addr); + } + +mapped: + return 0; + +not_mapped: + dev_err(hdev->dev, "virt addr 0x%llx is not mapped to phys addr\n", + virt_addr); + + return -EINVAL; +} + +static int _hl_mmu_v1_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, + u32 page_size, bool is_dram_addr) +{ + struct hl_device *hdev = ctx->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_mmu_properties *mmu_prop; + u64 hop0_addr = 0, hop0_pte_addr = 0, + hop1_addr = 0, hop1_pte_addr = 0, + hop2_addr = 0, hop2_pte_addr = 0, + hop3_addr = 0, hop3_pte_addr = 0, + hop4_addr = 0, hop4_pte_addr = 0, + curr_pte = 0; + bool hop1_new = false, hop2_new = false, hop3_new = false, + hop4_new = false, is_huge; + int rc = -ENOMEM; + + /* + * This mapping function can map a page or a huge page. For huge page + * there are only 3 hops rather than 4. Currently the DRAM allocation + * uses huge pages only but user memory could have been allocated with + * one of the two page sizes. Since this is a common code for all the + * three cases, we need this hugs page check. + */ + if (is_dram_addr) { + mmu_prop = &prop->dmmu; + is_huge = true; + } else if (page_size == prop->pmmu_huge.page_size) { + mmu_prop = &prop->pmmu_huge; + is_huge = true; + } else { + mmu_prop = &prop->pmmu; + is_huge = false; + } + + hop0_addr = get_hop0_addr(ctx); + hop0_pte_addr = get_hop0_pte_addr(ctx, mmu_prop, hop0_addr, virt_addr); + curr_pte = *(u64 *) (uintptr_t) hop0_pte_addr; + + hop1_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop1_new); + if (hop1_addr == ULLONG_MAX) + goto err; + + hop1_pte_addr = get_hop1_pte_addr(ctx, mmu_prop, hop1_addr, virt_addr); + curr_pte = *(u64 *) (uintptr_t) hop1_pte_addr; + + hop2_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop2_new); + if (hop2_addr == ULLONG_MAX) + goto err; + + hop2_pte_addr = get_hop2_pte_addr(ctx, mmu_prop, hop2_addr, virt_addr); + curr_pte = *(u64 *) (uintptr_t) hop2_pte_addr; + + hop3_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop3_new); + if (hop3_addr == ULLONG_MAX) + goto err; + + hop3_pte_addr = get_hop3_pte_addr(ctx, mmu_prop, hop3_addr, virt_addr); + curr_pte = *(u64 *) (uintptr_t) hop3_pte_addr; + + if (!is_huge) { + hop4_addr = get_alloc_next_hop_addr(ctx, curr_pte, &hop4_new); + if (hop4_addr == ULLONG_MAX) + goto err; + + hop4_pte_addr = get_hop4_pte_addr(ctx, mmu_prop, hop4_addr, + virt_addr); + curr_pte = *(u64 *) (uintptr_t) hop4_pte_addr; + } + + if (hdev->dram_default_page_mapping && is_dram_addr) { + u64 default_pte = (prop->mmu_dram_default_page_addr & + HOP_PHYS_ADDR_MASK) | LAST_MASK | + PAGE_PRESENT_MASK; + + if (curr_pte != default_pte) { + dev_err(hdev->dev, + "DRAM: mapping already exists for virt_addr 0x%llx\n", + virt_addr); + rc = -EINVAL; + goto err; + } + + if (hop1_new || hop2_new || hop3_new || hop4_new) { + dev_err(hdev->dev, + "DRAM mapping should not allocate more hops\n"); + rc = -EFAULT; + goto err; + } + } else if (curr_pte & PAGE_PRESENT_MASK) { + dev_err(hdev->dev, + "mapping already exists for virt_addr 0x%llx\n", + virt_addr); + + dev_dbg(hdev->dev, "hop0 pte: 0x%llx (0x%llx)\n", + *(u64 *) (uintptr_t) hop0_pte_addr, hop0_pte_addr); + dev_dbg(hdev->dev, "hop1 pte: 0x%llx (0x%llx)\n", + *(u64 *) (uintptr_t) hop1_pte_addr, hop1_pte_addr); + dev_dbg(hdev->dev, "hop2 pte: 0x%llx (0x%llx)\n", + *(u64 *) (uintptr_t) hop2_pte_addr, hop2_pte_addr); + dev_dbg(hdev->dev, "hop3 pte: 0x%llx (0x%llx)\n", + *(u64 *) (uintptr_t) hop3_pte_addr, hop3_pte_addr); + + if (!is_huge) + dev_dbg(hdev->dev, "hop4 pte: 0x%llx (0x%llx)\n", + *(u64 *) (uintptr_t) hop4_pte_addr, + hop4_pte_addr); + + rc = -EINVAL; + goto err; + } + + curr_pte = (phys_addr & HOP_PHYS_ADDR_MASK) | LAST_MASK + | PAGE_PRESENT_MASK; + + if (is_huge) + write_final_pte(ctx, hop3_pte_addr, curr_pte); + else + write_final_pte(ctx, hop4_pte_addr, curr_pte); + + if (hop1_new) { + curr_pte = + (hop1_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; + write_pte(ctx, hop0_pte_addr, curr_pte); + } + if (hop2_new) { + curr_pte = + (hop2_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; + write_pte(ctx, hop1_pte_addr, curr_pte); + get_pte(ctx, hop1_addr); + } + if (hop3_new) { + curr_pte = + (hop3_addr & HOP_PHYS_ADDR_MASK) | PAGE_PRESENT_MASK; + write_pte(ctx, hop2_pte_addr, curr_pte); + get_pte(ctx, hop2_addr); + } + + if (!is_huge) { + if (hop4_new) { + curr_pte = (hop4_addr & HOP_PHYS_ADDR_MASK) | + PAGE_PRESENT_MASK; + write_pte(ctx, hop3_pte_addr, curr_pte); + get_pte(ctx, hop3_addr); + } + + get_pte(ctx, hop4_addr); + } else { + get_pte(ctx, hop3_addr); + } + + return 0; + +err: + if (hop4_new) + free_hop(ctx, hop4_addr); + if (hop3_new) + free_hop(ctx, hop3_addr); + if (hop2_new) + free_hop(ctx, hop2_addr); + if (hop1_new) + free_hop(ctx, hop1_addr); + + return rc; +} + +/* + * hl_mmu_v1_swap_out - marks all mapping of the given ctx as swapped out + * + * @ctx: pointer to the context structure + * + */ +static void hl_mmu_v1_swap_out(struct hl_ctx *ctx) +{ + +} + +/* + * hl_mmu_v1_swap_in - marks all mapping of the given ctx as swapped in + * + * @ctx: pointer to the context structure + * + */ +static void hl_mmu_v1_swap_in(struct hl_ctx *ctx) +{ + +} + +/* + * hl_mmu_v1_prepare - prepare mmu for working with mmu v1 + * + * @hdev: pointer to the device structure + */ +void hl_mmu_v1_set_funcs(struct hl_device *hdev) +{ + struct hl_mmu_funcs *mmu = &hdev->mmu_func; + + mmu->init = hl_mmu_v1_init; + mmu->fini = hl_mmu_v1_fini; + mmu->ctx_init = hl_mmu_v1_ctx_init; + mmu->ctx_fini = hl_mmu_v1_ctx_fini; + mmu->map = _hl_mmu_v1_map; + mmu->unmap = _hl_mmu_v1_unmap; + mmu->flush = flush; + mmu->swap_out = hl_mmu_v1_swap_out; + mmu->swap_in = hl_mmu_v1_swap_in; +} diff --git a/drivers/misc/habanalabs/common/pci.c b/drivers/misc/habanalabs/common/pci.c index 2770f03b6cbb..4327e5704ebb 100644 --- a/drivers/misc/habanalabs/common/pci.c +++ b/drivers/misc/habanalabs/common/pci.c @@ -9,7 +9,6 @@ #include "../include/hw_ip/pci/pci_general.h" #include <linux/pci.h> -#include <linux/bitfield.h> #define HL_PLDM_PCI_ELBI_TIMEOUT_MSEC (HL_PCI_ELBI_TIMEOUT_MSEC * 10) @@ -339,12 +338,17 @@ static int hl_pci_set_dma_mask(struct hl_device *hdev) /** * hl_pci_init() - PCI initialization code. * @hdev: Pointer to hl_device structure. + * @cpu_boot_status_reg: status register of the device's CPU + * @boot_err0_reg: boot error register of the device's CPU + * @preboot_ver_timeout: how much to wait before bailing out on reading + * the preboot version * * Set DMA masks, initialize the PCI controller and map the PCI BARs. * * Return: 0 on success, non-zero for failure. */ -int hl_pci_init(struct hl_device *hdev) +int hl_pci_init(struct hl_device *hdev, u32 cpu_boot_status_reg, + u32 boot_err0_reg, u32 preboot_ver_timeout) { struct pci_dev *pdev = hdev->pdev; int rc; @@ -376,6 +380,15 @@ int hl_pci_init(struct hl_device *hdev) if (rc) goto unmap_pci_bars; + /* Before continuing in the initialization, we need to read the preboot + * version to determine whether we run with a security-enabled firmware + * The check will be done in each ASIC's specific code + */ + rc = hl_fw_read_preboot_ver(hdev, cpu_boot_status_reg, boot_err0_reg, + preboot_ver_timeout); + if (rc) + goto unmap_pci_bars; + return 0; unmap_pci_bars: diff --git a/drivers/misc/habanalabs/common/sysfs.c b/drivers/misc/habanalabs/common/sysfs.c index 5ae484cc84cd..3ceae87016b1 100644 --- a/drivers/misc/habanalabs/common/sysfs.c +++ b/drivers/misc/habanalabs/common/sysfs.c @@ -11,18 +11,18 @@ long hl_get_frequency(struct hl_device *hdev, u32 pll_index, bool curr) { - struct armcp_packet pkt; + struct cpucp_packet pkt; long result; int rc; memset(&pkt, 0, sizeof(pkt)); if (curr) - pkt.ctl = cpu_to_le32(ARMCP_PACKET_FREQUENCY_CURR_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_FREQUENCY_CURR_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); else - pkt.ctl = cpu_to_le32(ARMCP_PACKET_FREQUENCY_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_FREQUENCY_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.pll_index = cpu_to_le32(pll_index); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), @@ -40,13 +40,13 @@ long hl_get_frequency(struct hl_device *hdev, u32 pll_index, bool curr) void hl_set_frequency(struct hl_device *hdev, u32 pll_index, u64 freq) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_FREQUENCY_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_FREQUENCY_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.pll_index = cpu_to_le32(pll_index); pkt.value = cpu_to_le64(freq); @@ -61,14 +61,14 @@ void hl_set_frequency(struct hl_device *hdev, u32 pll_index, u64 freq) u64 hl_get_max_power(struct hl_device *hdev) { - struct armcp_packet pkt; + struct cpucp_packet pkt; long result; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_MAX_POWER_GET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_MAX_POWER_GET << + CPUCP_PKT_CTL_OPCODE_SHIFT); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), 0, &result); @@ -83,13 +83,13 @@ u64 hl_get_max_power(struct hl_device *hdev) void hl_set_max_power(struct hl_device *hdev) { - struct armcp_packet pkt; + struct cpucp_packet pkt; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_MAX_POWER_SET << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_MAX_POWER_SET << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.value = cpu_to_le64(hdev->max_power); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), @@ -112,7 +112,7 @@ static ssize_t armcp_kernel_ver_show(struct device *dev, { struct hl_device *hdev = dev_get_drvdata(dev); - return sprintf(buf, "%s", hdev->asic_prop.armcp_info.kernel_version); + return sprintf(buf, "%s", hdev->asic_prop.cpucp_info.kernel_version); } static ssize_t armcp_ver_show(struct device *dev, struct device_attribute *attr, @@ -120,7 +120,7 @@ static ssize_t armcp_ver_show(struct device *dev, struct device_attribute *attr, { struct hl_device *hdev = dev_get_drvdata(dev); - return sprintf(buf, "%s\n", hdev->asic_prop.armcp_info.armcp_version); + return sprintf(buf, "%s\n", hdev->asic_prop.cpucp_info.cpucp_version); } static ssize_t cpld_ver_show(struct device *dev, struct device_attribute *attr, @@ -129,7 +129,23 @@ static ssize_t cpld_ver_show(struct device *dev, struct device_attribute *attr, struct hl_device *hdev = dev_get_drvdata(dev); return sprintf(buf, "0x%08x\n", - hdev->asic_prop.armcp_info.cpld_version); + hdev->asic_prop.cpucp_info.cpld_version); +} + +static ssize_t cpucp_kernel_ver_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct hl_device *hdev = dev_get_drvdata(dev); + + return sprintf(buf, "%s", hdev->asic_prop.cpucp_info.kernel_version); +} + +static ssize_t cpucp_ver_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct hl_device *hdev = dev_get_drvdata(dev); + + return sprintf(buf, "%s\n", hdev->asic_prop.cpucp_info.cpucp_version); } static ssize_t infineon_ver_show(struct device *dev, @@ -138,7 +154,7 @@ static ssize_t infineon_ver_show(struct device *dev, struct hl_device *hdev = dev_get_drvdata(dev); return sprintf(buf, "0x%04x\n", - hdev->asic_prop.armcp_info.infineon_version); + hdev->asic_prop.cpucp_info.infineon_version); } static ssize_t fuse_ver_show(struct device *dev, struct device_attribute *attr, @@ -146,7 +162,7 @@ static ssize_t fuse_ver_show(struct device *dev, struct device_attribute *attr, { struct hl_device *hdev = dev_get_drvdata(dev); - return sprintf(buf, "%s\n", hdev->asic_prop.armcp_info.fuse_version); + return sprintf(buf, "%s\n", hdev->asic_prop.cpucp_info.fuse_version); } static ssize_t thermal_ver_show(struct device *dev, @@ -154,7 +170,7 @@ static ssize_t thermal_ver_show(struct device *dev, { struct hl_device *hdev = dev_get_drvdata(dev); - return sprintf(buf, "%s", hdev->asic_prop.armcp_info.thermal_version); + return sprintf(buf, "%s", hdev->asic_prop.cpucp_info.thermal_version); } static ssize_t preboot_btl_ver_show(struct device *dev, @@ -356,6 +372,8 @@ out: static DEVICE_ATTR_RO(armcp_kernel_ver); static DEVICE_ATTR_RO(armcp_ver); static DEVICE_ATTR_RO(cpld_ver); +static DEVICE_ATTR_RO(cpucp_kernel_ver); +static DEVICE_ATTR_RO(cpucp_ver); static DEVICE_ATTR_RO(device_type); static DEVICE_ATTR_RO(fuse_ver); static DEVICE_ATTR_WO(hard_reset); @@ -380,6 +398,8 @@ static struct attribute *hl_dev_attrs[] = { &dev_attr_armcp_kernel_ver.attr, &dev_attr_armcp_ver.attr, &dev_attr_cpld_ver.attr, + &dev_attr_cpucp_kernel_ver.attr, + &dev_attr_cpucp_ver.attr, &dev_attr_device_type.attr, &dev_attr_fuse_ver.attr, &dev_attr_hard_reset.attr, diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/misc/habanalabs/gaudi/gaudi.c index 4009b7df4caf..5f65a1691551 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/misc/habanalabs/gaudi/gaudi.c @@ -21,7 +21,6 @@ #include <linux/io-64-nonatomic-lo-hi.h> #include <linux/iommu.h> #include <linux/seq_file.h> -#include <linux/bitfield.h> /* * Gaudi security scheme: @@ -360,13 +359,14 @@ static int gaudi_memset_device_memory(struct hl_device *hdev, u64 addr, static int gaudi_run_tpc_kernel(struct hl_device *hdev, u64 tpc_kernel, u32 tpc_id); static int gaudi_mmu_clear_pgt_range(struct hl_device *hdev); -static int gaudi_armcp_info_get(struct hl_device *hdev); +static int gaudi_cpucp_info_get(struct hl_device *hdev); static void gaudi_disable_clock_gating(struct hl_device *hdev); static void gaudi_mmu_prepare(struct hl_device *hdev, u32 asid); static int gaudi_get_fixed_properties(struct hl_device *hdev) { struct asic_fixed_properties *prop = &hdev->asic_prop; + u32 num_sync_stream_queues = 0; int i; prop->max_queues = GAUDI_QUEUE_ID_SIZE; @@ -383,6 +383,7 @@ static int gaudi_get_fixed_properties(struct hl_device *hdev) prop->hw_queues_props[i].driver_only = 0; prop->hw_queues_props[i].requires_kernel_cb = 1; prop->hw_queues_props[i].supports_sync_stream = 1; + num_sync_stream_queues++; } else if (gaudi_queue_type[i] == QUEUE_TYPE_CPU) { prop->hw_queues_props[i].type = QUEUE_TYPE_CPU; prop->hw_queues_props[i].driver_only = 1; @@ -440,6 +441,7 @@ static int gaudi_get_fixed_properties(struct hl_device *hdev) prop->pmmu.end_addr = (VA_HOST_SPACE_START + VA_HOST_SPACE_SIZE / 2) - 1; prop->pmmu.page_size = PAGE_SIZE_4KB; + prop->pmmu.num_hops = MMU_ARCH_5_HOPS; /* PMMU and HPMMU are the same except of page size */ memcpy(&prop->pmmu_huge, &prop->pmmu, sizeof(prop->pmmu)); @@ -464,11 +466,16 @@ static int gaudi_get_fixed_properties(struct hl_device *hdev) prop->pcie_dbi_base_address = mmPCIE_DBI_BASE; prop->pcie_aux_dbi_reg_addr = CFG_BASE + mmPCIE_AUX_DBI; - strncpy(prop->armcp_info.card_name, GAUDI_DEFAULT_CARD_NAME, + strncpy(prop->cpucp_info.card_name, GAUDI_DEFAULT_CARD_NAME, CARD_NAME_MAX_LEN); prop->max_pending_cs = GAUDI_MAX_PENDING_CS; + prop->first_available_user_sob[HL_GAUDI_WS_DCORE] = + num_sync_stream_queues * HL_RSVD_SOBS; + prop->first_available_user_mon[HL_GAUDI_WS_DCORE] = + num_sync_stream_queues * HL_RSVD_MONS; + return 0; } @@ -592,10 +599,15 @@ static int gaudi_early_init(struct hl_device *hdev) prop->dram_pci_bar_size = pci_resource_len(pdev, HBM_BAR_ID); - rc = hl_pci_init(hdev); + rc = hl_pci_init(hdev, mmPSOC_GLOBAL_CONF_CPU_BOOT_STATUS, + mmCPU_BOOT_ERR0, GAUDI_BOOT_FIT_REQ_TIMEOUT_USEC); if (rc) goto free_queue_props; + /* GAUDI Firmware does not yet support security */ + prop->fw_security_disabled = true; + dev_info(hdev->dev, "firmware-level security is disabled\n"); + return 0; free_queue_props: @@ -675,10 +687,10 @@ static int _gaudi_init_tpc_mem(struct hl_device *hdev, init_tpc_mem_pkt->tsize = cpu_to_le32(tpc_kernel_size); - ctl = ((PACKET_LIN_DMA << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_LIN_DMA_CTL_LIN_SHIFT) | - (1 << GAUDI_PKT_CTL_RB_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT)); + ctl = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_LIN_DMA); + ctl |= FIELD_PREP(GAUDI_PKT_LIN_DMA_CTL_LIN_MASK, 1); + ctl |= FIELD_PREP(GAUDI_PKT_CTL_RB_MASK, 1); + ctl |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); init_tpc_mem_pkt->ctl = cpu_to_le32(ctl); @@ -780,13 +792,13 @@ static int gaudi_late_init(struct hl_device *hdev) struct gaudi_device *gaudi = hdev->asic_specific; int rc; - rc = gaudi->armcp_info_get(hdev); + rc = gaudi->cpucp_info_get(hdev); if (rc) { - dev_err(hdev->dev, "Failed to get armcp info\n"); + dev_err(hdev->dev, "Failed to get cpucp info\n"); return rc; } - rc = hl_fw_send_pci_access_msg(hdev, ARMCP_PACKET_ENABLE_PCI_ACCESS); + rc = hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_ENABLE_PCI_ACCESS); if (rc) { dev_err(hdev->dev, "Failed to enable PCI access from CPU\n"); return rc; @@ -811,7 +823,7 @@ static int gaudi_late_init(struct hl_device *hdev) return 0; disable_pci_access: - hl_fw_send_pci_access_msg(hdev, ARMCP_PACKET_DISABLE_PCI_ACCESS); + hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_DISABLE_PCI_ACCESS); return rc; } @@ -981,7 +993,7 @@ static int gaudi_sw_init(struct hl_device *hdev) } } - gaudi->armcp_info_get = gaudi_armcp_info_get; + gaudi->cpucp_info_get = gaudi_cpucp_info_get; gaudi->max_freq_value = GAUDI_MAX_CLK_FREQ; @@ -1853,9 +1865,11 @@ static void gaudi_init_pci_dma_qman(struct hl_device *hdev, int dma_id, WREG32(mmDMA0_QM_PQ_PI_0 + q_off, 0); WREG32(mmDMA0_QM_PQ_CI_0 + q_off, 0); - WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x74); - WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x14); - WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, QMAN_LDMA_SIZE_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_SRC_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_DST_OFFSET); WREG32(mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_0 + q_off, mtr_base_en_lo); WREG32(mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_0 + q_off, mtr_base_en_hi); @@ -1911,6 +1925,9 @@ static void gaudi_init_dma_core(struct hl_device *hdev, int dma_id) WREG32(mmDMA0_CORE_RD_MAX_OUTSTAND + dma_offset, 0); WREG32(mmDMA0_CORE_RD_MAX_SIZE + dma_offset, 0); + /* WA for H/W bug H3-2116 */ + WREG32(mmDMA0_CORE_LBW_MAX_OUTSTAND + dma_offset, 15); + /* STOP_ON bit implies no completion to operation in case of RAZWI */ if (hdev->stop_on_err) dma_err_cfg |= 1 << DMA0_CORE_ERR_CFG_STOP_ON_ERR_SHIFT; @@ -2010,13 +2027,19 @@ static void gaudi_init_hbm_dma_qman(struct hl_device *hdev, int dma_id, WREG32(mmDMA0_QM_PQ_PI_0 + q_off, 0); WREG32(mmDMA0_QM_PQ_CI_0 + q_off, 0); - WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x81BC); - WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x81B4); - WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_CPDMA_SIZE_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_SRC_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_DST_OFFSET); } else { - WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x74); - WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x14); - WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_LDMA_SIZE_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_SRC_OFFSET); + WREG32(mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_DST_OFFSET); /* Configure RAZWI IRQ */ dma_qm_err_cfg = HBM_DMA_QMAN_GLBL_ERR_CFG_MSG_EN_MASK; @@ -2120,13 +2143,19 @@ static void gaudi_init_mme_qman(struct hl_device *hdev, u32 mme_offset, WREG32(mmMME0_QM_PQ_PI_0 + q_off, 0); WREG32(mmMME0_QM_PQ_CI_0 + q_off, 0); - WREG32(mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x81BC); - WREG32(mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x81B4); - WREG32(mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_CPDMA_SIZE_OFFSET); + WREG32(mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_SRC_OFFSET); + WREG32(mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_DST_OFFSET); } else { - WREG32(mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x74); - WREG32(mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x14); - WREG32(mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_LDMA_SIZE_OFFSET); + WREG32(mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_SRC_OFFSET); + WREG32(mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_DST_OFFSET); /* Configure RAZWI IRQ */ mme_id = mme_offset / @@ -2234,13 +2263,19 @@ static void gaudi_init_tpc_qman(struct hl_device *hdev, u32 tpc_offset, WREG32(mmTPC0_QM_PQ_PI_0 + q_off, 0); WREG32(mmTPC0_QM_PQ_CI_0 + q_off, 0); - WREG32(mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x81BC); - WREG32(mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x81B4); - WREG32(mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_CPDMA_SIZE_OFFSET); + WREG32(mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_SRC_OFFSET); + WREG32(mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_CPDMA_DST_OFFSET); } else { - WREG32(mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x74); - WREG32(mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x14); - WREG32(mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + WREG32(mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 + q_off, + QMAN_LDMA_SIZE_OFFSET); + WREG32(mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_SRC_OFFSET); + WREG32(mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, + QMAN_LDMA_DST_OFFSET); /* Configure RAZWI IRQ */ tpc_id = tpc_offset / @@ -2321,7 +2356,8 @@ static void gaudi_init_tpc_qmans(struct hl_device *hdev) tpc_offset += mmTPC1_QM_GLBL_CFG0 - mmTPC0_QM_GLBL_CFG0; - gaudi->hw_cap_initialized |= 1 << (HW_CAP_TPC_SHIFT + tpc_id); + gaudi->hw_cap_initialized |= + FIELD_PREP(HW_CAP_TPC_MASK, 1 << tpc_id); } } @@ -2847,7 +2883,7 @@ static int gaudi_init_cpu_queues(struct hl_device *hdev, u32 cpu_timeout) if (err) { dev_err(hdev->dev, - "Failed to communicate with ARM CPU (ArmCP timeout)\n"); + "Failed to communicate with Device CPU (CPU-CP timeout)\n"); return -EIO; } @@ -2860,6 +2896,18 @@ static void gaudi_pre_hw_init(struct hl_device *hdev) /* Perform read from the device to make sure device is up */ RREG32(mmPCIE_DBI_DEVICE_ID_VENDOR_ID_REG); + /* Set the access through PCI bars (Linux driver only) as + * secured + */ + WREG32(mmPCIE_WRAP_LBW_PROT_OVR, + (PCIE_WRAP_LBW_PROT_OVR_RD_EN_MASK | + PCIE_WRAP_LBW_PROT_OVR_WR_EN_MASK)); + + /* Perform read to flush the waiting writes to ensure + * configuration was set in the device + */ + RREG32(mmPCIE_WRAP_LBW_PROT_OVR); + /* * Let's mark in the H/W that we have reached this point. We check * this value in the reset_before_init function to understand whether @@ -2868,31 +2916,6 @@ static void gaudi_pre_hw_init(struct hl_device *hdev) */ WREG32(mmHW_STATE, HL_DEVICE_HW_STATE_DIRTY); - /* Set the access through PCI bars (Linux driver only) as secured */ - WREG32(mmPCIE_WRAP_LBW_PROT_OVR, (PCIE_WRAP_LBW_PROT_OVR_RD_EN_MASK | - PCIE_WRAP_LBW_PROT_OVR_WR_EN_MASK)); - - /* Perform read to flush the waiting writes to ensure configuration - * was set in the device - */ - RREG32(mmPCIE_WRAP_LBW_PROT_OVR); - - if (hdev->axi_drain) { - WREG32(mmPCIE_WRAP_LBW_DRAIN_CFG, - 1 << PCIE_WRAP_LBW_DRAIN_CFG_EN_SHIFT); - WREG32(mmPCIE_WRAP_HBW_DRAIN_CFG, - 1 << PCIE_WRAP_HBW_DRAIN_CFG_EN_SHIFT); - - /* Perform read to flush the DRAIN cfg */ - RREG32(mmPCIE_WRAP_HBW_DRAIN_CFG); - } else { - WREG32(mmPCIE_WRAP_LBW_DRAIN_CFG, 0); - WREG32(mmPCIE_WRAP_HBW_DRAIN_CFG, 0); - - /* Perform read to flush the DRAIN cfg */ - RREG32(mmPCIE_WRAP_HBW_DRAIN_CFG); - } - /* Configure the reset registers. Must be done as early as possible * in case we fail during H/W initialization */ @@ -2900,13 +2923,13 @@ static void gaudi_pre_hw_init(struct hl_device *hdev) (CFG_RST_H_DMA_MASK | CFG_RST_H_MME_MASK | CFG_RST_H_SM_MASK | - CFG_RST_H_TPC_MASK)); + CFG_RST_H_TPC_7_MASK)); WREG32(mmPSOC_GLOBAL_CONF_SOFT_RST_CFG_L, CFG_RST_L_TPC_MASK); WREG32(mmPSOC_GLOBAL_CONF_SW_ALL_RST_CFG_H, (CFG_RST_H_HBM_MASK | - CFG_RST_H_TPC_MASK | + CFG_RST_H_TPC_7_MASK | CFG_RST_H_NIC_MASK | CFG_RST_H_SM_MASK | CFG_RST_H_DMA_MASK | @@ -3071,7 +3094,7 @@ static int gaudi_suspend(struct hl_device *hdev) { int rc; - rc = hl_fw_send_pci_access_msg(hdev, ARMCP_PACKET_DISABLE_PCI_ACCESS); + rc = hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_DISABLE_PCI_ACCESS); if (rc) dev_err(hdev->dev, "Failed to disable PCI access from CPU\n"); @@ -3084,17 +3107,16 @@ static int gaudi_resume(struct hl_device *hdev) } static int gaudi_cb_mmap(struct hl_device *hdev, struct vm_area_struct *vma, - u64 kaddress, phys_addr_t paddress, u32 size) + void *cpu_addr, dma_addr_t dma_addr, size_t size) { int rc; vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_DONTCOPY | VM_NORESERVE; - rc = remap_pfn_range(vma, vma->vm_start, paddress >> PAGE_SHIFT, - size, vma->vm_page_prot); + rc = dma_mmap_coherent(hdev->dev, vma, cpu_addr, dma_addr, size); if (rc) - dev_err(hdev->dev, "remap_pfn_range error %d", rc); + dev_err(hdev->dev, "dma_mmap_coherent error %d", rc); return rc; } @@ -3441,7 +3463,8 @@ static int gaudi_test_queue(struct hl_device *hdev, u32 hw_queue_id) &fence_dma_addr); if (!fence_ptr) { dev_err(hdev->dev, - "Failed to allocate memory for queue testing\n"); + "Failed to allocate memory for H/W queue %d testing\n", + hw_queue_id); return -ENOMEM; } @@ -3452,14 +3475,16 @@ static int gaudi_test_queue(struct hl_device *hdev, u32 hw_queue_id) GFP_KERNEL, &pkt_dma_addr); if (!fence_pkt) { dev_err(hdev->dev, - "Failed to allocate packet for queue testing\n"); + "Failed to allocate packet for H/W queue %d testing\n", + hw_queue_id); rc = -ENOMEM; goto free_fence_ptr; } - tmp = (PACKET_MSG_PROT << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_CTL_EB_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT); + tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_EB_MASK, 1); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); + fence_pkt->ctl = cpu_to_le32(tmp); fence_pkt->value = cpu_to_le32(fence_val); fence_pkt->addr = cpu_to_le64(fence_dma_addr); @@ -3469,7 +3494,8 @@ static int gaudi_test_queue(struct hl_device *hdev, u32 hw_queue_id) pkt_dma_addr); if (rc) { dev_err(hdev->dev, - "Failed to send fence packet\n"); + "Failed to send fence packet to H/W queue %d\n", + hw_queue_id); goto free_pkt; } @@ -3959,8 +3985,6 @@ static int gaudi_patch_dma_packet(struct hl_device *hdev, } } - new_dma_pkt->ctl = user_dma_pkt->ctl; - ctl = le32_to_cpu(user_dma_pkt->ctl); if (likely(dma_desc_cnt)) ctl &= ~GAUDI_PKT_CTL_EB_MASK; @@ -4105,8 +4129,9 @@ static int gaudi_parse_cb_mmu(struct hl_device *hdev, parser->patched_cb_size = parser->user_cb_size + sizeof(struct packet_msg_prot) * 2; - rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, parser->patched_cb_size, - &patched_cb_handle, HL_KERNEL_ASID_ID, false); + rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, hdev->kernel_ctx, + parser->patched_cb_size, false, false, + &patched_cb_handle); if (rc) { dev_err(hdev->dev, @@ -4178,8 +4203,9 @@ static int gaudi_parse_cb_no_mmu(struct hl_device *hdev, if (rc) goto free_userptr; - rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, parser->patched_cb_size, - &patched_cb_handle, HL_KERNEL_ASID_ID, false); + rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, hdev->kernel_ctx, + parser->patched_cb_size, false, false, + &patched_cb_handle); if (rc) { dev_err(hdev->dev, "Failed to allocate patched CB for DMA CS %d\n", rc); @@ -4275,11 +4301,11 @@ static void gaudi_add_end_of_cb_packets(struct hl_device *hdev, cq_pkt = (struct packet_msg_prot *) (uintptr_t) (kernel_address + len - (sizeof(struct packet_msg_prot) * 2)); - tmp = (PACKET_MSG_PROT << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT); + tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); if (eb) - tmp |= (1 << GAUDI_PKT_CTL_EB_SHIFT); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_EB_MASK, 1); cq_pkt->ctl = cpu_to_le32(tmp); cq_pkt->value = cpu_to_le32(cq_val); @@ -4287,8 +4313,8 @@ static void gaudi_add_end_of_cb_packets(struct hl_device *hdev, cq_pkt++; - tmp = (PACKET_MSG_PROT << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT); + tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); cq_pkt->ctl = cpu_to_le32(tmp); cq_pkt->value = cpu_to_le32(1); @@ -4320,11 +4346,12 @@ static int gaudi_memset_device_memory(struct hl_device *hdev, u64 addr, memset(lin_dma_pkt, 0, sizeof(*lin_dma_pkt)); cb_size = sizeof(*lin_dma_pkt); - ctl = ((PACKET_LIN_DMA << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_LIN_DMA_CTL_MEMSET_SHIFT) | - (1 << GAUDI_PKT_LIN_DMA_CTL_LIN_SHIFT) | - (1 << GAUDI_PKT_CTL_RB_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT)); + ctl = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_LIN_DMA); + ctl |= FIELD_PREP(GAUDI_PKT_LIN_DMA_CTL_MEMSET_MASK, 1); + ctl |= FIELD_PREP(GAUDI_PKT_LIN_DMA_CTL_LIN_MASK, 1); + ctl |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); + ctl |= FIELD_PREP(GAUDI_PKT_CTL_RB_MASK, 1); + lin_dma_pkt->ctl = cpu_to_le32(ctl); lin_dma_pkt->src_addr = cpu_to_le64(val); lin_dma_pkt->dst_addr |= cpu_to_le64(addr); @@ -4930,9 +4957,10 @@ static int gaudi_send_job_on_qman0(struct hl_device *hdev, fence_pkt = (struct packet_msg_prot *) (uintptr_t) (cb->kernel_address + job->job_cb_size - sizeof(struct packet_msg_prot)); - tmp = (PACKET_MSG_PROT << GAUDI_PKT_CTL_OPCODE_SHIFT) | - (1 << GAUDI_PKT_CTL_EB_SHIFT) | - (1 << GAUDI_PKT_CTL_MB_SHIFT); + tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_EB_MASK, 1); + tmp |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1); + fence_pkt->ctl = cpu_to_le32(tmp); fence_pkt->value = cpu_to_le32(GAUDI_QMAN0_FENCE_VAL); fence_pkt->addr = cpu_to_le64(fence_dma_addr); @@ -5606,7 +5634,7 @@ static bool gaudi_tpc_read_interrupts(struct hl_device *hdev, u8 tpc_id, bool soft_reset_required = false; /* Accessing the TPC_INTR_CAUSE registers requires disabling the clock - * gating, and thus cannot be done in ArmCP and should be done instead + * gating, and thus cannot be done in CPU-CP and should be done instead * by the driver. */ @@ -5653,21 +5681,25 @@ static void gaudi_print_clk_change_info(struct hl_device *hdev, { switch (event_type) { case GAUDI_EVENT_FIX_POWER_ENV_S: + hdev->clk_throttling_reason |= HL_CLK_THROTTLE_POWER; dev_info_ratelimited(hdev->dev, "Clock throttling due to power consumption\n"); break; case GAUDI_EVENT_FIX_POWER_ENV_E: + hdev->clk_throttling_reason &= ~HL_CLK_THROTTLE_POWER; dev_info_ratelimited(hdev->dev, "Power envelop is safe, back to optimal clock\n"); break; case GAUDI_EVENT_FIX_THERMAL_ENV_S: + hdev->clk_throttling_reason |= HL_CLK_THROTTLE_THERMAL; dev_info_ratelimited(hdev->dev, "Clock throttling due to overheating\n"); break; case GAUDI_EVENT_FIX_THERMAL_ENV_E: + hdev->clk_throttling_reason &= ~HL_CLK_THROTTLE_THERMAL; dev_info_ratelimited(hdev->dev, "Thermal envelop is safe, back to optimal clock\n"); break; @@ -6038,7 +6070,7 @@ static int gaudi_send_heartbeat(struct hl_device *hdev) return hl_fw_send_heartbeat(hdev); } -static int gaudi_armcp_info_get(struct hl_device *hdev) +static int gaudi_cpucp_info_get(struct hl_device *hdev) { struct gaudi_device *gaudi = hdev->asic_specific; struct asic_fixed_properties *prop = &hdev->asic_prop; @@ -6047,19 +6079,19 @@ static int gaudi_armcp_info_get(struct hl_device *hdev) if (!(gaudi->hw_cap_initialized & HW_CAP_CPU_Q)) return 0; - rc = hl_fw_armcp_info_get(hdev); + rc = hl_fw_cpucp_info_get(hdev); if (rc) return rc; - if (!strlen(prop->armcp_info.card_name)) - strncpy(prop->armcp_info.card_name, GAUDI_DEFAULT_CARD_NAME, + if (!strlen(prop->cpucp_info.card_name)) + strncpy(prop->cpucp_info.card_name, GAUDI_DEFAULT_CARD_NAME, CARD_NAME_MAX_LEN); - hdev->card_type = le32_to_cpu(hdev->asic_prop.armcp_info.card_type); + hdev->card_type = le32_to_cpu(hdev->asic_prop.cpucp_info.card_type); - if (hdev->card_type == armcp_card_type_pci) + if (hdev->card_type == cpucp_card_type_pci) prop->max_power_default = MAX_POWER_DEFAULT_PCI; - else if (hdev->card_type == armcp_card_type_pmc) + else if (hdev->card_type == cpucp_card_type_pmc) prop->max_power_default = MAX_POWER_DEFAULT_PMC; hdev->max_power = prop->max_power_default; @@ -6067,7 +6099,7 @@ static int gaudi_armcp_info_get(struct hl_device *hdev) return 0; } -static bool gaudi_is_device_idle(struct hl_device *hdev, u32 *mask, +static bool gaudi_is_device_idle(struct hl_device *hdev, u64 *mask, struct seq_file *s) { struct gaudi_device *gaudi = hdev->asic_specific; @@ -6099,7 +6131,7 @@ static bool gaudi_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << + *mask |= ((u64) !is_eng_idle) << (GAUDI_ENGINE_ID_DMA_0 + dma_id); if (s) seq_printf(s, fmt, dma_id, @@ -6122,7 +6154,8 @@ static bool gaudi_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << (GAUDI_ENGINE_ID_TPC_0 + i); + *mask |= ((u64) !is_eng_idle) << + (GAUDI_ENGINE_ID_TPC_0 + i); if (s) seq_printf(s, fmt, i, is_eng_idle ? "Y" : "N", @@ -6150,7 +6183,8 @@ static bool gaudi_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << (GAUDI_ENGINE_ID_MME_0 + i); + *mask |= ((u64) !is_eng_idle) << + (GAUDI_ENGINE_ID_MME_0 + i); if (s) { if (!is_slave) seq_printf(s, fmt, i, @@ -6288,6 +6322,15 @@ static int gaudi_run_tpc_kernel(struct hl_device *hdev, u64 tpc_kernel, 1000, kernel_timeout); + if (rc) { + dev_err(hdev->dev, + "Timeout while waiting for TPC%d vector pipe\n", + tpc_id); + hdev->asic_funcs->set_clock_gating(hdev); + mutex_unlock(&gaudi->clk_gate_mutex); + return -EIO; + } + rc = hl_poll_timeout( hdev, mmTPC0_CFG_WQ_INFLIGHT_CNTR + offset, @@ -6617,7 +6660,6 @@ static const struct hl_asic_funcs gaudi_funcs = { .send_cpu_message = gaudi_send_cpu_message, .get_hw_state = gaudi_get_hw_state, .pci_bars_map = gaudi_pci_bars_map, - .set_dram_bar_base = gaudi_set_hbm_bar_base, .init_iatu = gaudi_init_iatu, .rreg = hl_rreg, .wreg = hl_wreg, diff --git a/drivers/misc/habanalabs/gaudi/gaudiP.h b/drivers/misc/habanalabs/gaudi/gaudiP.h index 82137c3f3e2e..83ad2b0a3a61 100644 --- a/drivers/misc/habanalabs/gaudi/gaudiP.h +++ b/drivers/misc/habanalabs/gaudi/gaudiP.h @@ -35,8 +35,6 @@ #error "Number of MSI interrupts must be smaller or equal to GAUDI_MSI_ENTRIES" #endif -#define QMAN_FENCE_TIMEOUT_USEC 10000 /* 10 ms */ - #define CORESIGHT_TIMEOUT_USEC 100000 /* 100 ms */ #define GAUDI_MAX_CLK_FREQ 2200000000ull /* 2200 MHz */ @@ -44,7 +42,7 @@ #define MAX_POWER_DEFAULT_PCI 200000 /* 200W */ #define MAX_POWER_DEFAULT_PMC 350000 /* 350W */ -#define GAUDI_CPU_TIMEOUT_USEC 15000000 /* 15s */ +#define GAUDI_CPU_TIMEOUT_USEC 30000000 /* 30s */ #define TPC_ENABLED_MASK 0xFF @@ -86,6 +84,14 @@ #define DMA_CORE_OFFSET (mmDMA1_CORE_BASE - mmDMA0_CORE_BASE) +#define QMAN_LDMA_SRC_OFFSET (mmDMA0_CORE_SRC_BASE_LO - mmDMA0_CORE_CFG_0) +#define QMAN_LDMA_DST_OFFSET (mmDMA0_CORE_DST_BASE_LO - mmDMA0_CORE_CFG_0) +#define QMAN_LDMA_SIZE_OFFSET (mmDMA0_CORE_DST_TSIZE_0 - mmDMA0_CORE_CFG_0) + +#define QMAN_CPDMA_SRC_OFFSET (mmDMA0_QM_CQ_PTR_LO_4 - mmDMA0_CORE_CFG_0) +#define QMAN_CPDMA_DST_OFFSET (mmDMA0_CORE_DST_BASE_LO - mmDMA0_CORE_CFG_0) +#define QMAN_CPDMA_SIZE_OFFSET (mmDMA0_QM_CQ_TSIZE_4 - mmDMA0_CORE_CFG_0) + #define SIF_RTR_CTRL_OFFSET (mmSIF_RTR_CTRL_1_BASE - mmSIF_RTR_CTRL_0_BASE) #define NIF_RTR_CTRL_OFFSET (mmNIF_RTR_CTRL_1_BASE - mmNIF_RTR_CTRL_0_BASE) @@ -142,28 +148,28 @@ #define VA_HOST_SPACE_SIZE (VA_HOST_SPACE_END - \ VA_HOST_SPACE_START) /* 767TB */ -#define HW_CAP_PLL 0x00000001 -#define HW_CAP_HBM 0x00000002 -#define HW_CAP_MMU 0x00000004 -#define HW_CAP_MME 0x00000008 -#define HW_CAP_CPU 0x00000010 -#define HW_CAP_PCI_DMA 0x00000020 -#define HW_CAP_MSI 0x00000040 -#define HW_CAP_CPU_Q 0x00000080 -#define HW_CAP_HBM_DMA 0x00000100 -#define HW_CAP_CLK_GATE 0x00000200 -#define HW_CAP_SRAM_SCRAMBLER 0x00000400 -#define HW_CAP_HBM_SCRAMBLER 0x00000800 - -#define HW_CAP_TPC0 0x01000000 -#define HW_CAP_TPC1 0x02000000 -#define HW_CAP_TPC2 0x04000000 -#define HW_CAP_TPC3 0x08000000 -#define HW_CAP_TPC4 0x10000000 -#define HW_CAP_TPC5 0x20000000 -#define HW_CAP_TPC6 0x40000000 -#define HW_CAP_TPC7 0x80000000 -#define HW_CAP_TPC_MASK 0xFF000000 +#define HW_CAP_PLL BIT(0) +#define HW_CAP_HBM BIT(1) +#define HW_CAP_MMU BIT(2) +#define HW_CAP_MME BIT(3) +#define HW_CAP_CPU BIT(4) +#define HW_CAP_PCI_DMA BIT(5) +#define HW_CAP_MSI BIT(6) +#define HW_CAP_CPU_Q BIT(7) +#define HW_CAP_HBM_DMA BIT(8) +#define HW_CAP_CLK_GATE BIT(9) +#define HW_CAP_SRAM_SCRAMBLER BIT(10) +#define HW_CAP_HBM_SCRAMBLER BIT(11) + +#define HW_CAP_TPC0 BIT(24) +#define HW_CAP_TPC1 BIT(25) +#define HW_CAP_TPC2 BIT(26) +#define HW_CAP_TPC3 BIT(27) +#define HW_CAP_TPC4 BIT(28) +#define HW_CAP_TPC5 BIT(29) +#define HW_CAP_TPC6 BIT(30) +#define HW_CAP_TPC7 BIT(31) +#define HW_CAP_TPC_MASK GENMASK(31, 24) #define HW_CAP_TPC_SHIFT 24 #define GAUDI_CPU_PCI_MSB_ADDR(addr) (((addr) & GENMASK_ULL(49, 39)) >> 39) @@ -216,7 +222,7 @@ struct gaudi_internal_qman_info { /** * struct gaudi_device - ASIC specific manage structure. - * @armcp_info_get: get information on device from ArmCP + * @cpucp_info_get: get information on device from CPU-CP * @hw_queues_lock: protects the H/W queues from concurrent access. * @clk_gate_mutex: protects code areas that require clock gating to be disabled * temporarily @@ -239,7 +245,7 @@ struct gaudi_internal_qman_info { * 8-bit value so use u8. */ struct gaudi_device { - int (*armcp_info_get)(struct hl_device *hdev); + int (*cpucp_info_get)(struct hl_device *hdev); /* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock; diff --git a/drivers/misc/habanalabs/gaudi/gaudi_security.c b/drivers/misc/habanalabs/gaudi/gaudi_security.c index 8d5d6ddee6ed..2d7add0e5bcc 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_security.c +++ b/drivers/misc/habanalabs/gaudi/gaudi_security.c @@ -487,241 +487,241 @@ static void gaudi_init_mme_protection_bits(struct hl_device *hdev) pb_addr = (mmMME0_CTRL_RESET & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_CTRL_RESET & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_CTRL_RESET & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_QM_STALL & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_LOG_SHADOW & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_MIN & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); - mask |= 1 << ((mmMME0_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); + mask = 1U << ((mmMME0_CTRL_RESET & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_QM_STALL & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_LOG_SHADOW & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_MIN & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); + mask |= 1U << ((mmMME0_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_CTRL_SHADOW_0_STATUS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_CTRL_SHADOW_0_STATUS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); + mask = 1U << ((mmMME0_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -729,236 +729,235 @@ static void gaudi_init_mme_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_ARB_MST_CHOISE_PUSH_OFST_23 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmMME0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmMME0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME0_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME0_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmMME0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME1_CTRL_RESET & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME1_CTRL_RESET & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME1_CTRL_RESET & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_QM_STALL & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_LOG_SHADOW & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_MIN & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); - mask |= 1 << ((mmMME1_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); + mask = 1U << ((mmMME1_CTRL_RESET & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_QM_STALL & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_LOG_SHADOW & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_MIN & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); + mask |= 1U << ((mmMME1_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME1_CTRL_SHADOW_0_STATUS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME1_CTRL_SHADOW_0_STATUS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME1_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); + mask = 1U << ((mmMME1_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -966,241 +965,241 @@ static void gaudi_init_mme_protection_bits(struct hl_device *hdev) pb_addr = (mmMME2_CTRL_RESET & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_CTRL_RESET & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_CTRL_RESET & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_QM_STALL & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_LOG_SHADOW & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_MIN & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); - mask |= 1 << ((mmMME2_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); + mask = 1U << ((mmMME2_CTRL_RESET & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_QM_STALL & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_LOG_SHADOW & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_MIN & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); + mask |= 1U << ((mmMME2_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_CTRL_SHADOW_0_STATUS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_CTRL_SHADOW_0_STATUS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); + mask = 1U << ((mmMME2_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -1208,102 +1207,102 @@ static void gaudi_init_mme_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -1311,134 +1310,133 @@ static void gaudi_init_mme_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmMME2_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmMME2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmMME2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME2_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME2_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmMME2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME3_CTRL_RESET & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME3_CTRL_RESET & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME3_CTRL_RESET & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_QM_STALL & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_LOG_SHADOW & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_TH & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_MIN & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); - mask |= 1 << ((mmMME3_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); + mask = 1U << ((mmMME3_CTRL_RESET & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_QM_STALL & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_SYNC_OBJECT_FIFO_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_EUS_ROLLUP_CNT_ADD & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_LOG_SHADOW & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_DESC0 & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_TOKEN_UPDATE & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_TH & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_MIN & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_CTRL_EN & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_HISTORY_LOG_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_A_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_B_BF16 & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_A_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_A_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_B_FP32_ODD & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_DUMMY_B_FP32_EVEN & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_EU_POWER_SAVE_DISABLE & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_CS_DBG_BLOCK_ID & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_CS_DBG_STATUS_DROP_CNT & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_TE_CLOSE_CGATE & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_AGU_SM_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_AGU_SM_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_EZSYNC_OUT_CREDIT & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_PCU_RL_SAT_SEC & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_AGU_SYNC_MSG_AXI_USER & 0x7F) >> 2); + mask |= 1U << ((mmMME3_CTRL_QM_SLV_LBW_CLK_EN & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmMME3_CTRL_SHADOW_0_STATUS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmMME3_CTRL_SHADOW_0_STATUS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmMME3_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); + mask = 1U << ((mmMME3_CTRL_SHADOW_0_STATUS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -1486,199 +1484,199 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) pb_addr = (mmDMA0_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -1687,102 +1685,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_ARB_MST_CHOISE_PUSH_OFST_23 & ~0xFFF) + @@ -1790,290 +1788,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA0_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2082,102 +2079,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2186,290 +2183,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA1_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA1_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2478,102 +2474,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2582,290 +2578,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA2_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2874,102 +2869,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -2978,290 +2973,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA3_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA3_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -3270,102 +3264,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -3374,290 +3368,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA4_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA4_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -3666,102 +3659,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -3770,290 +3763,289 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA5_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA5_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -4062,102 +4054,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -4166,290 +4158,290 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA6_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + + mask = 1U << ((mmDMA6_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA6_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -4458,102 +4450,102 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -4562,606 +4554,605 @@ static void gaudi_init_dma_protection_bits(struct hl_device *hdev) word_offset = ((mmDMA7_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmDMA7_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA0_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA0_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA0_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA0_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA0_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA0_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA0_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA0_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA0_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA0_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA1_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA1_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA1_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA1_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA1_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA1_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA1_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA1_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA1_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA1_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA2_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA2_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA2_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA2_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA2_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA2_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA2_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA2_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA2_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA2_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA3_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA3_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA3_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA3_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA3_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA3_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA3_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA3_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA3_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA3_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA4_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA4_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA4_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA4_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA4_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA4_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA4_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA4_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA4_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA4_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA5_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA5_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA5_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA5_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA5_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA5_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA5_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA5_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA5_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA5_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA6_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA6_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA6_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA6_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA6_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA6_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA6_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA6_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA6_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA6_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_CORE_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_CORE_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_CORE_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); + mask = 1U << ((mmDMA7_CORE_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_LBW_MAX_OUTSTAND & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_CORE_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_CORE_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_CORE_PROT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_SECURE_PROPS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_NON_SECURE_PROPS & 0x7F) >> 2); + mask = 1U << ((mmDMA7_CORE_PROT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_SECURE_PROPS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_NON_SECURE_PROPS & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_CORE_RD_MAX_OUTSTAND & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_CORE_RD_MAX_OUTSTAND & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_ARCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_MAX_AWID & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_AWCACHE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_INFLIGHTS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_ERRMSG_WDATA & 0x7F) >> 2); + mask = 1U << ((mmDMA7_CORE_RD_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_ARCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_ARUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_MAX_OUTSTAND & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_MAX_AWID & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_AWCACHE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_INFLIGHTS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_ERRMSG_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_ERRMSG_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_ERRMSG_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_CORE_STS0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_CORE_STS0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_CORE_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_STS1 & 0x7F) >> 2); + mask = 1U << ((mmDMA7_CORE_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_STS1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmDMA7_CORE_RD_DBGMEM_ADD & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmDMA7_CORE_RD_DBGMEM_ADD & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmDMA7_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_RD_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_DESC_CNT & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_STS & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); - mask |= 1 << ((mmDMA7_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); + mask = 1U << ((mmDMA7_CORE_RD_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_RD_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_HBW_AXI_AR_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_HBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_LBW_AXI_AW_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_DESC_CNT & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_STS & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_RD_DESC_ID & 0x7F) >> 2); + mask |= 1U << ((mmDMA7_CORE_DBG_WR_DESC_ID & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); } @@ -5185,199 +5176,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC0_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -5387,102 +5378,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -5491,150 +5482,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC0_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC0_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC0_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC0_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC0_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC0_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC0_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC0_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC0_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC0_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -5643,199 +5633,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC1_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -5843,102 +5833,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -5947,150 +5937,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC1_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC1_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC1_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC1_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC1_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC1_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC1_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC1_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC1_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC1_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6099,199 +6088,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC2_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6299,102 +6288,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6402,150 +6391,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC2_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC2_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC2_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC2_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC2_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC2_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC2_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC2_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC2_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6554,199 +6542,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC3_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6754,102 +6742,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -6857,150 +6845,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC3_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC3_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC3_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC3_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC3_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC3_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC3_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC3_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC3_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7009,199 +6996,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC4_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7209,102 +7196,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7312,150 +7299,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC4_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC4_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC4_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC4_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC4_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC4_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC4_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC4_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC4_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7464,199 +7450,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC5_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7664,102 +7650,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7767,150 +7753,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC5_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC5_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC5_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC5_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC5_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC5_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC5_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC5_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC5_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -7919,199 +7904,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC6_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8119,102 +8104,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8223,85 +8208,84 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC6_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8309,65 +8293,65 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC6_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC6_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC6_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC6_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC6_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC6_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC6_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC6_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC6_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC6_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8376,199 +8360,199 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) pb_addr = (mmTPC7_QM_GLBL_CFG0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_GLBL_CFG0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_GLBL_CFG0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_CFG1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_ERR_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_STS1_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_LO_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_GLBL_CFG0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_CFG1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_ERR_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_NON_SECURE_PROPS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_STS1_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_MSG_EN_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_MSG_EN_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_MSG_EN_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_MSG_EN_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_MSG_EN_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_LO_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_PQ_BASE_HI_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_PQ_BASE_HI_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_PQ_BASE_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_BASE_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_SIZE_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_SIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_SIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_SIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_PI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_PI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_PI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_PI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_CFG1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS0_3 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_PQ_BASE_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_BASE_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_SIZE_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_SIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_SIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_SIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_PI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_PI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_PI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_PI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_CFG1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS0_3 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_PQ_STS1_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_PQ_STS1_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_PQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_PQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS0_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS1_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS1_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS1_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_STS1_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_0 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_PQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_PQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS0_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS1_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS1_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS1_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_STS1_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_0 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CQ_CTL_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CQ_CTL_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CQ_CTL_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CQ_CTL_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_LO_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_PTR_HI_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_TSIZE_STS_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CQ_CTL_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CQ_CTL_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CQ_CTL_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_CTL_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CQ_CTL_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_CTL_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_IFIFO_CNT_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_IFIFO_CNT_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_IFIFO_CNT_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_IFIFO_CNT_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CQ_IFIFO_CNT_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE0_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE1_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE2_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_MSG_BASE3_ADDR_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_TSIZE_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_SRC_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8578,102 +8562,102 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) word_offset = ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_LDMA_DST_BASE_LO_OFFSET_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CP_STS_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CP_STS_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CP_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CP_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_LO_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_LO_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_LO_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_LO_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_LO_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_HI_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_HI_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_HI_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_HI_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_CURRENT_INST_HI_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_BARRIER_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_BARRIER_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_BARRIER_CFG_2 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CP_BARRIER_CFG_3 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CP_BARRIER_CFG_3 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_DBG_0_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_DBG_0_1 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CP_BARRIER_CFG_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_BARRIER_CFG_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_DBG_0_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_DBG_0_1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_CP_DBG_0_2 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_CP_DBG_0_2 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_CP_DBG_0_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_DBG_0_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_DBG_0_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_CP_DBG_0_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_DBG_0_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_DBG_0_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_ARUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_ARUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_ARUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_ARUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_ARUSER_31_11_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_AWUSER_31_11_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_AWUSER_31_11_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_AWUSER_31_11_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_AWUSER_31_11_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CP_AWUSER_31_11_4 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_ARB_CFG_0 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_ARB_CFG_0 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_ARB_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_ARB_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_19 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_23 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_ARB_MST_AVAIL_CRED_24 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_ARB_MST_AVAIL_CRED_24 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_AVAIL_CRED_31 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); @@ -8681,150 +8665,149 @@ static void gaudi_init_tpc_protection_bits(struct hl_device *hdev) PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_ARB_MST_CHOISE_PUSH_OFST_23 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_ARB_MST_QUIET_PER & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_ARB_SLV_CHOISE_WDT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MSG_MAX_INFLIGHT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MSG_AWUSER_31_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MSG_AWUSER_SEC_PROP & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MSG_AWUSER_NON_SEC_PROP & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_ARB_STATE_STS & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_ARB_STATE_STS & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_ARB_STATE_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MSG_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_ERR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_ARB_STATE_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_CHOISE_FULLNESS_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MSG_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_SLV_CHOISE_Q_HEAD & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_ERR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_ERR_MSG_EN & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_ERR_STS_DRP & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_9 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_10 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_11 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_12 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_13 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_14 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_15 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_16 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_17 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_18 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_19 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_ARB_MST_CRED_STS_20 & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_ARB_MST_CRED_STS_20 & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CGM_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CGM_STS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CGM_CFG1 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_20 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_21 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_22 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_23 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_24 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_25 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_26 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_27 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_28 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_29 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_30 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_ARB_MST_CRED_STS_31 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CGM_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CGM_STS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CGM_CFG1 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_LOCAL_RANGE_BASE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_LOCAL_RANGE_BASE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_AXCACHE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_IND_GW_APB_CFG & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_IND_GW_APB_WDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_IND_GW_APB_RDATA & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_IND_GW_APB_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_QM_GLBL_ERR_WDATA & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_LOCAL_RANGE_BASE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_LOCAL_RANGE_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_CSMR_STRICT_PRIO_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_HBW_RD_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_LBW_WR_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_LBW_WR_RATE_LIM_CFG_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_HBW_RD_RATE_LIM_CFG_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_AXCACHE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_IND_GW_APB_CFG & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_IND_GW_APB_WDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_IND_GW_APB_RDATA & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_IND_GW_APB_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_ERR_ADDR_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_ERR_ADDR_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_QM_GLBL_ERR_WDATA & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_QM_GLBL_MEM_INIT_BUSY & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_QM_GLBL_MEM_INIT_BUSY & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); + mask = 1U << ((mmTPC7_QM_GLBL_MEM_INIT_BUSY & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_CFG_ROUND_CSR & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_CFG_ROUND_CSR & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_CFG_ROUND_CSR & 0x7F) >> 2); + mask = 1U << ((mmTPC7_CFG_ROUND_CSR & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_CFG_PROT & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_CFG_PROT & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_CFG_PROT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_VFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_SFLAGS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_STATUS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_TPC_STALL & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_RD_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_WR_RATE_LIMIT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_MSS_CONFIG & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_TPC_INTR_MASK & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_WQ_CREDITS & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_ARUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_ARUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_AWUSER_LO & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_AWUSER_HI & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_OPCODE_EXEC & 0x7F) >> 2); + mask = 1U << ((mmTPC7_CFG_PROT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_VFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_SFLAGS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_STATUS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_CFG_BASE_ADDRESS_HIGH & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_CFG_SUBTRACT_VALUE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_TPC_STALL & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_RD_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_WR_RATE_LIMIT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_MSS_CONFIG & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_TPC_INTR_CAUSE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_TPC_INTR_MASK & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_WQ_CREDITS & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_ARUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_ARUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_AWUSER_LO & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_AWUSER_HI & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_OPCODE_EXEC & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); pb_addr = (mmTPC7_CFG_TSB_CFG_MAX_SIZE & ~0xFFF) + PROT_BITS_OFFS; word_offset = ((mmTPC7_CFG_TSB_CFG_MAX_SIZE & PROT_BITS_OFFS) >> 7) << 2; - mask = 1 << ((mmTPC7_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_DBGMEM_ADD & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_DBGMEM_CTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_DBGMEM_RC & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); - mask |= 1 << ((mmTPC7_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); + mask = 1U << ((mmTPC7_CFG_TSB_CFG_MAX_SIZE & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_DBGMEM_ADD & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_DBGMEM_DATA_WR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_DBGMEM_DATA_RD & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_DBGMEM_CTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_DBGMEM_RC & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_TSB_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_WQ_INFLIGHT_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_WQ_LBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_WQ_HBW_TOTAL_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_IRQ_OCCOUPY_CNTR & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_CNTRL & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_PAT & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_0 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_1 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_2 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_3 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_4 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_5 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_6 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_7 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_8 & 0x7F) >> 2); + mask |= 1U << ((mmTPC7_CFG_FUNC_MBIST_MEM_9 & 0x7F) >> 2); WREG32(pb_addr + word_offset, ~mask); } diff --git a/drivers/misc/habanalabs/goya/goya.c b/drivers/misc/habanalabs/goya/goya.c index 33cd2ae653d2..5db52064ed9e 100644 --- a/drivers/misc/habanalabs/goya/goya.c +++ b/drivers/misc/habanalabs/goya/goya.c @@ -426,12 +426,14 @@ int goya_get_fixed_properties(struct hl_device *hdev) prop->dmmu.start_addr = VA_DDR_SPACE_START; prop->dmmu.end_addr = VA_DDR_SPACE_END; prop->dmmu.page_size = PAGE_SIZE_2MB; + prop->dmmu.num_hops = MMU_ARCH_5_HOPS; /* shifts and masks are the same in PMMU and DMMU */ memcpy(&prop->pmmu, &prop->dmmu, sizeof(prop->dmmu)); prop->pmmu.start_addr = VA_HOST_SPACE_START; prop->pmmu.end_addr = VA_HOST_SPACE_END; prop->pmmu.page_size = PAGE_SIZE_4KB; + prop->pmmu.num_hops = MMU_ARCH_5_HOPS; /* PMMU and HPMMU are the same except of page size */ memcpy(&prop->pmmu_huge, &prop->pmmu, sizeof(prop->pmmu)); @@ -449,7 +451,7 @@ int goya_get_fixed_properties(struct hl_device *hdev) prop->pcie_dbi_base_address = mmPCIE_DBI_BASE; prop->pcie_aux_dbi_reg_addr = CFG_BASE + mmPCIE_AUX_DBI; - strncpy(prop->armcp_info.card_name, GOYA_DEFAULT_CARD_NAME, + strncpy(prop->cpucp_info.card_name, GOYA_DEFAULT_CARD_NAME, CARD_NAME_MAX_LEN); prop->max_pending_cs = GOYA_MAX_PENDING_CS; @@ -598,10 +600,15 @@ static int goya_early_init(struct hl_device *hdev) prop->dram_pci_bar_size = pci_resource_len(pdev, DDR_BAR_ID); - rc = hl_pci_init(hdev); + rc = hl_pci_init(hdev, mmPSOC_GLOBAL_CONF_CPU_BOOT_STATUS, + mmCPU_BOOT_ERR0, GOYA_BOOT_FIT_REQ_TIMEOUT_USEC); if (rc) goto free_queue_props; + /* Goya Firmware does not support security */ + prop->fw_security_disabled = true; + dev_info(hdev->dev, "firmware-level security is disabled\n"); + if (!hdev->pldm) { val = RREG32(mmPSOC_GLOBAL_CONF_BOOT_STRAP_PINS); if (val & PSOC_GLOBAL_CONF_BOOT_STRAP_PINS_SRIOV_EN_MASK) @@ -727,9 +734,9 @@ int goya_late_init(struct hl_device *hdev) if (rc) return rc; - rc = goya_armcp_info_get(hdev); + rc = goya_cpucp_info_get(hdev); if (rc) { - dev_err(hdev->dev, "Failed to get armcp info %d\n", rc); + dev_err(hdev->dev, "Failed to get cpucp info %d\n", rc); return rc; } @@ -739,7 +746,7 @@ int goya_late_init(struct hl_device *hdev) */ WREG32(mmMMU_LOG2_DDR_SIZE, ilog2(prop->dram_size)); - rc = hl_fw_send_pci_access_msg(hdev, ARMCP_PACKET_ENABLE_PCI_ACCESS); + rc = hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_ENABLE_PCI_ACCESS); if (rc) { dev_err(hdev->dev, "Failed to enable PCI access from CPU %d\n", rc); @@ -2648,7 +2655,7 @@ int goya_suspend(struct hl_device *hdev) { int rc; - rc = hl_fw_send_pci_access_msg(hdev, ARMCP_PACKET_DISABLE_PCI_ACCESS); + rc = hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_DISABLE_PCI_ACCESS); if (rc) dev_err(hdev->dev, "Failed to disable PCI access from CPU\n"); @@ -2661,17 +2668,16 @@ int goya_resume(struct hl_device *hdev) } static int goya_cb_mmap(struct hl_device *hdev, struct vm_area_struct *vma, - u64 kaddress, phys_addr_t paddress, u32 size) + void *cpu_addr, dma_addr_t dma_addr, size_t size) { int rc; vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_DONTCOPY | VM_NORESERVE; - rc = remap_pfn_range(vma, vma->vm_start, paddress >> PAGE_SHIFT, - size, vma->vm_page_prot); + rc = dma_mmap_coherent(hdev->dev, vma, cpu_addr, dma_addr, size); if (rc) - dev_err(hdev->dev, "remap_pfn_range error %d", rc); + dev_err(hdev->dev, "dma_mmap_coherent error %d", rc); return rc; } @@ -2946,7 +2952,8 @@ int goya_test_queue(struct hl_device *hdev, u32 hw_queue_id) &fence_dma_addr); if (!fence_ptr) { dev_err(hdev->dev, - "Failed to allocate memory for queue testing\n"); + "Failed to allocate memory for H/W queue %d testing\n", + hw_queue_id); return -ENOMEM; } @@ -2957,7 +2964,8 @@ int goya_test_queue(struct hl_device *hdev, u32 hw_queue_id) GFP_KERNEL, &pkt_dma_addr); if (!fence_pkt) { dev_err(hdev->dev, - "Failed to allocate packet for queue testing\n"); + "Failed to allocate packet for H/W queue %d testing\n", + hw_queue_id); rc = -ENOMEM; goto free_fence_ptr; } @@ -2974,7 +2982,8 @@ int goya_test_queue(struct hl_device *hdev, u32 hw_queue_id) pkt_dma_addr); if (rc) { dev_err(hdev->dev, - "Failed to send fence packet\n"); + "Failed to send fence packet to H/W queue %d\n", + hw_queue_id); goto free_pkt; } @@ -3806,8 +3815,9 @@ static int goya_parse_cb_mmu(struct hl_device *hdev, parser->patched_cb_size = parser->user_cb_size + sizeof(struct packet_msg_prot) * 2; - rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, parser->patched_cb_size, - &patched_cb_handle, HL_KERNEL_ASID_ID, false); + rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, hdev->kernel_ctx, + parser->patched_cb_size, false, false, + &patched_cb_handle); if (rc) { dev_err(hdev->dev, @@ -3879,8 +3889,9 @@ static int goya_parse_cb_no_mmu(struct hl_device *hdev, if (rc) goto free_userptr; - rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, parser->patched_cb_size, - &patched_cb_handle, HL_KERNEL_ASID_ID, false); + rc = hl_cb_create(hdev, &hdev->kernel_cb_mgr, hdev->kernel_ctx, + parser->patched_cb_size, false, false, + &patched_cb_handle); if (rc) { dev_err(hdev->dev, "Failed to allocate patched CB for DMA CS %d\n", rc); @@ -4497,17 +4508,17 @@ static void goya_print_irq_info(struct hl_device *hdev, u16 event_type, static int goya_unmask_irq_arr(struct hl_device *hdev, u32 *irq_arr, size_t irq_arr_size) { - struct armcp_unmask_irq_arr_packet *pkt; + struct cpucp_unmask_irq_arr_packet *pkt; size_t total_pkt_size; long result; int rc; int irq_num_entries, irq_arr_index; __le32 *goya_irq_arr; - total_pkt_size = sizeof(struct armcp_unmask_irq_arr_packet) + + total_pkt_size = sizeof(struct cpucp_unmask_irq_arr_packet) + irq_arr_size; - /* data should be aligned to 8 bytes in order to ArmCP to copy it */ + /* data should be aligned to 8 bytes in order to CPU-CP to copy it */ total_pkt_size = (total_pkt_size + 0x7) & ~0x7; /* total_pkt_size is casted to u16 later on */ @@ -4531,8 +4542,8 @@ static int goya_unmask_irq_arr(struct hl_device *hdev, u32 *irq_arr, goya_irq_arr[irq_arr_index] = cpu_to_le32(irq_arr[irq_arr_index]); - pkt->armcp_pkt.ctl = cpu_to_le32(ARMCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt->cpucp_pkt.ctl = cpu_to_le32(CPUCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY << + CPUCP_PKT_CTL_OPCODE_SHIFT); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) pkt, total_pkt_size, 0, &result); @@ -4557,14 +4568,14 @@ static int goya_soft_reset_late_init(struct hl_device *hdev) static int goya_unmask_irq(struct hl_device *hdev, u16 event_type) { - struct armcp_packet pkt; + struct cpucp_packet pkt; long result; int rc; memset(&pkt, 0, sizeof(pkt)); - pkt.ctl = cpu_to_le32(ARMCP_PACKET_UNMASK_RAZWI_IRQ << - ARMCP_PKT_CTL_OPCODE_SHIFT); + pkt.ctl = cpu_to_le32(CPUCP_PACKET_UNMASK_RAZWI_IRQ << + CPUCP_PKT_CTL_OPCODE_SHIFT); pkt.value = cpu_to_le64(event_type); rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt), @@ -4580,18 +4591,22 @@ static void goya_print_clk_change_info(struct hl_device *hdev, u16 event_type) { switch (event_type) { case GOYA_ASYNC_EVENT_ID_FIX_POWER_ENV_S: + hdev->clk_throttling_reason |= HL_CLK_THROTTLE_POWER; dev_info_ratelimited(hdev->dev, "Clock throttling due to power consumption\n"); break; case GOYA_ASYNC_EVENT_ID_FIX_POWER_ENV_E: + hdev->clk_throttling_reason &= ~HL_CLK_THROTTLE_POWER; dev_info_ratelimited(hdev->dev, "Power envelop is safe, back to optimal clock\n"); break; case GOYA_ASYNC_EVENT_ID_FIX_THERMAL_ENV_S: + hdev->clk_throttling_reason |= HL_CLK_THROTTLE_THERMAL; dev_info_ratelimited(hdev->dev, "Clock throttling due to overheating\n"); break; case GOYA_ASYNC_EVENT_ID_FIX_THERMAL_ENV_E: + hdev->clk_throttling_reason &= ~HL_CLK_THROTTLE_THERMAL; dev_info_ratelimited(hdev->dev, "Thermal envelop is safe, back to optimal clock\n"); break; @@ -4638,7 +4653,8 @@ void goya_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_entry) case GOYA_ASYNC_EVENT_ID_L2_RAM_ECC: case GOYA_ASYNC_EVENT_ID_PSOC_GPIO_05_SW_RESET: goya_print_irq_info(hdev, event_type, false); - hl_device_reset(hdev, true, false); + if (hdev->hard_reset_on_fw_events) + hl_device_reset(hdev, true, false); break; case GOYA_ASYNC_EVENT_ID_PCIE_DEC: @@ -5096,7 +5112,7 @@ int goya_send_heartbeat(struct hl_device *hdev) return hl_fw_send_heartbeat(hdev); } -int goya_armcp_info_get(struct hl_device *hdev) +int goya_cpucp_info_get(struct hl_device *hdev) { struct goya_device *goya = hdev->asic_specific; struct asic_fixed_properties *prop = &hdev->asic_prop; @@ -5106,11 +5122,11 @@ int goya_armcp_info_get(struct hl_device *hdev) if (!(goya->hw_cap_initialized & HW_CAP_CPU_Q)) return 0; - rc = hl_fw_armcp_info_get(hdev); + rc = hl_fw_cpucp_info_get(hdev); if (rc) return rc; - dram_size = le64_to_cpu(prop->armcp_info.dram_size); + dram_size = le64_to_cpu(prop->cpucp_info.dram_size); if (dram_size) { if ((!is_power_of_2(dram_size)) || (dram_size < DRAM_PHYS_DEFAULT_SIZE)) { @@ -5124,8 +5140,8 @@ int goya_armcp_info_get(struct hl_device *hdev) prop->dram_end_address = prop->dram_base_address + dram_size; } - if (!strlen(prop->armcp_info.card_name)) - strncpy(prop->armcp_info.card_name, GOYA_DEFAULT_CARD_NAME, + if (!strlen(prop->cpucp_info.card_name)) + strncpy(prop->cpucp_info.card_name, GOYA_DEFAULT_CARD_NAME, CARD_NAME_MAX_LEN); return 0; @@ -5141,7 +5157,7 @@ static void goya_disable_clock_gating(struct hl_device *hdev) /* clock gating not supported in Goya */ } -static bool goya_is_device_idle(struct hl_device *hdev, u32 *mask, +static bool goya_is_device_idle(struct hl_device *hdev, u64 *mask, struct seq_file *s) { const char *fmt = "%-5d%-9s%#-14x%#-16x%#x\n"; @@ -5166,7 +5182,8 @@ static bool goya_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << (GOYA_ENGINE_ID_DMA_0 + i); + *mask |= ((u64) !is_eng_idle) << + (GOYA_ENGINE_ID_DMA_0 + i); if (s) seq_printf(s, dma_fmt, i, is_eng_idle ? "Y" : "N", qm_glbl_sts0, dma_core_sts0); @@ -5189,7 +5206,8 @@ static bool goya_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << (GOYA_ENGINE_ID_TPC_0 + i); + *mask |= ((u64) !is_eng_idle) << + (GOYA_ENGINE_ID_TPC_0 + i); if (s) seq_printf(s, fmt, i, is_eng_idle ? "Y" : "N", qm_glbl_sts0, cmdq_glbl_sts0, tpc_cfg_sts); @@ -5209,7 +5227,7 @@ static bool goya_is_device_idle(struct hl_device *hdev, u32 *mask, is_idle &= is_eng_idle; if (mask) - *mask |= !is_eng_idle << GOYA_ENGINE_ID_MME_0; + *mask |= ((u64) !is_eng_idle) << GOYA_ENGINE_ID_MME_0; if (s) { seq_printf(s, fmt, 0, is_eng_idle ? "Y" : "N", qm_glbl_sts0, cmdq_glbl_sts0, mme_arch_sts); @@ -5369,7 +5387,6 @@ static const struct hl_asic_funcs goya_funcs = { .send_cpu_message = goya_send_cpu_message, .get_hw_state = goya_get_hw_state, .pci_bars_map = goya_pci_bars_map, - .set_dram_bar_base = goya_set_ddr_bar_base, .init_iatu = goya_init_iatu, .rreg = hl_rreg, .wreg = hl_wreg, diff --git a/drivers/misc/habanalabs/goya/goyaP.h b/drivers/misc/habanalabs/goya/goyaP.h index bb7474ee9784..09b4006d4dc3 100644 --- a/drivers/misc/habanalabs/goya/goyaP.h +++ b/drivers/misc/habanalabs/goya/goyaP.h @@ -207,7 +207,7 @@ void goya_set_max_power(struct hl_device *hdev, u64 value); void goya_set_pll_profile(struct hl_device *hdev, enum hl_pll_frequency freq); void goya_add_device_attr(struct hl_device *hdev, struct attribute_group *dev_attr_grp); -int goya_armcp_info_get(struct hl_device *hdev); +int goya_cpucp_info_get(struct hl_device *hdev); int goya_debug_coresight(struct hl_device *hdev, void *data); void goya_halt_coresight(struct hl_device *hdev); diff --git a/drivers/misc/habanalabs/include/common/armcp_if.h b/drivers/misc/habanalabs/include/common/cpucp_if.h index 07f9972db28d..2a5c9cb3d505 100644 --- a/drivers/misc/habanalabs/include/common/armcp_if.h +++ b/drivers/misc/habanalabs/include/common/cpucp_if.h @@ -1,12 +1,12 @@ /* SPDX-License-Identifier: GPL-2.0 * - * Copyright 2016-2020 HabanaLabs, Ltd. + * Copyright 2020 HabanaLabs, Ltd. * All Rights Reserved. * */ -#ifndef ARMCP_IF_H -#define ARMCP_IF_H +#ifndef CPUCP_IF_H +#define CPUCP_IF_H #include <linux/types.h> @@ -50,16 +50,16 @@ enum pq_init_status { }; /* - * ArmCP Primary Queue Packets + * CpuCP Primary Queue Packets * * During normal operation, the host's kernel driver needs to send various - * messages to ArmCP, usually either to SET some value into a H/W periphery or + * messages to CpuCP, usually either to SET some value into a H/W periphery or * to GET the current value of some H/W periphery. For example, SET the * frequency of MME/TPC and GET the value of the thermal sensor. * * These messages can be initiated either by the User application or by the * host's driver itself, e.g. power management code. In either case, the - * communication from the host's driver to ArmCP will *always* be in + * communication from the host's driver to CpuCP will *always* be in * synchronous mode, meaning that the host will send a single message and poll * until the message was acknowledged and the results are ready (if results are * needed). @@ -73,21 +73,20 @@ enum pq_init_status { * * The message, inputs/outputs (if relevant) and fence object will be located * on the device DDR at an address that will be determined by the host's driver. - * During device initialization phase, the host will pass to ArmCP that address. + * During device initialization phase, the host will pass to CpuCP that address. * Most of the message types will contain inputs/outputs inside the message * itself. The common part of each message will contain the opcode of the * message (its type) and a field representing a fence object. * - * When the host's driver wishes to send a message to ArmCP, it will write the - * message contents to the device DDR, clear the fence object and then write the - * value 484 to the mmGIC_DISTRIBUTOR__5_GICD_SETSPI_NSR register to issue - * the 484 interrupt-id to the ARM core. + * When the host's driver wishes to send a message to CPU CP, it will write the + * message contents to the device DDR, clear the fence object and then write to + * the PSOC_ARC1_AUX_SW_INTR, to issue interrupt 121 to ARC Management CPU. * - * Upon receiving the 484 interrupt-id, ArmCP will read the message from the - * DDR. In case the message is a SET operation, ArmCP will first perform the + * Upon receiving the interrupt (#121), CpuCP will read the message from the + * DDR. In case the message is a SET operation, CpuCP will first perform the * operation and then write to the fence object on the device DDR. In case the - * message is a GET operation, ArmCP will first fill the results section on the - * device DDR and then write to the fence object. If an error occurred, ArmCP + * message is a GET operation, CpuCP will first fill the results section on the + * device DDR and then write to the fence object. If an error occurred, CpuCP * will fill the rc field with the right error code. * * In the meantime, the host's driver will poll on the fence object. Once the @@ -96,164 +95,174 @@ enum pq_init_status { * driver. * * To use QMAN packets, the opcode must be the QMAN opcode, shifted by 8 - * so the value being put by the host's driver matches the value read by ArmCP + * so the value being put by the host's driver matches the value read by CpuCP * * Non-QMAN packets should be limited to values 1 through (2^8 - 1) * * Detailed description: * - * ARMCP_PACKET_DISABLE_PCI_ACCESS - + * CPUCP_PACKET_DISABLE_PCI_ACCESS - * After receiving this packet the embedded CPU must NOT issue PCI * transactions (read/write) towards the Host CPU. This also include * sending MSI-X interrupts. * This packet is usually sent before the device is moved to D3Hot state. * - * ARMCP_PACKET_ENABLE_PCI_ACCESS - + * CPUCP_PACKET_ENABLE_PCI_ACCESS - * After receiving this packet the embedded CPU is allowed to issue PCI * transactions towards the Host CPU, including sending MSI-X interrupts. * This packet is usually send after the device is moved to D0 state. * - * ARMCP_PACKET_TEMPERATURE_GET - + * CPUCP_PACKET_TEMPERATURE_GET - * Fetch the current temperature / Max / Max Hyst / Critical / * Critical Hyst of a specified thermal sensor. The packet's * arguments specify the desired sensor and the field to get. * - * ARMCP_PACKET_VOLTAGE_GET - + * CPUCP_PACKET_VOLTAGE_GET - * Fetch the voltage / Max / Min of a specified sensor. The packet's * arguments specify the sensor and type. * - * ARMCP_PACKET_CURRENT_GET - + * CPUCP_PACKET_CURRENT_GET - * Fetch the current / Max / Min of a specified sensor. The packet's * arguments specify the sensor and type. * - * ARMCP_PACKET_FAN_SPEED_GET - + * CPUCP_PACKET_FAN_SPEED_GET - * Fetch the speed / Max / Min of a specified fan. The packet's * arguments specify the sensor and type. * - * ARMCP_PACKET_PWM_GET - + * CPUCP_PACKET_PWM_GET - * Fetch the pwm value / mode of a specified pwm. The packet's * arguments specify the sensor and type. * - * ARMCP_PACKET_PWM_SET - + * CPUCP_PACKET_PWM_SET - * Set the pwm value / mode of a specified pwm. The packet's * arguments specify the sensor, type and value. * - * ARMCP_PACKET_FREQUENCY_SET - + * CPUCP_PACKET_FREQUENCY_SET - * Set the frequency of a specified PLL. The packet's arguments specify * the PLL and the desired frequency. The actual frequency in the device * might differ from the requested frequency. * - * ARMCP_PACKET_FREQUENCY_GET - + * CPUCP_PACKET_FREQUENCY_GET - * Fetch the frequency of a specified PLL. The packet's arguments specify * the PLL. * - * ARMCP_PACKET_LED_SET - + * CPUCP_PACKET_LED_SET - * Set the state of a specified led. The packet's arguments * specify the led and the desired state. * - * ARMCP_PACKET_I2C_WR - + * CPUCP_PACKET_I2C_WR - * Write 32-bit value to I2C device. The packet's arguments specify the * I2C bus, address and value. * - * ARMCP_PACKET_I2C_RD - + * CPUCP_PACKET_I2C_RD - * Read 32-bit value from I2C device. The packet's arguments specify the * I2C bus and address. * - * ARMCP_PACKET_INFO_GET - + * CPUCP_PACKET_INFO_GET - * Fetch information from the device as specified in the packet's - * structure. The host's driver passes the max size it allows the ArmCP to + * structure. The host's driver passes the max size it allows the CpuCP to * write to the structure, to prevent data corruption in case of * mismatched driver/FW versions. * - * ARMCP_PACKET_FLASH_PROGRAM_REMOVED - this packet was removed + * CPUCP_PACKET_FLASH_PROGRAM_REMOVED - this packet was removed * - * ARMCP_PACKET_UNMASK_RAZWI_IRQ - + * CPUCP_PACKET_UNMASK_RAZWI_IRQ - * Unmask the given IRQ. The IRQ number is specified in the value field. * The packet is sent after receiving an interrupt and printing its * relevant information. * - * ARMCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY - + * CPUCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY - * Unmask the given IRQs. The IRQs numbers are specified in an array right - * after the armcp_packet structure, where its first element is the array + * after the cpucp_packet structure, where its first element is the array * length. The packet is sent after a soft reset was done in order to * handle any interrupts that were sent during the reset process. * - * ARMCP_PACKET_TEST - - * Test packet for ArmCP connectivity. The CPU will put the fence value + * CPUCP_PACKET_TEST - + * Test packet for CpuCP connectivity. The CPU will put the fence value * in the result field. * - * ARMCP_PACKET_FREQUENCY_CURR_GET - + * CPUCP_PACKET_FREQUENCY_CURR_GET - * Fetch the current frequency of a specified PLL. The packet's arguments * specify the PLL. * - * ARMCP_PACKET_MAX_POWER_GET - + * CPUCP_PACKET_MAX_POWER_GET - * Fetch the maximal power of the device. * - * ARMCP_PACKET_MAX_POWER_SET - + * CPUCP_PACKET_MAX_POWER_SET - * Set the maximal power of the device. The packet's arguments specify * the power. * - * ARMCP_PACKET_EEPROM_DATA_GET - - * Get EEPROM data from the ArmCP kernel. The buffer is specified in the + * CPUCP_PACKET_EEPROM_DATA_GET - + * Get EEPROM data from the CpuCP kernel. The buffer is specified in the * addr field. The CPU will put the returned data size in the result * field. In addition, the host's driver passes the max size it allows the - * ArmCP to write to the structure, to prevent data corruption in case of + * CpuCP to write to the structure, to prevent data corruption in case of * mismatched driver/FW versions. * - * ARMCP_PACKET_TEMPERATURE_SET - + * CPUCP_PACKET_TEMPERATURE_SET - * Set the value of the offset property of a specified thermal sensor. * The packet's arguments specify the desired sensor and the field to * set. * - * ARMCP_PACKET_VOLTAGE_SET - + * CPUCP_PACKET_VOLTAGE_SET - * Trigger the reset_history property of a specified voltage sensor. * The packet's arguments specify the desired sensor and the field to * set. * - * ARMCP_PACKET_CURRENT_SET - + * CPUCP_PACKET_CURRENT_SET - * Trigger the reset_history property of a specified current sensor. * The packet's arguments specify the desired sensor and the field to * set. + * + * CPUCP_PACKET_PLL_REG_GET + * Fetch register of PLL from the required PLL IP. + * The packet's arguments specify the PLL IP and the register to get. + * Each register is 32-bit value which is returned in result field. + * */ -enum armcp_packet_id { - ARMCP_PACKET_DISABLE_PCI_ACCESS = 1, /* internal */ - ARMCP_PACKET_ENABLE_PCI_ACCESS, /* internal */ - ARMCP_PACKET_TEMPERATURE_GET, /* sysfs */ - ARMCP_PACKET_VOLTAGE_GET, /* sysfs */ - ARMCP_PACKET_CURRENT_GET, /* sysfs */ - ARMCP_PACKET_FAN_SPEED_GET, /* sysfs */ - ARMCP_PACKET_PWM_GET, /* sysfs */ - ARMCP_PACKET_PWM_SET, /* sysfs */ - ARMCP_PACKET_FREQUENCY_SET, /* sysfs */ - ARMCP_PACKET_FREQUENCY_GET, /* sysfs */ - ARMCP_PACKET_LED_SET, /* debugfs */ - ARMCP_PACKET_I2C_WR, /* debugfs */ - ARMCP_PACKET_I2C_RD, /* debugfs */ - ARMCP_PACKET_INFO_GET, /* IOCTL */ - ARMCP_PACKET_FLASH_PROGRAM_REMOVED, - ARMCP_PACKET_UNMASK_RAZWI_IRQ, /* internal */ - ARMCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY, /* internal */ - ARMCP_PACKET_TEST, /* internal */ - ARMCP_PACKET_FREQUENCY_CURR_GET, /* sysfs */ - ARMCP_PACKET_MAX_POWER_GET, /* sysfs */ - ARMCP_PACKET_MAX_POWER_SET, /* sysfs */ - ARMCP_PACKET_EEPROM_DATA_GET, /* sysfs */ - ARMCP_RESERVED, - ARMCP_PACKET_TEMPERATURE_SET, /* sysfs */ - ARMCP_PACKET_VOLTAGE_SET, /* sysfs */ - ARMCP_PACKET_CURRENT_SET, /* sysfs */ +enum cpucp_packet_id { + CPUCP_PACKET_DISABLE_PCI_ACCESS = 1, /* internal */ + CPUCP_PACKET_ENABLE_PCI_ACCESS, /* internal */ + CPUCP_PACKET_TEMPERATURE_GET, /* sysfs */ + CPUCP_PACKET_VOLTAGE_GET, /* sysfs */ + CPUCP_PACKET_CURRENT_GET, /* sysfs */ + CPUCP_PACKET_FAN_SPEED_GET, /* sysfs */ + CPUCP_PACKET_PWM_GET, /* sysfs */ + CPUCP_PACKET_PWM_SET, /* sysfs */ + CPUCP_PACKET_FREQUENCY_SET, /* sysfs */ + CPUCP_PACKET_FREQUENCY_GET, /* sysfs */ + CPUCP_PACKET_LED_SET, /* debugfs */ + CPUCP_PACKET_I2C_WR, /* debugfs */ + CPUCP_PACKET_I2C_RD, /* debugfs */ + CPUCP_PACKET_INFO_GET, /* IOCTL */ + CPUCP_PACKET_FLASH_PROGRAM_REMOVED, + CPUCP_PACKET_UNMASK_RAZWI_IRQ, /* internal */ + CPUCP_PACKET_UNMASK_RAZWI_IRQ_ARRAY, /* internal */ + CPUCP_PACKET_TEST, /* internal */ + CPUCP_PACKET_FREQUENCY_CURR_GET, /* sysfs */ + CPUCP_PACKET_MAX_POWER_GET, /* sysfs */ + CPUCP_PACKET_MAX_POWER_SET, /* sysfs */ + CPUCP_PACKET_EEPROM_DATA_GET, /* sysfs */ + CPUCP_RESERVED, + CPUCP_PACKET_TEMPERATURE_SET, /* sysfs */ + CPUCP_PACKET_VOLTAGE_SET, /* sysfs */ + CPUCP_PACKET_CURRENT_SET, /* sysfs */ + CPUCP_PACKET_PCIE_THROUGHPUT_GET, /* internal */ + CPUCP_PACKET_PCIE_REPLAY_CNT_GET, /* internal */ + CPUCP_PACKET_TOTAL_ENERGY_GET, /* internal */ + CPUCP_PACKET_PLL_REG_GET, /* internal */ }; -#define ARMCP_PACKET_FENCE_VAL 0xFE8CE7A5 +#define CPUCP_PACKET_FENCE_VAL 0xFE8CE7A5 -#define ARMCP_PKT_CTL_RC_SHIFT 12 -#define ARMCP_PKT_CTL_RC_MASK 0x0000F000 +#define CPUCP_PKT_CTL_RC_SHIFT 12 +#define CPUCP_PKT_CTL_RC_MASK 0x0000F000 -#define ARMCP_PKT_CTL_OPCODE_SHIFT 16 -#define ARMCP_PKT_CTL_OPCODE_MASK 0x1FFF0000 +#define CPUCP_PKT_CTL_OPCODE_SHIFT 16 +#define CPUCP_PKT_CTL_OPCODE_MASK 0x1FFF0000 -struct armcp_packet { +struct cpucp_packet { union { __le64 value; /* For SET packets */ __le64 result; /* For GET packets */ @@ -277,71 +286,97 @@ struct armcp_packet { __u8 pad; /* unused */ }; + struct {/* For PLL register fetch */ + __le16 pll_type; + __le16 pll_reg; + }; + + /* For any general request */ + __le32 index; + /* For frequency get/set */ __le32 pll_index; /* For led set */ __le32 led_index; - /* For get Armcp info/EEPROM data */ + /* For get CpuCP info/EEPROM data */ __le32 data_max_size; }; __le32 reserved; }; -struct armcp_unmask_irq_arr_packet { - struct armcp_packet armcp_pkt; +struct cpucp_unmask_irq_arr_packet { + struct cpucp_packet cpucp_pkt; __le32 length; __le32 irqs[0]; }; -enum armcp_packet_rc { - armcp_packet_success, - armcp_packet_invalid, - armcp_packet_fault +enum cpucp_packet_rc { + cpucp_packet_success, + cpucp_packet_invalid, + cpucp_packet_fault }; /* - * armcp_temp_type should adhere to hwmon_temp_attributes + * cpucp_temp_type should adhere to hwmon_temp_attributes * defined in Linux kernel hwmon.h file */ -enum armcp_temp_type { - armcp_temp_input, - armcp_temp_max = 6, - armcp_temp_max_hyst, - armcp_temp_crit, - armcp_temp_crit_hyst, - armcp_temp_offset = 19, - armcp_temp_highest = 22, - armcp_temp_reset_history = 23 +enum cpucp_temp_type { + cpucp_temp_input, + cpucp_temp_max = 6, + cpucp_temp_max_hyst, + cpucp_temp_crit, + cpucp_temp_crit_hyst, + cpucp_temp_offset = 19, + cpucp_temp_highest = 22, + cpucp_temp_reset_history = 23 +}; + +enum cpucp_in_attributes { + cpucp_in_input, + cpucp_in_min, + cpucp_in_max, + cpucp_in_highest = 7, + cpucp_in_reset_history +}; + +enum cpucp_curr_attributes { + cpucp_curr_input, + cpucp_curr_min, + cpucp_curr_max, + cpucp_curr_highest = 7, + cpucp_curr_reset_history +}; + +enum cpucp_fan_attributes { + cpucp_fan_input, + cpucp_fan_min = 2, + cpucp_fan_max }; -enum armcp_in_attributes { - armcp_in_input, - armcp_in_min, - armcp_in_max, - armcp_in_highest = 7, - armcp_in_reset_history +enum cpucp_pwm_attributes { + cpucp_pwm_input, + cpucp_pwm_enable }; -enum armcp_curr_attributes { - armcp_curr_input, - armcp_curr_min, - armcp_curr_max, - armcp_curr_highest = 7, - armcp_curr_reset_history +enum cpucp_pcie_throughput_attributes { + cpucp_pcie_throughput_tx, + cpucp_pcie_throughput_rx }; -enum armcp_fan_attributes { - armcp_fan_input, - armcp_fan_min = 2, - armcp_fan_max +enum cpucp_pll_reg_attributes { + cpucp_pll_nr_reg, + cpucp_pll_nf_reg, + cpucp_pll_od_reg, + cpucp_pll_div_factor_reg, + cpucp_pll_div_sel_reg }; -enum armcp_pwm_attributes { - armcp_pwm_input, - armcp_pwm_enable +enum cpucp_pll_type_attributes { + cpucp_pll_cpu, + cpucp_pll_pci, }; /* Event Queue Packets */ @@ -351,32 +386,32 @@ struct eq_generic_event { }; /* - * ArmCP info + * CpuCP info */ #define CARD_NAME_MAX_LEN 16 #define VERSION_MAX_LEN 128 -#define ARMCP_MAX_SENSORS 128 +#define CPUCP_MAX_SENSORS 128 -struct armcp_sensor { +struct cpucp_sensor { __le32 type; __le32 flags; }; /** - * struct armcp_card_types - ASIC card type. - * @armcp_card_type_pci: PCI card. - * @armcp_card_type_pmc: PCI Mezzanine Card. + * struct cpucp_card_types - ASIC card type. + * @cpucp_card_type_pci: PCI card. + * @cpucp_card_type_pmc: PCI Mezzanine Card. */ -enum armcp_card_types { - armcp_card_type_pci, - armcp_card_type_pmc +enum cpucp_card_types { + cpucp_card_type_pci, + cpucp_card_type_pmc }; /** - * struct armcp_info - Info from ArmCP that is necessary to the host's driver + * struct cpucp_info - Info from CpuCP that is necessary to the host's driver * @sensors: available sensors description. - * @kernel_version: ArmCP linux kernel version. + * @kernel_version: CpuCP linux kernel version. * @reserved: reserved field. * @card_type: card configuration type. * @card_location: in a server, each card has different connections topology @@ -385,12 +420,12 @@ enum armcp_card_types { * @infineon_version: Infineon main DC-DC version. * @fuse_version: silicon production FUSE information. * @thermal_version: thermald S/W version. - * @armcp_version: ArmCP S/W version. + * @cpucp_version: CpuCP S/W version. * @dram_size: available DRAM size. * @card_name: card name that will be displayed in HWMON subsystem on the host */ -struct armcp_info { - struct armcp_sensor sensors[ARMCP_MAX_SENSORS]; +struct cpucp_info { + struct cpucp_sensor sensors[CPUCP_MAX_SENSORS]; __u8 kernel_version[VERSION_MAX_LEN]; __le32 reserved; __le32 card_type; @@ -399,9 +434,10 @@ struct armcp_info { __le32 infineon_version; __u8 fuse_version[VERSION_MAX_LEN]; __u8 thermal_version[VERSION_MAX_LEN]; - __u8 armcp_version[VERSION_MAX_LEN]; + __u8 cpucp_version[VERSION_MAX_LEN]; + __le32 reserved2; __le64 dram_size; char card_name[CARD_NAME_MAX_LEN]; }; -#endif /* ARMCP_IF_H */ +#endif /* CPUCP_IF_H */ diff --git a/drivers/misc/habanalabs/include/common/qman_if.h b/drivers/misc/habanalabs/include/common/qman_if.h index 0fdb49188ed7..7ed7739575ee 100644 --- a/drivers/misc/habanalabs/include/common/qman_if.h +++ b/drivers/misc/habanalabs/include/common/qman_if.h @@ -40,7 +40,7 @@ struct hl_bd { */ #define BD_CTL_COMP_OFFSET_SHIFT 16 -#define BD_CTL_COMP_OFFSET_MASK 0x00FF0000 +#define BD_CTL_COMP_OFFSET_MASK 0x0FFF0000 #define BD_CTL_COMP_DATA_SHIFT 0 #define BD_CTL_COMP_DATA_MASK 0x0000FFFF diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi.h b/drivers/misc/habanalabs/include/gaudi/gaudi.h index 8829891d3eef..f9ea897ae42c 100644 --- a/drivers/misc/habanalabs/include/gaudi/gaudi.h +++ b/drivers/misc/habanalabs/include/gaudi/gaudi.h @@ -44,6 +44,8 @@ #define MME_NUMBER_OF_MASTER_ENGINES 2 +#define MME_NUMBER_OF_SLAVE_ENGINES 2 + #define TPC_NUMBER_OF_ENGINES 8 #define DMA_NUMBER_OF_CHANNELS 8 diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_masks.h b/drivers/misc/habanalabs/include/gaudi/gaudi_masks.h index 3510c42d24e3..f395721060bd 100644 --- a/drivers/misc/habanalabs/include/gaudi/gaudi_masks.h +++ b/drivers/misc/habanalabs/include/gaudi/gaudi_masks.h @@ -12,191 +12,160 @@ /* Useful masks for bits in various registers */ #define PCI_DMA_QMAN_ENABLE (\ - (0xF << DMA0_QM_GLBL_CFG0_PQF_EN_SHIFT) | \ - (0xF << DMA0_QM_GLBL_CFG0_CQF_EN_SHIFT) | \ - (0xF << DMA0_QM_GLBL_CFG0_CP_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_CFG0_PQF_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_CFG0_CQF_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_CFG0_CP_EN_MASK, 0xF))) #define QMAN_EXTERNAL_MAKE_TRUSTED (\ - (0xF << DMA0_QM_GLBL_PROT_PQF_SHIFT) | \ - (0xF << DMA0_QM_GLBL_PROT_CQF_SHIFT) | \ - (0xF << DMA0_QM_GLBL_PROT_CP_SHIFT) | \ - (0x1 << DMA0_QM_GLBL_PROT_ERR_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_PROT_PQF_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_PROT_CQF_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_PROT_CP_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_PROT_ERR_MASK, 0x1))) #define QMAN_INTERNAL_MAKE_TRUSTED (\ - (0xF << DMA0_QM_GLBL_PROT_PQF_SHIFT) | \ - (0x1 << DMA0_QM_GLBL_PROT_ERR_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_PROT_PQF_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_PROT_ERR_MASK, 0x1))) #define HBM_DMA_QMAN_ENABLE (\ - (0xF << DMA0_QM_GLBL_CFG0_PQF_EN_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_CFG0_CQF_EN_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_CFG0_CP_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_CFG0_PQF_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_CFG0_CQF_EN_MASK, 0x1F)) | \ + (FIELD_PREP(DMA0_QM_GLBL_CFG0_CP_EN_MASK, 0x1F))) #define QMAN_MME_ENABLE (\ - (0xF << MME0_QM_GLBL_CFG0_PQF_EN_SHIFT) | \ - (0x1F << MME0_QM_GLBL_CFG0_CQF_EN_SHIFT) | \ - (0x1F << MME0_QM_GLBL_CFG0_CP_EN_SHIFT)) + (FIELD_PREP(MME0_QM_GLBL_CFG0_PQF_EN_MASK, 0xF)) | \ + (FIELD_PREP(MME0_QM_GLBL_CFG0_CQF_EN_MASK, 0x1F)) | \ + (FIELD_PREP(MME0_QM_GLBL_CFG0_CP_EN_MASK, 0x1F))) #define QMAN_TPC_ENABLE (\ - (0xF << TPC0_QM_GLBL_CFG0_PQF_EN_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_CFG0_CQF_EN_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_CFG0_CP_EN_SHIFT)) + (FIELD_PREP(TPC0_QM_GLBL_CFG0_PQF_EN_MASK, 0xF)) | \ + (FIELD_PREP(TPC0_QM_GLBL_CFG0_CQF_EN_MASK, 0x1F)) | \ + (FIELD_PREP(TPC0_QM_GLBL_CFG0_CP_EN_MASK, 0x1F))) #define QMAN_UPPER_CP_CGM_PWR_GATE_EN (\ - (0x20 << DMA0_QM_CGM_CFG_IDLE_TH_SHIFT) | \ - (0xA << DMA0_QM_CGM_CFG_G2F_TH_SHIFT) | \ - (0x10 << DMA0_QM_CGM_CFG_CP_IDLE_MASK_SHIFT) | \ - (1 << DMA0_QM_CGM_CFG_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_CGM_CFG_IDLE_TH_MASK, 0x20)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_G2F_TH_MASK, 0xA)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_CP_IDLE_MASK_MASK, 0x10)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_EN_MASK, 0x1))) #define QMAN_COMMON_CP_CGM_PWR_GATE_EN (\ - (0x20 << DMA0_QM_CGM_CFG_IDLE_TH_SHIFT) | \ - (0xA << DMA0_QM_CGM_CFG_G2F_TH_SHIFT) | \ - (0xF << DMA0_QM_CGM_CFG_CP_IDLE_MASK_SHIFT) | \ - (1 << DMA0_QM_CGM_CFG_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_CGM_CFG_IDLE_TH_MASK, 0x20)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_G2F_TH_MASK, 0xA)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_CP_IDLE_MASK_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_CGM_CFG_EN_MASK, 0x1))) #define PCI_DMA_QMAN_GLBL_ERR_CFG_MSG_EN_MASK (\ - (0xF << DMA0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_SHIFT) | \ - (0xF << DMA0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_SHIFT) | \ - (0xF << DMA0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_MASK, 0xF))) #define PCI_DMA_QMAN_GLBL_ERR_CFG_STOP_ON_ERR_EN_MASK (\ - (0xF << DMA0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_SHIFT) | \ - (0xF << DMA0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_SHIFT) | \ - (0xF << DMA0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_MASK, 0xF))) #define HBM_DMA_QMAN_GLBL_ERR_CFG_MSG_EN_MASK (\ - (0xF << DMA0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_MASK, 0x1F)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_MASK, 0x1F))) #define HBM_DMA_QMAN_GLBL_ERR_CFG_STOP_ON_ERR_EN_MASK (\ - (0xF << DMA0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << DMA0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_SHIFT)) + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_MASK, 0xF)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_MASK, 0x1F)) | \ + (FIELD_PREP(DMA0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_MASK, 0x1F))) #define TPC_QMAN_GLBL_ERR_CFG_MSG_EN_MASK (\ - (0xF << TPC0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_SHIFT)) + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_MASK, 0xF)) | \ + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_MASK, 0x1F)) | \ + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_MASK, 0x1F))) #define TPC_QMAN_GLBL_ERR_CFG_STOP_ON_ERR_EN_MASK (\ - (0xF << TPC0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << TPC0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_SHIFT)) + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_MASK, 0xF)) | \ + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_MASK, 0x1F)) | \ + (FIELD_PREP(TPC0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_MASK, 0x1F))) #define MME_QMAN_GLBL_ERR_CFG_MSG_EN_MASK (\ - (0xF << MME0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << MME0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_SHIFT) | \ - (0x1F << MME0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_SHIFT)) + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_PQF_ERR_MSG_EN_MASK, 0xF)) | \ + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_CQF_ERR_MSG_EN_MASK, 0x1F)) | \ + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_CP_ERR_MSG_EN_MASK, 0x1F))) #define MME_QMAN_GLBL_ERR_CFG_STOP_ON_ERR_EN_MASK (\ - (0xF << MME0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << MME0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_SHIFT) | \ - (0x1F << MME0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_SHIFT)) + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_PQF_STOP_ON_ERR_MASK, 0xF)) | \ + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_CQF_STOP_ON_ERR_MASK, 0x1F)) | \ + (FIELD_PREP(MME0_QM_GLBL_ERR_CFG_CP_STOP_ON_ERR_MASK, 0x1F))) -#define QMAN_CGM1_PWR_GATE_EN (0xA << DMA0_QM_CGM_CFG1_MASK_TH_SHIFT) +#define QMAN_CGM1_PWR_GATE_EN (FIELD_PREP(DMA0_QM_CGM_CFG1_MASK_TH_MASK, 0xA)) /* RESET registers configuration */ -#define CFG_RST_L_PSOC_SHIFT 0 -#define CFG_RST_L_PCIE_SHIFT 1 -#define CFG_RST_L_PCIE_IF_SHIFT 2 -#define CFG_RST_L_HBM_S_PLL_SHIFT 3 -#define CFG_RST_L_TPC_S_PLL_SHIFT 4 -#define CFG_RST_L_MME_S_PLL_SHIFT 5 -#define CFG_RST_L_CPU_PLL_SHIFT 6 -#define CFG_RST_L_PCIE_PLL_SHIFT 7 -#define CFG_RST_L_NIC_S_PLL_SHIFT 8 -#define CFG_RST_L_HBM_N_PLL_SHIFT 9 -#define CFG_RST_L_TPC_N_PLL_SHIFT 10 -#define CFG_RST_L_MME_N_PLL_SHIFT 11 -#define CFG_RST_L_NIC_N_PLL_SHIFT 12 -#define CFG_RST_L_DMA_W_PLL_SHIFT 13 -#define CFG_RST_L_SIF_W_PLL_SHIFT 14 -#define CFG_RST_L_MESH_W_PLL_SHIFT 15 -#define CFG_RST_L_SRAM_W_PLL_SHIFT 16 -#define CFG_RST_L_DMA_E_PLL_SHIFT 17 -#define CFG_RST_L_SIF_E_PLL_SHIFT 18 -#define CFG_RST_L_MESH_E_PLL_SHIFT 19 -#define CFG_RST_L_SRAM_E_PLL_SHIFT 20 -#define CFG_RST_L_IF_1_SHIFT 21 -#define CFG_RST_L_IF_0_SHIFT 22 -#define CFG_RST_L_IF_2_SHIFT 23 -#define CFG_RST_L_IF_3_SHIFT 24 -#define CFG_RST_L_TPC_0_SHIFT 25 -#define CFG_RST_L_TPC_1_SHIFT 26 -#define CFG_RST_L_TPC_2_SHIFT 27 -#define CFG_RST_L_TPC_3_SHIFT 28 -#define CFG_RST_L_TPC_4_SHIFT 29 -#define CFG_RST_L_TPC_5_SHIFT 30 -#define CFG_RST_L_TPC_6_SHIFT 31 -#define CFG_RST_H_TPC_7_SHIFT 0 -#define CFG_RST_H_MME_0_SHIFT 1 -#define CFG_RST_H_MME_1_SHIFT 2 -#define CFG_RST_H_MME_2_SHIFT 3 -#define CFG_RST_H_MME_3_SHIFT 4 -#define CFG_RST_H_HBM_0_SHIFT 5 -#define CFG_RST_H_HBM_1_SHIFT 6 -#define CFG_RST_H_HBM_2_SHIFT 7 -#define CFG_RST_H_HBM_3_SHIFT 8 -#define CFG_RST_H_NIC_0_SHIFT 9 -#define CFG_RST_H_NIC_1_SHIFT 10 -#define CFG_RST_H_NIC_2_SHIFT 11 -#define CFG_RST_H_NIC_3_SHIFT 12 -#define CFG_RST_H_NIC_4_SHIFT 13 -#define CFG_RST_H_SM_0_SHIFT 14 -#define CFG_RST_H_SM_1_SHIFT 15 -#define CFG_RST_H_SM_2_SHIFT 16 -#define CFG_RST_H_SM_3_SHIFT 17 -#define CFG_RST_H_DMA_0_SHIFT 18 -#define CFG_RST_H_DMA_1_SHIFT 19 -#define CFG_RST_H_CPU_SHIFT 20 -#define CFG_RST_H_MMU_SHIFT 21 - - -#define CFG_RST_H_DMA_MASK ((1 << CFG_RST_H_DMA_0_SHIFT) | \ - (1 << CFG_RST_H_DMA_1_SHIFT)) - -#define CFG_RST_H_CPU_MASK (1 << CFG_RST_H_CPU_SHIFT) -#define CFG_RST_H_MMU_MASK (1 << CFG_RST_H_MMU_SHIFT) - -#define CFG_RST_H_HBM_MASK ((1 << CFG_RST_H_HBM_0_SHIFT) | \ - (1 << CFG_RST_H_HBM_1_SHIFT) | \ - (1 << CFG_RST_H_HBM_2_SHIFT) | \ - (1 << CFG_RST_H_HBM_3_SHIFT)) - -#define CFG_RST_H_NIC_MASK ((1 << CFG_RST_H_NIC_0_SHIFT) | \ - (1 << CFG_RST_H_NIC_1_SHIFT) | \ - (1 << CFG_RST_H_NIC_2_SHIFT) | \ - (1 << CFG_RST_H_NIC_3_SHIFT) | \ - (1 << CFG_RST_H_NIC_4_SHIFT)) - -#define CFG_RST_H_SM_MASK ((1 << CFG_RST_H_SM_0_SHIFT) | \ - (1 << CFG_RST_H_SM_1_SHIFT) | \ - (1 << CFG_RST_H_SM_2_SHIFT) | \ - (1 << CFG_RST_H_SM_3_SHIFT)) - -#define CFG_RST_H_MME_MASK ((1 << CFG_RST_H_MME_0_SHIFT) | \ - (1 << CFG_RST_H_MME_1_SHIFT) | \ - (1 << CFG_RST_H_MME_2_SHIFT) | \ - (1 << CFG_RST_H_MME_3_SHIFT)) - -#define CFG_RST_L_PSOC_MASK (1 << CFG_RST_L_PSOC_SHIFT) - -#define CFG_RST_L_IF_MASK ((1 << CFG_RST_L_IF_0_SHIFT) | \ - (1 << CFG_RST_L_IF_1_SHIFT) | \ - (1 << CFG_RST_L_IF_2_SHIFT) | \ - (1 << CFG_RST_L_IF_3_SHIFT)) - -#define CFG_RST_L_TPC_MASK ((1 << CFG_RST_L_TPC_0_SHIFT) | \ - (1 << CFG_RST_L_TPC_1_SHIFT) | \ - (1 << CFG_RST_L_TPC_2_SHIFT) | \ - (1 << CFG_RST_L_TPC_3_SHIFT) | \ - (1 << CFG_RST_L_TPC_4_SHIFT) | \ - (1 << CFG_RST_L_TPC_5_SHIFT) | \ - (1 << CFG_RST_L_TPC_6_SHIFT)) - -#define CFG_RST_H_TPC_MASK (1 << CFG_RST_H_TPC_7_SHIFT) - -#define CA53_RESET (1 << CFG_RST_H_CPU_SHIFT) +#define CFG_RST_L_PSOC_MASK BIT_MASK(0) +#define CFG_RST_L_PCIE_MASK BIT_MASK(1) +#define CFG_RST_L_PCIE_IF_MASK BIT_MASK(2) +#define CFG_RST_L_HBM_S_PLL_MASK BIT_MASK(3) +#define CFG_RST_L_TPC_S_PLL_MASK BIT_MASK(4) +#define CFG_RST_L_MME_S_PLL_MASK BIT_MASK(5) +#define CFG_RST_L_CPU_PLL_MASK BIT_MASK(6) +#define CFG_RST_L_PCIE_PLL_MASK BIT_MASK(7) +#define CFG_RST_L_NIC_S_PLL_MASK BIT_MASK(8) +#define CFG_RST_L_HBM_N_PLL_MASK BIT_MASK(9) +#define CFG_RST_L_TPC_N_PLL_MASK BIT_MASK(10) +#define CFG_RST_L_MME_N_PLL_MASK BIT_MASK(11) +#define CFG_RST_L_NIC_N_PLL_MASK BIT_MASK(12) +#define CFG_RST_L_DMA_W_PLL_MASK BIT_MASK(13) +#define CFG_RST_L_SIF_W_PLL_MASK BIT_MASK(14) +#define CFG_RST_L_MESH_W_PLL_MASK BIT_MASK(15) +#define CFG_RST_L_SRAM_W_PLL_MASK BIT_MASK(16) +#define CFG_RST_L_DMA_E_PLL_MASK BIT_MASK(17) +#define CFG_RST_L_SIF_E_PLL_MASK BIT_MASK(18) +#define CFG_RST_L_MESH_E_PLL_MASK BIT_MASK(19) +#define CFG_RST_L_SRAM_E_PLL_MASK BIT_MASK(20) + +#define CFG_RST_L_IF_1_MASK BIT_MASK(21) +#define CFG_RST_L_IF_0_MASK BIT_MASK(22) +#define CFG_RST_L_IF_2_MASK BIT_MASK(23) +#define CFG_RST_L_IF_3_MASK BIT_MASK(24) +#define CFG_RST_L_IF_MASK GENMASK(24, 21) + +#define CFG_RST_L_TPC_0_MASK BIT_MASK(25) +#define CFG_RST_L_TPC_1_MASK BIT_MASK(26) +#define CFG_RST_L_TPC_2_MASK BIT_MASK(27) +#define CFG_RST_L_TPC_3_MASK BIT_MASK(28) +#define CFG_RST_L_TPC_4_MASK BIT_MASK(29) +#define CFG_RST_L_TPC_5_MASK BIT_MASK(30) +#define CFG_RST_L_TPC_6_MASK BIT_MASK(31) +#define CFG_RST_L_TPC_MASK GENMASK(31, 25) + +#define CFG_RST_H_TPC_7_MASK BIT_MASK(0) + +#define CFG_RST_H_MME_0_MASK BIT_MASK(1) +#define CFG_RST_H_MME_1_MASK BIT_MASK(2) +#define CFG_RST_H_MME_2_MASK BIT_MASK(3) +#define CFG_RST_H_MME_3_MASK BIT_MASK(4) +#define CFG_RST_H_MME_MASK GENMASK(4, 1) + +#define CFG_RST_H_HBM_0_MASK BIT_MASK(5) +#define CFG_RST_H_HBM_1_MASK BIT_MASK(6) +#define CFG_RST_H_HBM_2_MASK BIT_MASK(7) +#define CFG_RST_H_HBM_3_MASK BIT_MASK(8) +#define CFG_RST_H_HBM_MASK GENMASK(8, 5) + +#define CFG_RST_H_NIC_0_MASK BIT_MASK(9) +#define CFG_RST_H_NIC_1_MASK BIT_MASK(10) +#define CFG_RST_H_NIC_2_MASK BIT_MASK(11) +#define CFG_RST_H_NIC_3_MASK BIT_MASK(12) +#define CFG_RST_H_NIC_4_MASK BIT_MASK(13) +#define CFG_RST_H_NIC_MASK GENMASK(13, 9) + +#define CFG_RST_H_SM_0_MASK BIT_MASK(14) +#define CFG_RST_H_SM_1_MASK BIT_MASK(15) +#define CFG_RST_H_SM_2_MASK BIT_MASK(16) +#define CFG_RST_H_SM_3_MASK BIT_MASK(17) +#define CFG_RST_H_SM_MASK GENMASK(17, 14) + +#define CFG_RST_H_DMA_0_MASK BIT_MASK(18) +#define CFG_RST_H_DMA_1_MASK BIT_MASK(19) +#define CFG_RST_H_DMA_MASK GENMASK(19, 18) + +#define CFG_RST_H_CPU_MASK BIT_MASK(20) +#define CFG_RST_H_MMU_MASK BIT_MASK(21) #define UNIT_RST_L_PSOC_SHIFT 0 #define UNIT_RST_L_PCIE_SHIFT 1 diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h b/drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h index f25c60a2c243..977fb341a6e7 100644 --- a/drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h +++ b/drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h @@ -12,6 +12,7 @@ * PSOC scratch-pad registers */ #define mmHW_STATE mmPSOC_GLOBAL_CONF_SCRATCHPAD_0 +#define mmFUSE_VER_OFFSET mmPSOC_GLOBAL_CONF_SCRATCHPAD_22 #define mmCPU_CMD_STATUS_TO_HOST mmPSOC_GLOBAL_CONF_SCRATCHPAD_23 #define mmCPU_BOOT_ERR0 mmPSOC_GLOBAL_CONF_SCRATCHPAD_24 #define mmCPU_BOOT_ERR1 mmPSOC_GLOBAL_CONF_SCRATCHPAD_25 diff --git a/drivers/misc/habanalabs/include/goya/goya_reg_map.h b/drivers/misc/habanalabs/include/goya/goya_reg_map.h index 0195f62d7254..e56124265a05 100644 --- a/drivers/misc/habanalabs/include/goya/goya_reg_map.h +++ b/drivers/misc/habanalabs/include/goya/goya_reg_map.h @@ -22,6 +22,7 @@ #define mmCPU_CQ_BASE_ADDR_LOW mmPSOC_GLOBAL_CONF_SCRATCHPAD_8 #define mmCPU_CQ_BASE_ADDR_HIGH mmPSOC_GLOBAL_CONF_SCRATCHPAD_9 #define mmCPU_CQ_LENGTH mmPSOC_GLOBAL_CONF_SCRATCHPAD_10 +#define mmFUSE_VER_OFFSET mmPSOC_GLOBAL_CONF_SCRATCHPAD_22 #define mmCPU_CMD_STATUS_TO_HOST mmPSOC_GLOBAL_CONF_SCRATCHPAD_23 #define mmCPU_BOOT_ERR0 mmPSOC_GLOBAL_CONF_SCRATCHPAD_24 #define mmCPU_BOOT_ERR1 mmPSOC_GLOBAL_CONF_SCRATCHPAD_25 diff --git a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h b/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h index 468bb045fbd1..dedf20e8f956 100644 --- a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h +++ b/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h @@ -29,6 +29,8 @@ #define HOP3_SHIFT 21 #define HOP4_SHIFT 12 +#define MMU_ARCH_5_HOPS 5 + #define HOP_PHYS_ADDR_MASK (~FLAGS_MASK) #define HL_PTE_SIZE sizeof(u64) diff --git a/drivers/misc/hisi_hikey_usb.c b/drivers/misc/hisi_hikey_usb.c new file mode 100644 index 000000000000..cc93569e601c --- /dev/null +++ b/drivers/misc/hisi_hikey_usb.c @@ -0,0 +1,273 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Support for usb functionality of Hikey series boards + * based on Hisilicon Kirin Soc. + * + * Copyright (C) 2017-2018 Hilisicon Electronics Co., Ltd. + * http://www.huawei.com + * + * Authors: Yu Chen <chenyu56@huawei.com> + */ + +#include <linux/gpio/consumer.h> +#include <linux/kernel.h> +#include <linux/mod_devicetable.h> +#include <linux/module.h> +#include <linux/notifier.h> +#include <linux/of_gpio.h> +#include <linux/platform_device.h> +#include <linux/property.h> +#include <linux/regulator/consumer.h> +#include <linux/slab.h> +#include <linux/usb/role.h> + +#define DEVICE_DRIVER_NAME "hisi_hikey_usb" + +#define HUB_VBUS_POWER_ON 1 +#define HUB_VBUS_POWER_OFF 0 +#define USB_SWITCH_TO_HUB 1 +#define USB_SWITCH_TO_TYPEC 0 +#define TYPEC_VBUS_POWER_ON 1 +#define TYPEC_VBUS_POWER_OFF 0 + +struct hisi_hikey_usb { + struct device *dev; + struct gpio_desc *otg_switch; + struct gpio_desc *typec_vbus; + struct gpio_desc *hub_vbus; + struct gpio_desc *reset; + + struct regulator *regulator; + + struct usb_role_switch *hub_role_sw; + + struct usb_role_switch *dev_role_sw; + enum usb_role role; + + struct mutex lock; + struct work_struct work; + + struct notifier_block nb; +}; + +static void hub_power_ctrl(struct hisi_hikey_usb *hisi_hikey_usb, int value) +{ + int ret, status; + + if (hisi_hikey_usb->hub_vbus) + gpiod_set_value_cansleep(hisi_hikey_usb->hub_vbus, value); + + if (!hisi_hikey_usb->regulator) + return; + + status = regulator_is_enabled(hisi_hikey_usb->regulator); + if (status == !!value) + return; + + if (value) + ret = regulator_enable(hisi_hikey_usb->regulator); + else + ret = regulator_disable(hisi_hikey_usb->regulator); + + if (ret) + dev_err(hisi_hikey_usb->dev, + "Can't switch regulator state to %s\n", + value ? "enabled" : "disabled"); +} + +static void usb_switch_ctrl(struct hisi_hikey_usb *hisi_hikey_usb, + int switch_to) +{ + if (!hisi_hikey_usb->otg_switch) + return; + + gpiod_set_value_cansleep(hisi_hikey_usb->otg_switch, switch_to); +} + +static void usb_typec_power_ctrl(struct hisi_hikey_usb *hisi_hikey_usb, + int value) +{ + if (!hisi_hikey_usb->typec_vbus) + return; + + gpiod_set_value_cansleep(hisi_hikey_usb->typec_vbus, value); +} + +static void relay_set_role_switch(struct work_struct *work) +{ + struct hisi_hikey_usb *hisi_hikey_usb = container_of(work, + struct hisi_hikey_usb, + work); + struct usb_role_switch *sw; + enum usb_role role; + + if (!hisi_hikey_usb || !hisi_hikey_usb->dev_role_sw) + return; + + mutex_lock(&hisi_hikey_usb->lock); + switch (hisi_hikey_usb->role) { + case USB_ROLE_NONE: + usb_typec_power_ctrl(hisi_hikey_usb, TYPEC_VBUS_POWER_OFF); + usb_switch_ctrl(hisi_hikey_usb, USB_SWITCH_TO_HUB); + hub_power_ctrl(hisi_hikey_usb, HUB_VBUS_POWER_ON); + break; + case USB_ROLE_HOST: + hub_power_ctrl(hisi_hikey_usb, HUB_VBUS_POWER_OFF); + usb_switch_ctrl(hisi_hikey_usb, USB_SWITCH_TO_TYPEC); + usb_typec_power_ctrl(hisi_hikey_usb, TYPEC_VBUS_POWER_ON); + break; + case USB_ROLE_DEVICE: + hub_power_ctrl(hisi_hikey_usb, HUB_VBUS_POWER_OFF); + usb_typec_power_ctrl(hisi_hikey_usb, TYPEC_VBUS_POWER_OFF); + usb_switch_ctrl(hisi_hikey_usb, USB_SWITCH_TO_TYPEC); + break; + default: + break; + } + sw = hisi_hikey_usb->dev_role_sw; + role = hisi_hikey_usb->role; + mutex_unlock(&hisi_hikey_usb->lock); + + usb_role_switch_set_role(sw, role); +} + +static int hub_usb_role_switch_set(struct usb_role_switch *sw, enum usb_role role) +{ + struct hisi_hikey_usb *hisi_hikey_usb = usb_role_switch_get_drvdata(sw); + + if (!hisi_hikey_usb || !hisi_hikey_usb->dev_role_sw) + return -EINVAL; + + mutex_lock(&hisi_hikey_usb->lock); + hisi_hikey_usb->role = role; + mutex_unlock(&hisi_hikey_usb->lock); + + schedule_work(&hisi_hikey_usb->work); + + return 0; +} + +static int hisi_hikey_usb_parse_kirin970(struct platform_device *pdev, + struct hisi_hikey_usb *hisi_hikey_usb) +{ + struct regulator *regulator; + + regulator = devm_regulator_get(&pdev->dev, "hub-vdd"); + if (IS_ERR(regulator)) { + if (PTR_ERR(regulator) == -EPROBE_DEFER) { + dev_info(&pdev->dev, + "waiting for hub-vdd-supply to be probed\n"); + return PTR_ERR(regulator); + } + dev_err(&pdev->dev, + "get hub-vdd-supply failed with error %ld\n", + PTR_ERR(regulator)); + return PTR_ERR(regulator); + } + hisi_hikey_usb->regulator = regulator; + + hisi_hikey_usb->reset = devm_gpiod_get(&pdev->dev, "hub_reset_en_gpio", + GPIOD_OUT_HIGH); + if (IS_ERR(hisi_hikey_usb->reset)) + return PTR_ERR(hisi_hikey_usb->reset); + + return 0; +} + +static int hisi_hikey_usb_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct hisi_hikey_usb *hisi_hikey_usb; + struct usb_role_switch_desc hub_role_switch = {NULL}; + int ret; + + hisi_hikey_usb = devm_kzalloc(dev, sizeof(*hisi_hikey_usb), GFP_KERNEL); + if (!hisi_hikey_usb) + return -ENOMEM; + + hisi_hikey_usb->dev = &pdev->dev; + + hisi_hikey_usb->otg_switch = devm_gpiod_get(dev, "otg-switch", + GPIOD_OUT_HIGH); + if (IS_ERR(hisi_hikey_usb->otg_switch)) + return PTR_ERR(hisi_hikey_usb->otg_switch); + + hisi_hikey_usb->typec_vbus = devm_gpiod_get(dev, "typec-vbus", + GPIOD_OUT_LOW); + if (IS_ERR(hisi_hikey_usb->typec_vbus)) + return PTR_ERR(hisi_hikey_usb->typec_vbus); + + /* Parse Kirin 970-specific OF data */ + if (of_device_is_compatible(pdev->dev.of_node, + "hisilicon,kirin970_hikey_usbhub")) { + ret = hisi_hikey_usb_parse_kirin970(pdev, hisi_hikey_usb); + if (ret) + return ret; + } else { + /* hub-vdd33-en is optional */ + hisi_hikey_usb->hub_vbus = devm_gpiod_get_optional(dev, "hub-vdd33-en", + GPIOD_OUT_HIGH); + if (IS_ERR(hisi_hikey_usb->hub_vbus)) + return PTR_ERR(hisi_hikey_usb->hub_vbus); + } + + hisi_hikey_usb->dev_role_sw = usb_role_switch_get(dev); + if (!hisi_hikey_usb->dev_role_sw) + return -EPROBE_DEFER; + if (IS_ERR(hisi_hikey_usb->dev_role_sw)) + return PTR_ERR(hisi_hikey_usb->dev_role_sw); + + INIT_WORK(&hisi_hikey_usb->work, relay_set_role_switch); + mutex_init(&hisi_hikey_usb->lock); + + hub_role_switch.fwnode = dev_fwnode(dev); + hub_role_switch.set = hub_usb_role_switch_set; + hub_role_switch.driver_data = hisi_hikey_usb; + + hisi_hikey_usb->hub_role_sw = usb_role_switch_register(dev, + &hub_role_switch); + + if (IS_ERR(hisi_hikey_usb->hub_role_sw)) { + usb_role_switch_put(hisi_hikey_usb->dev_role_sw); + return PTR_ERR(hisi_hikey_usb->hub_role_sw); + } + + platform_set_drvdata(pdev, hisi_hikey_usb); + + return 0; +} + +static int hisi_hikey_usb_remove(struct platform_device *pdev) +{ + struct hisi_hikey_usb *hisi_hikey_usb = platform_get_drvdata(pdev); + + if (hisi_hikey_usb->hub_role_sw) + usb_role_switch_unregister(hisi_hikey_usb->hub_role_sw); + + if (hisi_hikey_usb->dev_role_sw) + usb_role_switch_put(hisi_hikey_usb->dev_role_sw); + + return 0; +} + +static const struct of_device_id id_table_hisi_hikey_usb[] = { + { .compatible = "hisilicon,gpio_hubv1" }, + { .compatible = "hisilicon,kirin970_hikey_usbhub" }, + {} +}; +MODULE_DEVICE_TABLE(of, id_table_hisi_hikey_usb); + +static struct platform_driver hisi_hikey_usb_driver = { + .probe = hisi_hikey_usb_probe, + .remove = hisi_hikey_usb_remove, + .driver = { + .name = DEVICE_DRIVER_NAME, + .of_match_table = id_table_hisi_hikey_usb, + }, +}; + +module_platform_driver(hisi_hikey_usb_driver); + +MODULE_AUTHOR("Yu Chen <chenyu56@huawei.com>"); +MODULE_DESCRIPTION("Driver Support for USB functionality of Hikey"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/misc/mei/Kconfig b/drivers/misc/mei/Kconfig index f5fd5b786607..c06581ffa7bd 100644 --- a/drivers/misc/mei/Kconfig +++ b/drivers/misc/mei/Kconfig @@ -46,4 +46,14 @@ config INTEL_MEI_TXE Supported SoCs: Intel Bay Trail +config INTEL_MEI_VIRTIO + tristate "Intel MEI interface emulation with virtio framework" + select INTEL_MEI + depends on X86 && PCI && VIRTIO_PCI + help + This module implements mei hw emulation over virtio transport. + The module will be called mei_virtio. + Enable this if your virtual machine supports virtual mei + device over virtio. + source "drivers/misc/mei/hdcp/Kconfig" diff --git a/drivers/misc/mei/Makefile b/drivers/misc/mei/Makefile index f1c76f7ee804..52aefaab5c1b 100644 --- a/drivers/misc/mei/Makefile +++ b/drivers/misc/mei/Makefile @@ -22,6 +22,9 @@ obj-$(CONFIG_INTEL_MEI_TXE) += mei-txe.o mei-txe-objs := pci-txe.o mei-txe-objs += hw-txe.o +obj-$(CONFIG_INTEL_MEI_VIRTIO) += mei-virtio.o +mei-virtio-objs := hw-virtio.o + mei-$(CONFIG_EVENT_TRACING) += mei-trace.o CFLAGS_mei-trace.o = -I$(src) diff --git a/drivers/misc/mei/bus-fixup.c b/drivers/misc/mei/bus-fixup.c index 07ba16d46690..4e30fa98fe7d 100644 --- a/drivers/misc/mei/bus-fixup.c +++ b/drivers/misc/mei/bus-fixup.c @@ -463,6 +463,17 @@ out: dev_dbg(bus->dev, "end of fixup match = %d\n", cldev->do_match); } +/** + * vt_support - enable on bus clients with vtag support + * + * @cldev: me clients device + */ +static void vt_support(struct mei_cl_device *cldev) +{ + if (cldev->me_cl->props.vt_supported == 1) + cldev->do_match = 1; +} + #define MEI_FIXUP(_uuid, _hook) { _uuid, _hook } static struct mei_fixup { @@ -476,6 +487,7 @@ static struct mei_fixup { MEI_FIXUP(MEI_UUID_WD, mei_wd), MEI_FIXUP(MEI_UUID_MKHIF_FIX, mei_mkhi_fix), MEI_FIXUP(MEI_UUID_HDCP, whitelist), + MEI_FIXUP(MEI_UUID_ANY, vt_support), }; /** diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c index a6dfc3ce1db2..9cdaa7f3af23 100644 --- a/drivers/misc/mei/bus.c +++ b/drivers/misc/mei/bus.c @@ -152,7 +152,7 @@ ssize_t __mei_cl_recv(struct mei_cl *cl, u8 *buf, size_t length, if (timeout) { rets = wait_event_interruptible_timeout (cl->rx_wait, - (!list_empty(&cl->rd_completed)) || + mei_cl_read_cb(cl, NULL) || (!mei_cl_is_connected(cl)), msecs_to_jiffies(timeout)); if (rets == 0) @@ -165,7 +165,7 @@ ssize_t __mei_cl_recv(struct mei_cl *cl, u8 *buf, size_t length, } else { if (wait_event_interruptible (cl->rx_wait, - (!list_empty(&cl->rd_completed)) || + mei_cl_read_cb(cl, NULL) || (!mei_cl_is_connected(cl)))) { if (signal_pending(current)) return -EINTR; @@ -198,7 +198,7 @@ copy: rets = r_length; free: - mei_io_cb_free(cb); + mei_cl_del_rd_completed(cl, cb); out: mutex_unlock(&bus->device_lock); @@ -496,6 +496,68 @@ static void mei_cl_bus_module_put(struct mei_cl_device *cldev) } /** + * mei_cl_bus_vtag - get bus vtag entry wrapper + * The tag for bus client is always first. + * + * @cl: host client + * + * Return: bus vtag or NULL + */ +static inline struct mei_cl_vtag *mei_cl_bus_vtag(struct mei_cl *cl) +{ + return list_first_entry_or_null(&cl->vtag_map, + struct mei_cl_vtag, list); +} + +/** + * mei_cl_bus_vtag_alloc - add bus client entry to vtag map + * + * @cldev: me client device + * + * Return: + * * 0 on success + * * -ENOMEM if memory allocation failed + */ +static int mei_cl_bus_vtag_alloc(struct mei_cl_device *cldev) +{ + struct mei_cl *cl = cldev->cl; + struct mei_cl_vtag *cl_vtag; + + /* + * Bail out if the client does not supports vtags + * or has already allocated one + */ + if (mei_cl_vt_support_check(cl) || mei_cl_bus_vtag(cl)) + return 0; + + cl_vtag = mei_cl_vtag_alloc(NULL, 0); + if (IS_ERR(cl_vtag)) + return -ENOMEM; + + list_add_tail(&cl_vtag->list, &cl->vtag_map); + + return 0; +} + +/** + * mei_cl_bus_vtag_free - remove the bus entry from vtag map + * + * @cldev: me client device + */ +static void mei_cl_bus_vtag_free(struct mei_cl_device *cldev) +{ + struct mei_cl *cl = cldev->cl; + struct mei_cl_vtag *cl_vtag; + + cl_vtag = mei_cl_bus_vtag(cl); + if (!cl_vtag) + return; + + list_del(&cl_vtag->list); + kfree(cl_vtag); +} + +/** * mei_cldev_enable - enable me client device * create connection with me client * @@ -531,9 +593,15 @@ int mei_cldev_enable(struct mei_cl_device *cldev) goto out; } + ret = mei_cl_bus_vtag_alloc(cldev); + if (ret) + goto out; + ret = mei_cl_connect(cl, cldev->me_cl, NULL); - if (ret < 0) + if (ret < 0) { dev_err(&cldev->dev, "cannot connect\n"); + mei_cl_bus_vtag_free(cldev); + } out: mutex_unlock(&bus->device_lock); @@ -586,6 +654,8 @@ int mei_cldev_disable(struct mei_cl_device *cldev) mutex_lock(&bus->device_lock); + mei_cl_bus_vtag_free(cldev); + if (!mei_cl_is_connected(cl)) { dev_dbg(bus->dev, "Already disconnected\n"); err = 0; @@ -810,6 +880,16 @@ static ssize_t fixed_show(struct device *dev, struct device_attribute *a, } static DEVICE_ATTR_RO(fixed); +static ssize_t vtag_show(struct device *dev, struct device_attribute *a, + char *buf) +{ + struct mei_cl_device *cldev = to_mei_cl_device(dev); + bool vt = mei_me_cl_vt(cldev->me_cl); + + return sprintf(buf, "%d", vt); +} +static DEVICE_ATTR_RO(vtag); + static ssize_t max_len_show(struct device *dev, struct device_attribute *a, char *buf) { @@ -827,6 +907,7 @@ static struct attribute *mei_cldev_attrs[] = { &dev_attr_modalias.attr, &dev_attr_max_conn.attr, &dev_attr_fixed.attr, + &dev_attr_vtag.attr, &dev_attr_max_len.attr, NULL, }; diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c index 2572887d99b6..d5c3f7d54634 100644 --- a/drivers/misc/mei/client.c +++ b/drivers/misc/mei/client.c @@ -355,6 +355,27 @@ static inline void mei_tx_cb_dequeue(struct mei_cl_cb *cb) } /** + * mei_cl_set_read_by_fp - set pending_read flag to vtag struct for given fp + * + * Locking: called under "dev->device_lock" lock + * + * @cl: mei client + * @fp: pointer to file structure + */ +static void mei_cl_set_read_by_fp(const struct mei_cl *cl, + const struct file *fp) +{ + struct mei_cl_vtag *cl_vtag; + + list_for_each_entry(cl_vtag, &cl->vtag_map, list) { + if (cl_vtag->fp == fp) { + cl_vtag->pending_read = true; + return; + } + } +} + +/** * mei_io_cb_init - allocate and initialize io callback * * @cl: mei client @@ -378,6 +399,8 @@ static struct mei_cl_cb *mei_io_cb_init(struct mei_cl *cl, cb->cl = cl; cb->buf_idx = 0; cb->fop_type = type; + cb->vtag = 0; + return cb; } @@ -406,14 +429,16 @@ static void mei_io_list_flush_cl(struct list_head *head, * * @head: An instance of our list structure * @cl: host client + * @fp: file pointer (matching cb file object), may be NULL */ static void mei_io_tx_list_free_cl(struct list_head *head, - const struct mei_cl *cl) + const struct mei_cl *cl, + const struct file *fp) { struct mei_cl_cb *cb, *next; list_for_each_entry_safe(cb, next, head, list) { - if (cl == cb->cl) + if (cl == cb->cl && (!fp || fp == cb->fp)) mei_tx_cb_dequeue(cb); } } @@ -434,6 +459,19 @@ static void mei_io_list_free_fp(struct list_head *head, const struct file *fp) } /** + * mei_cl_free_pending - free pending cb + * + * @cl: host client + */ +static void mei_cl_free_pending(struct mei_cl *cl) +{ + struct mei_cl_cb *cb; + + cb = list_first_entry_or_null(&cl->rd_pending, struct mei_cl_cb, list); + mei_io_cb_free(cb); +} + +/** * mei_cl_alloc_cb - a convenient wrapper for allocating read cb * * @cl: host client @@ -505,15 +543,19 @@ struct mei_cl_cb *mei_cl_enqueue_ctrl_wr_cb(struct mei_cl *cl, size_t length, * * Return: cb on success, NULL if cb is not found */ -struct mei_cl_cb *mei_cl_read_cb(const struct mei_cl *cl, const struct file *fp) +struct mei_cl_cb *mei_cl_read_cb(struct mei_cl *cl, const struct file *fp) { struct mei_cl_cb *cb; + struct mei_cl_cb *ret_cb = NULL; + spin_lock(&cl->rd_completed_lock); list_for_each_entry(cb, &cl->rd_completed, list) - if (!fp || fp == cb->fp) - return cb; - - return NULL; + if (!fp || fp == cb->fp) { + ret_cb = cb; + break; + } + spin_unlock(&cl->rd_completed_lock); + return ret_cb; } /** @@ -534,12 +576,17 @@ int mei_cl_flush_queues(struct mei_cl *cl, const struct file *fp) dev = cl->dev; cl_dbg(dev, cl, "remove list entry belonging to cl\n"); - mei_io_tx_list_free_cl(&cl->dev->write_list, cl); - mei_io_tx_list_free_cl(&cl->dev->write_waiting_list, cl); - mei_io_list_flush_cl(&cl->dev->ctrl_wr_list, cl); - mei_io_list_flush_cl(&cl->dev->ctrl_rd_list, cl); - mei_io_list_free_fp(&cl->rd_pending, fp); + mei_io_tx_list_free_cl(&cl->dev->write_list, cl, fp); + mei_io_tx_list_free_cl(&cl->dev->write_waiting_list, cl, fp); + /* free pending and control cb only in final flush */ + if (!fp) { + mei_io_list_flush_cl(&cl->dev->ctrl_wr_list, cl); + mei_io_list_flush_cl(&cl->dev->ctrl_rd_list, cl); + mei_cl_free_pending(cl); + } + spin_lock(&cl->rd_completed_lock); mei_io_list_free_fp(&cl->rd_completed, fp); + spin_unlock(&cl->rd_completed_lock); return 0; } @@ -557,6 +604,8 @@ static void mei_cl_init(struct mei_cl *cl, struct mei_device *dev) init_waitqueue_head(&cl->rx_wait); init_waitqueue_head(&cl->tx_wait); init_waitqueue_head(&cl->ev_wait); + INIT_LIST_HEAD(&cl->vtag_map); + spin_lock_init(&cl->rd_completed_lock); INIT_LIST_HEAD(&cl->rd_completed); INIT_LIST_HEAD(&cl->rd_pending); INIT_LIST_HEAD(&cl->link); @@ -752,8 +801,8 @@ static void mei_cl_set_disconnected(struct mei_cl *cl) return; cl->state = MEI_FILE_DISCONNECTED; - mei_io_tx_list_free_cl(&dev->write_list, cl); - mei_io_tx_list_free_cl(&dev->write_waiting_list, cl); + mei_io_tx_list_free_cl(&dev->write_list, cl, NULL); + mei_io_tx_list_free_cl(&dev->write_waiting_list, cl, NULL); mei_io_list_flush_cl(&dev->ctrl_rd_list, cl); mei_io_list_flush_cl(&dev->ctrl_wr_list, cl); mei_cl_wake_all(cl); @@ -1229,6 +1278,157 @@ static int mei_cl_tx_flow_ctrl_creds_reduce(struct mei_cl *cl) } /** + * mei_cl_vtag_alloc - allocate and fill the vtag structure + * + * @fp: pointer to file structure + * @vtag: vm tag + * + * Return: + * * Pointer to allocated struct - on success + * * ERR_PTR(-ENOMEM) on memory allocation failure + */ +struct mei_cl_vtag *mei_cl_vtag_alloc(struct file *fp, u8 vtag) +{ + struct mei_cl_vtag *cl_vtag; + + cl_vtag = kzalloc(sizeof(*cl_vtag), GFP_KERNEL); + if (!cl_vtag) + return ERR_PTR(-ENOMEM); + + INIT_LIST_HEAD(&cl_vtag->list); + cl_vtag->vtag = vtag; + cl_vtag->fp = fp; + + return cl_vtag; +} + +/** + * mei_cl_fp_by_vtag - obtain the file pointer by vtag + * + * @cl: host client + * @vtag: vm tag + * + * Return: + * * A file pointer - on success + * * ERR_PTR(-ENOENT) if vtag is not found in the client vtag list + */ +const struct file *mei_cl_fp_by_vtag(const struct mei_cl *cl, u8 vtag) +{ + struct mei_cl_vtag *vtag_l; + + list_for_each_entry(vtag_l, &cl->vtag_map, list) + if (vtag_l->vtag == vtag) + return vtag_l->fp; + + return ERR_PTR(-ENOENT); +} + +/** + * mei_cl_reset_read_by_vtag - reset pending_read flag by given vtag + * + * @cl: host client + * @vtag: vm tag + */ +static void mei_cl_reset_read_by_vtag(const struct mei_cl *cl, u8 vtag) +{ + struct mei_cl_vtag *vtag_l; + + list_for_each_entry(vtag_l, &cl->vtag_map, list) { + if (vtag_l->vtag == vtag) { + vtag_l->pending_read = false; + break; + } + } +} + +/** + * mei_cl_read_vtag_add_fc - add flow control for next pending reader + * in the vtag list + * + * @cl: host client + */ +static void mei_cl_read_vtag_add_fc(struct mei_cl *cl) +{ + struct mei_cl_vtag *cl_vtag; + + list_for_each_entry(cl_vtag, &cl->vtag_map, list) { + if (cl_vtag->pending_read) { + if (mei_cl_enqueue_ctrl_wr_cb(cl, + mei_cl_mtu(cl), + MEI_FOP_READ, + cl_vtag->fp)) + cl->rx_flow_ctrl_creds++; + break; + } + } +} + +/** + * mei_cl_vt_support_check - check if client support vtags + * + * @cl: host client + * + * Return: + * * 0 - supported, or not connected at all + * * -EOPNOTSUPP - vtags are not supported by client + */ +int mei_cl_vt_support_check(const struct mei_cl *cl) +{ + struct mei_device *dev = cl->dev; + + if (!dev->hbm_f_vt_supported) + return -EOPNOTSUPP; + + if (!cl->me_cl) + return 0; + + return cl->me_cl->props.vt_supported ? 0 : -EOPNOTSUPP; +} + +/** + * mei_cl_add_rd_completed - add read completed callback to list with lock + * and vtag check + * + * @cl: host client + * @cb: callback block + * + */ +void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) +{ + const struct file *fp; + + if (!mei_cl_vt_support_check(cl)) { + fp = mei_cl_fp_by_vtag(cl, cb->vtag); + if (IS_ERR(fp)) { + /* client already disconnected, discarding */ + mei_io_cb_free(cb); + return; + } + cb->fp = fp; + mei_cl_reset_read_by_vtag(cl, cb->vtag); + mei_cl_read_vtag_add_fc(cl); + } + + spin_lock(&cl->rd_completed_lock); + list_add_tail(&cb->list, &cl->rd_completed); + spin_unlock(&cl->rd_completed_lock); +} + +/** + * mei_cl_del_rd_completed - free read completed callback with lock + * + * @cl: host client + * @cb: callback block + * + */ +void mei_cl_del_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) +{ + spin_lock(&cl->rd_completed_lock); + mei_io_cb_free(cb); + spin_unlock(&cl->rd_completed_lock); +} + +/** * mei_cl_notify_fop2req - convert fop to proper request * * @fop: client notification start response command @@ -1483,13 +1683,17 @@ int mei_cl_read_start(struct mei_cl *cl, size_t length, const struct file *fp) return 0; /* HW currently supports only one pending read */ - if (cl->rx_flow_ctrl_creds) + if (cl->rx_flow_ctrl_creds) { + mei_cl_set_read_by_fp(cl, fp); return -EBUSY; + } cb = mei_cl_enqueue_ctrl_wr_cb(cl, length, MEI_FOP_READ, fp); if (!cb) return -ENOMEM; + mei_cl_set_read_by_fp(cl, fp); + rets = pm_runtime_get(dev->dev); if (rets < 0 && rets != -EINPROGRESS) { pm_runtime_put_noidle(dev->dev); @@ -1518,21 +1722,67 @@ nortpm: return rets; } +static inline u8 mei_ext_hdr_set_vtag(struct mei_ext_hdr *ext, u8 vtag) +{ + ext->type = MEI_EXT_HDR_VTAG; + ext->ext_payload[0] = vtag; + ext->length = mei_data2slots(sizeof(*ext)); + return ext->length; +} + /** - * mei_msg_hdr_init - initialize mei message header + * mei_msg_hdr_init - allocate and initialize mei message header * - * @mei_hdr: mei message header * @cb: message callback structure + * + * Return: a pointer to initialized header */ -static void mei_msg_hdr_init(struct mei_msg_hdr *mei_hdr, struct mei_cl_cb *cb) +static struct mei_msg_hdr *mei_msg_hdr_init(const struct mei_cl_cb *cb) { + size_t hdr_len; + struct mei_ext_meta_hdr *meta; + struct mei_ext_hdr *ext; + struct mei_msg_hdr *mei_hdr; + bool is_ext, is_vtag; + + if (!cb) + return ERR_PTR(-EINVAL); + + /* Extended header for vtag is attached only on the first fragment */ + is_vtag = (cb->vtag && cb->buf_idx == 0); + is_ext = is_vtag; + + /* Compute extended header size */ + hdr_len = sizeof(*mei_hdr); + + if (!is_ext) + goto setup_hdr; + + hdr_len += sizeof(*meta); + if (is_vtag) + hdr_len += sizeof(*ext); + +setup_hdr: + mei_hdr = kzalloc(hdr_len, GFP_KERNEL); + if (!mei_hdr) + return ERR_PTR(-ENOMEM); + mei_hdr->host_addr = mei_cl_host_addr(cb->cl); mei_hdr->me_addr = mei_cl_me_id(cb->cl); - mei_hdr->length = 0; - mei_hdr->reserved = 0; - mei_hdr->msg_complete = 0; - mei_hdr->dma_ring = 0; mei_hdr->internal = cb->internal; + mei_hdr->extended = is_ext; + + if (!is_ext) + goto out; + + meta = (struct mei_ext_meta_hdr *)mei_hdr->extension; + if (is_vtag) { + meta->count++; + meta->size += mei_ext_hdr_set_vtag(meta->hdrs, cb->vtag); + } +out: + mei_hdr->length = hdr_len - sizeof(*mei_hdr); + return mei_hdr; } /** @@ -1550,10 +1800,11 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb, { struct mei_device *dev; struct mei_msg_data *buf; - struct mei_msg_hdr mei_hdr; - size_t hdr_len = sizeof(mei_hdr); - size_t len; + struct mei_msg_hdr *mei_hdr = NULL; + size_t hdr_len; size_t hbuf_len, dr_len; + size_t buf_len; + size_t data_len; int hbuf_slots; u32 dr_slots; u32 dma_len; @@ -1579,7 +1830,7 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb, return 0; } - len = buf->size - cb->buf_idx; + buf_len = buf->size - cb->buf_idx; data = buf->data + cb->buf_idx; hbuf_slots = mei_hbuf_empty_slots(dev); if (hbuf_slots < 0) { @@ -1591,42 +1842,54 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb, dr_slots = mei_dma_ring_empty_slots(dev); dr_len = mei_slots2data(dr_slots); - mei_msg_hdr_init(&mei_hdr, cb); + mei_hdr = mei_msg_hdr_init(cb); + if (IS_ERR(mei_hdr)) { + rets = PTR_ERR(mei_hdr); + mei_hdr = NULL; + goto err; + } + + cl_dbg(dev, cl, "Extended Header %d vtag = %d\n", + mei_hdr->extended, cb->vtag); + + hdr_len = sizeof(*mei_hdr) + mei_hdr->length; /** * Split the message only if we can write the whole host buffer * otherwise wait for next time the host buffer is empty. */ - if (len + hdr_len <= hbuf_len) { - mei_hdr.length = len; - mei_hdr.msg_complete = 1; + if (hdr_len + buf_len <= hbuf_len) { + data_len = buf_len; + mei_hdr->msg_complete = 1; } else if (dr_slots && hbuf_len >= hdr_len + sizeof(dma_len)) { - mei_hdr.dma_ring = 1; - if (len > dr_len) - len = dr_len; + mei_hdr->dma_ring = 1; + if (buf_len > dr_len) + buf_len = dr_len; else - mei_hdr.msg_complete = 1; + mei_hdr->msg_complete = 1; - mei_hdr.length = sizeof(dma_len); - dma_len = len; + data_len = sizeof(dma_len); + dma_len = buf_len; data = &dma_len; } else if ((u32)hbuf_slots == mei_hbuf_depth(dev)) { - len = hbuf_len - hdr_len; - mei_hdr.length = len; + buf_len = hbuf_len - hdr_len; + data_len = buf_len; } else { + kfree(mei_hdr); return 0; } + mei_hdr->length += data_len; - if (mei_hdr.dma_ring) - mei_dma_ring_write(dev, buf->data + cb->buf_idx, len); + if (mei_hdr->dma_ring) + mei_dma_ring_write(dev, buf->data + cb->buf_idx, buf_len); + rets = mei_write_message(dev, mei_hdr, hdr_len, data, data_len); - rets = mei_write_message(dev, &mei_hdr, hdr_len, data, mei_hdr.length); if (rets) goto err; cl->status = 0; cl->writing_state = MEI_WRITING; - cb->buf_idx += len; + cb->buf_idx += buf_len; if (first_chunk) { if (mei_cl_tx_flow_ctrl_creds_reduce(cl)) { @@ -1635,12 +1898,14 @@ int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb, } } - if (mei_hdr.msg_complete) + if (mei_hdr->msg_complete) list_move_tail(&cb->list, &dev->write_waiting_list); + kfree(mei_hdr); return 0; err: + kfree(mei_hdr); cl->status = rets; list_move_tail(&cb->list, cmpl_list); return rets; @@ -1659,9 +1924,11 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb) { struct mei_device *dev; struct mei_msg_data *buf; - struct mei_msg_hdr mei_hdr; - size_t hdr_len = sizeof(mei_hdr); - size_t len, hbuf_len, dr_len; + struct mei_msg_hdr *mei_hdr = NULL; + size_t hdr_len; + size_t hbuf_len, dr_len; + size_t buf_len; + size_t data_len; int hbuf_slots; u32 dr_slots; u32 dma_len; @@ -1678,9 +1945,9 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb) dev = cl->dev; buf = &cb->buf; - len = buf->size; + buf_len = buf->size; - cl_dbg(dev, cl, "len=%zd\n", len); + cl_dbg(dev, cl, "buf_len=%zd\n", buf_len); blocking = cb->blocking; data = buf->data; @@ -1700,17 +1967,27 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb) if (rets < 0) goto err; - mei_msg_hdr_init(&mei_hdr, cb); + mei_hdr = mei_msg_hdr_init(cb); + if (IS_ERR(mei_hdr)) { + rets = -PTR_ERR(mei_hdr); + mei_hdr = NULL; + goto err; + } + + cl_dbg(dev, cl, "Extended Header %d vtag = %d\n", + mei_hdr->extended, cb->vtag); + + hdr_len = sizeof(*mei_hdr) + mei_hdr->length; if (rets == 0) { cl_dbg(dev, cl, "No flow control credentials: not sending.\n"); - rets = len; + rets = buf_len; goto out; } if (!mei_hbuf_acquire(dev)) { cl_dbg(dev, cl, "Cannot acquire the host buffer: not sending.\n"); - rets = len; + rets = buf_len; goto out; } @@ -1724,29 +2001,30 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb) dr_slots = mei_dma_ring_empty_slots(dev); dr_len = mei_slots2data(dr_slots); - if (len + hdr_len <= hbuf_len) { - mei_hdr.length = len; - mei_hdr.msg_complete = 1; + if (hdr_len + buf_len <= hbuf_len) { + data_len = buf_len; + mei_hdr->msg_complete = 1; } else if (dr_slots && hbuf_len >= hdr_len + sizeof(dma_len)) { - mei_hdr.dma_ring = 1; - if (len > dr_len) - len = dr_len; + mei_hdr->dma_ring = 1; + if (buf_len > dr_len) + buf_len = dr_len; else - mei_hdr.msg_complete = 1; + mei_hdr->msg_complete = 1; - mei_hdr.length = sizeof(dma_len); - dma_len = len; + data_len = sizeof(dma_len); + dma_len = buf_len; data = &dma_len; } else { - len = hbuf_len - hdr_len; - mei_hdr.length = len; + buf_len = hbuf_len - hdr_len; + data_len = buf_len; } - if (mei_hdr.dma_ring) - mei_dma_ring_write(dev, buf->data, len); + mei_hdr->length += data_len; + + if (mei_hdr->dma_ring) + mei_dma_ring_write(dev, buf->data, buf_len); + rets = mei_write_message(dev, mei_hdr, hdr_len, data, data_len); - rets = mei_write_message(dev, &mei_hdr, hdr_len, - data, mei_hdr.length); if (rets) goto err; @@ -1755,12 +2033,12 @@ ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb) goto err; cl->writing_state = MEI_WRITING; - cb->buf_idx = len; + cb->buf_idx = buf_len; /* restore return value */ - len = buf->size; + buf_len = buf->size; out: - if (mei_hdr.msg_complete) + if (mei_hdr->msg_complete) mei_tx_cb_enqueue(cb, &dev->write_waiting_list); else mei_tx_cb_enqueue(cb, &dev->write_list); @@ -1785,7 +2063,7 @@ out: } } - rets = len; + rets = buf_len; err: cl_dbg(dev, cl, "rpm: autosuspend\n"); pm_runtime_mark_last_busy(dev->dev); @@ -1793,10 +2071,11 @@ err: free: mei_io_cb_free(cb); + kfree(mei_hdr); + return rets; } - /** * mei_cl_complete - processes completed operation for a client * @@ -1820,7 +2099,7 @@ void mei_cl_complete(struct mei_cl *cl, struct mei_cl_cb *cb) break; case MEI_FOP_READ: - list_add_tail(&cb->list, &cl->rd_completed); + mei_cl_add_rd_completed(cl, cb); if (!mei_cl_is_fixed_address(cl) && !WARN_ON(!cl->rx_flow_ctrl_creds)) cl->rx_flow_ctrl_creds--; diff --git a/drivers/misc/mei/client.h b/drivers/misc/mei/client.h index 2f8954def591..64143d4ec758 100644 --- a/drivers/misc/mei/client.h +++ b/drivers/misc/mei/client.h @@ -94,6 +94,18 @@ static inline u8 mei_me_cl_fixed(const struct mei_me_client *me_cl) } /** + * mei_me_cl_vt - return me client vtag supported status + * + * @me_cl: me client + * + * Return: true if me client supports vt tagging + */ +static inline bool mei_me_cl_vt(const struct mei_me_client *me_cl) +{ + return me_cl->props.vt_supported == 1; +} + +/** * mei_me_cl_max_len - return me client max msg length * * @me_cl: me client @@ -121,8 +133,11 @@ int mei_cl_unlink(struct mei_cl *cl); struct mei_cl *mei_cl_alloc_linked(struct mei_device *dev); -struct mei_cl_cb *mei_cl_read_cb(const struct mei_cl *cl, - const struct file *fp); +struct mei_cl_cb *mei_cl_read_cb(struct mei_cl *cl, const struct file *fp); + +void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb); +void mei_cl_del_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb); + struct mei_cl_cb *mei_cl_alloc_cb(struct mei_cl *cl, size_t length, enum mei_cb_file_ops type, const struct file *fp); @@ -131,6 +146,9 @@ struct mei_cl_cb *mei_cl_enqueue_ctrl_wr_cb(struct mei_cl *cl, size_t length, const struct file *fp); int mei_cl_flush_queues(struct mei_cl *cl, const struct file *fp); +struct mei_cl_vtag *mei_cl_vtag_alloc(struct file *fp, u8 vtag); +const struct file *mei_cl_fp_by_vtag(const struct mei_cl *cl, u8 vtag); +int mei_cl_vt_support_check(const struct mei_cl *cl); /* * MEI input output function prototype */ diff --git a/drivers/misc/mei/debugfs.c b/drivers/misc/mei/debugfs.c index a26c716c61a1..3ab1a431d810 100644 --- a/drivers/misc/mei/debugfs.c +++ b/drivers/misc/mei/debugfs.c @@ -27,7 +27,7 @@ static int mei_dbgfs_meclients_show(struct seq_file *m, void *unused) down_read(&dev->me_clients_rwsem); - seq_puts(m, " |id|fix| UUID |con|msg len|sb|refc|\n"); + seq_puts(m, " |id|fix| UUID |con|msg len|sb|refc|vt|\n"); /* if the driver is not enabled the list won't be consistent */ if (dev->dev_state != MEI_DEV_ENABLED) @@ -37,14 +37,15 @@ static int mei_dbgfs_meclients_show(struct seq_file *m, void *unused) if (!mei_me_cl_get(me_cl)) continue; - seq_printf(m, "%2d|%2d|%3d|%pUl|%3d|%7d|%2d|%4d|\n", + seq_printf(m, "%2d|%2d|%3d|%pUl|%3d|%7d|%2d|%4d|%2d|\n", i++, me_cl->client_id, me_cl->props.fixed_address, &me_cl->props.protocol_name, me_cl->props.max_number_of_connections, me_cl->props.max_msg_length, me_cl->props.single_recv_buf, - kref_read(&me_cl->refcnt)); + kref_read(&me_cl->refcnt), + me_cl->props.vt_supported); mei_me_cl_put(me_cl); } @@ -103,6 +104,8 @@ static int mei_dbgfs_devstate_show(struct seq_file *m, void *unused) seq_printf(m, "\tFA: %01d\n", dev->hbm_f_fa_supported); seq_printf(m, "\tOS: %01d\n", dev->hbm_f_os_supported); seq_printf(m, "\tDR: %01d\n", dev->hbm_f_dr_supported); + seq_printf(m, "\tVT: %01d\n", dev->hbm_f_vt_supported); + seq_printf(m, "\tCAP: %01d\n", dev->hbm_f_cap_supported); } seq_printf(m, "pg: %s, %s\n", diff --git a/drivers/misc/mei/hbm.c b/drivers/misc/mei/hbm.c index 308caee86920..a97eb5d47705 100644 --- a/drivers/misc/mei/hbm.c +++ b/drivers/misc/mei/hbm.c @@ -125,19 +125,15 @@ void mei_hbm_reset(struct mei_device *dev) /** * mei_hbm_hdr - construct hbm header * - * @hdr: hbm header + * @mei_hdr: hbm header * @length: payload length */ -static inline void mei_hbm_hdr(struct mei_msg_hdr *hdr, size_t length) +static inline void mei_hbm_hdr(struct mei_msg_hdr *mei_hdr, size_t length) { - hdr->host_addr = 0; - hdr->me_addr = 0; - hdr->length = length; - hdr->msg_complete = 1; - hdr->dma_ring = 0; - hdr->reserved = 0; - hdr->internal = 0; + memset(mei_hdr, 0, sizeof(*mei_hdr)); + mei_hdr->length = length; + mei_hdr->msg_complete = 1; } /** @@ -326,6 +322,39 @@ static int mei_hbm_dma_setup_req(struct mei_device *dev) } /** + * mei_hbm_capabilities_req - request capabilities + * + * @dev: the device structure + * + * Return: 0 on success and < 0 on failure + */ +static int mei_hbm_capabilities_req(struct mei_device *dev) +{ + struct mei_msg_hdr mei_hdr; + struct hbm_capability_request req; + int ret; + + mei_hbm_hdr(&mei_hdr, sizeof(req)); + + memset(&req, 0, sizeof(req)); + req.hbm_cmd = MEI_HBM_CAPABILITIES_REQ_CMD; + if (dev->hbm_f_vt_supported) + req.capability_requested[0] = HBM_CAP_VT; + + ret = mei_hbm_write_message(dev, &mei_hdr, &req); + if (ret) { + dev_err(dev->dev, + "capabilities request write failed: ret = %d.\n", ret); + return ret; + } + + dev->hbm_state = MEI_HBM_CAP_SETUP; + dev->init_clients_timer = MEI_CLIENTS_INIT_TIMEOUT; + mei_schedule_stall_timer(dev); + return 0; +} + +/** * mei_hbm_enum_clients_req - sends enumeration client request message. * * @dev: the device structure @@ -1042,6 +1071,20 @@ static void mei_hbm_config_features(struct mei_device *dev) (dev->version.major_version == HBM_MAJOR_VERSION_DR && dev->version.minor_version >= HBM_MINOR_VERSION_DR)) dev->hbm_f_dr_supported = 1; + + /* VTag Support */ + dev->hbm_f_vt_supported = 0; + if (dev->version.major_version > HBM_MAJOR_VERSION_VT || + (dev->version.major_version == HBM_MAJOR_VERSION_VT && + dev->version.minor_version >= HBM_MINOR_VERSION_VT)) + dev->hbm_f_vt_supported = 1; + + /* Capability message Support */ + dev->hbm_f_cap_supported = 0; + if (dev->version.major_version > HBM_MAJOR_VERSION_CAP || + (dev->version.major_version == HBM_MAJOR_VERSION_CAP && + dev->version.minor_version >= HBM_MINOR_VERSION_CAP)) + dev->hbm_f_cap_supported = 1; } /** @@ -1075,6 +1118,7 @@ int mei_hbm_dispatch(struct mei_device *dev, struct mei_msg_hdr *hdr) struct hbm_host_enum_response *enum_res; struct hbm_dma_setup_response *dma_setup_res; struct hbm_add_client_request *add_cl_req; + struct hbm_capability_response *capability_res; int ret; struct mei_hbm_cl_cmd *cl_cmd; @@ -1138,6 +1182,13 @@ int mei_hbm_dispatch(struct mei_device *dev, struct mei_msg_hdr *hdr) return -EPROTO; } + if (dev->hbm_f_cap_supported) { + if (mei_hbm_capabilities_req(dev)) + return -EIO; + wake_up(&dev->wait_hbm_start); + break; + } + if (dev->hbm_f_dr_supported) { if (mei_dmam_ring_alloc(dev)) dev_info(dev->dev, "running w/o dma ring\n"); @@ -1159,6 +1210,38 @@ int mei_hbm_dispatch(struct mei_device *dev, struct mei_msg_hdr *hdr) wake_up(&dev->wait_hbm_start); break; + case MEI_HBM_CAPABILITIES_RES_CMD: + dev_dbg(dev->dev, "hbm: capabilities response: message received.\n"); + + dev->init_clients_timer = 0; + + if (dev->hbm_state != MEI_HBM_CAP_SETUP) { + dev_err(dev->dev, "hbm: capabilities response: state mismatch, [%d, %d]\n", + dev->dev_state, dev->hbm_state); + return -EPROTO; + } + + capability_res = (struct hbm_capability_response *)mei_msg; + if (!(capability_res->capability_granted[0] & HBM_CAP_VT)) + dev->hbm_f_vt_supported = 0; + + if (dev->hbm_f_dr_supported) { + if (mei_dmam_ring_alloc(dev)) + dev_info(dev->dev, "running w/o dma ring\n"); + if (mei_dma_ring_is_allocated(dev)) { + if (mei_hbm_dma_setup_req(dev)) + return -EIO; + break; + } + } + + dev->hbm_f_dr_supported = 0; + mei_dmam_ring_free(dev); + + if (mei_hbm_enum_clients_req(dev)) + return -EIO; + break; + case MEI_HBM_DMA_SETUP_RES_CMD: dev_dbg(dev->dev, "hbm: dma setup response: message received.\n"); diff --git a/drivers/misc/mei/hbm.h b/drivers/misc/mei/hbm.h index 5aa58cffdd2e..4d95e38e4ddf 100644 --- a/drivers/misc/mei/hbm.h +++ b/drivers/misc/mei/hbm.h @@ -16,6 +16,7 @@ struct mei_cl; * * @MEI_HBM_IDLE : protocol not started * @MEI_HBM_STARTING : start request message was sent + * @MEI_HBM_CAP_SETUP : capabilities request message was sent * @MEI_HBM_DR_SETUP : dma ring setup request message was sent * @MEI_HBM_ENUM_CLIENTS : enumeration request was sent * @MEI_HBM_CLIENT_PROPERTIES : acquiring clients properties @@ -25,6 +26,7 @@ struct mei_cl; enum mei_hbm_state { MEI_HBM_IDLE = 0, MEI_HBM_STARTING, + MEI_HBM_CAP_SETUP, MEI_HBM_DR_SETUP, MEI_HBM_ENUM_CLIENTS, MEI_HBM_CLIENT_PROPERTIES, diff --git a/drivers/misc/mei/hw-virtio.c b/drivers/misc/mei/hw-virtio.c new file mode 100644 index 000000000000..899dc1c5e7ca --- /dev/null +++ b/drivers/misc/mei/hw-virtio.c @@ -0,0 +1,874 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Intel Management Engine Interface (Intel MEI) Linux driver + * Copyright (c) 2018-2020, Intel Corporation. + */ +#include <linux/err.h> +#include <linux/module.h> +#include <linux/pm_runtime.h> +#include <linux/scatterlist.h> +#include <linux/spinlock.h> +#include <linux/slab.h> +#include <linux/virtio.h> +#include <linux/virtio_config.h> +#include <linux/virtio_ids.h> +#include <linux/atomic.h> + +#include "mei_dev.h" +#include "hbm.h" +#include "client.h" + +#define MEI_VIRTIO_RPM_TIMEOUT 500 +/* ACRN virtio device types */ +#ifndef VIRTIO_ID_MEI +#define VIRTIO_ID_MEI 0xFFFE /* virtio mei */ +#endif + +/** + * struct mei_virtio_cfg - settings passed from the virtio backend + * @buf_depth: read buffer depth in slots (4bytes) + * @hw_ready: hw is ready for operation + * @host_reset: synchronize reset with virtio backend + * @reserved: reserved for alignment + * @fw_status: FW status + */ +struct mei_virtio_cfg { + u32 buf_depth; + u8 hw_ready; + u8 host_reset; + u8 reserved[2]; + u32 fw_status[MEI_FW_STATUS_MAX]; +} __packed; + +struct mei_virtio_hw { + struct mei_device mdev; + char name[32]; + + struct virtqueue *in; + struct virtqueue *out; + + bool host_ready; + struct work_struct intr_handler; + + u32 *recv_buf; + u8 recv_rdy; + size_t recv_sz; + u32 recv_idx; + u32 recv_len; + + /* send buffer */ + atomic_t hbuf_ready; + const void *send_hdr; + const void *send_buf; + + struct mei_virtio_cfg cfg; +}; + +#define to_virtio_hw(_dev) container_of(_dev, struct mei_virtio_hw, mdev) + +/** + * mei_virtio_fw_status() - read status register of mei + * @dev: mei device + * @fw_status: fw status register values + * + * Return: always 0 + */ +static int mei_virtio_fw_status(struct mei_device *dev, + struct mei_fw_status *fw_status) +{ + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + fw_status->count = MEI_FW_STATUS_MAX; + virtio_cread_bytes(vdev, offsetof(struct mei_virtio_cfg, fw_status), + fw_status->status, sizeof(fw_status->status)); + return 0; +} + +/** + * mei_virtio_pg_state() - translate internal pg state + * to the mei power gating state + * There is no power management in ACRN mode always return OFF + * @dev: mei device + * + * Return: + * * MEI_PG_OFF - if aliveness is on (always) + * * MEI_PG_ON - (never) + */ +static inline enum mei_pg_state mei_virtio_pg_state(struct mei_device *dev) +{ + return MEI_PG_OFF; +} + +/** + * mei_virtio_hw_config() - configure hw dependent settings + * + * @dev: mei device + * + * Return: always 0 + */ +static int mei_virtio_hw_config(struct mei_device *dev) +{ + return 0; +} + +/** + * mei_virtio_hbuf_empty_slots() - counts write empty slots. + * @dev: the device structure + * + * Return: always return frontend buf size if buffer is ready, 0 otherwise + */ +static int mei_virtio_hbuf_empty_slots(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + return (atomic_read(&hw->hbuf_ready) == 1) ? hw->cfg.buf_depth : 0; +} + +/** + * mei_virtio_hbuf_is_ready() - checks if write buffer is ready + * @dev: the device structure + * + * Return: true if hbuf is ready + */ +static bool mei_virtio_hbuf_is_ready(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + return atomic_read(&hw->hbuf_ready) == 1; +} + +/** + * mei_virtio_hbuf_max_depth() - returns depth of FE write buffer. + * @dev: the device structure + * + * Return: size of frontend write buffer in bytes + */ +static u32 mei_virtio_hbuf_depth(const struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + return hw->cfg.buf_depth; +} + +/** + * mei_virtio_intr_clear() - clear and stop interrupts + * @dev: the device structure + */ +static void mei_virtio_intr_clear(struct mei_device *dev) +{ + /* + * In our virtio solution, there are two types of interrupts, + * vq interrupt and config change interrupt. + * 1) start/reset rely on virtio config changed interrupt; + * 2) send/recv rely on virtio virtqueue interrupts. + * They are all virtual interrupts. So, we don't have corresponding + * operation to do here. + */ +} + +/** + * mei_virtio_intr_enable() - enables mei BE virtqueues callbacks + * @dev: the device structure + */ +static void mei_virtio_intr_enable(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + virtio_config_enable(vdev); + + virtqueue_enable_cb(hw->in); + virtqueue_enable_cb(hw->out); +} + +/** + * mei_virtio_intr_disable() - disables mei BE virtqueues callbacks + * + * @dev: the device structure + */ +static void mei_virtio_intr_disable(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + virtio_config_disable(vdev); + + virtqueue_disable_cb(hw->in); + virtqueue_disable_cb(hw->out); +} + +/** + * mei_virtio_synchronize_irq() - wait for pending IRQ handlers for all + * virtqueue + * @dev: the device structure + */ +static void mei_virtio_synchronize_irq(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + /* + * Now, all IRQ handlers are converted to workqueue. + * Change synchronize irq to flush this work. + */ + flush_work(&hw->intr_handler); +} + +static void mei_virtio_free_outbufs(struct mei_virtio_hw *hw) +{ + kfree(hw->send_hdr); + kfree(hw->send_buf); + hw->send_hdr = NULL; + hw->send_buf = NULL; +} + +/** + * mei_virtio_write_message() - writes a message to mei virtio back-end service. + * @dev: the device structure + * @hdr: mei header of message + * @hdr_len: header length + * @data: message payload will be written + * @data_len: message payload length + * + * Return: + * * 0: on success + * * -EIO: if write has failed + * * -ENOMEM: on memory allocation failure + */ +static int mei_virtio_write_message(struct mei_device *dev, + const void *hdr, size_t hdr_len, + const void *data, size_t data_len) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct scatterlist sg[2]; + const void *hbuf, *dbuf; + int ret; + + if (WARN_ON(!atomic_add_unless(&hw->hbuf_ready, -1, 0))) + return -EIO; + + hbuf = kmemdup(hdr, hdr_len, GFP_KERNEL); + hw->send_hdr = hbuf; + + dbuf = kmemdup(data, data_len, GFP_KERNEL); + hw->send_buf = dbuf; + + if (!hbuf || !dbuf) { + ret = -ENOMEM; + goto fail; + } + + sg_init_table(sg, 2); + sg_set_buf(&sg[0], hbuf, hdr_len); + sg_set_buf(&sg[1], dbuf, data_len); + + ret = virtqueue_add_outbuf(hw->out, sg, 2, hw, GFP_KERNEL); + if (ret) { + dev_err(dev->dev, "failed to add outbuf\n"); + goto fail; + } + + virtqueue_kick(hw->out); + return 0; +fail: + + mei_virtio_free_outbufs(hw); + + return ret; +} + +/** + * mei_virtio_count_full_read_slots() - counts read full slots. + * @dev: the device structure + * + * Return: -EOVERFLOW if overflow, otherwise filled slots count + */ +static int mei_virtio_count_full_read_slots(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + if (hw->recv_idx > hw->recv_len) + return -EOVERFLOW; + + return hw->recv_len - hw->recv_idx; +} + +/** + * mei_virtio_read_hdr() - Reads 32bit dword from mei virtio receive buffer + * + * @dev: the device structure + * + * Return: 32bit dword of receive buffer (u32) + */ +static inline u32 mei_virtio_read_hdr(const struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + WARN_ON(hw->cfg.buf_depth < hw->recv_idx + 1); + + return hw->recv_buf[hw->recv_idx++]; +} + +static int mei_virtio_read(struct mei_device *dev, unsigned char *buffer, + unsigned long len) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + u32 slots = mei_data2slots(len); + + if (WARN_ON(hw->cfg.buf_depth < hw->recv_idx + slots)) + return -EOVERFLOW; + + /* + * Assumption: There is only one MEI message in recv_buf each time. + * Backend service need follow this rule too. + */ + memcpy(buffer, hw->recv_buf + hw->recv_idx, len); + hw->recv_idx += slots; + + return 0; +} + +static bool mei_virtio_pg_is_enabled(struct mei_device *dev) +{ + return false; +} + +static bool mei_virtio_pg_in_transition(struct mei_device *dev) +{ + return false; +} + +static void mei_virtio_add_recv_buf(struct mei_virtio_hw *hw) +{ + struct scatterlist sg; + + if (hw->recv_rdy) /* not needed */ + return; + + /* refill the recv_buf to IN virtqueue to get next message */ + sg_init_one(&sg, hw->recv_buf, mei_slots2data(hw->cfg.buf_depth)); + hw->recv_len = 0; + hw->recv_idx = 0; + hw->recv_rdy = 1; + virtqueue_add_inbuf(hw->in, &sg, 1, hw->recv_buf, GFP_KERNEL); + virtqueue_kick(hw->in); +} + +/** + * mei_virtio_hw_is_ready() - check whether the BE(hw) has turned ready + * @dev: mei device + * Return: bool + */ +static bool mei_virtio_hw_is_ready(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + virtio_cread(vdev, struct mei_virtio_cfg, + hw_ready, &hw->cfg.hw_ready); + + dev_dbg(dev->dev, "hw ready %d\n", hw->cfg.hw_ready); + + return hw->cfg.hw_ready; +} + +/** + * mei_virtio_hw_reset - resets virtio hw. + * + * @dev: the device structure + * @intr_enable: virtio use data/config callbacks + * + * Return: 0 on success an error code otherwise + */ +static int mei_virtio_hw_reset(struct mei_device *dev, bool intr_enable) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + dev_dbg(dev->dev, "hw reset\n"); + + dev->recvd_hw_ready = false; + hw->host_ready = false; + atomic_set(&hw->hbuf_ready, 0); + hw->recv_len = 0; + hw->recv_idx = 0; + + hw->cfg.host_reset = 1; + virtio_cwrite(vdev, struct mei_virtio_cfg, + host_reset, &hw->cfg.host_reset); + + mei_virtio_hw_is_ready(dev); + + if (intr_enable) + mei_virtio_intr_enable(dev); + + return 0; +} + +/** + * mei_virtio_hw_reset_release() - release device from the reset + * @dev: the device structure + */ +static void mei_virtio_hw_reset_release(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + struct virtio_device *vdev = dev_to_virtio(dev->dev); + + dev_dbg(dev->dev, "hw reset release\n"); + hw->cfg.host_reset = 0; + virtio_cwrite(vdev, struct mei_virtio_cfg, + host_reset, &hw->cfg.host_reset); +} + +/** + * mei_virtio_hw_ready_wait() - wait until the virtio(hw) has turned ready + * or timeout is reached + * @dev: mei device + * + * Return: 0 on success, error otherwise + */ +static int mei_virtio_hw_ready_wait(struct mei_device *dev) +{ + mutex_unlock(&dev->device_lock); + wait_event_timeout(dev->wait_hw_ready, + dev->recvd_hw_ready, + mei_secs_to_jiffies(MEI_HW_READY_TIMEOUT)); + mutex_lock(&dev->device_lock); + if (!dev->recvd_hw_ready) { + dev_err(dev->dev, "wait hw ready failed\n"); + return -ETIMEDOUT; + } + + dev->recvd_hw_ready = false; + return 0; +} + +/** + * mei_virtio_hw_start() - hw start routine + * @dev: mei device + * + * Return: 0 on success, error otherwise + */ +static int mei_virtio_hw_start(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + int ret; + + dev_dbg(dev->dev, "hw start\n"); + mei_virtio_hw_reset_release(dev); + + ret = mei_virtio_hw_ready_wait(dev); + if (ret) + return ret; + + mei_virtio_add_recv_buf(hw); + atomic_set(&hw->hbuf_ready, 1); + dev_dbg(dev->dev, "hw is ready\n"); + hw->host_ready = true; + + return 0; +} + +/** + * mei_virtio_host_is_ready() - check whether the FE has turned ready + * @dev: mei device + * + * Return: bool + */ +static bool mei_virtio_host_is_ready(struct mei_device *dev) +{ + struct mei_virtio_hw *hw = to_virtio_hw(dev); + + dev_dbg(dev->dev, "host ready %d\n", hw->host_ready); + + return hw->host_ready; +} + +/** + * mei_virtio_data_in() - The callback of recv virtqueue of virtio mei + * @vq: receiving virtqueue + */ +static void mei_virtio_data_in(struct virtqueue *vq) +{ + struct mei_virtio_hw *hw = vq->vdev->priv; + + /* disable interrupts (enabled again from in the interrupt worker) */ + virtqueue_disable_cb(hw->in); + + schedule_work(&hw->intr_handler); +} + +/** + * mei_virtio_data_out() - The callback of send virtqueue of virtio mei + * @vq: transmitting virtqueue + */ +static void mei_virtio_data_out(struct virtqueue *vq) +{ + struct mei_virtio_hw *hw = vq->vdev->priv; + + schedule_work(&hw->intr_handler); +} + +static void mei_virtio_intr_handler(struct work_struct *work) +{ + struct mei_virtio_hw *hw = + container_of(work, struct mei_virtio_hw, intr_handler); + struct mei_device *dev = &hw->mdev; + LIST_HEAD(complete_list); + s32 slots; + int rets = 0; + void *data; + unsigned int len; + + mutex_lock(&dev->device_lock); + + if (dev->dev_state == MEI_DEV_DISABLED) { + dev_warn(dev->dev, "Interrupt in disabled state.\n"); + mei_virtio_intr_disable(dev); + goto end; + } + + /* check if ME wants a reset */ + if (!mei_hw_is_ready(dev) && dev->dev_state != MEI_DEV_RESETTING) { + dev_warn(dev->dev, "BE service not ready: resetting.\n"); + schedule_work(&dev->reset_work); + goto end; + } + + /* check if we need to start the dev */ + if (!mei_host_is_ready(dev)) { + if (mei_hw_is_ready(dev)) { + dev_dbg(dev->dev, "we need to start the dev.\n"); + dev->recvd_hw_ready = true; + wake_up(&dev->wait_hw_ready); + } else { + dev_warn(dev->dev, "Spurious Interrupt\n"); + } + goto end; + } + + /* read */ + if (hw->recv_rdy) { + data = virtqueue_get_buf(hw->in, &len); + if (!data || !len) { + dev_dbg(dev->dev, "No data %d", len); + } else { + dev_dbg(dev->dev, "data_in %d\n", len); + WARN_ON(data != hw->recv_buf); + hw->recv_len = mei_data2slots(len); + hw->recv_rdy = 0; + } + } + + /* write */ + if (!atomic_read(&hw->hbuf_ready)) { + if (!virtqueue_get_buf(hw->out, &len)) { + dev_warn(dev->dev, "Failed to getbuf\n"); + } else { + mei_virtio_free_outbufs(hw); + atomic_inc(&hw->hbuf_ready); + } + } + + /* check slots available for reading */ + slots = mei_count_full_read_slots(dev); + while (slots > 0) { + dev_dbg(dev->dev, "slots to read = %08x\n", slots); + rets = mei_irq_read_handler(dev, &complete_list, &slots); + + if (rets && + (dev->dev_state != MEI_DEV_RESETTING && + dev->dev_state != MEI_DEV_POWER_DOWN)) { + dev_err(dev->dev, "mei_irq_read_handler ret = %d.\n", + rets); + schedule_work(&dev->reset_work); + goto end; + } + } + + dev->hbuf_is_ready = mei_hbuf_is_ready(dev); + + mei_irq_write_handler(dev, &complete_list); + + dev->hbuf_is_ready = mei_hbuf_is_ready(dev); + + mei_irq_compl_handler(dev, &complete_list); + + mei_virtio_add_recv_buf(hw); + +end: + if (dev->dev_state != MEI_DEV_DISABLED) { + if (!virtqueue_enable_cb(hw->in)) + schedule_work(&hw->intr_handler); + } + + mutex_unlock(&dev->device_lock); +} + +static void mei_virtio_config_changed(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw = vdev->priv; + struct mei_device *dev = &hw->mdev; + + virtio_cread(vdev, struct mei_virtio_cfg, + hw_ready, &hw->cfg.hw_ready); + + if (dev->dev_state == MEI_DEV_DISABLED) { + dev_dbg(dev->dev, "disabled state don't start\n"); + return; + } + + /* Run intr handler once to handle reset notify */ + schedule_work(&hw->intr_handler); +} + +static void mei_virtio_remove_vqs(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw = vdev->priv; + + virtqueue_detach_unused_buf(hw->in); + hw->recv_len = 0; + hw->recv_idx = 0; + hw->recv_rdy = 0; + + virtqueue_detach_unused_buf(hw->out); + + mei_virtio_free_outbufs(hw); + + vdev->config->del_vqs(vdev); +} + +/* + * There are two virtqueues, one is for send and another is for recv. + */ +static int mei_virtio_init_vqs(struct mei_virtio_hw *hw, + struct virtio_device *vdev) +{ + struct virtqueue *vqs[2]; + + vq_callback_t *cbs[] = { + mei_virtio_data_in, + mei_virtio_data_out, + }; + static const char * const names[] = { + "in", + "out", + }; + int ret; + + ret = virtio_find_vqs(vdev, 2, vqs, cbs, names, NULL); + if (ret) + return ret; + + hw->in = vqs[0]; + hw->out = vqs[1]; + + return 0; +} + +static const struct mei_hw_ops mei_virtio_ops = { + .fw_status = mei_virtio_fw_status, + .pg_state = mei_virtio_pg_state, + + .host_is_ready = mei_virtio_host_is_ready, + + .hw_is_ready = mei_virtio_hw_is_ready, + .hw_reset = mei_virtio_hw_reset, + .hw_config = mei_virtio_hw_config, + .hw_start = mei_virtio_hw_start, + + .pg_in_transition = mei_virtio_pg_in_transition, + .pg_is_enabled = mei_virtio_pg_is_enabled, + + .intr_clear = mei_virtio_intr_clear, + .intr_enable = mei_virtio_intr_enable, + .intr_disable = mei_virtio_intr_disable, + .synchronize_irq = mei_virtio_synchronize_irq, + + .hbuf_free_slots = mei_virtio_hbuf_empty_slots, + .hbuf_is_ready = mei_virtio_hbuf_is_ready, + .hbuf_depth = mei_virtio_hbuf_depth, + + .write = mei_virtio_write_message, + + .rdbuf_full_slots = mei_virtio_count_full_read_slots, + .read_hdr = mei_virtio_read_hdr, + .read = mei_virtio_read, +}; + +static int mei_virtio_probe(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw; + int ret; + + hw = devm_kzalloc(&vdev->dev, sizeof(*hw), GFP_KERNEL); + if (!hw) + return -ENOMEM; + + vdev->priv = hw; + + INIT_WORK(&hw->intr_handler, mei_virtio_intr_handler); + + ret = mei_virtio_init_vqs(hw, vdev); + if (ret) + goto vqs_failed; + + virtio_cread(vdev, struct mei_virtio_cfg, + buf_depth, &hw->cfg.buf_depth); + + hw->recv_buf = kzalloc(mei_slots2data(hw->cfg.buf_depth), GFP_KERNEL); + if (!hw->recv_buf) { + ret = -ENOMEM; + goto hbuf_failed; + } + atomic_set(&hw->hbuf_ready, 0); + + virtio_device_ready(vdev); + + mei_device_init(&hw->mdev, &vdev->dev, &mei_virtio_ops); + + pm_runtime_get_noresume(&vdev->dev); + pm_runtime_set_active(&vdev->dev); + pm_runtime_enable(&vdev->dev); + + ret = mei_start(&hw->mdev); + if (ret) + goto mei_start_failed; + + pm_runtime_set_autosuspend_delay(&vdev->dev, MEI_VIRTIO_RPM_TIMEOUT); + pm_runtime_use_autosuspend(&vdev->dev); + + ret = mei_register(&hw->mdev, &vdev->dev); + if (ret) + goto mei_failed; + + pm_runtime_put(&vdev->dev); + + return 0; + +mei_failed: + mei_stop(&hw->mdev); +mei_start_failed: + mei_cancel_work(&hw->mdev); + mei_disable_interrupts(&hw->mdev); + kfree(hw->recv_buf); +hbuf_failed: + vdev->config->del_vqs(vdev); +vqs_failed: + return ret; +} + +static int __maybe_unused mei_virtio_pm_runtime_idle(struct device *device) +{ + struct virtio_device *vdev = dev_to_virtio(device); + struct mei_virtio_hw *hw = vdev->priv; + + dev_dbg(&vdev->dev, "rpm: mei_virtio : runtime_idle\n"); + + if (!hw) + return -ENODEV; + + if (mei_write_is_idle(&hw->mdev)) + pm_runtime_autosuspend(device); + + return -EBUSY; +} + +static int __maybe_unused mei_virtio_pm_runtime_suspend(struct device *device) +{ + return 0; +} + +static int __maybe_unused mei_virtio_pm_runtime_resume(struct device *device) +{ + return 0; +} + +static int __maybe_unused mei_virtio_freeze(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw = vdev->priv; + + dev_dbg(&vdev->dev, "freeze\n"); + + if (!hw) + return -ENODEV; + + mei_stop(&hw->mdev); + mei_disable_interrupts(&hw->mdev); + cancel_work_sync(&hw->intr_handler); + vdev->config->reset(vdev); + mei_virtio_remove_vqs(vdev); + + return 0; +} + +static int __maybe_unused mei_virtio_restore(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw = vdev->priv; + int ret; + + dev_dbg(&vdev->dev, "restore\n"); + + if (!hw) + return -ENODEV; + + ret = mei_virtio_init_vqs(hw, vdev); + if (ret) + return ret; + + virtio_device_ready(vdev); + + ret = mei_restart(&hw->mdev); + if (ret) + return ret; + + /* Start timer if stopped in suspend */ + schedule_delayed_work(&hw->mdev.timer_work, HZ); + + return 0; +} + +static const struct dev_pm_ops mei_virtio_pm_ops = { + SET_RUNTIME_PM_OPS(mei_virtio_pm_runtime_suspend, + mei_virtio_pm_runtime_resume, + mei_virtio_pm_runtime_idle) +}; + +static void mei_virtio_remove(struct virtio_device *vdev) +{ + struct mei_virtio_hw *hw = vdev->priv; + + mei_stop(&hw->mdev); + mei_disable_interrupts(&hw->mdev); + cancel_work_sync(&hw->intr_handler); + mei_deregister(&hw->mdev); + vdev->config->reset(vdev); + mei_virtio_remove_vqs(vdev); + kfree(hw->recv_buf); + pm_runtime_disable(&vdev->dev); +} + +static struct virtio_device_id id_table[] = { + { VIRTIO_ID_MEI, VIRTIO_DEV_ANY_ID }, + { } +}; + +static struct virtio_driver mei_virtio_driver = { + .id_table = id_table, + .probe = mei_virtio_probe, + .remove = mei_virtio_remove, + .config_changed = mei_virtio_config_changed, + .driver = { + .name = KBUILD_MODNAME, + .owner = THIS_MODULE, + .pm = &mei_virtio_pm_ops, + }, +#ifdef CONFIG_PM_SLEEP + .freeze = mei_virtio_freeze, + .restore = mei_virtio_restore, +#endif +}; + +module_virtio_driver(mei_virtio_driver); +MODULE_DEVICE_TABLE(virtio, id_table); +MODULE_DESCRIPTION("Virtio MEI frontend driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/misc/mei/hw.h b/drivers/misc/mei/hw.h index 26fa92cb7f7a..8bac86c4d86b 100644 --- a/drivers/misc/mei/hw.h +++ b/drivers/misc/mei/hw.h @@ -25,7 +25,7 @@ /* * MEI Version */ -#define HBM_MINOR_VERSION 1 +#define HBM_MINOR_VERSION 2 #define HBM_MAJOR_VERSION 2 /* @@ -76,6 +76,18 @@ #define HBM_MINOR_VERSION_DR 1 #define HBM_MAJOR_VERSION_DR 2 +/* + * MEI version with vm tag support + */ +#define HBM_MINOR_VERSION_VT 2 +#define HBM_MAJOR_VERSION_VT 2 + +/* + * MEI version with capabilities message support + */ +#define HBM_MINOR_VERSION_CAP 2 +#define HBM_MAJOR_VERSION_CAP 2 + /* Host bus message command opcode */ #define MEI_HBM_CMD_OP_MSK 0x7f /* Host bus message command RESPONSE */ @@ -121,6 +133,9 @@ #define MEI_HBM_DMA_SETUP_REQ_CMD 0x12 #define MEI_HBM_DMA_SETUP_RES_CMD 0x92 +#define MEI_HBM_CAPABILITIES_REQ_CMD 0x13 +#define MEI_HBM_CAPABILITIES_RES_CMD 0x93 + /* * MEI Stop Reason * used by hbm_host_stop_request.reason @@ -182,17 +197,103 @@ enum mei_cl_connect_status { /* * Client Disconnect Status */ -enum mei_cl_disconnect_status { +enum mei_cl_disconnect_status { MEI_CL_DISCONN_SUCCESS = MEI_HBMS_SUCCESS }; /** + * enum mei_ext_hdr_type - extended header type used in + * extended header TLV + * + * @MEI_EXT_HDR_NONE: sentinel + * @MEI_EXT_HDR_VTAG: vtag header + */ +enum mei_ext_hdr_type { + MEI_EXT_HDR_NONE = 0, + MEI_EXT_HDR_VTAG = 1, +}; + +/** + * struct mei_ext_hdr - extend header descriptor (TLV) + * @type: enum mei_ext_hdr_type + * @length: length excluding descriptor + * @ext_payload: payload of the specific extended header + * @hdr: place holder for actual header + */ +struct mei_ext_hdr { + u8 type; + u8 length; + u8 ext_payload[2]; + u8 hdr[0]; +}; + +/** + * struct mei_ext_meta_hdr - extend header meta data + * @count: number of headers + * @size: total size of the extended header list excluding meta header + * @reserved: reserved + * @hdrs: extended headers TLV list + */ +struct mei_ext_meta_hdr { + u8 count; + u8 size; + u8 reserved[2]; + struct mei_ext_hdr hdrs[0]; +}; + +/* + * Extended header iterator functions + */ +/** + * mei_ext_hdr - extended header iterator begin + * + * @meta: meta header of the extended header list + * + * Return: + * The first extended header + */ +static inline struct mei_ext_hdr *mei_ext_begin(struct mei_ext_meta_hdr *meta) +{ + return meta->hdrs; +} + +/** + * mei_ext_last - check if the ext is the last one in the TLV list + * + * @meta: meta header of the extended header list + * @ext: a meta header on the list + * + * Return: true if ext is the last header on the list + */ +static inline bool mei_ext_last(struct mei_ext_meta_hdr *meta, + struct mei_ext_hdr *ext) +{ + return (u8 *)ext >= (u8 *)meta + sizeof(*meta) + (meta->size * 4); +} + +/** + *mei_ext_next - following extended header on the TLV list + * + * @ext: current extend header + * + * Context: The function does not check for the overflows, + * one should call mei_ext_last before. + * + * Return: The following extend header after @ext + */ +static inline struct mei_ext_hdr *mei_ext_next(struct mei_ext_hdr *ext) +{ + return (struct mei_ext_hdr *)(ext->hdr + (ext->length * 4)); +} + +/** * struct mei_msg_hdr - MEI BUS Interface Section * * @me_addr: device address * @host_addr: host address * @length: message length * @reserved: reserved + * @extended: message has extended header * @dma_ring: message is on dma ring * @internal: message is internal * @msg_complete: last packet of the message @@ -202,7 +303,8 @@ struct mei_msg_hdr { u32 me_addr:8; u32 host_addr:8; u32 length:9; - u32 reserved:4; + u32 reserved:3; + u32 extended:1; u32 dma_ring:1; u32 internal:1; u32 msg_complete:1; @@ -212,8 +314,6 @@ struct mei_msg_hdr { /* The length is up to 9 bits */ #define MEI_MSG_MAX_LEN_MASK GENMASK(9, 0) -#define MEI_MSG_HDR_MAX 2 - struct mei_bus_message { u8 hbm_cmd; u8 data[]; @@ -299,13 +399,26 @@ struct hbm_host_enum_response { u8 valid_addresses[32]; } __packed; +/** + * struct mei_client_properties - mei client properties + * + * @protocol_name: guid of the client + * @protocol_version: client protocol version + * @max_number_of_connections: number of possible connections. + * @fixed_address: fixed me address (0 if the client is dynamic) + * @single_recv_buf: 1 if all connections share a single receive buffer. + * @vt_supported: the client support vtag + * @reserved: reserved + * @max_msg_length: MTU of the client + */ struct mei_client_properties { uuid_le protocol_name; u8 protocol_version; u8 max_number_of_connections; u8 fixed_address; u8 single_recv_buf:1; - u8 reserved:7; + u8 vt_supported:1; + u8 reserved:6; u32 max_msg_length; } __packed; @@ -533,4 +646,29 @@ struct hbm_dma_ring_ctrl { u32 reserved4; } __packed; +/* virtual tag supported */ +#define HBM_CAP_VT BIT(0) + +/** + * struct hbm_capability_request - capability request from host to fw + * + * @hbm_cmd : bus message command header + * @capability_requested: bitmask of capabilities requested by host + */ +struct hbm_capability_request { + u8 hbm_cmd; + u8 capability_requested[3]; +} __packed; + +/** + * struct hbm_capability_response - capability response from fw to host + * + * @hbm_cmd : bus message command header + * @capability_granted: bitmask of capabilities granted by FW + */ +struct hbm_capability_response { + u8 hbm_cmd; + u8 capability_granted[3]; +} __packed; + #endif diff --git a/drivers/misc/mei/interrupt.c b/drivers/misc/mei/interrupt.c index c70a8c74cc57..326955b04fda 100644 --- a/drivers/misc/mei/interrupt.c +++ b/drivers/misc/mei/interrupt.c @@ -61,16 +61,21 @@ static inline int mei_cl_hbm_equal(struct mei_cl *cl, * * @dev: mei device * @hdr: message header + * @discard_len: the length of the message to discard (excluding header) */ -static void mei_irq_discard_msg(struct mei_device *dev, struct mei_msg_hdr *hdr) +static void mei_irq_discard_msg(struct mei_device *dev, struct mei_msg_hdr *hdr, + size_t discard_len) { - if (hdr->dma_ring) - mei_dma_ring_read(dev, NULL, hdr->extension[0]); + if (hdr->dma_ring) { + mei_dma_ring_read(dev, NULL, + hdr->extension[dev->rd_msg_hdr_count - 2]); + discard_len = 0; + } /* * no need to check for size as it is guarantied * that length fits into rd_msg_buf */ - mei_read_slots(dev, dev->rd_msg_buf, hdr->length); + mei_read_slots(dev, dev->rd_msg_buf, discard_len); dev_dbg(dev->dev, "discarding message " MEI_HDR_FMT "\n", MEI_HDR_PRM(hdr)); } @@ -80,18 +85,29 @@ static void mei_irq_discard_msg(struct mei_device *dev, struct mei_msg_hdr *hdr) * * @cl: reading client * @mei_hdr: header of mei client message + * @meta: extend meta header * @cmpl_list: completion list * * Return: always 0 */ static int mei_cl_irq_read_msg(struct mei_cl *cl, struct mei_msg_hdr *mei_hdr, + struct mei_ext_meta_hdr *meta, struct list_head *cmpl_list) { struct mei_device *dev = cl->dev; struct mei_cl_cb *cb; + size_t buf_sz; u32 length; + int ext_len; + + length = mei_hdr->length; + ext_len = 0; + if (mei_hdr->extended) { + ext_len = sizeof(*meta) + mei_slots2data(meta->size); + length -= ext_len; + } cb = list_first_entry_or_null(&cl->rd_pending, struct mei_cl_cb, list); if (!cb) { @@ -105,13 +121,50 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl, list_add_tail(&cb->list, &cl->rd_pending); } + if (mei_hdr->extended) { + struct mei_ext_hdr *ext; + struct mei_ext_hdr *vtag = NULL; + + ext = mei_ext_begin(meta); + do { + switch (ext->type) { + case MEI_EXT_HDR_VTAG: + vtag = ext; + break; + case MEI_EXT_HDR_NONE: + fallthrough; + default: + cb->status = -EPROTO; + break; + } + + ext = mei_ext_next(ext); + } while (!mei_ext_last(meta, ext)); + + if (!vtag) { + cl_dbg(dev, cl, "vtag not found in extended header.\n"); + cb->status = -EPROTO; + goto discard; + } + + cl_dbg(dev, cl, "vtag: %d\n", vtag->ext_payload[0]); + if (cb->vtag && cb->vtag != vtag->ext_payload[0]) { + cl_err(dev, cl, "mismatched tag: %d != %d\n", + cb->vtag, vtag->ext_payload[0]); + cb->status = -EPROTO; + goto discard; + } + cb->vtag = vtag->ext_payload[0]; + } + if (!mei_cl_is_connected(cl)) { cl_dbg(dev, cl, "not connected\n"); cb->status = -ENODEV; goto discard; } - length = mei_hdr->dma_ring ? mei_hdr->extension[0] : mei_hdr->length; + if (mei_hdr->dma_ring) + length = mei_hdr->extension[mei_data2slots(ext_len)]; buf_sz = length + cb->buf_idx; /* catch for integer overflow */ @@ -129,11 +182,13 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl, goto discard; } - if (mei_hdr->dma_ring) + if (mei_hdr->dma_ring) { mei_dma_ring_read(dev, cb->buf.data + cb->buf_idx, length); - - /* for DMA read 0 length to generate an interrupt to the device */ - mei_read_slots(dev, cb->buf.data + cb->buf_idx, mei_hdr->length); + /* for DMA read 0 length to generate interrupt to the device */ + mei_read_slots(dev, cb->buf.data + cb->buf_idx, 0); + } else { + mei_read_slots(dev, cb->buf.data + cb->buf_idx, length); + } cb->buf_idx += length; @@ -150,7 +205,7 @@ static int mei_cl_irq_read_msg(struct mei_cl *cl, discard: if (cb) list_move_tail(&cb->list, cmpl_list); - mei_irq_discard_msg(dev, mei_hdr); + mei_irq_discard_msg(dev, mei_hdr, length); return 0; } @@ -265,11 +320,16 @@ int mei_irq_read_handler(struct mei_device *dev, struct list_head *cmpl_list, s32 *slots) { struct mei_msg_hdr *mei_hdr; + struct mei_ext_meta_hdr *meta_hdr = NULL; struct mei_cl *cl; int ret; + u32 ext_meta_hdr_u32; + int i; + int ext_hdr_end; if (!dev->rd_msg_hdr[0]) { dev->rd_msg_hdr[0] = mei_read_hdr(dev); + dev->rd_msg_hdr_count = 1; (*slots)--; dev_dbg(dev->dev, "slots =%08x.\n", *slots); @@ -292,10 +352,34 @@ int mei_irq_read_handler(struct mei_device *dev, goto end; } + ext_hdr_end = 1; + + if (mei_hdr->extended) { + if (!dev->rd_msg_hdr[1]) { + ext_meta_hdr_u32 = mei_read_hdr(dev); + dev->rd_msg_hdr[1] = ext_meta_hdr_u32; + dev->rd_msg_hdr_count++; + (*slots)--; + dev_dbg(dev->dev, "extended header is %08x\n", + ext_meta_hdr_u32); + } + meta_hdr = ((struct mei_ext_meta_hdr *) + dev->rd_msg_hdr + 1); + ext_hdr_end = meta_hdr->size + 2; + for (i = dev->rd_msg_hdr_count; i < ext_hdr_end; i++) { + dev->rd_msg_hdr[i] = mei_read_hdr(dev); + dev_dbg(dev->dev, "extended header %d is %08x\n", i, + dev->rd_msg_hdr[i]); + dev->rd_msg_hdr_count++; + (*slots)--; + } + } + if (mei_hdr->dma_ring) { - dev->rd_msg_hdr[1] = mei_read_hdr(dev); + dev->rd_msg_hdr[ext_hdr_end] = mei_read_hdr(dev); + dev->rd_msg_hdr_count++; (*slots)--; - mei_hdr->length = 0; + mei_hdr->length -= sizeof(dev->rd_msg_hdr[ext_hdr_end]); } /* HBM message */ @@ -326,7 +410,7 @@ int mei_irq_read_handler(struct mei_device *dev, */ if (hdr_is_fixed(mei_hdr) || dev->dev_state == MEI_DEV_POWER_DOWN) { - mei_irq_discard_msg(dev, mei_hdr); + mei_irq_discard_msg(dev, mei_hdr, mei_hdr->length); ret = 0; goto reset_slots; } @@ -336,12 +420,13 @@ int mei_irq_read_handler(struct mei_device *dev, goto end; } - ret = mei_cl_irq_read_msg(cl, mei_hdr, cmpl_list); + ret = mei_cl_irq_read_msg(cl, mei_hdr, meta_hdr, cmpl_list); reset_slots: /* reset the number of slots and header */ memset(dev->rd_msg_hdr, 0, sizeof(dev->rd_msg_hdr)); + dev->rd_msg_hdr_count = 0; *slots = mei_count_full_read_slots(dev); if (*slots == -EOVERFLOW) { /* overflow - reset */ diff --git a/drivers/misc/mei/main.c b/drivers/misc/mei/main.c index 86ef5c1a7928..9f6682033ed7 100644 --- a/drivers/misc/mei/main.c +++ b/drivers/misc/mei/main.c @@ -81,6 +81,27 @@ err_unlock: } /** + * mei_cl_vtag_remove_by_fp - remove vtag that corresponds to fp from list + * + * @cl: host client + * @fp: pointer to file structure + * + */ +static void mei_cl_vtag_remove_by_fp(const struct mei_cl *cl, + const struct file *fp) +{ + struct mei_cl_vtag *vtag_l, *next; + + list_for_each_entry_safe(vtag_l, next, &cl->vtag_map, list) { + if (vtag_l->fp == fp) { + list_del(&vtag_l->list); + kfree(vtag_l); + return; + } + } +} + +/** * mei_release - the release function * * @inode: pointer to inode structure @@ -101,17 +122,35 @@ static int mei_release(struct inode *inode, struct file *file) mutex_lock(&dev->device_lock); + mei_cl_vtag_remove_by_fp(cl, file); + + if (!list_empty(&cl->vtag_map)) { + cl_dbg(dev, cl, "not the last vtag\n"); + mei_cl_flush_queues(cl, file); + rets = 0; + goto out; + } + rets = mei_cl_disconnect(cl); + /* + * Check again: This is necessary since disconnect releases the lock + * and another client can connect in the meantime. + */ + if (!list_empty(&cl->vtag_map)) { + cl_dbg(dev, cl, "not the last vtag after disconnect\n"); + mei_cl_flush_queues(cl, file); + goto out; + } - mei_cl_flush_queues(cl, file); + mei_cl_flush_queues(cl, NULL); cl_dbg(dev, cl, "removing\n"); mei_cl_unlink(cl); + kfree(cl); +out: file->private_data = NULL; - kfree(cl); - mutex_unlock(&dev->device_lock); return rets; } @@ -178,7 +217,7 @@ static ssize_t mei_read(struct file *file, char __user *ubuf, mutex_unlock(&dev->device_lock); if (wait_event_interruptible(cl->rx_wait, - !list_empty(&cl->rd_completed) || + mei_cl_read_cb(cl, file) || !mei_cl_is_connected(cl))) { if (signal_pending(current)) return -EINTR; @@ -229,7 +268,7 @@ copy_buffer: goto out; free: - mei_io_cb_free(cb); + mei_cl_del_rd_completed(cl, cb); *offset = 0; out: @@ -237,6 +276,28 @@ out: mutex_unlock(&dev->device_lock); return rets; } + +/** + * mei_cl_vtag_by_fp - obtain the vtag by file pointer + * + * @cl: host client + * @fp: pointer to file structure + * + * Return: vtag value on success, otherwise 0 + */ +static u8 mei_cl_vtag_by_fp(const struct mei_cl *cl, const struct file *fp) +{ + struct mei_cl_vtag *cl_vtag; + + if (!fp) + return 0; + + list_for_each_entry(cl_vtag, &cl->vtag_map, list) + if (cl_vtag->fp == fp) + return cl_vtag->vtag; + return 0; +} + /** * mei_write - the write function. * @@ -314,6 +375,7 @@ static ssize_t mei_write(struct file *file, const char __user *ubuf, rets = -ENOMEM; goto out; } + cb->vtag = mei_cl_vtag_by_fp(cl, file); rets = copy_from_user(cb->buf.data, ubuf, length); if (rets) { @@ -333,17 +395,18 @@ out: * mei_ioctl_connect_client - the connect to fw client IOCTL function * * @file: private data of the file object - * @data: IOCTL connect data, input and output parameters + * @in_client_uuid: requested UUID for connection + * @client: IOCTL connect data, output parameters * * Locking: called under "dev->device_lock" lock * * Return: 0 on success, <0 on failure. */ static int mei_ioctl_connect_client(struct file *file, - struct mei_connect_client_data *data) + const uuid_le *in_client_uuid, + struct mei_client *client) { struct mei_device *dev; - struct mei_client *client; struct mei_me_client *me_cl; struct mei_cl *cl; int rets; @@ -351,18 +414,15 @@ static int mei_ioctl_connect_client(struct file *file, cl = file->private_data; dev = cl->dev; - if (dev->dev_state != MEI_DEV_ENABLED) - return -ENODEV; - if (cl->state != MEI_FILE_INITIALIZING && cl->state != MEI_FILE_DISCONNECTED) return -EBUSY; /* find ME client we're trying to connect to */ - me_cl = mei_me_cl_by_uuid(dev, &data->in_client_uuid); + me_cl = mei_me_cl_by_uuid(dev, in_client_uuid); if (!me_cl) { dev_dbg(dev->dev, "Cannot connect to FW Client UUID = %pUl\n", - &data->in_client_uuid); + in_client_uuid); rets = -ENOTTY; goto end; } @@ -372,7 +432,7 @@ static int mei_ioctl_connect_client(struct file *file, !dev->allow_fixed_address : !dev->hbm_f_fa_supported; if (forbidden) { dev_dbg(dev->dev, "Connection forbidden to FW Client UUID = %pUl\n", - &data->in_client_uuid); + in_client_uuid); rets = -ENOTTY; goto end; } @@ -386,7 +446,6 @@ static int mei_ioctl_connect_client(struct file *file, me_cl->props.max_msg_length); /* prepare the output buffer */ - client = &data->out_client_properties; client->max_msg_length = me_cl->props.max_msg_length; client->protocol_version = me_cl->props.protocol_version; dev_dbg(dev->dev, "Can connect?\n"); @@ -399,6 +458,135 @@ end: } /** + * mei_vt_support_check - check if client support vtags + * + * Locking: called under "dev->device_lock" lock + * + * @dev: mei_device + * @uuid: client UUID + * + * Return: + * 0 - supported + * -ENOTTY - no such client + * -EOPNOTSUPP - vtags are not supported by client + */ +static int mei_vt_support_check(struct mei_device *dev, const uuid_le *uuid) +{ + struct mei_me_client *me_cl; + int ret; + + if (!dev->hbm_f_vt_supported) + return -EOPNOTSUPP; + + me_cl = mei_me_cl_by_uuid(dev, uuid); + if (!me_cl) { + dev_dbg(dev->dev, "Cannot connect to FW Client UUID = %pUl\n", + uuid); + return -ENOTTY; + } + ret = me_cl->props.vt_supported ? 0 : -EOPNOTSUPP; + mei_me_cl_put(me_cl); + + return ret; +} + +/** + * mei_ioctl_connect_vtag - connect to fw client with vtag IOCTL function + * + * @file: private data of the file object + * @in_client_uuid: requested UUID for connection + * @client: IOCTL connect data, output parameters + * @vtag: vm tag + * + * Locking: called under "dev->device_lock" lock + * + * Return: 0 on success, <0 on failure. + */ +static int mei_ioctl_connect_vtag(struct file *file, + const uuid_le *in_client_uuid, + struct mei_client *client, + u8 vtag) +{ + struct mei_device *dev; + struct mei_cl *cl; + struct mei_cl *pos; + struct mei_cl_vtag *cl_vtag; + + cl = file->private_data; + dev = cl->dev; + + dev_dbg(dev->dev, "FW Client %pUl vtag %d\n", in_client_uuid, vtag); + + switch (cl->state) { + case MEI_FILE_DISCONNECTED: + if (mei_cl_vtag_by_fp(cl, file) != vtag) { + dev_err(dev->dev, "reconnect with different vtag\n"); + return -EINVAL; + } + break; + case MEI_FILE_INITIALIZING: + /* malicious connect from another thread may push vtag */ + if (!IS_ERR(mei_cl_fp_by_vtag(cl, vtag))) { + dev_err(dev->dev, "vtag already filled\n"); + return -EINVAL; + } + + list_for_each_entry(pos, &dev->file_list, link) { + if (pos == cl) + continue; + if (!pos->me_cl) + continue; + + /* only search for same UUID */ + if (uuid_le_cmp(*mei_cl_uuid(pos), *in_client_uuid)) + continue; + + /* if tag already exist try another fp */ + if (!IS_ERR(mei_cl_fp_by_vtag(pos, vtag))) + continue; + + /* replace cl with acquired one */ + dev_dbg(dev->dev, "replacing with existing cl\n"); + mei_cl_unlink(cl); + kfree(cl); + file->private_data = pos; + cl = pos; + break; + } + + cl_vtag = mei_cl_vtag_alloc(file, vtag); + if (IS_ERR(cl_vtag)) + return -ENOMEM; + + list_add_tail(&cl_vtag->list, &cl->vtag_map); + break; + default: + return -EBUSY; + } + + while (cl->state != MEI_FILE_INITIALIZING && + cl->state != MEI_FILE_DISCONNECTED && + cl->state != MEI_FILE_CONNECTED) { + mutex_unlock(&dev->device_lock); + wait_event_timeout(cl->wait, + (cl->state == MEI_FILE_CONNECTED || + cl->state == MEI_FILE_DISCONNECTED || + cl->state == MEI_FILE_DISCONNECT_REQUIRED || + cl->state == MEI_FILE_DISCONNECT_REPLY), + mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT)); + mutex_lock(&dev->device_lock); + } + + if (!mei_cl_is_connected(cl)) + return mei_ioctl_connect_client(file, in_client_uuid, client); + + client->max_msg_length = cl->me_cl->props.max_msg_length; + client->protocol_version = cl->me_cl->props.protocol_version; + + return 0; +} + +/** * mei_ioctl_client_notify_request - * propagate event notification request to client * @@ -454,7 +642,11 @@ static long mei_ioctl(struct file *file, unsigned int cmd, unsigned long data) { struct mei_device *dev; struct mei_cl *cl = file->private_data; - struct mei_connect_client_data connect_data; + struct mei_connect_client_data conn; + struct mei_connect_client_data_vtag conn_vtag; + const uuid_le *cl_uuid; + struct mei_client *props; + u8 vtag; u32 notify_get, notify_req; int rets; @@ -475,20 +667,68 @@ static long mei_ioctl(struct file *file, unsigned int cmd, unsigned long data) switch (cmd) { case IOCTL_MEI_CONNECT_CLIENT: dev_dbg(dev->dev, ": IOCTL_MEI_CONNECT_CLIENT.\n"); - if (copy_from_user(&connect_data, (char __user *)data, - sizeof(connect_data))) { + if (copy_from_user(&conn, (char __user *)data, sizeof(conn))) { dev_dbg(dev->dev, "failed to copy data from userland\n"); rets = -EFAULT; goto out; } + cl_uuid = &conn.in_client_uuid; + props = &conn.out_client_properties; + vtag = 0; + + rets = mei_vt_support_check(dev, cl_uuid); + if (rets == -ENOTTY) + goto out; + if (!rets) + rets = mei_ioctl_connect_vtag(file, cl_uuid, props, + vtag); + else + rets = mei_ioctl_connect_client(file, cl_uuid, props); + if (rets) + goto out; + + /* if all is ok, copying the data back to user. */ + if (copy_to_user((char __user *)data, &conn, sizeof(conn))) { + dev_dbg(dev->dev, "failed to copy data to userland\n"); + rets = -EFAULT; + goto out; + } + + break; + + case IOCTL_MEI_CONNECT_CLIENT_VTAG: + dev_dbg(dev->dev, "IOCTL_MEI_CONNECT_CLIENT_VTAG\n"); + if (copy_from_user(&conn_vtag, (char __user *)data, + sizeof(conn_vtag))) { + dev_dbg(dev->dev, "failed to copy data from userland\n"); + rets = -EFAULT; + goto out; + } + + cl_uuid = &conn_vtag.connect.in_client_uuid; + props = &conn_vtag.out_client_properties; + vtag = conn_vtag.connect.vtag; + + rets = mei_vt_support_check(dev, cl_uuid); + if (rets == -EOPNOTSUPP) + dev_dbg(dev->dev, "FW Client %pUl does not support vtags\n", + cl_uuid); + if (rets) + goto out; + + if (!vtag) { + dev_dbg(dev->dev, "vtag can't be zero\n"); + rets = -EINVAL; + goto out; + } - rets = mei_ioctl_connect_client(file, &connect_data); + rets = mei_ioctl_connect_vtag(file, cl_uuid, props, vtag); if (rets) goto out; /* if all is ok, copying the data back to user. */ - if (copy_to_user((char __user *)data, &connect_data, - sizeof(connect_data))) { + if (copy_to_user((char __user *)data, &conn_vtag, + sizeof(conn_vtag))) { dev_dbg(dev->dev, "failed to copy data to userland\n"); rets = -EFAULT; goto out; @@ -572,7 +812,7 @@ static __poll_t mei_poll(struct file *file, poll_table *wait) if (req_events & (EPOLLIN | EPOLLRDNORM)) { poll_wait(file, &cl->rx_wait, wait); - if (!list_empty(&cl->rd_completed)) + if (mei_cl_read_cb(cl, file)) mask |= EPOLLIN | EPOLLRDNORM; else mei_cl_read_start(cl, mei_cl_mtu(cl), file); diff --git a/drivers/misc/mei/mei_dev.h b/drivers/misc/mei/mei_dev.h index d3a4f54c0ae7..2f4cc1a8aae8 100644 --- a/drivers/misc/mei/mei_dev.h +++ b/drivers/misc/mei/mei_dev.h @@ -174,6 +174,7 @@ struct mei_cl; * @fop_type: file operation type * @buf: buffer for data associated with the callback * @buf_idx: last read index + * @vtag: virtual tag * @fp: pointer to file structure * @status: io status of the cb * @internal: communication between driver and FW flag @@ -185,6 +186,7 @@ struct mei_cl_cb { enum mei_cb_file_ops fop_type; struct mei_msg_data buf; size_t buf_idx; + u8 vtag; const struct file *fp; int status; u32 internal:1; @@ -192,6 +194,21 @@ struct mei_cl_cb { }; /** + * struct mei_cl_vtag - file pointer to vtag mapping structure + * + * @list: link in map queue + * @fp: file pointer + * @vtag: corresponding vtag + * @pending_read: the read is pending on this file + */ +struct mei_cl_vtag { + struct list_head list; + const struct file *fp; + u8 vtag; + u8 pending_read:1; +}; + +/** * struct mei_cl - me client host representation * carried in file->private_data * @@ -207,6 +224,7 @@ struct mei_cl_cb { * @me_cl: fw client connected * @fp: file associated with client * @host_client_id: host id + * @vtag_map: vtag map * @tx_flow_ctrl_creds: transmit flow credentials * @rx_flow_ctrl_creds: receive flow credentials * @timer_count: watchdog timer for operation completion @@ -215,6 +233,7 @@ struct mei_cl_cb { * @tx_cb_queued: number of tx callbacks in queue * @writing_state: state of the tx * @rd_pending: pending read credits + * @rd_completed_lock: protects rd_completed queue * @rd_completed: completed read * * @cldev: device on the mei client bus @@ -232,6 +251,7 @@ struct mei_cl { struct mei_me_client *me_cl; const struct file *fp; u8 host_client_id; + struct list_head vtag_map; u8 tx_flow_ctrl_creds; u8 rx_flow_ctrl_creds; u8 timer_count; @@ -240,6 +260,7 @@ struct mei_cl { u8 tx_cb_queued; enum mei_file_transaction_states writing_state; struct list_head rd_pending; + spinlock_t rd_completed_lock; /* protects rd_completed queue */ struct list_head rd_completed; struct mei_cl_device *cldev; @@ -413,6 +434,7 @@ struct mei_fw_version { * * @rd_msg_buf : control messages buffer * @rd_msg_hdr : read message header storage + * @rd_msg_hdr_count : how many dwords were already read from header * * @hbuf_is_ready : query if the host host/write buffer is ready * @dr_dscr: DMA ring descriptors: TX, RX, and CTRL @@ -426,6 +448,8 @@ struct mei_fw_version { * @hbm_f_ie_supported : hbm feature immediate reply to enum request * @hbm_f_os_supported : hbm feature support OS ver message * @hbm_f_dr_supported : hbm feature dma ring supported + * @hbm_f_vt_supported : hbm feature vtag supported + * @hbm_f_cap_supported : hbm feature capabilities message supported * * @fw_ver : FW versions * @@ -494,7 +518,8 @@ struct mei_device { #endif /* CONFIG_PM */ unsigned char rd_msg_buf[MEI_RD_MSG_BUF_SIZE]; - u32 rd_msg_hdr[MEI_MSG_HDR_MAX]; + u32 rd_msg_hdr[MEI_RD_MSG_BUF_SIZE]; + int rd_msg_hdr_count; /* write buffer */ bool hbuf_is_ready; @@ -510,6 +535,8 @@ struct mei_device { unsigned int hbm_f_ie_supported:1; unsigned int hbm_f_os_supported:1; unsigned int hbm_f_dr_supported:1; + unsigned int hbm_f_vt_supported:1; + unsigned int hbm_f_cap_supported:1; struct mei_fw_version fw_ver[MEI_MAX_FW_VER_BLOCKS]; @@ -746,10 +773,11 @@ static inline void mei_dbgfs_deregister(struct mei_device *dev) {} int mei_register(struct mei_device *dev, struct device *parent); void mei_deregister(struct mei_device *dev); -#define MEI_HDR_FMT "hdr:host=%02d me=%02d len=%d dma=%1d internal=%1d comp=%1d" +#define MEI_HDR_FMT "hdr:host=%02d me=%02d len=%d dma=%1d ext=%1d internal=%1d comp=%1d" #define MEI_HDR_PRM(hdr) \ (hdr)->host_addr, (hdr)->me_addr, \ - (hdr)->length, (hdr)->dma_ring, (hdr)->internal, (hdr)->msg_complete + (hdr)->length, (hdr)->dma_ring, (hdr)->extended, \ + (hdr)->internal, (hdr)->msg_complete ssize_t mei_fw_status2str(struct mei_fw_status *fw_sts, char *buf, size_t len); /** diff --git a/drivers/misc/mic/scif/scif_nodeqp.c b/drivers/misc/mic/scif/scif_nodeqp.c index e0748be373f1..384ce08fa98a 100644 --- a/drivers/misc/mic/scif/scif_nodeqp.c +++ b/drivers/misc/mic/scif/scif_nodeqp.c @@ -363,7 +363,7 @@ scif_p2p_setsg(phys_addr_t pa, int page_size, int page_cnt) struct page *page; int i; - sg = kcalloc(page_cnt, sizeof(struct scatterlist), GFP_KERNEL); + sg = kmalloc_array(page_cnt, sizeof(struct scatterlist), GFP_KERNEL); if (!sg) return NULL; sg_init_table(sg, page_cnt); diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c index 2da3b474f486..18fb9d8b8a4b 100644 --- a/drivers/misc/mic/scif/scif_rma.c +++ b/drivers/misc/mic/scif/scif_rma.c @@ -1392,6 +1392,8 @@ retry: (prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0, pinned_pages->pages); if (nr_pages != pinned_pages->nr_pages) { + if (pinned_pages->nr_pages < 0) + pinned_pages->nr_pages = 0; if (try_upgrade) { if (ulimit) __scif_dec_pinned_vm_lock(mm, nr_pages); @@ -1408,7 +1410,6 @@ retry: if (pinned_pages->nr_pages < nr_pages) { err = -EFAULT; - pinned_pages->nr_pages = nr_pages; goto dec_pinned; } @@ -1421,7 +1422,6 @@ dec_pinned: __scif_dec_pinned_vm_lock(mm, nr_pages); /* Something went wrong! Rollback */ error_unmap: - pinned_pages->nr_pages = nr_pages; scif_destroy_pinned_pages(pinned_pages); *pages = NULL; dev_dbg(scif_info.mdev.this_device, diff --git a/drivers/misc/mic/vop/Makefile b/drivers/misc/mic/vop/Makefile index 579da3868c8e..51b9b0022786 100644 --- a/drivers/misc/mic/vop/Makefile +++ b/drivers/misc/mic/vop/Makefile @@ -3,7 +3,7 @@ # Makefile - Intel MIC Linux driver. # Copyright(c) 2016, Intel Corporation. # -obj-m := vop.o +obj-$(CONFIG_VOP) := vop.o vop-objs += vop_main.o vop-objs += vop_debugfs.o diff --git a/drivers/misc/mic/vop/vop_main.c b/drivers/misc/mic/vop/vop_main.c index 55e7f21e51f4..714b94f42d38 100644 --- a/drivers/misc/mic/vop/vop_main.c +++ b/drivers/misc/mic/vop/vop_main.c @@ -124,6 +124,7 @@ static void vop_transport_features(struct virtio_device *vdev) * creates virtio rings on preallocated memory. */ __virtio_clear_bit(vdev, VIRTIO_F_RING_PACKED); + __virtio_set_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); } static int vop_finalize_features(struct virtio_device *vdev) @@ -320,7 +321,7 @@ static struct virtqueue *vop_find_vq(struct virtio_device *dev, /* First assign the vring's allocated in host memory */ vqconfig = _vop_vq_config(vdev->desc) + index; memcpy_fromio(&config, vqconfig, sizeof(config)); - _vr_size = vring_size(le16_to_cpu(config.num), MIC_VIRTIO_RING_ALIGN); + _vr_size = round_up(vring_size(le16_to_cpu(config.num), MIC_VIRTIO_RING_ALIGN), 4); vr_size = PAGE_ALIGN(_vr_size + sizeof(struct _mic_vring_info)); va = vpdev->hw_ops->remap(vpdev, le64_to_cpu(config.address), vr_size); if (!va) diff --git a/drivers/misc/mic/vop/vop_vringh.c b/drivers/misc/mic/vop/vop_vringh.c index 30eac172f017..7014ffe88632 100644 --- a/drivers/misc/mic/vop/vop_vringh.c +++ b/drivers/misc/mic/vop/vop_vringh.c @@ -296,7 +296,7 @@ static int vop_virtio_add_device(struct vop_vdev *vdev, num = le16_to_cpu(vqconfig[i].num); mutex_init(&vvr->vr_mutex); - vr_size = PAGE_ALIGN(vring_size(num, MIC_VIRTIO_RING_ALIGN) + + vr_size = PAGE_ALIGN(round_up(vring_size(num, MIC_VIRTIO_RING_ALIGN), 4) + sizeof(struct _mic_vring_info)); vr->va = (void *) __get_free_pages(GFP_KERNEL | __GFP_ZERO, @@ -308,7 +308,7 @@ static int vop_virtio_add_device(struct vop_vdev *vdev, goto err; } vr->len = vr_size; - vr->info = vr->va + vring_size(num, MIC_VIRTIO_RING_ALIGN); + vr->info = vr->va + round_up(vring_size(num, MIC_VIRTIO_RING_ALIGN), 4); vr->info->magic = cpu_to_le32(MIC_MAGIC + vdev->virtio_id + i); vr_addr = dma_map_single(&vpdev->dev, vr->va, vr_size, DMA_BIDIRECTIONAL); @@ -602,6 +602,7 @@ static int vop_virtio_copy_from_user(struct vop_vdev *vdev, void __user *ubuf, size_t partlen; bool dma = VOP_USE_DMA && vi->dma_ch; int err = 0; + size_t offset = 0; if (dma) { dma_alignment = 1 << vi->dma_ch->device->copy_align; @@ -655,13 +656,20 @@ memcpy: * We are copying to IO below and should ideally use something * like copy_from_user_toio(..) if it existed. */ - if (copy_from_user((void __force *)dbuf, ubuf, len)) { - err = -EFAULT; - dev_err(vop_dev(vdev), "%s %d err %d\n", - __func__, __LINE__, err); - goto err; + while (len) { + partlen = min_t(size_t, len, VOP_INT_DMA_BUF_SIZE); + + if (copy_from_user(vvr->buf, ubuf + offset, partlen)) { + err = -EFAULT; + dev_err(vop_dev(vdev), "%s %d err %d\n", + __func__, __LINE__, err); + goto err; + } + memcpy_toio(dbuf + offset, vvr->buf, partlen); + offset += partlen; + vdev->out_bytes += partlen; + len -= partlen; } - vdev->out_bytes += len; err = 0; err: vpdev->hw_ops->unmap(vpdev, dbuf); diff --git a/drivers/misc/ocxl/Kconfig b/drivers/misc/ocxl/Kconfig index 6551007a066c..947294f6d7f4 100644 --- a/drivers/misc/ocxl/Kconfig +++ b/drivers/misc/ocxl/Kconfig @@ -9,9 +9,8 @@ config OCXL_BASE config OCXL tristate "OpenCAPI coherent accelerator support" - depends on PPC_POWERNV && PCI && EEH + depends on PPC_POWERNV && PCI && EEH && HOTPLUG_PCI_POWERNV select OCXL_BASE - select HOTPLUG_PCI_POWERNV default m help Select this option to enable the ocxl driver for Open diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c index b7a09b21ab36..aebfc53a2d09 100644 --- a/drivers/misc/ocxl/core.c +++ b/drivers/misc/ocxl/core.c @@ -327,14 +327,9 @@ static void free_function_dev(struct device *dev) static int set_function_device(struct ocxl_fn *fn, struct pci_dev *dev) { - int rc; - fn->dev.parent = &dev->dev; fn->dev.release = free_function_dev; - rc = dev_set_name(&fn->dev, "ocxlfn.%s", dev_name(&dev->dev)); - if (rc) - return rc; - return 0; + return dev_set_name(&fn->dev, "ocxlfn.%s", dev_name(&dev->dev)); } static int assign_function_actag(struct ocxl_fn *fn) diff --git a/drivers/misc/pvpanic.c b/drivers/misc/pvpanic.c index a6e1a8983e1f..e16a5e51006e 100644 --- a/drivers/misc/pvpanic.c +++ b/drivers/misc/pvpanic.c @@ -143,13 +143,7 @@ static void pvpanic_unregister_acpi_driver(void) {} static int pvpanic_mmio_probe(struct platform_device *pdev) { - struct resource *mem; - - mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); - if (!mem) - return -EINVAL; - - base = devm_ioremap_resource(&pdev->dev, mem); + base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(base)) return PTR_ERR(base); diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index 4cb7a5b19467..56dd98ab5a81 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -370,7 +370,7 @@ static struct attribute *uacce_dev_attrs[] = { static umode_t uacce_dev_is_visible(struct kobject *kobj, struct attribute *attr, int n) { - struct device *dev = container_of(kobj, struct device, kobj); + struct device *dev = kobj_to_dev(kobj); struct uacce_device *uacce = to_uacce_device(dev); if (((attr == &dev_attr_region_mmio_size.attr) && diff --git a/drivers/misc/vmw_vmci/vmci_queue_pair.c b/drivers/misc/vmw_vmci/vmci_queue_pair.c index 8531ae781195..c49065887e8f 100644 --- a/drivers/misc/vmw_vmci/vmci_queue_pair.c +++ b/drivers/misc/vmw_vmci/vmci_queue_pair.c @@ -657,8 +657,9 @@ static int qp_host_get_user_memory(u64 produce_uva, if (retval < (int)produce_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(produce) failed (retval=%d)", retval); - qp_release_pages(produce_q->kernel_if->u.h.header_page, - retval, false); + if (retval > 0) + qp_release_pages(produce_q->kernel_if->u.h.header_page, + retval, false); err = VMCI_ERROR_NO_MEM; goto out; } @@ -670,8 +671,9 @@ static int qp_host_get_user_memory(u64 produce_uva, if (retval < (int)consume_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(consume) failed (retval=%d)", retval); - qp_release_pages(consume_q->kernel_if->u.h.header_page, - retval, false); + if (retval > 0) + qp_release_pages(consume_q->kernel_if->u.h.header_page, + retval, false); qp_release_pages(produce_q->kernel_if->u.h.header_page, produce_q->kernel_if->num_pages, false); err = VMCI_ERROR_NO_MEM; diff --git a/drivers/nvmem/core.c b/drivers/nvmem/core.c index 6cd3edb2eaf6..a09ff8409f60 100644 --- a/drivers/nvmem/core.c +++ b/drivers/nvmem/core.c @@ -128,7 +128,7 @@ static ssize_t bin_attr_nvmem_read(struct file *filp, struct kobject *kobj, if (attr->private) dev = attr->private; else - dev = container_of(kobj, struct device, kobj); + dev = kobj_to_dev(kobj); nvmem = to_nvmem_device(dev); /* Stop the user from reading */ @@ -168,7 +168,7 @@ static ssize_t bin_attr_nvmem_write(struct file *filp, struct kobject *kobj, if (attr->private) dev = attr->private; else - dev = container_of(kobj, struct device, kobj); + dev = kobj_to_dev(kobj); nvmem = to_nvmem_device(dev); /* Stop the user from writing */ @@ -219,7 +219,7 @@ static umode_t nvmem_bin_attr_get_umode(struct nvmem_device *nvmem) static umode_t nvmem_bin_attr_is_visible(struct kobject *kobj, struct bin_attribute *attr, int i) { - struct device *dev = container_of(kobj, struct device, kobj); + struct device *dev = kobj_to_dev(kobj); struct nvmem_device *nvmem = to_nvmem_device(dev); return nvmem_bin_attr_get_umode(nvmem); @@ -321,7 +321,7 @@ static void nvmem_release(struct device *dev) { struct nvmem_device *nvmem = to_nvmem_device(dev); - ida_simple_remove(&nvmem_ida, nvmem->id); + ida_free(&nvmem_ida, nvmem->id); gpiod_put(nvmem->wp_gpio); kfree(nvmem); } @@ -361,16 +361,14 @@ static void nvmem_cell_add(struct nvmem_cell *cell) blocking_notifier_call_chain(&nvmem_notifier, NVMEM_CELL_ADD, cell); } -static int nvmem_cell_info_to_nvmem_cell(struct nvmem_device *nvmem, - const struct nvmem_cell_info *info, - struct nvmem_cell *cell) +static int nvmem_cell_info_to_nvmem_cell_nodup(struct nvmem_device *nvmem, + const struct nvmem_cell_info *info, + struct nvmem_cell *cell) { cell->nvmem = nvmem; cell->offset = info->offset; cell->bytes = info->bytes; - cell->name = kstrdup_const(info->name, GFP_KERNEL); - if (!cell->name) - return -ENOMEM; + cell->name = info->name; cell->bit_offset = info->bit_offset; cell->nbits = info->nbits; @@ -382,13 +380,30 @@ static int nvmem_cell_info_to_nvmem_cell(struct nvmem_device *nvmem, if (!IS_ALIGNED(cell->offset, nvmem->stride)) { dev_err(&nvmem->dev, "cell %s unaligned to nvmem stride %d\n", - cell->name, nvmem->stride); + cell->name ?: "<unknown>", nvmem->stride); return -EINVAL; } return 0; } +static int nvmem_cell_info_to_nvmem_cell(struct nvmem_device *nvmem, + const struct nvmem_cell_info *info, + struct nvmem_cell *cell) +{ + int err; + + err = nvmem_cell_info_to_nvmem_cell_nodup(nvmem, info, cell); + if (err) + return err; + + cell->name = kstrdup_const(info->name, GFP_KERNEL); + if (!cell->name) + return -ENOMEM; + + return 0; +} + /** * nvmem_add_cells() - Add cell information to an nvmem device * @@ -596,7 +611,7 @@ struct nvmem_device *nvmem_register(const struct nvmem_config *config) if (!nvmem) return ERR_PTR(-ENOMEM); - rval = ida_simple_get(&nvmem_ida, 0, 0, GFP_KERNEL); + rval = ida_alloc(&nvmem_ida, GFP_KERNEL); if (rval < 0) { kfree(nvmem); return ERR_PTR(rval); @@ -608,7 +623,7 @@ struct nvmem_device *nvmem_register(const struct nvmem_config *config) nvmem->wp_gpio = gpiod_get_optional(config->dev, "wp", GPIOD_OUT_HIGH); if (IS_ERR(nvmem->wp_gpio)) { - ida_simple_remove(&nvmem_ida, nvmem->id); + ida_free(&nvmem_ida, nvmem->id); rval = PTR_ERR(nvmem->wp_gpio); kfree(nvmem); return ERR_PTR(rval); @@ -835,6 +850,7 @@ struct nvmem_device *of_nvmem_device_get(struct device_node *np, const char *id) { struct device_node *nvmem_np; + struct nvmem_device *nvmem; int index = 0; if (id) @@ -844,7 +860,9 @@ struct nvmem_device *of_nvmem_device_get(struct device_node *np, const char *id) if (!nvmem_np) return ERR_PTR(-ENOENT); - return __nvmem_device_get(nvmem_np, device_match_of_node); + nvmem = __nvmem_device_get(nvmem_np, device_match_of_node); + of_node_put(nvmem_np); + return nvmem; } EXPORT_SYMBOL_GPL(of_nvmem_device_get); #endif @@ -1460,7 +1478,7 @@ ssize_t nvmem_device_cell_read(struct nvmem_device *nvmem, if (!nvmem) return -EINVAL; - rc = nvmem_cell_info_to_nvmem_cell(nvmem, info, &cell); + rc = nvmem_cell_info_to_nvmem_cell_nodup(nvmem, info, &cell); if (rc) return rc; @@ -1490,7 +1508,7 @@ int nvmem_device_cell_write(struct nvmem_device *nvmem, if (!nvmem) return -EINVAL; - rc = nvmem_cell_info_to_nvmem_cell(nvmem, info, &cell); + rc = nvmem_cell_info_to_nvmem_cell_nodup(nvmem, info, &cell); if (rc) return rc; diff --git a/drivers/nvmem/mtk-efuse.c b/drivers/nvmem/mtk-efuse.c index 856d9c3fc38e..6a537d959f14 100644 --- a/drivers/nvmem/mtk-efuse.c +++ b/drivers/nvmem/mtk-efuse.c @@ -28,19 +28,6 @@ static int mtk_reg_read(void *context, return 0; } -static int mtk_reg_write(void *context, - unsigned int reg, void *_val, size_t bytes) -{ - struct mtk_efuse_priv *priv = context; - u32 *val = _val; - int i = 0, words = bytes / 4; - - while (words--) - writel(*val++, priv->base + reg + (i++ * 4)); - - return 0; -} - static int mtk_efuse_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -61,7 +48,6 @@ static int mtk_efuse_probe(struct platform_device *pdev) econfig.stride = 4; econfig.word_size = 4; econfig.reg_read = mtk_reg_read; - econfig.reg_write = mtk_reg_write; econfig.size = resource_size(res); econfig.priv = priv; econfig.dev = dev; diff --git a/drivers/power/supply/bq27xxx_battery_hdq.c b/drivers/power/supply/bq27xxx_battery_hdq.c index 29771967df2e..d56b3e19e996 100644 --- a/drivers/power/supply/bq27xxx_battery_hdq.c +++ b/drivers/power/supply/bq27xxx_battery_hdq.c @@ -104,7 +104,7 @@ static void bq27xxx_battery_hdq_remove_slave(struct w1_slave *sl) bq27xxx_battery_teardown(di); } -static struct w1_family_ops bq27xxx_battery_hdq_fops = { +static const struct w1_family_ops bq27xxx_battery_hdq_fops = { .add_slave = bq27xxx_battery_hdq_add_slave, .remove_slave = bq27xxx_battery_hdq_remove_slave, }; diff --git a/drivers/power/supply/ds2760_battery.c b/drivers/power/supply/ds2760_battery.c index 11bed88a89fa..695bb6747400 100644 --- a/drivers/power/supply/ds2760_battery.c +++ b/drivers/power/supply/ds2760_battery.c @@ -795,7 +795,7 @@ static const struct of_device_id w1_ds2760_of_ids[] = { }; #endif -static struct w1_family_ops w1_ds2760_fops = { +static const struct w1_family_ops w1_ds2760_fops = { .add_slave = w1_ds2760_add_slave, .remove_slave = w1_ds2760_remove_slave, .groups = w1_ds2760_groups, diff --git a/drivers/power/supply/max1721x_battery.c b/drivers/power/supply/max1721x_battery.c index 9ca895b0dabb..1b1a36f8e929 100644 --- a/drivers/power/supply/max1721x_battery.c +++ b/drivers/power/supply/max1721x_battery.c @@ -431,7 +431,7 @@ static int devm_w1_max1721x_add_device(struct w1_slave *sl) return 0; } -static struct w1_family_ops w1_max1721x_fops = { +static const struct w1_family_ops w1_max1721x_fops = { .add_slave = devm_w1_max1721x_add_device, }; diff --git a/drivers/slimbus/core.c b/drivers/slimbus/core.c index ae1e248a8fb8..1d2bc181da05 100644 --- a/drivers/slimbus/core.c +++ b/drivers/slimbus/core.c @@ -301,8 +301,6 @@ int slim_unregister_controller(struct slim_controller *ctrl) { /* Remove all clients */ device_for_each_child(ctrl->dev, NULL, slim_ctrl_remove_device); - /* Enter Clock Pause */ - slim_ctrl_clk_pause(ctrl, false, 0); ida_simple_remove(&ctrl_ida, ctrl->id); return 0; @@ -326,8 +324,8 @@ void slim_report_absent(struct slim_device *sbdev) mutex_lock(&ctrl->lock); sbdev->is_laddr_valid = false; mutex_unlock(&ctrl->lock); - - ida_simple_remove(&ctrl->laddr_ida, sbdev->laddr); + if (!ctrl->get_laddr) + ida_simple_remove(&ctrl->laddr_ida, sbdev->laddr); slim_device_update_status(sbdev, SLIM_DEVICE_STATUS_DOWN); } EXPORT_SYMBOL_GPL(slim_report_absent); diff --git a/drivers/slimbus/qcom-ngd-ctrl.c b/drivers/slimbus/qcom-ngd-ctrl.c index 743ee7b4e63f..218aefc3531c 100644 --- a/drivers/slimbus/qcom-ngd-ctrl.c +++ b/drivers/slimbus/qcom-ngd-ctrl.c @@ -1277,9 +1277,13 @@ static void qcom_slim_ngd_qmi_del_server(struct qmi_handle *hdl, { struct qcom_slim_ngd_qmi *qmi = container_of(hdl, struct qcom_slim_ngd_qmi, svc_event_hdl); + struct qcom_slim_ngd_ctrl *ctrl = + container_of(qmi, struct qcom_slim_ngd_ctrl, qmi); qmi->svc_info.sq_node = 0; qmi->svc_info.sq_port = 0; + + qcom_slim_ngd_enable(ctrl, false); } static struct qmi_ops qcom_slim_ngd_qmi_svc_event_ops = { diff --git a/drivers/soundwire/Kconfig b/drivers/soundwire/Kconfig index fa2b4ab92ed9..016e74230bb7 100644 --- a/drivers/soundwire/Kconfig +++ b/drivers/soundwire/Kconfig @@ -24,6 +24,7 @@ config SOUNDWIRE_CADENCE config SOUNDWIRE_INTEL tristate "Intel SoundWire Master driver" select SOUNDWIRE_CADENCE + select SOUNDWIRE_GENERIC_ALLOCATION depends on ACPI && SND_SOC help SoundWire Intel Master driver. @@ -33,11 +34,15 @@ config SOUNDWIRE_INTEL config SOUNDWIRE_QCOM tristate "Qualcomm SoundWire Master driver" - depends on SLIMBUS + imply SLIMBUS depends on SND_SOC help SoundWire Qualcomm Master driver. If you have an Qualcomm platform which has a SoundWire Master then enable this config option to get the SoundWire support for that device + +config SOUNDWIRE_GENERIC_ALLOCATION + tristate + endif diff --git a/drivers/soundwire/Makefile b/drivers/soundwire/Makefile index 7c53ffae9f50..bf1e250d50dd 100644 --- a/drivers/soundwire/Makefile +++ b/drivers/soundwire/Makefile @@ -8,6 +8,9 @@ soundwire-bus-y := bus_type.o bus.o master.o slave.o mipi_disco.o stream.o \ sysfs_slave.o sysfs_slave_dpn.o obj-$(CONFIG_SOUNDWIRE) += soundwire-bus.o +soundwire-generic-allocation-objs := generic_bandwidth_allocation.o +obj-$(CONFIG_SOUNDWIRE_GENERIC_ALLOCATION) += soundwire-generic-allocation.o + ifdef CONFIG_DEBUG_FS soundwire-bus-y += debugfs.o endif diff --git a/drivers/soundwire/bus.c b/drivers/soundwire/bus.c index da0201693c24..8eaf31e76677 100644 --- a/drivers/soundwire/bus.c +++ b/drivers/soundwire/bus.c @@ -61,6 +61,12 @@ int sdw_bus_master_add(struct sdw_bus *bus, struct device *parent, return -EINVAL; } + if (!bus->compute_params) { + dev_err(bus->dev, + "Bandwidth allocation not configured, compute_params no set\n"); + return -EINVAL; + } + mutex_init(&bus->msg_lock); mutex_init(&bus->bus_lock); INIT_LIST_HEAD(&bus->slaves); @@ -255,6 +261,21 @@ static int sdw_reset_page(struct sdw_bus *bus, u16 dev_num) return ret; } +static int sdw_transfer_unlocked(struct sdw_bus *bus, struct sdw_msg *msg) +{ + int ret; + + ret = do_transfer(bus, msg); + if (ret != 0 && ret != -ENODATA) + dev_err(bus->dev, "trf on Slave %d failed:%d\n", + msg->dev_num, ret); + + if (msg->page) + sdw_reset_page(bus, msg->dev_num); + + return ret; +} + /** * sdw_transfer() - Synchronous transfer message to a SDW Slave device * @bus: SDW bus @@ -266,13 +287,7 @@ int sdw_transfer(struct sdw_bus *bus, struct sdw_msg *msg) mutex_lock(&bus->msg_lock); - ret = do_transfer(bus, msg); - if (ret != 0 && ret != -ENODATA) - dev_err(bus->dev, "trf on Slave %d failed:%d\n", - msg->dev_num, ret); - - if (msg->page) - sdw_reset_page(bus, msg->dev_num); + ret = sdw_transfer_unlocked(bus, msg); mutex_unlock(&bus->msg_lock); @@ -347,8 +362,8 @@ int sdw_fill_msg(struct sdw_msg *msg, struct sdw_slave *slave, return -EINVAL; } - msg->addr_page1 = (addr >> SDW_REG_SHIFT(SDW_SCP_ADDRPAGE1_MASK)); - msg->addr_page2 = (addr >> SDW_REG_SHIFT(SDW_SCP_ADDRPAGE2_MASK)); + msg->addr_page1 = FIELD_GET(SDW_SCP_ADDRPAGE1_MASK, addr); + msg->addr_page2 = FIELD_GET(SDW_SCP_ADDRPAGE2_MASK, addr); msg->addr |= BIT(15); msg->page = true; @@ -428,6 +443,39 @@ sdw_bwrite_no_pm(struct sdw_bus *bus, u16 dev_num, u32 addr, u8 value) return sdw_transfer(bus, &msg); } +int sdw_bread_no_pm_unlocked(struct sdw_bus *bus, u16 dev_num, u32 addr) +{ + struct sdw_msg msg; + u8 buf; + int ret; + + ret = sdw_fill_msg(&msg, NULL, addr, 1, dev_num, + SDW_MSG_FLAG_READ, &buf); + if (ret) + return ret; + + ret = sdw_transfer_unlocked(bus, &msg); + if (ret < 0) + return ret; + + return buf; +} +EXPORT_SYMBOL(sdw_bread_no_pm_unlocked); + +int sdw_bwrite_no_pm_unlocked(struct sdw_bus *bus, u16 dev_num, u32 addr, u8 value) +{ + struct sdw_msg msg; + int ret; + + ret = sdw_fill_msg(&msg, NULL, addr, 1, dev_num, + SDW_MSG_FLAG_WRITE, &value); + if (ret) + return ret; + + return sdw_transfer_unlocked(bus, &msg); +} +EXPORT_SYMBOL(sdw_bwrite_no_pm_unlocked); + static int sdw_read_no_pm(struct sdw_slave *slave, u32 addr) { @@ -699,6 +747,15 @@ static int sdw_program_device_num(struct sdw_bus *bus) if (!found) { /* TODO: Park this device in Group 13 */ + + /* + * add Slave device even if there is no platform + * firmware description. There will be no driver probe + * but the user/integration will be able to see the + * device, enumeration status and device number in sysfs + */ + sdw_slave_add(bus, &id, NULL); + dev_err(bus->dev, "Slave Entry not found\n"); } @@ -1051,6 +1108,12 @@ int sdw_configure_dpn_intr(struct sdw_slave *slave, int ret; u8 val = 0; + if (slave->bus->params.s_data_mode != SDW_PORT_DATA_MODE_NORMAL) { + dev_dbg(&slave->dev, "TEST FAIL interrupt %s\n", + enable ? "on" : "off"); + mask |= SDW_DPN_INT_TEST_FAIL; + } + addr = SDW_DPN_INTMASK(port); /* Set/Clear port ready interrupt mask */ @@ -1184,13 +1247,13 @@ static int sdw_initialize_slave(struct sdw_slave *slave) return ret; /* - * Set bus clash, parity and SCP implementation - * defined interrupt mask - * TODO: Read implementation defined interrupt mask - * from Slave property + * Set SCP_INT1_MASK register, typically bus clash and + * implementation-defined interrupt mask. The Parity detection + * may not always be correct on startup so its use is + * device-dependent, it might e.g. only be enabled in + * steady-state after a couple of frames. */ - val = SDW_SCP_INT1_IMPL_DEF | SDW_SCP_INT1_BUS_CLASH | - SDW_SCP_INT1_PARITY; + val = slave->prop.scp_int1_mask; /* Enable SCP interrupts */ ret = sdw_update(slave, SDW_SCP_INTMASK1, val, val); @@ -1362,6 +1425,8 @@ static int sdw_handle_slave_alerts(struct sdw_slave *slave) unsigned long port; bool slave_notify = false; u8 buf, buf2[2], _buf, _buf2[2]; + bool parity_check; + bool parity_quirk; sdw_modify_slave_status(slave, SDW_SLAVE_ALERT); @@ -1394,12 +1459,18 @@ static int sdw_handle_slave_alerts(struct sdw_slave *slave) * interrupt */ if (buf & SDW_SCP_INT1_PARITY) { - dev_err(&slave->dev, "Parity error detected\n"); + parity_check = slave->prop.scp_int1_mask & SDW_SCP_INT1_PARITY; + parity_quirk = !slave->first_interrupt_done && + (slave->prop.quirks & SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY); + + if (parity_check && !parity_quirk) + dev_err(&slave->dev, "Parity error detected\n"); clear |= SDW_SCP_INT1_PARITY; } if (buf & SDW_SCP_INT1_BUS_CLASH) { - dev_err(&slave->dev, "Bus clash error detected\n"); + if (slave->prop.scp_int1_mask & SDW_SCP_INT1_BUS_CLASH) + dev_err(&slave->dev, "Bus clash detected\n"); clear |= SDW_SCP_INT1_BUS_CLASH; } @@ -1411,16 +1482,18 @@ static int sdw_handle_slave_alerts(struct sdw_slave *slave) */ if (buf & SDW_SCP_INT1_IMPL_DEF) { - dev_dbg(&slave->dev, "Slave impl defined interrupt\n"); + if (slave->prop.scp_int1_mask & SDW_SCP_INT1_IMPL_DEF) { + dev_dbg(&slave->dev, "Slave impl defined interrupt\n"); + slave_notify = true; + } clear |= SDW_SCP_INT1_IMPL_DEF; - slave_notify = true; } /* Check port 0 - 3 interrupts */ port = buf & SDW_SCP_INT1_PORT0_3; /* To get port number corresponding to bits, shift it */ - port = port >> SDW_REG_SHIFT(SDW_SCP_INT1_PORT0_3); + port = FIELD_GET(SDW_SCP_INT1_PORT0_3, port); for_each_set_bit(bit, &port, 8) { sdw_handle_port_interrupt(slave, bit, &port_status[bit]); @@ -1468,6 +1541,9 @@ static int sdw_handle_slave_alerts(struct sdw_slave *slave) goto io_err; } + /* at this point all initial interrupt sources were handled */ + slave->first_interrupt_done = true; + /* * Read status again to ensure no new interrupts arrived * while servicing interrupts. @@ -1670,8 +1746,10 @@ void sdw_clear_slave_status(struct sdw_bus *bus, u32 request) if (!slave) continue; - if (slave->status != SDW_SLAVE_UNATTACHED) + if (slave->status != SDW_SLAVE_UNATTACHED) { sdw_modify_slave_status(slave, SDW_SLAVE_UNATTACHED); + slave->first_interrupt_done = false; + } /* keep track of request, used in pm_runtime resume */ slave->unattach_request = request; diff --git a/drivers/soundwire/bus.h b/drivers/soundwire/bus.h index 82484f741168..2e049d39c6e5 100644 --- a/drivers/soundwire/bus.h +++ b/drivers/soundwire/bus.h @@ -19,6 +19,8 @@ static inline int sdw_acpi_find_slaves(struct sdw_bus *bus) int sdw_of_find_slaves(struct sdw_bus *bus); void sdw_extract_slave_id(struct sdw_bus *bus, u64 addr, struct sdw_slave_id *id); +int sdw_slave_add(struct sdw_bus *bus, struct sdw_slave_id *id, + struct fwnode_handle *fwnode); int sdw_master_device_add(struct sdw_bus *bus, struct device *parent, struct fwnode_handle *fwnode); int sdw_master_device_del(struct sdw_bus *bus); @@ -69,6 +71,7 @@ struct sdw_msg { }; #define SDW_DOUBLE_RATE_FACTOR 2 +#define SDW_STRM_RATE_GROUPING 1 extern int sdw_rows[SDW_FRAME_ROWS]; extern int sdw_cols[SDW_FRAME_COLS]; @@ -154,9 +157,50 @@ int sdw_transfer_defer(struct sdw_bus *bus, struct sdw_msg *msg, int sdw_fill_msg(struct sdw_msg *msg, struct sdw_slave *slave, u32 addr, size_t count, u16 dev_num, u8 flags, u8 *buf); +/* Retrieve and return channel count from channel mask */ +static inline int sdw_ch_mask_to_ch(int ch_mask) +{ + int c = 0; + + for (c = 0; ch_mask; ch_mask >>= 1) + c += ch_mask & 1; + + return c; +} + +/* Fill transport parameter data structure */ +static inline void sdw_fill_xport_params(struct sdw_transport_params *params, + int port_num, bool grp_ctrl_valid, + int grp_ctrl, int sample_int, + int off1, int off2, + int hstart, int hstop, + int pack_mode, int lane_ctrl) +{ + params->port_num = port_num; + params->blk_grp_ctrl_valid = grp_ctrl_valid; + params->blk_grp_ctrl = grp_ctrl; + params->sample_interval = sample_int; + params->offset1 = off1; + params->offset2 = off2; + params->hstart = hstart; + params->hstop = hstop; + params->blk_pkg_mode = pack_mode; + params->lane_ctrl = lane_ctrl; +} + +/* Fill port parameter data structure */ +static inline void sdw_fill_port_params(struct sdw_port_params *params, + int port_num, int bps, + int flow_mode, int data_mode) +{ + params->num = port_num; + params->bps = bps; + params->flow_mode = flow_mode; + params->data_mode = data_mode; +} + /* Read-Modify-Write Slave register */ -static inline int -sdw_update(struct sdw_slave *slave, u32 addr, u8 mask, u8 val) +static inline int sdw_update(struct sdw_slave *slave, u32 addr, u8 mask, u8 val) { int tmp; @@ -168,6 +212,10 @@ sdw_update(struct sdw_slave *slave, u32 addr, u8 mask, u8 val) return sdw_write(slave, addr, tmp); } +/* broadcast read/write for tests */ +int sdw_bread_no_pm_unlocked(struct sdw_bus *bus, u16 dev_num, u32 addr); +int sdw_bwrite_no_pm_unlocked(struct sdw_bus *bus, u16 dev_num, u32 addr, u8 value); + /* * At the moment we only track Master-initiated hw_reset. * Additional fields can be added as needed diff --git a/drivers/soundwire/bus_type.c b/drivers/soundwire/bus_type.c index 6fba55898cf0..575b9bad99d5 100644 --- a/drivers/soundwire/bus_type.c +++ b/drivers/soundwire/bus_type.c @@ -84,6 +84,15 @@ static int sdw_drv_probe(struct device *dev) const struct sdw_device_id *id; int ret; + /* + * fw description is mandatory to bind + */ + if (!dev->fwnode) + return -ENODEV; + + if (!IS_ENABLED(CONFIG_ACPI) && !dev->of_node) + return -ENODEV; + id = sdw_get_device_id(slave, drv); if (!id) return -ENODEV; diff --git a/drivers/soundwire/cadence_master.c b/drivers/soundwire/cadence_master.c index 24eafe0aa1c3..9fa55164354a 100644 --- a/drivers/soundwire/cadence_master.c +++ b/drivers/soundwire/cadence_master.c @@ -13,6 +13,7 @@ #include <linux/io.h> #include <linux/module.h> #include <linux/mod_devicetable.h> +#include <linux/pm_runtime.h> #include <linux/soundwire/sdw_registers.h> #include <linux/soundwire/sdw.h> #include <sound/pcm_params.h> @@ -50,11 +51,14 @@ MODULE_PARM_DESC(cdns_mcp_int_mask, "Cadence MCP IntMask"); #define CDNS_MCP_CONTROL_BLOCK_WAKEUP BIT(0) #define CDNS_MCP_CMDCTRL 0x8 + +#define CDNS_MCP_CMDCTRL_INSERT_PARITY_ERR BIT(2) + #define CDNS_MCP_SSPSTAT 0xC #define CDNS_MCP_FRAME_SHAPE 0x10 #define CDNS_MCP_FRAME_SHAPE_INIT 0x14 #define CDNS_MCP_FRAME_SHAPE_COL_MASK GENMASK(2, 0) -#define CDNS_MCP_FRAME_SHAPE_ROW_OFFSET 3 +#define CDNS_MCP_FRAME_SHAPE_ROW_MASK GENMASK(7, 3) #define CDNS_MCP_CONFIG_UPDATE 0x18 #define CDNS_MCP_CONFIG_UPDATE_BIT BIT(0) @@ -129,8 +133,7 @@ MODULE_PARM_DESC(cdns_mcp_int_mask, "Cadence MCP IntMask"); #define CDNS_MCP_CMD_SSP_TAG BIT(31) #define CDNS_MCP_CMD_COMMAND GENMASK(30, 28) #define CDNS_MCP_CMD_DEV_ADDR GENMASK(27, 24) -#define CDNS_MCP_CMD_REG_ADDR_H GENMASK(23, 16) -#define CDNS_MCP_CMD_REG_ADDR_L GENMASK(15, 8) +#define CDNS_MCP_CMD_REG_ADDR GENMASK(23, 8) #define CDNS_MCP_CMD_REG_DATA GENMASK(7, 0) #define CDNS_MCP_CMD_READ 2 @@ -172,6 +175,7 @@ MODULE_PARM_DESC(cdns_mcp_int_mask, "Cadence MCP IntMask"); #define CDNS_DPN_HCTRL_LCTRL GENMASK(10, 8) #define CDNS_PORTCTRL 0x130 +#define CDNS_PORTCTRL_TEST_FAILED BIT(1) #define CDNS_PORTCTRL_DIRN BIT(7) #define CDNS_PORTCTRL_BANK_INVERT BIT(8) @@ -367,6 +371,85 @@ static int cdns_hw_reset(void *data, u64 value) DEFINE_DEBUGFS_ATTRIBUTE(cdns_hw_reset_fops, NULL, cdns_hw_reset, "%llu\n"); +static int cdns_parity_error_injection(void *data, u64 value) +{ + struct sdw_cdns *cdns = data; + struct sdw_bus *bus; + int ret; + + if (value != 1) + return -EINVAL; + + bus = &cdns->bus; + + /* + * Resume Master device. If this results in a bus reset, the + * Slave devices will re-attach and be re-enumerated. + */ + ret = pm_runtime_get_sync(bus->dev); + if (ret < 0 && ret != -EACCES) { + dev_err_ratelimited(cdns->dev, + "pm_runtime_get_sync failed in %s, ret %d\n", + __func__, ret); + pm_runtime_put_noidle(bus->dev); + return ret; + } + + /* + * wait long enough for Slave(s) to be in steady state. This + * does not need to be super precise. + */ + msleep(200); + + /* + * Take the bus lock here to make sure that any bus transactions + * will be queued while we inject a parity error on a dummy read + */ + mutex_lock(&bus->bus_lock); + + /* program hardware to inject parity error */ + cdns_updatel(cdns, CDNS_MCP_CMDCTRL, + CDNS_MCP_CMDCTRL_INSERT_PARITY_ERR, + CDNS_MCP_CMDCTRL_INSERT_PARITY_ERR); + + /* commit changes */ + cdns_updatel(cdns, CDNS_MCP_CONFIG_UPDATE, + CDNS_MCP_CONFIG_UPDATE_BIT, + CDNS_MCP_CONFIG_UPDATE_BIT); + + /* do a broadcast dummy read to avoid bus clashes */ + ret = sdw_bread_no_pm_unlocked(&cdns->bus, 0xf, SDW_SCP_DEVID_0); + dev_info(cdns->dev, "parity error injection, read: %d\n", ret); + + /* program hardware to disable parity error */ + cdns_updatel(cdns, CDNS_MCP_CMDCTRL, + CDNS_MCP_CMDCTRL_INSERT_PARITY_ERR, + 0); + + /* commit changes */ + cdns_updatel(cdns, CDNS_MCP_CONFIG_UPDATE, + CDNS_MCP_CONFIG_UPDATE_BIT, + CDNS_MCP_CONFIG_UPDATE_BIT); + + /* Continue bus operation with parity error injection disabled */ + mutex_unlock(&bus->bus_lock); + + /* Userspace changed the hardware state behind the kernel's back */ + add_taint(TAINT_USER, LOCKDEP_STILL_OK); + + /* + * allow Master device to enter pm_runtime suspend. This may + * also result in Slave devices suspending. + */ + pm_runtime_mark_last_busy(bus->dev); + pm_runtime_put_autosuspend(bus->dev); + + return 0; +} + +DEFINE_DEBUGFS_ATTRIBUTE(cdns_parity_error_fops, NULL, + cdns_parity_error_injection, "%llu\n"); + /** * sdw_cdns_debugfs_init() - Cadence debugfs init * @cdns: Cadence instance @@ -378,6 +461,9 @@ void sdw_cdns_debugfs_init(struct sdw_cdns *cdns, struct dentry *root) debugfs_create_file("cdns-hw-reset", 0200, root, cdns, &cdns_hw_reset_fops); + + debugfs_create_file("cdns-parity-error-injection", 0200, root, cdns, + &cdns_parity_error_fops); } EXPORT_SYMBOL_GPL(sdw_cdns_debugfs_init); @@ -417,8 +503,7 @@ cdns_fill_msg_resp(struct sdw_cdns *cdns, /* fill response */ for (i = 0; i < count; i++) - msg->buf[i + offset] = cdns->response_buf[i] >> - SDW_REG_SHIFT(CDNS_MCP_RESP_RDATA); + msg->buf[i + offset] = FIELD_GET(CDNS_MCP_RESP_RDATA, cdns->response_buf[i]); return SDW_CMD_OK; } @@ -441,14 +526,15 @@ _cdns_xfer_msg(struct sdw_cdns *cdns, struct sdw_msg *msg, int cmd, addr = msg->addr; for (i = 0; i < count; i++) { - data = msg->dev_num << SDW_REG_SHIFT(CDNS_MCP_CMD_DEV_ADDR); - data |= cmd << SDW_REG_SHIFT(CDNS_MCP_CMD_COMMAND); - data |= addr++ << SDW_REG_SHIFT(CDNS_MCP_CMD_REG_ADDR_L); + data = FIELD_PREP(CDNS_MCP_CMD_DEV_ADDR, msg->dev_num); + data |= FIELD_PREP(CDNS_MCP_CMD_COMMAND, cmd); + data |= FIELD_PREP(CDNS_MCP_CMD_REG_ADDR, addr); + addr++; if (msg->flags == SDW_MSG_FLAG_WRITE) data |= msg->buf[i + offset]; - data |= msg->ssp_sync << SDW_REG_SHIFT(CDNS_MCP_CMD_SSP_TAG); + data |= FIELD_PREP(CDNS_MCP_CMD_SSP_TAG, msg->ssp_sync); cdns_writel(cdns, base, data); base += CDNS_MCP_CMD_WORD_LEN; } @@ -483,12 +569,12 @@ cdns_program_scp_addr(struct sdw_cdns *cdns, struct sdw_msg *msg) cdns->msg_count = CDNS_SCP_RX_FIFOLEVEL; } - data[0] = msg->dev_num << SDW_REG_SHIFT(CDNS_MCP_CMD_DEV_ADDR); - data[0] |= 0x3 << SDW_REG_SHIFT(CDNS_MCP_CMD_COMMAND); + data[0] = FIELD_PREP(CDNS_MCP_CMD_DEV_ADDR, msg->dev_num); + data[0] |= FIELD_PREP(CDNS_MCP_CMD_COMMAND, 0x3); data[1] = data[0]; - data[0] |= SDW_SCP_ADDRPAGE1 << SDW_REG_SHIFT(CDNS_MCP_CMD_REG_ADDR_L); - data[1] |= SDW_SCP_ADDRPAGE2 << SDW_REG_SHIFT(CDNS_MCP_CMD_REG_ADDR_L); + data[0] |= FIELD_PREP(CDNS_MCP_CMD_REG_ADDR, SDW_SCP_ADDRPAGE1); + data[1] |= FIELD_PREP(CDNS_MCP_CMD_REG_ADDR, SDW_SCP_ADDRPAGE2); data[0] |= msg->addr_page1; data[1] |= msg->addr_page2; @@ -785,13 +871,35 @@ irqreturn_t sdw_cdns_irq(int irq, void *dev_id) dev_err_ratelimited(cdns->dev, "Bus clash for data word\n"); } + if (cdns->bus.params.m_data_mode != SDW_PORT_DATA_MODE_NORMAL && + int_status & CDNS_MCP_INT_DPINT) { + u32 port_intstat; + + /* just log which ports report an error */ + port_intstat = cdns_readl(cdns, CDNS_MCP_PORT_INTSTAT); + dev_err_ratelimited(cdns->dev, "DP interrupt: PortIntStat %8x\n", + port_intstat); + + /* clear status w/ write1 */ + cdns_writel(cdns, CDNS_MCP_PORT_INTSTAT, port_intstat); + } + if (int_status & CDNS_MCP_INT_SLAVE_MASK) { /* Mask the Slave interrupt and wake thread */ cdns_updatel(cdns, CDNS_MCP_INTMASK, CDNS_MCP_INT_SLAVE_MASK, 0); int_status &= ~CDNS_MCP_INT_SLAVE_MASK; - schedule_work(&cdns->work); + + /* + * Deal with possible race condition between interrupt + * handling and disabling interrupts on suspend. + * + * If the master is in the process of disabling + * interrupts, don't schedule a workqueue + */ + if (cdns->interrupt_enabled) + schedule_work(&cdns->work); } cdns_writel(cdns, CDNS_MCP_INTSTAT, int_status); @@ -900,7 +1008,9 @@ int sdw_cdns_enable_interrupt(struct sdw_cdns *cdns, bool state) mask |= CDNS_MCP_INT_CTRL_CLASH | CDNS_MCP_INT_DATA_CLASH | CDNS_MCP_INT_PARITY; - /* no detection of port interrupts for now */ + /* port interrupt limited to test modes for now */ + if (cdns->bus.params.m_data_mode != SDW_PORT_DATA_MODE_NORMAL) + mask |= CDNS_MCP_INT_DPINT; /* enable detection of RX fifo level */ mask |= CDNS_MCP_INT_RX_WL; @@ -924,6 +1034,19 @@ update_masks: slave_state = cdns_readl(cdns, CDNS_MCP_SLAVE_INTSTAT1); cdns_writel(cdns, CDNS_MCP_SLAVE_INTSTAT1, slave_state); } + cdns->interrupt_enabled = state; + + /* + * Complete any on-going status updates before updating masks, + * and cancel queued status updates. + * + * There could be a race with a new interrupt thrown before + * the 3 mask updates below are complete, so in the interrupt + * we use the 'interrupt_enabled' status to prevent new work + * from being queued. + */ + if (!state) + cancel_work_sync(&cdns->work); cdns_writel(cdns, CDNS_MCP_SLAVE_INTMASK0, slave_intmask0); cdns_writel(cdns, CDNS_MCP_SLAVE_INTMASK1, slave_intmask1); @@ -1041,9 +1164,10 @@ static u32 cdns_set_initial_frame_shape(int n_rows, int n_cols) int r; r = sdw_find_row_index(n_rows); - c = sdw_find_col_index(n_cols) & CDNS_MCP_FRAME_SHAPE_COL_MASK; + c = sdw_find_col_index(n_cols); - val = (r << CDNS_MCP_FRAME_SHAPE_ROW_OFFSET) | c; + val = FIELD_PREP(CDNS_MCP_FRAME_SHAPE_ROW_MASK, r); + val |= FIELD_PREP(CDNS_MCP_FRAME_SHAPE_COL_MASK, c); return val; } @@ -1170,12 +1294,9 @@ static int cdns_port_params(struct sdw_bus *bus, dpn_config = cdns_readl(cdns, dpn_config_off); - dpn_config |= ((p_params->bps - 1) << - SDW_REG_SHIFT(CDNS_DPN_CONFIG_WL)); - dpn_config |= (p_params->flow_mode << - SDW_REG_SHIFT(CDNS_DPN_CONFIG_PORT_FLOW)); - dpn_config |= (p_params->data_mode << - SDW_REG_SHIFT(CDNS_DPN_CONFIG_PORT_DAT)); + u32p_replace_bits(&dpn_config, (p_params->bps - 1), CDNS_DPN_CONFIG_WL); + u32p_replace_bits(&dpn_config, p_params->flow_mode, CDNS_DPN_CONFIG_PORT_FLOW); + u32p_replace_bits(&dpn_config, p_params->data_mode, CDNS_DPN_CONFIG_PORT_DAT); cdns_writel(cdns, dpn_config_off, dpn_config); @@ -1211,24 +1332,17 @@ static int cdns_transport_params(struct sdw_bus *bus, } dpn_config = cdns_readl(cdns, dpn_config_off); - - dpn_config |= (t_params->blk_grp_ctrl << - SDW_REG_SHIFT(CDNS_DPN_CONFIG_BGC)); - dpn_config |= (t_params->blk_pkg_mode << - SDW_REG_SHIFT(CDNS_DPN_CONFIG_BPM)); + u32p_replace_bits(&dpn_config, t_params->blk_grp_ctrl, CDNS_DPN_CONFIG_BGC); + u32p_replace_bits(&dpn_config, t_params->blk_pkg_mode, CDNS_DPN_CONFIG_BPM); cdns_writel(cdns, dpn_config_off, dpn_config); - dpn_offsetctrl |= (t_params->offset1 << - SDW_REG_SHIFT(CDNS_DPN_OFFSET_CTRL_1)); - dpn_offsetctrl |= (t_params->offset2 << - SDW_REG_SHIFT(CDNS_DPN_OFFSET_CTRL_2)); + u32p_replace_bits(&dpn_offsetctrl, t_params->offset1, CDNS_DPN_OFFSET_CTRL_1); + u32p_replace_bits(&dpn_offsetctrl, t_params->offset2, CDNS_DPN_OFFSET_CTRL_2); cdns_writel(cdns, dpn_offsetctrl_off, dpn_offsetctrl); - dpn_hctrl |= (t_params->hstart << - SDW_REG_SHIFT(CDNS_DPN_HCTRL_HSTART)); - dpn_hctrl |= (t_params->hstop << SDW_REG_SHIFT(CDNS_DPN_HCTRL_HSTOP)); - dpn_hctrl |= (t_params->lane_ctrl << - SDW_REG_SHIFT(CDNS_DPN_HCTRL_LCTRL)); + u32p_replace_bits(&dpn_hctrl, t_params->hstart, CDNS_DPN_HCTRL_HSTART); + u32p_replace_bits(&dpn_hctrl, t_params->hstop, CDNS_DPN_HCTRL_HSTOP); + u32p_replace_bits(&dpn_hctrl, t_params->lane_ctrl, CDNS_DPN_HCTRL_LCTRL); cdns_writel(cdns, dpn_hctrl_off, dpn_hctrl); cdns_writel(cdns, dpn_samplectrl_off, (t_params->sample_interval - 1)); @@ -1526,15 +1640,20 @@ void sdw_cdns_config_stream(struct sdw_cdns *cdns, { u32 offset, val = 0; - if (dir == SDW_DATA_DIR_RX) + if (dir == SDW_DATA_DIR_RX) { val = CDNS_PORTCTRL_DIRN; + if (cdns->bus.params.m_data_mode != SDW_PORT_DATA_MODE_NORMAL) + val |= CDNS_PORTCTRL_TEST_FAILED; + } offset = CDNS_PORTCTRL + pdi->num * CDNS_PORT_OFFSET; - cdns_updatel(cdns, offset, CDNS_PORTCTRL_DIRN, val); + cdns_updatel(cdns, offset, + CDNS_PORTCTRL_DIRN | CDNS_PORTCTRL_TEST_FAILED, + val); val = pdi->num; val |= CDNS_PDI_CONFIG_SOFT_RESET; - val |= ((1 << ch) - 1) << SDW_REG_SHIFT(CDNS_PDI_CONFIG_CHANNEL); + val |= FIELD_PREP(CDNS_PDI_CONFIG_CHANNEL, (1 << ch) - 1); cdns_writel(cdns, CDNS_PDI_CONFIG(pdi->num), val); } EXPORT_SYMBOL(sdw_cdns_config_stream); diff --git a/drivers/soundwire/cadence_master.h b/drivers/soundwire/cadence_master.h index 7638858397df..4d1aab5b5ec2 100644 --- a/drivers/soundwire/cadence_master.h +++ b/drivers/soundwire/cadence_master.h @@ -84,6 +84,8 @@ struct sdw_cdns_stream_config { * @bus: Bus handle * @stream_type: Stream type * @link_id: Master link id + * @hw_params: hw_params to be applied in .prepare step + * @suspended: status set when suspended, to be used in .prepare */ struct sdw_cdns_dma_data { char *name; @@ -92,6 +94,8 @@ struct sdw_cdns_dma_data { struct sdw_bus *bus; enum sdw_stream_type stream_type; int link_id; + struct snd_pcm_hw_params *hw_params; + bool suspended; }; /** @@ -129,6 +133,7 @@ struct sdw_cdns { bool link_up; unsigned int msg_count; + bool interrupt_enabled; struct work_struct work; diff --git a/drivers/soundwire/generic_bandwidth_allocation.c b/drivers/soundwire/generic_bandwidth_allocation.c new file mode 100644 index 000000000000..0bdef38c9a30 --- /dev/null +++ b/drivers/soundwire/generic_bandwidth_allocation.c @@ -0,0 +1,425 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-3-Clause) +// Copyright(c) 2015-2020 Intel Corporation. + +/* + * Bandwidth management algorithm based on 2^n gears + * + */ + +#include <linux/device.h> +#include <linux/module.h> +#include <linux/mod_devicetable.h> +#include <linux/slab.h> +#include <linux/soundwire/sdw.h> +#include "bus.h" + +#define SDW_STRM_RATE_GROUPING 1 + +struct sdw_group_params { + unsigned int rate; + int full_bw; + int payload_bw; + int hwidth; +}; + +struct sdw_group { + unsigned int count; + unsigned int max_size; + unsigned int *rates; +}; + +struct sdw_transport_data { + int hstart; + int hstop; + int block_offset; + int sub_block_offset; +}; + +static void sdw_compute_slave_ports(struct sdw_master_runtime *m_rt, + struct sdw_transport_data *t_data) +{ + struct sdw_slave_runtime *s_rt = NULL; + struct sdw_port_runtime *p_rt; + int port_bo, sample_int; + unsigned int rate, bps, ch = 0; + unsigned int slave_total_ch; + struct sdw_bus_params *b_params = &m_rt->bus->params; + + port_bo = t_data->block_offset; + + list_for_each_entry(s_rt, &m_rt->slave_rt_list, m_rt_node) { + rate = m_rt->stream->params.rate; + bps = m_rt->stream->params.bps; + sample_int = (m_rt->bus->params.curr_dr_freq / rate); + slave_total_ch = 0; + + list_for_each_entry(p_rt, &s_rt->port_list, port_node) { + ch = sdw_ch_mask_to_ch(p_rt->ch_mask); + + sdw_fill_xport_params(&p_rt->transport_params, + p_rt->num, false, + SDW_BLK_GRP_CNT_1, + sample_int, port_bo, port_bo >> 8, + t_data->hstart, + t_data->hstop, + (SDW_BLK_GRP_CNT_1 * ch), 0x0); + + sdw_fill_port_params(&p_rt->port_params, + p_rt->num, bps, + SDW_PORT_FLOW_MODE_ISOCH, + b_params->s_data_mode); + + port_bo += bps * ch; + slave_total_ch += ch; + } + + if (m_rt->direction == SDW_DATA_DIR_TX && + m_rt->ch_count == slave_total_ch) { + /* + * Slave devices were configured to access all channels + * of the stream, which indicates that they operate in + * 'mirror mode'. Make sure we reset the port offset for + * the next device in the list + */ + port_bo = t_data->block_offset; + } + } +} + +static void sdw_compute_master_ports(struct sdw_master_runtime *m_rt, + struct sdw_group_params *params, + int port_bo, int hstop) +{ + struct sdw_transport_data t_data = {0}; + struct sdw_port_runtime *p_rt; + struct sdw_bus *bus = m_rt->bus; + struct sdw_bus_params *b_params = &bus->params; + int sample_int, hstart = 0; + unsigned int rate, bps, ch, no_ch; + + rate = m_rt->stream->params.rate; + bps = m_rt->stream->params.bps; + ch = m_rt->ch_count; + sample_int = (bus->params.curr_dr_freq / rate); + + if (rate != params->rate) + return; + + t_data.hstop = hstop; + hstart = hstop - params->hwidth + 1; + t_data.hstart = hstart; + + list_for_each_entry(p_rt, &m_rt->port_list, port_node) { + no_ch = sdw_ch_mask_to_ch(p_rt->ch_mask); + + sdw_fill_xport_params(&p_rt->transport_params, p_rt->num, + false, SDW_BLK_GRP_CNT_1, sample_int, + port_bo, port_bo >> 8, hstart, hstop, + (SDW_BLK_GRP_CNT_1 * no_ch), 0x0); + + sdw_fill_port_params(&p_rt->port_params, + p_rt->num, bps, + SDW_PORT_FLOW_MODE_ISOCH, + b_params->m_data_mode); + + /* Check for first entry */ + if (!(p_rt == list_first_entry(&m_rt->port_list, + struct sdw_port_runtime, + port_node))) { + port_bo += bps * ch; + continue; + } + + t_data.hstart = hstart; + t_data.hstop = hstop; + t_data.block_offset = port_bo; + t_data.sub_block_offset = 0; + port_bo += bps * ch; + } + + sdw_compute_slave_ports(m_rt, &t_data); +} + +static void _sdw_compute_port_params(struct sdw_bus *bus, + struct sdw_group_params *params, int count) +{ + struct sdw_master_runtime *m_rt = NULL; + int hstop = bus->params.col - 1; + int block_offset, port_bo, i; + + /* Run loop for all groups to compute transport parameters */ + for (i = 0; i < count; i++) { + port_bo = 1; + block_offset = 1; + + list_for_each_entry(m_rt, &bus->m_rt_list, bus_node) { + sdw_compute_master_ports(m_rt, ¶ms[i], + port_bo, hstop); + + block_offset += m_rt->ch_count * + m_rt->stream->params.bps; + port_bo = block_offset; + } + + hstop = hstop - params[i].hwidth; + } +} + +static int sdw_compute_group_params(struct sdw_bus *bus, + struct sdw_group_params *params, + int *rates, int count) +{ + struct sdw_master_runtime *m_rt = NULL; + int sel_col = bus->params.col; + unsigned int rate, bps, ch; + int i, column_needed = 0; + + /* Calculate bandwidth per group */ + for (i = 0; i < count; i++) { + params[i].rate = rates[i]; + params[i].full_bw = bus->params.curr_dr_freq / params[i].rate; + } + + list_for_each_entry(m_rt, &bus->m_rt_list, bus_node) { + rate = m_rt->stream->params.rate; + bps = m_rt->stream->params.bps; + ch = m_rt->ch_count; + + for (i = 0; i < count; i++) { + if (rate == params[i].rate) + params[i].payload_bw += bps * ch; + } + } + + for (i = 0; i < count; i++) { + params[i].hwidth = (sel_col * + params[i].payload_bw + params[i].full_bw - 1) / + params[i].full_bw; + + column_needed += params[i].hwidth; + } + + if (column_needed > sel_col - 1) + return -EINVAL; + + return 0; +} + +static int sdw_add_element_group_count(struct sdw_group *group, + unsigned int rate) +{ + int num = group->count; + int i; + + for (i = 0; i <= num; i++) { + if (rate == group->rates[i]) + break; + + if (i != num) + continue; + + if (group->count >= group->max_size) { + unsigned int *rates; + + group->max_size += 1; + rates = krealloc(group->rates, + (sizeof(int) * group->max_size), + GFP_KERNEL); + if (!rates) + return -ENOMEM; + group->rates = rates; + } + + group->rates[group->count++] = rate; + } + + return 0; +} + +static int sdw_get_group_count(struct sdw_bus *bus, + struct sdw_group *group) +{ + struct sdw_master_runtime *m_rt; + unsigned int rate; + int ret = 0; + + group->count = 0; + group->max_size = SDW_STRM_RATE_GROUPING; + group->rates = kcalloc(group->max_size, sizeof(int), GFP_KERNEL); + if (!group->rates) + return -ENOMEM; + + list_for_each_entry(m_rt, &bus->m_rt_list, bus_node) { + rate = m_rt->stream->params.rate; + if (m_rt == list_first_entry(&bus->m_rt_list, + struct sdw_master_runtime, + bus_node)) { + group->rates[group->count++] = rate; + + } else { + ret = sdw_add_element_group_count(group, rate); + if (ret < 0) { + kfree(group->rates); + return ret; + } + } + } + + return ret; +} + +/** + * sdw_compute_port_params: Compute transport and port parameters + * + * @bus: SDW Bus instance + */ +static int sdw_compute_port_params(struct sdw_bus *bus) +{ + struct sdw_group_params *params = NULL; + struct sdw_group group; + int ret; + + ret = sdw_get_group_count(bus, &group); + if (ret < 0) + return ret; + + if (group.count == 0) + goto out; + + params = kcalloc(group.count, sizeof(*params), GFP_KERNEL); + if (!params) { + ret = -ENOMEM; + goto out; + } + + /* Compute transport parameters for grouped streams */ + ret = sdw_compute_group_params(bus, params, + &group.rates[0], group.count); + if (ret < 0) + goto free_params; + + _sdw_compute_port_params(bus, params, group.count); + +free_params: + kfree(params); +out: + kfree(group.rates); + + return ret; +} + +static int sdw_select_row_col(struct sdw_bus *bus, int clk_freq) +{ + struct sdw_master_prop *prop = &bus->prop; + int frame_int, frame_freq; + int r, c; + + for (c = 0; c < SDW_FRAME_COLS; c++) { + for (r = 0; r < SDW_FRAME_ROWS; r++) { + if (sdw_rows[r] != prop->default_row || + sdw_cols[c] != prop->default_col) + continue; + + frame_int = sdw_rows[r] * sdw_cols[c]; + frame_freq = clk_freq / frame_int; + + if ((clk_freq - (frame_freq * SDW_FRAME_CTRL_BITS)) < + bus->params.bandwidth) + continue; + + bus->params.row = sdw_rows[r]; + bus->params.col = sdw_cols[c]; + return 0; + } + } + + return -EINVAL; +} + +/** + * sdw_compute_bus_params: Compute bus parameters + * + * @bus: SDW Bus instance + */ +static int sdw_compute_bus_params(struct sdw_bus *bus) +{ + unsigned int max_dr_freq, curr_dr_freq = 0; + struct sdw_master_prop *mstr_prop = &bus->prop; + int i, clk_values, ret; + bool is_gear = false; + u32 *clk_buf; + + if (mstr_prop->num_clk_gears) { + clk_values = mstr_prop->num_clk_gears; + clk_buf = mstr_prop->clk_gears; + is_gear = true; + } else if (mstr_prop->num_clk_freq) { + clk_values = mstr_prop->num_clk_freq; + clk_buf = mstr_prop->clk_freq; + } else { + clk_values = 1; + clk_buf = NULL; + } + + max_dr_freq = mstr_prop->max_clk_freq * SDW_DOUBLE_RATE_FACTOR; + + for (i = 0; i < clk_values; i++) { + if (!clk_buf) + curr_dr_freq = max_dr_freq; + else + curr_dr_freq = (is_gear) ? + (max_dr_freq >> clk_buf[i]) : + clk_buf[i] * SDW_DOUBLE_RATE_FACTOR; + + if (curr_dr_freq <= bus->params.bandwidth) + continue; + + break; + + /* + * TODO: Check all the Slave(s) port(s) audio modes and find + * whether given clock rate is supported with glitchless + * transition. + */ + } + + if (i == clk_values) + return -EINVAL; + + ret = sdw_select_row_col(bus, curr_dr_freq); + if (ret < 0) + return -EINVAL; + + bus->params.curr_dr_freq = curr_dr_freq; + return 0; +} + +/** + * sdw_compute_params: Compute bus, transport and port parameters + * + * @bus: SDW Bus instance + */ +int sdw_compute_params(struct sdw_bus *bus) +{ + int ret; + + /* Computes clock frequency, frame shape and frame frequency */ + ret = sdw_compute_bus_params(bus); + if (ret < 0) { + dev_err(bus->dev, "Compute bus params failed: %d", ret); + return ret; + } + + /* Compute transport and port params */ + ret = sdw_compute_port_params(bus); + if (ret < 0) { + dev_err(bus->dev, "Compute transport params failed: %d", ret); + return ret; + } + + return 0; +} +EXPORT_SYMBOL(sdw_compute_params); + +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SoundWire Generic Bandwidth Allocation"); diff --git a/drivers/soundwire/intel.c b/drivers/soundwire/intel.c index a283670659a9..6a1e862b16c3 100644 --- a/drivers/soundwire/intel.c +++ b/drivers/soundwire/intel.c @@ -22,6 +22,24 @@ #include "bus.h" #include "intel.h" +#define INTEL_MASTER_SUSPEND_DELAY_MS 3000 + +/* + * debug/config flags for the Intel SoundWire Master. + * + * Since we may have multiple masters active, we can have up to 8 + * flags reused in each byte, with master0 using the ls-byte, etc. + */ + +#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME BIT(0) +#define SDW_INTEL_MASTER_DISABLE_CLOCK_STOP BIT(1) +#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE BIT(2) +#define SDW_INTEL_MASTER_DISABLE_MULTI_LINK BIT(3) + +static int md_flags; +module_param_named(sdw_md_flags, md_flags, int, 0444); +MODULE_PARM_DESC(sdw_md_flags, "SoundWire Intel Master device flags (0x0 all off)"); + /* Intel SHIM Registers Definition */ #define SDW_SHIM_LCAP 0x0 #define SDW_SHIM_LCTL 0x4 @@ -45,7 +63,9 @@ #define SDW_SHIM_WAKESTS 0x192 #define SDW_SHIM_LCTL_SPA BIT(0) +#define SDW_SHIM_LCTL_SPA_MASK GENMASK(3, 0) #define SDW_SHIM_LCTL_CPA BIT(8) +#define SDW_SHIM_LCTL_CPA_MASK GENMASK(11, 8) #define SDW_SHIM_SYNC_SYNCPRD_VAL_24 (24000 / SDW_CADENCE_GSYNC_KHZ - 1) #define SDW_SHIM_SYNC_SYNCPRD_VAL_38_4 (38400 / SDW_CADENCE_GSYNC_KHZ - 1) @@ -242,6 +262,42 @@ static int intel_reg_show(struct seq_file *s_file, void *data) } DEFINE_SHOW_ATTRIBUTE(intel_reg); +static int intel_set_m_datamode(void *data, u64 value) +{ + struct sdw_intel *sdw = data; + struct sdw_bus *bus = &sdw->cdns.bus; + + if (value > SDW_PORT_DATA_MODE_STATIC_1) + return -EINVAL; + + /* Userspace changed the hardware state behind the kernel's back */ + add_taint(TAINT_USER, LOCKDEP_STILL_OK); + + bus->params.m_data_mode = value; + + return 0; +} +DEFINE_DEBUGFS_ATTRIBUTE(intel_set_m_datamode_fops, NULL, + intel_set_m_datamode, "%llu\n"); + +static int intel_set_s_datamode(void *data, u64 value) +{ + struct sdw_intel *sdw = data; + struct sdw_bus *bus = &sdw->cdns.bus; + + if (value > SDW_PORT_DATA_MODE_STATIC_1) + return -EINVAL; + + /* Userspace changed the hardware state behind the kernel's back */ + add_taint(TAINT_USER, LOCKDEP_STILL_OK); + + bus->params.s_data_mode = value; + + return 0; +} +DEFINE_DEBUGFS_ATTRIBUTE(intel_set_s_datamode_fops, NULL, + intel_set_s_datamode, "%llu\n"); + static void intel_debugfs_init(struct sdw_intel *sdw) { struct dentry *root = sdw->cdns.bus.debugfs; @@ -254,6 +310,12 @@ static void intel_debugfs_init(struct sdw_intel *sdw) debugfs_create_file("intel-registers", 0400, sdw->debugfs, sdw, &intel_reg_fops); + debugfs_create_file("intel-m-datamode", 0200, sdw->debugfs, sdw, + &intel_set_m_datamode_fops); + + debugfs_create_file("intel-s-datamode", 0200, sdw->debugfs, sdw, + &intel_set_s_datamode_fops); + sdw_cdns_debugfs_init(&sdw->cdns, sdw->debugfs); } @@ -277,8 +339,8 @@ static int intel_link_power_up(struct sdw_intel *sdw) u32 *shim_mask = sdw->link_res->shim_mask; struct sdw_bus *bus = &sdw->cdns.bus; struct sdw_master_prop *prop = &bus->prop; - int spa_mask, cpa_mask; - int link_control; + u32 spa_mask, cpa_mask; + u32 link_control; int ret = 0; u32 syncprd; u32 sync_reg; @@ -301,33 +363,35 @@ static int intel_link_power_up(struct sdw_intel *sdw) syncprd = SDW_SHIM_SYNC_SYNCPRD_VAL_24; if (!*shim_mask) { + dev_dbg(sdw->cdns.dev, "%s: powering up all links\n", __func__); + /* we first need to program the SyncPRD/CPU registers */ dev_dbg(sdw->cdns.dev, "%s: first link up, programming SYNCPRD\n", __func__); /* set SyncPRD period */ sync_reg = intel_readl(shim, SDW_SHIM_SYNC); - sync_reg |= (syncprd << - SDW_REG_SHIFT(SDW_SHIM_SYNC_SYNCPRD)); + u32p_replace_bits(&sync_reg, syncprd, SDW_SHIM_SYNC_SYNCPRD); /* Set SyncCPU bit */ sync_reg |= SDW_SHIM_SYNC_SYNCCPU; intel_writel(shim, SDW_SHIM_SYNC, sync_reg); - } - /* Link power up sequence */ - link_control = intel_readl(shim, SDW_SHIM_LCTL); - spa_mask = (SDW_SHIM_LCTL_SPA << link_id); - cpa_mask = (SDW_SHIM_LCTL_CPA << link_id); - link_control |= spa_mask; + /* Link power up sequence */ + link_control = intel_readl(shim, SDW_SHIM_LCTL); - ret = intel_set_bit(shim, SDW_SHIM_LCTL, link_control, cpa_mask); - if (ret < 0) { - dev_err(sdw->cdns.dev, "Failed to power up link: %d\n", ret); - goto out; - } + /* only power-up enabled links */ + spa_mask = FIELD_PREP(SDW_SHIM_LCTL_SPA_MASK, sdw->link_res->link_mask); + cpa_mask = FIELD_PREP(SDW_SHIM_LCTL_CPA_MASK, sdw->link_res->link_mask); + + link_control |= spa_mask; + + ret = intel_set_bit(shim, SDW_SHIM_LCTL, link_control, cpa_mask); + if (ret < 0) { + dev_err(sdw->cdns.dev, "Failed to power up link: %d\n", ret); + goto out; + } - if (!*shim_mask) { /* SyncCPU will change once link is active */ ret = intel_wait_bit(shim, SDW_SHIM_SYNC, SDW_SHIM_SYNC_SYNCCPU, 0); @@ -426,7 +490,7 @@ static int intel_shim_init(struct sdw_intel *sdw, bool clock_stop) intel_shim_glue_to_master_ip(sdw); - act |= 0x1 << SDW_REG_SHIFT(SDW_SHIM_CTMCTL_DOAIS); + u16p_replace_bits(&act, 0x1, SDW_SHIM_CTMCTL_DOAIS); act |= SDW_SHIM_CTMCTL_DACTQE; act |= SDW_SHIM_CTMCTL_DODS; intel_writew(shim, SDW_SHIM_CTMCTL(link_id), act); @@ -463,9 +527,9 @@ static void intel_shim_wake(struct sdw_intel *sdw, bool wake_enable) mutex_unlock(sdw->link_res->shim_lock); } -static int __maybe_unused intel_link_power_down(struct sdw_intel *sdw) +static int intel_link_power_down(struct sdw_intel *sdw) { - int link_control, spa_mask, cpa_mask; + u32 link_control, spa_mask, cpa_mask; unsigned int link_id = sdw->instance; void __iomem *shim = sdw->link_res->shim; u32 *shim_mask = sdw->link_res->shim_mask; @@ -475,24 +539,37 @@ static int __maybe_unused intel_link_power_down(struct sdw_intel *sdw) intel_shim_master_ip_to_glue(sdw); - /* Link power down sequence */ - link_control = intel_readl(shim, SDW_SHIM_LCTL); - spa_mask = ~(SDW_SHIM_LCTL_SPA << link_id); - cpa_mask = (SDW_SHIM_LCTL_CPA << link_id); - link_control &= spa_mask; - - ret = intel_clear_bit(shim, SDW_SHIM_LCTL, link_control, cpa_mask); - if (!(*shim_mask & BIT(link_id))) dev_err(sdw->cdns.dev, "%s: Unbalanced power-up/down calls\n", __func__); *shim_mask &= ~BIT(link_id); + if (!*shim_mask) { + + dev_dbg(sdw->cdns.dev, "%s: powering down all links\n", __func__); + + /* Link power down sequence */ + link_control = intel_readl(shim, SDW_SHIM_LCTL); + + /* only power-down enabled links */ + spa_mask = FIELD_PREP(SDW_SHIM_LCTL_SPA_MASK, ~sdw->link_res->link_mask); + cpa_mask = FIELD_PREP(SDW_SHIM_LCTL_CPA_MASK, sdw->link_res->link_mask); + + link_control &= spa_mask; + + ret = intel_clear_bit(shim, SDW_SHIM_LCTL, link_control, cpa_mask); + } + + link_control = intel_readl(shim, SDW_SHIM_LCTL); + mutex_unlock(sdw->link_res->shim_lock); - if (ret < 0) + if (ret < 0) { + dev_err(sdw->cdns.dev, "%s: could not power down link\n", __func__); + return ret; + } sdw->cdns.link_up = false; return 0; @@ -538,6 +615,19 @@ static int intel_shim_sync_go_unlocked(struct sdw_intel *sdw) return ret; } +static int intel_shim_sync_go(struct sdw_intel *sdw) +{ + int ret; + + mutex_lock(sdw->link_res->shim_lock); + + ret = intel_shim_sync_go_unlocked(sdw); + + mutex_unlock(sdw->link_res->shim_lock); + + return ret; +} + /* * PDI routines */ @@ -551,12 +641,9 @@ static void intel_pdi_init(struct sdw_intel *sdw, /* PCM Stream Capability */ pcm_cap = intel_readw(shim, SDW_SHIM_PCMSCAP(link_id)); - config->pcm_bd = (pcm_cap & SDW_SHIM_PCMSCAP_BSS) >> - SDW_REG_SHIFT(SDW_SHIM_PCMSCAP_BSS); - config->pcm_in = (pcm_cap & SDW_SHIM_PCMSCAP_ISS) >> - SDW_REG_SHIFT(SDW_SHIM_PCMSCAP_ISS); - config->pcm_out = (pcm_cap & SDW_SHIM_PCMSCAP_OSS) >> - SDW_REG_SHIFT(SDW_SHIM_PCMSCAP_OSS); + config->pcm_bd = FIELD_GET(SDW_SHIM_PCMSCAP_BSS, pcm_cap); + config->pcm_in = FIELD_GET(SDW_SHIM_PCMSCAP_ISS, pcm_cap); + config->pcm_out = FIELD_GET(SDW_SHIM_PCMSCAP_OSS, pcm_cap); dev_dbg(sdw->cdns.dev, "PCM cap bd:%d in:%d out:%d\n", config->pcm_bd, config->pcm_in, config->pcm_out); @@ -564,12 +651,9 @@ static void intel_pdi_init(struct sdw_intel *sdw, /* PDM Stream Capability */ pdm_cap = intel_readw(shim, SDW_SHIM_PDMSCAP(link_id)); - config->pdm_bd = (pdm_cap & SDW_SHIM_PDMSCAP_BSS) >> - SDW_REG_SHIFT(SDW_SHIM_PDMSCAP_BSS); - config->pdm_in = (pdm_cap & SDW_SHIM_PDMSCAP_ISS) >> - SDW_REG_SHIFT(SDW_SHIM_PDMSCAP_ISS); - config->pdm_out = (pdm_cap & SDW_SHIM_PDMSCAP_OSS) >> - SDW_REG_SHIFT(SDW_SHIM_PDMSCAP_OSS); + config->pdm_bd = FIELD_GET(SDW_SHIM_PDMSCAP_BSS, pdm_cap); + config->pdm_in = FIELD_GET(SDW_SHIM_PDMSCAP_ISS, pdm_cap); + config->pdm_out = FIELD_GET(SDW_SHIM_PDMSCAP_OSS, pdm_cap); dev_dbg(sdw->cdns.dev, "PDM cap bd:%d in:%d out:%d\n", config->pdm_bd, config->pdm_in, config->pdm_out); @@ -596,8 +680,7 @@ intel_pdi_get_ch_cap(struct sdw_intel *sdw, unsigned int pdi_num, bool pcm) } else { count = intel_readw(shim, SDW_SHIM_PDMSCAP(link_id)); - count = ((count & SDW_SHIM_PDMSCAP_CPSS) >> - SDW_REG_SHIFT(SDW_SHIM_PDMSCAP_CPSS)); + count = FIELD_GET(SDW_SHIM_PDMSCAP_CPSS, count); } /* zero based values for channel count in register */ @@ -671,10 +754,9 @@ intel_pdi_shim_configure(struct sdw_intel *sdw, struct sdw_cdns_pdi *pdi) else pdi_conf &= ~(SDW_SHIM_PCMSYCM_DIR); - pdi_conf |= (pdi->intel_alh_id << - SDW_REG_SHIFT(SDW_SHIM_PCMSYCM_STREAM)); - pdi_conf |= (pdi->l_ch_num << SDW_REG_SHIFT(SDW_SHIM_PCMSYCM_LCHN)); - pdi_conf |= (pdi->h_ch_num << SDW_REG_SHIFT(SDW_SHIM_PCMSYCM_HCHN)); + u32p_replace_bits(&pdi_conf, pdi->intel_alh_id, SDW_SHIM_PCMSYCM_STREAM); + u32p_replace_bits(&pdi_conf, pdi->l_ch_num, SDW_SHIM_PCMSYCM_LCHN); + u32p_replace_bits(&pdi_conf, pdi->h_ch_num, SDW_SHIM_PCMSYCM_HCHN); intel_writew(shim, SDW_SHIM_PCMSYCHM(link_id, pdi->num), pdi_conf); } @@ -694,11 +776,8 @@ intel_pdi_alh_configure(struct sdw_intel *sdw, struct sdw_cdns_pdi *pdi) /* Program Stream config ALH register */ conf = intel_readl(alh, SDW_ALH_STRMZCFG(pdi->intel_alh_id)); - conf |= (SDW_ALH_STRMZCFG_DMAT_VAL << - SDW_REG_SHIFT(SDW_ALH_STRMZCFG_DMAT)); - - conf |= ((pdi->ch_count - 1) << - SDW_REG_SHIFT(SDW_ALH_STRMZCFG_CHN)); + u32p_replace_bits(&conf, SDW_ALH_STRMZCFG_DMAT_VAL, SDW_ALH_STRMZCFG_DMAT); + u32p_replace_bits(&conf, pdi->ch_count - 1, SDW_ALH_STRMZCFG_CHN); intel_writel(alh, SDW_ALH_STRMZCFG(pdi->intel_alh_id), conf); } @@ -807,10 +886,17 @@ unlock: static int intel_startup(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) { - /* - * TODO: add pm_runtime support here, the startup callback - * will make sure the IP is 'active' - */ + struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai); + int ret; + + ret = pm_runtime_get_sync(cdns->dev); + if (ret < 0 && ret != -EACCES) { + dev_err_ratelimited(cdns->dev, + "pm_runtime_get_sync failed in %s, ret %d\n", + __func__, ret); + pm_runtime_put_noidle(cdns->dev); + return ret; + } return 0; } @@ -856,6 +942,10 @@ static int intel_hw_params(struct snd_pcm_substream *substream, intel_pdi_alh_configure(sdw, pdi); sdw_cdns_config_stream(cdns, ch, dir, pdi); + /* store pdi and hw_params, may be needed in prepare step */ + dma->suspended = false; + dma->pdi = pdi; + dma->hw_params = params; /* Inform DSP about PDI stream number */ ret = intel_params_stream(sdw, substream, dai, params, @@ -899,7 +989,11 @@ error: static int intel_prepare(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) { + struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai); + struct sdw_intel *sdw = cdns_to_intel(cdns); struct sdw_cdns_dma_data *dma; + int ch, dir; + int ret = 0; dma = snd_soc_dai_get_dma_data(dai, substream); if (!dma) { @@ -908,43 +1002,35 @@ static int intel_prepare(struct snd_pcm_substream *substream, return -EIO; } - return sdw_prepare_stream(dma->stream); -} + if (dma->suspended) { + dma->suspended = false; -static int intel_trigger(struct snd_pcm_substream *substream, int cmd, - struct snd_soc_dai *dai) -{ - struct sdw_cdns_dma_data *dma; - int ret; - - dma = snd_soc_dai_get_dma_data(dai, substream); - if (!dma) { - dev_err(dai->dev, "failed to get dma data in %s", __func__); - return -EIO; - } + /* + * .prepare() is called after system resume, where we + * need to reinitialize the SHIM/ALH/Cadence IP. + * .prepare() is also called to deal with underflows, + * but in those cases we cannot touch ALH/SHIM + * registers + */ - switch (cmd) { - case SNDRV_PCM_TRIGGER_START: - case SNDRV_PCM_TRIGGER_PAUSE_RELEASE: - case SNDRV_PCM_TRIGGER_RESUME: - ret = sdw_enable_stream(dma->stream); - break; + /* configure stream */ + ch = params_channels(dma->hw_params); + if (substream->stream == SNDRV_PCM_STREAM_CAPTURE) + dir = SDW_DATA_DIR_RX; + else + dir = SDW_DATA_DIR_TX; - case SNDRV_PCM_TRIGGER_PAUSE_PUSH: - case SNDRV_PCM_TRIGGER_SUSPEND: - case SNDRV_PCM_TRIGGER_STOP: - ret = sdw_disable_stream(dma->stream); - break; + intel_pdi_shim_configure(sdw, dma->pdi); + intel_pdi_alh_configure(sdw, dma->pdi); + sdw_cdns_config_stream(cdns, ch, dir, dma->pdi); - default: - ret = -EINVAL; - break; + /* Inform DSP about PDI stream number */ + ret = intel_params_stream(sdw, substream, dai, + dma->hw_params, + sdw->instance, + dma->pdi->intel_alh_id); } - if (ret) - dev_err(dai->dev, - "%s trigger %d failed: %d", - __func__, cmd, ret); return ret; } @@ -960,12 +1046,12 @@ intel_hw_free(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) if (!dma) return -EIO; - ret = sdw_deprepare_stream(dma->stream); - if (ret) { - dev_err(dai->dev, "sdw_deprepare_stream: failed %d", ret); - return ret; - } - + /* + * The sdw stream state will transition to RELEASED when stream-> + * master_list is empty. So the stream state will transition to + * DEPREPARED for the first cpu-dai and to RELEASED for the last + * cpu-dai. + */ ret = sdw_stream_remove_master(&cdns->bus, dma->stream); if (ret < 0) { dev_err(dai->dev, "remove master from stream %s failed: %d\n", @@ -979,13 +1065,42 @@ intel_hw_free(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) return ret; } + dma->hw_params = NULL; + dma->pdi = NULL; + return 0; } static void intel_shutdown(struct snd_pcm_substream *substream, struct snd_soc_dai *dai) { + struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai); + pm_runtime_mark_last_busy(cdns->dev); + pm_runtime_put_autosuspend(cdns->dev); +} + +static int intel_component_dais_suspend(struct snd_soc_component *component) +{ + struct sdw_cdns_dma_data *dma; + struct snd_soc_dai *dai; + + for_each_component_dais(component, dai) { + /* + * we don't have a .suspend dai_ops, and we don't have access + * to the substream, so let's mark both capture and playback + * DMA contexts as suspended + */ + dma = dai->playback_dma_data; + if (dma) + dma->suspended = true; + + dma = dai->capture_dma_data; + if (dma) + dma->suspended = true; + } + + return 0; } static int intel_pcm_set_sdw_stream(struct snd_soc_dai *dai, @@ -1011,7 +1126,7 @@ static void *intel_get_sdw_stream(struct snd_soc_dai *dai, dma = dai->capture_dma_data; if (!dma) - return NULL; + return ERR_PTR(-EINVAL); return dma->stream; } @@ -1020,7 +1135,6 @@ static const struct snd_soc_dai_ops intel_pcm_dai_ops = { .startup = intel_startup, .hw_params = intel_hw_params, .prepare = intel_prepare, - .trigger = intel_trigger, .hw_free = intel_hw_free, .shutdown = intel_shutdown, .set_sdw_stream = intel_pcm_set_sdw_stream, @@ -1031,7 +1145,6 @@ static const struct snd_soc_dai_ops intel_pdm_dai_ops = { .startup = intel_startup, .hw_params = intel_hw_params, .prepare = intel_prepare, - .trigger = intel_trigger, .hw_free = intel_hw_free, .shutdown = intel_shutdown, .set_sdw_stream = intel_pdm_set_sdw_stream, @@ -1040,6 +1153,7 @@ static const struct snd_soc_dai_ops intel_pdm_dai_ops = { static const struct snd_soc_component_driver dai_component = { .name = "soundwire", + .suspend = intel_component_dais_suspend }; static int intel_create_dai(struct sdw_cdns *cdns, @@ -1207,10 +1321,7 @@ static int intel_init(struct sdw_intel *sdw) intel_shim_init(sdw, clock_stop); - if (clock_stop) - return 0; - - return sdw_cdns_init(&sdw->cdns); + return 0; } /* @@ -1249,6 +1360,9 @@ static int intel_master_probe(struct platform_device *pdev) /* set driver data, accessed by snd_soc_dai_get_drvdata() */ dev_set_drvdata(dev, cdns); + /* use generic bandwidth allocation algorithm */ + sdw->cdns.bus.compute_params = sdw_compute_params; + ret = sdw_bus_master_add(bus, dev, dev->fwnode); if (ret) { dev_err(dev, "sdw_bus_master_add fail: %d\n", ret); @@ -1259,6 +1373,11 @@ static int intel_master_probe(struct platform_device *pdev) dev_info(dev, "SoundWire master %d is disabled, will be ignored\n", bus->link_id); + /* + * Ignore BIOS err_threshold, it's a really bad idea when dealing + * with multiple hardware synchronized links + */ + bus->prop.err_threshold = 0; return 0; } @@ -1270,6 +1389,9 @@ int intel_master_startup(struct platform_device *pdev) struct sdw_cdns *cdns = dev_get_drvdata(dev); struct sdw_intel *sdw = cdns_to_intel(cdns); struct sdw_bus *bus = &cdns->bus; + int link_flags; + bool multi_link; + u32 clock_stop_quirks; int ret; if (bus->prop.hw_disabled) { @@ -1279,7 +1401,23 @@ int intel_master_startup(struct platform_device *pdev) return 0; } - /* Initialize shim, controller and Cadence IP */ + link_flags = md_flags >> (bus->link_id * 8); + multi_link = !(link_flags & SDW_INTEL_MASTER_DISABLE_MULTI_LINK); + if (!multi_link) { + dev_dbg(dev, "Multi-link is disabled\n"); + bus->multi_link = false; + } else { + /* + * hardware-based synchronization is required regardless + * of the number of segments used by a stream: SSP-based + * synchronization is gated by gsync when the multi-master + * mode is set. + */ + bus->multi_link = true; + bus->hw_sync_min_links = 1; + } + + /* Initialize shim, controller */ ret = intel_init(sdw); if (ret) goto err_init; @@ -1298,12 +1436,33 @@ int intel_master_startup(struct platform_device *pdev) goto err_init; } + /* + * follow recommended programming flows to avoid timeouts when + * gsync is enabled + */ + if (multi_link) + intel_shim_sync_arm(sdw); + + ret = sdw_cdns_init(cdns); + if (ret < 0) { + dev_err(dev, "unable to initialize Cadence IP\n"); + goto err_interrupt; + } + ret = sdw_cdns_exit_reset(cdns); if (ret < 0) { dev_err(dev, "unable to exit bus reset sequence\n"); goto err_interrupt; } + if (multi_link) { + ret = intel_shim_sync_go(sdw); + if (ret < 0) { + dev_err(dev, "sync go failed: %d\n", ret); + goto err_interrupt; + } + } + /* Register DAIs */ ret = intel_register_dai(sdw); if (ret) { @@ -1314,6 +1473,47 @@ int intel_master_startup(struct platform_device *pdev) intel_debugfs_init(sdw); + /* Enable runtime PM */ + if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME)) { + pm_runtime_set_autosuspend_delay(dev, + INTEL_MASTER_SUSPEND_DELAY_MS); + pm_runtime_use_autosuspend(dev); + pm_runtime_mark_last_busy(dev); + + pm_runtime_set_active(dev); + pm_runtime_enable(dev); + } + + clock_stop_quirks = sdw->link_res->clock_stop_quirks; + if (clock_stop_quirks & SDW_INTEL_CLK_STOP_NOT_ALLOWED) { + /* + * To keep the clock running we need to prevent + * pm_runtime suspend from happening by increasing the + * reference count. + * This quirk is specified by the parent PCI device in + * case of specific latency requirements. It will have + * no effect if pm_runtime is disabled by the user via + * a module parameter for testing purposes. + */ + pm_runtime_get_noresume(dev); + } + + /* + * The runtime PM status of Slave devices is "Unsupported" + * until they report as ATTACHED. If they don't, e.g. because + * there are no Slave devices populated or if the power-on is + * delayed or dependent on a power switch, the Master will + * remain active and prevent its parent from suspending. + * + * Conditionally force the pm_runtime core to re-evaluate the + * Master status in the absence of any Slave activity. A quirk + * is provided to e.g. deal with Slaves that may be powered on + * with a delay. A more complete solution would require the + * definition of Master properties. + */ + if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE)) + pm_runtime_idle(dev); + return 0; err_interrupt: @@ -1329,6 +1529,11 @@ static int intel_master_remove(struct platform_device *pdev) struct sdw_intel *sdw = cdns_to_intel(cdns); struct sdw_bus *bus = &cdns->bus; + /* + * Since pm_runtime is already disabled, we don't decrease + * the refcount when the clock_stop_quirk is + * SDW_INTEL_CLK_STOP_NOT_ALLOWED + */ if (!bus->prop.hw_disabled) { intel_debugfs_exit(sdw); sdw_cdns_enable_interrupt(cdns, false); @@ -1376,12 +1581,408 @@ int intel_master_process_wakeen_event(struct platform_device *pdev) return 0; } +/* + * PM calls + */ + +#ifdef CONFIG_PM + +static int __maybe_unused intel_suspend(struct device *dev) +{ + struct sdw_cdns *cdns = dev_get_drvdata(dev); + struct sdw_intel *sdw = cdns_to_intel(cdns); + struct sdw_bus *bus = &cdns->bus; + u32 clock_stop_quirks; + int ret; + + if (bus->prop.hw_disabled) { + dev_dbg(dev, "SoundWire master %d is disabled, ignoring\n", + bus->link_id); + return 0; + } + + if (pm_runtime_suspended(dev)) { + dev_dbg(dev, "%s: pm_runtime status: suspended\n", __func__); + + clock_stop_quirks = sdw->link_res->clock_stop_quirks; + + if ((clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET || + !clock_stop_quirks) && + !pm_runtime_suspended(dev->parent)) { + + /* + * if we've enabled clock stop, and the parent + * is still active, disable shim wake. The + * SHIM registers are not accessible if the + * parent is already pm_runtime suspended so + * it's too late to change that configuration + */ + + intel_shim_wake(sdw, false); + } + + return 0; + } + + ret = sdw_cdns_enable_interrupt(cdns, false); + if (ret < 0) { + dev_err(dev, "cannot disable interrupts on suspend\n"); + return ret; + } + + ret = intel_link_power_down(sdw); + if (ret) { + dev_err(dev, "Link power down failed: %d", ret); + return ret; + } + + intel_shim_wake(sdw, false); + + return 0; +} + +static int intel_suspend_runtime(struct device *dev) +{ + struct sdw_cdns *cdns = dev_get_drvdata(dev); + struct sdw_intel *sdw = cdns_to_intel(cdns); + struct sdw_bus *bus = &cdns->bus; + u32 clock_stop_quirks; + int ret; + + if (bus->prop.hw_disabled) { + dev_dbg(dev, "SoundWire master %d is disabled, ignoring\n", + bus->link_id); + return 0; + } + + clock_stop_quirks = sdw->link_res->clock_stop_quirks; + + if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) { + + ret = sdw_cdns_enable_interrupt(cdns, false); + if (ret < 0) { + dev_err(dev, "cannot disable interrupts on suspend\n"); + return ret; + } + + ret = intel_link_power_down(sdw); + if (ret) { + dev_err(dev, "Link power down failed: %d", ret); + return ret; + } + + intel_shim_wake(sdw, false); + + } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET || + !clock_stop_quirks) { + ret = sdw_cdns_clock_stop(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable clock stop on suspend\n"); + return ret; + } + + ret = sdw_cdns_enable_interrupt(cdns, false); + if (ret < 0) { + dev_err(dev, "cannot disable interrupts on suspend\n"); + return ret; + } + + ret = intel_link_power_down(sdw); + if (ret) { + dev_err(dev, "Link power down failed: %d", ret); + return ret; + } + + intel_shim_wake(sdw, true); + } else { + dev_err(dev, "%s clock_stop_quirks %x unsupported\n", + __func__, clock_stop_quirks); + ret = -EINVAL; + } + + return ret; +} + +static int __maybe_unused intel_resume(struct device *dev) +{ + struct sdw_cdns *cdns = dev_get_drvdata(dev); + struct sdw_intel *sdw = cdns_to_intel(cdns); + struct sdw_bus *bus = &cdns->bus; + int link_flags; + bool multi_link; + int ret; + + if (bus->prop.hw_disabled) { + dev_dbg(dev, "SoundWire master %d is disabled, ignoring\n", + bus->link_id); + return 0; + } + + link_flags = md_flags >> (bus->link_id * 8); + multi_link = !(link_flags & SDW_INTEL_MASTER_DISABLE_MULTI_LINK); + + if (pm_runtime_suspended(dev)) { + dev_dbg(dev, "%s: pm_runtime status was suspended, forcing active\n", __func__); + + /* follow required sequence from runtime_pm.rst */ + pm_runtime_disable(dev); + pm_runtime_set_active(dev); + pm_runtime_mark_last_busy(dev); + pm_runtime_enable(dev); + + link_flags = md_flags >> (bus->link_id * 8); + + if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE)) + pm_runtime_idle(dev); + } + + ret = intel_init(sdw); + if (ret) { + dev_err(dev, "%s failed: %d", __func__, ret); + return ret; + } + + /* + * make sure all Slaves are tagged as UNATTACHED and provide + * reason for reinitialization + */ + sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET); + + ret = sdw_cdns_enable_interrupt(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable interrupts during resume\n"); + return ret; + } + + /* + * follow recommended programming flows to avoid timeouts when + * gsync is enabled + */ + if (multi_link) + intel_shim_sync_arm(sdw); + + ret = sdw_cdns_init(&sdw->cdns); + if (ret < 0) { + dev_err(dev, "unable to initialize Cadence IP during resume\n"); + return ret; + } + + ret = sdw_cdns_exit_reset(cdns); + if (ret < 0) { + dev_err(dev, "unable to exit bus reset sequence during resume\n"); + return ret; + } + + if (multi_link) { + ret = intel_shim_sync_go(sdw); + if (ret < 0) { + dev_err(dev, "sync go failed during resume\n"); + return ret; + } + } + + /* + * after system resume, the pm_runtime suspend() may kick in + * during the enumeration, before any children device force the + * master device to remain active. Using pm_runtime_get() + * routines is not really possible, since it'd prevent the + * master from suspending. + * A reasonable compromise is to update the pm_runtime + * counters and delay the pm_runtime suspend by several + * seconds, by when all enumeration should be complete. + */ + pm_runtime_mark_last_busy(dev); + + return ret; +} + +static int intel_resume_runtime(struct device *dev) +{ + struct sdw_cdns *cdns = dev_get_drvdata(dev); + struct sdw_intel *sdw = cdns_to_intel(cdns); + struct sdw_bus *bus = &cdns->bus; + u32 clock_stop_quirks; + bool clock_stop0; + int link_flags; + bool multi_link; + int status; + int ret; + + if (bus->prop.hw_disabled) { + dev_dbg(dev, "SoundWire master %d is disabled, ignoring\n", + bus->link_id); + return 0; + } + + link_flags = md_flags >> (bus->link_id * 8); + multi_link = !(link_flags & SDW_INTEL_MASTER_DISABLE_MULTI_LINK); + + clock_stop_quirks = sdw->link_res->clock_stop_quirks; + + if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) { + ret = intel_init(sdw); + if (ret) { + dev_err(dev, "%s failed: %d", __func__, ret); + return ret; + } + + /* + * make sure all Slaves are tagged as UNATTACHED and provide + * reason for reinitialization + */ + sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET); + + ret = sdw_cdns_enable_interrupt(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable interrupts during resume\n"); + return ret; + } + + /* + * follow recommended programming flows to avoid + * timeouts when gsync is enabled + */ + if (multi_link) + intel_shim_sync_arm(sdw); + + ret = sdw_cdns_init(&sdw->cdns); + if (ret < 0) { + dev_err(dev, "unable to initialize Cadence IP during resume\n"); + return ret; + } + + ret = sdw_cdns_exit_reset(cdns); + if (ret < 0) { + dev_err(dev, "unable to exit bus reset sequence during resume\n"); + return ret; + } + + if (multi_link) { + ret = intel_shim_sync_go(sdw); + if (ret < 0) { + dev_err(dev, "sync go failed during resume\n"); + return ret; + } + } + } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) { + ret = intel_init(sdw); + if (ret) { + dev_err(dev, "%s failed: %d", __func__, ret); + return ret; + } + + /* + * An exception condition occurs for the CLK_STOP_BUS_RESET + * case if one or more masters remain active. In this condition, + * all the masters are powered on for they are in the same power + * domain. Master can preserve its context for clock stop0, so + * there is no need to clear slave status and reset bus. + */ + clock_stop0 = sdw_cdns_is_clock_stop(&sdw->cdns); + + if (!clock_stop0) { + + /* + * make sure all Slaves are tagged as UNATTACHED and + * provide reason for reinitialization + */ + + status = SDW_UNATTACH_REQUEST_MASTER_RESET; + sdw_clear_slave_status(bus, status); + + ret = sdw_cdns_enable_interrupt(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable interrupts during resume\n"); + return ret; + } + + /* + * follow recommended programming flows to avoid + * timeouts when gsync is enabled + */ + if (multi_link) + intel_shim_sync_arm(sdw); + + /* + * Re-initialize the IP since it was powered-off + */ + sdw_cdns_init(&sdw->cdns); + + } else { + ret = sdw_cdns_enable_interrupt(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable interrupts during resume\n"); + return ret; + } + } + + ret = sdw_cdns_clock_restart(cdns, !clock_stop0); + if (ret < 0) { + dev_err(dev, "unable to restart clock during resume\n"); + return ret; + } + + if (!clock_stop0) { + ret = sdw_cdns_exit_reset(cdns); + if (ret < 0) { + dev_err(dev, "unable to exit bus reset sequence during resume\n"); + return ret; + } + + if (multi_link) { + ret = intel_shim_sync_go(sdw); + if (ret < 0) { + dev_err(sdw->cdns.dev, "sync go failed during resume\n"); + return ret; + } + } + } + } else if (!clock_stop_quirks) { + + clock_stop0 = sdw_cdns_is_clock_stop(&sdw->cdns); + if (!clock_stop0) + dev_err(dev, "%s invalid configuration, clock was not stopped", __func__); + + ret = intel_init(sdw); + if (ret) { + dev_err(dev, "%s failed: %d", __func__, ret); + return ret; + } + + ret = sdw_cdns_enable_interrupt(cdns, true); + if (ret < 0) { + dev_err(dev, "cannot enable interrupts during resume\n"); + return ret; + } + + ret = sdw_cdns_clock_restart(cdns, false); + if (ret < 0) { + dev_err(dev, "unable to resume master during resume\n"); + return ret; + } + } else { + dev_err(dev, "%s clock_stop_quirks %x unsupported\n", + __func__, clock_stop_quirks); + ret = -EINVAL; + } + + return ret; +} + +#endif + +static const struct dev_pm_ops intel_pm = { + SET_SYSTEM_SLEEP_PM_OPS(intel_suspend, intel_resume) + SET_RUNTIME_PM_OPS(intel_suspend_runtime, intel_resume_runtime, NULL) +}; + static struct platform_driver sdw_intel_drv = { .probe = intel_master_probe, .remove = intel_master_remove, .driver = { .name = "intel-sdw", - }, + .pm = &intel_pm, + } }; module_platform_driver(sdw_intel_drv); diff --git a/drivers/soundwire/intel.h b/drivers/soundwire/intel.h index 4ea3d262d249..76820d0b9deb 100644 --- a/drivers/soundwire/intel.h +++ b/drivers/soundwire/intel.h @@ -17,6 +17,8 @@ * @dev: device implementing hw_params and free callbacks * @shim_lock: mutex to handle access to shared SHIM registers * @shim_mask: global pointer to check SHIM register initialization + * @clock_stop_quirks: mask defining requested behavior on pm_suspend + * @link_mask: global mask needed for power-up/down sequences * @cdns: Cadence master descriptor * @list: used to walk-through all masters exposed by the same controller */ @@ -31,6 +33,8 @@ struct sdw_intel_link_res { struct device *dev; struct mutex *shim_lock; /* protect shared registers */ u32 *shim_mask; + u32 clock_stop_quirks; + u32 link_mask; struct sdw_cdns *cdns; struct list_head list; }; diff --git a/drivers/soundwire/intel_init.c b/drivers/soundwire/intel_init.c index 047252a91c9e..cabdadb09a1b 100644 --- a/drivers/soundwire/intel_init.c +++ b/drivers/soundwire/intel_init.c @@ -13,6 +13,7 @@ #include <linux/io.h> #include <linux/module.h> #include <linux/platform_device.h> +#include <linux/pm_runtime.h> #include <linux/soundwire/sdw_intel.h> #include "cadence_master.h" #include "intel.h" @@ -68,8 +69,13 @@ static int sdw_intel_cleanup(struct sdw_intel_ctx *ctx) if (!(link_mask & BIT(i))) continue; - if (link->pdev) + if (link->pdev) { + pm_runtime_disable(&link->pdev->dev); platform_device_unregister(link->pdev); + } + + if (!link->clock_stop_quirks) + pm_runtime_put_noidle(link->dev); } return 0; @@ -246,8 +252,10 @@ static struct sdw_intel_ctx link->ops = res->ops; link->dev = res->dev; + link->clock_stop_quirks = res->clock_stop_quirks; link->shim_lock = &ctx->shim_lock; link->shim_mask = &ctx->shim_mask; + link->link_mask = link_mask; memset(&pdevinfo, 0, sizeof(pdevinfo)); @@ -334,6 +342,16 @@ sdw_intel_startup_controller(struct sdw_intel_ctx *ctx) continue; intel_master_startup(link->pdev); + + if (!link->clock_stop_quirks) { + /* + * we need to prevent the parent PCI device + * from entering pm_runtime suspend, so that + * power rails to the SoundWire IP are not + * turned off. + */ + pm_runtime_get_noresume(link->dev); + } } return 0; @@ -365,7 +383,7 @@ static acpi_status sdw_intel_acpi_cb(acpi_handle handle, u32 level, * Name(_ADR, 0x40000000), with bits 31..28 representing the * SoundWire link so filter accordingly */ - if ((adr & GENMASK(31, 28)) >> 28 != SDW_LINK_TYPE) + if (FIELD_GET(GENMASK(31, 28), adr) != SDW_LINK_TYPE) return AE_OK; /* keep going */ /* device found, stop namespace walk */ diff --git a/drivers/soundwire/master.c b/drivers/soundwire/master.c index 5f0b2189defe..3488bb824e84 100644 --- a/drivers/soundwire/master.c +++ b/drivers/soundwire/master.c @@ -154,6 +154,7 @@ int sdw_master_device_add(struct sdw_bus *bus, struct device *parent, bus->dev = &md->dev; bus->md = md; + pm_runtime_enable(&bus->md->dev); device_register_err: return ret; } @@ -166,6 +167,7 @@ device_register_err: */ int sdw_master_device_del(struct sdw_bus *bus) { + pm_runtime_disable(&bus->md->dev); device_unregister(bus->dev); return 0; diff --git a/drivers/soundwire/mipi_disco.c b/drivers/soundwire/mipi_disco.c index 4ae62b452b8c..55a9c51c84c1 100644 --- a/drivers/soundwire/mipi_disco.c +++ b/drivers/soundwire/mipi_disco.c @@ -289,7 +289,7 @@ int sdw_slave_read_prop(struct sdw_slave *slave) struct sdw_slave_prop *prop = &slave->prop; struct device *dev = &slave->dev; struct fwnode_handle *port; - int num_of_ports, nval, i, dp0 = 0; + int nval; device_property_read_u32(dev, "mipi-sdw-sw-interface-revision", &prop->mipi_revision); @@ -352,7 +352,6 @@ int sdw_slave_read_prop(struct sdw_slave *slave) return -ENOMEM; sdw_slave_read_dp0(slave, port, prop->dp0_prop); - dp0 = 1; } /* @@ -383,21 +382,6 @@ int sdw_slave_read_prop(struct sdw_slave *slave) sdw_slave_read_dpn(slave, prop->sink_dpn_prop, nval, prop->sink_ports, "sink"); - /* some ports are bidirectional so check total ports by ORing */ - nval = prop->source_ports | prop->sink_ports; - num_of_ports = hweight32(nval) + dp0; /* add DP0 */ - - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - return 0; } EXPORT_SYMBOL(sdw_slave_read_prop); diff --git a/drivers/soundwire/qcom.c b/drivers/soundwire/qcom.c index 915c2cf0c274..fbca4ebf63e9 100644 --- a/drivers/soundwire/qcom.c +++ b/drivers/soundwire/qcom.c @@ -34,6 +34,7 @@ #define SWRM_INTERRUPT_STATUS_SPECIAL_CMD_ID_FINISHED BIT(10) #define SWRM_INTERRUPT_MASK_ADDR 0x204 #define SWRM_INTERRUPT_CLEAR 0x208 +#define SWRM_INTERRUPT_CPU_EN 0x210 #define SWRM_CMD_FIFO_WR_CMD 0x300 #define SWRM_CMD_FIFO_RD_CMD 0x304 #define SWRM_CMD_FIFO_CMD 0x308 @@ -43,19 +44,17 @@ #define SWRM_CMD_FIFO_RD_FIFO_ADDR 0x318 #define SWRM_ENUMERATOR_CFG_ADDR 0x500 #define SWRM_MCP_FRAME_CTRL_BANK_ADDR(m) (0x101C + 0x40 * (m)) -#define SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_SHFT 3 #define SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_BMSK GENMASK(2, 0) #define SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_BMSK GENMASK(7, 3) -#define SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_SHFT 0 #define SWRM_MCP_CFG_ADDR 0x1048 #define SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_BMSK GENMASK(21, 17) -#define SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_SHFT 0x11 #define SWRM_DEF_CMD_NO_PINGS 0x1f #define SWRM_MCP_STATUS 0x104C #define SWRM_MCP_STATUS_BANK_NUM_MASK BIT(0) #define SWRM_MCP_SLV_STATUS 0x1090 #define SWRM_MCP_SLV_STATUS_MASK GENMASK(1, 0) #define SWRM_DP_PORT_CTRL_BANK(n, m) (0x1124 + 0x100 * (n - 1) + 0x40 * m) +#define SWRM_DP_BLOCK_CTRL3_BANK(n, m) (0x1138 + 0x100 * (n - 1) + 0x40 * m) #define SWRM_DP_PORT_CTRL_EN_CHAN_SHFT 0x18 #define SWRM_DP_PORT_CTRL_OFFSET2_SHFT 0x10 #define SWRM_DP_PORT_CTRL_OFFSET1_SHFT 0x08 @@ -67,11 +66,6 @@ #define SWRM_REG_VAL_PACK(data, dev, id, reg) \ ((reg) | ((id) << 16) | ((dev) << 20) | ((data) << 24)) -#define SWRM_MAX_ROW_VAL 0 /* Rows = 48 */ -#define SWRM_DEFAULT_ROWS 48 -#define SWRM_MIN_COL_VAL 0 /* Cols = 2 */ -#define SWRM_DEFAULT_COL 16 -#define SWRM_MAX_COL_VAL 7 #define SWRM_SPECIAL_CMD_ID 0xF #define MAX_FREQ_NUM 1 #define TIMEOUT_MS (2 * HZ) @@ -84,12 +78,14 @@ struct qcom_swrm_port_config { u8 si; u8 off1; u8 off2; + u8 bp_mode; }; struct qcom_swrm_ctrl { struct sdw_bus bus; struct device *dev; struct regmap *regmap; + void __iomem *mmio; struct completion *comp; struct work_struct slave_work; /* read/write lock */ @@ -103,6 +99,8 @@ struct qcom_swrm_ctrl { unsigned int version; int num_din_ports; int num_dout_ports; + int cols_index; + int rows_index; unsigned long dout_port_mask; unsigned long din_port_mask; struct qcom_swrm_port_config pconfig[QCOM_SDW_MAX_PORTS]; @@ -112,9 +110,24 @@ struct qcom_swrm_ctrl { int (*reg_write)(struct qcom_swrm_ctrl *ctrl, int reg, int val); }; +struct qcom_swrm_data { + u32 default_cols; + u32 default_rows; +}; + +static struct qcom_swrm_data swrm_v1_3_data = { + .default_rows = 48, + .default_cols = 16, +}; + +static struct qcom_swrm_data swrm_v1_5_data = { + .default_rows = 50, + .default_cols = 16, +}; + #define to_qcom_sdw(b) container_of(b, struct qcom_swrm_ctrl, bus) -static int qcom_swrm_abh_reg_read(struct qcom_swrm_ctrl *ctrl, int reg, +static int qcom_swrm_ahb_reg_read(struct qcom_swrm_ctrl *ctrl, int reg, u32 *val) { struct regmap *wcd_regmap = ctrl->regmap; @@ -154,6 +167,20 @@ static int qcom_swrm_ahb_reg_write(struct qcom_swrm_ctrl *ctrl, return SDW_CMD_OK; } +static int qcom_swrm_cpu_reg_read(struct qcom_swrm_ctrl *ctrl, int reg, + u32 *val) +{ + *val = readl(ctrl->mmio + reg); + return SDW_CMD_OK; +} + +static int qcom_swrm_cpu_reg_write(struct qcom_swrm_ctrl *ctrl, int reg, + int val) +{ + writel(val, ctrl->mmio + reg); + return SDW_CMD_OK; +} + static int qcom_swrm_cmd_fifo_wr_cmd(struct qcom_swrm_ctrl *ctrl, u8 cmd_data, u8 dev_addr, u16 reg_addr) { @@ -284,8 +311,8 @@ static int qcom_swrm_init(struct qcom_swrm_ctrl *ctrl) u32 val; /* Clear Rows and Cols */ - val = (SWRM_MAX_ROW_VAL << SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_SHFT | - SWRM_MIN_COL_VAL << SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_SHFT); + val = FIELD_PREP(SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_BMSK, ctrl->rows_index); + val |= FIELD_PREP(SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_BMSK, ctrl->cols_index); ctrl->reg_write(ctrl, SWRM_MCP_FRAME_CTRL_BANK_ADDR(0), val); @@ -298,9 +325,7 @@ static int qcom_swrm_init(struct qcom_swrm_ctrl *ctrl) /* Configure No pings */ ctrl->reg_read(ctrl, SWRM_MCP_CFG_ADDR, &val); - val &= ~SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_BMSK; - val |= (SWRM_DEF_CMD_NO_PINGS << - SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_SHFT); + u32p_replace_bits(&val, SWRM_DEF_CMD_NO_PINGS, SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_BMSK); ctrl->reg_write(ctrl, SWRM_MCP_CFG_ADDR, val); /* Configure number of retries of a read/write cmd */ @@ -310,6 +335,12 @@ static int qcom_swrm_init(struct qcom_swrm_ctrl *ctrl) ctrl->reg_write(ctrl, SWRM_COMP_CFG_ADDR, SWRM_COMP_CFG_IRQ_LEVEL_OR_PULSE_MSK | SWRM_COMP_CFG_ENABLE_MSK); + + /* enable CPU IRQs */ + if (ctrl->mmio) { + ctrl->reg_write(ctrl, SWRM_INTERRUPT_CPU_EN, + SWRM_INTERRUPT_STATUS_RMSK); + } return 0; } @@ -355,11 +386,8 @@ static int qcom_swrm_pre_bank_switch(struct sdw_bus *bus) ctrl->reg_read(ctrl, reg, &val); - val &= ~SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_BMSK; - val &= ~SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_BMSK; - - val |= (SWRM_MAX_ROW_VAL << SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_SHFT | - SWRM_MAX_COL_VAL << SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_SHFT); + u32p_replace_bits(&val, ctrl->cols_index, SWRM_MCP_FRAME_CTRL_BANK_COL_CTRL_BMSK); + u32p_replace_bits(&val, ctrl->rows_index, SWRM_MCP_FRAME_CTRL_BANK_ROW_CTRL_BMSK); return ctrl->reg_write(ctrl, reg, val); } @@ -378,14 +406,22 @@ static int qcom_swrm_transport_params(struct sdw_bus *bus, { struct qcom_swrm_ctrl *ctrl = to_qcom_sdw(bus); u32 value; + int reg = SWRM_DP_PORT_CTRL_BANK((params->port_num), bank); + int ret; value = params->offset1 << SWRM_DP_PORT_CTRL_OFFSET1_SHFT; value |= params->offset2 << SWRM_DP_PORT_CTRL_OFFSET2_SHFT; value |= params->sample_interval - 1; - return ctrl->reg_write(ctrl, - SWRM_DP_PORT_CTRL_BANK((params->port_num), bank), - value); + ret = ctrl->reg_write(ctrl, reg, value); + + if (!ret && params->blk_pkg_mode) { + reg = SWRM_DP_BLOCK_CTRL3_BANK(params->port_num, bank); + + ret = ctrl->reg_write(ctrl, reg, 1); + } + + return ret; } static int qcom_swrm_port_enable(struct sdw_bus *bus, @@ -433,6 +469,7 @@ static int qcom_swrm_compute_params(struct sdw_bus *bus) p_rt->transport_params.sample_interval = pcfg->si + 1; p_rt->transport_params.offset1 = pcfg->off1; p_rt->transport_params.offset2 = pcfg->off2; + p_rt->transport_params.blk_pkg_mode = pcfg->bp_mode; } list_for_each_entry(s_rt, &m_rt->slave_rt_list, m_rt_node) { @@ -443,6 +480,7 @@ static int qcom_swrm_compute_params(struct sdw_bus *bus) pcfg->si + 1; p_rt->transport_params.offset1 = pcfg->off1; p_rt->transport_params.offset2 = pcfg->off2; + p_rt->transport_params.blk_pkg_mode = pcfg->bp_mode; i++; } } @@ -689,12 +727,13 @@ static int qcom_swrm_get_port_config(struct qcom_swrm_ctrl *ctrl) u8 off1[QCOM_SDW_MAX_PORTS]; u8 off2[QCOM_SDW_MAX_PORTS]; u8 si[QCOM_SDW_MAX_PORTS]; + u8 bp_mode[QCOM_SDW_MAX_PORTS] = { 0, }; int i, ret, nports, val; ctrl->reg_read(ctrl, SWRM_COMP_PARAMS, &val); - ctrl->num_dout_ports = val & SWRM_COMP_PARAMS_DOUT_PORTS_MASK; - ctrl->num_din_ports = (val & SWRM_COMP_PARAMS_DIN_PORTS_MASK) >> 5; + ctrl->num_dout_ports = FIELD_GET(SWRM_COMP_PARAMS_DOUT_PORTS_MASK, val); + ctrl->num_din_ports = FIELD_GET(SWRM_COMP_PARAMS_DIN_PORTS_MASK, val); ret = of_property_read_u32(np, "qcom,din-ports", &val); if (ret) @@ -731,10 +770,13 @@ static int qcom_swrm_get_port_config(struct qcom_swrm_ctrl *ctrl) if (ret) return ret; + ret = of_property_read_u8_array(np, "qcom,ports-block-pack-mode", + bp_mode, nports); for (i = 0; i < nports; i++) { ctrl->pconfig[i].si = si[i]; ctrl->pconfig[i].off1 = off1[i]; ctrl->pconfig[i].off2 = off2[i]; + ctrl->pconfig[i].bp_mode = bp_mode[i]; } return 0; @@ -746,6 +788,7 @@ static int qcom_swrm_probe(struct platform_device *pdev) struct sdw_master_prop *prop; struct sdw_bus_params *params; struct qcom_swrm_ctrl *ctrl; + const struct qcom_swrm_data *data; int ret; u32 val; @@ -753,15 +796,25 @@ static int qcom_swrm_probe(struct platform_device *pdev) if (!ctrl) return -ENOMEM; + data = of_device_get_match_data(dev); + ctrl->rows_index = sdw_find_row_index(data->default_rows); + ctrl->cols_index = sdw_find_col_index(data->default_cols); +#if IS_ENABLED(CONFIG_SLIMBUS) if (dev->parent->bus == &slimbus_bus) { - ctrl->reg_read = qcom_swrm_abh_reg_read; +#else + if (false) { +#endif + ctrl->reg_read = qcom_swrm_ahb_reg_read; ctrl->reg_write = qcom_swrm_ahb_reg_write; ctrl->regmap = dev_get_regmap(dev->parent, NULL); if (!ctrl->regmap) return -EINVAL; } else { - /* Only WCD based SoundWire controller is supported */ - return -ENOTSUPP; + ctrl->reg_read = qcom_swrm_cpu_reg_read; + ctrl->reg_write = qcom_swrm_cpu_reg_write; + ctrl->mmio = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(ctrl->mmio)) + return PTR_ERR(ctrl->mmio); } ctrl->irq = of_irq_get(dev->of_node, 0); @@ -795,8 +848,8 @@ static int qcom_swrm_probe(struct platform_device *pdev) params = &ctrl->bus.params; params->max_dr_freq = DEFAULT_CLK_FREQ; params->curr_dr_freq = DEFAULT_CLK_FREQ; - params->col = SWRM_DEFAULT_COL; - params->row = SWRM_DEFAULT_ROWS; + params->col = data->default_cols; + params->row = data->default_rows; ctrl->reg_read(ctrl, SWRM_MCP_STATUS, &val); params->curr_bank = val & SWRM_MCP_STATUS_BANK_NUM_MASK; params->next_bank = !params->curr_bank; @@ -806,8 +859,8 @@ static int qcom_swrm_probe(struct platform_device *pdev) prop->num_clk_gears = 0; prop->num_clk_freq = MAX_FREQ_NUM; prop->clk_freq = &qcom_swrm_freq_tbl[0]; - prop->default_col = SWRM_DEFAULT_COL; - prop->default_row = SWRM_DEFAULT_ROWS; + prop->default_col = data->default_cols; + prop->default_row = data->default_rows; ctrl->reg_read(ctrl, SWRM_COMP_HW_VERSION, &ctrl->version); @@ -858,7 +911,8 @@ static int qcom_swrm_remove(struct platform_device *pdev) } static const struct of_device_id qcom_swrm_of_match[] = { - { .compatible = "qcom,soundwire-v1.3.0", }, + { .compatible = "qcom,soundwire-v1.3.0", .data = &swrm_v1_3_data }, + { .compatible = "qcom,soundwire-v1.5.1", .data = &swrm_v1_5_data }, {/* sentinel */}, }; diff --git a/drivers/soundwire/slave.c b/drivers/soundwire/slave.c index 0839445ee07b..a08f4081c1c4 100644 --- a/drivers/soundwire/slave.c +++ b/drivers/soundwire/slave.c @@ -6,6 +6,7 @@ #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> #include "bus.h" +#include "sysfs_local.h" static void sdw_slave_release(struct device *dev) { @@ -20,11 +21,12 @@ struct device_type sdw_slave_type = { .uevent = sdw_slave_uevent, }; -static int sdw_slave_add(struct sdw_bus *bus, - struct sdw_slave_id *id, struct fwnode_handle *fwnode) +int sdw_slave_add(struct sdw_bus *bus, + struct sdw_slave_id *id, struct fwnode_handle *fwnode) { struct sdw_slave *slave; int ret; + int i; slave = kzalloc(sizeof(*slave), GFP_KERNEL); if (!slave) @@ -50,6 +52,7 @@ static int sdw_slave_add(struct sdw_bus *bus, slave->dev.bus = &sdw_bus_type; slave->dev.of_node = of_node_get(to_of_node(fwnode)); slave->dev.type = &sdw_slave_type; + slave->dev.groups = sdw_slave_status_attr_groups; slave->bus = bus; slave->status = SDW_SLAVE_UNATTACHED; init_completion(&slave->enumeration_complete); @@ -57,6 +60,10 @@ static int sdw_slave_add(struct sdw_bus *bus, slave->dev_num = 0; init_completion(&slave->probe_complete); slave->probed = false; + slave->first_interrupt_done = false; + + for (i = 0; i < SDW_MAX_PORTS; i++) + init_completion(&slave->port_ready[i]); mutex_lock(&bus->bus_lock); list_add_tail(&slave->node, &bus->slaves); @@ -102,7 +109,7 @@ static bool find_slave(struct sdw_bus *bus, } /* Extract link id from ADR, Bit 51 to 48 (included) */ - link_id = (addr >> 48) & GENMASK(3, 0); + link_id = SDW_DISCO_LINK_ID(addr); /* Check for link_id match */ if (link_id != bus->link_id) diff --git a/drivers/soundwire/stream.c b/drivers/soundwire/stream.c index 6e36deb505b1..1099b5d1262b 100644 --- a/drivers/soundwire/stream.c +++ b/drivers/soundwire/stream.c @@ -25,8 +25,10 @@ int sdw_rows[SDW_FRAME_ROWS] = {48, 50, 60, 64, 75, 80, 125, 147, 96, 100, 120, 128, 150, 160, 250, 0, 192, 200, 240, 256, 72, 144, 90, 180}; +EXPORT_SYMBOL(sdw_rows); int sdw_cols[SDW_FRAME_COLS] = {2, 4, 6, 8, 10, 12, 14, 16}; +EXPORT_SYMBOL(sdw_cols); int sdw_find_col_index(int col) { @@ -100,9 +102,7 @@ static int _sdw_program_slave_port_params(struct sdw_bus *bus, return ret; /* Program DPN_SampleCtrl2 register */ - wbuf = (t_params->sample_interval - 1); - wbuf &= SDW_DPN_SAMPLECTRL_HIGH; - wbuf >>= SDW_REG_SHIFT(SDW_DPN_SAMPLECTRL_HIGH); + wbuf = FIELD_GET(SDW_DPN_SAMPLECTRL_HIGH, t_params->sample_interval - 1); ret = sdw_write(slave, addr3, wbuf); if (ret < 0) { @@ -111,9 +111,8 @@ static int _sdw_program_slave_port_params(struct sdw_bus *bus, } /* Program DPN_HCtrl register */ - wbuf = t_params->hstart; - wbuf <<= SDW_REG_SHIFT(SDW_DPN_HCTRL_HSTART); - wbuf |= t_params->hstop; + wbuf = FIELD_PREP(SDW_DPN_HCTRL_HSTART, t_params->hstart); + wbuf |= FIELD_PREP(SDW_DPN_HCTRL_HSTOP, t_params->hstop); ret = sdw_write(slave, addr4, wbuf); if (ret < 0) @@ -157,8 +156,8 @@ static int sdw_program_slave_port_params(struct sdw_bus *bus, } /* Program DPN_PortCtrl register */ - wbuf = p_params->data_mode << SDW_REG_SHIFT(SDW_DPN_PORTCTRL_DATAMODE); - wbuf |= p_params->flow_mode; + wbuf = FIELD_PREP(SDW_DPN_PORTCTRL_DATAMODE, p_params->data_mode); + wbuf |= FIELD_PREP(SDW_DPN_PORTCTRL_FLOWMODE, p_params->flow_mode); ret = sdw_update(s_rt->slave, addr1, 0xF, wbuf); if (ret < 0) { @@ -444,7 +443,8 @@ static int sdw_prep_deprep_slave_ports(struct sdw_bus *bus, prep_ch.bank = bus->params.next_bank; - if (dpn_prop->imp_def_interrupts || !dpn_prop->simple_ch_prep_sm) + if (dpn_prop->imp_def_interrupts || !dpn_prop->simple_ch_prep_sm || + bus->params.s_data_mode != SDW_PORT_DATA_MODE_NORMAL) intr = true; /* @@ -689,9 +689,9 @@ static int sdw_bank_switch(struct sdw_bus *bus, int m_rt_count) /* * Set the multi_link flag only when both the hardware supports - * and there is a stream handled by multiple masters + * and hardware-based sync is required */ - multi_link = bus->multi_link && (m_rt_count > 1); + multi_link = bus->multi_link && (m_rt_count >= bus->hw_sync_min_links); if (multi_link) ret = sdw_transfer_defer(bus, wr_msg, &bus->defer_msg); @@ -761,13 +761,16 @@ static int do_bank_switch(struct sdw_stream_runtime *stream) const struct sdw_master_ops *ops; struct sdw_bus *bus; bool multi_link = false; + int m_rt_count; int ret = 0; + m_rt_count = stream->m_rt_count; + list_for_each_entry(m_rt, &stream->master_list, stream_node) { bus = m_rt->bus; ops = bus->ops; - if (bus->multi_link) { + if (bus->multi_link && m_rt_count >= bus->hw_sync_min_links) { multi_link = true; mutex_lock(&bus->msg_lock); } @@ -788,7 +791,7 @@ static int do_bank_switch(struct sdw_stream_runtime *stream) * synchronized across all Masters and happens later as a * part of post_bank_switch ops. */ - ret = sdw_bank_switch(bus, stream->m_rt_count); + ret = sdw_bank_switch(bus, m_rt_count); if (ret < 0) { dev_err(bus->dev, "Bank switch failed: %d\n", ret); goto error; @@ -814,7 +817,7 @@ static int do_bank_switch(struct sdw_stream_runtime *stream) ret); goto error; } - } else if (bus->multi_link && stream->m_rt_count > 1) { + } else if (multi_link) { dev_err(bus->dev, "Post bank switch ops not implemented\n"); goto error; @@ -832,7 +835,7 @@ static int do_bank_switch(struct sdw_stream_runtime *stream) goto error; } - if (bus->multi_link) + if (multi_link) mutex_unlock(&bus->msg_lock); } @@ -1784,6 +1787,16 @@ static int _sdw_deprepare_stream(struct sdw_stream_runtime *stream) bus->params.bandwidth -= m_rt->stream->params.rate * m_rt->ch_count * m_rt->stream->params.bps; + /* Compute params */ + if (bus->compute_params) { + ret = bus->compute_params(bus); + if (ret < 0) { + dev_err(bus->dev, "Compute params failed: %d", + ret); + return ret; + } + } + /* Program params */ ret = sdw_program_params(bus, false); if (ret < 0) { @@ -1913,7 +1926,7 @@ void sdw_shutdown_stream(void *sdw_substream) sdw_stream = snd_soc_dai_get_sdw_stream(dai, substream->stream); - if (!sdw_stream) { + if (IS_ERR(sdw_stream)) { dev_err(rtd->dev, "no stream found for DAI %s", dai->name); return; } diff --git a/drivers/soundwire/sysfs_local.h b/drivers/soundwire/sysfs_local.h index ff60adee3c41..7268bc24c538 100644 --- a/drivers/soundwire/sysfs_local.h +++ b/drivers/soundwire/sysfs_local.h @@ -8,6 +8,10 @@ * SDW sysfs APIs - */ +/* basic attributes to report status of Slave (attachment, dev_num) */ +extern const struct attribute_group *sdw_slave_status_attr_groups[]; + +/* additional device-managed properties reported after driver probe */ int sdw_slave_sysfs_init(struct sdw_slave *slave); int sdw_slave_sysfs_dpn_init(struct sdw_slave *slave); diff --git a/drivers/soundwire/sysfs_slave.c b/drivers/soundwire/sysfs_slave.c index f510071b0add..b48b6617a396 100644 --- a/drivers/soundwire/sysfs_slave.c +++ b/drivers/soundwire/sysfs_slave.c @@ -16,9 +16,13 @@ /* * The sysfs for Slave reflects the MIPI description as given - * in the MIPI DisCo spec + * in the MIPI DisCo spec. + * status and device_number come directly from the MIPI SoundWire + * 1.x specification. * * Base file is device + * |---- status + * |---- device_number * |---- modalias * |---- dev-properties * |---- mipi_revision @@ -212,3 +216,55 @@ int sdw_slave_sysfs_init(struct sdw_slave *slave) return 0; } + +/* + * the status is shown in capital letters for UNATTACHED and RESERVED + * on purpose, to highligh users to the fact that these status values + * are not expected. + */ +static const char *const slave_status[] = { + [SDW_SLAVE_UNATTACHED] = "UNATTACHED", + [SDW_SLAVE_ATTACHED] = "Attached", + [SDW_SLAVE_ALERT] = "Alert", + [SDW_SLAVE_RESERVED] = "RESERVED", +}; + +static ssize_t status_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct sdw_slave *slave = dev_to_sdw_dev(dev); + + return sprintf(buf, "%s\n", slave_status[slave->status]); +} +static DEVICE_ATTR_RO(status); + +static ssize_t device_number_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct sdw_slave *slave = dev_to_sdw_dev(dev); + + if (slave->status == SDW_SLAVE_UNATTACHED) + return sprintf(buf, "%s", "N/A"); + else + return sprintf(buf, "%d", slave->dev_num); +} +static DEVICE_ATTR_RO(device_number); + +static struct attribute *slave_status_attrs[] = { + &dev_attr_status.attr, + &dev_attr_device_number.attr, + NULL, +}; + +/* + * we don't use ATTRIBUTES_GROUP here since the group is used in a + * separate file and can't be handled as a static. + */ +static const struct attribute_group sdw_slave_status_attr_group = { + .attrs = slave_status_attrs, +}; + +const struct attribute_group *sdw_slave_status_attr_groups[] = { + &sdw_slave_status_attr_group, + NULL +}; diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c index 73efb80815db..6dca744e39e9 100644 --- a/drivers/uio/uio.c +++ b/drivers/uio/uio.c @@ -1048,8 +1048,6 @@ void uio_unregister_device(struct uio_info *info) idev = info->uio_dev; - uio_free_minor(idev); - mutex_lock(&idev->info_lock); uio_dev_del_attributes(idev); @@ -1064,6 +1062,8 @@ void uio_unregister_device(struct uio_info *info) device_unregister(&idev->dev); + uio_free_minor(idev); + return; } EXPORT_SYMBOL_GPL(uio_unregister_device); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index cbc1f25c79ab..80c5f9c16ec1 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -32,4 +32,6 @@ config FSL_HV_MANAGER partition shuts down. source "drivers/virt/vboxguest/Kconfig" + +source "drivers/virt/nitro_enclaves/Kconfig" endif diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index fd331247c27a..f28425ce4b39 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -5,3 +5,5 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o obj-y += vboxguest/ + +obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c index 1b0b11b55d2a..46ee0a0998b6 100644 --- a/drivers/virt/fsl_hypervisor.c +++ b/drivers/virt/fsl_hypervisor.c @@ -157,7 +157,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) unsigned int i; long ret = 0; - int num_pinned; /* return value from get_user_pages() */ + int num_pinned = 0; /* return value from get_user_pages_fast() */ phys_addr_t remote_paddr; /* The next address in the remote buffer */ uint32_t count; /* The number of bytes left to copy */ @@ -174,7 +174,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) return -EINVAL; /* - * The array of pages returned by get_user_pages() covers only + * The array of pages returned by get_user_pages_fast() covers only * page-aligned memory. Since the user buffer is probably not * page-aligned, we need to handle the discrepancy. * @@ -224,7 +224,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) /* * 'pages' is an array of struct page pointers that's initialized by - * get_user_pages(). + * get_user_pages_fast(). */ pages = kcalloc(num_pages, sizeof(struct page *), GFP_KERNEL); if (!pages) { @@ -241,7 +241,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) if (!sg_list_unaligned) { pr_debug("fsl-hv: could not allocate S/G list\n"); ret = -ENOMEM; - goto exit; + goto free_pages; } sg_list = PTR_ALIGN(sg_list_unaligned, sizeof(struct fh_sg_list)); @@ -250,7 +250,6 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) num_pages, param.source != -1 ? FOLL_WRITE : 0, pages); if (num_pinned != num_pages) { - /* get_user_pages() failed */ pr_debug("fsl-hv: could not lock source buffer\n"); ret = (num_pinned < 0) ? num_pinned : -EFAULT; goto exit; @@ -292,13 +291,13 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) virt_to_phys(sg_list), num_pages); exit: - if (pages) { - for (i = 0; i < num_pages; i++) - if (pages[i]) - put_page(pages[i]); + if (pages && (num_pinned > 0)) { + for (i = 0; i < num_pinned; i++) + put_page(pages[i]); } kfree(sg_list_unaligned); +free_pages: kfree(pages); if (!ret) diff --git a/drivers/virt/nitro_enclaves/Kconfig b/drivers/virt/nitro_enclaves/Kconfig new file mode 100644 index 000000000000..8c9387a232df --- /dev/null +++ b/drivers/virt/nitro_enclaves/Kconfig @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +# Amazon Nitro Enclaves (NE) support. +# Nitro is a hypervisor that has been developed by Amazon. + +# TODO: Add dependency for ARM64 once NE is supported on Arm platforms. For now, +# the NE kernel driver can be built for aarch64 arch. +# depends on (ARM64 || X86) && HOTPLUG_CPU && PCI && SMP + +config NITRO_ENCLAVES + tristate "Nitro Enclaves Support" + depends on X86 && HOTPLUG_CPU && PCI && SMP + help + This driver consists of support for enclave lifetime management + for Nitro Enclaves (NE). + + To compile this driver as a module, choose M here. + The module will be called nitro_enclaves. diff --git a/drivers/virt/nitro_enclaves/Makefile b/drivers/virt/nitro_enclaves/Makefile new file mode 100644 index 000000000000..da61260f2be6 --- /dev/null +++ b/drivers/virt/nitro_enclaves/Makefile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +# Enclave lifetime management support for Nitro Enclaves (NE). + +obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves.o + +nitro_enclaves-y := ne_pci_dev.o ne_misc_dev.o diff --git a/drivers/virt/nitro_enclaves/ne_misc_dev.c b/drivers/virt/nitro_enclaves/ne_misc_dev.c new file mode 100644 index 000000000000..f06622b48d69 --- /dev/null +++ b/drivers/virt/nitro_enclaves/ne_misc_dev.c @@ -0,0 +1,1733 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +/** + * DOC: Enclave lifetime management driver for Nitro Enclaves (NE). + * Nitro is a hypervisor that has been developed by Amazon. + */ + +#include <linux/anon_inodes.h> +#include <linux/capability.h> +#include <linux/cpu.h> +#include <linux/device.h> +#include <linux/file.h> +#include <linux/hugetlb.h> +#include <linux/limits.h> +#include <linux/list.h> +#include <linux/miscdevice.h> +#include <linux/mm.h> +#include <linux/mman.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/nitro_enclaves.h> +#include <linux/pci.h> +#include <linux/poll.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <uapi/linux/vm_sockets.h> + +#include "ne_misc_dev.h" +#include "ne_pci_dev.h" + +/** + * NE_CPUS_SIZE - Size for max 128 CPUs, for now, in a cpu-list string, comma + * separated. The NE CPU pool includes CPUs from a single NUMA + * node. + */ +#define NE_CPUS_SIZE (512) + +/** + * NE_EIF_LOAD_OFFSET - The offset where to copy the Enclave Image Format (EIF) + * image in enclave memory. + */ +#define NE_EIF_LOAD_OFFSET (8 * 1024UL * 1024UL) + +/** + * NE_MIN_ENCLAVE_MEM_SIZE - The minimum memory size an enclave can be launched + * with. + */ +#define NE_MIN_ENCLAVE_MEM_SIZE (64 * 1024UL * 1024UL) + +/** + * NE_MIN_MEM_REGION_SIZE - The minimum size of an enclave memory region. + */ +#define NE_MIN_MEM_REGION_SIZE (2 * 1024UL * 1024UL) + +/** + * NE_PARENT_VM_CID - The CID for the vsock device of the primary / parent VM. + */ +#define NE_PARENT_VM_CID (3) + +static long ne_ioctl(struct file *file, unsigned int cmd, unsigned long arg); + +static const struct file_operations ne_fops = { + .owner = THIS_MODULE, + .llseek = noop_llseek, + .unlocked_ioctl = ne_ioctl, +}; + +static struct miscdevice ne_misc_dev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "nitro_enclaves", + .fops = &ne_fops, + .mode = 0660, +}; + +struct ne_devs ne_devs = { + .ne_misc_dev = &ne_misc_dev, +}; + +/* + * TODO: Update logic to create new sysfs entries instead of using + * a kernel parameter e.g. if multiple sysfs files needed. + */ +static int ne_set_kernel_param(const char *val, const struct kernel_param *kp); + +static const struct kernel_param_ops ne_cpu_pool_ops = { + .get = param_get_string, + .set = ne_set_kernel_param, +}; + +static char ne_cpus[NE_CPUS_SIZE]; +static struct kparam_string ne_cpus_arg = { + .maxlen = sizeof(ne_cpus), + .string = ne_cpus, +}; + +module_param_cb(ne_cpus, &ne_cpu_pool_ops, &ne_cpus_arg, 0644); +/* https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html#cpu-lists */ +MODULE_PARM_DESC(ne_cpus, "<cpu-list> - CPU pool used for Nitro Enclaves"); + +/** + * struct ne_cpu_pool - CPU pool used for Nitro Enclaves. + * @avail_threads_per_core: Available full CPU cores to be dedicated to + * enclave(s). The cpumasks from the array, indexed + * by core id, contain all the threads from the + * available cores, that are not set for created + * enclave(s). The full CPU cores are part of the + * NE CPU pool. + * @mutex: Mutex for the access to the NE CPU pool. + * @nr_parent_vm_cores : The size of the available threads per core array. + * The total number of CPU cores available on the + * primary / parent VM. + * @nr_threads_per_core: The number of threads that a full CPU core has. + * @numa_node: NUMA node of the CPUs in the pool. + */ +struct ne_cpu_pool { + cpumask_var_t *avail_threads_per_core; + struct mutex mutex; + unsigned int nr_parent_vm_cores; + unsigned int nr_threads_per_core; + int numa_node; +}; + +static struct ne_cpu_pool ne_cpu_pool; + +/** + * ne_check_enclaves_created() - Verify if at least one enclave has been created. + * @void: No parameters provided. + * + * Context: Process context. + * Return: + * * True if at least one enclave is created. + * * False otherwise. + */ +static bool ne_check_enclaves_created(void) +{ + struct ne_pci_dev *ne_pci_dev = ne_devs.ne_pci_dev; + bool ret = false; + + if (!ne_pci_dev) + return ret; + + mutex_lock(&ne_pci_dev->enclaves_list_mutex); + + if (!list_empty(&ne_pci_dev->enclaves_list)) + ret = true; + + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); + + return ret; +} + +/** + * ne_setup_cpu_pool() - Set the NE CPU pool after handling sanity checks such + * as not sharing CPU cores with the primary / parent VM + * or not using CPU 0, which should remain available for + * the primary / parent VM. Offline the CPUs from the + * pool after the checks passed. + * @ne_cpu_list: The CPU list used for setting NE CPU pool. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_setup_cpu_pool(const char *ne_cpu_list) +{ + int core_id = -1; + unsigned int cpu = 0; + cpumask_var_t cpu_pool; + unsigned int cpu_sibling = 0; + unsigned int i = 0; + int numa_node = -1; + int rc = -EINVAL; + + if (!zalloc_cpumask_var(&cpu_pool, GFP_KERNEL)) + return -ENOMEM; + + mutex_lock(&ne_cpu_pool.mutex); + + rc = cpulist_parse(ne_cpu_list, cpu_pool); + if (rc < 0) { + pr_err("%s: Error in cpulist parse [rc=%d]\n", ne_misc_dev.name, rc); + + goto free_pool_cpumask; + } + + cpu = cpumask_any(cpu_pool); + if (cpu >= nr_cpu_ids) { + pr_err("%s: No CPUs available in CPU pool\n", ne_misc_dev.name); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + + /* + * Check if the CPUs are online, to further get info about them + * e.g. numa node, core id, siblings. + */ + for_each_cpu(cpu, cpu_pool) + if (cpu_is_offline(cpu)) { + pr_err("%s: CPU %d is offline, has to be online to get its metadata\n", + ne_misc_dev.name, cpu); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + + /* + * Check if the CPUs from the NE CPU pool are from the same NUMA node. + */ + for_each_cpu(cpu, cpu_pool) + if (numa_node < 0) { + numa_node = cpu_to_node(cpu); + if (numa_node < 0) { + pr_err("%s: Invalid NUMA node %d\n", + ne_misc_dev.name, numa_node); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + } else { + if (numa_node != cpu_to_node(cpu)) { + pr_err("%s: CPUs with different NUMA nodes\n", + ne_misc_dev.name); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + } + + /* + * Check if CPU 0 and its siblings are included in the provided CPU pool + * They should remain available for the primary / parent VM. + */ + if (cpumask_test_cpu(0, cpu_pool)) { + pr_err("%s: CPU 0 has to remain available\n", ne_misc_dev.name); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + + for_each_cpu(cpu_sibling, topology_sibling_cpumask(0)) { + if (cpumask_test_cpu(cpu_sibling, cpu_pool)) { + pr_err("%s: CPU sibling %d for CPU 0 is in CPU pool\n", + ne_misc_dev.name, cpu_sibling); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + } + + /* + * Check if CPU siblings are included in the provided CPU pool. The + * expectation is that full CPU cores are made available in the CPU pool + * for enclaves. + */ + for_each_cpu(cpu, cpu_pool) { + for_each_cpu(cpu_sibling, topology_sibling_cpumask(cpu)) { + if (!cpumask_test_cpu(cpu_sibling, cpu_pool)) { + pr_err("%s: CPU %d is not in CPU pool\n", + ne_misc_dev.name, cpu_sibling); + + rc = -EINVAL; + + goto free_pool_cpumask; + } + } + } + + /* Calculate the number of threads from a full CPU core. */ + cpu = cpumask_any(cpu_pool); + for_each_cpu(cpu_sibling, topology_sibling_cpumask(cpu)) + ne_cpu_pool.nr_threads_per_core++; + + ne_cpu_pool.nr_parent_vm_cores = nr_cpu_ids / ne_cpu_pool.nr_threads_per_core; + + ne_cpu_pool.avail_threads_per_core = kcalloc(ne_cpu_pool.nr_parent_vm_cores, + sizeof(*ne_cpu_pool.avail_threads_per_core), + GFP_KERNEL); + if (!ne_cpu_pool.avail_threads_per_core) { + rc = -ENOMEM; + + goto free_pool_cpumask; + } + + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + if (!zalloc_cpumask_var(&ne_cpu_pool.avail_threads_per_core[i], GFP_KERNEL)) { + rc = -ENOMEM; + + goto free_cores_cpumask; + } + + /* + * Split the NE CPU pool in threads per core to keep the CPU topology + * after offlining the CPUs. + */ + for_each_cpu(cpu, cpu_pool) { + core_id = topology_core_id(cpu); + if (core_id < 0 || core_id >= ne_cpu_pool.nr_parent_vm_cores) { + pr_err("%s: Invalid core id %d for CPU %d\n", + ne_misc_dev.name, core_id, cpu); + + rc = -EINVAL; + + goto clear_cpumask; + } + + cpumask_set_cpu(cpu, ne_cpu_pool.avail_threads_per_core[core_id]); + } + + /* + * CPUs that are given to enclave(s) should not be considered online + * by Linux anymore, as the hypervisor will degrade them to floating. + * The physical CPUs (full cores) are carved out of the primary / parent + * VM and given to the enclave VM. The same number of vCPUs would run + * on less pCPUs for the primary / parent VM. + * + * We offline them here, to not degrade performance and expose correct + * topology to Linux and user space. + */ + for_each_cpu(cpu, cpu_pool) { + rc = remove_cpu(cpu); + if (rc != 0) { + pr_err("%s: CPU %d is not offlined [rc=%d]\n", + ne_misc_dev.name, cpu, rc); + + goto online_cpus; + } + } + + free_cpumask_var(cpu_pool); + + ne_cpu_pool.numa_node = numa_node; + + mutex_unlock(&ne_cpu_pool.mutex); + + return 0; + +online_cpus: + for_each_cpu(cpu, cpu_pool) + add_cpu(cpu); +clear_cpumask: + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + cpumask_clear(ne_cpu_pool.avail_threads_per_core[i]); +free_cores_cpumask: + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + free_cpumask_var(ne_cpu_pool.avail_threads_per_core[i]); + kfree(ne_cpu_pool.avail_threads_per_core); +free_pool_cpumask: + free_cpumask_var(cpu_pool); + ne_cpu_pool.nr_parent_vm_cores = 0; + ne_cpu_pool.nr_threads_per_core = 0; + ne_cpu_pool.numa_node = -1; + mutex_unlock(&ne_cpu_pool.mutex); + + return rc; +} + +/** + * ne_teardown_cpu_pool() - Online the CPUs from the NE CPU pool and cleanup the + * CPU pool. + * @void: No parameters provided. + * + * Context: Process context. + */ +static void ne_teardown_cpu_pool(void) +{ + unsigned int cpu = 0; + unsigned int i = 0; + int rc = -EINVAL; + + mutex_lock(&ne_cpu_pool.mutex); + + if (!ne_cpu_pool.nr_parent_vm_cores) { + mutex_unlock(&ne_cpu_pool.mutex); + + return; + } + + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) { + for_each_cpu(cpu, ne_cpu_pool.avail_threads_per_core[i]) { + rc = add_cpu(cpu); + if (rc != 0) + pr_err("%s: CPU %d is not onlined [rc=%d]\n", + ne_misc_dev.name, cpu, rc); + } + + cpumask_clear(ne_cpu_pool.avail_threads_per_core[i]); + + free_cpumask_var(ne_cpu_pool.avail_threads_per_core[i]); + } + + kfree(ne_cpu_pool.avail_threads_per_core); + ne_cpu_pool.nr_parent_vm_cores = 0; + ne_cpu_pool.nr_threads_per_core = 0; + ne_cpu_pool.numa_node = -1; + + mutex_unlock(&ne_cpu_pool.mutex); +} + +/** + * ne_set_kernel_param() - Set the NE CPU pool value via the NE kernel parameter. + * @val: NE CPU pool string value. + * @kp : NE kernel parameter associated with the NE CPU pool. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_set_kernel_param(const char *val, const struct kernel_param *kp) +{ + char error_val[] = ""; + int rc = -EINVAL; + + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + + if (ne_check_enclaves_created()) { + pr_err("%s: The CPU pool is used by enclave(s)\n", ne_misc_dev.name); + + return -EPERM; + } + + ne_teardown_cpu_pool(); + + rc = ne_setup_cpu_pool(val); + if (rc < 0) { + pr_err("%s: Error in setup CPU pool [rc=%d]\n", ne_misc_dev.name, rc); + + param_set_copystring(error_val, kp); + + return rc; + } + + rc = param_set_copystring(val, kp); + if (rc < 0) { + pr_err("%s: Error in param set copystring [rc=%d]\n", ne_misc_dev.name, rc); + + ne_teardown_cpu_pool(); + + param_set_copystring(error_val, kp); + + return rc; + } + + return 0; +} + +/** + * ne_donated_cpu() - Check if the provided CPU is already used by the enclave. + * @ne_enclave : Private data associated with the current enclave. + * @cpu: CPU to check if already used. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * True if the provided CPU is already used by the enclave. + * * False otherwise. + */ +static bool ne_donated_cpu(struct ne_enclave *ne_enclave, unsigned int cpu) +{ + if (cpumask_test_cpu(cpu, ne_enclave->vcpu_ids)) + return true; + + return false; +} + +/** + * ne_get_unused_core_from_cpu_pool() - Get the id of a full core from the + * NE CPU pool. + * @void: No parameters provided. + * + * Context: Process context. This function is called with the ne_enclave and + * ne_cpu_pool mutexes held. + * Return: + * * Core id. + * * -1 if no CPU core available in the pool. + */ +static int ne_get_unused_core_from_cpu_pool(void) +{ + int core_id = -1; + unsigned int i = 0; + + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + if (!cpumask_empty(ne_cpu_pool.avail_threads_per_core[i])) { + core_id = i; + + break; + } + + return core_id; +} + +/** + * ne_set_enclave_threads_per_core() - Set the threads of the provided core in + * the enclave data structure. + * @ne_enclave : Private data associated with the current enclave. + * @core_id: Core id to get its threads from the NE CPU pool. + * @vcpu_id: vCPU id part of the provided core. + * + * Context: Process context. This function is called with the ne_enclave and + * ne_cpu_pool mutexes held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_set_enclave_threads_per_core(struct ne_enclave *ne_enclave, + int core_id, u32 vcpu_id) +{ + unsigned int cpu = 0; + + if (core_id < 0 && vcpu_id == 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "No CPUs available in NE CPU pool\n"); + + return -NE_ERR_NO_CPUS_AVAIL_IN_POOL; + } + + if (core_id < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "CPU %d is not in NE CPU pool\n", vcpu_id); + + return -NE_ERR_VCPU_NOT_IN_CPU_POOL; + } + + if (core_id >= ne_enclave->nr_parent_vm_cores) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Invalid core id %d - ne_enclave\n", core_id); + + return -NE_ERR_VCPU_INVALID_CPU_CORE; + } + + for_each_cpu(cpu, ne_cpu_pool.avail_threads_per_core[core_id]) + cpumask_set_cpu(cpu, ne_enclave->threads_per_core[core_id]); + + cpumask_clear(ne_cpu_pool.avail_threads_per_core[core_id]); + + return 0; +} + +/** + * ne_get_cpu_from_cpu_pool() - Get a CPU from the NE CPU pool, either from the + * remaining sibling(s) of a CPU core or the first + * sibling of a new CPU core. + * @ne_enclave : Private data associated with the current enclave. + * @vcpu_id: vCPU to get from the NE CPU pool. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_get_cpu_from_cpu_pool(struct ne_enclave *ne_enclave, u32 *vcpu_id) +{ + int core_id = -1; + unsigned int cpu = 0; + unsigned int i = 0; + int rc = -EINVAL; + + /* + * If previously allocated a thread of a core to this enclave, first + * check remaining sibling(s) for new CPU allocations, so that full + * CPU cores are used for the enclave. + */ + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) + for_each_cpu(cpu, ne_enclave->threads_per_core[i]) + if (!ne_donated_cpu(ne_enclave, cpu)) { + *vcpu_id = cpu; + + return 0; + } + + mutex_lock(&ne_cpu_pool.mutex); + + /* + * If no remaining siblings, get a core from the NE CPU pool and keep + * track of all the threads in the enclave threads per core data structure. + */ + core_id = ne_get_unused_core_from_cpu_pool(); + + rc = ne_set_enclave_threads_per_core(ne_enclave, core_id, *vcpu_id); + if (rc < 0) + goto unlock_mutex; + + *vcpu_id = cpumask_any(ne_enclave->threads_per_core[core_id]); + + rc = 0; + +unlock_mutex: + mutex_unlock(&ne_cpu_pool.mutex); + + return rc; +} + +/** + * ne_get_vcpu_core_from_cpu_pool() - Get from the NE CPU pool the id of the + * core associated with the provided vCPU. + * @vcpu_id: Provided vCPU id to get its associated core id. + * + * Context: Process context. This function is called with the ne_enclave and + * ne_cpu_pool mutexes held. + * Return: + * * Core id. + * * -1 if the provided vCPU is not in the pool. + */ +static int ne_get_vcpu_core_from_cpu_pool(u32 vcpu_id) +{ + int core_id = -1; + unsigned int i = 0; + + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + if (cpumask_test_cpu(vcpu_id, ne_cpu_pool.avail_threads_per_core[i])) { + core_id = i; + + break; + } + + return core_id; +} + +/** + * ne_check_cpu_in_cpu_pool() - Check if the given vCPU is in the available CPUs + * from the pool. + * @ne_enclave : Private data associated with the current enclave. + * @vcpu_id: ID of the vCPU to check if available in the NE CPU pool. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_check_cpu_in_cpu_pool(struct ne_enclave *ne_enclave, u32 vcpu_id) +{ + int core_id = -1; + unsigned int i = 0; + int rc = -EINVAL; + + if (ne_donated_cpu(ne_enclave, vcpu_id)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "CPU %d already used\n", vcpu_id); + + return -NE_ERR_VCPU_ALREADY_USED; + } + + /* + * If previously allocated a thread of a core to this enclave, but not + * the full core, first check remaining sibling(s). + */ + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) + if (cpumask_test_cpu(vcpu_id, ne_enclave->threads_per_core[i])) + return 0; + + mutex_lock(&ne_cpu_pool.mutex); + + /* + * If no remaining siblings, get from the NE CPU pool the core + * associated with the vCPU and keep track of all the threads in the + * enclave threads per core data structure. + */ + core_id = ne_get_vcpu_core_from_cpu_pool(vcpu_id); + + rc = ne_set_enclave_threads_per_core(ne_enclave, core_id, vcpu_id); + if (rc < 0) + goto unlock_mutex; + + rc = 0; + +unlock_mutex: + mutex_unlock(&ne_cpu_pool.mutex); + + return rc; +} + +/** + * ne_add_vcpu_ioctl() - Add a vCPU to the slot associated with the current + * enclave. + * @ne_enclave : Private data associated with the current enclave. + * @vcpu_id: ID of the CPU to be associated with the given slot, + * apic id on x86. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_add_vcpu_ioctl(struct ne_enclave *ne_enclave, u32 vcpu_id) +{ + struct ne_pci_dev_cmd_reply cmd_reply = {}; + struct pci_dev *pdev = ne_devs.ne_pci_dev->pdev; + int rc = -EINVAL; + struct slot_add_vcpu_req slot_add_vcpu_req = {}; + + if (ne_enclave->mm != current->mm) + return -EIO; + + slot_add_vcpu_req.slot_uid = ne_enclave->slot_uid; + slot_add_vcpu_req.vcpu_id = vcpu_id; + + rc = ne_do_request(pdev, SLOT_ADD_VCPU, + &slot_add_vcpu_req, sizeof(slot_add_vcpu_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in slot add vCPU [rc=%d]\n", rc); + + return rc; + } + + cpumask_set_cpu(vcpu_id, ne_enclave->vcpu_ids); + + ne_enclave->nr_vcpus++; + + return 0; +} + +/** + * ne_sanity_check_user_mem_region() - Sanity check the user space memory + * region received during the set user + * memory region ioctl call. + * @ne_enclave : Private data associated with the current enclave. + * @mem_region : User space memory region to be sanity checked. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_sanity_check_user_mem_region(struct ne_enclave *ne_enclave, + struct ne_user_memory_region mem_region) +{ + struct ne_mem_region *ne_mem_region = NULL; + + if (ne_enclave->mm != current->mm) + return -EIO; + + if (mem_region.memory_size & (NE_MIN_MEM_REGION_SIZE - 1)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "User space memory size is not multiple of 2 MiB\n"); + + return -NE_ERR_INVALID_MEM_REGION_SIZE; + } + + if (!IS_ALIGNED(mem_region.userspace_addr, NE_MIN_MEM_REGION_SIZE)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "User space address is not 2 MiB aligned\n"); + + return -NE_ERR_UNALIGNED_MEM_REGION_ADDR; + } + + if ((mem_region.userspace_addr & (NE_MIN_MEM_REGION_SIZE - 1)) || + !access_ok((void __user *)(unsigned long)mem_region.userspace_addr, + mem_region.memory_size)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Invalid user space address range\n"); + + return -NE_ERR_INVALID_MEM_REGION_ADDR; + } + + list_for_each_entry(ne_mem_region, &ne_enclave->mem_regions_list, + mem_region_list_entry) { + u64 memory_size = ne_mem_region->memory_size; + u64 userspace_addr = ne_mem_region->userspace_addr; + + if ((userspace_addr <= mem_region.userspace_addr && + mem_region.userspace_addr < (userspace_addr + memory_size)) || + (mem_region.userspace_addr <= userspace_addr && + (mem_region.userspace_addr + mem_region.memory_size) > userspace_addr)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "User space memory region already used\n"); + + return -NE_ERR_MEM_REGION_ALREADY_USED; + } + } + + return 0; +} + +/** + * ne_sanity_check_user_mem_region_page() - Sanity check a page from the user space + * memory region received during the set + * user memory region ioctl call. + * @ne_enclave : Private data associated with the current enclave. + * @mem_region_page: Page from the user space memory region to be sanity checked. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_sanity_check_user_mem_region_page(struct ne_enclave *ne_enclave, + struct page *mem_region_page) +{ + if (!PageHuge(mem_region_page)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Not a hugetlbfs page\n"); + + return -NE_ERR_MEM_NOT_HUGE_PAGE; + } + + if (page_size(mem_region_page) & (NE_MIN_MEM_REGION_SIZE - 1)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Page size not multiple of 2 MiB\n"); + + return -NE_ERR_INVALID_PAGE_SIZE; + } + + if (ne_enclave->numa_node != page_to_nid(mem_region_page)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Page is not from NUMA node %d\n", + ne_enclave->numa_node); + + return -NE_ERR_MEM_DIFFERENT_NUMA_NODE; + } + + return 0; +} + +/** + * ne_set_user_memory_region_ioctl() - Add user space memory region to the slot + * associated with the current enclave. + * @ne_enclave : Private data associated with the current enclave. + * @mem_region : User space memory region to be associated with the given slot. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_set_user_memory_region_ioctl(struct ne_enclave *ne_enclave, + struct ne_user_memory_region mem_region) +{ + long gup_rc = 0; + unsigned long i = 0; + unsigned long max_nr_pages = 0; + unsigned long memory_size = 0; + struct ne_mem_region *ne_mem_region = NULL; + unsigned long nr_phys_contig_mem_regions = 0; + struct pci_dev *pdev = ne_devs.ne_pci_dev->pdev; + struct page **phys_contig_mem_regions = NULL; + int rc = -EINVAL; + + rc = ne_sanity_check_user_mem_region(ne_enclave, mem_region); + if (rc < 0) + return rc; + + ne_mem_region = kzalloc(sizeof(*ne_mem_region), GFP_KERNEL); + if (!ne_mem_region) + return -ENOMEM; + + max_nr_pages = mem_region.memory_size / NE_MIN_MEM_REGION_SIZE; + + ne_mem_region->pages = kcalloc(max_nr_pages, sizeof(*ne_mem_region->pages), + GFP_KERNEL); + if (!ne_mem_region->pages) { + rc = -ENOMEM; + + goto free_mem_region; + } + + phys_contig_mem_regions = kcalloc(max_nr_pages, sizeof(*phys_contig_mem_regions), + GFP_KERNEL); + if (!phys_contig_mem_regions) { + rc = -ENOMEM; + + goto free_mem_region; + } + + do { + i = ne_mem_region->nr_pages; + + if (i == max_nr_pages) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Reached max nr of pages in the pages data struct\n"); + + rc = -ENOMEM; + + goto put_pages; + } + + gup_rc = get_user_pages(mem_region.userspace_addr + memory_size, 1, FOLL_GET, + ne_mem_region->pages + i, NULL); + if (gup_rc < 0) { + rc = gup_rc; + + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in get user pages [rc=%d]\n", rc); + + goto put_pages; + } + + rc = ne_sanity_check_user_mem_region_page(ne_enclave, ne_mem_region->pages[i]); + if (rc < 0) + goto put_pages; + + /* + * TODO: Update once handled non-contiguous memory regions + * received from user space or contiguous physical memory regions + * larger than 2 MiB e.g. 8 MiB. + */ + phys_contig_mem_regions[i] = ne_mem_region->pages[i]; + + memory_size += page_size(ne_mem_region->pages[i]); + + ne_mem_region->nr_pages++; + } while (memory_size < mem_region.memory_size); + + /* + * TODO: Update once handled non-contiguous memory regions received + * from user space or contiguous physical memory regions larger than + * 2 MiB e.g. 8 MiB. + */ + nr_phys_contig_mem_regions = ne_mem_region->nr_pages; + + if ((ne_enclave->nr_mem_regions + nr_phys_contig_mem_regions) > + ne_enclave->max_mem_regions) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Reached max memory regions %lld\n", + ne_enclave->max_mem_regions); + + rc = -NE_ERR_MEM_MAX_REGIONS; + + goto put_pages; + } + + for (i = 0; i < nr_phys_contig_mem_regions; i++) { + u64 phys_region_addr = page_to_phys(phys_contig_mem_regions[i]); + u64 phys_region_size = page_size(phys_contig_mem_regions[i]); + + if (phys_region_size & (NE_MIN_MEM_REGION_SIZE - 1)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Physical mem region size is not multiple of 2 MiB\n"); + + rc = -EINVAL; + + goto put_pages; + } + + if (!IS_ALIGNED(phys_region_addr, NE_MIN_MEM_REGION_SIZE)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Physical mem region address is not 2 MiB aligned\n"); + + rc = -EINVAL; + + goto put_pages; + } + } + + ne_mem_region->memory_size = mem_region.memory_size; + ne_mem_region->userspace_addr = mem_region.userspace_addr; + + list_add(&ne_mem_region->mem_region_list_entry, &ne_enclave->mem_regions_list); + + for (i = 0; i < nr_phys_contig_mem_regions; i++) { + struct ne_pci_dev_cmd_reply cmd_reply = {}; + struct slot_add_mem_req slot_add_mem_req = {}; + + slot_add_mem_req.slot_uid = ne_enclave->slot_uid; + slot_add_mem_req.paddr = page_to_phys(phys_contig_mem_regions[i]); + slot_add_mem_req.size = page_size(phys_contig_mem_regions[i]); + + rc = ne_do_request(pdev, SLOT_ADD_MEM, + &slot_add_mem_req, sizeof(slot_add_mem_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in slot add mem [rc=%d]\n", rc); + + kfree(phys_contig_mem_regions); + + /* + * Exit here without put pages as memory regions may + * already been added. + */ + return rc; + } + + ne_enclave->mem_size += slot_add_mem_req.size; + ne_enclave->nr_mem_regions++; + } + + kfree(phys_contig_mem_regions); + + return 0; + +put_pages: + for (i = 0; i < ne_mem_region->nr_pages; i++) + put_page(ne_mem_region->pages[i]); +free_mem_region: + kfree(phys_contig_mem_regions); + kfree(ne_mem_region->pages); + kfree(ne_mem_region); + + return rc; +} + +/** + * ne_start_enclave_ioctl() - Trigger enclave start after the enclave resources, + * such as memory and CPU, have been set. + * @ne_enclave : Private data associated with the current enclave. + * @enclave_start_info : Enclave info that includes enclave cid and flags. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_start_enclave_ioctl(struct ne_enclave *ne_enclave, + struct ne_enclave_start_info *enclave_start_info) +{ + struct ne_pci_dev_cmd_reply cmd_reply = {}; + unsigned int cpu = 0; + struct enclave_start_req enclave_start_req = {}; + unsigned int i = 0; + struct pci_dev *pdev = ne_devs.ne_pci_dev->pdev; + int rc = -EINVAL; + + if (!ne_enclave->nr_mem_regions) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave has no mem regions\n"); + + return -NE_ERR_NO_MEM_REGIONS_ADDED; + } + + if (ne_enclave->mem_size < NE_MIN_ENCLAVE_MEM_SIZE) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave memory is less than %ld\n", + NE_MIN_ENCLAVE_MEM_SIZE); + + return -NE_ERR_ENCLAVE_MEM_MIN_SIZE; + } + + if (!ne_enclave->nr_vcpus) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave has no vCPUs\n"); + + return -NE_ERR_NO_VCPUS_ADDED; + } + + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) + for_each_cpu(cpu, ne_enclave->threads_per_core[i]) + if (!cpumask_test_cpu(cpu, ne_enclave->vcpu_ids)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Full CPU cores not used\n"); + + return -NE_ERR_FULL_CORES_NOT_USED; + } + + enclave_start_req.enclave_cid = enclave_start_info->enclave_cid; + enclave_start_req.flags = enclave_start_info->flags; + enclave_start_req.slot_uid = ne_enclave->slot_uid; + + rc = ne_do_request(pdev, ENCLAVE_START, + &enclave_start_req, sizeof(enclave_start_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in enclave start [rc=%d]\n", rc); + + return rc; + } + + ne_enclave->state = NE_STATE_RUNNING; + + enclave_start_info->enclave_cid = cmd_reply.enclave_cid; + + return 0; +} + +/** + * ne_enclave_ioctl() - Ioctl function provided by the enclave file. + * @file: File associated with this ioctl function. + * @cmd: The command that is set for the ioctl call. + * @arg: The argument that is provided for the ioctl call. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static long ne_enclave_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + struct ne_enclave *ne_enclave = file->private_data; + + switch (cmd) { + case NE_ADD_VCPU: { + int rc = -EINVAL; + u32 vcpu_id = 0; + + if (copy_from_user(&vcpu_id, (void __user *)arg, sizeof(vcpu_id))) + return -EFAULT; + + mutex_lock(&ne_enclave->enclave_info_mutex); + + if (ne_enclave->state != NE_STATE_INIT) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave is not in init state\n"); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return -NE_ERR_NOT_IN_INIT_STATE; + } + + if (vcpu_id >= (ne_enclave->nr_parent_vm_cores * + ne_enclave->nr_threads_per_core)) { + dev_err_ratelimited(ne_misc_dev.this_device, + "vCPU id higher than max CPU id\n"); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return -NE_ERR_INVALID_VCPU; + } + + if (!vcpu_id) { + /* Use the CPU pool for choosing a CPU for the enclave. */ + rc = ne_get_cpu_from_cpu_pool(ne_enclave, &vcpu_id); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in get CPU from pool [rc=%d]\n", + rc); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return rc; + } + } else { + /* Check if the provided vCPU is available in the NE CPU pool. */ + rc = ne_check_cpu_in_cpu_pool(ne_enclave, vcpu_id); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in check CPU %d in pool [rc=%d]\n", + vcpu_id, rc); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return rc; + } + } + + rc = ne_add_vcpu_ioctl(ne_enclave, vcpu_id); + if (rc < 0) { + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return rc; + } + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + if (copy_to_user((void __user *)arg, &vcpu_id, sizeof(vcpu_id))) + return -EFAULT; + + return 0; + } + + case NE_GET_IMAGE_LOAD_INFO: { + struct ne_image_load_info image_load_info = {}; + + if (copy_from_user(&image_load_info, (void __user *)arg, sizeof(image_load_info))) + return -EFAULT; + + mutex_lock(&ne_enclave->enclave_info_mutex); + + if (ne_enclave->state != NE_STATE_INIT) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave is not in init state\n"); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return -NE_ERR_NOT_IN_INIT_STATE; + } + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + if (!image_load_info.flags || + image_load_info.flags >= NE_IMAGE_LOAD_MAX_FLAG_VAL) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Incorrect flag in enclave image load info\n"); + + return -NE_ERR_INVALID_FLAG_VALUE; + } + + if (image_load_info.flags == NE_EIF_IMAGE) + image_load_info.memory_offset = NE_EIF_LOAD_OFFSET; + + if (copy_to_user((void __user *)arg, &image_load_info, sizeof(image_load_info))) + return -EFAULT; + + return 0; + } + + case NE_SET_USER_MEMORY_REGION: { + struct ne_user_memory_region mem_region = {}; + int rc = -EINVAL; + + if (copy_from_user(&mem_region, (void __user *)arg, sizeof(mem_region))) + return -EFAULT; + + if (mem_region.flags >= NE_MEMORY_REGION_MAX_FLAG_VAL) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Incorrect flag for user memory region\n"); + + return -NE_ERR_INVALID_FLAG_VALUE; + } + + mutex_lock(&ne_enclave->enclave_info_mutex); + + if (ne_enclave->state != NE_STATE_INIT) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave is not in init state\n"); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return -NE_ERR_NOT_IN_INIT_STATE; + } + + rc = ne_set_user_memory_region_ioctl(ne_enclave, mem_region); + if (rc < 0) { + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return rc; + } + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return 0; + } + + case NE_START_ENCLAVE: { + struct ne_enclave_start_info enclave_start_info = {}; + int rc = -EINVAL; + + if (copy_from_user(&enclave_start_info, (void __user *)arg, + sizeof(enclave_start_info))) + return -EFAULT; + + if (enclave_start_info.flags >= NE_ENCLAVE_START_MAX_FLAG_VAL) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Incorrect flag in enclave start info\n"); + + return -NE_ERR_INVALID_FLAG_VALUE; + } + + /* + * Do not use well-known CIDs - 0, 1, 2 - for enclaves. + * VMADDR_CID_ANY = -1U + * VMADDR_CID_HYPERVISOR = 0 + * VMADDR_CID_LOCAL = 1 + * VMADDR_CID_HOST = 2 + * Note: 0 is used as a placeholder to auto-generate an enclave CID. + * http://man7.org/linux/man-pages/man7/vsock.7.html + */ + if (enclave_start_info.enclave_cid > 0 && + enclave_start_info.enclave_cid <= VMADDR_CID_HOST) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Well-known CID value, not to be used for enclaves\n"); + + return -NE_ERR_INVALID_ENCLAVE_CID; + } + + if (enclave_start_info.enclave_cid == U32_MAX) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Well-known CID value, not to be used for enclaves\n"); + + return -NE_ERR_INVALID_ENCLAVE_CID; + } + + /* + * Do not use the CID of the primary / parent VM for enclaves. + */ + if (enclave_start_info.enclave_cid == NE_PARENT_VM_CID) { + dev_err_ratelimited(ne_misc_dev.this_device, + "CID of the parent VM, not to be used for enclaves\n"); + + return -NE_ERR_INVALID_ENCLAVE_CID; + } + + /* 64-bit CIDs are not yet supported for the vsock device. */ + if (enclave_start_info.enclave_cid > U32_MAX) { + dev_err_ratelimited(ne_misc_dev.this_device, + "64-bit CIDs not yet supported for the vsock device\n"); + + return -NE_ERR_INVALID_ENCLAVE_CID; + } + + mutex_lock(&ne_enclave->enclave_info_mutex); + + if (ne_enclave->state != NE_STATE_INIT) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Enclave is not in init state\n"); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return -NE_ERR_NOT_IN_INIT_STATE; + } + + rc = ne_start_enclave_ioctl(ne_enclave, &enclave_start_info); + if (rc < 0) { + mutex_unlock(&ne_enclave->enclave_info_mutex); + + return rc; + } + + mutex_unlock(&ne_enclave->enclave_info_mutex); + + if (copy_to_user((void __user *)arg, &enclave_start_info, + sizeof(enclave_start_info))) + return -EFAULT; + + return 0; + } + + default: + return -ENOTTY; + } + + return 0; +} + +/** + * ne_enclave_remove_all_mem_region_entries() - Remove all memory region entries + * from the enclave data structure. + * @ne_enclave : Private data associated with the current enclave. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + */ +static void ne_enclave_remove_all_mem_region_entries(struct ne_enclave *ne_enclave) +{ + unsigned long i = 0; + struct ne_mem_region *ne_mem_region = NULL; + struct ne_mem_region *ne_mem_region_tmp = NULL; + + list_for_each_entry_safe(ne_mem_region, ne_mem_region_tmp, + &ne_enclave->mem_regions_list, + mem_region_list_entry) { + list_del(&ne_mem_region->mem_region_list_entry); + + for (i = 0; i < ne_mem_region->nr_pages; i++) + put_page(ne_mem_region->pages[i]); + + kfree(ne_mem_region->pages); + + kfree(ne_mem_region); + } +} + +/** + * ne_enclave_remove_all_vcpu_id_entries() - Remove all vCPU id entries from + * the enclave data structure. + * @ne_enclave : Private data associated with the current enclave. + * + * Context: Process context. This function is called with the ne_enclave mutex held. + */ +static void ne_enclave_remove_all_vcpu_id_entries(struct ne_enclave *ne_enclave) +{ + unsigned int cpu = 0; + unsigned int i = 0; + + mutex_lock(&ne_cpu_pool.mutex); + + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) { + for_each_cpu(cpu, ne_enclave->threads_per_core[i]) + /* Update the available NE CPU pool. */ + cpumask_set_cpu(cpu, ne_cpu_pool.avail_threads_per_core[i]); + + free_cpumask_var(ne_enclave->threads_per_core[i]); + } + + mutex_unlock(&ne_cpu_pool.mutex); + + kfree(ne_enclave->threads_per_core); + + free_cpumask_var(ne_enclave->vcpu_ids); +} + +/** + * ne_pci_dev_remove_enclave_entry() - Remove the enclave entry from the data + * structure that is part of the NE PCI + * device private data. + * @ne_enclave : Private data associated with the current enclave. + * @ne_pci_dev : Private data associated with the PCI device. + * + * Context: Process context. This function is called with the ne_pci_dev enclave + * mutex held. + */ +static void ne_pci_dev_remove_enclave_entry(struct ne_enclave *ne_enclave, + struct ne_pci_dev *ne_pci_dev) +{ + struct ne_enclave *ne_enclave_entry = NULL; + struct ne_enclave *ne_enclave_entry_tmp = NULL; + + list_for_each_entry_safe(ne_enclave_entry, ne_enclave_entry_tmp, + &ne_pci_dev->enclaves_list, enclave_list_entry) { + if (ne_enclave_entry->slot_uid == ne_enclave->slot_uid) { + list_del(&ne_enclave_entry->enclave_list_entry); + + break; + } + } +} + +/** + * ne_enclave_release() - Release function provided by the enclave file. + * @inode: Inode associated with this file release function. + * @file: File associated with this release function. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_enclave_release(struct inode *inode, struct file *file) +{ + struct ne_pci_dev_cmd_reply cmd_reply = {}; + struct enclave_stop_req enclave_stop_request = {}; + struct ne_enclave *ne_enclave = file->private_data; + struct ne_pci_dev *ne_pci_dev = ne_devs.ne_pci_dev; + struct pci_dev *pdev = ne_pci_dev->pdev; + int rc = -EINVAL; + struct slot_free_req slot_free_req = {}; + + if (!ne_enclave) + return 0; + + /* + * Early exit in case there is an error in the enclave creation logic + * and fput() is called on the cleanup path. + */ + if (!ne_enclave->slot_uid) + return 0; + + /* + * Acquire the enclave list mutex before the enclave mutex + * in order to avoid deadlocks with @ref ne_event_work_handler. + */ + mutex_lock(&ne_pci_dev->enclaves_list_mutex); + mutex_lock(&ne_enclave->enclave_info_mutex); + + if (ne_enclave->state != NE_STATE_INIT && ne_enclave->state != NE_STATE_STOPPED) { + enclave_stop_request.slot_uid = ne_enclave->slot_uid; + + rc = ne_do_request(pdev, ENCLAVE_STOP, + &enclave_stop_request, sizeof(enclave_stop_request), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in enclave stop [rc=%d]\n", rc); + + goto unlock_mutex; + } + + memset(&cmd_reply, 0, sizeof(cmd_reply)); + } + + slot_free_req.slot_uid = ne_enclave->slot_uid; + + rc = ne_do_request(pdev, SLOT_FREE, + &slot_free_req, sizeof(slot_free_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in slot free [rc=%d]\n", rc); + + goto unlock_mutex; + } + + ne_pci_dev_remove_enclave_entry(ne_enclave, ne_pci_dev); + ne_enclave_remove_all_mem_region_entries(ne_enclave); + ne_enclave_remove_all_vcpu_id_entries(ne_enclave); + + mutex_unlock(&ne_enclave->enclave_info_mutex); + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); + + kfree(ne_enclave); + + return 0; + +unlock_mutex: + mutex_unlock(&ne_enclave->enclave_info_mutex); + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); + + return rc; +} + +/** + * ne_enclave_poll() - Poll functionality used for enclave out-of-band events. + * @file: File associated with this poll function. + * @wait: Poll table data structure. + * + * Context: Process context. + * Return: + * * Poll mask. + */ +static __poll_t ne_enclave_poll(struct file *file, poll_table *wait) +{ + __poll_t mask = 0; + struct ne_enclave *ne_enclave = file->private_data; + + poll_wait(file, &ne_enclave->eventq, wait); + + if (!ne_enclave->has_event) + return mask; + + mask = POLLHUP; + + return mask; +} + +static const struct file_operations ne_enclave_fops = { + .owner = THIS_MODULE, + .llseek = noop_llseek, + .poll = ne_enclave_poll, + .unlocked_ioctl = ne_enclave_ioctl, + .release = ne_enclave_release, +}; + +/** + * ne_create_vm_ioctl() - Alloc slot to be associated with an enclave. Create + * enclave file descriptor to be further used for enclave + * resources handling e.g. memory regions and CPUs. + * @ne_pci_dev : Private data associated with the PCI device. + * @slot_uid: Generated unique slot id associated with an enclave. + * + * Context: Process context. This function is called with the ne_pci_dev enclave + * mutex held. + * Return: + * * Enclave fd on success. + * * Negative return value on failure. + */ +static int ne_create_vm_ioctl(struct ne_pci_dev *ne_pci_dev, u64 *slot_uid) +{ + struct ne_pci_dev_cmd_reply cmd_reply = {}; + int enclave_fd = -1; + struct file *enclave_file = NULL; + unsigned int i = 0; + struct ne_enclave *ne_enclave = NULL; + struct pci_dev *pdev = ne_pci_dev->pdev; + int rc = -EINVAL; + struct slot_alloc_req slot_alloc_req = {}; + + mutex_lock(&ne_cpu_pool.mutex); + + for (i = 0; i < ne_cpu_pool.nr_parent_vm_cores; i++) + if (!cpumask_empty(ne_cpu_pool.avail_threads_per_core[i])) + break; + + if (i == ne_cpu_pool.nr_parent_vm_cores) { + dev_err_ratelimited(ne_misc_dev.this_device, + "No CPUs available in CPU pool\n"); + + mutex_unlock(&ne_cpu_pool.mutex); + + return -NE_ERR_NO_CPUS_AVAIL_IN_POOL; + } + + mutex_unlock(&ne_cpu_pool.mutex); + + ne_enclave = kzalloc(sizeof(*ne_enclave), GFP_KERNEL); + if (!ne_enclave) + return -ENOMEM; + + mutex_lock(&ne_cpu_pool.mutex); + + ne_enclave->nr_parent_vm_cores = ne_cpu_pool.nr_parent_vm_cores; + ne_enclave->nr_threads_per_core = ne_cpu_pool.nr_threads_per_core; + ne_enclave->numa_node = ne_cpu_pool.numa_node; + + mutex_unlock(&ne_cpu_pool.mutex); + + ne_enclave->threads_per_core = kcalloc(ne_enclave->nr_parent_vm_cores, + sizeof(*ne_enclave->threads_per_core), GFP_KERNEL); + if (!ne_enclave->threads_per_core) { + rc = -ENOMEM; + + goto free_ne_enclave; + } + + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) + if (!zalloc_cpumask_var(&ne_enclave->threads_per_core[i], GFP_KERNEL)) { + rc = -ENOMEM; + + goto free_cpumask; + } + + if (!zalloc_cpumask_var(&ne_enclave->vcpu_ids, GFP_KERNEL)) { + rc = -ENOMEM; + + goto free_cpumask; + } + + enclave_fd = get_unused_fd_flags(O_CLOEXEC); + if (enclave_fd < 0) { + rc = enclave_fd; + + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in getting unused fd [rc=%d]\n", rc); + + goto free_cpumask; + } + + enclave_file = anon_inode_getfile("ne-vm", &ne_enclave_fops, ne_enclave, O_RDWR); + if (IS_ERR(enclave_file)) { + rc = PTR_ERR(enclave_file); + + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in anon inode get file [rc=%d]\n", rc); + + goto put_fd; + } + + rc = ne_do_request(pdev, SLOT_ALLOC, + &slot_alloc_req, sizeof(slot_alloc_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) { + dev_err_ratelimited(ne_misc_dev.this_device, + "Error in slot alloc [rc=%d]\n", rc); + + goto put_file; + } + + init_waitqueue_head(&ne_enclave->eventq); + ne_enclave->has_event = false; + mutex_init(&ne_enclave->enclave_info_mutex); + ne_enclave->max_mem_regions = cmd_reply.mem_regions; + INIT_LIST_HEAD(&ne_enclave->mem_regions_list); + ne_enclave->mm = current->mm; + ne_enclave->slot_uid = cmd_reply.slot_uid; + ne_enclave->state = NE_STATE_INIT; + + list_add(&ne_enclave->enclave_list_entry, &ne_pci_dev->enclaves_list); + + *slot_uid = ne_enclave->slot_uid; + + fd_install(enclave_fd, enclave_file); + + return enclave_fd; + +put_file: + fput(enclave_file); +put_fd: + put_unused_fd(enclave_fd); +free_cpumask: + free_cpumask_var(ne_enclave->vcpu_ids); + for (i = 0; i < ne_enclave->nr_parent_vm_cores; i++) + free_cpumask_var(ne_enclave->threads_per_core[i]); + kfree(ne_enclave->threads_per_core); +free_ne_enclave: + kfree(ne_enclave); + + return rc; +} + +/** + * ne_ioctl() - Ioctl function provided by the NE misc device. + * @file: File associated with this ioctl function. + * @cmd: The command that is set for the ioctl call. + * @arg: The argument that is provided for the ioctl call. + * + * Context: Process context. + * Return: + * * Ioctl result (e.g. enclave file descriptor) on success. + * * Negative return value on failure. + */ +static long ne_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + switch (cmd) { + case NE_CREATE_VM: { + int enclave_fd = -1; + struct file *enclave_file = NULL; + struct ne_pci_dev *ne_pci_dev = ne_devs.ne_pci_dev; + int rc = -EINVAL; + u64 slot_uid = 0; + + mutex_lock(&ne_pci_dev->enclaves_list_mutex); + + enclave_fd = ne_create_vm_ioctl(ne_pci_dev, &slot_uid); + if (enclave_fd < 0) { + rc = enclave_fd; + + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); + + return rc; + } + + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); + + if (copy_to_user((void __user *)arg, &slot_uid, sizeof(slot_uid))) { + enclave_file = fget(enclave_fd); + /* Decrement file refs to have release() called. */ + fput(enclave_file); + fput(enclave_file); + put_unused_fd(enclave_fd); + + return -EFAULT; + } + + return enclave_fd; + } + + default: + return -ENOTTY; + } + + return 0; +} + +static int __init ne_init(void) +{ + mutex_init(&ne_cpu_pool.mutex); + + return pci_register_driver(&ne_pci_driver); +} + +static void __exit ne_exit(void) +{ + pci_unregister_driver(&ne_pci_driver); + + ne_teardown_cpu_pool(); +} + +module_init(ne_init); +module_exit(ne_exit); + +MODULE_AUTHOR("Amazon.com, Inc. or its affiliates"); +MODULE_DESCRIPTION("Nitro Enclaves Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/virt/nitro_enclaves/ne_misc_dev.h b/drivers/virt/nitro_enclaves/ne_misc_dev.h new file mode 100644 index 000000000000..2a4d2224baba --- /dev/null +++ b/drivers/virt/nitro_enclaves/ne_misc_dev.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +#ifndef _NE_MISC_DEV_H_ +#define _NE_MISC_DEV_H_ + +#include <linux/cpumask.h> +#include <linux/list.h> +#include <linux/miscdevice.h> +#include <linux/mm.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/wait.h> + +#include "ne_pci_dev.h" + +/** + * struct ne_mem_region - Entry in the enclave user space memory regions list. + * @mem_region_list_entry: Entry in the list of enclave memory regions. + * @memory_size: Size of the user space memory region. + * @nr_pages: Number of pages that make up the memory region. + * @pages: Pages that make up the user space memory region. + * @userspace_addr: User space address of the memory region. + */ +struct ne_mem_region { + struct list_head mem_region_list_entry; + u64 memory_size; + unsigned long nr_pages; + struct page **pages; + u64 userspace_addr; +}; + +/** + * struct ne_enclave - Per-enclave data used for enclave lifetime management. + * @enclave_info_mutex : Mutex for accessing this internal state. + * @enclave_list_entry : Entry in the list of created enclaves. + * @eventq: Wait queue used for out-of-band event notifications + * triggered from the PCI device event handler to + * the enclave process via the poll function. + * @has_event: Variable used to determine if the out-of-band event + * was triggered. + * @max_mem_regions: The maximum number of memory regions that can be + * handled by the hypervisor. + * @mem_regions_list: Enclave user space memory regions list. + * @mem_size: Enclave memory size. + * @mm : Enclave process abstraction mm data struct. + * @nr_mem_regions: Number of memory regions associated with the enclave. + * @nr_parent_vm_cores : The size of the threads per core array. The + * total number of CPU cores available on the + * parent / primary VM. + * @nr_threads_per_core: The number of threads that a full CPU core has. + * @nr_vcpus: Number of vcpus associated with the enclave. + * @numa_node: NUMA node of the enclave memory and CPUs. + * @slot_uid: Slot unique id mapped to the enclave. + * @state: Enclave state, updated during enclave lifetime. + * @threads_per_core: Enclave full CPU cores array, indexed by core id, + * consisting of cpumasks with all their threads. + * Full CPU cores are taken from the NE CPU pool + * and are available to the enclave. + * @vcpu_ids: Cpumask of the vCPUs that are set for the enclave. + */ +struct ne_enclave { + struct mutex enclave_info_mutex; + struct list_head enclave_list_entry; + wait_queue_head_t eventq; + bool has_event; + u64 max_mem_regions; + struct list_head mem_regions_list; + u64 mem_size; + struct mm_struct *mm; + unsigned int nr_mem_regions; + unsigned int nr_parent_vm_cores; + unsigned int nr_threads_per_core; + unsigned int nr_vcpus; + int numa_node; + u64 slot_uid; + u16 state; + cpumask_var_t *threads_per_core; + cpumask_var_t vcpu_ids; +}; + +/** + * enum ne_state - States available for an enclave. + * @NE_STATE_INIT: The enclave has not been started yet. + * @NE_STATE_RUNNING: The enclave was started and is running as expected. + * @NE_STATE_STOPPED: The enclave exited without userspace interaction. + */ +enum ne_state { + NE_STATE_INIT = 0, + NE_STATE_RUNNING = 2, + NE_STATE_STOPPED = U16_MAX, +}; + +/** + * struct ne_devs - Data structure to keep refs to the NE misc and PCI devices. + * @ne_misc_dev: Nitro Enclaves misc device. + * @ne_pci_dev : Nitro Enclaves PCI device. + */ +struct ne_devs { + struct miscdevice *ne_misc_dev; + struct ne_pci_dev *ne_pci_dev; +}; + +/* Nitro Enclaves (NE) data structure for keeping refs to the NE misc and PCI devices. */ +extern struct ne_devs ne_devs; + +#endif /* _NE_MISC_DEV_H_ */ diff --git a/drivers/virt/nitro_enclaves/ne_pci_dev.c b/drivers/virt/nitro_enclaves/ne_pci_dev.c new file mode 100644 index 000000000000..b9c1de41e300 --- /dev/null +++ b/drivers/virt/nitro_enclaves/ne_pci_dev.c @@ -0,0 +1,625 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +/** + * DOC: Nitro Enclaves (NE) PCI device driver. + */ + +#include <linux/delay.h> +#include <linux/device.h> +#include <linux/list.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/nitro_enclaves.h> +#include <linux/pci.h> +#include <linux/types.h> +#include <linux/wait.h> + +#include "ne_misc_dev.h" +#include "ne_pci_dev.h" + +/** + * NE_DEFAULT_TIMEOUT_MSECS - Default timeout to wait for a reply from + * the NE PCI device. + */ +#define NE_DEFAULT_TIMEOUT_MSECS (120000) /* 120 sec */ + +static const struct pci_device_id ne_pci_ids[] = { + { PCI_DEVICE(PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_NE) }, + { 0, } +}; + +MODULE_DEVICE_TABLE(pci, ne_pci_ids); + +/** + * ne_submit_request() - Submit command request to the PCI device based on the + * command type. + * @pdev: PCI device to send the command to. + * @cmd_type: Command type of the request sent to the PCI device. + * @cmd_request: Command request payload. + * @cmd_request_size: Size of the command request payload. + * + * Context: Process context. This function is called with the ne_pci_dev mutex held. + */ +static void ne_submit_request(struct pci_dev *pdev, enum ne_pci_dev_cmd_type cmd_type, + void *cmd_request, size_t cmd_request_size) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + memcpy_toio(ne_pci_dev->iomem_base + NE_SEND_DATA, cmd_request, cmd_request_size); + + iowrite32(cmd_type, ne_pci_dev->iomem_base + NE_COMMAND); +} + +/** + * ne_retrieve_reply() - Retrieve reply from the PCI device. + * @pdev: PCI device to receive the reply from. + * @cmd_reply: Command reply payload. + * @cmd_reply_size: Size of the command reply payload. + * + * Context: Process context. This function is called with the ne_pci_dev mutex held. + */ +static void ne_retrieve_reply(struct pci_dev *pdev, struct ne_pci_dev_cmd_reply *cmd_reply, + size_t cmd_reply_size) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + memcpy_fromio(cmd_reply, ne_pci_dev->iomem_base + NE_RECV_DATA, cmd_reply_size); +} + +/** + * ne_wait_for_reply() - Wait for a reply of a PCI device command. + * @pdev: PCI device for which a reply is waited. + * + * Context: Process context. This function is called with the ne_pci_dev mutex held. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_wait_for_reply(struct pci_dev *pdev) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + int rc = -EINVAL; + + /* + * TODO: Update to _interruptible and handle interrupted wait event + * e.g. -ERESTARTSYS, incoming signals + update timeout, if needed. + */ + rc = wait_event_timeout(ne_pci_dev->cmd_reply_wait_q, + atomic_read(&ne_pci_dev->cmd_reply_avail) != 0, + msecs_to_jiffies(NE_DEFAULT_TIMEOUT_MSECS)); + if (!rc) + return -ETIMEDOUT; + + return 0; +} + +int ne_do_request(struct pci_dev *pdev, enum ne_pci_dev_cmd_type cmd_type, + void *cmd_request, size_t cmd_request_size, + struct ne_pci_dev_cmd_reply *cmd_reply, size_t cmd_reply_size) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + int rc = -EINVAL; + + if (cmd_type <= INVALID_CMD || cmd_type >= MAX_CMD) { + dev_err_ratelimited(&pdev->dev, "Invalid cmd type=%u\n", cmd_type); + + return -EINVAL; + } + + if (!cmd_request) { + dev_err_ratelimited(&pdev->dev, "Null cmd request for cmd type=%u\n", + cmd_type); + + return -EINVAL; + } + + if (cmd_request_size > NE_SEND_DATA_SIZE) { + dev_err_ratelimited(&pdev->dev, "Invalid req size=%zu for cmd type=%u\n", + cmd_request_size, cmd_type); + + return -EINVAL; + } + + if (!cmd_reply) { + dev_err_ratelimited(&pdev->dev, "Null cmd reply for cmd type=%u\n", + cmd_type); + + return -EINVAL; + } + + if (cmd_reply_size > NE_RECV_DATA_SIZE) { + dev_err_ratelimited(&pdev->dev, "Invalid reply size=%zu for cmd type=%u\n", + cmd_reply_size, cmd_type); + + return -EINVAL; + } + + /* + * Use this mutex so that the PCI device handles one command request at + * a time. + */ + mutex_lock(&ne_pci_dev->pci_dev_mutex); + + atomic_set(&ne_pci_dev->cmd_reply_avail, 0); + + ne_submit_request(pdev, cmd_type, cmd_request, cmd_request_size); + + rc = ne_wait_for_reply(pdev); + if (rc < 0) { + dev_err_ratelimited(&pdev->dev, "Error in wait for reply for cmd type=%u [rc=%d]\n", + cmd_type, rc); + + goto unlock_mutex; + } + + ne_retrieve_reply(pdev, cmd_reply, cmd_reply_size); + + atomic_set(&ne_pci_dev->cmd_reply_avail, 0); + + if (cmd_reply->rc < 0) { + rc = cmd_reply->rc; + + dev_err_ratelimited(&pdev->dev, "Error in cmd process logic, cmd type=%u [rc=%d]\n", + cmd_type, rc); + + goto unlock_mutex; + } + + rc = 0; + +unlock_mutex: + mutex_unlock(&ne_pci_dev->pci_dev_mutex); + + return rc; +} + +/** + * ne_reply_handler() - Interrupt handler for retrieving a reply matching a + * request sent to the PCI device for enclave lifetime + * management. + * @irq: Received interrupt for a reply sent by the PCI device. + * @args: PCI device private data structure. + * + * Context: Interrupt context. + * Return: + * * IRQ_HANDLED on handled interrupt. + */ +static irqreturn_t ne_reply_handler(int irq, void *args) +{ + struct ne_pci_dev *ne_pci_dev = (struct ne_pci_dev *)args; + + atomic_set(&ne_pci_dev->cmd_reply_avail, 1); + + /* TODO: Update to _interruptible. */ + wake_up(&ne_pci_dev->cmd_reply_wait_q); + + return IRQ_HANDLED; +} + +/** + * ne_event_work_handler() - Work queue handler for notifying enclaves on a + * state change received by the event interrupt + * handler. + * @work: Item containing the NE PCI device for which an out-of-band event + * was issued. + * + * An out-of-band event is being issued by the Nitro Hypervisor when at least + * one enclave is changing state without client interaction. + * + * Context: Work queue context. + */ +static void ne_event_work_handler(struct work_struct *work) +{ + struct ne_pci_dev_cmd_reply cmd_reply = {}; + struct ne_enclave *ne_enclave = NULL; + struct ne_pci_dev *ne_pci_dev = + container_of(work, struct ne_pci_dev, notify_work); + struct pci_dev *pdev = ne_pci_dev->pdev; + int rc = -EINVAL; + struct slot_info_req slot_info_req = {}; + + mutex_lock(&ne_pci_dev->enclaves_list_mutex); + + /* + * Iterate over all enclaves registered for the Nitro Enclaves + * PCI device and determine for which enclave(s) the out-of-band event + * is corresponding to. + */ + list_for_each_entry(ne_enclave, &ne_pci_dev->enclaves_list, enclave_list_entry) { + mutex_lock(&ne_enclave->enclave_info_mutex); + + /* + * Enclaves that were never started cannot receive out-of-band + * events. + */ + if (ne_enclave->state != NE_STATE_RUNNING) + goto unlock; + + slot_info_req.slot_uid = ne_enclave->slot_uid; + + rc = ne_do_request(pdev, SLOT_INFO, + &slot_info_req, sizeof(slot_info_req), + &cmd_reply, sizeof(cmd_reply)); + if (rc < 0) + dev_err(&pdev->dev, "Error in slot info [rc=%d]\n", rc); + + /* Notify enclave process that the enclave state changed. */ + if (ne_enclave->state != cmd_reply.state) { + ne_enclave->state = cmd_reply.state; + + ne_enclave->has_event = true; + + wake_up_interruptible(&ne_enclave->eventq); + } + +unlock: + mutex_unlock(&ne_enclave->enclave_info_mutex); + } + + mutex_unlock(&ne_pci_dev->enclaves_list_mutex); +} + +/** + * ne_event_handler() - Interrupt handler for PCI device out-of-band events. + * This interrupt does not supply any data in the MMIO + * region. It notifies a change in the state of any of + * the launched enclaves. + * @irq: Received interrupt for an out-of-band event. + * @args: PCI device private data structure. + * + * Context: Interrupt context. + * Return: + * * IRQ_HANDLED on handled interrupt. + */ +static irqreturn_t ne_event_handler(int irq, void *args) +{ + struct ne_pci_dev *ne_pci_dev = (struct ne_pci_dev *)args; + + queue_work(ne_pci_dev->event_wq, &ne_pci_dev->notify_work); + + return IRQ_HANDLED; +} + +/** + * ne_setup_msix() - Setup MSI-X vectors for the PCI device. + * @pdev: PCI device to setup the MSI-X for. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_setup_msix(struct pci_dev *pdev) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + int nr_vecs = 0; + int rc = -EINVAL; + + nr_vecs = pci_msix_vec_count(pdev); + if (nr_vecs < 0) { + rc = nr_vecs; + + dev_err(&pdev->dev, "Error in getting vec count [rc=%d]\n", rc); + + return rc; + } + + rc = pci_alloc_irq_vectors(pdev, nr_vecs, nr_vecs, PCI_IRQ_MSIX); + if (rc < 0) { + dev_err(&pdev->dev, "Error in alloc MSI-X vecs [rc=%d]\n", rc); + + return rc; + } + + /* + * This IRQ gets triggered every time the PCI device responds to a + * command request. The reply is then retrieved, reading from the MMIO + * space of the PCI device. + */ + rc = request_irq(pci_irq_vector(pdev, NE_VEC_REPLY), ne_reply_handler, + 0, "enclave_cmd", ne_pci_dev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in request irq reply [rc=%d]\n", rc); + + goto free_irq_vectors; + } + + ne_pci_dev->event_wq = create_singlethread_workqueue("ne_pci_dev_wq"); + if (!ne_pci_dev->event_wq) { + rc = -ENOMEM; + + dev_err(&pdev->dev, "Cannot get wq for dev events [rc=%d]\n", rc); + + goto free_reply_irq_vec; + } + + INIT_WORK(&ne_pci_dev->notify_work, ne_event_work_handler); + + /* + * This IRQ gets triggered every time any enclave's state changes. Its + * handler then scans for the changes and propagates them to the user + * space. + */ + rc = request_irq(pci_irq_vector(pdev, NE_VEC_EVENT), ne_event_handler, + 0, "enclave_evt", ne_pci_dev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in request irq event [rc=%d]\n", rc); + + goto destroy_wq; + } + + return 0; + +destroy_wq: + destroy_workqueue(ne_pci_dev->event_wq); +free_reply_irq_vec: + free_irq(pci_irq_vector(pdev, NE_VEC_REPLY), ne_pci_dev); +free_irq_vectors: + pci_free_irq_vectors(pdev); + + return rc; +} + +/** + * ne_teardown_msix() - Teardown MSI-X vectors for the PCI device. + * @pdev: PCI device to teardown the MSI-X for. + * + * Context: Process context. + */ +static void ne_teardown_msix(struct pci_dev *pdev) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + free_irq(pci_irq_vector(pdev, NE_VEC_EVENT), ne_pci_dev); + + flush_work(&ne_pci_dev->notify_work); + flush_workqueue(ne_pci_dev->event_wq); + destroy_workqueue(ne_pci_dev->event_wq); + + free_irq(pci_irq_vector(pdev, NE_VEC_REPLY), ne_pci_dev); + + pci_free_irq_vectors(pdev); +} + +/** + * ne_pci_dev_enable() - Select the PCI device version and enable it. + * @pdev: PCI device to select version for and then enable. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_pci_dev_enable(struct pci_dev *pdev) +{ + u8 dev_enable_reply = 0; + u16 dev_version_reply = 0; + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + iowrite16(NE_VERSION_MAX, ne_pci_dev->iomem_base + NE_VERSION); + + dev_version_reply = ioread16(ne_pci_dev->iomem_base + NE_VERSION); + if (dev_version_reply != NE_VERSION_MAX) { + dev_err(&pdev->dev, "Error in pci dev version cmd\n"); + + return -EIO; + } + + iowrite8(NE_ENABLE_ON, ne_pci_dev->iomem_base + NE_ENABLE); + + dev_enable_reply = ioread8(ne_pci_dev->iomem_base + NE_ENABLE); + if (dev_enable_reply != NE_ENABLE_ON) { + dev_err(&pdev->dev, "Error in pci dev enable cmd\n"); + + return -EIO; + } + + return 0; +} + +/** + * ne_pci_dev_disable() - Disable the PCI device. + * @pdev: PCI device to disable. + * + * Context: Process context. + */ +static void ne_pci_dev_disable(struct pci_dev *pdev) +{ + u8 dev_disable_reply = 0; + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + const unsigned int sleep_time = 10; /* 10 ms */ + unsigned int sleep_time_count = 0; + + iowrite8(NE_ENABLE_OFF, ne_pci_dev->iomem_base + NE_ENABLE); + + /* + * Check for NE_ENABLE_OFF in a loop, to handle cases when the device + * state is not immediately set to disabled and going through a + * transitory state of disabling. + */ + while (sleep_time_count < NE_DEFAULT_TIMEOUT_MSECS) { + dev_disable_reply = ioread8(ne_pci_dev->iomem_base + NE_ENABLE); + if (dev_disable_reply == NE_ENABLE_OFF) + return; + + msleep_interruptible(sleep_time); + sleep_time_count += sleep_time; + } + + dev_disable_reply = ioread8(ne_pci_dev->iomem_base + NE_ENABLE); + if (dev_disable_reply != NE_ENABLE_OFF) + dev_err(&pdev->dev, "Error in pci dev disable cmd\n"); +} + +/** + * ne_pci_probe() - Probe function for the NE PCI device. + * @pdev: PCI device to match with the NE PCI driver. + * @id : PCI device id table associated with the NE PCI driver. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + struct ne_pci_dev *ne_pci_dev = NULL; + int rc = -EINVAL; + + ne_pci_dev = kzalloc(sizeof(*ne_pci_dev), GFP_KERNEL); + if (!ne_pci_dev) + return -ENOMEM; + + rc = pci_enable_device(pdev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in pci dev enable [rc=%d]\n", rc); + + goto free_ne_pci_dev; + } + + rc = pci_request_regions_exclusive(pdev, "nitro_enclaves"); + if (rc < 0) { + dev_err(&pdev->dev, "Error in pci request regions [rc=%d]\n", rc); + + goto disable_pci_dev; + } + + ne_pci_dev->iomem_base = pci_iomap(pdev, PCI_BAR_NE, 0); + if (!ne_pci_dev->iomem_base) { + rc = -ENOMEM; + + dev_err(&pdev->dev, "Error in pci iomap [rc=%d]\n", rc); + + goto release_pci_regions; + } + + pci_set_drvdata(pdev, ne_pci_dev); + + rc = ne_setup_msix(pdev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in pci dev msix setup [rc=%d]\n", rc); + + goto iounmap_pci_bar; + } + + ne_pci_dev_disable(pdev); + + rc = ne_pci_dev_enable(pdev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in ne_pci_dev enable [rc=%d]\n", rc); + + goto teardown_msix; + } + + atomic_set(&ne_pci_dev->cmd_reply_avail, 0); + init_waitqueue_head(&ne_pci_dev->cmd_reply_wait_q); + INIT_LIST_HEAD(&ne_pci_dev->enclaves_list); + mutex_init(&ne_pci_dev->enclaves_list_mutex); + mutex_init(&ne_pci_dev->pci_dev_mutex); + ne_pci_dev->pdev = pdev; + + ne_devs.ne_pci_dev = ne_pci_dev; + + rc = misc_register(ne_devs.ne_misc_dev); + if (rc < 0) { + dev_err(&pdev->dev, "Error in misc dev register [rc=%d]\n", rc); + + goto disable_ne_pci_dev; + } + + return 0; + +disable_ne_pci_dev: + ne_devs.ne_pci_dev = NULL; + ne_pci_dev_disable(pdev); +teardown_msix: + ne_teardown_msix(pdev); +iounmap_pci_bar: + pci_set_drvdata(pdev, NULL); + pci_iounmap(pdev, ne_pci_dev->iomem_base); +release_pci_regions: + pci_release_regions(pdev); +disable_pci_dev: + pci_disable_device(pdev); +free_ne_pci_dev: + kfree(ne_pci_dev); + + return rc; +} + +/** + * ne_pci_remove() - Remove function for the NE PCI device. + * @pdev: PCI device associated with the NE PCI driver. + * + * Context: Process context. + */ +static void ne_pci_remove(struct pci_dev *pdev) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + misc_deregister(ne_devs.ne_misc_dev); + + ne_devs.ne_pci_dev = NULL; + + ne_pci_dev_disable(pdev); + + ne_teardown_msix(pdev); + + pci_set_drvdata(pdev, NULL); + + pci_iounmap(pdev, ne_pci_dev->iomem_base); + + pci_release_regions(pdev); + + pci_disable_device(pdev); + + kfree(ne_pci_dev); +} + +/** + * ne_pci_shutdown() - Shutdown function for the NE PCI device. + * @pdev: PCI device associated with the NE PCI driver. + * + * Context: Process context. + */ +static void ne_pci_shutdown(struct pci_dev *pdev) +{ + struct ne_pci_dev *ne_pci_dev = pci_get_drvdata(pdev); + + if (!ne_pci_dev) + return; + + misc_deregister(ne_devs.ne_misc_dev); + + ne_devs.ne_pci_dev = NULL; + + ne_pci_dev_disable(pdev); + + ne_teardown_msix(pdev); + + pci_set_drvdata(pdev, NULL); + + pci_iounmap(pdev, ne_pci_dev->iomem_base); + + pci_release_regions(pdev); + + pci_disable_device(pdev); + + kfree(ne_pci_dev); +} + +/* + * TODO: Add suspend / resume functions for power management w/ CONFIG_PM, if + * needed. + */ +/* NE PCI device driver. */ +struct pci_driver ne_pci_driver = { + .name = "nitro_enclaves", + .id_table = ne_pci_ids, + .probe = ne_pci_probe, + .remove = ne_pci_remove, + .shutdown = ne_pci_shutdown, +}; diff --git a/drivers/virt/nitro_enclaves/ne_pci_dev.h b/drivers/virt/nitro_enclaves/ne_pci_dev.h new file mode 100644 index 000000000000..8bfbc6607818 --- /dev/null +++ b/drivers/virt/nitro_enclaves/ne_pci_dev.h @@ -0,0 +1,327 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +#ifndef _NE_PCI_DEV_H_ +#define _NE_PCI_DEV_H_ + +#include <linux/atomic.h> +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/pci_ids.h> +#include <linux/wait.h> + +/** + * DOC: Nitro Enclaves (NE) PCI device + */ + +/** + * PCI_DEVICE_ID_NE - Nitro Enclaves PCI device id. + */ +#define PCI_DEVICE_ID_NE (0xe4c1) +/** + * PCI_BAR_NE - Nitro Enclaves PCI device MMIO BAR. + */ +#define PCI_BAR_NE (0x03) + +/** + * DOC: Device registers in the NE PCI device MMIO BAR + */ + +/** + * NE_ENABLE - (1 byte) Register to notify the device that the driver is using + * it (Read/Write). + */ +#define NE_ENABLE (0x0000) +#define NE_ENABLE_OFF (0x00) +#define NE_ENABLE_ON (0x01) + +/** + * NE_VERSION - (2 bytes) Register to select the device run-time version + * (Read/Write). + */ +#define NE_VERSION (0x0002) +#define NE_VERSION_MAX (0x0001) + +/** + * NE_COMMAND - (4 bytes) Register to notify the device what command was + * requested (Write-Only). + */ +#define NE_COMMAND (0x0004) + +/** + * NE_EVTCNT - (4 bytes) Register to notify the driver that a reply or a device + * event is available (Read-Only): + * - Lower half - command reply counter + * - Higher half - out-of-band device event counter + */ +#define NE_EVTCNT (0x000c) +#define NE_EVTCNT_REPLY_SHIFT (0) +#define NE_EVTCNT_REPLY_MASK (0x0000ffff) +#define NE_EVTCNT_REPLY(cnt) (((cnt) & NE_EVTCNT_REPLY_MASK) >> \ + NE_EVTCNT_REPLY_SHIFT) +#define NE_EVTCNT_EVENT_SHIFT (16) +#define NE_EVTCNT_EVENT_MASK (0xffff0000) +#define NE_EVTCNT_EVENT(cnt) (((cnt) & NE_EVTCNT_EVENT_MASK) >> \ + NE_EVTCNT_EVENT_SHIFT) + +/** + * NE_SEND_DATA - (240 bytes) Buffer for sending the command request payload + * (Read/Write). + */ +#define NE_SEND_DATA (0x0010) + +/** + * NE_RECV_DATA - (240 bytes) Buffer for receiving the command reply payload + * (Read-Only). + */ +#define NE_RECV_DATA (0x0100) + +/** + * DOC: Device MMIO buffer sizes + */ + +/** + * NE_SEND_DATA_SIZE / NE_RECV_DATA_SIZE - 240 bytes for send / recv buffer. + */ +#define NE_SEND_DATA_SIZE (240) +#define NE_RECV_DATA_SIZE (240) + +/** + * DOC: MSI-X interrupt vectors + */ + +/** + * NE_VEC_REPLY - MSI-X vector used for command reply notification. + */ +#define NE_VEC_REPLY (0) + +/** + * NE_VEC_EVENT - MSI-X vector used for out-of-band events e.g. enclave crash. + */ +#define NE_VEC_EVENT (1) + +/** + * enum ne_pci_dev_cmd_type - Device command types. + * @INVALID_CMD: Invalid command. + * @ENCLAVE_START: Start an enclave, after setting its resources. + * @ENCLAVE_GET_SLOT: Get the slot uid of an enclave. + * @ENCLAVE_STOP: Terminate an enclave. + * @SLOT_ALLOC : Allocate a slot for an enclave. + * @SLOT_FREE: Free the slot allocated for an enclave + * @SLOT_ADD_MEM: Add a memory region to an enclave slot. + * @SLOT_ADD_VCPU: Add a vCPU to an enclave slot. + * @SLOT_COUNT : Get the number of allocated slots. + * @NEXT_SLOT: Get the next slot in the list of allocated slots. + * @SLOT_INFO: Get the info for a slot e.g. slot uid, vCPUs count. + * @SLOT_ADD_BULK_VCPUS: Add a number of vCPUs, not providing CPU ids. + * @MAX_CMD: A gatekeeper for max possible command type. + */ +enum ne_pci_dev_cmd_type { + INVALID_CMD = 0, + ENCLAVE_START = 1, + ENCLAVE_GET_SLOT = 2, + ENCLAVE_STOP = 3, + SLOT_ALLOC = 4, + SLOT_FREE = 5, + SLOT_ADD_MEM = 6, + SLOT_ADD_VCPU = 7, + SLOT_COUNT = 8, + NEXT_SLOT = 9, + SLOT_INFO = 10, + SLOT_ADD_BULK_VCPUS = 11, + MAX_CMD, +}; + +/** + * DOC: Device commands - payload structure for requests and replies. + */ + +/** + * struct enclave_start_req - ENCLAVE_START request. + * @slot_uid: Slot unique id mapped to the enclave to start. + * @enclave_cid: Context ID (CID) for the enclave vsock device. + * If 0, CID is autogenerated. + * @flags: Flags for the enclave to start with (e.g. debug mode). + */ +struct enclave_start_req { + u64 slot_uid; + u64 enclave_cid; + u64 flags; +}; + +/** + * struct enclave_get_slot_req - ENCLAVE_GET_SLOT request. + * @enclave_cid: Context ID (CID) for the enclave vsock device. + */ +struct enclave_get_slot_req { + u64 enclave_cid; +}; + +/** + * struct enclave_stop_req - ENCLAVE_STOP request. + * @slot_uid: Slot unique id mapped to the enclave to stop. + */ +struct enclave_stop_req { + u64 slot_uid; +}; + +/** + * struct slot_alloc_req - SLOT_ALLOC request. + * @unused: In order to avoid weird sizeof edge cases. + */ +struct slot_alloc_req { + u8 unused; +}; + +/** + * struct slot_free_req - SLOT_FREE request. + * @slot_uid: Slot unique id mapped to the slot to free. + */ +struct slot_free_req { + u64 slot_uid; +}; + +/* TODO: Add flags field to the request to add memory region. */ +/** + * struct slot_add_mem_req - SLOT_ADD_MEM request. + * @slot_uid: Slot unique id mapped to the slot to add the memory region to. + * @paddr: Physical address of the memory region to add to the slot. + * @size: Memory size, in bytes, of the memory region to add to the slot. + */ +struct slot_add_mem_req { + u64 slot_uid; + u64 paddr; + u64 size; +}; + +/** + * struct slot_add_vcpu_req - SLOT_ADD_VCPU request. + * @slot_uid: Slot unique id mapped to the slot to add the vCPU to. + * @vcpu_id: vCPU ID of the CPU to add to the enclave. + * @padding: Padding for the overall data structure. + */ +struct slot_add_vcpu_req { + u64 slot_uid; + u32 vcpu_id; + u8 padding[4]; +}; + +/** + * struct slot_count_req - SLOT_COUNT request. + * @unused: In order to avoid weird sizeof edge cases. + */ +struct slot_count_req { + u8 unused; +}; + +/** + * struct next_slot_req - NEXT_SLOT request. + * @slot_uid: Slot unique id of the next slot in the iteration. + */ +struct next_slot_req { + u64 slot_uid; +}; + +/** + * struct slot_info_req - SLOT_INFO request. + * @slot_uid: Slot unique id mapped to the slot to get information about. + */ +struct slot_info_req { + u64 slot_uid; +}; + +/** + * struct slot_add_bulk_vcpus_req - SLOT_ADD_BULK_VCPUS request. + * @slot_uid: Slot unique id mapped to the slot to add vCPUs to. + * @nr_vcpus: Number of vCPUs to add to the slot. + */ +struct slot_add_bulk_vcpus_req { + u64 slot_uid; + u64 nr_vcpus; +}; + +/** + * struct ne_pci_dev_cmd_reply - NE PCI device command reply. + * @rc : Return code of the logic that processed the request. + * @padding0: Padding for the overall data structure. + * @slot_uid: Valid for all commands except SLOT_COUNT. + * @enclave_cid: Valid for ENCLAVE_START command. + * @slot_count : Valid for SLOT_COUNT command. + * @mem_regions: Valid for SLOT_ALLOC and SLOT_INFO commands. + * @mem_size: Valid for SLOT_INFO command. + * @nr_vcpus: Valid for SLOT_INFO command. + * @flags: Valid for SLOT_INFO command. + * @state: Valid for SLOT_INFO command. + * @padding1: Padding for the overall data structure. + */ +struct ne_pci_dev_cmd_reply { + s32 rc; + u8 padding0[4]; + u64 slot_uid; + u64 enclave_cid; + u64 slot_count; + u64 mem_regions; + u64 mem_size; + u64 nr_vcpus; + u64 flags; + u16 state; + u8 padding1[6]; +}; + +/** + * struct ne_pci_dev - Nitro Enclaves (NE) PCI device. + * @cmd_reply_avail: Variable set if a reply has been sent by the + * PCI device. + * @cmd_reply_wait_q: Wait queue for handling command reply from the + * PCI device. + * @enclaves_list: List of the enclaves managed by the PCI device. + * @enclaves_list_mutex: Mutex for accessing the list of enclaves. + * @event_wq: Work queue for handling out-of-band events + * triggered by the Nitro Hypervisor which require + * enclave state scanning and propagation to the + * enclave process. + * @iomem_base : MMIO region of the PCI device. + * @notify_work: Work item for every received out-of-band event. + * @pci_dev_mutex: Mutex for accessing the PCI device MMIO space. + * @pdev: PCI device data structure. + */ +struct ne_pci_dev { + atomic_t cmd_reply_avail; + wait_queue_head_t cmd_reply_wait_q; + struct list_head enclaves_list; + struct mutex enclaves_list_mutex; + struct workqueue_struct *event_wq; + void __iomem *iomem_base; + struct work_struct notify_work; + struct mutex pci_dev_mutex; + struct pci_dev *pdev; +}; + +/** + * ne_do_request() - Submit command request to the PCI device based on the command + * type and retrieve the associated reply. + * @pdev: PCI device to send the command to and receive the reply from. + * @cmd_type: Command type of the request sent to the PCI device. + * @cmd_request: Command request payload. + * @cmd_request_size: Size of the command request payload. + * @cmd_reply: Command reply payload. + * @cmd_reply_size: Size of the command reply payload. + * + * Context: Process context. This function uses the ne_pci_dev mutex to handle + * one command at a time. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +int ne_do_request(struct pci_dev *pdev, enum ne_pci_dev_cmd_type cmd_type, + void *cmd_request, size_t cmd_request_size, + struct ne_pci_dev_cmd_reply *cmd_reply, + size_t cmd_reply_size); + +/* Nitro Enclaves (NE) PCI device driver */ +extern struct pci_driver ne_pci_driver; + +#endif /* _NE_PCI_DEV_H_ */ diff --git a/drivers/virt/vboxguest/vboxguest_linux.c b/drivers/virt/vboxguest/vboxguest_linux.c index 32c2c52f7e84..73eb34849eab 100644 --- a/drivers/virt/vboxguest/vboxguest_linux.c +++ b/drivers/virt/vboxguest/vboxguest_linux.c @@ -35,7 +35,7 @@ static u32 vbg_misc_device_requestor(struct inode *inode) VMMDEV_REQUESTOR_CON_DONT_KNOW | VMMDEV_REQUESTOR_TRUST_NOT_GIVEN; - if (from_kuid(current_user_ns(), current->cred->uid) == 0) + if (from_kuid(current_user_ns(), current_uid()) == 0) requestor |= VMMDEV_REQUESTOR_USR_ROOT; else requestor |= VMMDEV_REQUESTOR_USR_USER; @@ -202,13 +202,8 @@ static int vbg_input_open(struct input_dev *input) { struct vbg_dev *gdev = input_get_drvdata(input); u32 feat = VMMDEV_MOUSE_GUEST_CAN_ABSOLUTE | VMMDEV_MOUSE_NEW_PROTOCOL; - int ret; - ret = vbg_core_set_mouse_status(gdev, feat); - if (ret) - return ret; - - return 0; + return vbg_core_set_mouse_status(gdev, feat); } /** diff --git a/drivers/w1/masters/mxc_w1.c b/drivers/w1/masters/mxc_w1.c index 1ca880e01476..090cbbf9e1e2 100644 --- a/drivers/w1/masters/mxc_w1.c +++ b/drivers/w1/masters/mxc_w1.c @@ -7,7 +7,7 @@ #include <linux/clk.h> #include <linux/delay.h> #include <linux/io.h> -#include <linux/jiffies.h> +#include <linux/ktime.h> #include <linux/module.h> #include <linux/mod_devicetable.h> #include <linux/platform_device.h> @@ -40,12 +40,12 @@ struct mxc_w1_device { static u8 mxc_w1_ds2_reset_bus(void *data) { struct mxc_w1_device *dev = data; - unsigned long timeout; + ktime_t timeout; writeb(MXC_W1_CONTROL_RPP, dev->regs + MXC_W1_CONTROL); /* Wait for reset sequence 511+512us, use 1500us for sure */ - timeout = jiffies + usecs_to_jiffies(1500); + timeout = ktime_add_us(ktime_get(), 1500); udelay(511 + 512); @@ -55,7 +55,7 @@ static u8 mxc_w1_ds2_reset_bus(void *data) /* PST bit is valid after the RPP bit is self-cleared */ if (!(ctrl & MXC_W1_CONTROL_RPP)) return !(ctrl & MXC_W1_CONTROL_PST); - } while (time_is_after_jiffies(timeout)); + } while (ktime_before(ktime_get(), timeout)); return 1; } @@ -68,12 +68,12 @@ static u8 mxc_w1_ds2_reset_bus(void *data) static u8 mxc_w1_ds2_touch_bit(void *data, u8 bit) { struct mxc_w1_device *dev = data; - unsigned long timeout; + ktime_t timeout; writeb(MXC_W1_CONTROL_WR(bit), dev->regs + MXC_W1_CONTROL); /* Wait for read/write bit (60us, Max 120us), use 200us for sure */ - timeout = jiffies + usecs_to_jiffies(200); + timeout = ktime_add_us(ktime_get(), 200); udelay(60); @@ -83,7 +83,7 @@ static u8 mxc_w1_ds2_touch_bit(void *data, u8 bit) /* RDST bit is valid after the WR1/RD bit is self-cleared */ if (!(ctrl & MXC_W1_CONTROL_WR(bit))) return !!(ctrl & MXC_W1_CONTROL_RDST); - } while (time_is_after_jiffies(timeout)); + } while (ktime_before(ktime_get(), timeout)); return 0; } diff --git a/drivers/w1/slaves/w1_ds2405.c b/drivers/w1/slaves/w1_ds2405.c index 86cd97309d87..1d9a1183e83f 100644 --- a/drivers/w1/slaves/w1_ds2405.c +++ b/drivers/w1/slaves/w1_ds2405.c @@ -206,7 +206,7 @@ static struct attribute *w1_ds2405_attrs[] = { ATTRIBUTE_GROUPS(w1_ds2405); -static struct w1_family_ops w1_ds2405_fops = { +static const struct w1_family_ops w1_ds2405_fops = { .groups = w1_ds2405_groups }; diff --git a/drivers/w1/slaves/w1_ds2406.c b/drivers/w1/slaves/w1_ds2406.c index 762e5e4e2b48..6c269af73c80 100644 --- a/drivers/w1/slaves/w1_ds2406.c +++ b/drivers/w1/slaves/w1_ds2406.c @@ -138,7 +138,7 @@ static void w1_f12_remove_slave(struct w1_slave *sl) &(w1_f12_sysfs_bin_files[i])); } -static struct w1_family_ops w1_f12_fops = { +static const struct w1_family_ops w1_f12_fops = { .add_slave = w1_f12_add_slave, .remove_slave = w1_f12_remove_slave, }; diff --git a/drivers/w1/slaves/w1_ds2408.c b/drivers/w1/slaves/w1_ds2408.c index 83f8d94bb814..ad102c577122 100644 --- a/drivers/w1/slaves/w1_ds2408.c +++ b/drivers/w1/slaves/w1_ds2408.c @@ -336,7 +336,7 @@ static const struct attribute_group *w1_f29_groups[] = { NULL, }; -static struct w1_family_ops w1_f29_fops = { +static const struct w1_family_ops w1_f29_fops = { .add_slave = w1_f29_disable_test_mode, .groups = w1_f29_groups, }; diff --git a/drivers/w1/slaves/w1_ds2413.c b/drivers/w1/slaves/w1_ds2413.c index f1fb18afbcea..c8cfac555b48 100644 --- a/drivers/w1/slaves/w1_ds2413.c +++ b/drivers/w1/slaves/w1_ds2413.c @@ -143,7 +143,7 @@ static const struct attribute_group *w1_f3a_groups[] = { NULL, }; -static struct w1_family_ops w1_f3a_fops = { +static const struct w1_family_ops w1_f3a_fops = { .groups = w1_f3a_groups, }; diff --git a/drivers/w1/slaves/w1_ds2423.c b/drivers/w1/slaves/w1_ds2423.c index f4367282dcc1..b6bd18d5b3f6 100644 --- a/drivers/w1/slaves/w1_ds2423.c +++ b/drivers/w1/slaves/w1_ds2423.c @@ -117,7 +117,7 @@ static struct attribute *w1_f1d_attrs[] = { }; ATTRIBUTE_GROUPS(w1_f1d); -static struct w1_family_ops w1_f1d_fops = { +static const struct w1_family_ops w1_f1d_fops = { .groups = w1_f1d_groups, }; diff --git a/drivers/w1/slaves/w1_ds2430.c b/drivers/w1/slaves/w1_ds2430.c index 75bb8a88620b..0ea7d779d17a 100644 --- a/drivers/w1/slaves/w1_ds2430.c +++ b/drivers/w1/slaves/w1_ds2430.c @@ -279,7 +279,7 @@ static const struct attribute_group *w1_f14_groups[] = { NULL, }; -static struct w1_family_ops w1_f14_fops = { +static const struct w1_family_ops w1_f14_fops = { .groups = w1_f14_groups, }; diff --git a/drivers/w1/slaves/w1_ds2431.c b/drivers/w1/slaves/w1_ds2431.c index e5bd7e2354d7..6856b1c29e17 100644 --- a/drivers/w1/slaves/w1_ds2431.c +++ b/drivers/w1/slaves/w1_ds2431.c @@ -278,7 +278,7 @@ static const struct attribute_group *w1_f2d_groups[] = { NULL, }; -static struct w1_family_ops w1_f2d_fops = { +static const struct w1_family_ops w1_f2d_fops = { .groups = w1_f2d_groups, }; diff --git a/drivers/w1/slaves/w1_ds2433.c b/drivers/w1/slaves/w1_ds2433.c index 1f805c86517a..0f72df15a024 100644 --- a/drivers/w1/slaves/w1_ds2433.c +++ b/drivers/w1/slaves/w1_ds2433.c @@ -288,7 +288,7 @@ static void w1_f23_remove_slave(struct w1_slave *sl) #endif /* CONFIG_W1_SLAVE_DS2433_CRC */ } -static struct w1_family_ops w1_f23_fops = { +static const struct w1_family_ops w1_f23_fops = { .add_slave = w1_f23_add_slave, .remove_slave = w1_f23_remove_slave, .groups = w1_f23_groups, diff --git a/drivers/w1/slaves/w1_ds2438.c b/drivers/w1/slaves/w1_ds2438.c index d199e5a25cc0..5cfb0ae23e91 100644 --- a/drivers/w1/slaves/w1_ds2438.c +++ b/drivers/w1/slaves/w1_ds2438.c @@ -412,7 +412,7 @@ static const struct attribute_group *w1_ds2438_groups[] = { NULL, }; -static struct w1_family_ops w1_ds2438_fops = { +static const struct w1_family_ops w1_ds2438_fops = { .groups = w1_ds2438_groups, }; diff --git a/drivers/w1/slaves/w1_ds250x.c b/drivers/w1/slaves/w1_ds250x.c index e507117444d8..7592c7050d1d 100644 --- a/drivers/w1/slaves/w1_ds250x.c +++ b/drivers/w1/slaves/w1_ds250x.c @@ -215,7 +215,7 @@ static int w1_eprom_add_slave(struct w1_slave *sl) return PTR_ERR_OR_ZERO(nvmem); } -static struct w1_family_ops w1_eprom_fops = { +static const struct w1_family_ops w1_eprom_fops = { .add_slave = w1_eprom_add_slave, }; diff --git a/drivers/w1/slaves/w1_ds2780.c b/drivers/w1/slaves/w1_ds2780.c index c689b1b987b8..c281fe5ed688 100644 --- a/drivers/w1/slaves/w1_ds2780.c +++ b/drivers/w1/slaves/w1_ds2780.c @@ -141,7 +141,7 @@ static void w1_ds2780_remove_slave(struct w1_slave *sl) platform_device_unregister(pdev); } -static struct w1_family_ops w1_ds2780_fops = { +static const struct w1_family_ops w1_ds2780_fops = { .add_slave = w1_ds2780_add_slave, .remove_slave = w1_ds2780_remove_slave, .groups = w1_ds2780_groups, diff --git a/drivers/w1/slaves/w1_ds2781.c b/drivers/w1/slaves/w1_ds2781.c index 84d6ceec5da5..f0d393ae070b 100644 --- a/drivers/w1/slaves/w1_ds2781.c +++ b/drivers/w1/slaves/w1_ds2781.c @@ -138,7 +138,7 @@ static void w1_ds2781_remove_slave(struct w1_slave *sl) platform_device_unregister(pdev); } -static struct w1_family_ops w1_ds2781_fops = { +static const struct w1_family_ops w1_ds2781_fops = { .add_slave = w1_ds2781_add_slave, .remove_slave = w1_ds2781_remove_slave, .groups = w1_ds2781_groups, diff --git a/drivers/w1/slaves/w1_ds2805.c b/drivers/w1/slaves/w1_ds2805.c index ccb753a474b1..206186db727d 100644 --- a/drivers/w1/slaves/w1_ds2805.c +++ b/drivers/w1/slaves/w1_ds2805.c @@ -281,7 +281,7 @@ static void w1_f0d_remove_slave(struct w1_slave *sl) sysfs_remove_bin_file(&sl->dev.kobj, &w1_f0d_bin_attr); } -static struct w1_family_ops w1_f0d_fops = { +static const struct w1_family_ops w1_f0d_fops = { .add_slave = w1_f0d_add_slave, .remove_slave = w1_f0d_remove_slave, }; diff --git a/drivers/w1/slaves/w1_ds28e04.c b/drivers/w1/slaves/w1_ds28e04.c index 8a640f159078..e4f336111edc 100644 --- a/drivers/w1/slaves/w1_ds28e04.c +++ b/drivers/w1/slaves/w1_ds28e04.c @@ -410,7 +410,7 @@ static void w1_f1C_remove_slave(struct w1_slave *sl) sl->family_data = NULL; } -static struct w1_family_ops w1_f1C_fops = { +static const struct w1_family_ops w1_f1C_fops = { .add_slave = w1_f1C_add_slave, .remove_slave = w1_f1C_remove_slave, .groups = w1_f1C_groups, diff --git a/drivers/w1/slaves/w1_ds28e17.c b/drivers/w1/slaves/w1_ds28e17.c index 046ddda83df9..6b00db7169ab 100644 --- a/drivers/w1/slaves/w1_ds28e17.c +++ b/drivers/w1/slaves/w1_ds28e17.c @@ -741,7 +741,7 @@ static void w1_f19_remove_slave(struct w1_slave *sl) /* Declarations within the w1 subsystem. */ -static struct w1_family_ops w1_f19_fops = { +static const struct w1_family_ops w1_f19_fops = { .add_slave = w1_f19_add_slave, .remove_slave = w1_f19_remove_slave, .groups = w1_f19_groups, diff --git a/drivers/w1/slaves/w1_therm.c b/drivers/w1/slaves/w1_therm.c index c1b4eda16719..cddf60b7309c 100644 --- a/drivers/w1/slaves/w1_therm.c +++ b/drivers/w1/slaves/w1_therm.c @@ -17,6 +17,7 @@ #include <linux/delay.h> #include <linux/hwmon.h> #include <linux/string.h> +#include <linux/jiffies.h> #include <linux/w1.h> @@ -65,6 +66,32 @@ static u16 bulk_read_device_counter; /* =0 as per C standard */ #define MIN_TEMP -55 /* min temperature that can be mesured */ #define MAX_TEMP 125 /* max temperature that can be mesured */ +/* Allowed values for sysfs conv_time attribute */ +#define CONV_TIME_DEFAULT 0 +#define CONV_TIME_MEASURE 1 + +/* Bits in sysfs "features" value */ +#define W1_THERM_CHECK_RESULT 1 /* Enable conversion success check */ +#define W1_THERM_POLL_COMPLETION 2 /* Poll for conversion completion */ +#define W1_THERM_FEATURES_MASK 3 /* All values mask */ + +/* Poll period in milliseconds. Should be less then a shortest operation on the device */ +#define W1_POLL_PERIOD 32 +#define W1_POLL_CONVERT_TEMP 2000 /* Timeout for W1_CONVERT_TEMP, ms */ +#define W1_POLL_RECALL_EEPROM 500 /* Timeout for W1_RECALL_EEPROM, ms*/ + +/* Masks for resolution functions, work with all devices */ +/* Bit mask for config register for all devices, bits 7,6,5 */ +#define W1_THERM_RESOLUTION_MASK 0xE0 +/* Bit offset of resolution in config register for all devices */ +#define W1_THERM_RESOLUTION_SHIFT 5 +/* Bit offset of resolution in config register for all devices */ +#define W1_THERM_RESOLUTION_SHIFT 5 +/* Add this to bit value to get resolution */ +#define W1_THERM_RESOLUTION_MIN 9 +/* Maximum allowed value */ +#define W1_THERM_RESOLUTION_MAX 14 + /* Helpers Macros */ /* @@ -89,6 +116,20 @@ static u16 bulk_read_device_counter; /* =0 as per C standard */ (((struct w1_therm_family_data *)(sl->family_data))->resolution) /* + * return the conv_time_override of the sl slave + * always test family data existence before using this macro + */ + #define SLAVE_CONV_TIME_OVERRIDE(sl) \ + (((struct w1_therm_family_data *)(sl->family_data))->conv_time_override) + +/* + * return the features of the sl slave + * always test family data existence before using this macro + */ + #define SLAVE_FEATURES(sl) \ + (((struct w1_therm_family_data *)(sl->family_data))->features) + +/* * return whether or not a converT command has been issued to the slave * * 0: no bulk read is pending * * -1: conversion is in progress @@ -136,6 +177,8 @@ struct w1_therm_family_converter { * -x error or undefined * @resolution: current device resolution * @convert_triggered: conversion state of the device + * @conv_time_override: user selected conversion time or CONV_TIME_DEFAULT + * @features: bit mask - enable temperature validity check, poll for completion * @specific_functions: pointer to struct of device specific function */ struct w1_therm_family_data { @@ -144,6 +187,8 @@ struct w1_therm_family_data { int external_powered; int resolution; int convert_triggered; + int conv_time_override; + unsigned int features; struct w1_therm_family_converter *specific_functions; }; @@ -285,6 +330,19 @@ static ssize_t therm_bulk_read_store(struct device *device, static ssize_t therm_bulk_read_show(struct device *device, struct device_attribute *attr, char *buf); +static ssize_t conv_time_show(struct device *device, + struct device_attribute *attr, char *buf); + +static ssize_t conv_time_store(struct device *device, + struct device_attribute *attr, const char *buf, + size_t size); + +static ssize_t features_show(struct device *device, + struct device_attribute *attr, char *buf); + +static ssize_t features_store(struct device *device, + struct device_attribute *attr, const char *buf, + size_t size); /* Attributes declarations */ static DEVICE_ATTR_RW(w1_slave); @@ -294,6 +352,8 @@ static DEVICE_ATTR_RO(ext_power); static DEVICE_ATTR_RW(resolution); static DEVICE_ATTR_WO(eeprom); static DEVICE_ATTR_RW(alarms); +static DEVICE_ATTR_RW(conv_time); +static DEVICE_ATTR_RW(features); static DEVICE_ATTR_RW(therm_bulk_read); /* attribut at master level */ @@ -328,6 +388,8 @@ static struct attribute *w1_therm_attrs[] = { &dev_attr_resolution.attr, &dev_attr_eeprom.attr, &dev_attr_alarms.attr, + &dev_attr_conv_time.attr, + &dev_attr_features.attr, NULL, }; @@ -337,6 +399,8 @@ static struct attribute *w1_ds18s20_attrs[] = { &dev_attr_ext_power.attr, &dev_attr_eeprom.attr, &dev_attr_alarms.attr, + &dev_attr_conv_time.attr, + &dev_attr_features.attr, NULL, }; @@ -348,6 +412,8 @@ static struct attribute *w1_ds28ea00_attrs[] = { &dev_attr_resolution.attr, &dev_attr_eeprom.attr, &dev_attr_alarms.attr, + &dev_attr_conv_time.attr, + &dev_attr_features.attr, NULL, }; @@ -409,21 +475,21 @@ static const struct hwmon_chip_info w1_chip_info = { /* Family operations */ -static struct w1_family_ops w1_therm_fops = { +static const struct w1_family_ops w1_therm_fops = { .add_slave = w1_therm_add_slave, .remove_slave = w1_therm_remove_slave, .groups = w1_therm_groups, .chip_info = W1_CHIPINFO, }; -static struct w1_family_ops w1_ds18s20_fops = { +static const struct w1_family_ops w1_ds18s20_fops = { .add_slave = w1_therm_add_slave, .remove_slave = w1_therm_remove_slave, .groups = w1_ds18s20_groups, .chip_info = W1_CHIPINFO, }; -static struct w1_family_ops w1_ds28ea00_fops = { +static const struct w1_family_ops w1_ds28ea00_fops = { .add_slave = w1_therm_add_slave, .remove_slave = w1_therm_remove_slave, .groups = w1_ds28ea00_groups, @@ -466,7 +532,12 @@ static inline int w1_DS18B20_convert_time(struct w1_slave *sl) if (!sl->family_data) return -ENODEV; /* device unknown */ - /* return time in ms for conversion operation */ + if (SLAVE_CONV_TIME_OVERRIDE(sl) != CONV_TIME_DEFAULT) + return SLAVE_CONV_TIME_OVERRIDE(sl); + + /* Return the conversion time, depending on resolution, + * select maximum conversion time among all compatible devices + */ switch (SLAVE_RESOLUTION(sl)) { case 9: ret = 95; @@ -478,6 +549,14 @@ static inline int w1_DS18B20_convert_time(struct w1_slave *sl) ret = 375; break; case 12: + ret = 750; + break; + case 13: + ret = 850; /* GX20MH01 only. Datasheet says 500ms, but that's not enough. */ + break; + case 14: + ret = 1600; /* GX20MH01 only. Datasheet says 1000ms - not enough */ + break; default: ret = 750; } @@ -486,8 +565,13 @@ static inline int w1_DS18B20_convert_time(struct w1_slave *sl) static inline int w1_DS18S20_convert_time(struct w1_slave *sl) { - (void)(sl); - return 750; /* always 750ms for DS18S20 */ + if (!sl->family_data) + return -ENODEV; /* device unknown */ + + if (SLAVE_CONV_TIME_OVERRIDE(sl) == CONV_TIME_DEFAULT) + return 750; /* default for DS18S20 */ + else + return SLAVE_CONV_TIME_OVERRIDE(sl); } static inline int w1_DS18B20_write_data(struct w1_slave *sl, @@ -506,52 +590,71 @@ static inline int w1_DS18S20_write_data(struct w1_slave *sl, static inline int w1_DS18B20_set_resolution(struct w1_slave *sl, int val) { int ret; - u8 new_config_register[3]; /* array of data to be written */ - struct therm_info info; + struct therm_info info, info2; - /* resolution of DS18B20 is in the range [9..12] bits */ - if (val < 9 || val > 12) + /* DS18B20 resolution is 9 to 12 bits */ + /* GX20MH01 resolution is 9 to 14 bits */ + if (val < W1_THERM_RESOLUTION_MIN || val > W1_THERM_RESOLUTION_MAX) return -EINVAL; - val -= 9; /* soustract 9 the lowest resolution in bit */ - val = (val << 5); /* shift to position bit 5 & bit 6 */ + /* Calc bit value from resolution */ + val = (val - W1_THERM_RESOLUTION_MIN) << W1_THERM_RESOLUTION_SHIFT; /* * Read the scratchpad to change only the required bits * (bit5 & bit 6 from byte 4) */ ret = read_scratchpad(sl, &info); - if (!ret) { - new_config_register[0] = info.rom[2]; - new_config_register[1] = info.rom[3]; - /* config register is byte 4 & mask 0b10011111*/ - new_config_register[2] = (info.rom[4] & 0x9F) | - (u8) val; - } else + + if (ret) return ret; + + info.rom[4] &= ~W1_THERM_RESOLUTION_MASK; + info.rom[4] |= val; + /* Write data in the device RAM */ - ret = w1_DS18B20_write_data(sl, new_config_register); + ret = w1_DS18B20_write_data(sl, info.rom + 2); + if (ret) + return ret; - return ret; + /* Have to read back the resolution to verify an actual value + * GX20MH01 and DS18B20 are indistinguishable by family number, but resolutions differ + * Some DS18B20 clones don't support resolution change + */ + ret = read_scratchpad(sl, &info2); + if (ret) + /* Scratchpad read fail */ + return ret; + + if ((info2.rom[4] & W1_THERM_RESOLUTION_MASK) == (info.rom[4] & W1_THERM_RESOLUTION_MASK)) + return 0; + + /* Resolution verify error */ + return -EIO; } static inline int w1_DS18B20_get_resolution(struct w1_slave *sl) { int ret; - u8 config_register; + int resolution; struct therm_info info; ret = read_scratchpad(sl, &info); - if (!ret) { - config_register = info.rom[4]; /* config register is byte 4 */ - config_register &= 0x60; /* 0b01100000 keep only bit 5 & 6 */ - config_register = (config_register >> 5); /* shift */ - config_register += 9; /* add 9 the lowest resolution in bit */ - ret = (int) config_register; - } - return ret; + if (ret) + return ret; + + resolution = ((info.rom[4] & W1_THERM_RESOLUTION_MASK) >> W1_THERM_RESOLUTION_SHIFT) + + W1_THERM_RESOLUTION_MIN; + /* GX20MH01 has one special case: + * >=14 means 14 bits when getting resolution from bit value. + * Other devices have no more then 12 bits. + */ + if (resolution > W1_THERM_RESOLUTION_MAX) + resolution = W1_THERM_RESOLUTION_MAX; + + return resolution; } /** @@ -564,11 +667,28 @@ static inline int w1_DS18B20_get_resolution(struct w1_slave *sl) */ static inline int w1_DS18B20_convert_temp(u8 rom[9]) { - s16 t = le16_to_cpup((__le16 *)rom); + int t; + u32 bv; + + /* Config register bit R2 = 1 - GX20MH01 in 13 or 14 bit resolution mode */ + if (rom[4] & 0x80) { + /* Signed 16-bit value to unsigned, cpu order */ + bv = le16_to_cpup((__le16 *)rom); + + /* Insert two temperature bits from config register */ + /* Avoid arithmetic shift of signed value */ + bv = (bv << 2) | (rom[4] & 3); + + t = (int) sign_extend32(bv, 17); /* Degrees, lowest bit is 2^-6 */ + return (t*1000)/64; /* Millidegrees */ + } + t = (int)le16_to_cpup((__le16 *)rom); return t*1000/16; } + + /** * w1_DS18S20_convert_temp() - temperature computation for DS18S20 * @rom: data read from device RAM (8 data bytes + 1 CRC byte) @@ -600,6 +720,7 @@ static inline int w1_DS18S20_convert_temp(u8 rom[9]) } /* Device capability description */ +/* GX20MH01 device shares family number and structure with DS18B20 */ static struct w1_therm_family_converter w1_therm_families[] = { { @@ -621,6 +742,7 @@ static struct w1_therm_family_converter w1_therm_families[] = { .bulk_read = true }, { + /* Also used for GX20MH01 */ .f = &w1_therm_family_DS18B20, .convert = w1_DS18B20_convert_temp, .get_conversion_time = w1_DS18B20_convert_time, @@ -700,6 +822,22 @@ static inline bool bus_mutex_lock(struct mutex *lock) } /** + * check_family_data() - Check if family data and specific functions are present + * @sl: W1 device data + * + * Return: 0 - OK, negative value - error + */ +static int check_family_data(struct w1_slave *sl) +{ + if ((!sl->family_data) || (!SLAVE_SPECIFIC_FUNC(sl))) { + dev_info(&sl->dev, + "%s: Device is not supported by the driver\n", __func__); + return -EINVAL; /* No device family */ + } + return 0; +} + +/** * support_bulk_read() - check if slave support bulk read * @sl: device to check the ability * @@ -883,6 +1021,34 @@ static int reset_select_slave(struct w1_slave *sl) return 0; } +/** + * w1_poll_completion - Poll for operation completion, with timeout + * @dev_master: the device master of the bus + * @tout_ms: timeout in milliseconds + * + * The device is answering 0's while an operation is in progress and 1's after it completes + * Timeout may happen if the previous command was not recognised due to a line noise + * + * Return: 0 - OK, negative error - timeout + */ +static int w1_poll_completion(struct w1_master *dev_master, int tout_ms) +{ + int i; + + for (i = 0; i < tout_ms/W1_POLL_PERIOD; i++) { + /* Delay is before poll, for device to recognize a command */ + msleep(W1_POLL_PERIOD); + + /* Compare all 8 bits to mitigate a noise on the bus */ + if (w1_read_8(dev_master) == 0xFF) + break; + } + if (i == tout_ms/W1_POLL_PERIOD) + return -EIO; + + return 0; +} + static int convert_t(struct w1_slave *sl, struct therm_info *info) { struct w1_master *dev_master = sl->master; @@ -898,6 +1064,13 @@ static int convert_t(struct w1_slave *sl, struct therm_info *info) (!SLAVE_POWERMODE(sl) && w1_strong_pullup)); + if (strong_pullup && SLAVE_FEATURES(sl) & W1_THERM_POLL_COMPLETION) { + dev_warn(&sl->dev, + "%s: Disabling W1_THERM_POLL_COMPLETION in parasite power mode.\n", + __func__); + SLAVE_FEATURES(sl) &= ~W1_THERM_POLL_COMPLETION; + } + /* get conversion duration device and id dependent */ t_conv = conversion_time(sl); @@ -933,15 +1106,38 @@ static int convert_t(struct w1_slave *sl, struct therm_info *info) } mutex_unlock(&dev_master->bus_mutex); } else { /*no device need pullup */ - mutex_unlock(&dev_master->bus_mutex); - - sleep_rem = msleep_interruptible(t_conv); - if (sleep_rem != 0) { - ret = -EINTR; - goto dec_refcnt; + if (SLAVE_FEATURES(sl) & W1_THERM_POLL_COMPLETION) { + ret = w1_poll_completion(dev_master, W1_POLL_CONVERT_TEMP); + if (ret) { + dev_dbg(&sl->dev, "%s: Timeout\n", __func__); + goto mt_unlock; + } + mutex_unlock(&dev_master->bus_mutex); + } else { + /* Fixed delay */ + mutex_unlock(&dev_master->bus_mutex); + sleep_rem = msleep_interruptible(t_conv); + if (sleep_rem != 0) { + ret = -EINTR; + goto dec_refcnt; + } } } ret = read_scratchpad(sl, info); + + /* If enabled, check for conversion success */ + if ((SLAVE_FEATURES(sl) & W1_THERM_CHECK_RESULT) && + (info->rom[6] == 0xC) && + ((info->rom[1] == 0x5 && info->rom[0] == 0x50) || + (info->rom[1] == 0x7 && info->rom[0] == 0xFF)) + ) { + /* Invalid reading (scratchpad byte 6 = 0xC) + * due to insufficient conversion time + * or power failure. + */ + ret = -EIO; + } + goto dec_refcnt; } @@ -955,6 +1151,76 @@ error: return ret; } +static int conv_time_measure(struct w1_slave *sl, int *conv_time) +{ + struct therm_info inf, + *info = &inf; + struct w1_master *dev_master = sl->master; + int max_trying = W1_THERM_MAX_TRY; + int ret = -ENODEV; + bool strong_pullup; + + if (!sl->family_data) + goto error; + + strong_pullup = (w1_strong_pullup == 2 || + (!SLAVE_POWERMODE(sl) && + w1_strong_pullup)); + + if (strong_pullup) { + pr_info("%s: Measure with strong_pullup is not supported.\n", __func__); + return -EINVAL; + } + + memset(info->rom, 0, sizeof(info->rom)); + + /* prevent the slave from going away in sleep */ + atomic_inc(THERM_REFCNT(sl->family_data)); + + if (!bus_mutex_lock(&dev_master->bus_mutex)) { + ret = -EAGAIN; /* Didn't acquire the mutex */ + goto dec_refcnt; + } + + while (max_trying-- && ret) { /* ret should be 0 */ + info->verdict = 0; + info->crc = 0; + /* safe version to select slave */ + if (!reset_select_slave(sl)) { + int j_start, j_end; + + /*no device need pullup */ + w1_write_8(dev_master, W1_CONVERT_TEMP); + + j_start = jiffies; + ret = w1_poll_completion(dev_master, W1_POLL_CONVERT_TEMP); + if (ret) { + dev_dbg(&sl->dev, "%s: Timeout\n", __func__); + goto mt_unlock; + } + j_end = jiffies; + /* 1.2x increase for variation and changes over temperature range */ + *conv_time = jiffies_to_msecs(j_end-j_start)*12/10; + pr_debug("W1 Measure complete, conv_time = %d, HZ=%d.\n", + *conv_time, HZ); + if (*conv_time <= CONV_TIME_MEASURE) { + ret = -EIO; + goto mt_unlock; + } + mutex_unlock(&dev_master->bus_mutex); + ret = read_scratchpad(sl, info); + goto dec_refcnt; + } + + } +mt_unlock: + mutex_unlock(&dev_master->bus_mutex); +dec_refcnt: + atomic_dec(THERM_REFCNT(sl->family_data)); +error: + return ret; +} + static int read_scratchpad(struct w1_slave *sl, struct therm_info *info) { struct w1_master *dev_master = sl->master; @@ -1118,10 +1384,7 @@ static int recall_eeprom(struct w1_slave *sl) if (!reset_select_slave(sl)) { w1_write_8(dev_master, W1_RECALL_EEPROM); - - ret = 1; /* Slave will pull line to 0 */ - while (ret) - ret = 1 - w1_touch_bit(dev_master, 1); + ret = w1_poll_completion(dev_master, W1_POLL_RECALL_EEPROM); } } @@ -1345,11 +1608,13 @@ static ssize_t w1_slave_store(struct device *device, } if (ret) { - dev_info(device, - "%s: writing error %d\n", __func__, ret); - /* return size to avoid call back again */ - } else - SLAVE_RESOLUTION(sl) = val; + dev_warn(device, "%s: Set resolution - error %d\n", __func__, ret); + /* Propagate error to userspace */ + return ret; + } + SLAVE_RESOLUTION(sl) = val; + /* Reset the conversion time to default - it depends on resolution */ + SLAVE_CONV_TIME_OVERRIDE(sl) = CONV_TIME_DEFAULT; return size; /* always return size to avoid infinite calling */ } @@ -1465,12 +1730,12 @@ static ssize_t resolution_store(struct device *device, /* get the correct function depending on the device */ ret = SLAVE_SPECIFIC_FUNC(sl)->set_resolution(sl, val); - if (ret) { - dev_info(device, - "%s: writing error %d\n", __func__, ret); - /* return size to avoid call back again */ - } else - SLAVE_RESOLUTION(sl) = val; + if (ret) + return ret; + + SLAVE_RESOLUTION(sl) = val; + /* Reset the conversion time to default because it depends on resolution */ + SLAVE_CONV_TIME_OVERRIDE(sl) = CONV_TIME_DEFAULT; return size; } @@ -1660,6 +1925,96 @@ show_result: return sprintf(buf, "%d\n", ret); } +static ssize_t conv_time_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + struct w1_slave *sl = dev_to_w1_slave(device); + + if ((!sl->family_data) || (!SLAVE_SPECIFIC_FUNC(sl))) { + dev_info(device, + "%s: Device is not supported by the driver\n", __func__); + return 0; /* No device family */ + } + return sprintf(buf, "%d\n", conversion_time(sl)); +} + +static ssize_t conv_time_store(struct device *device, + struct device_attribute *attr, const char *buf, size_t size) +{ + int val, ret = 0; + struct w1_slave *sl = dev_to_w1_slave(device); + + if (kstrtoint(buf, 10, &val)) /* converting user entry to int */ + return -EINVAL; + + if (check_family_data(sl)) + return -ENODEV; + + if (val != CONV_TIME_MEASURE) { + if (val >= CONV_TIME_DEFAULT) + SLAVE_CONV_TIME_OVERRIDE(sl) = val; + else + return -EINVAL; + + } else { + int conv_time; + + ret = conv_time_measure(sl, &conv_time); + if (ret) + return -EIO; + SLAVE_CONV_TIME_OVERRIDE(sl) = conv_time; + } + return size; +} + +static ssize_t features_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + struct w1_slave *sl = dev_to_w1_slave(device); + + if ((!sl->family_data) || (!SLAVE_SPECIFIC_FUNC(sl))) { + dev_info(device, + "%s: Device not supported by the driver\n", __func__); + return 0; /* No device family */ + } + return sprintf(buf, "%u\n", SLAVE_FEATURES(sl)); +} + +static ssize_t features_store(struct device *device, + struct device_attribute *attr, const char *buf, size_t size) +{ + int val, ret = 0; + bool strong_pullup; + struct w1_slave *sl = dev_to_w1_slave(device); + + ret = kstrtouint(buf, 10, &val); /* converting user entry to int */ + if (ret) + return -EINVAL; /* invalid number */ + + if ((!sl->family_data) || (!SLAVE_SPECIFIC_FUNC(sl))) { + dev_info(device, "%s: Device not supported by the driver\n", __func__); + return -ENODEV; + } + + if ((val & W1_THERM_FEATURES_MASK) != val) + return -EINVAL; + + SLAVE_FEATURES(sl) = val; + + strong_pullup = (w1_strong_pullup == 2 || + (!SLAVE_POWERMODE(sl) && + w1_strong_pullup)); + + if (strong_pullup && SLAVE_FEATURES(sl) & W1_THERM_POLL_COMPLETION) { + dev_warn(&sl->dev, + "%s: W1_THERM_POLL_COMPLETION disabled in parasite power mode.\n", + __func__); + SLAVE_FEATURES(sl) &= ~W1_THERM_POLL_COMPLETION; + } + + return size; +} + #if IS_REACHABLE(CONFIG_HWMON) static int w1_read_temp(struct device *device, u32 attr, int channel, long *val) diff --git a/drivers/w1/w1.c b/drivers/w1/w1.c index e58c7592008d..15a2ee32f116 100644 --- a/drivers/w1/w1.c +++ b/drivers/w1/w1.c @@ -160,7 +160,7 @@ static const struct attribute_group *w1_slave_default_groups[] = { NULL, }; -static struct w1_family_ops w1_default_fops = { +static const struct w1_family_ops w1_default_fops = { .groups = w1_slave_default_groups, }; @@ -613,7 +613,7 @@ end: static int w1_family_notify(unsigned long action, struct w1_slave *sl) { - struct w1_family_ops *fops; + const struct w1_family_ops *fops; int err; fops = sl->family->fops; diff --git a/fs/Makefile b/fs/Makefile index d72ee2ce7af0..7bb2a05fda1f 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -13,7 +13,8 @@ obj-y := open.o read_write.o file_table.o super.o \ seq_file.o xattr.o libfs.o fs-writeback.o \ pnode.o splice.o sync.o utimes.o d_path.o \ stack.o fs_struct.o statfs.o fs_pin.o nsfs.o \ - fs_types.o fs_context.o fs_parser.o fsopen.o init.o + fs_types.o fs_context.o fs_parser.o fsopen.o init.o \ + kernel_read_file.o ifeq ($(CONFIG_BLOCK),y) obj-y += buffer.o block_dev.o direct-io.o mpage.o diff --git a/fs/exec.c b/fs/exec.c index 07910f5032e7..3f83e94bb0ba 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -23,6 +23,7 @@ * formats. */ +#include <linux/kernel_read_file.h> #include <linux/slab.h> #include <linux/file.h> #include <linux/fdtable.h> @@ -950,137 +951,6 @@ struct file *open_exec(const char *name) } EXPORT_SYMBOL(open_exec); -int kernel_read_file(struct file *file, void **buf, loff_t *size, - loff_t max_size, enum kernel_read_file_id id) -{ - loff_t i_size, pos; - ssize_t bytes = 0; - int ret; - - if (!S_ISREG(file_inode(file)->i_mode) || max_size < 0) - return -EINVAL; - - ret = deny_write_access(file); - if (ret) - return ret; - - ret = security_kernel_read_file(file, id); - if (ret) - goto out; - - i_size = i_size_read(file_inode(file)); - if (i_size <= 0) { - ret = -EINVAL; - goto out; - } - if (i_size > SIZE_MAX || (max_size > 0 && i_size > max_size)) { - ret = -EFBIG; - goto out; - } - - if (id != READING_FIRMWARE_PREALLOC_BUFFER) - *buf = vmalloc(i_size); - if (!*buf) { - ret = -ENOMEM; - goto out; - } - - pos = 0; - while (pos < i_size) { - bytes = kernel_read(file, *buf + pos, i_size - pos, &pos); - if (bytes < 0) { - ret = bytes; - goto out_free; - } - - if (bytes == 0) - break; - } - - if (pos != i_size) { - ret = -EIO; - goto out_free; - } - - ret = security_kernel_post_read_file(file, *buf, i_size, id); - if (!ret) - *size = pos; - -out_free: - if (ret < 0) { - if (id != READING_FIRMWARE_PREALLOC_BUFFER) { - vfree(*buf); - *buf = NULL; - } - } - -out: - allow_write_access(file); - return ret; -} -EXPORT_SYMBOL_GPL(kernel_read_file); - -int kernel_read_file_from_path(const char *path, void **buf, loff_t *size, - loff_t max_size, enum kernel_read_file_id id) -{ - struct file *file; - int ret; - - if (!path || !*path) - return -EINVAL; - - file = filp_open(path, O_RDONLY, 0); - if (IS_ERR(file)) - return PTR_ERR(file); - - ret = kernel_read_file(file, buf, size, max_size, id); - fput(file); - return ret; -} -EXPORT_SYMBOL_GPL(kernel_read_file_from_path); - -int kernel_read_file_from_path_initns(const char *path, void **buf, - loff_t *size, loff_t max_size, - enum kernel_read_file_id id) -{ - struct file *file; - struct path root; - int ret; - - if (!path || !*path) - return -EINVAL; - - task_lock(&init_task); - get_fs_root(init_task.fs, &root); - task_unlock(&init_task); - - file = file_open_root(root.dentry, root.mnt, path, O_RDONLY, 0); - path_put(&root); - if (IS_ERR(file)) - return PTR_ERR(file); - - ret = kernel_read_file(file, buf, size, max_size, id); - fput(file); - return ret; -} -EXPORT_SYMBOL_GPL(kernel_read_file_from_path_initns); - -int kernel_read_file_from_fd(int fd, void **buf, loff_t *size, loff_t max_size, - enum kernel_read_file_id id) -{ - struct fd f = fdget(fd); - int ret = -EBADF; - - if (!f.file) - goto out; - - ret = kernel_read_file(f.file, buf, size, max_size, id); -out: - fdput(f); - return ret; -} -EXPORT_SYMBOL_GPL(kernel_read_file_from_fd); - #if defined(CONFIG_HAVE_AOUT) || defined(CONFIG_BINFMT_FLAT) || \ defined(CONFIG_BINFMT_ELF_FDPIC) ssize_t read_code(struct file *file, unsigned long addr, loff_t pos, size_t len) diff --git a/fs/kernel_read_file.c b/fs/kernel_read_file.c new file mode 100644 index 000000000000..90d255fbdd9b --- /dev/null +++ b/fs/kernel_read_file.c @@ -0,0 +1,189 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/fs.h> +#include <linux/fs_struct.h> +#include <linux/kernel_read_file.h> +#include <linux/security.h> +#include <linux/vmalloc.h> + +/** + * kernel_read_file() - read file contents into a kernel buffer + * + * @file file to read from + * @offset where to start reading from (see below). + * @buf pointer to a "void *" buffer for reading into (if + * *@buf is NULL, a buffer will be allocated, and + * @buf_size will be ignored) + * @buf_size size of buf, if already allocated. If @buf not + * allocated, this is the largest size to allocate. + * @file_size if non-NULL, the full size of @file will be + * written here. + * @id the kernel_read_file_id identifying the type of + * file contents being read (for LSMs to examine) + * + * @offset must be 0 unless both @buf and @file_size are non-NULL + * (i.e. the caller must be expecting to read partial file contents + * via an already-allocated @buf, in at most @buf_size chunks, and + * will be able to determine when the entire file was read by + * checking @file_size). This isn't a recommended way to read a + * file, though, since it is possible that the contents might + * change between calls to kernel_read_file(). + * + * Returns number of bytes read (no single read will be bigger + * than INT_MAX), or negative on error. + * + */ +int kernel_read_file(struct file *file, loff_t offset, void **buf, + size_t buf_size, size_t *file_size, + enum kernel_read_file_id id) +{ + loff_t i_size, pos; + size_t copied; + void *allocated = NULL; + bool whole_file; + int ret; + + if (offset != 0 && (!*buf || !file_size)) + return -EINVAL; + + if (!S_ISREG(file_inode(file)->i_mode)) + return -EINVAL; + + ret = deny_write_access(file); + if (ret) + return ret; + + i_size = i_size_read(file_inode(file)); + if (i_size <= 0) { + ret = -EINVAL; + goto out; + } + /* The file is too big for sane activities. */ + if (i_size > INT_MAX) { + ret = -EFBIG; + goto out; + } + /* The entire file cannot be read in one buffer. */ + if (!file_size && offset == 0 && i_size > buf_size) { + ret = -EFBIG; + goto out; + } + + whole_file = (offset == 0 && i_size <= buf_size); + ret = security_kernel_read_file(file, id, whole_file); + if (ret) + goto out; + + if (file_size) + *file_size = i_size; + + if (!*buf) + *buf = allocated = vmalloc(i_size); + if (!*buf) { + ret = -ENOMEM; + goto out; + } + + pos = offset; + copied = 0; + while (copied < buf_size) { + ssize_t bytes; + size_t wanted = min_t(size_t, buf_size - copied, + i_size - pos); + + bytes = kernel_read(file, *buf + copied, wanted, &pos); + if (bytes < 0) { + ret = bytes; + goto out_free; + } + + if (bytes == 0) + break; + copied += bytes; + } + + if (whole_file) { + if (pos != i_size) { + ret = -EIO; + goto out_free; + } + + ret = security_kernel_post_read_file(file, *buf, i_size, id); + } + +out_free: + if (ret < 0) { + if (allocated) { + vfree(*buf); + *buf = NULL; + } + } + +out: + allow_write_access(file); + return ret == 0 ? copied : ret; +} +EXPORT_SYMBOL_GPL(kernel_read_file); + +int kernel_read_file_from_path(const char *path, loff_t offset, void **buf, + size_t buf_size, size_t *file_size, + enum kernel_read_file_id id) +{ + struct file *file; + int ret; + + if (!path || !*path) + return -EINVAL; + + file = filp_open(path, O_RDONLY, 0); + if (IS_ERR(file)) + return PTR_ERR(file); + + ret = kernel_read_file(file, offset, buf, buf_size, file_size, id); + fput(file); + return ret; +} +EXPORT_SYMBOL_GPL(kernel_read_file_from_path); + +int kernel_read_file_from_path_initns(const char *path, loff_t offset, + void **buf, size_t buf_size, + size_t *file_size, + enum kernel_read_file_id id) +{ + struct file *file; + struct path root; + int ret; + + if (!path || !*path) + return -EINVAL; + + task_lock(&init_task); + get_fs_root(init_task.fs, &root); + task_unlock(&init_task); + + file = file_open_root(root.dentry, root.mnt, path, O_RDONLY, 0); + path_put(&root); + if (IS_ERR(file)) + return PTR_ERR(file); + + ret = kernel_read_file(file, offset, buf, buf_size, file_size, id); + fput(file); + return ret; +} +EXPORT_SYMBOL_GPL(kernel_read_file_from_path_initns); + +int kernel_read_file_from_fd(int fd, loff_t offset, void **buf, + size_t buf_size, size_t *file_size, + enum kernel_read_file_id id) +{ + struct fd f = fdget(fd); + int ret = -EBADF; + + if (!f.file) + goto out; + + ret = kernel_read_file(f.file, offset, buf, buf_size, file_size, id); +out: + fdput(f); + return ret; +} +EXPORT_SYMBOL_GPL(kernel_read_file_from_fd); diff --git a/include/dt-bindings/interconnect/qcom,icc.h b/include/dt-bindings/interconnect/qcom,icc.h new file mode 100644 index 000000000000..cd34f36daaaa --- /dev/null +++ b/include/dt-bindings/interconnect/qcom,icc.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + */ + +#ifndef __DT_BINDINGS_INTERCONNECT_QCOM_ICC_H +#define __DT_BINDINGS_INTERCONNECT_QCOM_ICC_H + +/* + * The AMC bucket denotes constraints that are applied to hardware when + * icc_set_bw() completes, whereas the WAKE and SLEEP constraints are applied + * when the execution environment transitions between active and low power mode. + */ +#define QCOM_ICC_BUCKET_AMC 0 +#define QCOM_ICC_BUCKET_WAKE 1 +#define QCOM_ICC_BUCKET_SLEEP 2 +#define QCOM_ICC_NUM_BUCKETS 3 + +#define QCOM_ICC_TAG_AMC (1 << QCOM_ICC_BUCKET_AMC) +#define QCOM_ICC_TAG_WAKE (1 << QCOM_ICC_BUCKET_WAKE) +#define QCOM_ICC_TAG_SLEEP (1 << QCOM_ICC_BUCKET_SLEEP) +#define QCOM_ICC_TAG_ACTIVE_ONLY (QCOM_ICC_TAG_AMC | QCOM_ICC_TAG_WAKE) +#define QCOM_ICC_TAG_ALWAYS (QCOM_ICC_TAG_AMC | QCOM_ICC_TAG_WAKE |\ + QCOM_ICC_TAG_SLEEP) + +#endif diff --git a/include/dt-bindings/interconnect/qcom,osm-l3.h b/include/dt-bindings/interconnect/qcom,osm-l3.h index 54858ff7674d..61ef649ae565 100644 --- a/include/dt-bindings/interconnect/qcom,osm-l3.h +++ b/include/dt-bindings/interconnect/qcom,osm-l3.h @@ -9,4 +9,7 @@ #define MASTER_OSM_L3_APPS 0 #define SLAVE_OSM_L3 1 +#define MASTER_EPSS_L3_APPS 0 +#define SLAVE_EPSS_L3_SHARED 1 + #endif diff --git a/include/dt-bindings/interconnect/qcom,sm8150.h b/include/dt-bindings/interconnect/qcom,sm8150.h new file mode 100644 index 000000000000..a25684680c42 --- /dev/null +++ b/include/dt-bindings/interconnect/qcom,sm8150.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Qualcomm SM8150 interconnect IDs + * + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + */ + +#ifndef __DT_BINDINGS_INTERCONNECT_QCOM_SM8150_H +#define __DT_BINDINGS_INTERCONNECT_QCOM_SM8150_H + +#define MASTER_A1NOC_CFG 0 +#define MASTER_QUP_0 1 +#define MASTER_EMAC 2 +#define MASTER_UFS_MEM 3 +#define MASTER_USB3 4 +#define MASTER_USB3_1 5 +#define A1NOC_SNOC_SLV 6 +#define SLAVE_SERVICE_A1NOC 7 + +#define MASTER_A2NOC_CFG 0 +#define MASTER_QDSS_BAM 1 +#define MASTER_QSPI 2 +#define MASTER_QUP_1 3 +#define MASTER_QUP_2 4 +#define MASTER_SENSORS_AHB 5 +#define MASTER_TSIF 6 +#define MASTER_CNOC_A2NOC 7 +#define MASTER_CRYPTO_CORE_0 8 +#define MASTER_IPA 9 +#define MASTER_PCIE 10 +#define MASTER_PCIE_1 11 +#define MASTER_QDSS_ETR 12 +#define MASTER_SDCC_2 13 +#define MASTER_SDCC_4 14 +#define A2NOC_SNOC_SLV 15 +#define SLAVE_ANOC_PCIE_GEM_NOC 16 +#define SLAVE_SERVICE_A2NOC 17 + +#define MASTER_CAMNOC_HF0_UNCOMP 0 +#define MASTER_CAMNOC_HF1_UNCOMP 1 +#define MASTER_CAMNOC_SF_UNCOMP 2 +#define SLAVE_CAMNOC_UNCOMP 3 + +#define MASTER_NPU 0 +#define SLAVE_CDSP_MEM_NOC 1 + +#define MASTER_SPDM 0 +#define SNOC_CNOC_MAS 1 +#define MASTER_QDSS_DAP 2 +#define SLAVE_A1NOC_CFG 3 +#define SLAVE_A2NOC_CFG 4 +#define SLAVE_AHB2PHY_SOUTH 5 +#define SLAVE_AOP 6 +#define SLAVE_AOSS 7 +#define SLAVE_CAMERA_CFG 8 +#define SLAVE_CLK_CTL 9 +#define SLAVE_CDSP_CFG 10 +#define SLAVE_RBCPR_CX_CFG 11 +#define SLAVE_RBCPR_MMCX_CFG 12 +#define SLAVE_RBCPR_MX_CFG 13 +#define SLAVE_CRYPTO_0_CFG 14 +#define SLAVE_CNOC_DDRSS 15 +#define SLAVE_DISPLAY_CFG 16 +#define SLAVE_EMAC_CFG 17 +#define SLAVE_GLM 18 +#define SLAVE_GRAPHICS_3D_CFG 19 +#define SLAVE_IMEM_CFG 20 +#define SLAVE_IPA_CFG 21 +#define SLAVE_CNOC_MNOC_CFG 22 +#define SLAVE_NPU_CFG 23 +#define SLAVE_PCIE_0_CFG 24 +#define SLAVE_PCIE_1_CFG 25 +#define SLAVE_NORTH_PHY_CFG 26 +#define SLAVE_PIMEM_CFG 27 +#define SLAVE_PRNG 28 +#define SLAVE_QDSS_CFG 29 +#define SLAVE_QSPI 30 +#define SLAVE_QUP_2 31 +#define SLAVE_QUP_1 32 +#define SLAVE_QUP_0 33 +#define SLAVE_SDCC_2 34 +#define SLAVE_SDCC_4 35 +#define SLAVE_SNOC_CFG 36 +#define SLAVE_SPDM_WRAPPER 37 +#define SLAVE_SPSS_CFG 38 +#define SLAVE_SSC_CFG 39 +#define SLAVE_TCSR 40 +#define SLAVE_TLMM_EAST 41 +#define SLAVE_TLMM_NORTH 42 +#define SLAVE_TLMM_SOUTH 43 +#define SLAVE_TLMM_WEST 44 +#define SLAVE_TSIF 45 +#define SLAVE_UFS_CARD_CFG 46 +#define SLAVE_UFS_MEM_CFG 47 +#define SLAVE_USB3 48 +#define SLAVE_USB3_1 49 +#define SLAVE_VENUS_CFG 50 +#define SLAVE_VSENSE_CTRL_CFG 51 +#define SLAVE_CNOC_A2NOC 52 +#define SLAVE_SERVICE_CNOC 53 + +#define MASTER_CNOC_DC_NOC 0 +#define SLAVE_LLCC_CFG 1 +#define SLAVE_GEM_NOC_CFG 2 + +#define MASTER_AMPSS_M0 0 +#define MASTER_GPU_TCU 1 +#define MASTER_SYS_TCU 2 +#define MASTER_GEM_NOC_CFG 3 +#define MASTER_COMPUTE_NOC 4 +#define MASTER_GRAPHICS_3D 5 +#define MASTER_MNOC_HF_MEM_NOC 6 +#define MASTER_MNOC_SF_MEM_NOC 7 +#define MASTER_GEM_NOC_PCIE_SNOC 8 +#define MASTER_SNOC_GC_MEM_NOC 9 +#define MASTER_SNOC_SF_MEM_NOC 10 +#define MASTER_ECC 11 +#define SLAVE_MSS_PROC_MS_MPU_CFG 12 +#define SLAVE_ECC 13 +#define SLAVE_GEM_NOC_SNOC 14 +#define SLAVE_LLCC 15 +#define SLAVE_SERVICE_GEM_NOC 16 + +#define MASTER_IPA_CORE 0 +#define SLAVE_IPA_CORE 1 + +#define MASTER_LLCC 0 +#define SLAVE_EBI_CH0 1 + +#define MASTER_CNOC_MNOC_CFG 0 +#define MASTER_CAMNOC_HF0 1 +#define MASTER_CAMNOC_HF1 2 +#define MASTER_CAMNOC_SF 3 +#define MASTER_MDP_PORT0 4 +#define MASTER_MDP_PORT1 5 +#define MASTER_ROTATOR 6 +#define MASTER_VIDEO_P0 7 +#define MASTER_VIDEO_P1 8 +#define MASTER_VIDEO_PROC 9 +#define SLAVE_MNOC_SF_MEM_NOC 10 +#define SLAVE_MNOC_HF_MEM_NOC 11 +#define SLAVE_SERVICE_MNOC 12 + +#define MASTER_SNOC_CFG 0 +#define A1NOC_SNOC_MAS 1 +#define A2NOC_SNOC_MAS 2 +#define MASTER_GEM_NOC_SNOC 3 +#define MASTER_PIMEM 4 +#define MASTER_GIC 5 +#define SLAVE_APPSS 6 +#define SNOC_CNOC_SLV 7 +#define SLAVE_SNOC_GEM_NOC_GC 8 +#define SLAVE_SNOC_GEM_NOC_SF 9 +#define SLAVE_OCIMEM 10 +#define SLAVE_PIMEM 11 +#define SLAVE_SERVICE_SNOC 12 +#define SLAVE_PCIE_0 13 +#define SLAVE_PCIE_1 14 +#define SLAVE_QDSS_STM 15 +#define SLAVE_TCU 16 + +#endif diff --git a/include/dt-bindings/interconnect/qcom,sm8250.h b/include/dt-bindings/interconnect/qcom,sm8250.h new file mode 100644 index 000000000000..1b4d9fbe888d --- /dev/null +++ b/include/dt-bindings/interconnect/qcom,sm8250.h @@ -0,0 +1,172 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Qualcomm SM8250 interconnect IDs + * + * Copyright (c) 2020, The Linux Foundation. All rights reserved. + */ + +#ifndef __DT_BINDINGS_INTERCONNECT_QCOM_SM8250_H +#define __DT_BINDINGS_INTERCONNECT_QCOM_SM8250_H + +#define MASTER_A1NOC_CFG 0 +#define MASTER_QSPI_0 1 +#define MASTER_QUP_1 2 +#define MASTER_QUP_2 3 +#define MASTER_TSIF 4 +#define MASTER_PCIE_2 5 +#define MASTER_SDCC_4 6 +#define MASTER_UFS_MEM 7 +#define MASTER_USB3 8 +#define MASTER_USB3_1 9 +#define A1NOC_SNOC_SLV 10 +#define SLAVE_ANOC_PCIE_GEM_NOC_1 11 +#define SLAVE_SERVICE_A1NOC 12 + +#define MASTER_A2NOC_CFG 0 +#define MASTER_QDSS_BAM 1 +#define MASTER_QUP_0 2 +#define MASTER_CNOC_A2NOC 3 +#define MASTER_CRYPTO_CORE_0 4 +#define MASTER_IPA 5 +#define MASTER_PCIE 6 +#define MASTER_PCIE_1 7 +#define MASTER_QDSS_ETR 8 +#define MASTER_SDCC_2 9 +#define MASTER_UFS_CARD 10 +#define A2NOC_SNOC_SLV 11 +#define SLAVE_ANOC_PCIE_GEM_NOC 12 +#define SLAVE_SERVICE_A2NOC 13 + +#define MASTER_NPU 0 +#define SLAVE_CDSP_MEM_NOC 1 + +#define SNOC_CNOC_MAS 0 +#define MASTER_QDSS_DAP 1 +#define SLAVE_A1NOC_CFG 2 +#define SLAVE_A2NOC_CFG 3 +#define SLAVE_AHB2PHY_SOUTH 4 +#define SLAVE_AHB2PHY_NORTH 5 +#define SLAVE_AOSS 6 +#define SLAVE_CAMERA_CFG 7 +#define SLAVE_CLK_CTL 8 +#define SLAVE_CDSP_CFG 9 +#define SLAVE_RBCPR_CX_CFG 10 +#define SLAVE_RBCPR_MMCX_CFG 11 +#define SLAVE_RBCPR_MX_CFG 12 +#define SLAVE_CRYPTO_0_CFG 13 +#define SLAVE_CX_RDPM 14 +#define SLAVE_DCC_CFG 15 +#define SLAVE_CNOC_DDRSS 16 +#define SLAVE_DISPLAY_CFG 17 +#define SLAVE_GRAPHICS_3D_CFG 18 +#define SLAVE_IMEM_CFG 19 +#define SLAVE_IPA_CFG 20 +#define SLAVE_IPC_ROUTER_CFG 21 +#define SLAVE_LPASS 22 +#define SLAVE_CNOC_MNOC_CFG 23 +#define SLAVE_NPU_CFG 24 +#define SLAVE_PCIE_0_CFG 25 +#define SLAVE_PCIE_1_CFG 26 +#define SLAVE_PCIE_2_CFG 27 +#define SLAVE_PDM 28 +#define SLAVE_PIMEM_CFG 29 +#define SLAVE_PRNG 30 +#define SLAVE_QDSS_CFG 31 +#define SLAVE_QSPI_0 32 +#define SLAVE_QUP_0 33 +#define SLAVE_QUP_1 34 +#define SLAVE_QUP_2 35 +#define SLAVE_SDCC_2 36 +#define SLAVE_SDCC_4 37 +#define SLAVE_SNOC_CFG 38 +#define SLAVE_TCSR 39 +#define SLAVE_TLMM_NORTH 40 +#define SLAVE_TLMM_SOUTH 41 +#define SLAVE_TLMM_WEST 42 +#define SLAVE_TSIF 43 +#define SLAVE_UFS_CARD_CFG 44 +#define SLAVE_UFS_MEM_CFG 45 +#define SLAVE_USB3 46 +#define SLAVE_USB3_1 47 +#define SLAVE_VENUS_CFG 48 +#define SLAVE_VSENSE_CTRL_CFG 49 +#define SLAVE_CNOC_A2NOC 50 +#define SLAVE_SERVICE_CNOC 51 + +#define MASTER_CNOC_DC_NOC 0 +#define SLAVE_LLCC_CFG 1 +#define SLAVE_GEM_NOC_CFG 2 + +#define MASTER_GPU_TCU 0 +#define MASTER_SYS_TCU 1 +#define MASTER_AMPSS_M0 2 +#define MASTER_GEM_NOC_CFG 3 +#define MASTER_COMPUTE_NOC 4 +#define MASTER_GRAPHICS_3D 5 +#define MASTER_MNOC_HF_MEM_NOC 6 +#define MASTER_MNOC_SF_MEM_NOC 7 +#define MASTER_ANOC_PCIE_GEM_NOC 8 +#define MASTER_SNOC_GC_MEM_NOC 9 +#define MASTER_SNOC_SF_MEM_NOC 10 +#define SLAVE_GEM_NOC_SNOC 11 +#define SLAVE_LLCC 12 +#define SLAVE_MEM_NOC_PCIE_SNOC 13 +#define SLAVE_SERVICE_GEM_NOC_1 14 +#define SLAVE_SERVICE_GEM_NOC_2 15 +#define SLAVE_SERVICE_GEM_NOC 16 + +#define MASTER_IPA_CORE 0 +#define SLAVE_IPA_CORE 1 + +#define MASTER_LLCC 0 +#define SLAVE_EBI_CH0 1 + +#define MASTER_CNOC_MNOC_CFG 0 +#define MASTER_CAMNOC_HF 1 +#define MASTER_CAMNOC_ICP 2 +#define MASTER_CAMNOC_SF 3 +#define MASTER_VIDEO_P0 4 +#define MASTER_VIDEO_P1 5 +#define MASTER_VIDEO_PROC 6 +#define MASTER_MDP_PORT0 7 +#define MASTER_MDP_PORT1 8 +#define MASTER_ROTATOR 9 +#define SLAVE_MNOC_HF_MEM_NOC 10 +#define SLAVE_MNOC_SF_MEM_NOC 11 +#define SLAVE_SERVICE_MNOC 12 + +#define MASTER_NPU_SYS 0 +#define MASTER_NPU_CDP 1 +#define MASTER_NPU_NOC_CFG 2 +#define SLAVE_NPU_CAL_DP0 3 +#define SLAVE_NPU_CAL_DP1 4 +#define SLAVE_NPU_CP 5 +#define SLAVE_NPU_INT_DMA_BWMON_CFG 6 +#define SLAVE_NPU_DPM 7 +#define SLAVE_ISENSE_CFG 8 +#define SLAVE_NPU_LLM_CFG 9 +#define SLAVE_NPU_TCM 10 +#define SLAVE_NPU_COMPUTE_NOC 11 +#define SLAVE_SERVICE_NPU_NOC 12 + +#define MASTER_SNOC_CFG 0 +#define A1NOC_SNOC_MAS 1 +#define A2NOC_SNOC_MAS 2 +#define MASTER_GEM_NOC_SNOC 3 +#define MASTER_GEM_NOC_PCIE_SNOC 4 +#define MASTER_PIMEM 5 +#define MASTER_GIC 6 +#define SLAVE_APPSS 7 +#define SNOC_CNOC_SLV 8 +#define SLAVE_SNOC_GEM_NOC_GC 9 +#define SLAVE_SNOC_GEM_NOC_SF 10 +#define SLAVE_OCIMEM 11 +#define SLAVE_PIMEM 12 +#define SLAVE_SERVICE_SNOC 13 +#define SLAVE_PCIE_0 14 +#define SLAVE_PCIE_1 15 +#define SLAVE_PCIE_2 16 +#define SLAVE_QDSS_STM 17 +#define SLAVE_TCU 18 + +#endif diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 58fffdecdbfd..7d3c87e5b97c 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -208,6 +208,7 @@ struct coresight_device { /* sysfs links between components */ int nr_links; bool has_conns_grp; + bool ect_enabled; /* true only if associated ect device is enabled */ }; /* @@ -324,7 +325,7 @@ struct coresight_ops { const struct coresight_ops_ect *ect_ops; }; -#ifdef CONFIG_CORESIGHT +#if IS_ENABLED(CONFIG_CORESIGHT) extern struct coresight_device * coresight_register(struct coresight_desc *desc); extern void coresight_unregister(struct coresight_device *csdev); diff --git a/include/linux/firmware.h b/include/linux/firmware.h index cb3e2c06ed8a..c15acadc6cf4 100644 --- a/include/linux/firmware.h +++ b/include/linux/firmware.h @@ -53,6 +53,9 @@ int request_firmware_direct(const struct firmware **fw, const char *name, struct device *device); int request_firmware_into_buf(const struct firmware **firmware_p, const char *name, struct device *device, void *buf, size_t size); +int request_partial_firmware_into_buf(const struct firmware **firmware_p, + const char *name, struct device *device, + void *buf, size_t size, size_t offset); void release_firmware(const struct firmware *fw); #else @@ -102,6 +105,15 @@ static inline int request_firmware_into_buf(const struct firmware **firmware_p, return -EINVAL; } +static inline int request_partial_firmware_into_buf + (const struct firmware **firmware_p, + const char *name, + struct device *device, + void *buf, size_t size, size_t offset) +{ + return -EINVAL; +} + #endif int firmware_request_cache(struct device *device, const char *name); diff --git a/include/linux/fs.h b/include/linux/fs.h index 5815f7d4dbf4..ae97d87a00d2 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2852,45 +2852,6 @@ static inline void i_readcount_inc(struct inode *inode) #endif extern int do_pipe_flags(int *, int); -#define __kernel_read_file_id(id) \ - id(UNKNOWN, unknown) \ - id(FIRMWARE, firmware) \ - id(FIRMWARE_PREALLOC_BUFFER, firmware) \ - id(FIRMWARE_EFI_EMBEDDED, firmware) \ - id(MODULE, kernel-module) \ - id(KEXEC_IMAGE, kexec-image) \ - id(KEXEC_INITRAMFS, kexec-initramfs) \ - id(POLICY, security-policy) \ - id(X509_CERTIFICATE, x509-certificate) \ - id(MAX_ID, ) - -#define __fid_enumify(ENUM, dummy) READING_ ## ENUM, -#define __fid_stringify(dummy, str) #str, - -enum kernel_read_file_id { - __kernel_read_file_id(__fid_enumify) -}; - -static const char * const kernel_read_file_str[] = { - __kernel_read_file_id(__fid_stringify) -}; - -static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) -{ - if ((unsigned)id >= READING_MAX_ID) - return kernel_read_file_str[READING_UNKNOWN]; - - return kernel_read_file_str[id]; -} - -extern int kernel_read_file(struct file *, void **, loff_t *, loff_t, - enum kernel_read_file_id); -extern int kernel_read_file_from_path(const char *, void **, loff_t *, loff_t, - enum kernel_read_file_id); -extern int kernel_read_file_from_path_initns(const char *, void **, loff_t *, loff_t, - enum kernel_read_file_id); -extern int kernel_read_file_from_fd(int, void **, loff_t *, loff_t, - enum kernel_read_file_id); extern ssize_t kernel_read(struct file *, void *, size_t, loff_t *); ssize_t __kernel_read(struct file *file, void *buf, size_t count, loff_t *pos); extern ssize_t kernel_write(struct file *, const void *, size_t, loff_t *); diff --git a/include/linux/fsl/mc.h b/include/linux/fsl/mc.h index a428c61ead6e..db244874e834 100644 --- a/include/linux/fsl/mc.h +++ b/include/linux/fsl/mc.h @@ -3,6 +3,7 @@ * Freescale Management Complex (MC) bus public interface * * Copyright (C) 2014-2016 Freescale Semiconductor, Inc. + * Copyright 2019-2020 NXP * Author: German Rivera <German.Rivera@freescale.com> * */ @@ -148,6 +149,13 @@ struct fsl_mc_obj_desc { */ #define FSL_MC_IS_DPRC 0x0001 +/* Region flags */ +/* Indicates that region can be mapped as cacheable */ +#define FSL_MC_REGION_CACHEABLE 0x00000001 + +/* Indicates that region can be mapped as shareable */ +#define FSL_MC_REGION_SHAREABLE 0x00000002 + /** * struct fsl_mc_device - MC object device object * @dev: Linux driver model device object @@ -161,6 +169,7 @@ struct fsl_mc_obj_desc { * @regions: pointer to array of MMIO region entries * @irqs: pointer to array of pointers to interrupts allocated to this device * @resource: generic resource associated with this MC object device, if any. + * @driver_override: driver name to force a match * * Generic device object for MC object devices that are "attached" to a * MC bus. @@ -186,7 +195,7 @@ struct fsl_mc_device { struct device dev; u64 dma_mask; u16 flags; - u16 icid; + u32 icid; u16 mc_handle; struct fsl_mc_io *mc_io; struct fsl_mc_obj_desc obj_desc; @@ -194,6 +203,7 @@ struct fsl_mc_device { struct fsl_mc_device_irq **irqs; struct fsl_mc_resource *resource; struct device_link *consumer_link; + char *driver_override; }; #define to_fsl_mc_device(_dev) \ @@ -514,6 +524,35 @@ static inline bool is_fsl_mc_bus_dpdmai(const struct fsl_mc_device *mc_dev) return mc_dev->dev.type == &fsl_mc_bus_dpdmai_type; } +#define DPRC_RESET_OPTION_NON_RECURSIVE 0x00000001 +int dprc_reset_container(struct fsl_mc_io *mc_io, + u32 cmd_flags, + u16 token, + int child_container_id, + u32 options); + +int dprc_scan_container(struct fsl_mc_device *mc_bus_dev, + bool alloc_interrupts); + +void dprc_remove_devices(struct fsl_mc_device *mc_bus_dev, + struct fsl_mc_obj_desc *obj_desc_array, + int num_child_objects_in_mc); + +int dprc_cleanup(struct fsl_mc_device *mc_dev); + +int dprc_setup(struct fsl_mc_device *mc_dev); + +/** + * Maximum number of total IRQs that can be pre-allocated for an MC bus' + * IRQ pool + */ +#define FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS 256 + +int fsl_mc_populate_irq_pool(struct fsl_mc_device *mc_bus_dev, + unsigned int irq_count); + +void fsl_mc_cleanup_irq_pool(struct fsl_mc_device *mc_bus_dev); + /* * Data Path Buffer Pool (DPBP) API * Contains initialization APIs and runtime control APIs for DPBP diff --git a/include/linux/ima.h b/include/linux/ima.h index d15100de6cdd..8fa7bcfb2da2 100644 --- a/include/linux/ima.h +++ b/include/linux/ima.h @@ -7,6 +7,7 @@ #ifndef _LINUX_IMA_H #define _LINUX_IMA_H +#include <linux/kernel_read_file.h> #include <linux/fs.h> #include <linux/security.h> #include <linux/kexec.h> @@ -19,8 +20,11 @@ extern void ima_post_create_tmpfile(struct inode *inode); extern void ima_file_free(struct file *file); extern int ima_file_mmap(struct file *file, unsigned long prot); extern int ima_file_mprotect(struct vm_area_struct *vma, unsigned long prot); -extern int ima_load_data(enum kernel_load_data_id id); -extern int ima_read_file(struct file *file, enum kernel_read_file_id id); +extern int ima_load_data(enum kernel_load_data_id id, bool contents); +extern int ima_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id id, char *description); +extern int ima_read_file(struct file *file, enum kernel_read_file_id id, + bool contents); extern int ima_post_read_file(struct file *file, void *buf, loff_t size, enum kernel_read_file_id id); extern void ima_post_path_mknod(struct dentry *dentry); @@ -77,12 +81,20 @@ static inline int ima_file_mprotect(struct vm_area_struct *vma, return 0; } -static inline int ima_load_data(enum kernel_load_data_id id) +static inline int ima_load_data(enum kernel_load_data_id id, bool contents) { return 0; } -static inline int ima_read_file(struct file *file, enum kernel_read_file_id id) +static inline int ima_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id id, + char *description) +{ + return 0; +} + +static inline int ima_read_file(struct file *file, enum kernel_read_file_id id, + bool contents) { return 0; } diff --git a/include/linux/interconnect-provider.h b/include/linux/interconnect-provider.h index 4735518de515..6bd01f7159c6 100644 --- a/include/linux/interconnect-provider.h +++ b/include/linux/interconnect-provider.h @@ -15,6 +15,17 @@ struct icc_node; struct of_phandle_args; /** + * struct icc_node_data - icc node data + * + * @node: icc node + * @tag: tag + */ +struct icc_node_data { + struct icc_node *node; + u32 tag; +}; + +/** * struct icc_onecell_data - driver data for onecell interconnect providers * * @num_nodes: number of nodes in this device @@ -38,7 +49,9 @@ struct icc_node *of_icc_xlate_onecell(struct of_phandle_args *spec, * @aggregate: pointer to device specific aggregate operation function * @pre_aggregate: pointer to device specific function that is called * before the aggregation begins (optional) + * @get_bw: pointer to device specific function to get current bandwidth * @xlate: provider-specific callback for mapping nodes from phandle arguments + * @xlate_extended: vendor-specific callback for mapping node data from phandle arguments * @dev: the device this interconnect provider belongs to * @users: count of active users * @inter_set: whether inter-provider pairs will be configured with @set @@ -51,7 +64,9 @@ struct icc_provider { int (*aggregate)(struct icc_node *node, u32 tag, u32 avg_bw, u32 peak_bw, u32 *agg_avg, u32 *agg_peak); void (*pre_aggregate)(struct icc_node *node); + int (*get_bw)(struct icc_node *node, u32 *avg, u32 *peak); struct icc_node* (*xlate)(struct of_phandle_args *spec, void *data); + struct icc_node_data* (*xlate_extended)(struct of_phandle_args *spec, void *data); struct device *dev; int users; bool inter_set; @@ -73,6 +88,8 @@ struct icc_provider { * @req_list: a list of QoS constraint requests associated with this node * @avg_bw: aggregated value of average bandwidth requests from all consumers * @peak_bw: aggregated value of peak bandwidth requests from all consumers + * @init_avg: average bandwidth value that is read from the hardware during init + * @init_peak: peak bandwidth value that is read from the hardware during init * @data: pointer to private data */ struct icc_node { @@ -89,6 +106,8 @@ struct icc_node { struct hlist_head req_list; u32 avg_bw; u32 peak_bw; + u32 init_avg; + u32 init_peak; void *data; }; @@ -105,7 +124,8 @@ void icc_node_del(struct icc_node *node); int icc_nodes_remove(struct icc_provider *provider); int icc_provider_add(struct icc_provider *provider); int icc_provider_del(struct icc_provider *provider); -struct icc_node *of_icc_get_from_provider(struct of_phandle_args *spec); +struct icc_node_data *of_icc_get_from_provider(struct of_phandle_args *spec); +void icc_sync_state(struct device *dev); #else @@ -157,7 +177,7 @@ static inline int icc_provider_del(struct icc_provider *provider) return -ENOTSUPP; } -static inline struct icc_node *of_icc_get_from_provider(struct of_phandle_args *spec) +static inline struct icc_node_data *of_icc_get_from_provider(struct of_phandle_args *spec) { return ERR_PTR(-ENOTSUPP); } diff --git a/include/linux/interconnect.h b/include/linux/interconnect.h index 3a63d98613fc..f2dd2fc8d3cd 100644 --- a/include/linux/interconnect.h +++ b/include/linux/interconnect.h @@ -23,6 +23,28 @@ struct icc_path; struct device; +/** + * struct icc_bulk_data - Data used for bulk icc operations. + * + * @path: reference to the interconnect path (internal use) + * @name: the name from the "interconnect-names" DT property + * @avg_bw: average bandwidth in icc units + * @peak_bw: peak bandwidth in icc units + */ +struct icc_bulk_data { + struct icc_path *path; + const char *name; + u32 avg_bw; + u32 peak_bw; +}; + +int __must_check of_icc_bulk_get(struct device *dev, int num_paths, + struct icc_bulk_data *paths); +void icc_bulk_put(int num_paths, struct icc_bulk_data *paths); +int icc_bulk_set_bw(int num_paths, const struct icc_bulk_data *paths); +int icc_bulk_enable(int num_paths, const struct icc_bulk_data *paths); +void icc_bulk_disable(int num_paths, const struct icc_bulk_data *paths); + #if IS_ENABLED(CONFIG_INTERCONNECT) struct icc_path *icc_get(struct device *dev, const int src_id, diff --git a/include/linux/kernel_read_file.h b/include/linux/kernel_read_file.h new file mode 100644 index 000000000000..575ffa1031d3 --- /dev/null +++ b/include/linux/kernel_read_file.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_KERNEL_READ_FILE_H +#define _LINUX_KERNEL_READ_FILE_H + +#include <linux/file.h> +#include <linux/types.h> + +/* This is a list of *what* is being read, not *how* nor *where*. */ +#define __kernel_read_file_id(id) \ + id(UNKNOWN, unknown) \ + id(FIRMWARE, firmware) \ + id(MODULE, kernel-module) \ + id(KEXEC_IMAGE, kexec-image) \ + id(KEXEC_INITRAMFS, kexec-initramfs) \ + id(POLICY, security-policy) \ + id(X509_CERTIFICATE, x509-certificate) \ + id(MAX_ID, ) + +#define __fid_enumify(ENUM, dummy) READING_ ## ENUM, +#define __fid_stringify(dummy, str) #str, + +enum kernel_read_file_id { + __kernel_read_file_id(__fid_enumify) +}; + +static const char * const kernel_read_file_str[] = { + __kernel_read_file_id(__fid_stringify) +}; + +static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) +{ + if ((unsigned int)id >= READING_MAX_ID) + return kernel_read_file_str[READING_UNKNOWN]; + + return kernel_read_file_str[id]; +} + +int kernel_read_file(struct file *file, loff_t offset, + void **buf, size_t buf_size, + size_t *file_size, + enum kernel_read_file_id id); +int kernel_read_file_from_path(const char *path, loff_t offset, + void **buf, size_t buf_size, + size_t *file_size, + enum kernel_read_file_id id); +int kernel_read_file_from_path_initns(const char *path, loff_t offset, + void **buf, size_t buf_size, + size_t *file_size, + enum kernel_read_file_id id); +int kernel_read_file_from_fd(int fd, loff_t offset, + void **buf, size_t buf_size, + size_t *file_size, + enum kernel_read_file_id id); + +#endif /* _LINUX_KERNEL_READ_FILE_H */ diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 2a8c74d99015..32a940117e7a 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -184,9 +184,11 @@ LSM_HOOK(void, LSM_RET_VOID, cred_getsecid, const struct cred *c, u32 *secid) LSM_HOOK(int, 0, kernel_act_as, struct cred *new, u32 secid) LSM_HOOK(int, 0, kernel_create_files_as, struct cred *new, struct inode *inode) LSM_HOOK(int, 0, kernel_module_request, char *kmod_name) -LSM_HOOK(int, 0, kernel_load_data, enum kernel_load_data_id id) +LSM_HOOK(int, 0, kernel_load_data, enum kernel_load_data_id id, bool contents) +LSM_HOOK(int, 0, kernel_post_load_data, char *buf, loff_t size, + enum kernel_load_data_id id, char *description) LSM_HOOK(int, 0, kernel_read_file, struct file *file, - enum kernel_read_file_id id) + enum kernel_read_file_id id, bool contents) LSM_HOOK(int, 0, kernel_post_read_file, struct file *file, char *buf, loff_t size, enum kernel_read_file_id id) LSM_HOOK(int, 0, task_fix_setuid, struct cred *new, const struct cred *old, diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h index 9e2e3e63719d..8814e3d5952d 100644 --- a/include/linux/lsm_hooks.h +++ b/include/linux/lsm_hooks.h @@ -635,12 +635,23 @@ * @kernel_load_data: * Load data provided by userspace. * @id kernel load data identifier + * @contents if a subsequent @kernel_post_load_data will be called. * Return 0 if permission is granted. + * @kernel_post_load_data: + * Load data provided by a non-file source (usually userspace buffer). + * @buf pointer to buffer containing the data contents. + * @size length of the data contents. + * @id kernel load data identifier + * @description a text description of what was loaded, @id-specific + * Return 0 if permission is granted. + * This must be paired with a prior @kernel_load_data call that had + * @contents set to true. * @kernel_read_file: * Read a file specified by userspace. * @file contains the file structure pointing to the file being read * by the kernel. * @id kernel read file identifier + * @contents if a subsequent @kernel_post_read_file will be called. * Return 0 if permission is granted. * @kernel_post_read_file: * Read a file specified by userspace. @@ -649,6 +660,8 @@ * @buf pointer to buffer containing the file contents. * @size length of the file contents. * @id kernel read file identifier + * This must be paired with a prior @kernel_read_file call that had + * @contents set to true. * Return 0 if permission is granted. * @task_fix_setuid: * Update the module's state after setting one or more of the user diff --git a/include/linux/mhi.h b/include/linux/mhi.h index c4a940d98912..d4841e5a5f45 100644 --- a/include/linux/mhi.h +++ b/include/linux/mhi.h @@ -9,13 +9,14 @@ #include <linux/device.h> #include <linux/dma-direction.h> #include <linux/mutex.h> -#include <linux/rwlock_types.h> #include <linux/skbuff.h> #include <linux/slab.h> -#include <linux/spinlock_types.h> +#include <linux/spinlock.h> #include <linux/wait.h> #include <linux/workqueue.h> +#define MHI_MAX_OEM_PK_HASH_SEGMENTS 16 + struct mhi_chan; struct mhi_event; struct mhi_ctxt; @@ -85,13 +86,15 @@ enum mhi_ch_type { }; /** - * struct image_info - Firmware and RDDM table table - * @mhi_buf - Buffer for firmware and RDDM table - * @entries - # of entries in table + * struct image_info - Firmware and RDDM table + * @mhi_buf: Buffer for firmware and RDDM table + * @entries: # of entries in table */ struct image_info { struct mhi_buf *mhi_buf; + /* private: from internal.h */ struct bhi_vec_entry *bhi_vec; + /* public: */ u32 entries; }; @@ -276,9 +279,9 @@ struct mhi_controller_config { u32 timeout_ms; u32 buf_len; u32 num_channels; - struct mhi_channel_config *ch_cfg; + const struct mhi_channel_config *ch_cfg; u32 num_events; - struct mhi_event_config *event_cfg; + const struct mhi_event_config *event_cfg; bool use_bounce_buf; bool m2_no_db; }; @@ -288,6 +291,7 @@ struct mhi_controller_config { * @cntrl_dev: Pointer to the struct device of physical bus acting as the MHI * controller (required) * @mhi_dev: MHI device instance for the controller + * @debugfs_dentry: MHI controller debugfs directory * @regs: Base address of MHI MMIO register space (required) * @bhi: Points to base of MHI BHI register space * @bhie: Points to base of MHI BHIe register space @@ -308,12 +312,13 @@ struct mhi_controller_config { * @total_ev_rings: Total # of event rings allocated * @hw_ev_rings: Number of hardware event rings * @sw_ev_rings: Number of software event rings - * @nr_irqs_req: Number of IRQs required to operate (optional) * @nr_irqs: Number of IRQ allocated by bus master (required) * @family_number: MHI controller family number * @device_number: MHI controller device number * @major_version: MHI controller major revision number * @minor_version: MHI controller minor revision number + * @serial_number: MHI controller serial number obtained from BHI + * @oem_pk_hash: MHI controller OEM PK Hash obtained from BHI * @mhi_event: MHI event ring configurations table * @mhi_cmd: MHI command ring configurations table * @mhi_ctxt: MHI device context, shared memory between host and device @@ -326,6 +331,7 @@ struct mhi_controller_config { * @dev_state: MHI device state * @dev_wake: Device wakeup count * @pending_pkts: Pending packets for the controller + * @M0, M2, M3: Counters to track number of device MHI state changes * @transition_list: List of MHI state transitions * @transition_lock: Lock for protecting MHI state transition list * @wlock: Lock for protecting device wakeup @@ -364,6 +370,7 @@ struct mhi_controller_config { struct mhi_controller { struct device *cntrl_dev; struct mhi_device *mhi_dev; + struct dentry *debugfs_dentry; void __iomem *regs; void __iomem *bhi; void __iomem *bhie; @@ -385,12 +392,13 @@ struct mhi_controller { u32 total_ev_rings; u32 hw_ev_rings; u32 sw_ev_rings; - u32 nr_irqs_req; u32 nr_irqs; u32 family_number; u32 device_number; u32 major_version; u32 minor_version; + u32 serial_number; + u32 oem_pk_hash[MHI_MAX_OEM_PK_HASH_SEGMENTS]; struct mhi_event *mhi_event; struct mhi_cmd *mhi_cmd; @@ -405,6 +413,7 @@ struct mhi_controller { enum mhi_state dev_state; atomic_t dev_wake; atomic_t pending_pkts; + u32 M0, M2, M3; struct list_head transition_list; spinlock_t transition_lock; spinlock_t wlock; @@ -436,10 +445,10 @@ struct mhi_controller { }; /** - * struct mhi_device - Structure representing a MHI device which binds - * to channels + * struct mhi_device - Structure representing an MHI device which binds + * to channels or is associated with controllers * @id: Pointer to MHI device ID struct - * @chan_name: Name of the channel to which the device binds + * @name: Name of the associated MHI device * @mhi_cntrl: Controller the device belongs to * @ul_chan: UL channel for the device * @dl_chan: DL channel for the device @@ -451,7 +460,7 @@ struct mhi_controller { */ struct mhi_device { const struct mhi_device_id *id; - const char *chan_name; + const char *name; struct mhi_controller *mhi_cntrl; struct mhi_chan *ul_chan; struct mhi_chan *dl_chan; @@ -518,12 +527,24 @@ struct mhi_driver { #define to_mhi_device(dev) container_of(dev, struct mhi_device, dev) /** + * mhi_alloc_controller - Allocate the MHI Controller structure + * Allocate the mhi_controller structure using zero initialized memory + */ +struct mhi_controller *mhi_alloc_controller(void); + +/** + * mhi_free_controller - Free the MHI Controller structure + * Free the mhi_controller structure which was previously allocated + */ +void mhi_free_controller(struct mhi_controller *mhi_cntrl); + +/** * mhi_register_controller - Register MHI controller * @mhi_cntrl: MHI controller to register * @config: Configuration to use for the controller */ int mhi_register_controller(struct mhi_controller *mhi_cntrl, - struct mhi_controller_config *config); + const struct mhi_controller_config *config); /** * mhi_unregister_controller - Unregister MHI controller @@ -593,7 +614,7 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl); /** * mhi_sync_power_up - Start MHI power up sequence and wait till the device - * device enters valid EE state + * enters valid EE state * @mhi_cntrl: MHI controller */ int mhi_sync_power_up(struct mhi_controller *mhi_cntrl); diff --git a/include/linux/miscdevice.h b/include/linux/miscdevice.h index c7a93002a3c1..0676f18093f9 100644 --- a/include/linux/miscdevice.h +++ b/include/linux/miscdevice.h @@ -7,9 +7,9 @@ #include <linux/device.h> /* - * These allocations are managed by device@lanana.org. If you use an - * entry that is not in assigned your entry may well be moved and - * reassigned, or set dynamic if a fixed value is not justified. + * These allocations are managed by device@lanana.org. If you need + * an entry that is not assigned here, it can be moved and + * reassigned or dynamically set if a fixed value is not justified. */ #define PSMOUSE_MINOR 1 @@ -93,14 +93,14 @@ extern void misc_deregister(struct miscdevice *misc); /* * Helper macro for drivers that don't do anything special in the initcall. - * This helps in eleminating of boilerplate code. + * This helps to eliminate boilerplate code. */ #define builtin_misc_device(__misc_device) \ builtin_driver(__misc_device, misc_register) /* * Helper macro for drivers that don't do anything special in module init / exit - * call. This helps in eleminating of boilerplate code. + * call. This helps to eliminate boilerplate code. */ #define module_misc_device(__misc_device) \ module_driver(__misc_device, misc_register, misc_deregister) diff --git a/include/linux/nitro_enclaves.h b/include/linux/nitro_enclaves.h new file mode 100644 index 000000000000..d91ef2bfdf47 --- /dev/null +++ b/include/linux/nitro_enclaves.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +#ifndef _LINUX_NITRO_ENCLAVES_H_ +#define _LINUX_NITRO_ENCLAVES_H_ + +#include <uapi/linux/nitro_enclaves.h> + +#endif /* _LINUX_NITRO_ENCLAVES_H_ */ diff --git a/include/linux/security.h b/include/linux/security.h index 0a0a03b36a3b..bc2725491560 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -23,6 +23,7 @@ #ifndef __LINUX_SECURITY_H #define __LINUX_SECURITY_H +#include <linux/kernel_read_file.h> #include <linux/key.h> #include <linux/capability.h> #include <linux/fs.h> @@ -386,8 +387,12 @@ void security_cred_getsecid(const struct cred *c, u32 *secid); int security_kernel_act_as(struct cred *new, u32 secid); int security_kernel_create_files_as(struct cred *new, struct inode *inode); int security_kernel_module_request(char *kmod_name); -int security_kernel_load_data(enum kernel_load_data_id id); -int security_kernel_read_file(struct file *file, enum kernel_read_file_id id); +int security_kernel_load_data(enum kernel_load_data_id id, bool contents); +int security_kernel_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id id, + char *description); +int security_kernel_read_file(struct file *file, enum kernel_read_file_id id, + bool contents); int security_kernel_post_read_file(struct file *file, char *buf, loff_t size, enum kernel_read_file_id id); int security_task_fix_setuid(struct cred *new, const struct cred *old, @@ -1013,13 +1018,21 @@ static inline int security_kernel_module_request(char *kmod_name) return 0; } -static inline int security_kernel_load_data(enum kernel_load_data_id id) +static inline int security_kernel_load_data(enum kernel_load_data_id id, bool contents) +{ + return 0; +} + +static inline int security_kernel_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id id, + char *description) { return 0; } static inline int security_kernel_read_file(struct file *file, - enum kernel_read_file_id id) + enum kernel_read_file_id id, + bool contents) { return 0; } diff --git a/include/linux/soundwire/sdw.h b/include/linux/soundwire/sdw.h index 76052f12c9f7..41cc1192f9aa 100644 --- a/include/linux/soundwire/sdw.h +++ b/include/linux/soundwire/sdw.h @@ -5,6 +5,7 @@ #define __SOUNDWIRE_H #include <linux/mod_devicetable.h> +#include <linux/bitfield.h> struct sdw_bus; struct sdw_slave; @@ -38,7 +39,8 @@ struct sdw_slave; #define SDW_FRAME_CTRL_BITS 48 #define SDW_MAX_DEVICES 11 -#define SDW_VALID_PORT_RANGE(n) ((n) <= 14 && (n) >= 1) +#define SDW_MAX_PORTS 15 +#define SDW_VALID_PORT_RANGE(n) ((n) < SDW_MAX_PORTS && (n) >= 1) enum { SDW_PORT_DIRN_SINK = 0, @@ -355,6 +357,8 @@ struct sdw_dpn_prop { * @dp0_prop: Data Port 0 properties * @src_dpn_prop: Source Data Port N properties * @sink_dpn_prop: Sink Data Port N properties + * @scp_int1_mask: SCP_INT1_MASK desired settings + * @quirks: bitmask identifying deltas from the MIPI specification */ struct sdw_slave_prop { u32 mipi_revision; @@ -376,8 +380,12 @@ struct sdw_slave_prop { struct sdw_dp0_prop *dp0_prop; struct sdw_dpn_prop *src_dpn_prop; struct sdw_dpn_prop *sink_dpn_prop; + u8 scp_int1_mask; + u32 quirks; }; +#define SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY BIT(0) + /** * struct sdw_master_prop - Master properties * @revision: MIPI spec version of the implementation @@ -455,13 +463,19 @@ struct sdw_slave_id { * * The MIPI DisCo for SoundWire defines in addition the link_id as bits 51:48 */ +#define SDW_DISCO_LINK_ID_MASK GENMASK_ULL(51, 48) +#define SDW_VERSION_MASK GENMASK_ULL(47, 44) +#define SDW_UNIQUE_ID_MASK GENMASK_ULL(43, 40) +#define SDW_MFG_ID_MASK GENMASK_ULL(39, 24) +#define SDW_PART_ID_MASK GENMASK_ULL(23, 8) +#define SDW_CLASS_ID_MASK GENMASK_ULL(7, 0) -#define SDW_DISCO_LINK_ID(adr) (((adr) >> 48) & GENMASK(3, 0)) -#define SDW_VERSION(adr) (((adr) >> 44) & GENMASK(3, 0)) -#define SDW_UNIQUE_ID(adr) (((adr) >> 40) & GENMASK(3, 0)) -#define SDW_MFG_ID(adr) (((adr) >> 24) & GENMASK(15, 0)) -#define SDW_PART_ID(adr) (((adr) >> 8) & GENMASK(15, 0)) -#define SDW_CLASS_ID(adr) ((adr) & GENMASK(7, 0)) +#define SDW_DISCO_LINK_ID(addr) FIELD_GET(SDW_DISCO_LINK_ID_MASK, addr) +#define SDW_VERSION(addr) FIELD_GET(SDW_VERSION_MASK, addr) +#define SDW_UNIQUE_ID(addr) FIELD_GET(SDW_UNIQUE_ID_MASK, addr) +#define SDW_MFG_ID(addr) FIELD_GET(SDW_MFG_ID_MASK, addr) +#define SDW_PART_ID(addr) FIELD_GET(SDW_PART_ID_MASK, addr) +#define SDW_CLASS_ID(addr) FIELD_GET(SDW_CLASS_ID_MASK, addr) /** * struct sdw_slave_intr_status - Slave interrupt status @@ -540,6 +554,10 @@ enum sdw_port_prep_ops { * @bandwidth: Current bandwidth * @col: Active columns * @row: Active rows + * @s_data_mode: NORMAL, STATIC or PRBS mode for all Slave ports + * @m_data_mode: NORMAL, STATIC or PRBS mode for all Master ports. The value + * should be the same to detect transmission issues, but can be different to + * test the interrupt reports */ struct sdw_bus_params { enum sdw_reg_bank curr_bank; @@ -549,6 +567,8 @@ struct sdw_bus_params { unsigned int bandwidth; unsigned int col; unsigned int row; + int s_data_mode; + int m_data_mode; }; /** @@ -606,6 +626,8 @@ struct sdw_slave_ops { * between the Master suspending and the codec resuming, and make sure that * when the Master triggered a reset the Slave is properly enumerated and * initialized + * @first_interrupt_done: status flag tracking if the interrupt handling + * for a Slave happens for the first time after enumeration */ struct sdw_slave { struct sdw_slave_id id; @@ -618,7 +640,7 @@ struct sdw_slave { struct dentry *debugfs; #endif struct list_head node; - struct completion *port_ready; + struct completion port_ready[SDW_MAX_PORTS]; enum sdw_clk_stop_mode curr_clk_stop_mode; u16 dev_num; u16 dev_num_sticky; @@ -627,6 +649,7 @@ struct sdw_slave { struct completion enumeration_complete; struct completion initialization_complete; u32 unattach_request; + bool first_interrupt_done; }; #define dev_to_sdw_dev(_dev) container_of(_dev, struct sdw_slave, dev) @@ -827,6 +850,11 @@ struct sdw_master_ops { * @multi_link: Store bus property that indicates if multi links * are supported. This flag is populated by drivers after reading * appropriate firmware (ACPI/DT). + * @hw_sync_min_links: Number of links used by a stream above which + * hardware-based synchronization is required. This value is only + * meaningful if multi_link is set. If set to 1, hardware-based + * synchronization will be used even if a stream only uses a single + * SoundWire segment. */ struct sdw_bus { struct device *dev; @@ -850,6 +878,7 @@ struct sdw_bus { unsigned int clk_stop_timeout; u32 bank_switch_timeout; bool multi_link; + int hw_sync_min_links; }; int sdw_bus_master_add(struct sdw_bus *bus, struct device *parent, @@ -941,6 +970,9 @@ struct sdw_stream_runtime { struct sdw_stream_runtime *sdw_alloc_stream(const char *stream_name); void sdw_release_stream(struct sdw_stream_runtime *stream); + +int sdw_compute_params(struct sdw_bus *bus); + int sdw_stream_add_master(struct sdw_bus *bus, struct sdw_stream_config *stream_config, struct sdw_port_config *port_config, diff --git a/include/linux/soundwire/sdw_registers.h b/include/linux/soundwire/sdw_registers.h index 5d3c271af7d1..f420e8059779 100644 --- a/include/linux/soundwire/sdw_registers.h +++ b/include/linux/soundwire/sdw_registers.h @@ -5,13 +5,6 @@ #define __SDW_REGISTERS_H /* - * typically we define register and shifts but if one observes carefully, - * the shift can be generated from MASKS using few bit primitaives like ffs - * etc, so we use that and avoid defining shifts - */ -#define SDW_REG_SHIFT(n) (ffs(n) - 1) - -/* * SDW registers as defined by MIPI 1.2 Spec */ #define SDW_REGADDR GENMASK(14, 0) diff --git a/include/linux/spi/eeprom.h b/include/linux/spi/eeprom.h index aceccf9c71fb..1cca3dd5a748 100644 --- a/include/linux/spi/eeprom.h +++ b/include/linux/spi/eeprom.h @@ -14,7 +14,7 @@ struct spi_eeprom { u32 byte_len; char name[10]; - u16 page_size; /* for writes */ + u32 page_size; /* for writes */ u16 flags; #define EE_ADDR1 0x0001 /* 8 bit addrs */ #define EE_ADDR2 0x0002 /* 16 bit addrs */ diff --git a/include/linux/trace.h b/include/linux/trace.h index 36d255d66f88..886a4ffd9d45 100644 --- a/include/linux/trace.h +++ b/include/linux/trace.h @@ -3,6 +3,11 @@ #define _LINUX_TRACE_H #ifdef CONFIG_TRACING + +#define TRACE_EXPORT_FUNCTION BIT(0) +#define TRACE_EXPORT_EVENT BIT(1) +#define TRACE_EXPORT_MARKER BIT(2) + /* * The trace export - an export of Ftrace output. The trace_export * can process traces and export them to a registered destination as @@ -15,10 +20,12 @@ * next - pointer to the next trace_export * write - copy traces which have been delt with ->commit() to * the destination + * flags - which ftrace to be exported */ struct trace_export { struct trace_export __rcu *next; void (*write)(struct trace_export *, const void *, unsigned int); + int flags; }; int register_ftrace_export(struct trace_export *export); diff --git a/include/linux/w1.h b/include/linux/w1.h index cebf3464bc03..949d3b10e531 100644 --- a/include/linux/w1.h +++ b/include/linux/w1.h @@ -269,7 +269,7 @@ struct w1_family { struct list_head family_entry; u8 fid; - struct w1_family_ops *fops; + const struct w1_family_ops *fops; const struct of_device_id *of_match_table; diff --git a/include/uapi/linux/coresight-stm.h b/include/uapi/linux/coresight-stm.h index 8847dbf24151..7ff3709c01b8 100644 --- a/include/uapi/linux/coresight-stm.h +++ b/include/uapi/linux/coresight-stm.h @@ -5,6 +5,7 @@ #include <linux/const.h> #define STM_FLAG_TIMESTAMPED _BITUL(3) +#define STM_FLAG_MARKED _BITUL(4) #define STM_FLAG_GUARANTEED _BITUL(7) /* diff --git a/include/uapi/linux/mei.h b/include/uapi/linux/mei.h index c6aec86cc5de..4f3638489d01 100644 --- a/include/uapi/linux/mei.h +++ b/include/uapi/linux/mei.h @@ -66,4 +66,53 @@ struct mei_connect_client_data { */ #define IOCTL_MEI_NOTIFY_GET _IOR('H', 0x03, __u32) +/** + * struct mei_connect_client_vtag - mei client information struct with vtag + * + * @in_client_uuid: UUID of client to connect + * @vtag: virtual tag + * @reserved: reserved for future use + */ +struct mei_connect_client_vtag { + uuid_le in_client_uuid; + __u8 vtag; + __u8 reserved[3]; +}; + +/** + * struct mei_connect_client_data_vtag - IOCTL connect data union + * + * @connect: input connect data + * @out_client_properties: output client data + */ +struct mei_connect_client_data_vtag { + union { + struct mei_connect_client_vtag connect; + struct mei_client out_client_properties; + }; +}; + +/** + * DOC: + * This IOCTL is used to associate the current file descriptor with a + * FW Client (given by UUID), and virtual tag (vtag). + * The IOCTL opens a communication channel between a host client and + * a FW client on a tagged channel. From this point on, every read + * and write will communicate with the associated FW client with + * on the tagged channel. + * Upone close() the communication is terminated. + * + * The IOCTL argument is a struct with a union that contains + * the input parameter and the output parameter for this IOCTL. + * + * The input parameter is UUID of the FW Client, a vtag [0,255] + * The output parameter is the properties of the FW client + * (FW protocool version and max message size). + * + * Clients that do not support tagged connection + * will respond with -EOPNOTSUPP. + */ +#define IOCTL_MEI_CONNECT_CLIENT_VTAG \ + _IOWR('H', 0x04, struct mei_connect_client_data_vtag) + #endif /* _LINUX_MEI_H */ diff --git a/include/uapi/linux/nitro_enclaves.h b/include/uapi/linux/nitro_enclaves.h new file mode 100644 index 000000000000..b945073fe544 --- /dev/null +++ b/include/uapi/linux/nitro_enclaves.h @@ -0,0 +1,359 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +#ifndef _UAPI_LINUX_NITRO_ENCLAVES_H_ +#define _UAPI_LINUX_NITRO_ENCLAVES_H_ + +#include <linux/types.h> + +/** + * DOC: Nitro Enclaves (NE) Kernel Driver Interface + */ + +/** + * NE_CREATE_VM - The command is used to create a slot that is associated with + * an enclave VM. + * The generated unique slot id is an output parameter. + * The ioctl can be invoked on the /dev/nitro_enclaves fd, before + * setting any resources, such as memory and vCPUs, for an + * enclave. Memory and vCPUs are set for the slot mapped to an enclave. + * A NE CPU pool has to be set before calling this function. The + * pool can be set after the NE driver load, using + * /sys/module/nitro_enclaves/parameters/ne_cpus. + * Its format is the detailed in the cpu-lists section: + * https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html + * CPU 0 and its siblings have to remain available for the + * primary / parent VM, so they cannot be set for enclaves. Full + * CPU core(s), from the same NUMA node, need(s) to be included + * in the CPU pool. + * + * Context: Process context. + * Return: + * * Enclave file descriptor - Enclave file descriptor used with + * ioctl calls to set vCPUs and memory + * regions, then start the enclave. + * * -1 - There was a failure in the ioctl logic. + * On failure, errno is set to: + * * EFAULT - copy_to_user() failure. + * * ENOMEM - Memory allocation failure for internal + * bookkeeping variables. + * * NE_ERR_NO_CPUS_AVAIL_IN_POOL - No NE CPU pool set / no CPUs available + * in the pool. + * * Error codes from get_unused_fd_flags() and anon_inode_getfile(). + * * Error codes from the NE PCI device request. + */ +#define NE_CREATE_VM _IOR(0xAE, 0x20, __u64) + +/** + * NE_ADD_VCPU - The command is used to set a vCPU for an enclave. The vCPU can + * be auto-chosen from the NE CPU pool or it can be set by the + * caller, with the note that it needs to be available in the NE + * CPU pool. Full CPU core(s), from the same NUMA node, need(s) to + * be associated with an enclave. + * The vCPU id is an input / output parameter. If its value is 0, + * then a CPU is chosen from the enclave CPU pool and returned via + * this parameter. + * The ioctl can be invoked on the enclave fd, before an enclave + * is started. + * + * Context: Process context. + * Return: + * * 0 - Logic succesfully completed. + * * -1 - There was a failure in the ioctl logic. + * On failure, errno is set to: + * * EFAULT - copy_from_user() / copy_to_user() failure. + * * ENOMEM - Memory allocation failure for internal + * bookkeeping variables. + * * EIO - Current task mm is not the same as the one + * that created the enclave. + * * NE_ERR_NO_CPUS_AVAIL_IN_POOL - No CPUs available in the NE CPU pool. + * * NE_ERR_VCPU_ALREADY_USED - The provided vCPU is already used. + * * NE_ERR_VCPU_NOT_IN_CPU_POOL - The provided vCPU is not available in the + * NE CPU pool. + * * NE_ERR_VCPU_INVALID_CPU_CORE - The core id of the provided vCPU is invalid + * or out of range. + * * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state + * (init = before being started). + * * NE_ERR_INVALID_VCPU - The provided vCPU is not in the available + * CPUs range. + * * Error codes from the NE PCI device request. + */ +#define NE_ADD_VCPU _IOWR(0xAE, 0x21, __u32) + +/** + * NE_GET_IMAGE_LOAD_INFO - The command is used to get information needed for + * in-memory enclave image loading e.g. offset in + * enclave memory to start placing the enclave image. + * The image load info is an input / output parameter. + * It includes info provided by the caller - flags - + * and returns the offset in enclave memory where to + * start placing the enclave image. + * The ioctl can be invoked on the enclave fd, before + * an enclave is started. + * + * Context: Process context. + * Return: + * * 0 - Logic succesfully completed. + * * -1 - There was a failure in the ioctl logic. + * On failure, errno is set to: + * * EFAULT - copy_from_user() / copy_to_user() failure. + * * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state (init = + * before being started). + * * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid. + */ +#define NE_GET_IMAGE_LOAD_INFO _IOWR(0xAE, 0x22, struct ne_image_load_info) + +/** + * NE_SET_USER_MEMORY_REGION - The command is used to set a memory region for an + * enclave, given the allocated memory from the + * userspace. Enclave memory needs to be from the + * same NUMA node as the enclave CPUs. + * The user memory region is an input parameter. It + * includes info provided by the caller - flags, + * memory size and userspace address. + * The ioctl can be invoked on the enclave fd, + * before an enclave is started. + * + * Context: Process context. + * Return: + * * 0 - Logic succesfully completed. + * * -1 - There was a failure in the ioctl logic. + * On failure, errno is set to: + * * EFAULT - copy_from_user() failure. + * * EINVAL - Invalid physical memory region(s) e.g. + * unaligned address. + * * EIO - Current task mm is not the same as + * the one that created the enclave. + * * ENOMEM - Memory allocation failure for internal + * bookkeeping variables. + * * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state + * (init = before being started). + * * NE_ERR_INVALID_MEM_REGION_SIZE - The memory size of the region is not + * multiple of 2 MiB. + * * NE_ERR_INVALID_MEM_REGION_ADDR - Invalid user space address given. + * * NE_ERR_UNALIGNED_MEM_REGION_ADDR - Unaligned user space address given. + * * NE_ERR_MEM_REGION_ALREADY_USED - The memory region is already used. + * * NE_ERR_MEM_NOT_HUGE_PAGE - The memory region is not backed by + * huge pages. + * * NE_ERR_MEM_DIFFERENT_NUMA_NODE - The memory region is not from the same + * NUMA node as the CPUs. + * * NE_ERR_MEM_MAX_REGIONS - The number of memory regions set for + * the enclave reached maximum. + * * NE_ERR_INVALID_PAGE_SIZE - The memory region is not backed by + * pages multiple of 2 MiB. + * * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid. + * * Error codes from get_user_pages(). + * * Error codes from the NE PCI device request. + */ +#define NE_SET_USER_MEMORY_REGION _IOW(0xAE, 0x23, struct ne_user_memory_region) + +/** + * NE_START_ENCLAVE - The command is used to trigger enclave start after the + * enclave resources, such as memory and CPU, have been set. + * The enclave start info is an input / output parameter. It + * includes info provided by the caller - enclave cid and + * flags - and returns the cid (if input cid is 0). + * The ioctl can be invoked on the enclave fd, after an + * enclave slot is created and resources, such as memory and + * vCPUs are set for an enclave. + * + * Context: Process context. + * Return: + * * 0 - Logic succesfully completed. + * * -1 - There was a failure in the ioctl logic. + * On failure, errno is set to: + * * EFAULT - copy_from_user() / copy_to_user() failure. + * * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state + * (init = before being started). + * * NE_ERR_NO_MEM_REGIONS_ADDED - No memory regions are set. + * * NE_ERR_NO_VCPUS_ADDED - No vCPUs are set. + * * NE_ERR_FULL_CORES_NOT_USED - Full core(s) not set for the enclave. + * * NE_ERR_ENCLAVE_MEM_MIN_SIZE - Enclave memory is less than minimum + * memory size (64 MiB). + * * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid. + * * NE_ERR_INVALID_ENCLAVE_CID - The provided enclave CID is invalid. + * * Error codes from the NE PCI device request. + */ +#define NE_START_ENCLAVE _IOWR(0xAE, 0x24, struct ne_enclave_start_info) + +/** + * DOC: NE specific error codes + */ + +/** + * NE_ERR_VCPU_ALREADY_USED - The provided vCPU is already used. + */ +#define NE_ERR_VCPU_ALREADY_USED (256) +/** + * NE_ERR_VCPU_NOT_IN_CPU_POOL - The provided vCPU is not available in the + * NE CPU pool. + */ +#define NE_ERR_VCPU_NOT_IN_CPU_POOL (257) +/** + * NE_ERR_VCPU_INVALID_CPU_CORE - The core id of the provided vCPU is invalid + * or out of range of the NE CPU pool. + */ +#define NE_ERR_VCPU_INVALID_CPU_CORE (258) +/** + * NE_ERR_INVALID_MEM_REGION_SIZE - The user space memory region size is not + * multiple of 2 MiB. + */ +#define NE_ERR_INVALID_MEM_REGION_SIZE (259) +/** + * NE_ERR_INVALID_MEM_REGION_ADDR - The user space memory region address range + * is invalid. + */ +#define NE_ERR_INVALID_MEM_REGION_ADDR (260) +/** + * NE_ERR_UNALIGNED_MEM_REGION_ADDR - The user space memory region address is + * not aligned. + */ +#define NE_ERR_UNALIGNED_MEM_REGION_ADDR (261) +/** + * NE_ERR_MEM_REGION_ALREADY_USED - The user space memory region is already used. + */ +#define NE_ERR_MEM_REGION_ALREADY_USED (262) +/** + * NE_ERR_MEM_NOT_HUGE_PAGE - The user space memory region is not backed by + * contiguous physical huge page(s). + */ +#define NE_ERR_MEM_NOT_HUGE_PAGE (263) +/** + * NE_ERR_MEM_DIFFERENT_NUMA_NODE - The user space memory region is backed by + * pages from different NUMA nodes than the CPUs. + */ +#define NE_ERR_MEM_DIFFERENT_NUMA_NODE (264) +/** + * NE_ERR_MEM_MAX_REGIONS - The supported max memory regions per enclaves has + * been reached. + */ +#define NE_ERR_MEM_MAX_REGIONS (265) +/** + * NE_ERR_NO_MEM_REGIONS_ADDED - The command to start an enclave is triggered + * and no memory regions are added. + */ +#define NE_ERR_NO_MEM_REGIONS_ADDED (266) +/** + * NE_ERR_NO_VCPUS_ADDED - The command to start an enclave is triggered and no + * vCPUs are added. + */ +#define NE_ERR_NO_VCPUS_ADDED (267) +/** + * NE_ERR_ENCLAVE_MEM_MIN_SIZE - The enclave memory size is lower than the + * minimum supported. + */ +#define NE_ERR_ENCLAVE_MEM_MIN_SIZE (268) +/** + * NE_ERR_FULL_CORES_NOT_USED - The command to start an enclave is triggered and + * full CPU cores are not set. + */ +#define NE_ERR_FULL_CORES_NOT_USED (269) +/** + * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state when setting + * resources or triggering start. + */ +#define NE_ERR_NOT_IN_INIT_STATE (270) +/** + * NE_ERR_INVALID_VCPU - The provided vCPU is out of range of the available CPUs. + */ +#define NE_ERR_INVALID_VCPU (271) +/** + * NE_ERR_NO_CPUS_AVAIL_IN_POOL - The command to create an enclave is triggered + * and no CPUs are available in the pool. + */ +#define NE_ERR_NO_CPUS_AVAIL_IN_POOL (272) +/** + * NE_ERR_INVALID_PAGE_SIZE - The user space memory region is not backed by pages + * multiple of 2 MiB. + */ +#define NE_ERR_INVALID_PAGE_SIZE (273) +/** + * NE_ERR_INVALID_FLAG_VALUE - The provided flag value is invalid. + */ +#define NE_ERR_INVALID_FLAG_VALUE (274) +/** + * NE_ERR_INVALID_ENCLAVE_CID - The provided enclave CID is invalid, either + * being a well-known value or the CID of the + * parent / primary VM. + */ +#define NE_ERR_INVALID_ENCLAVE_CID (275) + +/** + * DOC: Image load info flags + */ + +/** + * NE_EIF_IMAGE - Enclave Image Format (EIF) + */ +#define NE_EIF_IMAGE (0x01) + +#define NE_IMAGE_LOAD_MAX_FLAG_VAL (0x02) + +/** + * struct ne_image_load_info - Info necessary for in-memory enclave image + * loading (in / out). + * @flags: Flags to determine the enclave image type + * (e.g. Enclave Image Format - EIF) (in). + * @memory_offset: Offset in enclave memory where to start placing the + * enclave image (out). + */ +struct ne_image_load_info { + __u64 flags; + __u64 memory_offset; +}; + +/** + * DOC: User memory region flags + */ + +/** + * NE_DEFAULT_MEMORY_REGION - Memory region for enclave general usage. + */ +#define NE_DEFAULT_MEMORY_REGION (0x00) + +#define NE_MEMORY_REGION_MAX_FLAG_VAL (0x01) + +/** + * struct ne_user_memory_region - Memory region to be set for an enclave (in). + * @flags: Flags to determine the usage for the memory region (in). + * @memory_size: The size, in bytes, of the memory region to be set for + * an enclave (in). + * @userspace_addr: The start address of the userspace allocated memory of + * the memory region to set for an enclave (in). + */ +struct ne_user_memory_region { + __u64 flags; + __u64 memory_size; + __u64 userspace_addr; +}; + +/** + * DOC: Enclave start info flags + */ + +/** + * NE_ENCLAVE_PRODUCTION_MODE - Start enclave in production mode. + */ +#define NE_ENCLAVE_PRODUCTION_MODE (0x00) +/** + * NE_ENCLAVE_DEBUG_MODE - Start enclave in debug mode. + */ +#define NE_ENCLAVE_DEBUG_MODE (0x01) + +#define NE_ENCLAVE_START_MAX_FLAG_VAL (0x02) + +/** + * struct ne_enclave_start_info - Setup info necessary for enclave start (in / out). + * @flags: Flags for the enclave to start with (e.g. debug mode) (in). + * @enclave_cid: Context ID (CID) for the enclave vsock device. If 0 as + * input, the CID is autogenerated by the hypervisor and + * returned back as output by the driver (in / out). + */ +struct ne_enclave_start_info { + __u64 flags; + __u64 enclave_cid; +}; + +#endif /* _UAPI_LINUX_NITRO_ENCLAVES_H_ */ diff --git a/include/uapi/misc/fastrpc.h b/include/uapi/misc/fastrpc.h index 07de2b7aac85..0a89f95463f6 100644 --- a/include/uapi/misc/fastrpc.h +++ b/include/uapi/misc/fastrpc.h @@ -10,8 +10,9 @@ #define FASTRPC_IOCTL_INVOKE _IOWR('R', 3, struct fastrpc_invoke) #define FASTRPC_IOCTL_INIT_ATTACH _IO('R', 4) #define FASTRPC_IOCTL_INIT_CREATE _IOWR('R', 5, struct fastrpc_init_create) -#define FASTRPC_IOCTL_MMAP _IOWR('R', 6, struct fastrpc_req_mmap) -#define FASTRPC_IOCTL_MUNMAP _IOWR('R', 7, struct fastrpc_req_munmap) +#define FASTRPC_IOCTL_MMAP _IOWR('R', 6, struct fastrpc_req_mmap) +#define FASTRPC_IOCTL_MUNMAP _IOWR('R', 7, struct fastrpc_req_munmap) +#define FASTRPC_IOCTL_INIT_ATTACH_SNS _IO('R', 8) struct fastrpc_invoke_args { __u64 ptr; diff --git a/include/uapi/misc/habanalabs.h b/include/uapi/misc/habanalabs.h index d5c4f983b7a8..9705b8adb60c 100644 --- a/include/uapi/misc/habanalabs.h +++ b/include/uapi/misc/habanalabs.h @@ -264,6 +264,10 @@ enum hl_device_status { * HL_INFO_TIME_SYNC - Retrieve the device's time alongside the host's time * for synchronization. * HL_INFO_CS_COUNTERS - Retrieve command submission counters + * HL_INFO_PCI_COUNTERS - Retrieve PCI counters + * HL_INFO_CLK_THROTTLE_REASON - Retrieve clock throttling reason + * HL_INFO_SYNC_MANAGER - Retrieve sync manager info per dcore + * HL_INFO_TOTAL_ENERGY - Retrieve total energy consumption */ #define HL_INFO_HW_IP_INFO 0 #define HL_INFO_HW_EVENTS 1 @@ -276,6 +280,10 @@ enum hl_device_status { #define HL_INFO_RESET_COUNT 9 #define HL_INFO_TIME_SYNC 10 #define HL_INFO_CS_COUNTERS 11 +#define HL_INFO_PCI_COUNTERS 12 +#define HL_INFO_CLK_THROTTLE_REASON 13 +#define HL_INFO_SYNC_MANAGER 14 +#define HL_INFO_TOTAL_ENERGY 15 #define HL_INFO_VERSION_MAX_LEN 128 #define HL_INFO_CARD_NAME_MAX_LEN 16 @@ -289,7 +297,7 @@ struct hl_info_hw_ip_info { __u32 device_id; /* PCI Device ID */ __u32 module_id; /* For mezzanine cards in servers (From OCP spec.) */ __u32 reserved[2]; - __u32 armcp_cpld_version; + __u32 cpld_version; __u32 psoc_pci_pll_nr; __u32 psoc_pci_pll_nf; __u32 psoc_pci_pll_od; @@ -297,7 +305,7 @@ struct hl_info_hw_ip_info { __u8 tpc_enabled_mask; __u8 dram_enabled; __u8 pad[2]; - __u8 armcp_version[HL_INFO_VERSION_MAX_LEN]; + __u8 cpucp_version[HL_INFO_VERSION_MAX_LEN]; __u8 card_name[HL_INFO_CARD_NAME_MAX_LEN]; }; @@ -313,6 +321,12 @@ struct hl_info_hw_idle { * Bits definition is according to `enum <chip>_enging_id'. */ __u32 busy_engines_mask; + + /* + * Extended Bitmask of busy engines. + * Bits definition is according to `enum <chip>_enging_id'. + */ + __u64 busy_engines_mask_ext; }; struct hl_info_device_status { @@ -341,17 +355,60 @@ struct hl_info_time_sync { }; /** + * struct hl_info_pci_counters - pci counters + * @rx_throughput: PCI rx throughput KBps + * @tx_throughput: PCI tx throughput KBps + * @replay_cnt: PCI replay counter + */ +struct hl_info_pci_counters { + __u64 rx_throughput; + __u64 tx_throughput; + __u64 replay_cnt; +}; + +#define HL_CLK_THROTTLE_POWER 0x1 +#define HL_CLK_THROTTLE_THERMAL 0x2 + +/** + * struct hl_info_clk_throttle - clock throttling reason + * @clk_throttling_reason: each bit represents a clk throttling reason + */ +struct hl_info_clk_throttle { + __u32 clk_throttling_reason; +}; + +/** + * struct hl_info_energy - device energy information + * @total_energy_consumption: total device energy consumption + */ +struct hl_info_energy { + __u64 total_energy_consumption; +}; + +/** + * struct hl_info_sync_manager - sync manager information + * @first_available_sync_object: first available sob + * @first_available_monitor: first available monitor + */ +struct hl_info_sync_manager { + __u32 first_available_sync_object; + __u32 first_available_monitor; +}; + +/** * struct hl_info_cs_counters - command submission counters * @out_of_mem_drop_cnt: dropped due to memory allocation issue * @parsing_drop_cnt: dropped due to error in packet parsing * @queue_full_drop_cnt: dropped due to queue full * @device_in_reset_drop_cnt: dropped due to device in reset + * @max_cs_in_flight_drop_cnt: dropped due to maximum CS in-flight */ struct hl_cs_counters { __u64 out_of_mem_drop_cnt; __u64 parsing_drop_cnt; __u64 queue_full_drop_cnt; __u64 device_in_reset_drop_cnt; + __u64 max_cs_in_flight_drop_cnt; }; struct hl_info_cs_counters { @@ -359,6 +416,13 @@ struct hl_info_cs_counters { struct hl_cs_counters ctx_cs_counters; }; +enum gaudi_dcores { + HL_GAUDI_WS_DCORE, + HL_GAUDI_WN_DCORE, + HL_GAUDI_EN_DCORE, + HL_GAUDI_ES_DCORE +}; + struct hl_info_args { /* Location of relevant struct in userspace */ __u64 return_pointer; @@ -375,6 +439,10 @@ struct hl_info_args { __u32 op; union { + /* Dcore id for which the information is relevant. + * For Gaudi refer to 'enum gaudi_dcores' + */ + __u32 dcore_id; /* Context ID - Currently not in use */ __u32 ctx_id; /* Period value for utilization rate (100ms - 1000ms, in 100ms @@ -394,6 +462,9 @@ struct hl_info_args { /* 2MB minus 32 bytes for 2xMSG_PROT */ #define HL_MAX_CB_SIZE (0x200000 - 32) +/* Indicates whether the command buffer should be mapped to the device's MMU */ +#define HL_CB_FLAGS_MAP 0x1 + struct hl_cb_in { /* Handle of CB or 0 if we want to create one */ __u64 cb_handle; @@ -405,7 +476,8 @@ struct hl_cb_in { __u32 cb_size; /* Context ID - Currently not in use */ __u32 ctx_id; - __u32 pad; + /* HL_CB_FLAGS_* */ + __u32 flags; }; struct hl_cb_out { @@ -788,6 +860,12 @@ struct hl_debug_args { * When creating a new CB, the IOCTL returns a handle of it, and the user-space * process needs to use that handle to mmap the buffer so it can access them. * + * In some instances, the device must access the command buffer through the + * device's MMU, and thus its memory should be mapped. In these cases, user can + * indicate the driver that such a mapping is required. + * The resulting device virtual address will be used internally by the driver, + * and won't be returned to user. + * */ #define HL_IOCTL_CB \ _IOWR('H', 0x02, union hl_cb_args) @@ -846,6 +924,9 @@ struct hl_debug_args { * inside the kernel until the CS has finished or until the user-requested * timeout has expired. * + * If the timeout value is 0, the driver won't sleep at all. It will check + * the status of the CS and return immediately + * * The return value of the IOCTL is a standard Linux error code. The possible * values are: * diff --git a/kernel/kexec.c b/kernel/kexec.c index f977786fe498..c82c6c06f051 100644 --- a/kernel/kexec.c +++ b/kernel/kexec.c @@ -205,7 +205,7 @@ static inline int kexec_load_check(unsigned long nr_segments, return -EPERM; /* Permit LSMs and IMA to fail the kexec */ - result = security_kernel_load_data(LOADING_KEXEC_IMAGE); + result = security_kernel_load_data(LOADING_KEXEC_IMAGE, false); if (result < 0) return result; diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index ca40bef75a61..84f7316792a7 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -24,6 +24,7 @@ #include <linux/elf.h> #include <linux/elfcore.h> #include <linux/kernel.h> +#include <linux/kernel_read_file.h> #include <linux/syscalls.h> #include <linux/vmalloc.h> #include "kexec_internal.h" @@ -219,13 +220,12 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd, { int ret; void *ldata; - loff_t size; - ret = kernel_read_file_from_fd(kernel_fd, &image->kernel_buf, - &size, INT_MAX, READING_KEXEC_IMAGE); - if (ret) + ret = kernel_read_file_from_fd(kernel_fd, 0, &image->kernel_buf, + INT_MAX, NULL, READING_KEXEC_IMAGE); + if (ret < 0) return ret; - image->kernel_buf_len = size; + image->kernel_buf_len = ret; /* Call arch image probe handlers */ ret = arch_kexec_kernel_image_probe(image, image->kernel_buf, @@ -241,12 +241,13 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd, #endif /* It is possible that there no initramfs is being loaded */ if (!(flags & KEXEC_FILE_NO_INITRAMFS)) { - ret = kernel_read_file_from_fd(initrd_fd, &image->initrd_buf, - &size, INT_MAX, + ret = kernel_read_file_from_fd(initrd_fd, 0, &image->initrd_buf, + INT_MAX, NULL, READING_KEXEC_INITRAMFS); - if (ret) + if (ret < 0) goto out; - image->initrd_buf_len = size; + image->initrd_buf_len = ret; + ret = 0; } if (cmdline_len) { diff --git a/kernel/module.c b/kernel/module.c index c075a18103fb..9d9f2400d94e 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -18,6 +18,7 @@ #include <linux/fs.h> #include <linux/sysfs.h> #include <linux/kernel.h> +#include <linux/kernel_read_file.h> #include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/elf.h> @@ -3013,7 +3014,7 @@ static int copy_module_from_user(const void __user *umod, unsigned long len, if (info->len < sizeof(*(info->hdr))) return -ENOEXEC; - err = security_kernel_load_data(LOADING_MODULE); + err = security_kernel_load_data(LOADING_MODULE, true); if (err) return err; @@ -3023,11 +3024,17 @@ static int copy_module_from_user(const void __user *umod, unsigned long len, return -ENOMEM; if (copy_chunked_from_user(info->hdr, umod, info->len) != 0) { - vfree(info->hdr); - return -EFAULT; + err = -EFAULT; + goto out; } - return 0; + err = security_kernel_post_load_data((char *)info->hdr, info->len, + LOADING_MODULE, "init_module"); +out: + if (err) + vfree(info->hdr); + + return err; } static void free_copy(struct load_info *info) @@ -4043,8 +4050,7 @@ SYSCALL_DEFINE3(init_module, void __user *, umod, SYSCALL_DEFINE3(finit_module, int, fd, const char __user *, uargs, int, flags) { struct load_info info = { }; - loff_t size; - void *hdr; + void *hdr = NULL; int err; err = may_init_module(); @@ -4057,12 +4063,12 @@ SYSCALL_DEFINE3(finit_module, int, fd, const char __user *, uargs, int, flags) |MODULE_INIT_IGNORE_VERMAGIC)) return -EINVAL; - err = kernel_read_file_from_fd(fd, &hdr, &size, INT_MAX, + err = kernel_read_file_from_fd(fd, 0, &hdr, INT_MAX, NULL, READING_MODULE); - if (err) + if (err < 0) return err; info.hdr = hdr; - info.len = size; + info.len = err; return load_module(&info, uargs, flags); } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 25b72a73608a..4c187740c0df 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -251,6 +251,145 @@ unsigned long long ns2usecs(u64 nsec) return nsec; } +static void +trace_process_export(struct trace_export *export, + struct ring_buffer_event *event, int flag) +{ + struct trace_entry *entry; + unsigned int size = 0; + + if (export->flags & flag) { + entry = ring_buffer_event_data(event); + size = ring_buffer_event_length(event); + export->write(export, entry, size); + } +} + +static DEFINE_MUTEX(ftrace_export_lock); + +static struct trace_export __rcu *ftrace_exports_list __read_mostly; + +static DEFINE_STATIC_KEY_FALSE(trace_function_exports_enabled); +static DEFINE_STATIC_KEY_FALSE(trace_event_exports_enabled); +static DEFINE_STATIC_KEY_FALSE(trace_marker_exports_enabled); + +static inline void ftrace_exports_enable(struct trace_export *export) +{ + if (export->flags & TRACE_EXPORT_FUNCTION) + static_branch_inc(&trace_function_exports_enabled); + + if (export->flags & TRACE_EXPORT_EVENT) + static_branch_inc(&trace_event_exports_enabled); + + if (export->flags & TRACE_EXPORT_MARKER) + static_branch_inc(&trace_marker_exports_enabled); +} + +static inline void ftrace_exports_disable(struct trace_export *export) +{ + if (export->flags & TRACE_EXPORT_FUNCTION) + static_branch_dec(&trace_function_exports_enabled); + + if (export->flags & TRACE_EXPORT_EVENT) + static_branch_dec(&trace_event_exports_enabled); + + if (export->flags & TRACE_EXPORT_MARKER) + static_branch_dec(&trace_marker_exports_enabled); +} + +static void ftrace_exports(struct ring_buffer_event *event, int flag) +{ + struct trace_export *export; + + preempt_disable_notrace(); + + export = rcu_dereference_raw_check(ftrace_exports_list); + while (export) { + trace_process_export(export, event, flag); + export = rcu_dereference_raw_check(export->next); + } + + preempt_enable_notrace(); +} + +static inline void +add_trace_export(struct trace_export **list, struct trace_export *export) +{ + rcu_assign_pointer(export->next, *list); + /* + * We are entering export into the list but another + * CPU might be walking that list. We need to make sure + * the export->next pointer is valid before another CPU sees + * the export pointer included into the list. + */ + rcu_assign_pointer(*list, export); +} + +static inline int +rm_trace_export(struct trace_export **list, struct trace_export *export) +{ + struct trace_export **p; + + for (p = list; *p != NULL; p = &(*p)->next) + if (*p == export) + break; + + if (*p != export) + return -1; + + rcu_assign_pointer(*p, (*p)->next); + + return 0; +} + +static inline void +add_ftrace_export(struct trace_export **list, struct trace_export *export) +{ + ftrace_exports_enable(export); + + add_trace_export(list, export); +} + +static inline int +rm_ftrace_export(struct trace_export **list, struct trace_export *export) +{ + int ret; + + ret = rm_trace_export(list, export); + ftrace_exports_disable(export); + + return ret; +} + +int register_ftrace_export(struct trace_export *export) +{ + if (WARN_ON_ONCE(!export->write)) + return -1; + + mutex_lock(&ftrace_export_lock); + + add_ftrace_export(&ftrace_exports_list, export); + + mutex_unlock(&ftrace_export_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(register_ftrace_export); + +int unregister_ftrace_export(struct trace_export *export) +{ + int ret; + + mutex_lock(&ftrace_export_lock); + + ret = rm_ftrace_export(&ftrace_exports_list, export); + + mutex_unlock(&ftrace_export_lock); + + return ret; +} +EXPORT_SYMBOL_GPL(unregister_ftrace_export); + /* trace_flags holds trace_options default values */ #define TRACE_DEFAULT_FLAGS \ (FUNCTION_DEFAULT_FLAGS | \ @@ -2699,6 +2838,8 @@ void trace_event_buffer_commit(struct trace_event_buffer *fbuffer) if (static_key_false(&tracepoint_printk_key.key)) output_printk(fbuffer); + if (static_branch_unlikely(&trace_event_exports_enabled)) + ftrace_exports(fbuffer->event, TRACE_EXPORT_EVENT); event_trigger_unlock_commit_regs(fbuffer->trace_file, fbuffer->buffer, fbuffer->event, fbuffer->entry, fbuffer->flags, fbuffer->pc, fbuffer->regs); @@ -2742,129 +2883,6 @@ trace_buffer_unlock_commit_nostack(struct trace_buffer *buffer, __buffer_unlock_commit(buffer, event); } -static void -trace_process_export(struct trace_export *export, - struct ring_buffer_event *event) -{ - struct trace_entry *entry; - unsigned int size = 0; - - entry = ring_buffer_event_data(event); - size = ring_buffer_event_length(event); - export->write(export, entry, size); -} - -static DEFINE_MUTEX(ftrace_export_lock); - -static struct trace_export __rcu *ftrace_exports_list __read_mostly; - -static DEFINE_STATIC_KEY_FALSE(ftrace_exports_enabled); - -static inline void ftrace_exports_enable(void) -{ - static_branch_enable(&ftrace_exports_enabled); -} - -static inline void ftrace_exports_disable(void) -{ - static_branch_disable(&ftrace_exports_enabled); -} - -static void ftrace_exports(struct ring_buffer_event *event) -{ - struct trace_export *export; - - preempt_disable_notrace(); - - export = rcu_dereference_raw_check(ftrace_exports_list); - while (export) { - trace_process_export(export, event); - export = rcu_dereference_raw_check(export->next); - } - - preempt_enable_notrace(); -} - -static inline void -add_trace_export(struct trace_export **list, struct trace_export *export) -{ - rcu_assign_pointer(export->next, *list); - /* - * We are entering export into the list but another - * CPU might be walking that list. We need to make sure - * the export->next pointer is valid before another CPU sees - * the export pointer included into the list. - */ - rcu_assign_pointer(*list, export); -} - -static inline int -rm_trace_export(struct trace_export **list, struct trace_export *export) -{ - struct trace_export **p; - - for (p = list; *p != NULL; p = &(*p)->next) - if (*p == export) - break; - - if (*p != export) - return -1; - - rcu_assign_pointer(*p, (*p)->next); - - return 0; -} - -static inline void -add_ftrace_export(struct trace_export **list, struct trace_export *export) -{ - if (*list == NULL) - ftrace_exports_enable(); - - add_trace_export(list, export); -} - -static inline int -rm_ftrace_export(struct trace_export **list, struct trace_export *export) -{ - int ret; - - ret = rm_trace_export(list, export); - if (*list == NULL) - ftrace_exports_disable(); - - return ret; -} - -int register_ftrace_export(struct trace_export *export) -{ - if (WARN_ON_ONCE(!export->write)) - return -1; - - mutex_lock(&ftrace_export_lock); - - add_ftrace_export(&ftrace_exports_list, export); - - mutex_unlock(&ftrace_export_lock); - - return 0; -} -EXPORT_SYMBOL_GPL(register_ftrace_export); - -int unregister_ftrace_export(struct trace_export *export) -{ - int ret; - - mutex_lock(&ftrace_export_lock); - - ret = rm_ftrace_export(&ftrace_exports_list, export); - - mutex_unlock(&ftrace_export_lock); - - return ret; -} -EXPORT_SYMBOL_GPL(unregister_ftrace_export); - void trace_function(struct trace_array *tr, unsigned long ip, unsigned long parent_ip, unsigned long flags, @@ -2884,8 +2902,8 @@ trace_function(struct trace_array *tr, entry->parent_ip = parent_ip; if (!call_filter_check_discard(call, entry, buffer, event)) { - if (static_branch_unlikely(&ftrace_exports_enabled)) - ftrace_exports(event); + if (static_branch_unlikely(&trace_function_exports_enabled)) + ftrace_exports(event, TRACE_EXPORT_FUNCTION); __buffer_unlock_commit(buffer, event); } } @@ -6678,6 +6696,8 @@ tracing_mark_write(struct file *filp, const char __user *ubuf, } else entry->buf[cnt] = '\0'; + if (static_branch_unlikely(&trace_marker_exports_enabled)) + ftrace_exports(event, TRACE_EXPORT_MARKER); __buffer_unlock_commit(buffer, event); if (tt) diff --git a/lib/test_firmware.c b/lib/test_firmware.c index 06c955057756..2baa275a6ddf 100644 --- a/lib/test_firmware.c +++ b/lib/test_firmware.c @@ -52,6 +52,9 @@ struct test_batched_req { * @name: the name of the firmware file to look for * @into_buf: when the into_buf is used if this is true * request_firmware_into_buf() will be used instead. + * @buf_size: size of buf to allocate when into_buf is true + * @file_offset: file offset to request when calling request_firmware_into_buf + * @partial: partial read opt when calling request_firmware_into_buf * @sync_direct: when the sync trigger is used if this is true * request_firmware_direct() will be used instead. * @send_uevent: whether or not to send a uevent for async requests @@ -91,6 +94,9 @@ struct test_batched_req { struct test_config { char *name; bool into_buf; + size_t buf_size; + size_t file_offset; + bool partial; bool sync_direct; bool send_uevent; u8 num_requests; @@ -185,6 +191,9 @@ static int __test_firmware_config_init(void) test_fw_config->num_requests = TEST_FIRMWARE_NUM_REQS; test_fw_config->send_uevent = true; test_fw_config->into_buf = false; + test_fw_config->buf_size = TEST_FIRMWARE_BUF_SIZE; + test_fw_config->file_offset = 0; + test_fw_config->partial = false; test_fw_config->sync_direct = false; test_fw_config->req_firmware = request_firmware; test_fw_config->test_result = 0; @@ -238,28 +247,35 @@ static ssize_t config_show(struct device *dev, dev_name(dev)); if (test_fw_config->name) - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "name:\t%s\n", test_fw_config->name); else - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "name:\tEMTPY\n"); - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "num_requests:\t%u\n", test_fw_config->num_requests); - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "send_uevent:\t\t%s\n", test_fw_config->send_uevent ? "FW_ACTION_HOTPLUG" : "FW_ACTION_NOHOTPLUG"); - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "into_buf:\t\t%s\n", test_fw_config->into_buf ? "true" : "false"); - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, + "buf_size:\t%zu\n", test_fw_config->buf_size); + len += scnprintf(buf + len, PAGE_SIZE - len, + "file_offset:\t%zu\n", test_fw_config->file_offset); + len += scnprintf(buf + len, PAGE_SIZE - len, + "partial:\t\t%s\n", + test_fw_config->partial ? "true" : "false"); + len += scnprintf(buf + len, PAGE_SIZE - len, "sync_direct:\t\t%s\n", test_fw_config->sync_direct ? "true" : "false"); - len += scnprintf(buf+len, PAGE_SIZE - len, + len += scnprintf(buf + len, PAGE_SIZE - len, "read_fw_idx:\t%u\n", test_fw_config->read_fw_idx); mutex_unlock(&test_fw_mutex); @@ -317,6 +333,30 @@ static ssize_t test_dev_config_show_bool(char *buf, bool val) return snprintf(buf, PAGE_SIZE, "%d\n", val); } +static int test_dev_config_update_size_t(const char *buf, + size_t size, + size_t *cfg) +{ + int ret; + long new; + + ret = kstrtol(buf, 10, &new); + if (ret) + return ret; + + mutex_lock(&test_fw_mutex); + *(size_t *)cfg = new; + mutex_unlock(&test_fw_mutex); + + /* Always return full write size even if we didn't consume all */ + return size; +} + +static ssize_t test_dev_config_show_size_t(char *buf, size_t val) +{ + return snprintf(buf, PAGE_SIZE, "%zu\n", val); +} + static ssize_t test_dev_config_show_int(char *buf, int val) { return snprintf(buf, PAGE_SIZE, "%d\n", val); @@ -402,6 +442,83 @@ static ssize_t config_into_buf_show(struct device *dev, } static DEVICE_ATTR_RW(config_into_buf); +static ssize_t config_buf_size_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int rc; + + mutex_lock(&test_fw_mutex); + if (test_fw_config->reqs) { + pr_err("Must call release_all_firmware prior to changing config\n"); + rc = -EINVAL; + mutex_unlock(&test_fw_mutex); + goto out; + } + mutex_unlock(&test_fw_mutex); + + rc = test_dev_config_update_size_t(buf, count, + &test_fw_config->buf_size); + +out: + return rc; +} + +static ssize_t config_buf_size_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return test_dev_config_show_size_t(buf, test_fw_config->buf_size); +} +static DEVICE_ATTR_RW(config_buf_size); + +static ssize_t config_file_offset_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int rc; + + mutex_lock(&test_fw_mutex); + if (test_fw_config->reqs) { + pr_err("Must call release_all_firmware prior to changing config\n"); + rc = -EINVAL; + mutex_unlock(&test_fw_mutex); + goto out; + } + mutex_unlock(&test_fw_mutex); + + rc = test_dev_config_update_size_t(buf, count, + &test_fw_config->file_offset); + +out: + return rc; +} + +static ssize_t config_file_offset_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return test_dev_config_show_size_t(buf, test_fw_config->file_offset); +} +static DEVICE_ATTR_RW(config_file_offset); + +static ssize_t config_partial_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + return test_dev_config_update_bool(buf, + count, + &test_fw_config->partial); +} + +static ssize_t config_partial_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return test_dev_config_show_bool(buf, test_fw_config->partial); +} +static DEVICE_ATTR_RW(config_partial); + static ssize_t config_sync_direct_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) @@ -659,11 +776,21 @@ static int test_fw_run_batch_request(void *data) if (!test_buf) return -ENOSPC; - req->rc = request_firmware_into_buf(&req->fw, - req->name, - req->dev, - test_buf, - TEST_FIRMWARE_BUF_SIZE); + if (test_fw_config->partial) + req->rc = request_partial_firmware_into_buf + (&req->fw, + req->name, + req->dev, + test_buf, + test_fw_config->buf_size, + test_fw_config->file_offset); + else + req->rc = request_firmware_into_buf + (&req->fw, + req->name, + req->dev, + test_buf, + test_fw_config->buf_size); if (!req->fw) kfree(test_buf); } else { @@ -936,6 +1063,9 @@ static struct attribute *test_dev_attrs[] = { TEST_FW_DEV_ATTR(config_name), TEST_FW_DEV_ATTR(config_num_requests), TEST_FW_DEV_ATTR(config_into_buf), + TEST_FW_DEV_ATTR(config_buf_size), + TEST_FW_DEV_ATTR(config_file_offset), + TEST_FW_DEV_ATTR(config_partial), TEST_FW_DEV_ATTR(config_sync_direct), TEST_FW_DEV_ATTR(config_send_uevent), TEST_FW_DEV_ATTR(config_read_fw_idx), diff --git a/samples/mic/mpssd/mpssd.c b/samples/mic/mpssd/mpssd.c index a11bf6c5b53b..c03a05d498f0 100644 --- a/samples/mic/mpssd/mpssd.c +++ b/samples/mic/mpssd/mpssd.c @@ -403,9 +403,9 @@ mic_virtio_copy(struct mic_info *mic, int fd, static inline unsigned _vring_size(unsigned int num, unsigned long align) { - return ((sizeof(struct vring_desc) * num + sizeof(__u16) * (3 + num) + return _ALIGN_UP(((sizeof(struct vring_desc) * num + sizeof(__u16) * (3 + num) + align - 1) & ~(align - 1)) - + sizeof(__u16) * 3 + sizeof(struct vring_used_elem) * num; + + sizeof(__u16) * 3 + sizeof(struct vring_used_elem) * num, 4); } /* @@ -612,7 +612,7 @@ virtio_net(void *arg) copy.out_len, hdr->gso_type); #endif #ifdef DEBUG - disp_iovec(mic, copy, __func__, __LINE__); + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d read from tap 0x%lx\n", mic->name, __func__, __LINE__, len); @@ -632,7 +632,7 @@ virtio_net(void *arg) if (!err) verify_out_len(mic, ©); #ifdef DEBUG - disp_iovec(mic, copy, __func__, __LINE__); + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d wrote to net 0x%lx\n", mic->name, __func__, __LINE__, sum_iovec_len(©)); @@ -681,12 +681,12 @@ virtio_net(void *arg) sizeof(struct virtio_net_hdr); verify_out_len(mic, ©); #ifdef DEBUG - disp_iovec(mic, copy, __func__, + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d ", mic->name, __func__, __LINE__); mpsslog("read from net 0x%lx\n", - sum_iovec_len(copy)); + sum_iovec_len(©)); #endif len = writev(net_poll[NET_FD_TUN].fd, copy.iov, copy.iovcnt); @@ -814,7 +814,7 @@ virtio_console(void *arg) len = readv(pty_fd, copy.iov, copy.iovcnt); if (len > 0) { #ifdef DEBUG - disp_iovec(mic, copy, __func__, __LINE__); + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d read from tap 0x%lx\n", mic->name, __func__, __LINE__, len); @@ -834,10 +834,10 @@ virtio_console(void *arg) if (!err) verify_out_len(mic, ©); #ifdef DEBUG - disp_iovec(mic, copy, __func__, __LINE__); + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d wrote to net 0x%lx\n", mic->name, __func__, __LINE__, - sum_iovec_len(copy)); + sum_iovec_len(©)); #endif /* Reinitialize IOV for next run */ iov0->iov_len = PAGE_SIZE; @@ -866,12 +866,12 @@ virtio_console(void *arg) iov1->iov_len = copy.out_len; verify_out_len(mic, ©); #ifdef DEBUG - disp_iovec(mic, copy, __func__, + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d ", mic->name, __func__, __LINE__); mpsslog("read from net 0x%lx\n", - sum_iovec_len(copy)); + sum_iovec_len(©)); #endif len = writev(pty_fd, copy.iov, copy.iovcnt); @@ -883,7 +883,7 @@ virtio_console(void *arg) sum_iovec_len(©)); } else { #ifdef DEBUG - disp_iovec(mic, copy, __func__, + disp_iovec(mic, ©, __func__, __LINE__); mpsslog("%s %s %d ", mic->name, __func__, diff --git a/samples/nitro_enclaves/.gitignore b/samples/nitro_enclaves/.gitignore new file mode 100644 index 000000000000..827934129c90 --- /dev/null +++ b/samples/nitro_enclaves/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +ne_ioctl_sample diff --git a/samples/nitro_enclaves/Makefile b/samples/nitro_enclaves/Makefile new file mode 100644 index 000000000000..a3ec78fefb52 --- /dev/null +++ b/samples/nitro_enclaves/Makefile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +# Enclave lifetime management support for Nitro Enclaves (NE) - ioctl sample +# usage. + +.PHONY: all clean + +CFLAGS += -Wall + +all: + $(CC) $(CFLAGS) -o ne_ioctl_sample ne_ioctl_sample.c -lpthread + +clean: + rm -f ne_ioctl_sample diff --git a/samples/nitro_enclaves/ne_ioctl_sample.c b/samples/nitro_enclaves/ne_ioctl_sample.c new file mode 100644 index 000000000000..480b763142b3 --- /dev/null +++ b/samples/nitro_enclaves/ne_ioctl_sample.c @@ -0,0 +1,883 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. + */ + +/** + * DOC: Sample flow of using the ioctl interface provided by the Nitro Enclaves (NE) + * kernel driver. + * + * Usage + * ----- + * + * Load the nitro_enclaves module, setting also the enclave CPU pool. The + * enclave CPUs need to be full cores from the same NUMA node. CPU 0 and its + * siblings have to remain available for the primary / parent VM, so they + * cannot be included in the enclave CPU pool. + * + * See the cpu list section from the kernel documentation. + * https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html#cpu-lists + * + * insmod drivers/virt/nitro_enclaves/nitro_enclaves.ko + * lsmod + * + * The CPU pool can be set at runtime, after the kernel module is loaded. + * + * echo <cpu-list> > /sys/module/nitro_enclaves/parameters/ne_cpus + * + * NUMA and CPU siblings information can be found using: + * + * lscpu + * /proc/cpuinfo + * + * Check the online / offline CPU list. The CPUs from the pool should be + * offlined. + * + * lscpu + * + * Check dmesg for any warnings / errors through the NE driver lifetime / usage. + * The NE logs contain the "nitro_enclaves" or "pci 0000:00:02.0" pattern. + * + * dmesg + * + * Setup hugetlbfs huge pages. The memory needs to be from the same NUMA node as + * the enclave CPUs. + * + * https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html + * + * By default, the allocation of hugetlb pages are distributed on all possible + * NUMA nodes. Use the following configuration files to set the number of huge + * pages from a NUMA node: + * + * /sys/devices/system/node/node<X>/hugepages/hugepages-2048kB/nr_hugepages + * /sys/devices/system/node/node<X>/hugepages/hugepages-1048576kB/nr_hugepages + * + * or, if not on a system with multiple NUMA nodes, can also set the number + * of 2 MiB / 1 GiB huge pages using + * + * /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + * /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages + * + * In this example 256 hugepages of 2 MiB are used. + * + * Build and run the NE sample. + * + * make -C samples/nitro_enclaves clean + * make -C samples/nitro_enclaves + * ./samples/nitro_enclaves/ne_ioctl_sample <path_to_enclave_image> + * + * Unload the nitro_enclaves module. + * + * rmmod nitro_enclaves + * lsmod + */ + +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> +#include <fcntl.h> +#include <limits.h> +#include <poll.h> +#include <pthread.h> +#include <string.h> +#include <sys/eventfd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/socket.h> +#include <sys/stat.h> +#include <sys/types.h> +#include <unistd.h> + +#include <linux/mman.h> +#include <linux/nitro_enclaves.h> +#include <linux/vm_sockets.h> + +/** + * NE_DEV_NAME - Nitro Enclaves (NE) misc device that provides the ioctl interface. + */ +#define NE_DEV_NAME "/dev/nitro_enclaves" + +/** + * NE_POLL_WAIT_TIME - Timeout in seconds for each poll event. + */ +#define NE_POLL_WAIT_TIME (60) +/** + * NE_POLL_WAIT_TIME_MS - Timeout in milliseconds for each poll event. + */ +#define NE_POLL_WAIT_TIME_MS (NE_POLL_WAIT_TIME * 1000) + +/** + * NE_SLEEP_TIME - Amount of time in seconds for the process to keep the enclave alive. + */ +#define NE_SLEEP_TIME (300) + +/** + * NE_DEFAULT_NR_VCPUS - Default number of vCPUs set for an enclave. + */ +#define NE_DEFAULT_NR_VCPUS (2) + +/** + * NE_MIN_MEM_REGION_SIZE - Minimum size of a memory region - 2 MiB. + */ +#define NE_MIN_MEM_REGION_SIZE (2 * 1024 * 1024) + +/** + * NE_DEFAULT_NR_MEM_REGIONS - Default number of memory regions of 2 MiB set for + * an enclave. + */ +#define NE_DEFAULT_NR_MEM_REGIONS (256) + +/** + * NE_IMAGE_LOAD_HEARTBEAT_CID - Vsock CID for enclave image loading heartbeat logic. + */ +#define NE_IMAGE_LOAD_HEARTBEAT_CID (3) +/** + * NE_IMAGE_LOAD_HEARTBEAT_PORT - Vsock port for enclave image loading heartbeat logic. + */ +#define NE_IMAGE_LOAD_HEARTBEAT_PORT (9000) +/** + * NE_IMAGE_LOAD_HEARTBEAT_VALUE - Heartbeat value for enclave image loading. + */ +#define NE_IMAGE_LOAD_HEARTBEAT_VALUE (0xb7) + +/** + * struct ne_user_mem_region - User space memory region set for an enclave. + * @userspace_addr: Address of the user space memory region. + * @memory_size: Size of the user space memory region. + */ +struct ne_user_mem_region { + void *userspace_addr; + size_t memory_size; +}; + +/** + * ne_create_vm() - Create a slot for the enclave VM. + * @ne_dev_fd: The file descriptor of the NE misc device. + * @slot_uid: The generated slot uid for the enclave. + * @enclave_fd : The generated file descriptor for the enclave. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_create_vm(int ne_dev_fd, unsigned long *slot_uid, int *enclave_fd) +{ + int rc = -EINVAL; + *enclave_fd = ioctl(ne_dev_fd, NE_CREATE_VM, slot_uid); + + if (*enclave_fd < 0) { + rc = *enclave_fd; + switch (errno) { + case NE_ERR_NO_CPUS_AVAIL_IN_POOL: { + printf("Error in create VM, no CPUs available in the NE CPU pool\n"); + + break; + } + + default: + printf("Error in create VM [%m]\n"); + } + + return rc; + } + + return 0; +} + + +/** + * ne_poll_enclave_fd() - Thread function for polling the enclave fd. + * @data: Argument provided for the polling function. + * + * Context: Process context. + * Return: + * * NULL on success / failure. + */ +void *ne_poll_enclave_fd(void *data) +{ + int enclave_fd = *(int *)data; + struct pollfd fds[1] = {}; + int i = 0; + int rc = -EINVAL; + + printf("Running from poll thread, enclave fd %d\n", enclave_fd); + + fds[0].fd = enclave_fd; + fds[0].events = POLLIN | POLLERR | POLLHUP; + + /* Keep on polling until the current process is terminated. */ + while (1) { + printf("[iter %d] Polling ...\n", i); + + rc = poll(fds, 1, NE_POLL_WAIT_TIME_MS); + if (rc < 0) { + printf("Error in poll [%m]\n"); + + return NULL; + } + + i++; + + if (!rc) { + printf("Poll: %d seconds elapsed\n", + i * NE_POLL_WAIT_TIME); + + continue; + } + + printf("Poll received value 0x%x\n", fds[0].revents); + + if (fds[0].revents & POLLHUP) { + printf("Received POLLHUP\n"); + + return NULL; + } + + if (fds[0].revents & POLLNVAL) { + printf("Received POLLNVAL\n"); + + return NULL; + } + } + + return NULL; +} + +/** + * ne_alloc_user_mem_region() - Allocate a user space memory region for an enclave. + * @ne_user_mem_region: User space memory region allocated using hugetlbfs. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_alloc_user_mem_region(struct ne_user_mem_region *ne_user_mem_region) +{ + /** + * Check available hugetlb encodings for different huge page sizes in + * include/uapi/linux/mman.h. + */ + ne_user_mem_region->userspace_addr = mmap(NULL, ne_user_mem_region->memory_size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | + MAP_HUGETLB | MAP_HUGE_2MB, -1, 0); + if (ne_user_mem_region->userspace_addr == MAP_FAILED) { + printf("Error in mmap memory [%m]\n"); + + return -1; + } + + return 0; +} + +/** + * ne_load_enclave_image() - Place the enclave image in the enclave memory. + * @enclave_fd : The file descriptor associated with the enclave. + * @ne_user_mem_regions: User space memory regions allocated for the enclave. + * @enclave_image_path : The file path of the enclave image. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_load_enclave_image(int enclave_fd, struct ne_user_mem_region ne_user_mem_regions[], + char *enclave_image_path) +{ + unsigned char *enclave_image = NULL; + int enclave_image_fd = -1; + size_t enclave_image_size = 0; + size_t enclave_memory_size = 0; + unsigned long i = 0; + size_t image_written_bytes = 0; + struct ne_image_load_info image_load_info = { + .flags = NE_EIF_IMAGE, + }; + struct stat image_stat_buf = {}; + int rc = -EINVAL; + size_t temp_image_offset = 0; + + for (i = 0; i < NE_DEFAULT_NR_MEM_REGIONS; i++) + enclave_memory_size += ne_user_mem_regions[i].memory_size; + + rc = stat(enclave_image_path, &image_stat_buf); + if (rc < 0) { + printf("Error in get image stat info [%m]\n"); + + return rc; + } + + enclave_image_size = image_stat_buf.st_size; + + if (enclave_memory_size < enclave_image_size) { + printf("The enclave memory is smaller than the enclave image size\n"); + + return -ENOMEM; + } + + rc = ioctl(enclave_fd, NE_GET_IMAGE_LOAD_INFO, &image_load_info); + if (rc < 0) { + switch (errno) { + case NE_ERR_NOT_IN_INIT_STATE: { + printf("Error in get image load info, enclave not in init state\n"); + + break; + } + + case NE_ERR_INVALID_FLAG_VALUE: { + printf("Error in get image load info, provided invalid flag\n"); + + break; + } + + default: + printf("Error in get image load info [%m]\n"); + } + + return rc; + } + + printf("Enclave image offset in enclave memory is %lld\n", + image_load_info.memory_offset); + + enclave_image_fd = open(enclave_image_path, O_RDONLY); + if (enclave_image_fd < 0) { + printf("Error in open enclave image file [%m]\n"); + + return enclave_image_fd; + } + + enclave_image = mmap(NULL, enclave_image_size, PROT_READ, + MAP_PRIVATE, enclave_image_fd, 0); + if (enclave_image == MAP_FAILED) { + printf("Error in mmap enclave image [%m]\n"); + + return -1; + } + + temp_image_offset = image_load_info.memory_offset; + + for (i = 0; i < NE_DEFAULT_NR_MEM_REGIONS; i++) { + size_t bytes_to_write = 0; + size_t memory_offset = 0; + size_t memory_size = ne_user_mem_regions[i].memory_size; + size_t remaining_bytes = 0; + void *userspace_addr = ne_user_mem_regions[i].userspace_addr; + + if (temp_image_offset >= memory_size) { + temp_image_offset -= memory_size; + + continue; + } else if (temp_image_offset != 0) { + memory_offset = temp_image_offset; + memory_size -= temp_image_offset; + temp_image_offset = 0; + } + + remaining_bytes = enclave_image_size - image_written_bytes; + bytes_to_write = memory_size < remaining_bytes ? + memory_size : remaining_bytes; + + memcpy(userspace_addr + memory_offset, + enclave_image + image_written_bytes, bytes_to_write); + + image_written_bytes += bytes_to_write; + + if (image_written_bytes == enclave_image_size) + break; + } + + munmap(enclave_image, enclave_image_size); + + close(enclave_image_fd); + + return 0; +} + +/** + * ne_set_user_mem_region() - Set a user space memory region for the given enclave. + * @enclave_fd : The file descriptor associated with the enclave. + * @ne_user_mem_region : User space memory region to be set for the enclave. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_set_user_mem_region(int enclave_fd, struct ne_user_mem_region ne_user_mem_region) +{ + struct ne_user_memory_region mem_region = { + .flags = NE_DEFAULT_MEMORY_REGION, + .memory_size = ne_user_mem_region.memory_size, + .userspace_addr = (__u64)ne_user_mem_region.userspace_addr, + }; + int rc = -EINVAL; + + rc = ioctl(enclave_fd, NE_SET_USER_MEMORY_REGION, &mem_region); + if (rc < 0) { + switch (errno) { + case NE_ERR_NOT_IN_INIT_STATE: { + printf("Error in set user memory region, enclave not in init state\n"); + + break; + } + + case NE_ERR_INVALID_MEM_REGION_SIZE: { + printf("Error in set user memory region, mem size not multiple of 2 MiB\n"); + + break; + } + + case NE_ERR_INVALID_MEM_REGION_ADDR: { + printf("Error in set user memory region, invalid user space address\n"); + + break; + } + + case NE_ERR_UNALIGNED_MEM_REGION_ADDR: { + printf("Error in set user memory region, unaligned user space address\n"); + + break; + } + + case NE_ERR_MEM_REGION_ALREADY_USED: { + printf("Error in set user memory region, memory region already used\n"); + + break; + } + + case NE_ERR_MEM_NOT_HUGE_PAGE: { + printf("Error in set user memory region, not backed by huge pages\n"); + + break; + } + + case NE_ERR_MEM_DIFFERENT_NUMA_NODE: { + printf("Error in set user memory region, different NUMA node than CPUs\n"); + + break; + } + + case NE_ERR_MEM_MAX_REGIONS: { + printf("Error in set user memory region, max memory regions reached\n"); + + break; + } + + case NE_ERR_INVALID_PAGE_SIZE: { + printf("Error in set user memory region, has page not multiple of 2 MiB\n"); + + break; + } + + case NE_ERR_INVALID_FLAG_VALUE: { + printf("Error in set user memory region, provided invalid flag\n"); + + break; + } + + default: + printf("Error in set user memory region [%m]\n"); + } + + return rc; + } + + return 0; +} + +/** + * ne_free_mem_regions() - Unmap all the user space memory regions that were set + * aside for the enclave. + * @ne_user_mem_regions: The user space memory regions associated with an enclave. + * + * Context: Process context. + */ +static void ne_free_mem_regions(struct ne_user_mem_region ne_user_mem_regions[]) +{ + unsigned int i = 0; + + for (i = 0; i < NE_DEFAULT_NR_MEM_REGIONS; i++) + munmap(ne_user_mem_regions[i].userspace_addr, + ne_user_mem_regions[i].memory_size); +} + +/** + * ne_add_vcpu() - Add a vCPU to the given enclave. + * @enclave_fd : The file descriptor associated with the enclave. + * @vcpu_id: vCPU id to be set for the enclave, either provided or + * auto-generated (if provided vCPU id is 0). + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_add_vcpu(int enclave_fd, unsigned int *vcpu_id) +{ + int rc = -EINVAL; + + rc = ioctl(enclave_fd, NE_ADD_VCPU, vcpu_id); + if (rc < 0) { + switch (errno) { + case NE_ERR_NO_CPUS_AVAIL_IN_POOL: { + printf("Error in add vcpu, no CPUs available in the NE CPU pool\n"); + + break; + } + + case NE_ERR_VCPU_ALREADY_USED: { + printf("Error in add vcpu, the provided vCPU is already used\n"); + + break; + } + + case NE_ERR_VCPU_NOT_IN_CPU_POOL: { + printf("Error in add vcpu, the provided vCPU is not in the NE CPU pool\n"); + + break; + } + + case NE_ERR_VCPU_INVALID_CPU_CORE: { + printf("Error in add vcpu, the core id of the provided vCPU is invalid\n"); + + break; + } + + case NE_ERR_NOT_IN_INIT_STATE: { + printf("Error in add vcpu, enclave not in init state\n"); + + break; + } + + case NE_ERR_INVALID_VCPU: { + printf("Error in add vcpu, the provided vCPU is out of avail CPUs range\n"); + + break; + } + + default: + printf("Error in add vcpu [%m]\n"); + + } + return rc; + } + + return 0; +} + +/** + * ne_start_enclave() - Start the given enclave. + * @enclave_fd : The file descriptor associated with the enclave. + * @enclave_start_info : Enclave metadata used for starting e.g. vsock CID. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_start_enclave(int enclave_fd, struct ne_enclave_start_info *enclave_start_info) +{ + int rc = -EINVAL; + + rc = ioctl(enclave_fd, NE_START_ENCLAVE, enclave_start_info); + if (rc < 0) { + switch (errno) { + case NE_ERR_NOT_IN_INIT_STATE: { + printf("Error in start enclave, enclave not in init state\n"); + + break; + } + + case NE_ERR_NO_MEM_REGIONS_ADDED: { + printf("Error in start enclave, no memory regions have been added\n"); + + break; + } + + case NE_ERR_NO_VCPUS_ADDED: { + printf("Error in start enclave, no vCPUs have been added\n"); + + break; + } + + case NE_ERR_FULL_CORES_NOT_USED: { + printf("Error in start enclave, enclave has no full cores set\n"); + + break; + } + + case NE_ERR_ENCLAVE_MEM_MIN_SIZE: { + printf("Error in start enclave, enclave memory is less than min size\n"); + + break; + } + + case NE_ERR_INVALID_FLAG_VALUE: { + printf("Error in start enclave, provided invalid flag\n"); + + break; + } + + case NE_ERR_INVALID_ENCLAVE_CID: { + printf("Error in start enclave, provided invalid enclave CID\n"); + + break; + } + + default: + printf("Error in start enclave [%m]\n"); + } + + return rc; + } + + return 0; +} + +/** + * ne_start_enclave_check_booted() - Start the enclave and wait for a hearbeat + * from it, on a newly created vsock channel, + * to check it has booted. + * @enclave_fd : The file descriptor associated with the enclave. + * + * Context: Process context. + * Return: + * * 0 on success. + * * Negative return value on failure. + */ +static int ne_start_enclave_check_booted(int enclave_fd) +{ + struct sockaddr_vm client_vsock_addr = {}; + int client_vsock_fd = -1; + socklen_t client_vsock_len = sizeof(client_vsock_addr); + struct ne_enclave_start_info enclave_start_info = {}; + struct pollfd fds[1] = {}; + int rc = -EINVAL; + unsigned char recv_buf = 0; + struct sockaddr_vm server_vsock_addr = { + .svm_family = AF_VSOCK, + .svm_cid = NE_IMAGE_LOAD_HEARTBEAT_CID, + .svm_port = NE_IMAGE_LOAD_HEARTBEAT_PORT, + }; + int server_vsock_fd = -1; + + server_vsock_fd = socket(AF_VSOCK, SOCK_STREAM, 0); + if (server_vsock_fd < 0) { + rc = server_vsock_fd; + + printf("Error in socket [%m]\n"); + + return rc; + } + + rc = bind(server_vsock_fd, (struct sockaddr *)&server_vsock_addr, + sizeof(server_vsock_addr)); + if (rc < 0) { + printf("Error in bind [%m]\n"); + + goto out; + } + + rc = listen(server_vsock_fd, 1); + if (rc < 0) { + printf("Error in listen [%m]\n"); + + goto out; + } + + rc = ne_start_enclave(enclave_fd, &enclave_start_info); + if (rc < 0) + goto out; + + printf("Enclave started, CID %llu\n", enclave_start_info.enclave_cid); + + fds[0].fd = server_vsock_fd; + fds[0].events = POLLIN; + + rc = poll(fds, 1, NE_POLL_WAIT_TIME_MS); + if (rc < 0) { + printf("Error in poll [%m]\n"); + + goto out; + } + + if (!rc) { + printf("Poll timeout, %d seconds elapsed\n", NE_POLL_WAIT_TIME); + + rc = -ETIMEDOUT; + + goto out; + } + + if ((fds[0].revents & POLLIN) == 0) { + printf("Poll received value %d\n", fds[0].revents); + + rc = -EINVAL; + + goto out; + } + + rc = accept(server_vsock_fd, (struct sockaddr *)&client_vsock_addr, + &client_vsock_len); + if (rc < 0) { + printf("Error in accept [%m]\n"); + + goto out; + } + + client_vsock_fd = rc; + + /* + * Read the heartbeat value that the init process in the enclave sends + * after vsock connect. + */ + rc = read(client_vsock_fd, &recv_buf, sizeof(recv_buf)); + if (rc < 0) { + printf("Error in read [%m]\n"); + + goto out; + } + + if (rc != sizeof(recv_buf) || recv_buf != NE_IMAGE_LOAD_HEARTBEAT_VALUE) { + printf("Read %d instead of %d\n", recv_buf, + NE_IMAGE_LOAD_HEARTBEAT_VALUE); + + goto out; + } + + /* Write the heartbeat value back. */ + rc = write(client_vsock_fd, &recv_buf, sizeof(recv_buf)); + if (rc < 0) { + printf("Error in write [%m]\n"); + + goto out; + } + + rc = 0; + +out: + close(server_vsock_fd); + + return rc; +} + +int main(int argc, char *argv[]) +{ + int enclave_fd = -1; + unsigned int i = 0; + int ne_dev_fd = -1; + struct ne_user_mem_region ne_user_mem_regions[NE_DEFAULT_NR_MEM_REGIONS] = {}; + unsigned int ne_vcpus[NE_DEFAULT_NR_VCPUS] = {}; + int rc = -EINVAL; + pthread_t thread_id = 0; + unsigned long slot_uid = 0; + + if (argc != 2) { + printf("Usage: %s <path_to_enclave_image>\n", argv[0]); + + exit(EXIT_FAILURE); + } + + if (strlen(argv[1]) >= PATH_MAX) { + printf("The size of the path to enclave image is higher than max path\n"); + + exit(EXIT_FAILURE); + } + + ne_dev_fd = open(NE_DEV_NAME, O_RDWR | O_CLOEXEC); + if (ne_dev_fd < 0) { + printf("Error in open NE device [%m]\n"); + + exit(EXIT_FAILURE); + } + + printf("Creating enclave slot ...\n"); + + rc = ne_create_vm(ne_dev_fd, &slot_uid, &enclave_fd); + + close(ne_dev_fd); + + if (rc < 0) + exit(EXIT_FAILURE); + + printf("Enclave fd %d\n", enclave_fd); + + rc = pthread_create(&thread_id, NULL, ne_poll_enclave_fd, (void *)&enclave_fd); + if (rc < 0) { + printf("Error in thread create [%m]\n"); + + close(enclave_fd); + + exit(EXIT_FAILURE); + } + + for (i = 0; i < NE_DEFAULT_NR_MEM_REGIONS; i++) { + ne_user_mem_regions[i].memory_size = NE_MIN_MEM_REGION_SIZE; + + rc = ne_alloc_user_mem_region(&ne_user_mem_regions[i]); + if (rc < 0) { + printf("Error in alloc userspace memory region, iter %d\n", i); + + goto release_enclave_fd; + } + } + + rc = ne_load_enclave_image(enclave_fd, ne_user_mem_regions, argv[1]); + if (rc < 0) + goto release_enclave_fd; + + for (i = 0; i < NE_DEFAULT_NR_MEM_REGIONS; i++) { + rc = ne_set_user_mem_region(enclave_fd, ne_user_mem_regions[i]); + if (rc < 0) { + printf("Error in set memory region, iter %d\n", i); + + goto release_enclave_fd; + } + } + + printf("Enclave memory regions were added\n"); + + for (i = 0; i < NE_DEFAULT_NR_VCPUS; i++) { + /* + * The vCPU is chosen from the enclave vCPU pool, if the value + * of the vcpu_id is 0. + */ + ne_vcpus[i] = 0; + rc = ne_add_vcpu(enclave_fd, &ne_vcpus[i]); + if (rc < 0) { + printf("Error in add vcpu, iter %d\n", i); + + goto release_enclave_fd; + } + + printf("Added vCPU %d to the enclave\n", ne_vcpus[i]); + } + + printf("Enclave vCPUs were added\n"); + + rc = ne_start_enclave_check_booted(enclave_fd); + if (rc < 0) { + printf("Error in the enclave start / image loading heartbeat logic [rc=%d]\n", rc); + + goto release_enclave_fd; + } + + printf("Entering sleep for %d seconds ...\n", NE_SLEEP_TIME); + + sleep(NE_SLEEP_TIME); + + close(enclave_fd); + + ne_free_mem_regions(ne_user_mem_regions); + + exit(EXIT_SUCCESS); + +release_enclave_fd: + close(enclave_fd); + ne_free_mem_regions(ne_user_mem_regions); + + exit(EXIT_FAILURE); +} diff --git a/security/integrity/digsig.c b/security/integrity/digsig.c index e9cbadade74b..0f518dcfde05 100644 --- a/security/integrity/digsig.c +++ b/security/integrity/digsig.c @@ -10,6 +10,7 @@ #include <linux/sched.h> #include <linux/slab.h> #include <linux/cred.h> +#include <linux/kernel_read_file.h> #include <linux/key-type.h> #include <linux/digsig.h> #include <linux/vmalloc.h> @@ -169,17 +170,18 @@ int __init integrity_add_key(const unsigned int id, const void *data, int __init integrity_load_x509(const unsigned int id, const char *path) { - void *data; - loff_t size; + void *data = NULL; + size_t size; int rc; key_perm_t perm; - rc = kernel_read_file_from_path(path, &data, &size, 0, + rc = kernel_read_file_from_path(path, 0, &data, INT_MAX, NULL, READING_X509_CERTIFICATE); if (rc < 0) { pr_err("Unable to open file: %s (%d)", path, rc); return rc; } + size = rc; perm = (KEY_POS_ALL & ~KEY_POS_SETATTR) | KEY_USR_VIEW | KEY_USR_READ; diff --git a/security/integrity/ima/ima_fs.c b/security/integrity/ima/ima_fs.c index e3fcad871861..ea8ff8a07b36 100644 --- a/security/integrity/ima/ima_fs.c +++ b/security/integrity/ima/ima_fs.c @@ -13,6 +13,7 @@ */ #include <linux/fcntl.h> +#include <linux/kernel_read_file.h> #include <linux/slab.h> #include <linux/init.h> #include <linux/seq_file.h> @@ -272,9 +273,9 @@ static const struct file_operations ima_ascii_measurements_ops = { static ssize_t ima_read_policy(char *path) { - void *data; + void *data = NULL; char *datap; - loff_t size; + size_t size; int rc, pathlen = strlen(path); char *p; @@ -283,11 +284,14 @@ static ssize_t ima_read_policy(char *path) datap = path; strsep(&datap, "\n"); - rc = kernel_read_file_from_path(path, &data, &size, 0, READING_POLICY); + rc = kernel_read_file_from_path(path, 0, &data, INT_MAX, NULL, + READING_POLICY); if (rc < 0) { pr_err("Unable to open file: %s (%d)", path, rc); return rc; } + size = rc; + rc = 0; datap = data; while (size > 0 && (p = strsep(&datap, "\n"))) { diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c index 8a91711ca79b..82c9d62bcb11 100644 --- a/security/integrity/ima/ima_main.c +++ b/security/integrity/ima/ima_main.c @@ -18,6 +18,7 @@ #include <linux/module.h> #include <linux/file.h> #include <linux/binfmts.h> +#include <linux/kernel_read_file.h> #include <linux/mount.h> #include <linux/mman.h> #include <linux/slab.h> @@ -601,6 +602,7 @@ void ima_post_path_mknod(struct dentry *dentry) * ima_read_file - pre-measure/appraise hook decision based on policy * @file: pointer to the file to be measured/appraised/audit * @read_id: caller identifier + * @contents: whether a subsequent call will be made to ima_post_read_file() * * Permit reading a file based on policy. The policy rules are written * in terms of the policy identifier. Appraising the integrity of @@ -608,22 +610,37 @@ void ima_post_path_mknod(struct dentry *dentry) * * For permission return 0, otherwise return -EACCES. */ -int ima_read_file(struct file *file, enum kernel_read_file_id read_id) +int ima_read_file(struct file *file, enum kernel_read_file_id read_id, + bool contents) { + enum ima_hooks func; + u32 secid; + /* - * READING_FIRMWARE_PREALLOC_BUFFER - * * Do devices using pre-allocated memory run the risk of the * firmware being accessible to the device prior to the completion * of IMA's signature verification any more than when using two - * buffers? + * buffers? It may be desirable to include the buffer address + * in this API and walk all the dma_map_single() mappings to check. */ - return 0; + + /* + * There will be a call made to ima_post_read_file() with + * a filled buffer, so we don't need to perform an extra + * read early here. + */ + if (contents) + return 0; + + /* Read entire file for all partial reads. */ + func = read_idmap[read_id] ?: FILE_CHECK; + security_task_getsecid(current, &secid); + return process_measurement(file, current_cred(), secid, NULL, + 0, MAY_READ, func); } const int read_idmap[READING_MAX_ID] = { [READING_FIRMWARE] = FIRMWARE_CHECK, - [READING_FIRMWARE_PREALLOC_BUFFER] = FIRMWARE_CHECK, [READING_MODULE] = MODULE_CHECK, [READING_KEXEC_IMAGE] = KEXEC_KERNEL_CHECK, [READING_KEXEC_INITRAMFS] = KEXEC_INITRAMFS_CHECK, @@ -649,15 +666,6 @@ int ima_post_read_file(struct file *file, void *buf, loff_t size, enum ima_hooks func; u32 secid; - if (!file && read_id == READING_FIRMWARE) { - if ((ima_appraise & IMA_APPRAISE_FIRMWARE) && - (ima_appraise & IMA_APPRAISE_ENFORCE)) { - pr_err("Prevent firmware loading_store.\n"); - return -EACCES; /* INTEGRITY_UNKNOWN */ - } - return 0; - } - /* permit signed certs */ if (!file && read_id == READING_X509_CERTIFICATE) return 0; @@ -677,6 +685,8 @@ int ima_post_read_file(struct file *file, void *buf, loff_t size, /** * ima_load_data - appraise decision based on policy * @id: kernel load data caller identifier + * @contents: whether the full contents will be available in a later + * call to ima_post_load_data(). * * Callers of this LSM hook can not measure, appraise, or audit the * data provided by userspace. Enforce policy rules requring a file @@ -684,7 +694,7 @@ int ima_post_read_file(struct file *file, void *buf, loff_t size, * * For permission return 0, otherwise return -EACCES. */ -int ima_load_data(enum kernel_load_data_id id) +int ima_load_data(enum kernel_load_data_id id, bool contents) { bool ima_enforce, sig_enforce; @@ -705,7 +715,7 @@ int ima_load_data(enum kernel_load_data_id id) } break; case LOADING_FIRMWARE: - if (ima_enforce && (ima_appraise & IMA_APPRAISE_FIRMWARE)) { + if (ima_enforce && (ima_appraise & IMA_APPRAISE_FIRMWARE) && !contents) { pr_err("Prevent firmware sysfs fallback loading.\n"); return -EACCES; /* INTEGRITY_UNKNOWN */ } @@ -724,6 +734,35 @@ int ima_load_data(enum kernel_load_data_id id) return 0; } +/** + * ima_post_load_data - appraise decision based on policy + * @buf: pointer to in memory file contents + * @size: size of in memory file contents + * @id: kernel load data caller identifier + * @description: @id-specific description of contents + * + * Measure/appraise/audit in memory buffer based on policy. Policy rules + * are written in terms of a policy identifier. + * + * On success return 0. On integrity appraisal error, assuming the file + * is in policy and IMA-appraisal is in enforcing mode, return -EACCES. + */ +int ima_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id load_id, + char *description) +{ + if (load_id == LOADING_FIRMWARE) { + if ((ima_appraise & IMA_APPRAISE_FIRMWARE) && + (ima_appraise & IMA_APPRAISE_ENFORCE)) { + pr_err("Prevent firmware loading_store.\n"); + return -EACCES; /* INTEGRITY_UNKNOWN */ + } + return 0; + } + + return 0; +} + /* * process_buffer_measurement - Measure the buffer to ima log. * @inode: inode associated with the object being measured (NULL for KEY_CHECK) diff --git a/security/integrity/ima/ima_policy.c b/security/integrity/ima/ima_policy.c index b4de33074b37..3b0b43e18ecf 100644 --- a/security/integrity/ima/ima_policy.c +++ b/security/integrity/ima/ima_policy.c @@ -9,6 +9,7 @@ #include <linux/init.h> #include <linux/list.h> +#include <linux/kernel_read_file.h> #include <linux/fs.h> #include <linux/security.h> #include <linux/magic.h> diff --git a/security/loadpin/loadpin.c b/security/loadpin/loadpin.c index 670a1aebb8a1..b12f7d986b1e 100644 --- a/security/loadpin/loadpin.c +++ b/security/loadpin/loadpin.c @@ -11,6 +11,7 @@ #include <linux/module.h> #include <linux/fs.h> +#include <linux/kernel_read_file.h> #include <linux/lsm_hooks.h> #include <linux/mount.h> #include <linux/blkdev.h> @@ -117,11 +118,21 @@ static void loadpin_sb_free_security(struct super_block *mnt_sb) } } -static int loadpin_read_file(struct file *file, enum kernel_read_file_id id) +static int loadpin_read_file(struct file *file, enum kernel_read_file_id id, + bool contents) { struct super_block *load_root; const char *origin = kernel_read_file_id_str(id); + /* + * If we will not know that we'll be seeing the full contents + * then we cannot trust a load will be complete and unchanged + * off disk. Treat all contents=false hooks as if there were + * no associated file struct. + */ + if (!contents) + file = NULL; + /* If the file id is excluded, ignore the pinning. */ if ((unsigned int)id < ARRAY_SIZE(ignore_read_file_id) && ignore_read_file_id[id]) { @@ -176,9 +187,9 @@ static int loadpin_read_file(struct file *file, enum kernel_read_file_id id) return 0; } -static int loadpin_load_data(enum kernel_load_data_id id) +static int loadpin_load_data(enum kernel_load_data_id id, bool contents) { - return loadpin_read_file(NULL, (enum kernel_read_file_id) id); + return loadpin_read_file(NULL, (enum kernel_read_file_id) id, contents); } static struct security_hook_list loadpin_hooks[] __lsm_ro_after_init = { diff --git a/security/security.c b/security/security.c index 70a7ad357bc6..a28045dc9e7f 100644 --- a/security/security.c +++ b/security/security.c @@ -16,6 +16,7 @@ #include <linux/export.h> #include <linux/init.h> #include <linux/kernel.h> +#include <linux/kernel_read_file.h> #include <linux/lsm_hooks.h> #include <linux/integrity.h> #include <linux/ima.h> @@ -1671,14 +1672,15 @@ int security_kernel_module_request(char *kmod_name) return integrity_kernel_module_request(kmod_name); } -int security_kernel_read_file(struct file *file, enum kernel_read_file_id id) +int security_kernel_read_file(struct file *file, enum kernel_read_file_id id, + bool contents) { int ret; - ret = call_int_hook(kernel_read_file, 0, file, id); + ret = call_int_hook(kernel_read_file, 0, file, id, contents); if (ret) return ret; - return ima_read_file(file, id); + return ima_read_file(file, id, contents); } EXPORT_SYMBOL_GPL(security_kernel_read_file); @@ -1694,17 +1696,31 @@ int security_kernel_post_read_file(struct file *file, char *buf, loff_t size, } EXPORT_SYMBOL_GPL(security_kernel_post_read_file); -int security_kernel_load_data(enum kernel_load_data_id id) +int security_kernel_load_data(enum kernel_load_data_id id, bool contents) { int ret; - ret = call_int_hook(kernel_load_data, 0, id); + ret = call_int_hook(kernel_load_data, 0, id, contents); if (ret) return ret; - return ima_load_data(id); + return ima_load_data(id, contents); } EXPORT_SYMBOL_GPL(security_kernel_load_data); +int security_kernel_post_load_data(char *buf, loff_t size, + enum kernel_load_data_id id, + char *description) +{ + int ret; + + ret = call_int_hook(kernel_post_load_data, 0, buf, size, id, + description); + if (ret) + return ret; + return ima_post_load_data(buf, size, id, description); +} +EXPORT_SYMBOL_GPL(security_kernel_post_load_data); + int security_task_fix_setuid(struct cred *new, const struct cred *old, int flags) { diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 2dabd58b126a..6b1826fc3658 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -24,6 +24,7 @@ #include <linux/init.h> #include <linux/kd.h> #include <linux/kernel.h> +#include <linux/kernel_read_file.h> #include <linux/tracehook.h> #include <linux/errno.h> #include <linux/sched/signal.h> @@ -4005,13 +4006,14 @@ static int selinux_kernel_module_from_file(struct file *file) } static int selinux_kernel_read_file(struct file *file, - enum kernel_read_file_id id) + enum kernel_read_file_id id, + bool contents) { int rc = 0; switch (id) { case READING_MODULE: - rc = selinux_kernel_module_from_file(file); + rc = selinux_kernel_module_from_file(contents ? file : NULL); break; default: break; @@ -4020,7 +4022,7 @@ static int selinux_kernel_read_file(struct file *file, return rc; } -static int selinux_kernel_load_data(enum kernel_load_data_id id) +static int selinux_kernel_load_data(enum kernel_load_data_id id, bool contents) { int rc = 0; diff --git a/sound/soc/codecs/max98373-sdw.c b/sound/soc/codecs/max98373-sdw.c index e4675cfff7b2..13f74499e46e 100644 --- a/sound/soc/codecs/max98373-sdw.c +++ b/sound/soc/codecs/max98373-sdw.c @@ -15,6 +15,7 @@ #include <linux/of.h> #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> +#include <linux/soundwire/sdw_registers.h> #include "max98373.h" #include "max98373-sdw.h" @@ -282,11 +283,13 @@ static const struct dev_pm_ops max98373_pm = { static int max98373_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_BUS_CLASH | SDW_SCP_INT1_PARITY; + /* BITMAP: 00001000 Dataport 3 is active */ prop->source_ports = BIT(3); /* BITMAP: 00000010 Dataport 1 is active */ @@ -295,7 +298,6 @@ static int max98373_read_prop(struct sdw_slave *slave) prop->clk_stop_timeout = 20; nval = hweight32(prop->source_ports); - num_of_ports = nval; prop->src_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->src_dpn_prop), GFP_KERNEL); @@ -315,7 +317,6 @@ static int max98373_read_prop(struct sdw_slave *slave) /* do this again for sink now */ nval = hweight32(prop->sink_ports); - num_of_ports += nval; prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->sink_dpn_prop), GFP_KERNEL); @@ -333,17 +334,6 @@ static int max98373_read_prop(struct sdw_slave *slave) i++; } - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/rt1308-sdw.c b/sound/soc/codecs/rt1308-sdw.c index 56e952a904a3..977be6f2b817 100644 --- a/sound/soc/codecs/rt1308-sdw.c +++ b/sound/soc/codecs/rt1308-sdw.c @@ -118,11 +118,14 @@ static int rt1308_clock_config(struct device *dev) static int rt1308_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports = 1; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_BUS_CLASH | SDW_SCP_INT1_PARITY; + prop->quirks = SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY; + prop->paging_support = true; /* first we need to allocate memory for set bits in port lists */ @@ -131,7 +134,6 @@ static int rt1308_read_prop(struct sdw_slave *slave) /* for sink */ nval = hweight32(prop->sink_ports); - num_of_ports += nval; prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->sink_dpn_prop), GFP_KERNEL); @@ -149,17 +151,6 @@ static int rt1308_read_prop(struct sdw_slave *slave) i++; } - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/rt5682-sdw.c b/sound/soc/codecs/rt5682-sdw.c index 94bf6bee78e6..feb2db95829e 100644 --- a/sound/soc/codecs/rt5682-sdw.c +++ b/sound/soc/codecs/rt5682-sdw.c @@ -19,6 +19,7 @@ #include <linux/mutex.h> #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> +#include <linux/soundwire/sdw_registers.h> #include <sound/core.h> #include <sound/pcm.h> #include <sound/pcm_params.h> @@ -537,11 +538,15 @@ static int rt5682_update_status(struct sdw_slave *slave, static int rt5682_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports = 1; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_IMPL_DEF | SDW_SCP_INT1_BUS_CLASH | + SDW_SCP_INT1_PARITY; + prop->quirks = SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY; + prop->paging_support = false; /* first we need to allocate memory for set bits in port lists */ @@ -549,7 +554,6 @@ static int rt5682_read_prop(struct sdw_slave *slave) prop->sink_ports = 0x2; /* BITMAP: 00000010 */ nval = hweight32(prop->source_ports); - num_of_ports += nval; prop->src_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->src_dpn_prop), GFP_KERNEL); @@ -569,7 +573,6 @@ static int rt5682_read_prop(struct sdw_slave *slave) /* do this again for sink now */ nval = hweight32(prop->sink_ports); - num_of_ports += nval; prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->sink_dpn_prop), GFP_KERNEL); @@ -587,17 +590,6 @@ static int rt5682_read_prop(struct sdw_slave *slave) i++; } - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/rt700-sdw.c b/sound/soc/codecs/rt700-sdw.c index 1d24bf040718..77deda9fd969 100644 --- a/sound/soc/codecs/rt700-sdw.c +++ b/sound/soc/codecs/rt700-sdw.c @@ -11,6 +11,7 @@ #include <linux/mod_devicetable.h> #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> +#include <linux/soundwire/sdw_registers.h> #include <linux/module.h> #include <linux/regmap.h> #include <sound/soc.h> @@ -333,11 +334,15 @@ static int rt700_update_status(struct sdw_slave *slave, static int rt700_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports = 1; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_IMPL_DEF | SDW_SCP_INT1_BUS_CLASH | + SDW_SCP_INT1_PARITY; + prop->quirks = SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY; + prop->paging_support = false; /* first we need to allocate memory for set bits in port lists */ @@ -345,7 +350,6 @@ static int rt700_read_prop(struct sdw_slave *slave) prop->sink_ports = 0xA; /* BITMAP: 00001010 */ nval = hweight32(prop->source_ports); - num_of_ports += nval; prop->src_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->src_dpn_prop), GFP_KERNEL); @@ -365,7 +369,6 @@ static int rt700_read_prop(struct sdw_slave *slave) /* do this again for sink now */ nval = hweight32(prop->sink_ports); - num_of_ports += nval; prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->sink_dpn_prop), GFP_KERNEL); @@ -383,17 +386,6 @@ static int rt700_read_prop(struct sdw_slave *slave) i++; } - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/rt711-sdw.c b/sound/soc/codecs/rt711-sdw.c index 7efff130a638..2eb115cf9d41 100644 --- a/sound/soc/codecs/rt711-sdw.c +++ b/sound/soc/codecs/rt711-sdw.c @@ -11,6 +11,7 @@ #include <linux/mod_devicetable.h> #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> +#include <linux/soundwire/sdw_registers.h> #include <linux/module.h> #include <linux/regmap.h> #include <sound/soc.h> @@ -337,11 +338,15 @@ static int rt711_update_status(struct sdw_slave *slave, static int rt711_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports = 1; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_IMPL_DEF | SDW_SCP_INT1_BUS_CLASH | + SDW_SCP_INT1_PARITY; + prop->quirks = SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY; + prop->paging_support = false; /* first we need to allocate memory for set bits in port lists */ @@ -349,7 +354,6 @@ static int rt711_read_prop(struct sdw_slave *slave) prop->sink_ports = 0x8; /* BITMAP: 00001000 */ nval = hweight32(prop->source_ports); - num_of_ports += nval; prop->src_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->src_dpn_prop), GFP_KERNEL); @@ -369,7 +373,6 @@ static int rt711_read_prop(struct sdw_slave *slave) /* do this again for sink now */ nval = hweight32(prop->sink_ports); - num_of_ports += nval; prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->sink_dpn_prop), GFP_KERNEL); @@ -387,17 +390,6 @@ static int rt711_read_prop(struct sdw_slave *slave) i++; } - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/rt715-sdw.c b/sound/soc/codecs/rt715-sdw.c index 68a36739f1b0..6b8ea62d5178 100644 --- a/sound/soc/codecs/rt715-sdw.c +++ b/sound/soc/codecs/rt715-sdw.c @@ -12,6 +12,7 @@ #include <linux/mod_devicetable.h> #include <linux/soundwire/sdw.h> #include <linux/soundwire/sdw_type.h> +#include <linux/soundwire/sdw_registers.h> #include <linux/module.h> #include <linux/of.h> #include <linux/regmap.h> @@ -431,11 +432,15 @@ static int rt715_update_status(struct sdw_slave *slave, static int rt715_read_prop(struct sdw_slave *slave) { struct sdw_slave_prop *prop = &slave->prop; - int nval, i, num_of_ports = 1; + int nval, i; u32 bit; unsigned long addr; struct sdw_dpn_prop *dpn; + prop->scp_int1_mask = SDW_SCP_INT1_IMPL_DEF | SDW_SCP_INT1_BUS_CLASH | + SDW_SCP_INT1_PARITY; + prop->quirks = SDW_SLAVE_QUIRKS_INVALID_INITIAL_PARITY; + prop->paging_support = false; /* first we need to allocate memory for set bits in port lists */ @@ -443,7 +448,6 @@ static int rt715_read_prop(struct sdw_slave *slave) prop->sink_ports = 0x0; /* BITMAP: 00000000 */ nval = hweight32(prop->source_ports); - num_of_ports += nval; prop->src_dpn_prop = devm_kcalloc(&slave->dev, nval, sizeof(*prop->src_dpn_prop), GFP_KERNEL); @@ -460,36 +464,6 @@ static int rt715_read_prop(struct sdw_slave *slave) i++; } - /* do this again for sink now */ - nval = hweight32(prop->sink_ports); - num_of_ports += nval; - prop->sink_dpn_prop = devm_kcalloc(&slave->dev, nval, - sizeof(*prop->sink_dpn_prop), - GFP_KERNEL); - if (!prop->sink_dpn_prop) - return -ENOMEM; - - dpn = prop->sink_dpn_prop; - i = 0; - addr = prop->sink_ports; - for_each_set_bit(bit, &addr, 32) { - dpn[i].num = bit; - dpn[i].simple_ch_prep_sm = true; - dpn[i].ch_prep_timeout = 10; - i++; - } - - /* Allocate port_ready based on num_of_ports */ - slave->port_ready = devm_kcalloc(&slave->dev, num_of_ports, - sizeof(*slave->port_ready), - GFP_KERNEL); - if (!slave->port_ready) - return -ENOMEM; - - /* Initialize completion */ - for (i = 0; i < num_of_ports; i++) - init_completion(&slave->port_ready[i]); - /* set the timeout values */ prop->clk_stop_timeout = 20; diff --git a/sound/soc/codecs/wsa881x.c b/sound/soc/codecs/wsa881x.c index d39d479e2378..68e774e69c85 100644 --- a/sound/soc/codecs/wsa881x.c +++ b/sound/soc/codecs/wsa881x.c @@ -1112,6 +1112,7 @@ static int wsa881x_probe(struct sdw_slave *pdev, wsa881x->sconfig.type = SDW_STREAM_PDM; pdev->prop.sink_ports = GENMASK(WSA881X_MAX_SWR_PORTS, 0); pdev->prop.sink_dpn_prop = wsa_sink_dpn_prop; + pdev->prop.scp_int1_mask = SDW_SCP_INT1_BUS_CLASH | SDW_SCP_INT1_PARITY; gpiod_direction_output(wsa881x->sd_n, 1); wsa881x->regmap = devm_regmap_init_sdw(pdev, &wsa881x_regmap_config); diff --git a/tools/testing/selftests/firmware/fw_filesystem.sh b/tools/testing/selftests/firmware/fw_filesystem.sh index fcc281373b4d..c2a2a100114b 100755 --- a/tools/testing/selftests/firmware/fw_filesystem.sh +++ b/tools/testing/selftests/firmware/fw_filesystem.sh @@ -149,6 +149,26 @@ config_unset_into_buf() echo 0 > $DIR/config_into_buf } +config_set_buf_size() +{ + echo $1 > $DIR/config_buf_size +} + +config_set_file_offset() +{ + echo $1 > $DIR/config_file_offset +} + +config_set_partial() +{ + echo 1 > $DIR/config_partial +} + +config_unset_partial() +{ + echo 0 > $DIR/config_partial +} + config_set_sync_direct() { echo 1 > $DIR/config_sync_direct @@ -207,6 +227,35 @@ read_firmwares() done } +read_partial_firmwares() +{ + if [ "$(cat $DIR/config_into_buf)" == "1" ]; then + fwfile="${FW_INTO_BUF}" + else + fwfile="${FW}" + fi + + if [ "$1" = "xzonly" ]; then + fwfile="${fwfile}-orig" + fi + + # Strip fwfile down to match partial offset and length + partial_data="$(cat $fwfile)" + partial_data="${partial_data:$2:$3}" + + for i in $(seq 0 3); do + config_set_read_fw_idx $i + + read_firmware="$(cat $DIR/read_firmware)" + + # Verify the contents are what we expect. + if [ $read_firmware != $partial_data ]; then + echo "request #$i: partial firmware was not loaded" >&2 + exit 1 + fi + done +} + read_firmwares_expect_nofile() { for i in $(seq 0 3); do @@ -242,6 +291,21 @@ test_batched_request_firmware_into_buf_nofile() echo "OK" } +test_request_partial_firmware_into_buf_nofile() +{ + echo -n "Test request_partial_firmware_into_buf() off=$1 size=$2 nofile: " + config_reset + config_set_name nope-test-firmware.bin + config_set_into_buf + config_set_partial + config_set_buf_size $2 + config_set_file_offset $1 + config_trigger_sync + read_firmwares_expect_nofile + release_all_firmware + echo "OK" +} + test_batched_request_firmware_direct_nofile() { echo -n "Batched request_firmware_direct() nofile try #$1: " @@ -356,6 +420,21 @@ test_request_firmware_nowait_custom() echo "OK" } +test_request_partial_firmware_into_buf() +{ + echo -n "Test request_partial_firmware_into_buf() off=$1 size=$2: " + config_reset + config_set_name $TEST_FIRMWARE_INTO_BUF_FILENAME + config_set_into_buf + config_set_partial + config_set_buf_size $2 + config_set_file_offset $1 + config_trigger_sync + read_partial_firmwares normal $1 $2 + release_all_firmware + echo "OK" +} + # Only continue if batched request triggers are present on the # test-firmware driver test_config_present @@ -383,6 +462,12 @@ for i in $(seq 1 5); do test_request_firmware_nowait_custom $i normal done +# Partial loads cannot use fallback, so do not repeat tests. +test_request_partial_firmware_into_buf 0 10 +test_request_partial_firmware_into_buf 0 5 +test_request_partial_firmware_into_buf 1 6 +test_request_partial_firmware_into_buf 2 10 + # Test for file not found, errors are expected, the failure would be # a hung task, which would require a hard reset. echo @@ -407,6 +492,12 @@ for i in $(seq 1 5); do test_request_firmware_nowait_custom_nofile $i done +# Partial loads cannot use fallback, so do not repeat tests. +test_request_partial_firmware_into_buf_nofile 0 10 +test_request_partial_firmware_into_buf_nofile 0 5 +test_request_partial_firmware_into_buf_nofile 1 6 +test_request_partial_firmware_into_buf_nofile 2 10 + test "$HAS_FW_LOADER_COMPRESS" != "yes" && exit 0 # test with both files present diff --git a/tools/testing/selftests/lkdtm/run.sh b/tools/testing/selftests/lkdtm/run.sh index 8383eb89d88a..bb7a1775307b 100755 --- a/tools/testing/selftests/lkdtm/run.sh +++ b/tools/testing/selftests/lkdtm/run.sh @@ -82,7 +82,7 @@ dmesg > "$DMESG" ($SHELL -c 'cat <(echo '"$test"') >'"$TRIGGER" 2>/dev/null) || true # Record and dump the results -dmesg | diff --changed-group-format='%>' --unchanged-group-format='' "$DMESG" - > "$LOG" || true +dmesg | comm --nocheck-order -13 "$DMESG" - > "$LOG" || true cat "$LOG" # Check for expected output |