arm64: rockchip: add MIPI CSI camera pipeline support for RK3576

[hz2]

Adds initial MIPI CSI-2 camera capture support for the RK3576 SoC, porting the existing RK3588 pipeline to the RK3576. The two SoCs share the same IP blocks (Innosilicon CSI DPHY, Synopsys DW CSI-2 Host, Rockchip VICAP) with minor differences in instance count and register base addresses.

RK3576 has 2 Inno CSI DPHY instances, 5 CSI Host controllers, and a VICAP unit with 5 MIPI input ports (vs 6 on RK3588).

Register addresses and layouts were verified against the RK3576 TRM V1.2 Part 2 and the Rockchip BSP kernel (develop-6.1 branch).

Changes:

• phy: rockchip: inno-csidphy: add rockchip,rk3576-csi-dphy compatible
• media: synopsys: dw-mipi-csi2rx: add rockchip,rk3576-mipi-csi2 compatible
• media: rockchip: rkcif: add rockchip,rk3576-vicap with 5-port MIPI support
• arm64: dts: rockchip: add VICAP, CSI Host, CSIDPHY, and GRF nodes to rk3576.dtsi

This is still missing DT binding YAML updates and board-level DTS wiring (sensor not yet identified for Flipper One). Compile-tested with the flipperone-linux-build-scripts config, RK3576 DTBs builds.

Closes #6

[alchark]

Thanks for your contribution!

As we are targeting mainline Linux kernel, we'll need to follow the mainline Linux contribution rules - in particular the part about the Developer Certificate of Origin there (i.e. please include your real name and email address in both the patch author information and the Signed-off-by: trailer to assert the DCO).

Are you certain that there is no register-level difference vs. RK3588? Rockchip's downstream driver has some GRF registers used by RK3588 but not RK3576

Please also check if https://docs.kernel.org/process/coding-assistants.html applies and also if Rockchip authorship/sign-off needs to be preserved in any of the patches (depending on how much of this was copied from the downstream develop-6.1 kernel)

[hz2]

Thanks for getting back so quickly!

As we are targeting mainline Linux kernel, we'll need to follow the mainline Linux contribution rules - in particular the part about the Developer Certificate of Origin there (i.e. please include your real name and email address in both the patch author information and the Signed-off-by: trailer to assert the DCO).

added DCO, somewhat familiar with the process

Are you certain that there is no register-level difference vs. RK3588? Rockchip's downstream driver has some GRF registers used by RK3588 but not RK3576

RK3576 uses sys_grf for lane select at 0x14 (GRF_SOC_CON5), where RK3588 uses 0x308 (GRF_SOC_CON2). Mainline doesn't have sys_grf at all currently, only writes GRF for lane and clock enable i believe. the register layout is identical between RK3576 and RK3588 when looking at BSP source and TRM, however, sys_grf is missing from this series and will be added. for the flipper-one use case, per-PHY GRF writes should be sufficient? i'll add sys_grf support in a follow-up or fold it into this series before submitting to the lists, open to your thoughts here as well.

Please also check if https://docs.kernel.org/process/coding-assistants.html applies

ack, will review and comply

also if Rockchip authorship/sign-off needs to be preserved in any of the patches (depending on how much of this was copied from the downstream develop-6.1 kernel)

no code was copied from the BSP driver, the driver changes follow mainline patterns (RK3588 entries already in-tree) and register addresses come from TRM and BSP DTS so no Rockchip sign-off should be needed here but please keep me honest.

plan would be to address the sys_grf gap i mentioned and add DT binding YAML updates then submit a proper patch series to:

• linux-media@vger.kernel.org
• linux-rockchip@lists.infradead.org

accordingly. happy to hear if you think anything else needs to change before that.

ps: wouldn't hurt to get a dev flipper one to test with 👀

[alchark]

FWIW, our builds didn't enable the CSI2 driver and its siblings up to now - I've just changed that. It builds fine with your patches, thank you!

Do you have any RK3576 based board to test it on? It would be great to also add the necessary board DTS nodes along with sensor-specific DTSO

[hz2]

FWIW, our builds didn't enable the CSI2 driver and its siblings up to now - I've just changed that. It builds fine with your patches, thank you!

great! i tried it in a nix-shell (probably should have mentioned that) previously when building locally

Do you have any RK3576 based board to test it on?

i do not atm (why i asked about a dev board)

It would be great to also add the necessary board DTS nodes along with sensor-specific DTSO

agree that the sensor wiring should go in a DTSO, assuming sensor choice isn't finalized, right? happy to work on the overlay once ik what module to target

[hz2]

will be sending a RFC to mailing list to denote i haven't tested on hardware and get feedback

[alchark]

Ouch, this looks ugly. Firstly, you are not guarding this block by a particular SoC compatible but use a SoC specific MMIO address. Secondly, this should probably be encoded as a data attribute somewhere instead of hardcoding register addresses.

[hz2]

Ouch, this looks ugly.

Thanks! lol

Fair enough, one thing I can't verify without hardware: does rk3576 full mode actually need the SYS_GRF_SOC_CON5 write, or is the reset default already full mode? That decides whether I drop it cleanly or rework it in place.

When it comes back I'd carry the lane select bit as a phandle arg, rockchip,sys-grf = <&sys_grf 1>, read with syscon_regmap_lookup_by_phandle_args, rather than sniffing the MMIO address

[hz2]

I'd like to drop the sys_grf lane select from this series and bring it back as a proper follow-up. The precedent is your own upstream RK3588 CSI DPHY support: the BSP has the lane select (rk3588_grf_dphy_regs, GRF_SOC_CON2 bit 6) but the mainline rk3588 variant left it out, and the upstream rk3588 csi_dphy nodes carry only rockchip,grf, never a sys_grf. RK3576 is the same IP and the target boards all use a single 2 lane PHY in full mode, so the core port should stand on its own the same way.

The one thing I cannot verify: does RK3576 full mode actually work without writing SYS_GRF_SOC_CON5? TRM Part 2 only gives the bit meaning (bit 1 = DPHY0, bit 2 = DPHY1, 0 = full, 1 = split), not the reset value, and the BSP writes it explicitly even for full mode. If POR is full mode, dropping is safe. If not, I should rework it in-series instead. If you'd rather keep it in-series, the plan is to carry the lane select bit as a DT phandle arg, rockchip,sys-grf = <&sys_grf 1>, read with syscon_regmap_lookup_by_phandle_args, so there's no MMIO address sniffing and no offset sentinel.

Open to a different representation.

[alchark]

Thanks for the update!

I've recently rebased our development branch, so had to reapply these manually. Haven't yet had chance to runtime test it, but there is a bunch of warnings during the DT schema validation - can you please address?

/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: syscon@2603a000 (rockchip,rk3576-csidphy-grf): False schema does not allow [[23, 492]]
        from schema $id: http://devicetree.org/schemas/soc/rockchip/grf.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: video-capture@27c10000 (rockchip,rk3576-vicap): resets: [[23, 321], [23, 322], [23, 320], [23, 332], [23, 333], [23, 334], [23, 335], [23, 336]] is too short
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3588-vicap.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27c80000 (rockchip,rk3576-mipi-csi2): clock-names: ['pclk', 'iclk'] is too long
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27c80000 (rockchip,rk3576-mipi-csi2): clocks: [[23, 369], [23, 375]] is too long
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27c80000 (rockchip,rk3576-mipi-csi2): clock-names:0: 'per' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27c80000 (rockchip,rk3576-mipi-csi2): clock-names:1: 'pixel' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27c90000 (rockchip,rk3576-mipi-csi2): clock-names:0: 'per' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27ca0000 (rockchip,rk3576-mipi-csi2): clock-names:0: 'per' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27cb0000 (rockchip,rk3576-mipi-csi2): clock-names:0: 'per' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml
/home/alchark/linux/arch/arm64/boot/dts/rockchip/rk3576-flipper-one-rev-f0b0c1.dtb: csi@27cc0000 (rockchip,rk3576-mipi-csi2): clock-names:0: 'per' was expected
        from schema $id: http://devicetree.org/schemas/media/rockchip,rk3568-mipi-csi2.yaml

[alchark]

Apparently something's also off with the power domains handling, as the boot stalls with the following messages once a camera module is enabled (it gets unstuck after a while, but this doesn't seem right):

[   13.536766] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 27b00100.video-codec
[   13.537787] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 27c10000.video-capture
[   13.538795] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 27cb0000.csi
[   13.539724] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 27d50000.sai
[   13.540675] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 27ea0000.spdif-tx
[   13.541649] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 2a600000.sai
[   13.542577] rockchip-pm-domain 27380000.power-management:power-controller: sync_state() pending due to 2a620000.sai

Could you please check @hz2?

Thanks a lot!

[hz2]

Apparently something's also off with the power domains handling, as the boot stalls with the following messages once a camera module is enabled (it gets unstuck after a while, but this doesn't seem right)

i dug into this and think it is a kernel config gap rather than a DT driver bug and would think this would be harmless once the drivers are built in.

each node in your log is a consumer of VI/VO/audio power domain whose device link has not gone active yet, so the controller defers and you see "pending due to ..." and the fw_devlink timeout then force-syncs, which is the "gets unstuck after awhile" part, so it's a deferred cleanup, not a true hang.

i didn't see the drivers I have and the config from the build scripts:

# CONFIG_VIDEO_ROCKCHIP_CIF is not set
# CONFIG_VIDEO_DW_MIPI_CSI2RX is not set
# CONFIG_PHY_ROCKCHIP_INNO_CSIDPHY is not set

with those off, enabling vicap/csi/csi_dphy is an overlay creates power-domain callers that can never probe. could you check the config on your side? you mentioned enabling the CSI2 driver previously, want to confirm that PHY_ROCKCHIP_INNO_CSIDPHY (and not CIF) did not make it into the same both. If all three enabled and a node does not bind, a dmesg log from you would help with this (or hardware sent out). i can't verify the config gap without a hardware boot.

for what it's worth, the power-domain wiring on the rk3576 camera nodes is the same shape as the rk3588 camera nodes now on the branch so if a real sync_state issue remained after the drivers are enabled, I would expect rk3588 to show it too.