Re: [PATCH v8 00/11] Add support for SBI v0.2 and CPU hotplug

From: Atish Patra
Date: Wed Feb 19 2020 - 20:16:17 EST


On Wed, 2020-02-19 at 13:48 -0800, Palmer Dabbelt wrote:
> On Tue, 11 Feb 2020 17:48:11 PST (-0800), Atish Patra wrote:
> > The Supervisor Binary Interface(SBI) specification[1] now defines a
> > base extension that provides extendability to add future extensions
> > while maintaining backward compatibility with previous versions.
> > The new version is defined as 0.2 and older version is marked as
> > 0.1.
> >
> > This series adds support v0.2 and a unified calling convention
> > implementation between 0.1 and 0.2. It also add other SBI v0.2
> > functionality defined in [2]. The base support for SBI v0.2 is
> > already
> > available in OpenSBI v0.5. It also adds SBI HSM extension and cpu-
> > hotplug
> > support for RISC-V which requires additional patches[3] in OpenSBI.
>
> Now that 0.2-rc1 has been tagged we should really start to get this
> into shape
> to merge this. My biggest worry is being able to put together a
> kernel that
> can boot on both 0.1 and 0.2 SBIs, with the hart lottery being my
> major worry
> there. I just skimmed this, but I was expected to see something
> like
>
> diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
> index 271860fc2c3f..849ba75959ba 100644
> --- a/arch/riscv/kernel/head.S
> +++ b/arch/riscv/kernel/head.S
> @@ -187,6 +187,7 @@ relocate:
> la a3, .Lsecondary_park
> csrw CSR_TVEC, a3
>
> +#ifdef CONFIG_SBI_V01
> slli a3, a0, LGREG
> la a1, __cpu_up_stack_pointer
> la a2, __cpu_up_task_pointer
> @@ -212,7 +213,10 @@ relocate:
> #endif
>
> tail smp_callin
> -#endif
> +#else /* !CONFIG_SBI_V01 */
> + pr_warn("multiple harts booted an SBI v0.2+ only kernel");
> +#endif /* CONFIG_SBI_V01 */
> +#endif /* CONFIG_SMP */
>
> END(_start)
>
> but I don't. Is there something else doing this?
>

Yes. Current series ensures that it boots both 0.1 and 0.2 SBIs. It
is done based on dynamic detection of HSM extension and cpu_ops. You
can take a look at the patch 9.

+#if IS_ENABLED(CONFIG_RISCV_SBI)
+ if (sbi_probe_extension(SBI_EXT_HSM) > 0) {
+ if (!cpuid)
+ pr_info("SBI v0.2 HSM extension detected\n");
+ cpu_ops[cpuid] = &cpu_ops_sbi;
+ } else
+#endif
+ cpu_ops[cpuid] = &cpu_ops_spinwait;

https://patchwork.kernel.org/patch/11377359/

For SBI v0.1, probing will fail for HSM extension and cpu_ops_spinwait
will be set as th cpu_ops.


> > [1]
> > https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc
> > [2] https://github.com/riscv/riscv-sbi-doc/pull/27
> > [3]
> > http://lists.infradead.org/pipermail/opensbi/2020-January/001050.html
> >
> > The patches are also available in following github repositery.
> >
> > OpenSBI : https://github.com/atishp04/opensbi/tree/sbi_hsm_v1
> > Linux Kernel: https://github.com/atishp04/linux/tree/sbi_v0.2_v8
> >
> > Changes from v7->v8:
> > 1. Refactored to code to have modular cpu_ops calls.
> > 2. Refactored HSM extension from sbi.c to cpu_ops_sbi.c.
> > 3. Fix plic driver to handle cpu hotplug.
> >
> > Changes from v6->v7:
> > 1. Rebased on v5.5
> > 2. Fixed few compilation issues for !CONFIG_SMP and
> > !CONFIG_RISCV_SBI
> > 3. Added SBI HSM extension
> > 4. Add CPU hotplug support
> >
> > Changes from v5->v6
> > 1. Fixed few compilation issues around config.
> > 2. Fixed hart mask generation issues for RFENCE & IPI extensions.
> >
> > Changes from v4->v5
> > 1. Fixed few minor comments related to static & inline.
> > 2. Make sure that every patch is boot tested individually.
> >
> > Changes from v3->v4.
> > 1. Rebased on for-next.
> > 2. Fixed issuses with checkpatch --strict.
> > 3. Unfied all IPI/fence related functions.
> > 4. Added Hfence related SBI calls.
> >
> > Changes from v2->v3.
> > 1. Moved v0.1 extensions to a new config.
> > 2. Added support for relacement extensions of v0.1 extensions.
> >
> > Changes from v1->v2
> > 1. Removed the legacy calling convention.
> > 2. Moved all SBI related calls to sbi.c.
> > 3. Moved all SBI related macros to uapi.
> >
> > Atish Patra (11):
> > RISC-V: Mark existing SBI as 0.1 SBI.
> > RISC-V: Add basic support for SBI v0.2
> > RISC-V: Add SBI v0.2 extension definitions
> > RISC-V: Introduce a new config for SBI v0.1
> > RISC-V: Implement new SBI v0.2 extensions
> > RISC-V: Move relocate and few other functions out of __init
> > RISC-V: Add cpu_ops and modify default booting method
> > RISC-V: Add SBI HSM extension
> > RISC-V: Add supported for ordered booting method using HSM
> > irqchip/sifive-plic: Initialize the plic handler when cpu comes
> > online
> > RISC-V: Support cpu hotplug
> >
> > arch/riscv/Kconfig | 19 +-
> > arch/riscv/include/asm/cpu_ops.h | 46 +++
> > arch/riscv/include/asm/sbi.h | 194 ++++++----
> > arch/riscv/include/asm/smp.h | 24 ++
> > arch/riscv/kernel/Makefile | 6 +
> > arch/riscv/kernel/cpu-hotplug.c | 87 +++++
> > arch/riscv/kernel/cpu_ops.c | 48 +++
> > arch/riscv/kernel/cpu_ops_sbi.c | 113 ++++++
> > arch/riscv/kernel/cpu_ops_spinwait.c | 42 +++
> > arch/riscv/kernel/head.S | 179 +++++----
> > arch/riscv/kernel/sbi.c | 524
> > ++++++++++++++++++++++++++-
> > arch/riscv/kernel/setup.c | 24 +-
> > arch/riscv/kernel/smpboot.c | 56 +--
> > arch/riscv/kernel/traps.c | 2 +-
> > arch/riscv/kernel/vmlinux.lds.S | 5 +-
> > drivers/irqchip/irq-sifive-plic.c | 34 +-
> > include/linux/cpuhotplug.h | 1 +
> > 17 files changed, 1227 insertions(+), 177 deletions(-)
> > create mode 100644 arch/riscv/include/asm/cpu_ops.h
> > create mode 100644 arch/riscv/kernel/cpu-hotplug.c
> > create mode 100644 arch/riscv/kernel/cpu_ops.c
> > create mode 100644 arch/riscv/kernel/cpu_ops_sbi.c
> > create mode 100644 arch/riscv/kernel/cpu_ops_spinwait.c

--
Regards,
Atish