RE: [PATCH v2 0/3] media: pci: intel: ivsc: Add driver of Intel Visual Sensing Controller(IVSC)
From: Wu, Wentong
Date: Tue Feb 14 2023 - 01:35:01 EST
Hi Srinivas,
Thanks for your review
> -----Original Message-----
> From: Pandruvada, Srinivas <srinivas.pandruvada@xxxxxxxxx>
> Sent: Monday, February 13, 2023 11:21 AM
>
> On Mon, 2023-02-13 at 10:23 +0800, Wentong Wu wrote:
> >
>
> v2 without change log, why?
This addressed 'mei_csi.c:(.text+0x1a6): undefined reference to __udivdi3' issue on i386 arch. I will add change log on following version.
>
> > Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a
> > companion chip designed to provide secure and low power vision
> > capability to IA platforms. IVSC is available in existing commercial
> > platforms from multiple OEMs.
> >
> > The primary use case of IVSC is to bring in context awareness. IVSC
> > interfaces directly with the platform main camera sensor via a CSI-2
> > link and processes the image data with the embedded AI engine. The
> > detected events are sent over I2C to ISH (Intel Sensor Hub) for
> > additional data fusion from multiple sensors. The fusion results are
> > used to implement advanced use cases like:
> > - Face detection to unlock screen
> > - Detect user presence to manage backlight setting or waking up
> > system
> >
> > Since the Image Processing Unit(IPU) used on the host processor needs
> > to configure the CSI-2 link in normal camera usages, the CSI-2 link
> > and camera sensor can only be used in mutually-exclusive ways by host
> > IPU and IVSC. By default the IVSC owns the CSI-2 link and camera
> > sensor. The IPU driver can take ownership of the CSI-2 link and camera
> > sensor using interfaces provided by this IVSC driver.
> >
> > Switching ownership requires an interface with two different hardware
> > modules inside IVSC. The software interface to these modules is via
> > Intel MEI (The Intel Management Engine) commands. These two hardware
> > modules have two different MEI UUIDs to enumerate. These hardware
> > modules
> > are:
> > - ACE (Algorithm Context Engine): This module is for algorithm
> > computing when IVSC owns camera sensor. Also ACE module controls
> > camera sensor's ownership. This hardware module is used to set
> > ownership of camera sensor.
> > - CSI (Camera Serial Interface): This module is used to route camera
> > sensor data either to IVSC or to host for IPU driver and application.
> >
> > IVSC also provides a privacy mode. When privacy mode is turned on,
> > camera sensor can't be used. This means that both ACE and host IPU
> > can't get image data. And when this mode is turned on, host IPU driver
> > is informed via a registered callback, so that user can be notified.
> >
> > In summary, to acquire ownership of camera by IPU driver, first ACE
> > module needs to be informed of ownership and then to setup MIPI CSI-2
> > link for the camera sensor and IPU.
> >
> > Implementation:
> > There are two different drivers to handle ACE and CSI hardware modules
> > inside IVSC.
> > - mei_csi: MEI client driver to send commands and receive
> > notifications from CSI module.
> > - mei_ace: MEI client driver to send commands and get status from ACE
> > module.
> > Interface is exposed via ivsc.h to acquire and release camera sensor
> > and
> > CSI-2 link.
> >
> > Below diagram shows connections of IVSC/ISH/IPU/Camera sensor.
> > ---------------------------------------------------------------------
> > --------
> > > Host
> > > Processor
> > > |
> > >
> > > |
> > > ----------------- ----------------- -------------
> > > -- |
> > > | | | | |
> > > | I2C |
> > > | IPU | | ISH | |camera
> > > driver|--| |
> > > | | | | |
> > > | | |
> > > ----------------- ----------------- -------------
> > > -- | |
> > > | |
> > > | | |
> > > | | -------------
> > > -- | |
> > > | | |
> > > | | |
> > > | | | IVSC driver
> > > | | |
> > > | | |
> > > | | |
> > > | | -------------
> > > -- | |
> > > | |
> > > | | |
> > ----------------|-----------------------|----------------------|-----
> > ----|---
> > | CSI | I2C
> > |SPI |
> > | |
> > | |
> > ----------------|-----------------------|----------------
> > | |
> > > IVSC | |
> > > | |
> > > | |
> > > | |
> > > ----------------- ----------------- |
> > > | |
> > > | | | | |
> > > | |
> > > | CSI | | ACE | |------
> > > | |
> > > | | | |
> > > | |
> > > ----------------- -----------------
> > > | |
> > > | | I2C
> > > | |
> > ----------------|-----------------------|----------------
> > |
> > | CSI
> > | |
> > |
> > | |
> > --------------------------------
> > |
> > | | I2C
> > |
> > | camera sensor |-------------------------
> > ----|
> > | |
> > --------------------------------
> >
> > Wentong Wu (3):
> > media: pci: intel: ivsc: Add CSI submodule
> > media: pci: intel: ivsc: Add ACE submodule
> > media: pci: intel: ivsc: Add acquire/release API for ivsc
> >
> > drivers/media/pci/Kconfig | 1 +
> > drivers/media/pci/intel/Makefile | 2 +
> > drivers/media/pci/intel/ivsc/Kconfig | 12 +
> > drivers/media/pci/intel/ivsc/Makefile | 7 +
> > drivers/media/pci/intel/ivsc/ivsc.c | 84 +++++
> > drivers/media/pci/intel/ivsc/mei_ace.c | 472
> > +++++++++++++++++++++++++
> > drivers/media/pci/intel/ivsc/mei_ace.h | 36 ++
> > drivers/media/pci/intel/ivsc/mei_csi.c | 342 ++++++++++++++++++
> > drivers/media/pci/intel/ivsc/mei_csi.h | 60 ++++
> > include/linux/ivsc.h | 74 ++++
> > 10 files changed, 1090 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ivsc/Kconfig
> > create mode 100644 drivers/media/pci/intel/ivsc/Makefile
> > create mode 100644 drivers/media/pci/intel/ivsc/ivsc.c
> > create mode 100644 drivers/media/pci/intel/ivsc/mei_ace.c
> > create mode 100644 drivers/media/pci/intel/ivsc/mei_ace.h
> > create mode 100644 drivers/media/pci/intel/ivsc/mei_csi.c
> > create mode 100644 drivers/media/pci/intel/ivsc/mei_csi.h
> > create mode 100644 include/linux/ivsc.h
> >