Re: [Syzkaller & bisect] There was "iommufd_test" WARNING in v6.2 in guest

[Pengfei Xu]

Hi Jason,

On 2023-02-24 at 10:48:45 -0400, Jason Gunthorpe wrote:
> On Fri, Feb 24, 2023 at 01:43:38PM +0800, Pengfei Xu wrote:
> > Hi Jason,
> > 
> > Greeting!
> > 
> > Platform: ADL-S and x86 platforms
> > Host kernel 6.2.0-rc7 which doesn't enable iommufd by host kconfig setting as
> > below:
> > "# CONFIG_IOMMUFD is not set"
> > 
> > Guest kconfig with iommufd enabled: https://github.com/xupengfe/syzkaller_logs/blob/main/230224_044002_iommufd_test/kconfig_origin
> > Reproduced code: https://github.com/xupengfe/syzkaller_logs/blob/main/230224_044002_iommufd_test/repro.c
> > v6.2 problem dmesg: https://github.com/xupengfe/syzkaller_logs/blob/main/230224_044002_iommufd_test/v6.2_c9c3395d5e3dcc6daee66c6908354d47bf98cb0c_dmesg.log
> > Bisect info: https://github.com/xupengfe/syzkaller_logs/blob/main/230224_044002_iommufd_test/bisect_info.log
> > 
> > There was "iommufd_test" WARNING in v6.2 in guest:
> > [   32.012827] ------------[ cut here ]------------
> > [   32.013027] WARNING: CPU: 1 PID: 393 at drivers/iommu/iommufd/selftest.c:403 iommufd_test+0xb19/0x16f0
> > [   32.013410] Modules linked in:
> 
> I didn't include the IOMMU_TEST_OP_MD_CHECK_REFS in the syzkaller
> descriptions, how did you hit this?
> 
I am new to iommufd feature, and I'm not sure how syzkaller got to this
unexpected test point.

Anyway, I provided syzkaller repo top commit id info on that ADL-S:
commit bcdf85f8bd3fccff5bc9507a589c4847d9b35405 (HEAD -> master)
Author: Dmitry Vyukov <dvyukov@xxxxxxxxxx>
Date:   Fri Feb 17 09:58:24 2023 +0100
    dashboard/config/linux: enable DWARF4 for all

And I didn't change the file "syzkaller/sys/linux/dev_iommu.txt.const":
"
...
AT_FDCWD = 18446744073709551516
IOMMU_DESTROY = 15232, mips64le:ppc64le:536886144             // 15232= 0x3B80  =  IOMMU_DESTROY  =  IOMMUFD_CMD_BASE
IOMMU_IOAS_ALLOC = 15233, mips64le:ppc64le:536886145
IOMMU_IOAS_ALLOW_IOVAS = 15234, mips64le:ppc64le:536886146
IOMMU_IOAS_COPY = 15235, mips64le:ppc64le:536886147
IOMMU_IOAS_IOVA_RANGES = 15236, mips64le:ppc64le:536886148
IOMMU_IOAS_MAP = 15237, mips64le:ppc64le:536886149
...
IOMMU_TEST_CMD = 15264, mips64le:ppc64le:536886176       // 15232(IOMMU_DESTROY=IOMMUFD_CMD_BASE) +  32 =   15264 =  0x3ba0
IOMMU_TEST_CMD_SIZE = 56         // 56(0x38  for CMD size)
IOMMU_TEST_OP_ACCESS_PAGES = 7   // no IOMMU_TEST_OP_MD_CHECK_REFS:4 in the const file
IOMMU_TEST_OP_ACCESS_RW = 8
IOMMU_TEST_OP_ADD_RESERVED = 1
IOMMU_TEST_OP_CREATE_ACCESS = 5
IOMMU_TEST_OP_DESTROY_ACCESS_PAGES = 6
IOMMU_TEST_OP_MOCK_DOMAIN = 2
IOMMU_TEST_OP_SET_TEMP_MEMORY_LIMIT = 9
IOMMU_VFIO_IOAS = 15240, mips64le:ppc64le:536886152
...
"
I didn't change syzkaller/sys/linux/dev_iommu.txt and
syzkaller/sys/linux/test/dev_iommu also.

# [root@p-adls01 syzkaller]# pwd
/root/syzkaller
[root@p-adls01 syzkaller]# git log
commit bcdf85f8bd3fccff5bc9507a589c4847d9b35405 (HEAD -> master)
Author: Dmitry Vyukov <dvyukov@xxxxxxxxxx>
Date:   Fri Feb 17 09:58:24 2023 +0100
    dashboard/config/linux: enable DWARF4 for all
[root@p-adls01 syzkaller]# git diff
[root@p-adls01 syzkaller]# git status
On branch master
Your branch is behind 'origin/master' by 51 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working tree clean
[root@p-adls01 syzkaller]#


I'm not sure how syzkaller hit this unexpected test point:
https://github.com/xupengfe/syzkaller_logs/blob/c7cc00a25b3f5d1efd4b83bf3cfc552049628c96/230224_044002_iommufd_test/repro.c#L34
"
  *(uint32_t*)0x20000000 = 0x38;
  *(uint32_t*)0x20000004 = 4;
...
  syscall(__NR_ioctl, r[0], 0x3ba0, 0x20000000ul);
"

Checked IOMMU_TEST_OP_MD_CHECK_REFS in linux_kernel/drivers/iommu/iommufd/iommufd_test.h
"
enum {
        IOMMU_TEST_OP_ADD_RESERVED = 1,
        IOMMU_TEST_OP_MOCK_DOMAIN,
        IOMMU_TEST_OP_MD_CHECK_MAP,
        IOMMU_TEST_OP_MD_CHECK_REFS,          // 4
        IOMMU_TEST_OP_CREATE_ACCESS,
        IOMMU_TEST_OP_DESTROY_ACCESS_PAGES,
        IOMMU_TEST_OP_ACCESS_PAGES,
        IOMMU_TEST_OP_ACCESS_RW,
        IOMMU_TEST_OP_SET_TEMP_MEMORY_LIMIT,
};
...
#define IOMMU_TEST_CMD _IO(IOMMUFD_TYPE, IOMMUFD_CMD_BASE + 32)
"
Thanks!

> Anyhow, it is a small issue in the test suite because uptr + length overflows:
> 
  Thanks a lot for your patch! I'm glad it's helpful.

  Thanks!
  BR.

> --- a/drivers/iommu/iommufd/selftest.c
> +++ b/drivers/iommu/iommufd/selftest.c
> @@ -568,13 +568,17 @@ static int iommufd_test_md_check_refs(struct iommufd_ucmd *ucmd,
>                                       void __user *uptr, size_t length,
>                                       unsigned int refs)
>  {
> -       if (length % PAGE_SIZE || (uintptr_t)uptr % PAGE_SIZE)
> +       uintptr_t end;
> +
> +       if (length % PAGE_SIZE || (uintptr_t)uptr % PAGE_SIZE ||
> +           check_add_overflow((uintptr_t)uptr, (uintptr_t)length, &end))
>                 return -EINVAL;
> 
> Jason