Re: [PATCH bpf-next v5] virtio_net: add XDP meta data support

[Yuya Kusakabe]

On 2/24/20 1:05 PM, Jason Wang wrote:
> 
> On 2020/2/23 äå4:14, Michael S. Tsirkin wrote:
>> On Fri, Feb 21, 2020 at 05:36:08PM +0900, Yuya Kusakabe wrote:
>>> On 2/21/20 1:23 PM, Jason Wang wrote:
>>>> On 2020/2/20 äå4:55, Yuya Kusakabe wrote:
>>>>> Implement support for transferring XDP meta data into skb for
>>>>> virtio_net driver; before calling into the program, xdp.data_meta points
>>>>> to xdp.data, where on program return with pass verdict, we call
>>>>> into skb_metadata_set().
>>>>>
>>>>> Tested with the script at
>>>>> https://github.com/higebu/virtio_net-xdp-metadata-test.
>>>>>
>>>>> Fixes: de8f3a83b0a0 ("bpf: add meta pointer for direct access")
>>>> I'm not sure this is correct since virtio-net claims to not support metadata by calling xdp_set_data_meta_invalid()?
>>> virtio_net doesn't support by calling xdp_set_data_meta_invalid() for now.
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/drivers/net/virtio_net.c?id=e42da4c62abb547d9c9138e0e7fcd1f36057b5e8#n686
>>> https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/drivers/net/virtio_net.c?id=e42da4c62abb547d9c9138e0e7fcd1f36057b5e8#n842
>>>
>>> And xdp_set_data_meta_invalid() are added by de8f3a83b0a0.
>>>
>>> $ git blame ./drivers/net/virtio_net.c | grep xdp_set_data_meta_invalid
>>> de8f3a83b0a0f (Daniel BorkmannÂÂÂÂÂÂÂÂÂÂ 2017-09-25 02:25:51 +0200Â 686)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ xdp_set_data_meta_invalid(&xdp);
>>> de8f3a83b0a0f (Daniel BorkmannÂÂÂÂÂÂÂÂÂÂ 2017-09-25 02:25:51 +0200Â 842)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ xdp_set_data_meta_invalid(&xdp);
>>>
>>> So I added `Fixes: de8f3a83b0a0 ("bpf: add meta pointer for direct access")` to the comment.
>>>
>>>>> Signed-off-by: Yuya Kusakabe<yuya.kusakabe@xxxxxxxxx>
>>>>> ---
>>>>> v5:
>>>>> ÂÂ - page_to_skb(): copy vnet header if hdr_valid without checking metasize.
>>>>> ÂÂ - receive_small(): do not copy vnet header if xdp_prog is availavle.
>>>>> ÂÂ - __virtnet_xdp_xmit_one(): remove the xdp_set_data_meta_invalid().
>>>>> ÂÂ - improve comments.
>>>>> v4:
>>>>> ÂÂ - improve commit message
>>>>> v3:
>>>>> ÂÂ - fix preserve the vnet header in receive_small().
>>>>> v2:
>>>>> ÂÂ - keep copy untouched in page_to_skb().
>>>>> ÂÂ - preserve the vnet header in receive_small().
>>>>> ÂÂ - fix indentation.
>>>>> ---
>>>>> ÂÂ drivers/net/virtio_net.c | 54 ++++++++++++++++++++++++----------------
>>>>> ÂÂ 1 file changed, 33 insertions(+), 21 deletions(-)
>>>>>
>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>>> index 2fe7a3188282..4ea0ae60c000 100644
>>>>> --- a/drivers/net/virtio_net.c
>>>>> +++ b/drivers/net/virtio_net.c
>>>>> @@ -371,7 +371,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct receive_queue *rq,
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct page *page, unsigned int offset,
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned int len, unsigned int truesize,
>>>>> -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ bool hdr_valid)
>>>>> +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ bool hdr_valid, unsigned int metasize)
>>>>> ÂÂ {
>>>>> ÂÂÂÂÂÂ struct sk_buff *skb;
>>>>> ÂÂÂÂÂÂ struct virtio_net_hdr_mrg_rxbuf *hdr;
>>>>> @@ -393,6 +393,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>>>> ÂÂÂÂÂÂ else
>>>>> ÂÂÂÂÂÂÂÂÂÂ hdr_padded_len = sizeof(struct padded_vnet_hdr);
>>>>> ÂÂ +ÂÂÂ /* hdr_valid means no XDP, so we can copy the vnet header */
>>>>> ÂÂÂÂÂÂ if (hdr_valid)
>>>>> ÂÂÂÂÂÂÂÂÂÂ memcpy(hdr, p, hdr_len);
>>>>> ÂÂ @@ -405,6 +406,11 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>>>> ÂÂÂÂÂÂÂÂÂÂ copy = skb_tailroom(skb);
>>>>> ÂÂÂÂÂÂ skb_put_data(skb, p, copy);
>>>>> ÂÂ +ÂÂÂ if (metasize) {
>>>>> +ÂÂÂÂÂÂÂ __skb_pull(skb, metasize);
>>>>> +ÂÂÂÂÂÂÂ skb_metadata_set(skb, metasize);
>>>>> +ÂÂÂ }
>>>>> +
>>>>> ÂÂÂÂÂÂ len -= copy;
>>>>> ÂÂÂÂÂÂ offset += copy;
>>>>> ÂÂ @@ -450,10 +456,6 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi,
>>>>> ÂÂÂÂÂÂ struct virtio_net_hdr_mrg_rxbuf *hdr;
>>>>> ÂÂÂÂÂÂ int err;
>>>>> ÂÂ -ÂÂÂ /* virtqueue want to use data area in-front of packet */
>>>>> -ÂÂÂ if (unlikely(xdpf->metasize > 0))
>>>>> -ÂÂÂÂÂÂÂ return -EOPNOTSUPP;
>>>>> -
>>>>> ÂÂÂÂÂÂ if (unlikely(xdpf->headroom < vi->hdr_len))
>>>>> ÂÂÂÂÂÂÂÂÂÂ return -EOVERFLOW;
>>>>> ÂÂ @@ -644,6 +646,7 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>>>> ÂÂÂÂÂÂ unsigned int delta = 0;
>>>>> ÂÂÂÂÂÂ struct page *xdp_page;
>>>>> ÂÂÂÂÂÂ int err;
>>>>> +ÂÂÂ unsigned int metasize = 0;
>>>>> ÂÂ ÂÂÂÂÂ len -= vi->hdr_len;
>>>>> ÂÂÂÂÂÂ stats->bytes += len;
>>>>> @@ -683,8 +686,8 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>>>> ÂÂ ÂÂÂÂÂÂÂÂÂ xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
>>>>> ÂÂÂÂÂÂÂÂÂÂ xdp.data = xdp.data_hard_start + xdp_headroom;
>>>>> -ÂÂÂÂÂÂÂ xdp_set_data_meta_invalid(&xdp);
>>>>> ÂÂÂÂÂÂÂÂÂÂ xdp.data_end = xdp.data + len;
>>>>> +ÂÂÂÂÂÂÂ xdp.data_meta = xdp.data;
>>>>> ÂÂÂÂÂÂÂÂÂÂ xdp.rxq = &rq->xdp_rxq;
>>>>> ÂÂÂÂÂÂÂÂÂÂ orig_data = xdp.data;
>>>>> ÂÂÂÂÂÂÂÂÂÂ act = bpf_prog_run_xdp(xdp_prog, &xdp);
>>>>> @@ -695,6 +698,7 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ /* Recalculate length in case bpf program changed it */
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ delta = orig_data - xdp.data;
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ len = xdp.data_end - xdp.data;
>>>>> +ÂÂÂÂÂÂÂÂÂÂÂ metasize = xdp.data - xdp.data_meta;
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ break;
>>>>> ÂÂÂÂÂÂÂÂÂÂ case XDP_TX:
>>>>> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ stats->xdp_tx++;
>>>>> @@ -735,11 +739,14 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>>>> ÂÂÂÂÂÂ }
>>>>> ÂÂÂÂÂÂ skb_reserve(skb, headroom - delta);
>>>>> ÂÂÂÂÂÂ skb_put(skb, len);
>>>>> -ÂÂÂ if (!delta) {
>>>>> +ÂÂÂ if (!xdp_prog) {
>>>>> ÂÂÂÂÂÂÂÂÂÂ buf += header_offset;
>>>>> ÂÂÂÂÂÂÂÂÂÂ memcpy(skb_vnet_hdr(skb), buf, vi->hdr_len);
>>>>> ÂÂÂÂÂÂ } /* keep zeroed vnet hdr since packet was changed by bpf */
>>>> I prefer to make this an independent patch and cc stable.
>>>>
>>>> Other looks good.
>>>>
>>>> Thanks
>>> I see. So I need to revert to delta from xdp_prog?
>>>
>>> Thank you.
>> So maybe send a 2 patch series: 1/2 is this chunk with the appropriate
>> description. Actually for netdev David prefers that people do not
>> cc stable directly, just include Fixes tag and mention in the
>> commit log it's also needed for stable. Patch 2/2 is the rest
>> handling metadata.
> 
> 
> +1
> 
> Thanks
> 
> 

Thank you for the detailed explanation. I will make a 2 patch series.