Re: VM_RESERVED and PG_reserved : Allocating memory for video buffers

[Laurent Pinchart]

On Friday 16 December 2005 02:27, Nick Piggin wrote:
> Laurent Pinchart wrote:
> > Hi everybody,
> >
> > I'm writing a Linux driver for a USB Video Class compliant USB device. I
> > manage to understand pretty much everything on my own until the point
> > where I have to allocate video buffers.
> >
> > I read other drivers to understand how they proceed. Most of them used
> > vmalloc with SetPageReserved and remap_pfn_range to map the memory to
> > user space. I thought I understood that, when I noticed that
> > vm_insert_page has been added in 2.6.15. I wasn't sure how to prevent
> > pages from being swapped out, so I read the excellent "Understanding the
> > Linux Virtual Memory Manager", but I'm still not sure to understand
> > everything. This is where I ask for your help.
> >
> > I need to allocate big buffers, so vmalloc is the way to go, as I don't
> > need contiguous memory. I need to map those buffers to user space, and I
> > understand that vm_insert_page will do the job nicely. My fears come from
> > pages being swapped out. I suppose I need to prevent that, as a page
> > fault in interrupt is a Bad Thing(TM). I'm not sure how PG_reserved and
> > VM_RESERVED interract with eachother. Can kernel pages be swapped out if
> > they are not mapped to user space ? Or does kswapd only walk VMAs when it
> > tries to find pages that will be swapped out ? If the later is true, is
> > it enough to set VM_RESERVED on the VMA in the mmap handler ?
>
> PG_reserved no longer does anything (except catching bugs in old code).
> If you are writing new code, you shouldn't use it. Don't copy rvmalloc,
> you should be able to use vmalloc directly.
>
> vm_insert_page is indeed the right interface for mapping these pages
> into userspace.

Ok, that's what I intended to do.

> You do not have to worry about pages being swapped out, and you shouldn't
> need to set any unusual vma or page flags. kswapd only walks the lru lists,
> and it won't even look at any other pages.

I'd still like to understand how things work (I'm one of those programmers who 
don't like to code without understanding).

I think I understand how disk buffers or non-shared pages mapped by a regular 
file can be reclaimed, but I have trouble with anonymous pages and shared 
pages.

First of all, I haven't been to find a definition of an anonymous page. I 
understand it as a page of memory not backed by a file (pages allocated by 
vmalloc for instance). If this is wrong, what I'm about to say if probably 
very wrong as well.

Are anonymous pages ever added to the LRU active list ? I suppose they are 
not, which is why they are not reclaimed.

How does the kernel handle shared pages, (if for instance two processes map a 
regular file with MAP_SHARED) ? They can't be reclaimed before all processes 
which map them have had their PTEs modified. Is this where reverse mapping 
comes into play ?

Finally, how are devices which map anonymous kernel memory to user space 
handled ? When a page is inserted in a process VMA using vm_insert_page, it 
becomes shared between the kernel and user space. Does the kernel see the 
page as a regular device backed page, and put it in the LRU active list ? You 
said I shouldn't need to set any unusual VMA or page flags. What's the exact 
purpose of VM_RESERVED and VM_IO then ? And when should they be set ?

Hope I'm not bothering you too much with all those questions. I don't feel at 
ease when developping kernel code if I don't have at least a basic 
understanding of what I'm doing.

Laurent Pinchart

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/