Re: Fwd: [Nepomuk] Better support for (desktop) file search / indexingapplications

[Lijo Antony]

On 03/10/2013 08:51 AM, Simeon Bird wrote:
> Hi,
>
> We (nepomuk) recently looked at using fanotify, and indeed we would
> need user watches, support for moves and recursive directory watches
> (we need to support the case where /home is not a separate filesystem)
> before it would be useful to us. If you are interested in adding
> these, we would be delighted to use nepomuk as a test-case for them.
>
> We were wondering also if it would be possible to extend inotify a
> little? Our wishlist is:
>
> 1) Recursive folder watches
> 2) When a file moves, some way to get the destination without watching
> the directory it moved to, so moves can be tracked without watching
> every file on the system.

I am also interested in these features. As of now, my solutions are,

1) When the limit is reached, ask the user to increase the limit and 
restart the application.

2) When top level directory move is detected, do a file system search 
based on inode to find out the new location. Very slow and not fool proof.

I would also like to any other solutions for these problems. I am yet to 
look into fanotify.

-lijo

[leaving the rest for reference]

> I understand that there are reasons of security and performance why
> you cannot implement 1), but is 2) possible? Maybe by extending
> IN_MOVED_TO, or adding a new event type?
>
> 2) is actually in some ways the more severe problem for us. As well as
> being an indexer, nepomuk is a system that allows you to store file
> metadata such as ratings. When users move the files, they want the
> metadata to move too, so we need to track where the file moved, and
> thus at the moment we recursively watch everything. This is
> particularly problematic with removable media; because a lot of people
> will plug in an external drive and then move files onto it, we have to
> watch every drive as soon as it is plugged in. If we were able to get
> the destination of move events without watching the destination
> directory, we could watch only those directories with interesting
> metadata in, which would make things a lot easier.
>
> inotify move tracking would also be useful for other things - eg, a
> text editor could use inotify to see if a file it has open has moved
> and offer to re-open the file in its new location, which is impossible
> at the moment.
>
> Since the lack of recursive watches is really a problem because we
> have a tendency to run out of watches, it would also really help if
> the default limit was a bit higher -  most people seem to have > 8000
> folders, but I suspect far fewer have > 32000 (probably excepting
> those who are indexing kernel source trees: I have 21000, and half of
> that is KDE source).
>
> Would any of this be possible? If you happen to know of a better way
> to track moves using existing tools, that would be even better.
>
> Thanks,
> Simeon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/