[KinoSearch] Possible Locking Improvements

[Marvin Humphrey]

On Jan 28, 2007, at 11:42 PM, Miles Crawford wrote:
> This sounds like it relates to my earlier thread about indexing  
> with an NFS-driver clustered setup, and I just wanted to chime in  
> and say that if there was a particular KinoSearch class I could  
> extend in order to do all the requisite locking via a sufficiently  
> atomic store, like our db, that would be a good solution for us.

The clustered writer problem is solved in svn trunk.  Add a unique  
host_id param to InvIndexer->new:

     my $invindexer = KinoSearch::InvIndexer->new(
         invindex    => $invindex,
         host_id     => $hostname   # default: ''
     );

Both host_id and process id will be written to the lock file.  If an  
existing lockfile is detected, first the host_id is compared, and if  
it doesn't match, the locking attempt fails.  If host_id matches,  
then the pid is examined.  If the pid is active, the locking attempt  
fails; if it's not, the lockfile is silently zapped and indexing  
proceeds.

> That seems like it could even enable a single NFS-stored index if  
> extended to handle all the cases where kinosearch needs to avoid  
> collisions with itself. ;)

Reads on NFS remain a problem, but I'm back-burnering working on that  
till later.

I'm toying with the idea of adding no-op methods lock_for_reading()  
and release_read_lock() to IndexReader, and is_locked_for_reading()  
to InvIndexer.  Determined individuals would subclass and override.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/



_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch