[KinoSearch] Issue with KinoSearch

[Marvin Humphrey]

On Apr 24, 2008, at 6:01 PM, Ox Gibson wrote:

> When trying to create a new invIndexer, I keeping getting this  
> error...

> kino_FSFolder_open_outstream at c_src/KinoSearch/Store/FSFolder.c: 
> 55: Can't
> open 'seg_t.ds': File exists

OK, I've now duplicated the problem.  Thanks for the report.

The last KS releases, both maint and devel, were primarily intended to  
ratchet up paranoia about clobbering files for security reasons.   
Throughout the KS code base, instead of clobbering, we now unlink then  
perform a safe open using (O_CREAT | O_EXCL).  It looks like this  
problem has arisen because I missed an unlink.

seg_t.ds is an old file left behind from an aborted indexing session  
-- maybe a crash, or maybe a quit prior to calling finish().  With  
earlier versions, the InvIndexer would just clobber this file.  Now  
it's trying to perform a safe open, but the unlink hasn't happened  
first.

I believe that the solution is for InvIndexer to sweep the invindex  
directory for unused KS files before it tries to initialize the  
SegWriter.  I'll try and put together a new release in the next couple  
of days that does that.  In the meantime, you should be able to clear  
the block by moving any files that start with 'seg_t' out of the  
invindex directory.

> I am using the following code to create a new invIndexer:
>
> sub invindexer {
>     my $self = shift;
>     my $path = $self->get_maildir->get_config_path . "/kino";
>     my $hostname = hostname();
>     die "Can't get unique hostname" unless $hostname;
>
>     my $invindex = GT::Maildir::KinoSearch::Schema->open($path);
>     my $lock_factory = KinoSearch::Store::LockFactory->new(
>         folder    => $invindex->get_folder,
>         agent_id  => $hostname,
>     );

Since you're using a lock_factory... is this invindex is on an NFS  
volume?

Do you know whether an indexing session crashed before completion, and  
if so what the cause was?  The fix is the same regardless; I'd just  
like to know what aborted the indexing session.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/


_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch