[KinoSearch] KinoSearch Death

[Marvin Humphrey]

On Oct 19, 2006, at 5:19 PM, Chris Nandor wrote:

> At 16:44 -0700 2006.10.19, Marvin Humphrey wrote:
>> I'm pretty sure you're using a Unix variant ;) so any file currently
>> in use that gets unlinked is going to stick around until the last
>> filehandle lets it go, and thus that seems unlikely to be the cause.
>
> Yes, but I was thinking maybe it didn't actually keep all the files  
> open
> that it might need to read.
>
>
>>> So maybe it is NOT safe to update a live invindex?
>>
>> I'm really not sure what conclusion to draw, based on that error
>> message.  How many times have you seen it?
>
> It's in the error log for today 113 times.

Hmm.  You say you've been indexing once every five minutes, and you  
checked that at around 5:00? 16 x 60 / 5 = 192.  Perhaps if the times  
when the errors occurred are relatively evenly spaced at 5 minute  
intervals it might have something to do with updating.

> Fuller log is attached.

One thing that's interesting about your error message is the last  
number, which is errno.

[Thu Oct 19 23:08:48 2006] [error] refill: tried to read 1024 bytes,  
got 0: 0 at

It's zero.  That's frustrating.  We're reading from a PerlIO* object  
(which is the C struct underpinning a Perl filehandle).  We asked it  
for 1024 bytes, and it gave us zero.  But it didn't set errno.

When PerlIO_read fails, it's supposed to set errno to a positive  
number indicating the type of error that occurred -- just like the  
stdio function fread().  Here's a snippet from its documentation in  
perlapio:

     Returns a byte count if successful (which may be zero or positive),
     returns negative value and sets errno on error.

So, it looks like we've tickled a bug in PerlIO_read.  It thinks it  
succeeded by giving us 0 bytes, even though we asked for 1024.

Can you please check to see errno is in your other log entries?

FYI, among the refactoring tasks I've already concluded is the  
complete removal of *all* PerlIO functions from KinoSearch's IO  
classes.  InStream and OutStream classes now use stdio exclusively  
for file system i/o.  So maybe this bug has been inadvertently  
squashed already.

I would ordinarily ask you to try the subversion trunk but it isn't  
stable just this moment.  Since you are on the lucy-dev list, you  
know what I'm working on.  Charmonizer has to be finished for Lucy,  
then copied back into KinoSearch.  After that I have to squash a few  
memory leaks.  Probably by Monday.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/



_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch