[KinoSearch] KinoSearch Death
marvin at rectangular.com
Thu Oct 19 18:54:57 PDT 2006
On Oct 19, 2006, at 5:19 PM, Chris Nandor wrote:
> At 16:44 -0700 2006.10.19, Marvin Humphrey wrote:
>> I'm pretty sure you're using a Unix variant ;) so any file currently
>> in use that gets unlinked is going to stick around until the last
>> filehandle lets it go, and thus that seems unlikely to be the cause.
> Yes, but I was thinking maybe it didn't actually keep all the files
> that it might need to read.
>>> So maybe it is NOT safe to update a live invindex?
>> I'm really not sure what conclusion to draw, based on that error
>> message. How many times have you seen it?
> It's in the error log for today 113 times.
Hmm. You say you've been indexing once every five minutes, and you
checked that at around 5:00? 16 x 60 / 5 = 192. Perhaps if the times
when the errors occurred are relatively evenly spaced at 5 minute
intervals it might have something to do with updating.
> Fuller log is attached.
One thing that's interesting about your error message is the last
number, which is errno.
[Thu Oct 19 23:08:48 2006] [error] refill: tried to read 1024 bytes,
got 0: 0 at
It's zero. That's frustrating. We're reading from a PerlIO* object
(which is the C struct underpinning a Perl filehandle). We asked it
for 1024 bytes, and it gave us zero. But it didn't set errno.
When PerlIO_read fails, it's supposed to set errno to a positive
number indicating the type of error that occurred -- just like the
stdio function fread(). Here's a snippet from its documentation in
Returns a byte count if successful (which may be zero or positive),
returns negative value and sets errno on error.
So, it looks like we've tickled a bug in PerlIO_read. It thinks it
succeeded by giving us 0 bytes, even though we asked for 1024.
Can you please check to see errno is in your other log entries?
FYI, among the refactoring tasks I've already concluded is the
complete removal of *all* PerlIO functions from KinoSearch's IO
classes. InStream and OutStream classes now use stdio exclusively
for file system i/o. So maybe this bug has been inadvertently
I would ordinarily ask you to try the subversion trunk but it isn't
stable just this moment. Since you are on the lucy-dev list, you
know what I'm working on. Charmonizer has to be finished for Lucy,
then copied back into KinoSearch. After that I have to squash a few
memory leaks. Probably by Monday.
KinoSearch mailing list
KinoSearch at rectangular.com
More information about the kinosearch