[KinoSearch] Merging indexes, etc

[henka at cityweb.co.za]

>
> On Oct 26, 2006, at 6:36 AM, henka at cityweb.co.za wrote:
>
>> Splitting the process into two steps (delete_docs_by_term() and
>> add_invindexes()) unfortunately results in the error:
>>
>> Can't locate object method "_release_locks" via package
>> "self" (perhaps
>> you forgot to load "self"?) at
>> /usr/lib/perl5/site_perl/5.8.7/i486-linux/KinoSearch/InvIndexer.pm
>> line
>> 273.
>
> Hmm.  What the hell is "self" doing there?  That really suggests
> memory corruption of some kind -- the pointer that used to lead Perl
> to the class name has been reused and is now pointing to something
> completely different and wrong.  That's why I suspected problems with
> DESTROY.
>
> I have one more idea, but after that, I'm going to need a failing
> test case I can go tear into.
>
>> The logic I used:
>>
>> my $iv;
>> foreach (sub-indexes) {
>>   $iv= KinoSearch::InvIndexer->new(...);
>>   $iv->delete_docs_by_term(...);
>>   $iv->finish(optimize=>0);
>>   $iv= KinoSearch::InvIndexer->new(...); # same sub-index as previous
>>   $iv->add_invindexes(...);
>>   $iv->finish(optimize=>0);
>> }
>
> Try undef-ing the InvIndexer before assigning another one on top of
> it.  Take a look at the create_test_invindex() function in <http://
> www.rectangular.com/svn/kinosearch/trunk/buildlib/
> KinoSearchTestInvIndex.pm> for a template.
>
> I'm not sure that you're seeing the same problem I've been seeing,
> but code like in your example, where a new InvIndexer instance is
> created in the same scalar variable, has caused me problems in the
> past.  The symptoms were slightly different: it was lock contention
> rather than a failed call to _release_locks.
>
> The sequence "$invindexer = new_invindexer(); $invindexer =
> new_invindexer();" is problematic, because the second assignment
> statement triggers this order of operations:
>
>       1) Create new InvIndexer, put it in a mortal SV on
>          the Perl stack.
>       2) Assign the new InvIndexer object to the scalar
>          which previously held another invindexer.
>       3) The old scalar's reference count is decremented
>          and the old InvIndexer's DESTROY method gets called.
>
> In step 1, the new InvIndexer requests a lock.  But the old
> InvIndexer doesn't release its lock until step 3.
>
> I'd like to solve this problem, but I'm not sure how to do it in the
> general case.  I've thought of doing something like this, using
> closures:
>
>      # in new()
>      $self->{release_locks} = sub { $self->do_release_locks };
>
>      # in finish()
>      $self->{release_locks}->();
>      undef $self->{release_locks};
>
>      # in DESTROY
>      $self->{release_locks}->() if defined $self->{release_locks};
>
> But I think that would still fail if finish() never gets called.

Thanks for the detailed response.  Will try the undef approach and let you
know.


_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch