<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>I am indexing a large number of files, in batches. In order
to allow the files to change and be re-indexed, I call <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>$invindexer->delete_docs_by_term($term);<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Before each insertion. <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I managed to index around 140000 files, but then I hit a problem
with the following message:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Couldn't open file
'/home/qatest/kinosearch/invindex/_10613.f0': File exists at<o:p></o:p></p>
<p class=MsoNormal>/usr/local/lib/perl/5.8.8/KinoSearch/Store/FSInvIndex.pm
line 88<o:p></o:p></p>
<p class=MsoNormal>
KinoSearch::Store::FSInvIndex::open_outstream('KinoSearch::Store::FSInvIndex=HASH(0x88829bc)',
'_10613.f0') called at /usr/local/lib<o:p></o:p></p>
<p class=MsoNormal>/perl/5.8.8/KinoSearch/Index/SegWriter.pm line 40<o:p></o:p></p>
<p class=MsoNormal>
KinoSearch::Index::SegWriter::init_instance('KinoSearch::Index::SegWriter=HASH(0x8c9e634)')
called at /usr/local/lib/perl/5.8.8/Kino<o:p></o:p></p>
<p class=MsoNormal>Search/Util/Class.pm line 31<o:p></o:p></p>
<p class=MsoNormal>
KinoSearch::Util::Class::new('KinoSearch::Index::SegWriter', 'invindex',
'KinoSearch::Store::FSInvIndex=HASH(0x88829bc)', 'seg_name'<o:p></o:p></p>
<p class=MsoNormal>, '_10613', 'finfos',
'KinoSearch::Index::FieldInfos=HASH(0x8c9e1d8)', 'field_sims',
'HASH(0x8882ee4)', ...) called at /usr/local/lib/perl/5<o:p></o:p></p>
<p class=MsoNormal>.8.8/KinoSearch/InvIndexer.pm line 152<o:p></o:p></p>
<p class=MsoNormal>
KinoSearch::InvIndexer::_delayed_init('KinoSearch::InvIndexer=HASH(0x8882b78)')
called at /usr/local/lib/perl/5.8.8/KinoSearch/InvIn<o:p></o:p></p>
<p class=MsoNormal>dexer.pm line 262<o:p></o:p></p>
<p class=MsoNormal>
KinoSearch::InvIndexer::delete_docs_by_term('KinoSearch::InvIndexer=HASH(0x8882b78)',
'KinoSearch::Index::Term=HASH(0x8882cbc)') cal<o:p></o:p></p>
<p class=MsoNormal>led at index.pl line 185<o:p></o:p></p>
<p class=MsoNormal>
main::handleJob('KinoSearch::InvIndexer=HASH(0x8882b78)',
'26ba90ca1fa8354ffb00f43b59b58223f2f07b35') called at index.pl line 80<o:p></o:p></p>
<p class=MsoNormal> eval {...} called
at index.pl line 79<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>The contents of the invindexer directory are the following<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 2360312453 2008-10-20 20:44
_10455.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 2331193220 2008-10-20 22:51
_10583.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 351002378 2008-10-20 23:06
_10599.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 13298110 2008-10-20
23:07 _10600.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 41360185 2008-10-20
23:08 _10601.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 20375600 2008-10-20
23:09 _10602.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 60127418 2008-10-20
23:11 _10603.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 14264840 2008-10-20
23:12 _10604.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 14161480 2008-10-20
23:12 _10605.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 14208046 2008-10-20
23:13 _10606.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 14408859 2008-10-20
23:14 _10607.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 15721448 2008-10-20
23:15 _10608.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 28738202 2008-10-20
23:16 _10609.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 47820729 2008-10-20
23:17 _10610.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 16592842 2008-10-20
23:18 _10611.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 89080573 2008-11-10
13:05 _10612.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f0<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f1<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f10<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f11<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f12<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f2<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f3<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f4<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f5<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f6<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f7<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f8<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.f9<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 8111867
2008-11-10 13:06 _10613.fdt<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.fdx<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 0 2008-11-10 13:06
_10613.srt<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 4111442230 2008-10-11 01:17
_228.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest qatest 4448882741 2008-10-19 17:08
_9338.cfs<o:p></o:p></p>
<p class=MsoNormal>-rw-r--r-- 1 qatest
qatest 215 2008-11-10 13:05 segments<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>If I move all the _10613.* files to another directory it
will look like it is indexing again, but I think the ‘finish’ is
not working, so that on my next batch run I will get the same problem above. I
guess that if I delete these files I am basically not adding that data to the
index. Is that true? I tried adding optimize => 1 to the finish, but it made
no difference.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I make the indexer, add a batch full of documents (e..g 200
or so of delete_docs_by_term, then add_doc), then call <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>$invindexer->finish; <o:p></o:p></p>
<p class=MsoNormal>$invindexer->_release_locks();<o:p></o:p></p>
<p class=MsoNormal>$invindexer = undef;<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>And exit.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I read some other posts with update problems e.g. <a
href="http://www.gossamer-threads.com/lists/kinosearch/discuss/3249">http://www.gossamer-threads.com/lists/kinosearch/discuss/3249</a>,
and upgraded to the latest from svn (3883 at the time I did it). Both the
version on cpan and the latest had the same exact error message.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Any suggestions? <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks in advance<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Matt Williamson<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>