[KinoSearch] Serialized Schema

[Marvin Humphrey]

On Nov 6, 2007, at 4:08 AM, Peter Karman wrote:

> I really like the JSON::Syck (and accompanying YAML::Syck) parser.  
> libsyck is
> BSD-licensed and very fast.

The JSON::XS documentation has a "comparison" section where it runs  
through the available alternatives, including JSON::Syck. Here's the  
gripe list for JSON::Syck 0.21:

   * Very buggy (often crashes).

   * Very inflexible (no human-readable format supported, format  
pretty much
     undocumented. I need at least a format for easy reading by  
humans and a
     single-line compact format for use in a protocol, and preferably  
a way to
     generate ASCII-only JSON texts).

   * Completely broken (and confusingly documented) Unicode handling  
(unicode
     escapes are not working properly, you need to set  
ImplicitUnicode to
     I<different> values on en- and decoding to get symmetric  
behaviour).

   * No roundtripping (simple cases work, but this depends on wether  
the scalar
     value was used in a numeric context or not).

   * Dumping hashes may skip hash values depending on iterator state.

   * Unmaintained (maintainer unresponsive for many months, bugs are not
     getting fixed).

   * Does not check input for validity (i.e. will accept non-JSON  
input and
     return "something" instead of raising an exception. This is a  
security
     issue: imagine two banks transfering money between each other  
using JSON.
     One bank might parse a given non-JSON request and deduct money,  
while the
     other might reject the transaction with a syntax error. While a  
good
     protocol will at least recover, that is extra unnecessary work  
and the
     transaction will still not succeed).

JSON::Syck is at 0.26 now, so it *is* maintained and some bugs are  
getting fixed.  Looking at the Changes file, though, I don't see any  
mention of the rest.

Also, JSON::XS 1.52 has a better CPAN Testers report than YAML::Syck  
0.99, the distro that contains JSON::Syck: 55 passes, 5 N/A vs 94  
passes, 11 failures.

I think the main underlying difference is that libsyck has to deal  
with YAML while JSON::XS is just JSON, and JSON is a much easier spec  
to implement than YAML.  Douglas Crockford, author of the JSON spec,  
got KISS right -- and the YAML people didn't.

Something to bear in mind while writing the InvIndex file format spec.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/



_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch