#clojure logs

2008-08-09

07:05Chouserkotarak: chimp works as advertised.
07:05Chouserkotarak: I'll have to let it settle in a bit before I can provide any useful opinions.
07:53kotarakChouser: thanks for testing :)
12:34arohnerhas anyone tried alternatives to an RDBS in clojure?
12:34arohnerI'm working on a project using lots of bayes statistics, and using SQL to do my queries looks like it will be very messy
12:35arohnerI would really love a DB I can query using lisp, but it seems like postgres/mysql is the only game in town
12:44arohnerI guess I don't need an alternative to RDBS, I want an alternative to SQL
12:55shoover1I've been wondering how Berkeley DB would work as a key/value store for Clojure. There is a pure Java implementation, and supposedly it's supposed to work well with JavaBeans.
12:59shooverI'm just thinking out loud, as I don't know of an existing implementation for Clojure. But in this system I imagine being able to query using your keys, Clojure fns, or list comprehensions.
13:04arohnerinteresting
13:04arohnerthe problem is, my data actually fits in an RDBS well
13:05arohnerlots of user sessions, each session is a bunch of questions
13:05arohnerI just don't want to use SQL to get to it
13:06arohnerthe fact that SQL is not composable is very irritating
13:07moxleyThere's a lightweight persistence framework inside Compojure-- persist.clj
13:09arohnerhmm..
13:09arohnerthen I would need to write the code to add an index to a set of rows
13:15moxleyyeah, it doesn't help much there
13:17arohnerI'm also slightly worried about what happens when the dataset becomes larger than RAM
13:22joubertI've used BDB as a datastore from Clojure
13:24shooverjoubert: Oh?! What'd you think?
13:26joubertwell, the way I persist values is to encode the data (structure) to the UTF-8 stream of bytes that make up the result of its (pr-str)
13:26joubertThen I decode doing the reverse
13:26jouberte.g.:
13:26joubert(defn decode-data-structure-from-persistence [persisted-representation]
13:26joubert (let [string-representation (new String persisted-representation *utf-8*)]
13:26joubert (read (new java.io.PushbackReader (new java.io.StringReader string-representation)))))
13:26joubert(defn encode-data-structure-for-persistence [data-structure]
13:26joubert (let [string-representation (pr-str data-structure)]
13:26joubert (. string-representation (getBytes *utf-8*))))
13:26joubertthe limitation is that I cannot persist seqs
13:27joubertI can share some code if interested
13:27shooverPlease paste that snippet into http://paste.lisp.org/new
13:28joubertthe encode and decode?
13:28shooveryes, the functions above
13:29lisppaste8joubert pasted "Encode / Decode Clojure data for persistence (e.g. to BDB)" at http://paste.lisp.org/display/65076
13:31joubertI should also have shown: (def *utf-8* "UTF-8")
13:31shooverThat's cool. And then you use Clojure/Java interop for creating and querying your db?
13:32joubertyeah, I have functions to put and get values, as well as the plumbing you need to setup the BDB environment & database
13:32joubertI also have support for keys
13:32joubertthat allow referential logic
13:32joubert(BDB calls them SecondaryKeys)
13:36shooverSo that would work for all of Clojure's readable data structures, but seqs you would have to evaluate to a list or vector first
13:36joubertright
13:36joubertmaybe I should post my code to the clojure group files area
13:37joubertit is only about 170 lines, and would obviate a lot of grunt code you'd need to write
13:38shooverYes, that or submit to clojure-contrib, as long as you're willing to license it appropriately
13:38joubertcurrently GPL 3, but I guess most clojure things are CPL?
13:38joubertI'm happy to change to CPL
13:39shooverclojure-contrib is CPL. I'm not sure, but I think things submitted to the group are automatically CPL
13:40rhickeyjoubert: to contribute to clojure-contrib you'll need to submit a CA: http://clojure.org/contributing
13:41joubertOK
13:41arohner_rich: is there any way to view the change history to the docs on clojure.org?
13:42arohner_there have been several groups posts where someone asks you to change the docs, and you reply "done". I don't see a good way to find out what changed.
13:42rhickeyarohner_: not right now, there might be a way with the wiki software...
13:51rhickeyarohner_: http://clojure.org/space/changes
13:52arohner_cool. thanks!
14:32drewrarohner_: You could look into an object persistence solution, like Terracotta.
14:34arohner_I'll check it out. Thanks
14:36arohner_that's interesting, though not quite what I'm looking for
14:37arohner_my biggest headache is SQL
14:38arohner_the queries I'm writing are ugly, and I can't help but think it could be easier using map, filter and reduce
14:39arohner_but then I want to be able to declare indexes on my dataset, and have the library handle moving stuff out of RAM when not necessary
14:39arohner_basically a normal RDBMS that I can query using lisp
14:40rhickeyarohner_: there is: http://clojure.org/api#index
14:42arohner_how do I call it?
14:42arohner_index
14:42arohner_java.lang.Exception: Unable to resolve symbol: index in this context
14:43arohner_ah, nm
14:43rhickeyit's in the clojre.set namespace
14:43rhickeyclojure.set
14:48arohner_I don't quite understand what it's supposed to do from the docstring
14:52arohner_oh look, a helpful example at the bottom of set.clj :-)
15:07arohner_all of the subtypes of map have the same literal syntax, {}, right?
15:07arohner_so to find out what type of map you have, you should do (class map)?
15:07rhickeyarohner_: right now, yes
16:28jamiirhickey: It looks like stream fusion might be really effective combined with seq. You couldnt do it statically because clojure is late binding, but maybe hotspot could be persuaded to use it?
16:33rhickeyjamii: I have no idea if optimizations like that are present in hotspot
16:36jamiiI wouldnt expect them to be useful in java. Its a simple compiler optmisations in an early binding language. It would work really well with clojures seqs but it would require some ability to hot swap code. I wonder if hotspot exposes any of its functionality.