2011-05-23
| 00:10 | technomancy | wolfjb: you probably just want to use clojure.java.io/copy between a file and a zipoutputstream |
| 00:16 | wolfjb | technomancy: thanks! amalloy and ataggart have been very helpful in leading me toward that solution as well |
| 00:33 | wolfjb | I'm not understanding apply very well tonight, I thought I could call (apply addentry files zip-outputstream) which would turn into (addentry file zip-outputstream) for each file in files. however, that isn't working right. |
| 00:33 | technomancy | yeah; apparently I didn't read the backlog |
| 00:33 | ataggart | ,(apply + [1 2 3]) |
| 00:33 | clojurebot | 6 |
| 00:34 | ataggart | wolfjb: you probably want doseq |
| 00:34 | wolfjb | ah |
| 00:34 | ataggart | e.g., (doseq [f files] ...) |
| 00:35 | amalloy | wolfjb: apply "expands" the last argument (which must be a seq) in-place to result in a function call with more arguments |
| 00:35 | amalloy | &(apply (partial list '+) 1 2 [3 4 5 6]) |
| 00:35 | sexpbot | ⟹ (+ 1 2 3 4 5 6) |
| 00:35 | wolfjb | ah |
| 00:35 | wolfjb | (I feel like I'm saying that a lot) |
| 00:35 | wolfjb | heh |
| 00:35 | amalloy | then you're doing it right |
| 00:35 | wolfjb | thanks for the help |
| 00:36 | wolfjb | you guys are awesome! |
| 00:37 | wolfjb | woot! my zip stuff works now! |
| 03:41 | amalloy | do agents have a reset!-like functionality, or do i have to send them (constantly x)? |
| 06:24 | Vinzent | I'm using appengine-magic and have created a protocol and extended it to the appengine entity, but when I'm calling protocol fn on the entity queried from database, implementation for Object is chosen |
| 06:29 | Vinzent | example: http://paste.org/pastebin/view/33786 |
| 06:30 | Vinzent | What could be the problem? |
| 07:09 | Cozey | Sorry for an OT, but I'd like to query Lisp programmers -- if you use/have to use Python or Ruby, which one do you choose and why? |
| 07:14 | manutter | Lisp programmers aren't awake this early ;) |
| 07:15 | Cozey | :-))) |
| 07:16 | vijaykiran | Hi all a quick question - I want to create a simple REST based app using compojure - which JSON library should I use ? |
| 07:20 | Vinzent | vijaykiran, take a look at clojure.contrib.json |
| 07:22 | raek | Vinzent: clojure.contrib.json has now been lifted to its own project: https://github.com/clojure/data.json |
| 07:22 | raek | [org.clojure/data.json "0.1.0"] |
| 07:22 | raek | (works with both clojure 1.2 and 1.3) |
| 07:22 | raek | vijaykiran: ^ |
| 07:22 | Vinzent | oh, ok, i thought it requires 1.3 |
| 07:22 | vijaykiran | Vinzent & raek: thanks! |
| 07:23 | timvisher | hey all |
| 07:23 | timvisher | i find myself pulling out google whenever i want to look for a reference to a namespace |
| 07:24 | timvisher | `doc` is great if i know the function i'm looking up |
| 07:24 | timvisher | `find-doc` is great if i don't, but it doesn't seem to work for namespace prefixes, i.e. `(find-doc "clojure.pprint/*")` doesn't return all the functions in that namespace |
| 07:24 | raek | Cozey: kinda OT answer: I like Python 3 because it has a good string model (i.e. strings are sequences of characters rather than bytes, so you don't work with the data in its encoded form) |
| 07:24 | timvisher | is there any way to get the references that i find at clojure.org at the repl? |
| 07:25 | raek | ...but I've heard that they have fixed the string model in Ruby too |
| 07:25 | manutter | timvisher: dunno about the repl, but I use clojuredocs.org for a lot of that kind of stuff |
| 07:26 | timvisher | manutter: thanks for the pointer! hadn't seen that before |
| 07:26 | raek | timvisher: print-namespace-doc |
| 07:26 | timvisher | i'm primarily interested in it because i want access to these docs offline, and i want them to be specific to what the version i'm working with |
| 07:26 | Vinzent | timvisher, not exactly what you want, but i use auto-complete which shows all public vars and doc-strings in the ns |
| 07:27 | Cozey | raek: will Ruby has similar migrating difficulties as python2->python3 ? |
| 07:27 | timvisher | hmm |
| 07:28 | timvisher | raek: that looks like what i want, but calling it like `(print-namespace-doc 'clojure.pprint)` returns clojure.pprint \n nil |
| 07:28 | raek | Cozey: from what I have heard, they added the support in a non-major version |
| 07:28 | Cozey | mhm |
| 07:28 | Vinzent | timvisher, it prints docstring of the ns |
| 07:28 | raek | and that sounds scary to me |
| 07:29 | timvisher | vinzent: ah |
| 07:29 | cemerick | timvisher: there's also http://clojureatlas.com </self-promotion> |
| 07:30 | timvisher | cemerick: that looks cool to |
| 07:30 | timvisher | i still want the offline thing. |
| 07:30 | raek | I'm so happy that Clojure got the string model thing right... |
| 07:30 | timvisher | maybe it just doesn't exist at the moment |
| 07:31 | timvisher | i thought there may be a simple function that they call to generate the online docs |
| 07:31 | raek | well, it's mostly Java's effort... |
| 07:31 | cemerick | timvisher: you'll be able to download a static copy of your atlas very shortly :-) |
| 07:31 | timvisher | but it's probably akin to javadoc |
| 07:31 | raek | timvisher: you can clone the gh-pages branch of clojure |
| 07:31 | Vinzent | I'm using appengine-magic and have created a protocol and extended it to the appengine entity, but when I'm calling protocol fn on the entity queried from database, implementation for Object is chosen |
| 07:31 | raek | that way you get the same html files that are on the site |
| 07:31 | Vinzent | example: http://paste.org/pastebin/view/33786 What could be the problem? |
| 07:32 | timvisher | raek: that's an interesting option |
| 07:32 | Cozey | yeah. i need to code something in python and it is a pain in a way now. but from github i see many clojure people useing ruby as welll |
| 07:32 | timvisher | hadn't considered doing that yet |
| 07:35 | Vinzent | No ideas? Am I missing something obvious? |
| 07:35 | manutter | timvisher: btw, if you poke around clojuredocs.org, there's a downloadable standalone archive. I got it running on my laptop fairly easily (though iirc it's a Rails webapp!) |
| 07:36 | raek | Vinzent: maybe you extend the protocol to the wrong type or something... |
| 07:37 | raek | also, I'm not very sure how ambiguities are handled (your type is both of the entity type and of the Object type) |
| 07:37 | timvisher | manutter: thanks. that's a litle like Apple using those windows mobile devices for awhile in there stores for their mobile point of sale devices in their stores. :) |
| 07:38 | manutter | Heh |
| 07:38 | timvisher | is there an easier way than including multiple Exceptions in a catch clause than multiple (catch...) forms? |
| 07:38 | Vinzent | raek, no, when i'm creating object explicitly it works well, so type can't be wrong... |
| 07:38 | timvisher | i.e. a seq of Exception types or something along those lines? |
| 07:38 | Vinzent | raek, but every type is an Object! |
| 07:38 | timvisher | (catch [FooException IndexException BlahException]...)? |
| 07:38 | raek | Vinzent: http://groups.google.com/group/clojure-dev/browse_thread/thread/fb3a0b03bf3ef8ca/d335cce2f83049b3?lnk=gst&q=prefer+mechanism+for+protocols#d335cce2f83049b3 |
| 07:39 | manutter | timvisher: I guess you could just catch RuntimeException and then check for which subclass the actual exception is? |
| 07:39 | timvisher | manutter: mmm. trying to avoid that |
| 07:40 | timvisher | i could always extract the catch function out and then call that from each catch statement |
| 07:40 | raek | timvisher: you can with a macro |
| 07:40 | manutter | true |
| 07:43 | Vinzent | raek, but my entity type is an descendant of the Object; i believe such cases are handled well |
| 07:43 | Vinzent | anyway i'll check this |
| 07:55 | Vinzent | raek, thanks, you are right: i've removed default implementations and all works fine now; i'm completely confused with that |
| 08:00 | raek | Vinzent: with 'extend', you can "mix" the implementation from multiple sources. it expects a map, so you use ordinary functions (assoc etc) to compose the new implementation |
| 08:01 | raek | Vinzent: an example https://github.com/clojure/clojure/blob/b578c69d7480f621841ebcafdfa98e33fcb765f6/src/clj/clojure/java/io.clj#L158 |
| 08:04 | Vinzent | raek, yes, i know that, just thought protocols "know" about inheritance |
| 08:05 | Vinzent | so user don't have to extend protocol on his type unless he needs special behaviour |
| 08:06 | raek | I know that they were not designed to provide inhertitance, but I cannot present any of the "inheritance is bad" arguments |
| 08:06 | raek | well, except for reuse with maps in extend |
| 08:07 | raek | all I know is that protocols were never meant to be a class system like that of java |
| 08:13 | dnolen | Vinzent: raek: a longish but excellent explication around the issues around inheritance. tl;dr, implementations have many assumptions backed in, inheritance locks you into those assumptions. Better to program against protocols. In Clojure the benefits are obvious, trivial to create things from scratch which interoperate cleanly w/ the entire system - i.e. your own IPersistentSet. |
| 08:13 | dnolen | http://www2.parc.com/csl/groups/sda/projects/oi/towards-talk/transcript.html |
| 08:15 | dnolen | inheritance only gets you convenience, protocols get you much further - scalability and sustainability. |
| 08:35 | Vinzent | dnolen, thanks, I'll definitely read it tonight! |
| 08:39 | dnolen | Vinzent: it's a good read. Objective-C also takes the inheritance is bad stance + explicit support for protocols. Java interfaces were basically lifted from Objective-C protocols as far as I understand. http://www.virtualschool.edu/objectivec/influenceOnJava.html |
| 08:40 | dnolen | old ideas. |
| 08:57 | Vinzent | dnolen, interesting point. I don't know much about obj-c (just heard that it's "right" cpp), but it would be good to learn (at least briefly) other, non-lispy languages to understand clojure better |
| 09:07 | Vinzent | btw, can I somehow re-extend protocol dinamically? |
| 09:10 | dnolen | Vinzent: if you mean add methods to a protocol, then no. |
| 09:12 | Vinzent | dnolen, no, I mean redefining from the repl/recompiling, but looks like the problem is in appengine |
| 09:12 | dnolen | Vinzent: oh yeah, definitely. |
| 10:17 | amac_ | hmm, anyone know how I can use contrib.sql/insert-records and find the auto-inc primary key the record was assigned... querying on the provided columns might bring back multiple rows |
| 10:47 | cemerick | OK, continuing on my stream of numerics-related ignorance: bigints are contagious, so (+ 1 XXXN) => XXYN. Why aren't BigDecimals contagious as well? As things stand, (+ 1.0 1e400M) => Infinity. |
| 10:48 | cemerick | It seems like the loss of precision there is undesirable…? |
| 11:08 | joegallo | cemerick: 1.0 as a double isn't very precise, though, is it? wouldn't making the result be a bigdecimal mean a magical gain in precision? double + bigdecimal ==> double because you can't manufacture precision out of nowhere. the same doesn't apply for an integer because they are already precise. (caveat: i have no clue what i'm talking about.) |
| 11:16 | cemerick | joegallo: fair point, that may be it. |
| 11:18 | TimMc | or... maybe it's a bug :-) |
| 11:18 | cemerick | Perhaps the "right" thing to do might be to promote as necessary, and then flatten the precision of the result down to the least-precise input you have. |
| 11:19 | cemerick | TimMc: I don't think so; there's never been any discussion of BigDecimal contagion, so I have to assume that that's intended. I figure I'll ask some stupid questions to understand why, though. :-) |
| 11:20 | TimMc | joegallo's explanation is pretty convincing from a semantics perspective, though |
| 11:21 | TimMc | Does BigDecimal has a larger exponent range than doubles? |
| 11:21 | cemerick | arbitrarily so |
| 11:22 | TimMc | I suppose it has to. |
| 11:22 | TimMc | Also, I can't English today. >_< |
| 11:23 | pjstadig | yeah i think it is basically what joegallo says |
| 11:23 | pjstadig | there's like essentially a subset relationship between the various ints |
| 11:23 | pjstadig | every long can also be represented by a big int |
| 11:23 | pjstadig | but i guess it's not the same way with floating point, or so i've been lead to understand |
| 11:30 | cemerick | pjstadig: NaN and infinity are unrepresentable in BigDecimal (AFAICT), so perhaps that's it. |
| 11:32 | pjstadig | there is also a difference between 1.1 and 1.10 in terms of precision (or accuracy i forget?) so upcasting from a double to a BigDecimal may essentially add zeroes onto the end of the base or something |
| 11:33 | TimMc | cemerick: Ooof, good point. |
| 11:33 | pjstadig | (caveat: i also have no clue what i'm talking about) |
| 11:34 | cemerick | pjstadig: Same here, but negatory on that last one: |
| 11:34 | cemerick | ,(java.math.BigDecimal/valueOf 1.1) |
| 11:34 | clojurebot | 1.1M |
| 11:34 | cemerick | *shrug* |
| 11:35 | pjstadig | cemerick: well it prints that way, but how is in represented in memory? |
| 11:35 | pjstadig | meh |
| 11:35 | pjstadig | it's a problem i don't have :) (having to worry about all this precision stuff) |
| 11:36 | cemerick | ,(.precision (java.math.BigDecimal/valueOf 1.1)) |
| 11:36 | TimMc | ,(java.math.BigDecimal. Double.POSITIVE_INFINITY) |
| 11:36 | clojurebot | 2 |
| 11:36 | clojurebot | java.lang.ClassNotFoundException: Double.POSITIVE_INFINITY |
| 11:36 | TimMc | ,(java.math.BigDecimal. Double/POSITIVE_INFINITY) |
| 11:36 | cemerick | so, it's not inflating precision or anything |
| 11:36 | clojurebot | java.lang.NumberFormatException: Infinite or NaN |
| 11:36 | cemerick | yeah, there's no infinity in BD |
| 11:36 | TimMc | however |
| 11:36 | TimMc | ,(.doubleValue 1e9999999999999999999999) |
| 11:36 | clojurebot | Infinity |
| 11:37 | pjstadig | i have on my reading list some paper (that was footnoted in JoC) about how floating points are evil...guess i should read it |
| 11:44 | dnolen | ,(/ (BigDecimal. 1.05) 3) |
| 11:44 | clojurebot | java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result. |
| 11:45 | dnolen | ^ a really good reason for no BigDecimal contagion |
| 11:47 | cemerick | dnolen: Good point. |
| 11:48 | cemerick | I guess * and / involving BigDecimals could rationalize them first… |
| 11:48 | TimMc | dnolen: Not sure I see it... |
| 11:48 | TimMc | could pass in a rounding mode. |
| 11:48 | TimMc | / could, that is |
| 11:50 | dnolen | TimMc: if it's rounding what's the point? |
| 11:50 | TimMc | hm |
| 13:38 | amalloy | technomancy: looks like http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8708#10 is fixed in trunk |
| 13:41 | technomancy | amalloy: sweet; thanks for following up on that |
| 13:41 | amalloy | technomancy: i couldn't bear to have the bug blamed on me, see :) |
| 13:42 | technomancy | heh. do you know how long the bug's been around? |
| 13:42 | technomancy | wondering if I should drop the check or if I should assume for a while that all 24 users are affected |
| 13:42 | amalloy | technomancy: uh. well, i reported it last week. if you mean, how long it's been broken, i don't know |
| 13:42 | amalloy | but git blame ought to reveal it pretty quickly |
| 13:43 | edw | Yay, amalloy and technomancy! Thanks for looking into this. |
| 14:33 | coopernurse | trying to get my clojure/emacs/slime setup going on Aquaemacs. Installed clojure-mode via marmelade, but when I restart emacs the mode isn't available. do I need to add anything to my .emacs file? |
| 14:33 | coopernurse | I'm a little confused where the .el files were installed to |
| 14:35 | technomancy | it should be in ~/.emacs.d/elpa, but in general forks of GNU Emacs are not supported. |
| 14:35 | coopernurse | ah, ok thanks |
| 14:35 | coopernurse | I see stuff here: /Users/james/Library/Application Support/Aquamacs Emacs/elpa |
| 14:36 | coopernurse | but it looks like they're not being loaded when Aquamacs starts |
| 14:36 | coopernurse | I'll try the other emacs distro |
| 14:37 | desertman | sorry i seem to have a connection problem |
| 14:41 | desertman | i don't know if i already posted this or not because of a connection problem but i am trying to undertstand contrib.sql http://richhickey.github.com/clojure-contrib/sql-api.html , ie (do-commands "show databases") returns a clojure.lang.ArraySeq$ArraySeq_int, I would like to get a list of the databases. |
| 14:42 | ordnungswidrig | *grr* is there any sensible imap mail client on osx. thunderbird eats my cpu |
| 14:43 | drewr | people like sparrowapp.com |
| 14:43 | raek | desertman: clojure.lang.ArraySeq *is* a seq. are you sure you're not using (a potentially implicit) str instead of some of the printing functions? (pr prn pr-str prn-str) |
| 14:43 | drewr | I find it doesn't support gmail's imap well |
| 14:44 | ordnungswidrig | drewr: I have a *real* imap server :) |
| 14:44 | raek | ,(pr-str (seq (to-array [1 2 3]))) |
| 14:44 | clojurebot | "(1 2 3)" |
| 14:44 | raek | ,(str (seq (to-array [1 2 3]))) |
| 14:44 | clojurebot | "(1 2 3)" |
| 14:44 | raek | hrm |
| 14:44 | raek | ,(str (lazy-seq [1 2 3])) |
| 14:45 | clojurebot | "clojure.lang.LazySeq@7861" |
| 14:45 | drewr | ordnungswidrig: you could try it then |
| 14:45 | ordnungswidrig | drewr: I will. thanks for the point |
| 14:45 | coopernurse | I was missing: (package-initialize) |
| 14:45 | ordnungswidrig | I tried of measure the number of retries of a dosync block by incrementing a atom |
| 14:45 | coopernurse | in .emacs.. I'm a ELPA n00b |
| 14:46 | ordnungswidrig | But I do not really trust the numbers |
| 14:46 | manutter | ,(apply str (lazy-seq [1 2 3])) |
| 14:46 | clojurebot | "123" |
| 14:46 | manutter | hrm, what was that other command now... |
| 14:46 | raek | ,(apply str (lazy-seq [(lazy-seq [1]) (lazy-seq [2]) (lazy-seq [3])])) |
| 14:46 | clojurebot | "clojure.lang.LazySeq@20clojure.lang.LazySeq@21clojure.lang.LazySeq@22" |
| 14:46 | desertman | (type (do-commands "show databases")) returns clojure.lang.ArraySeq$ArraySeq_int , which is an array of integers |
| 14:47 | raek | desertman: not actually an array, but an array seq |
| 14:47 | ordnungswidrig | using pcalls to start 100 parallel executions on 2 refs on a 2 core machine should give some retries, shouldn't it? |
| 14:47 | raek | ,(seq (to-array [1 2 3])) |
| 14:47 | clojurebot | (1 2 3) |
| 14:47 | raek | ,(type (seq (to-array [1 2 3]))) |
| 14:47 | clojurebot | clojure.lang.ArraySeq |
| 14:48 | raek | desertman: it works and prints like an ordinary sequence |
| 14:48 | manutter | desertman: ah, yes, you're getting back the results of executing a sql command |
| 14:48 | manutter | desertman: do-commands is intended for non-select sql commands like drops and inserts |
| 14:49 | manutter | desertman: You're probably getting back a 1 element array containing the number of rows returned by the "show databases" command |
| 14:50 | desertman | yes |
| 14:51 | manutter | If you want to see the actual database names, you need (with-query-results results ["show databases"] (doseq [row results] (println row))) |
| 14:52 | manutter | or something like that |
| 14:53 | desertman | ok |
| 14:55 | desertman | <manutter> - yes, that works, thank you |
| 14:57 | manutter | :) |
| 15:00 | desertman | <manutter> although the syntax is a bit strange to me, , because it seems that '(doseq [row results] (println row))' is being passed as an argument to with-query-results results ? |
| 15:01 | manutter | It's the body of the with-query-results macro -- wraps your code in some setup/cleanup stuff |
| 15:03 | manutter | ,(doc clojure-contrib.sql/with-query-results) |
| 15:03 | clojurebot | Cool story bro. |
| 15:03 | desertman | ok, i wouldn't have seen that in the docs, but i guess you need to look at the source to know what these things do, although i'm not there yet |
| 15:04 | coopernurse | Ok, got my emacs/slime setup working. thanks for the tip on the other emacs distro |
| 15:04 | coopernurse | I never cared for all the extra junk in aquamacs anyhow |
| 15:05 | technomancy | "training wheels on a humvee" is one description I've heard =) |
| 15:05 | manutter | desertman: there's a few other macros that do similar things, like with-out-str etc |
| 15:06 | coopernurse | technomancy: heh.. after years of Java development, this clojure/repl/slime setup is pretty interesting. love reloading code in the repl without restarting the vm |
| 15:07 | coopernurse | I'm going through some project euler problems and realizing I know _nothing_ about FP :-) |
| 15:07 | coopernurse | my solutions are 10x too long |
| 15:11 | manutter | ,(doc clojure-contrib.sql/with-query-results) |
| 15:11 | clojurebot | Cool story bro. |
| 15:12 | manutter | no, doh, that's what I typed last time |
| 15:12 | manutter | ,(doc clojure.contrib.sql/with-query-results) |
| 15:12 | clojurebot | "([results sql-params & body]); Executes a query, then evaluates body with results bound to a seq of the results. sql-params is a vector containing a string providing the (optionally parameterized) SQL query followed by values for any parameters." |
| 15:12 | manutter | there we go |
| 16:03 | markoman | hi. is there some standard way to formalize strings to ascii form? I need this for searching string from datastore |
| 16:04 | markoman | im thinking something like: Caté1 -> cate1 |
| 16:07 | ampleyfly | you want cate1 and Caté1 to collide, then? |
| 16:07 | coopernurse | markoman: stackoverflow thread that is relevant to what you want to do |
| 16:07 | coopernurse | http://stackoverflow.com/questions/5651124/is-there-a-way-to-dumb-down-text-from-unicode-to-ascii |
| 16:07 | coopernurse | not clojure specific.. |
| 16:08 | coopernurse | but it seems like your first challenge is to make a mapping, since ascii can't represent all unicode characters (obviously), so it's not clear what heuristic you want to use |
| 16:08 | TimMc | markoman: What should بينظير ڀٽو become? |
| 16:08 | coopernurse | TimMc: right |
| 16:09 | markoman | good question, i have no idea... |
| 16:09 | hiredman | you render each string as set of paths and then find best fits |
| 16:09 | markoman | and is there a upper and lower case version of بينظير ڀٽو? |
| 16:09 | TimMc | I work at a company that makes transliteration software, but I don't happen to know the open source answer to this. |
| 16:09 | hiredman | good luck |
| 16:16 | markoman | there is this example in java: String unicode = "Unicode: \u30e6\u30eb\u30b3\u30fc\u30c9"; byte[] bytes = unicode.getBytes("US-ASCII"); how could I test it on clojure repl? |
| 16:17 | TimMc | markoman: That will just get you munged data. |
| 16:18 | raek | ,(String. (.getBytes "Unicode: \u30e6\u30eb\u30b3\u30fc\u30c9" "US-ASCII") "US-ASCII") |
| 16:18 | clojurebot | "Unicode: ?????" |
| 16:19 | amalloy | better than Unrecognizable Rectangles |
| 16:19 | raek | the default behaviour seems to be to replace unrepresentable characters with \? |
| 16:20 | markoman | yeah, it replace nontransable chars to question marks, not good, but some start |
| 16:21 | amalloy | markoman: it is a start down the path of evil. unicode is a much larger set than ascii; trying to downconvert will bring only pain |
| 16:21 | raek | amalloy: tell that to "Mr. Yurukôdo" |
| 16:21 | raek | ahem. "Yurikôdo", i.e. "Unicode" |
| 16:23 | raek | I think that's it not very useful for the Japanese (the string contains Katakana) if their text is replaced with questionmarks |
| 16:23 | markoman | thats true |
| 16:23 | amalloy | that's what i said, right? don't convert unicode to ascii? |
| 16:23 | markoman | well then is string to upper and lower case smart enough, it works with unicode? |
| 16:24 | raek | amalloy: oh, sorry. I accidentally read "better than Unrecognizable Rectangles" as a comment on "Unicode: ?????" |
| 16:24 | amalloy | markoman: insofar as such a thing is possible, yes, it works |
| 16:24 | amalloy | raek: yeah, sorry. better, but not actually good |
| 16:25 | amalloy | "Unicode: ?????" neatly sums up the developer's feelings: "I clearly don't understand Unicode" |
| 16:25 | raek | :) |
| 16:25 | markoman | I could also try to make a map, as suggested, but for special punctuations only, partial solution |
| 16:25 | TimMc | markoman: http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail |
| 16:26 | raek | markoman: if you only consider the latin alphabet, maybe you could normalize the string to NFKD form (Normalization Form Compatibility Decomposition) and strip all the combining characters |
| 16:26 | amalloy | TimMc: that too. but java's toLowercase gets that right, doesn't it? |
| 16:27 | TimMc | amalloy: What does "right" mean? |
| 16:27 | TimMc | You have to know the language of origin. |
| 16:27 | amalloy | converts uppercase dotted-i to lowercase dotted-i, and likewise for dotless? |
| 16:27 | raek | markoman: this can problably do something useful: http://site.icu-project.org/ |
| 16:28 | amalloy | i guess not in all cases |
| 16:28 | amalloy | i see what you mean about language of origin |
| 16:28 | markoman | basically, if I use lower case to this: بينظير ڀٽو does the meaning change totally different? |
| 16:28 | raek | it basically knows the stuff about the letters that human know |
| 16:28 | markoman | it would be a problem for getting right results with searches... |
| 16:29 | raek | markoman: it won't change it, since the Arabic script does not have the lower/upper case distinction |
| 16:29 | raek | ,(.toLowerCase "بينظير ڀٽو") |
| 16:29 | clojurebot | "بينظير ڀٽو" |
| 16:29 | TimMc | markoman: Are you beginning to see why there are are so few search engines? :-) |
| 16:29 | amalloy | TimMc: b/c for turkish lowercase of I is dotless, but for english the lowercase is dotted? that's the point you're making, right? |
| 16:30 | TimMc | amalloy: Indeed. |
| 16:30 | raek | ,(let [s "بينظير ڀٽو"] (= s (.toLowerCase s) (.toUpperCase s))) |
| 16:30 | clojurebot | true |
| 16:30 | pdk | turkish has two separate letters for I |
| 16:30 | coopernurse | different subject - anyone out there running clojure on a VPS with limited ram? |
| 16:30 | pdk | one has a dot on both lower/upper cases, one has a dot on neither |
| 16:31 | amalloy | coopernurse: for given values of "limited" |
| 16:31 | coopernurse | I'm interested in how small you were able to get the JVM - if you tried.. |
| 16:31 | coopernurse | I was playing with compojure on a rackspace cloud 512mb instance and it worked fine |
| 16:31 | coopernurse | jvm rss was about 125mb |
| 16:31 | coopernurse | with hello world loaded |
| 16:32 | coopernurse | I was curious if anyone has tried to get that smaller.. I had start/max heap set to 32mb |
| 16:32 | hiredman | on osx I can't seem to get the rss of the jvm below 80mb no matter what settings I use |
| 16:32 | coopernurse | hiredman: yeah, that's about what I've seen too |
| 16:32 | hiredman | just fiddling around with it |
| 16:32 | markoman | simple things come easily complicated in this digital world |
| 16:33 | TimMc | markoman: You're trying to make a program that can deal with any language. :-P |
| 16:33 | markoman | but hey thanks, I got something to chew |
| 16:33 | TimMc | Rather... it turns out that search engines are not language-agnostic. |
| 16:34 | markoman | nah, Im finding limits and making compromises then :) |
| 16:35 | TimMc | I wish I could share some of the crazy stuff we're dealing with in our own software, but I'm not really sure what's confidential and what isn't. |
| 16:36 | markoman | *promises not to copy and paste everything on twitter* |
| 16:37 | markoman | I get gray hair even with basic character encodings |
| 16:37 | amalloy | haha i'd like to see someone try to distribute their whole codebase in tweet-sized chunks |
| 16:37 | TimMc | public static void main(String[] args) { #loloutofspace |
| 16:41 | raek | markoman: ##(apply str (remove #(= (Character/getType %) Character/NON_SPACING_MARK) (java.text.Normalizer/normalize "åäö" java.text.Normalizer$Form/NFKD))) |
| 16:41 | sexpbot | ⟹ "aao" |
| 16:44 | markoman | hmh... on intelliJ repl it gives questionmarks, need to test lein repl |
| 16:45 | TimMc | markoman: Make sure to have rlwrap installed -- it handles Unicode printing. |
| 16:46 | raek | markoman: install rlwrap if you don't have it. jline, which is the default, is broken and does not have support for UTF-8 |
| 16:46 | markoman | thats clojure lib, or lein extension or java lib? |
| 16:46 | raek | markoman: OS package |
| 16:47 | raek | then the test to confirm everything works: ##(seq "åäö") ;should return *three* characters |
| 16:47 | sexpbot | ⟹ (\å \ä \ö) |
| 16:49 | markoman | ok, found from mac port |
| 16:52 | cemerick | hiredman, coopernurse: I'll bet that mapped byte buffers are being used for your bootclasspath, so a very large chunk of that 80MB is rt.jar and friends. |
| 16:52 | technomancy | so... how hard would it be to create a lazy seq from which items could be dropped without calculating them? |
| 16:53 | coopernurse | cemerick: interesting. I'll google "mapped byte buffers" and learn something |
| 16:53 | hiredman | technomancy: very |
| 16:53 | technomancy | I'm thinking if drop rebound some kind of var, the lazy seq construction could check it and simply skip the work where possible if it knew the results could be discarded. |
| 16:53 | hiredman | (defprotocol internal-drop ...) |
| 16:54 | hiredman | technomancy: seems easier to just have a lazy-seq of delays |
| 16:54 | technomancy | hiredman: did we have this conversation already? I have a vague idea I once determined it'd be harder than it sounds, but I don't recall why. |
| 16:55 | technomancy | certainly easier, but less elegant. |
| 16:55 | cemerick | coopernurse: Looking at URLClassLoader, it's loading class byte[]s from ByteBuffers, and there's a comment that makes me think they're direct 99% of the time (would certainly make sense if the jars are on disk). |
| 16:55 | hiredman | technomancy: yeah, I vaguely recall that, but don't recall specifics |
| 16:57 | markoman | hmh, i still get questionmarks after installing rlwrap and restarting apps |
| 16:57 | hiredman | ~google java file encoding |
| 16:57 | clojurebot | First, out of 323000 results is: |
| 16:57 | clojurebot | utf 8 - Setting the default Java character encoding? - Stack Overflow |
| 16:57 | clojurebot | http://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding |
| 16:57 | coopernurse | cemerick: ah. so does that mean I'm seeing disk blocks that are mmap'd in the RSS figure? |
| 16:57 | markoman | is there something else I need to include when repl starts up? |
| 16:57 | coopernurse | I have a tenuous grasp on how mmap works |
| 16:59 | cemerick | coopernurse: Perhaps. And, if those jars are being mmap'ed from Java, then the corresponding memory allocation happens outside of the heap. |
| 16:59 | cemerick | Thus, the disconnect between -Xmx and such and rss. |
| 16:59 | coopernurse | cemerick: ah, cool. I always wondered why that difference was so huge |
| 16:59 | coopernurse | looks like -XX:MaxDirectMemorySize may be a way to tune that? |
| 17:01 | cemerick | coopernurse: I'll bet that's a hard limit that will cause an exception to be thrown when the classloader attempts to mmap the files, not prompt it to fall back to a non-mmap code path. |
| 17:01 | cemerick | coopernurse: which option reference are you looking at? I don't see that one on http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html |
| 17:01 | coopernurse | cemerick: interesting. and it appears the default is 64MB |
| 17:02 | coopernurse | cemerick: http://download.oracle.com/docs/cd/E15289_01/doc.40/e15062/optionxx.htm |
| 17:02 | coopernurse | cemerick: oops, that's jrockit.. |
| 17:04 | coopernurse | cermerick: normal java -X doesn't show it.. oh well. but I suspect you've explained why you can't really get a JVM down below 100MB RSS in practice. |
| 17:05 | cemerick | coopernurse: there's a *lot* of options that aren't listed on the page I linked. There's a more comprehensive reference, I just can't find it at the moment. |
| 17:06 | markoman | when repl starts on intelliJ it says "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -Dfile.encoding=UTF-8 " but still I get question marks when printing scandic letters |
| 17:07 | offby1 | markoman: might have to do with your terminal program. |
| 17:07 | offby1 | Or your choice of font. |
| 17:09 | coopernurse | cemerick: I see. my goal is to see how many JVMs you could reasonably fit in a given amount of RAM for small-ish webapps. My preference would be to use a separate JVM per app so I can restart them separately. I realize that's not Java's preference, but it would make the Compojure web apps more similar to Ruby/Python apps from an ops standpoint |
| 17:10 | coopernurse | cemerick: hot swapping the WAR files is another option, but my experience with that has been less than stellar (run out of PermGen due to classloader leaks, etc) |
| 17:12 | cemerick | coopernurse: undeploy/redeploy is the good middle ground; you could probably run a dozen or more small webapps in 128MB of heap, and deploy them independently. |
| 17:12 | coopernurse | cemerick: undeploy/redploy - is that a jetty feature? |
| 17:13 | cemerick | coopernurse: any (worthwhile) container supports undeploying an app while leaving the container and other apps undisturbed |
| 17:13 | cemerick | jetty, tomcat, glassfish are all suitable |
| 17:13 | S11001001 | except for the permgen thing |
| 17:14 | cemerick | turn on class GC and you should be fine |
| 17:14 | cemerick | or, use a VM that doesn't have permgen (jrockit is now free) |
| 17:16 | cemerick | coopernurse: I'm not saying you can have heroku-in-a-box on this route, but it certainly works for handfuls of apps. 1:1 app:container setups are never going to be efficient in any way (unless there's a magic config parameter that we've not discovered that will eliminate the rss issue). |
| 17:17 | coopernurse | cemerick: agreed. the JVM seems to favor long running processes. redeploying seems like a decent route -- I guess I've just been bit by that in the past. perhaps I should take a look at jrockit |
| 17:18 | cemerick | coopernurse: some containers (weblogic, IIRC?) have previously offered a "hot redeploy" option that, yes, have never worked as advertised. Doing discrete undeploy + deploy operations are a whole different (simpler) approach, though, and "just work". |
| 17:19 | markoman | yeah, on terminal i needed to do: export JAVA_OPTS="-Dfile.encoding=utf-8" |
| 17:19 | cemerick | Also, the shorter your uptime is, the less you'll get JITed code. |
| 17:19 | coopernurse | cemerick: interesting, I would have figured hot deploy does the same thing as undeploy/redeploy - or is the latter just not atomic, and therefore more reliable? |
| 17:20 | markoman | but intelliJ is harder to get functioning on repl, need to test it on clj files tomorrow, sometimes they still work, just repl not |
| 17:21 | markoman | thanks guys, more next time |
| 17:22 | cemerick | coopernurse: exactly, those are two operations that you run / script as part of your "redeploy" process. Things marketed as "hot redeploy" try to be tricky, probably attempting to do things like what JavaRebel et al. do. |
| 17:23 | cemerick | (and, JavaRebel works well when it can — but there are always changes that cause it to fall over and force a full rebuild / deploy cycle) |
| 18:34 | ssideris | hello, has anyone seen the "make-dependency does not exist" error with leiningen native-deps 1.0.5? |
| 18:35 | ssideris | it seems that native-deps is a bit sensitive to the version of leiningen or something... |
| 18:36 | technomancy | ssideris: what lein version? |
| 18:36 | ssideris | 1.5.0 |
| 18:36 | ssideris | (on windows if that's relevant) |
| 18:36 | technomancy | odd; that should work. |
| 18:36 | technomancy | if you use lein 1.6.0-SNAPSHOT you don't need a plugin for native-deps |
| 18:37 | ssideris | oh, ok let me give it a go |
| 18:37 | ssideris | thanks |
| 18:37 | ssideris | and thanks for leiningen btw :-) |
| 18:37 | technomancy | sure =) |
| 18:39 | technomancy | having make-dependency move was a long time ago though; 1.0.5 shouldn't be affected by that |
| 18:43 | ssideris | sorry just a clarification: does that mean that lein 1.6.0-SNAPSHOT does not differentiate between native and non-native deps? |
| 18:43 | ssideris | do I put everything under :dependencies? |
| 18:44 | technomancy | ssideris: not yet; so far it works the same as the plugin |
| 18:45 | technomancy | except you don't need a separate "lein native-deps" step |
| 18:45 | technomancy | but you do need them in :native-deps in project.clj |
| 18:45 | technomancy | hopefully by the time it's released you can put them in :dependencies though. |
| 18:46 | ssideris | hm, that's odd, there must be something really weird with my setup because it didn't seem to pick up :native-dependencies\ |
| 18:46 | ssideris | it's being ignored... |
| 18:47 | technomancy | hmm... it's very new code; not well-tested yet. could you open an issue? |
| 18:47 | ssideris | ok |
| 18:49 | technomancy | what's the dependency? |
| 18:50 | ssideris | [penumbra/lwjgl "2.4.2"] |
| 18:54 | ssideris | technomancy: https://github.com/technomancy/leiningen/issues/207 |
| 18:54 | ssideris | opened! |
| 18:54 | ssideris | thanks |
| 19:47 | technomancy | ssideris: if you can take a look it would help to get it fixed faster; I don't have any experience with native deps and have a few search features I need to finish first |
| 19:47 | technomancy | but I'll get to it before the release if not |
| 19:50 | ssideris | technomancy: erm, wouldn't know where to start, but if you can direct me a bit I would be happy to give it a go |
| 19:51 | technomancy | here's the function in question: https://github.com/technomancy/leiningen/blob/master/src/leiningen/deps.clj#L144 |
| 19:51 | technomancy | pretty much taken verbatim from the plugin |
| 19:53 | ssideris | ok, I'll see if I find anything, but don't put too much faith in my clojure abilities :-) |
| 21:16 | mec | Is there anywhere that has the syntax for literal defrecords in alpha 7? |
| 21:17 | amalloy | mec: it's just #user.Foo[1 2] or #user.Foo{:a 1 :b 2} |
| 21:18 | ataggart | http://dev.clojure.org/display/design/defrecord+improvements |
| 21:20 | ataggart | and it's not just for records, but a generic constructor literal. |
| 21:31 | mec | ataggart: so its good for any java class? |
| 21:38 | ataggart | yes, e.g. #java.util.Date[12345] |
| 21:38 | ataggart | bear in mind that this is really intended for print-dup, not something you should be typing out. |
| 22:12 | WuHoUnited | hello clojurebot |
| 22:14 | WuHoUnited | (doc clojurebot) |
| 22:14 | clojurebot | It's greek to me. |
| 22:14 | WuHoUnited | (source clojurebot) |
| 22:15 | amalloy | clojurebot: source? |
| 22:15 | clojurebot | source is http://github.com/hiredman/clojurebot/tree/master |
| 22:15 | amalloy | WuHoUnited: ^ |
| 22:15 | WuHoUnited | ^_^ |
| 22:15 | amalloy | $whatis source |
| 22:15 | sexpbot | source is http://github.com/cognitivedissonance/sexpbot |
| 22:15 | amalloy | (also that if you're interested) |