2009-08-31
| 00:13 | Anniepoo | what's the difference between clojure.main and clojure.lang.Repl? |
| 00:24 | rlb | Anniepoo: main is for running applications, Repl is for an interactive read-eval-print loop. |
| 00:24 | rlb | i.e. a prompt |
| 00:26 | Anniepoo | Other than that main exits after loading it's cmd line arg scripts and the Repl gives, obviously, the Repl, are there differences? |
| 00:28 | Anniepoo | realized while doing some work on a project involving Clojure on Google App Engine that I'm slightly fuzzy on this stuff |
| 00:29 | rlb | I don't know offhand. I imagine they might set up the environment a bit differently. |
| 00:29 | Anniepoo | ok, thanks rlb |
| 00:29 | rlb | sure |
| 00:30 | hiredman | clojure.main is newer and combines clojure.lang.Repl and Script |
| 00:30 | hiredman | with a few extra doodads |
| 00:31 | Anniepoo | ah, thanks |
| 00:50 | j-dot | Anniepoo: have you had any specific problems incorporating Clojure in an App Engine project? |
| 00:51 | benatkin | Can anyone tell me why (let [+ 3] +) evals to 3, while (let [+ 3] (::+ + 4)) evals to 4, and not 7? |
| 00:52 | j-dot | ,::+ |
| 00:52 | clojurebot | :sandbox/+ |
| 00:52 | j-dot | the ::+ does not represent the function clojure.core/+ |
| 00:53 | j-dot | it represents the keyword :current-ns/+ |
| 00:53 | Anniepoo | j-dot - no, it's gone amazingly smoothly so far |
| 00:53 | Anniepoo | mike hinchy's got more expertise doing it, he gave a talk at baclojure on it |
| 00:53 | j-dot | Anniepoo: glad to hear :) |
| 00:53 | j-dot | nice |
| 00:53 | Anniepoo | but we're doing it, and I'm new to Google App engine, my partner doesn't know Clojure |
| 00:55 | Anniepoo | hmmm.... I've got slime running in emacs, I think I've got swank in emacs talking properly to swank-server in an external repl, but slime is attempting to evaluate my clojure as lisp |
| 00:55 | j-dot | I haven't done much more than play, but I've had good luck so far with this: http://github.com/jmcconnell/appengine-clj/tree/master |
| 00:56 | benatkin | j-dot: thanks. I didn't account for the possibility that ::+ wasn't adding. I thought it would have barfed on the number 3 as its first param, though. |
| 00:56 | Anniepoo | hey, thanks!! |
| 00:57 | j-dot | my pleasure |
| 00:57 | Anniepoo | yah, this is amazingly great, it will save us a ton of work |
| 00:57 | j-dot | glad to hear! |
| 00:59 | j-dot | benatkin: I haven't looked into it, but I bet that ::+, when applied as a function, is seeing that it can't be applied to the value +, since it's not a map, and is returning the default value 4 |
| 00:59 | j-dot | anyway ... off to bed for me |
| 00:59 | benatkin | j-dot: g'night. I figured out that keywords just return their default value when you don't use a hash :) |
| 01:15 | Anniepoo | I've got slime up and running, apparently, in emacs, but C-x C-e still gives me lisp |
| 01:46 | noidi | when defining a map, is it possible to refer to another key's value? |
| 01:47 | noidi | something like {:r 5, :d (* 2 (% :r))}, where the % would refer to the map itself |
| 01:49 | ambient | Anniepoo C-c C-k evaluates the full file for me and C-M-x evaluates expression SLIME/Clojure |
| 01:49 | ambient | iirc |
| 01:49 | Anniepoo | thanks ambient |
| 01:49 | ambient | those can be found on the SLIME menu i think |
| 01:49 | Anniepoo | I see no menus |
| 01:50 | ambient | emacs -nw? use the gui |
| 01:50 | Anniepoo | windows native emacs |
| 01:50 | Anniepoo | it trashes it's menus when I load .emacs |
| 01:50 | ambient | well i have that also |
| 01:51 | Anniepoo | somehow I'm clobbering them when I get the slime stuff going |
| 01:51 | ambient | strange |
| 01:51 | Anniepoo | and C-M-x gives me lisp, not clojure, as well |
| 01:52 | ambient | well then it's well and borked |
| 01:52 | Anniepoo | ok |
| 01:52 | Anniepoo | that's useful to know, thanks |
| 01:52 | ambient | why not just install ClojureBox? |
| 01:53 | ambient | just set HOME to your user directory so .emacs goes to the right place |
| 01:53 | Anniepoo | hmm... I think it's getting the right emacs |
| 01:53 | Anniepoo | s/emacs/.emacs/ |
| 01:54 | ambient | i wrote down how I did the whole install thingie http://tommih.blogspot.com/2009/08/emacs-clojure-and-windows.html |
| 01:54 | Anniepoo | ah, lovely |
| 01:54 | Anniepoo | thanks |
| 01:55 | Anniepoo | I'm afraid to follow it, danlei spent like 5 hours helping me get this set up |
| 01:56 | ambient | well it's not that hard, just set some paths, install git, ant, java and clojure-mode and hit GO |
| 01:56 | ambient | done |
| 01:56 | tomoj | damn, 5 hours? |
| 01:56 | tomoj | is your emacs like, totally pimped out? |
| 01:56 | Anniepoo | no |
| 01:56 | Anniepoo | I'm a complete emacs noobie |
| 01:58 | Anniepoo | does this get slime and swank and such? |
| 01:58 | ambient | this has pretty cood "cheat-sheet" for all things emacs & clojure http://lifeofaprogrammergeek.blogspot.com/2009/03/learning-clojure-and-emacs.html |
| 01:58 | ambient | yes |
| 01:59 | ambient | but I'd recommend just installing ClojureBox |
| 01:59 | ambient | http://clojure.bighugh.com/ |
| 01:59 | ambient | just click a button and everything works |
| 01:59 | Anniepoo | ok, this looks like it might do it |
| 02:00 | Anniepoo | I'll save off my .emacs and try it |
| 02:00 | Anniepoo | danlei will kill me |
| 02:00 | ambient | and .emacs.d |
| 02:03 | Anniepoo | Pray for my soul |
| 02:04 | ambient | you probably had just one line wrong on the .emacs anyway :D |
| 02:04 | Anniepoo | oh, it was a long saga |
| 02:04 | ambient | it's not that complicated |
| 02:04 | ambient | ELPA & clojure-install make things extremely simple |
| 02:06 | tomoj | apparently clojure-install doesn't work on windows |
| 02:07 | ambient | um, it does. I did it twice yesterday, with windows 7 and windows vista, separate machines |
| 02:07 | tomoj | maybe depends on which git you have then |
| 02:07 | ambient | yes, it does, and also ant, and java |
| 02:07 | ambient | if you don't have those in the path it will break |
| 02:08 | tomoj | if you use cygwin git then it has no chance of working, right? |
| 02:08 | tomoj | I dunno, I'm just speculating, I haven't touched windows in years |
| 02:08 | ambient | i have no idea |
| 02:09 | ambient | you could just probably make a link of the executable and place it in emacs root in the worst scenario |
| 02:11 | Anniepoo | I seem to be in a slime repl |
| 02:11 | ambient | write (+ 1 1) and (defn foo [x] (+ 1 x)) and stuff to see if it works |
| 02:12 | Anniepoo | seems to be |
| 02:12 | Anniepoo | and I typed in some clojure that isn't valid CLisp to make sure |
| 02:13 | ambient | im using mostly REPL myself to learn Clojure |
| 02:13 | ambient | alt-n and alt-p rewind and forward the previous commands |
| 02:14 | Anniepoo | C-M-x isn't producing any visible results |
| 02:15 | ambient | it only works when you have file open and click definitioni in that |
| 02:15 | ambient | in repl you just press the enter key |
| 02:15 | Anniepoo | ok, my mistake |
| 02:15 | ambient | but yeah, im still myself learning slime and emacs, there are a lot of useful keybindings |
| 02:16 | Anniepoo | I opened a file, typed in (+ 1 2) I get 3 in the minibuffer |
| 02:16 | LauJensen | Awesome :) |
| 02:16 | Anniepoo | but if I change it to a println the output doesn't appear |
| 02:17 | Anniepoo | ah! |
| 02:17 | Anniepoo | output is at repl |
| 02:17 | Anniepoo | 8cD |
| 02:17 | LauJensen | Anniepoo: You're on SLIME now ? |
| 02:17 | ambient | you do know how repl works? |
| 02:17 | Anniepoo | I'm snug as a bug in a rug, happy as a clam |
| 02:18 | Anniepoo | well, I know what a repl is, I've used one befor |
| 02:18 | Anniepoo | before |
| 02:18 | Anniepoo | I'm coming off using intellij to write clojure |
| 02:18 | Anniepoo | (feel like I'm coming off heroin) |
| 02:18 | LauJensen | Anniepoo: You're on SLIME now ? |
| 02:18 | Anniepoo | I am indeed sliming away |
| 02:19 | Anniepoo | I'm all slimy |
| 02:19 | ambient | i dont even know how to paste into emacs from windows |
| 02:19 | ambient | i probably should find out |
| 02:20 | LauJensen | Alright, then just one word of caution. Printlns and similar outputs to *out* will go into the slime-repl-buffer, except when its coming from a thread, then it will most likely go into the inferior-lisp-buffer, or sometimes, no buffer at all :) Has tripped me up a few times |
| 02:20 | LauJensen | C-y = paste |
| 02:20 | Anniepoo | ah, good to know |
| 02:20 | Anniepoo | C-y isn't doing it for me |
| 02:20 | LauJensen | (setq x-select-enable-clipboard t) |
| 02:20 | LauJensen | Add that to .emacs or eval in ielm |
| 02:20 | clojurebot | emacs is best configured for Clojure with instructions at http://technomancy.us/126 |
| 02:20 | ambient | LauJensen gracias |
| 02:21 | LauJensen | np |
| 02:21 | ambient | strange though, it didn't work previously |
| 02:23 | Anniepoo | any idea where clojure-box will put .emacs? |
| 02:23 | ambient | C-x d <enter> |
| 02:23 | tomoj | one thing I found pretty cool about slime is that the slime-inspector shows java methods |
| 02:23 | ambient | and you're in that directory |
| 02:23 | ambient | tomoj how? |
| 02:23 | tomoj | so if you do C-c I Math <RET>, you see all the methods in java.lang.MAth |
| 02:24 | tomoj | it doesn't know how to look at clojure stuff yet though :/ |
| 02:24 | ambient | um, clojure stuff works with just tab, right? |
| 02:24 | tomoj | well completion |
| 02:24 | ambient | C-c I is undefined :/ |
| 02:25 | tomoj | well, the function is slime-inspect |
| 02:25 | tomoj | guess I got a binding from somewhere |
| 02:25 | tomoj | also C-c C-d d (slime-describe-symbol) is cool |
| 02:25 | LauJensen | ambient: C-c I (capital i) ? |
| 02:26 | ambient | yes |
| 02:26 | Anniepoo | ah, for others interst it's in application data |
| 02:26 | LauJensen | Anniepoo: Is Clojurebox using technomancys install-mode ? |
| 02:26 | ambient | Anniepoo set HOME environment variable to your home dir |
| 02:26 | ambient | and then copy .emacs* there |
| 02:27 | LauJensen | ambient: C-h b ? |
| 02:27 | ambient | perhaps I should revise my clojure installation according to technomancy |
| 02:27 | Anniepoo | clojurebox thinks hope is the application data dir |
| 02:29 | ambient | LauJensen nvm I was in the wrong buffer |
| 02:29 | Anniepoo | ah, lovely. |
| 02:29 | Anniepoo | fixing my HOME even got my menus back |
| 02:31 | ambient | is there any way to get a listing of all working SLIME functionality with clojure? |
| 02:31 | ambient | some functions seem to be unmapped |
| 02:31 | ambient | or not working |
| 02:34 | ambient | back to reading this http://common-lisp.net/project/slime/doc/html/ |
| 02:34 | Anniepoo | strange, now it won't let me make a new file |
| 02:35 | jdz | it? |
| 02:35 | Anniepoo | emacs |
| 02:35 | jdz | how are you doing it? |
| 02:35 | LauJensen | Anniepoo: Given what reason ? |
| 02:36 | Anniepoo | ah, it's just the windows menu is somehow messed up |
| 02:36 | ambient | C-x b shows all buffers |
| 02:37 | LauJensen | C-x f => find / create files |
| 02:37 | ambient | idk what logic the buffers menu uses, but it often seems to be missing some entries |
| 02:37 | jdz | * C-x C-f |
| 02:37 | Anniepoo | C-x C-f threw me into some weird non GUI file browser in a buffer at one point |
| 02:37 | Anniepoo | and the actual windows file->new menu is borked |
| 02:37 | jdz | that's what you want most of the time |
| 02:38 | jdz | who'd want to browse through tons of folders by pointy-clicking when you can easily find what you want with tab-completion? |
| 02:38 | Anniepoo | yah, but I couldn't figure out how to make a new file in that |
| 02:39 | LauJensen | jdz: and besides, ido-find-file, you just type boostrap.clj, and it'll find the file for you, no matter where its hiding :) |
| 02:39 | ambient | bootstrap.clj? |
| 02:39 | jdz | Anniepoo: you just type the path, and the file will be created when you save it. |
| 02:40 | jdz | Anniepoo: type the path in find-file query |
| 02:40 | LauJensen | Just at example. Ido keeps a history of your files, so you never have to type the fullpath, just the name or part of the name |
| 02:40 | Anniepoo | yah, just their attempt to add 'real' windows menus is borked |
| 02:40 | ambient | oh ok |
| 02:40 | Anniepoo | ah, lovely, |
| 02:40 | Anniepoo | swimming in info, but there's something called ido? |
| 02:41 | Anniepoo | cause file browsing is pretty absurd |
| 02:41 | ambient | M-x ido- <tab> |
| 02:41 | ambient | M-x is pretty sweet with tab completion |
| 02:43 | LauJensen | Anniepoo: I believe that if you (require 'git-emacs) in your .emacs, it will automatically give you git integration AND ido. So that C-x f from now on will complete paths for you. just type "core" for instance, and way a sec, it would find "/foo/bar/core.clj" if you've previously accessed it |
| 02:43 | jdz | again, it's C-x C-f |
| 02:44 | Anniepoo | I seem to have ido |
| 02:44 | Anniepoo | M-x ido- tab gives me a buffer with a bunch of commands |
| 02:44 | Anniepoo | so what's this thing do? |
| 02:44 | LauJensen | You have it, but its not bound and loaded |
| 02:44 | LauJensen | I gotta jet, be back l8r |
| 02:44 | Anniepoo | thanks for the help |
| 02:45 | jdz | Anniepoo: most probably you'll want to bind C-x C-f from plain find-file to ido-find-file in your .emacs |
| 02:45 | Anniepoo | foo, lost paredit |
| 02:47 | Anniepoo | and am getting smart indent 8cP |
| 02:48 | Anniepoo | well, thanks all |
| 02:49 | Anniepoo | have beaten myself enuf 4 2nite |
| 03:34 | cark | http://java.ociweb.com/mark/stm/article.html |
| 04:01 | ambient | how do i trigger time-based events in clojure? |
| 04:01 | ambient | i have a stack that has different time stamps sorted in chronological order, pop takes the first one |
| 04:02 | ambient | and according to time, i have to trigger those events but i dont want to write a busy loop |
| 04:02 | tomoj | what other possibility could there be? |
| 04:02 | ambient | eg. callbacks |
| 04:03 | ambient | because the next time is always known when the event triggers |
| 04:03 | tomoj | but someone has to call a callback |
| 04:03 | ambient | busy loops are so ugly :( |
| 04:03 | ambient | oh well... |
| 04:04 | tomoj | isn't pretty much everything running on your computer a busy loop? |
| 04:04 | ambient | in theory there has to be only one busy loop |
| 04:04 | jdz | threads are used for this usually, |
| 04:05 | hiredman | ambient: there are all kinds of cool stuff in executors |
| 04:05 | tomoj | yeah, mabye I dunno what a "busy loop" is |
| 04:05 | ambient | while 1: do stuff until break |
| 04:06 | ambient | s/1/true |
| 04:06 | hiredman | http://gist.github.com/178350 |
| 04:06 | tomoj | oh, I thought you meant spawn a thread that checks if the task's time has come and then sleeps for a bit |
| 04:06 | ambient | hiredman nice, thanks |
| 04:06 | ambient | tomoj well i could do that also, then the loop would be in that thread |
| 04:07 | tomoj | yeah |
| 04:08 | hiredman | (sched/fixedrate {:task #(dump-dict-is config) :start-delay 1 :rate 10 :unit (:minu |
| 04:08 | hiredman | er |
| 04:09 | ambient | although if i know the time difference i can just call (Thread/sleep time-delta) |
| 04:09 | ambient | heureka |
| 04:09 | hiredman | (sched/fixedrate {:task #(dump-dict-is config) :start-delay 1 :rate 10 :unit (:minutes sched/unit)}) |
| 04:09 | ambient | the rate is not fixed |
| 04:09 | tomoj | ambient: do people in your locale say "heureka" or do you just know greek? |
| 04:10 | tomoj | everyone around here says just "eureka" |
| 04:10 | ambient | it just felt right |
| 04:10 | ambient | no, people in my locale dont speak greek |
| 04:11 | tomoj | I thought maybe in other places "heureka" is just the word they use |
| 04:11 | tomoj | which would be awesome |
| 04:11 | ambient | well i think it is so |
| 04:12 | ambient | http://fi.wikipedia.org/wiki/Heureka |
| 04:18 | eevar2 | ambient: java.util.Timer & java.util.TimerTask? |
| 04:18 | hiredman | :( |
| 04:18 | Fossi | german, finish and polish seem to be those that keep the h |
| 04:19 | hiredman | the scheduling stuff from the executors framework is much better than Timer |
| 04:19 | tomoj | I'm guessing in english we lost all the h's that didn't come through latin |
| 04:20 | eevar2 | though the stuff in java.util.concurrent is probably better, yes |
| 04:44 | triyo | I see the apply and reduce functions can perform the same action cases I have tried. """(apply + [1 2 3])) -> 6 AND (reduce + [1 2 3])) -> 6""" What is the major difference between the two? |
| 04:47 | tomoj | (apply + [1 2 3]) means (+ 1 2 3) |
| 04:47 | tomoj | (reduce + [1 2 3]) is like (+ (+ 1 2) 3) |
| 04:48 | tomoj | they happen to be the same for + but won't be the same for most functions |
| 04:48 | triyo | tomoj: thanks. I see though that apply wouldn't always return the same result as reduce of course...make sense now reading the doc string |
| 04:49 | tomoj | in most of the places I use reduce, apply would cause an error |
| 04:49 | tomoj | (most functions don't take an arbitrary number of arguments) |
| 04:50 | jdz | arbitrary number, which can also be infinity :) |
| 04:50 | tomoj | I think reduce is faster with + |
| 04:52 | tomoj | and it works better with lazy seqs, I believe |
| 04:52 | jdz | better how? |
| 04:52 | tomoj | (apply + (range 10000000)) blows my heap, but (reduce + (range 10000000)) works fine |
| 04:52 | tomoj | reduce doesn't need the whole seq at once, apply does |
| 04:53 | jdz | i remember apply not needing the whole sequence at once... |
| 04:53 | jdz | might be some specific cases |
| 04:53 | jdz | oh, right |
| 04:53 | tomoj | how could it not? it has to pass the whole seq as arguments to a function |
| 04:53 | jdz | in case a function has a &rest parameter |
| 04:53 | tomoj | ah, yeah |
| 05:04 | tomoj | yup, true |
| 05:04 | tomoj | that's pretty neat |
| 05:06 | tomoj | (apply + ..) only seems to be slower than (reduce + ..) if you're not holding the head |
| 05:12 | tomoj | but in that case I'm seeing a 35% speedup |
| 05:16 | LauJensen | Any nginx experts here? |
| 05:17 | LauJensen | nginxperts? :) |
| 05:17 | tomoj | I've used it.. wouldn't say I'm an expert |
| 05:18 | LauJensen | I'm using it to proxy all requests to port 80 to my jetty server on another port. The problem is my logging in Jetty shows all users to come from 127.0.0.1 - how do I get around this? |
| 05:19 | tomoj | you've just got "proxy_pass ..."? |
| 05:19 | eevar2 | LauJensen: http://forum.nginx.org/read.php?2,4805,4809 |
| 05:20 | tomoj | indeed |
| 05:20 | LauJensen | tomoj: yea |
| 05:20 | eevar2 | the proxy_set_header bits in particular |
| 05:20 | tomoj | I think the block here should be enough http://wiki.nginx.org/NginxHttpProxyModule |
| 05:20 | LauJensen | oh yea, great if that works |
| 05:20 | LauJensen | fantastic |
| 05:21 | LauJensen | Clojure community never ceases to amaze me with its versatility :) |
| 05:21 | tomoj | I still don't really understand deploying jetty stuff :( |
| 05:23 | LauJensen | What dont you understand? |
| 05:25 | tomoj | in the ruby world there's no threading really |
| 05:25 | tomoj | so you have nginx proxy to a cluster of processes |
| 05:25 | tomoj | but I'm thinking maybe you only need one jetty instance on each box? |
| 05:26 | tomoj | never did any java web stuff so servlets and all are new to me |
| 05:26 | eevar2 | one jetty instance should do, yes |
| 05:31 | LauJensen | Jetty can listen of a given port using run-server. It takes an argument of a servlet and each servlet handles various routes, ie "/" "/foo" "/favicon.ico" and so on. I recommend loading your namespaces from a central boot.clj, which then opens your sites/servlets on various ports. |
| 05:31 | LauJensen | (I'm speaking of Compojures approach) |
| 05:32 | eevar2 | anyone had a look at nginx built-in caching support? |
| 05:32 | LauJensen | no - but I've just tested, and that snippit puts another field in the header, so now I'm logging all IPs :) |
| 05:33 | LauJensen | mysql> select remote from traffic; |
| 05:33 | LauJensen | +----------------+ |
| 05:33 | LauJensen | | remote | |
| 05:33 | LauJensen | +----------------+ |
| 05:33 | LauJensen | | 90.185.128.253 | |
| 05:33 | LauJensen | | 90.185.128.253 | |
| 05:33 | LauJensen | ... |
| 05:33 | ambient | wouldnt it be a good idea to build a sql frontend for mysql too in clojure? |
| 05:33 | eevar2 | which headers did you have to set? x-real-ip, x-forwarded-for, or both? |
| 05:34 | eevar2 | and host? |
| 05:35 | LauJensen | ambient: You mean like ClojureQL ? |
| 05:35 | ambient | oh man, this concurrency thing is giving me a headache |
| 05:35 | LauJensen | eevar, just x-real-ip |
| 05:35 | eevar2 | oki |
| 05:35 | LauJensen | ~clojureql |
| 05:35 | clojurebot | clojureql is http://gitorious.org/clojureql |
| 05:35 | ambient | LauJensen oh, you've already done so :) |
| 05:36 | LauJensen | Yes sir, its also what I use for logging traffic on my site, like http://groups.google.com/group/compojure/browse_thread/thread/2f1729608be2b072 |
| 05:36 | LauJensen | I was looking for some feedback on it. Am I logging the right stuff? To much, to little? |
| 05:37 | ambient | set different log levels and you can switch on/off when you feel like it *shrug* |
| 05:38 | LauJensen | I already have that, but its rolled in a debug macro, which goes quote verbose when its on |
| 05:39 | LauJensen | something like |
| 05:39 | LauJensen | (debug "launching thread" "launched" (send-off 0 (myfunc))) |
| 05:39 | LauJensen | TIMESTAMP: Launching thread |
| 05:39 | LauJensen | TIMESTAMP: launched |
| 05:39 | LauJensen | TIMESTAMP: Theads own debugging... |
| 05:42 | tomoj | what's (send-off 0 ...) ? |
| 05:42 | eevar2 | LauJensen: I think most existing logging libraries do buffered, asynchronous, logging, btw |
| 05:43 | eevar2 | committing every entry to the db might add some overhead |
| 05:43 | LauJensen | eevar, its a good idea to do it async, each transaction runs at ~10 msecs. I'm mostly interested in the data I'm trapping right now. Is it enough for doing statistics later on, or should I get more ? |
| 05:45 | LauJensen | Hmm, the buffer is a good idea. I should cluster about 500 rows together before committing, having an agent which checks regularily. |
| 05:45 | LauJensen | Calling mysql 10 times per visitor can quickly flood the cpu |
| 05:46 | eevar2 | i'd just leave it to the web server tbh. ;) |
| 05:46 | LauJensen | nginx supports dropping logs in mysql ? |
| 05:46 | eevar2 | you could always import them later |
| 05:47 | eevar2 | if you ever add caching, some requests won't actually hit the app server either |
| 06:01 | tomoj | has anyone tried decreasing the wait times in ants.clj? seems to screw things up |
| 06:02 | LauJensen | Way back when, yea I think so. Dont remember anything breaking though |
| 06:04 | tomoj | I guess 10ms for the ant sleep time just leaves no time to do the animation |
| 06:04 | tomoj | so the animation locks up |
| 06:22 | juhal | apparently clojure compiles and loads a new class every time I call eval |
| 06:22 | juhal | sigh |
| 06:23 | jdz | why would you call eval? |
| 06:23 | juhal | because I need to evaluate clojure expressions |
| 06:23 | jdz | are you sure? |
| 06:24 | jdz | i mean, are they user-supplied and you have no control over them whatsoever? |
| 06:24 | juhal | yes |
| 06:25 | jdz | and what's bad about the compilation, then? |
| 06:25 | juhal | well the expressions don't change so recompiling them every time doesn't make sense |
| 06:26 | jdz | then don't |
| 06:26 | jdz | if the expressions don't change, why are you calling eval on them more than once? |
| 06:27 | juhal | because I need their side-effects |
| 06:30 | jdz | if nobody more knowledgeable than me offers something else, you can wrap your expressions so that eval returns function, which you can then call when you want |
| 06:31 | juhal | hmm that sounds like it could work, thanks |
| 06:36 | jdz | but anyway blingly executing random code is not a very good thing to do... |
| 06:37 | Chousuke | The mention of both side-effects AND eval makes me highly suspicious, but I have no idea what you're trying to do :/ |
| 06:50 | tomoj | hrmm.. clojure-http-client currently gives you the headers with keys being header names |
| 06:50 | tomoj | but header names are supposed to be case-insensitive |
| 06:51 | tomoj | I can't think of a good way to fix this |
| 06:51 | tomoj | you could provide a function for looking up headers in the header map in a case-insensitive way, but this isn't as pretty as just using the headers as a map |
| 06:51 | tomoj | would be nice if the headers map had symbols as keys with the header names downcased, but that's not backwards compatible |
| 06:52 | tomoj | in ruby I would just monkeypatch the header map to downcase keys before looking them up :( |
| 06:57 | tomoj | s/symbols/keywords/ |
| 07:18 | AWizzArd | Is it possible to have a regex which splits a string at the empty string following the regex [,;]? So, like splitting on , or ; but without consuming those chars? |
| 07:20 | jdz | regular expressions will create more trouble than they solve |
| 07:39 | cgrand | AWizzArd: #"(?=[,;])" |
| 08:28 | AWizzArd | cgrand: works, thanks |
| 08:38 | LauJensen | ,(sort > (vals {:foo 5 :bar 2 :baz 10})) |
| 08:38 | clojurebot | (10 5 2) |
| 08:38 | LauJensen | Whats a pretty way to flip that back to the keys, so I get (:baz :foo :bar) ? |
| 08:41 | Chouser | ,(map key (sort-by val > {:foo 5 :bar 2 :baz 10})) |
| 08:41 | clojurebot | (:baz :foo :bar) |
| 08:42 | LauJensen | (sort-by val) thats a new one. Thanks alot |
| 08:45 | tomoj | never realized how useful it is that maps act as seqs of mapentries |
| 08:50 | LauJensen | Rich has a few good ideas like that :) |
| 08:51 | tomoj | goddamn't |
| 08:52 | tomoj | my university was like "oh hey looks like tom finished writing a scraper for the course schedule, let's change all the markup around!" |
| 08:53 | LauJensen | Yea being an IT admin has its perks :) |
| 08:54 | jdz | tomoj: you should instead wrote a program that writes course schedule scrapers :) |
| 08:55 | tomoj | jdz: that's be a good idea, but the data's not stable so I don't see how I could do it easily |
| 08:56 | jdz | what do you mean data is not stable? |
| 08:56 | jdz | and it's not about being easy to do, but doing it once :) |
| 08:56 | Fossi | *couhg* xls *couhg* ? ;) |
| 08:56 | tomoj | but I mean, every time the markup changes, I need to have the program write a new scraper |
| 08:57 | tomoj | but by that time the expected data is all different |
| 08:57 | jdz | oh boy oh boy |
| 08:57 | Fossi | wow. not that i can't type, but i can't type twice as much today :) |
| 08:57 | jdz | have you ever heard of machine learning thingie? |
| 08:57 | tomoj | yeah but I don't have target data to train on |
| 08:57 | jdz | then generate it |
| 08:58 | tomoj | I mean I could go visit the site and manually scrape the data into some reasonable format |
| 08:58 | tomoj | I guess doing that is easier than rewriting the scraper every time the markup changes |
| 08:58 | jdz | not to mention all the things you learn along the way |
| 08:59 | tomoj | something like scrubyt, you say what the data should be and it figures out how to scrape it |
| 08:59 | jdz | oh, look, somebody has written the program for you! :) |
| 08:59 | tomoj | (or tries to) |
| 09:00 | tomoj | that one doesn't work for me though :/ |
| 09:00 | tomoj | first of all it's in ruby, not clojure |
| 09:01 | tomoj | and second there are some special relationships my scraper-writing program would need to understand which scrubyt doesn't |
| 09:01 | LauJensen | tomoj: Are you using enlive ? |
| 09:01 | tomoj | LauJensen: yep |
| 09:01 | LauJensen | k, isnt it a simple change to adapt it ? |
| 09:01 | tomoj | this time it is, I think |
| 09:02 | tomoj | but in general it won't be |
| 09:02 | tomoj | the markup on the site I'm scraping is really bad |
| 09:02 | LauJensen | oh |
| 09:03 | tomoj | so like in one table cell they have "MW<br/>F" and later in the row something like "2pm to 3pm<br/>10am to 11am" |
| 09:03 | tomoj | and that means the class starts at 2pm on monday and wednesday but 10am on friday |
| 09:04 | tomoj | stuff that enlive can't do on its own, so I have to have logic after pulling stuff out with enlive, and that logic could be painful to change |
| 09:04 | tomoj | they're government workers though so since they just made a change it'll probably be a long time until the next change :) |
| 09:06 | LauJensen | tomoj: File a feature request, I'm sure cgrand has a one-liner which both scrapes the website and gets the goverment workers fired. |
| 09:06 | tomoj | haha |
| 09:07 | tomoj | I actually work here too but not in the department that does this stuff.. wish I could convince them to just put it up as XML or JSON |
| 09:07 | tomoj | actually they must have some internal machine-readable representation because counselors get the data in a different interface.. hmm |
| 09:09 | LauJensen | hehe, now you're thinking |
| 09:11 | Chouser | in my experience, constantly fixing a fragile web scraper is often more reliable and less work than getting access to the machine-readable format internal to the organization. |
| 09:14 | tomoj | yeah, I don't think they'd let me at that anyway |
| 09:14 | tomoj | I'm not sure they'd even let me scrape it if they knew I was doing it |
| 09:23 | rabidsnail | Is there a clojure equivalent to Python's locals() function? |
| 09:24 | jdz | what does it do? |
| 09:24 | rabidsnail | It gives you a map of all of all of the variables in the current context. |
| 09:25 | jdz | current context? |
| 09:25 | rabidsnail | current scope |
| 09:26 | jdz | i'm pretty sure there is no such function |
| 09:26 | jdz | because you can see all them variables by looking at the source code |
| 09:26 | jdz | the don't magically appear while the program is running |
| 09:26 | Chouser | not really -- you might be able to use the JVM debugging api to get at some of that, but I'm not sure. |
| 09:26 | rabidsnail | bother |
| 09:27 | jdz | *they |
| 09:28 | rabidsnail | Django uses it to stick all of the local varaibles in error pages when debug mode is on, for example |
| 09:28 | rabidsnail | It's very useful |
| 09:28 | jdz | then what Chouser said |
| 09:41 | gko | Is there an operator to return from a function, like Common Lisp (return ...) or (return-from ...) ? |
| 09:42 | ole3 | no |
| 09:42 | ole3 | gko: no |
| 09:46 | triyo | I am trying to understand the difference between defining metadata for an object using #^{} and with-meta. If I use the former, (println (meta myfunc)) prints nil. However if I define the metada using with-meta then it print the metada of the object. |
| 09:46 | Chousuke | triyo: #^{:foo 'bar} assigns metadata to the *symbol* at read-time. |
| 09:47 | Chousuke | triyo: with-meta assigns it to whatever object you pass to it. |
| 09:48 | Chouser | just to keep things confusing, the compiler or various macros often copy metadata from a symbol to some useful runtime object. |
| 09:48 | Chouser | did I say confusing? I meant convenient. |
| 09:48 | Chouser | ,(meta #^{:foo 'bar} []) |
| 09:48 | clojurebot | {:foo bar} |
| 09:49 | Chouser | ,(let [my-vec []] (meta #^{:foo 'bar} my-vec)) |
| 09:49 | clojurebot | nil |
| 09:49 | Chousuke | ,(meta #^{:bar 'foo} foo) ; fail |
| 09:49 | clojurebot | java.lang.Exception: Unable to resolve symbol: foo in this context |
| 09:49 | Chousuke | ,(meta '#^{:bar 'foo} foo) ; no fail |
| 09:49 | clojurebot | {:bar (quote foo)} |
| 09:49 | Chousuke | note the location of the quote |
| 09:50 | Chousuke | ,(meta '#^{:bar foo} foo) |
| 09:50 | clojurebot | {:bar foo} |
| 09:50 | triyo | is the attr-map #^{} meant to do the same as with-meta? |
| 09:51 | Chousuke | no. |
| 09:51 | Chousuke | as said, with-meta happens at runtime. #^{} is read-time |
| 09:52 | Chousuke | and as such, #^{} is only useful for whatever the compiler can read. |
| 09:52 | Chousuke | eg. vectors, lists, symbols, maps, etc. |
| 09:52 | Chousuke | er, s/compiler/reader/ |
| 09:58 | triyo | Chousuke: oh I see. you need to quote the attr-map |
| 10:07 | jdz | hmm, that '#^{metadata here} symbol-name looks very very confusing... |
| 10:07 | jdz | had to think about it like for 5 minutes until i understood what's going on... |
| 10:09 | Chousuke | triyo: no, it's actually quoting the symbol |
| 10:09 | Chousuke | triyo: the attr-map disappears after read-time |
| 10:09 | Chousuke | ,(macroexpand '#^{:foo bar} test) |
| 10:09 | clojurebot | test |
| 10:09 | triyo | Chousuke: got it now, I perfectly understand it. thx |
| 10:10 | jdz | ye, there usually is no space between quote and the symbol :/ |
| 10:10 | Chousuke | yeah, it's a bit weird. |
| 10:10 | Chousuke | You need it with macros too, if you want to produce type-hinted code. |
| 10:10 | jdz | haven't had a need for those yet |
| 10:10 | Chousuke | at least, sometimes |
| 10:11 | jdz | but now i'll be better equipped |
| 10:11 | Chousuke | I wonder if I'm done toying with my chroots now. |
| 10:11 | Chousuke | I unstalled Debian stable AND unstable on my serverbox :P |
| 10:11 | jdz | maybe writing '#^{meta data}whatever may help some. maybe not. |
| 10:12 | ankou | hi, anybody using slime here? If it try to use M-.(slime-edit-definition) on a function I get a NullPointerException |
| 10:12 | jdz | unstalling is like installing and uninstalling, all at once? :) |
| 10:12 | Chousuke | heh |
| 10:13 | Chousuke | I needed some newer software from unstable but didn't feel like keeping them up-to-date myself. |
| 10:13 | ankou | *but only with my own functions |
| 10:13 | Chousuke | so, I installed schroot and debootstrap'd myself a sid environment :P |
| 10:13 | Chousuke | and now I have a sid-shell alias that takes me to it |
| 10:38 | AWizzArd | Is there a simple way to do comp, but give the functions in reverse order, to match -> and the thinking? So, maybe a function pipe? |
| 10:38 | stuartsierra | Somebody wrote such a thing and posted it on the list, I think. |
| 10:41 | stuartsierra | Maybe this is it: http://www.mail-archive.com/clojure@googlegroups.com/msg08098.html |
| 10:42 | stuartsierra | Although (apply comp (reverse fns)) ought to work as well. |
| 10:42 | gko | ole3: OK |
| 10:46 | stuartsierra | Or just use -> in a #() |
| 10:52 | AWizzArd | Maybe I am one of the few who uses this regularily |
| 10:56 | AWizzArd | I have my own pipe function (not macro), which does essantially forward the evaluation to comp. But I find this could go into core. Users of comp will think first what should be done first, navigate with arrow keys back, then write down what comes next, use arrow keys again, and so on. |
| 10:56 | AWizzArd | Pretty much like the -> |
| 10:59 | Chouser | Rich has talked about adding a -> that puts the args on the right end instead of the left. |
| 10:59 | Chouser | I think the hold up so far has been the name |
| 11:00 | stuartsierra | I find I use -> and comp less than I expect. |
| 11:01 | stuartsierra | I started working on a Clojure package manager this weekend. |
| 11:03 | stuartsierra | But then I wondered - how many Clojure "packages" are ready for a formal "release"? |
| 11:05 | AWizzArd | Chouser: is that a new operator which turns around the args or will the existing -> require the reverse order? |
| 11:06 | AWizzArd | It's just that comp doesn't match the way of thinking about it. |
| 11:29 | AWizzArd | Chouser: is it possible to get the number of matches of a regexp and use it for substitution? For example, I want to replace all digits between 1-9 with a 9, but only for at least 2 subsequent digits. |
| 11:30 | AWizzArd | When I do (.replaceAll s "[1-9]([1-9]+)" "9$1") it does not replace each digit |
| 11:30 | AWizzArd | ,(.replaceAll "Hello 1 and 2340" "[1-9]([1-9]+)" "9$1") |
| 11:30 | clojurebot | "Hello 1 and 9340" |
| 11:30 | stuartsierra | You could try c.c.str-utils2/replace with a function as the replacement. |
| 11:31 | AWizzArd | ,(.replaceAll "Hello 1 and 2340" "[1-9]([1-9])+" "9$1") |
| 11:31 | clojurebot | "Hello 1 and 940" |
| 11:38 | Chouser | AWizzArd: -> would not change |
| 11:40 | AWizzArd | Chouser: good |
| 11:40 | AWizzArd | ,(.replaceAll "Hello 1 and 2340" "[1-9](?=[1-9])+" "9$1") |
| 11:40 | clojurebot | java.lang.IndexOutOfBoundsException: No group 1 |
| 11:40 | AWizzArd | ,(.replaceAll "Hello 1 and 2340" "[1-9](?=[1-9])+" "9") |
| 11:40 | clojurebot | "Hello 1 and 9940" |
| 11:40 | AWizzArd | ,(.replaceAll "Hello 1 and 2340" "[1-9](?=[1-9])" "9") |
| 11:40 | clojurebot | "Hello 1 and 9940" |
| 11:41 | Chouser | is that the output you want? |
| 11:43 | AWizzArd | no, I hoped for "Hello 1 and 9999" |
| 11:44 | AWizzArd | no, I hoped for "Hello 1 and 9990" |
| 11:44 | AWizzArd | all digits between 1-9 shuold be replaced by a 9, if there are least 2 subsequent digits |
| 11:45 | AWizzArd | but zeros inbetween may be a problem |
| 11:47 | Chouser | ,(.replaceAll "Hello 1 and 2340" "[1-9](?=[0-9][0-9])" "9") |
| 11:47 | clojurebot | "Hello 1 and 9940" |
| 11:48 | tomoj | I don't get it.. there aren't two subsequent digits after the 4 in 2340 |
| 11:48 | Chouser | ,(.replaceAll "Hello 1 and 2340" "[1-9](?=[0-9])" "9") |
| 11:48 | clojurebot | "Hello 1 and 9990" |
| 11:49 | LauJensen | Gents, Im doing a webstat module and for this I need some webservice that I can post IPs to and in return I want information on names, adresses etc. Anyone of you know of such a service available freely? |
| 11:49 | AWizzArd | ,(.replaceAll "Hello 15 and 4 and 0123/1234567890" "[1-9](?=[0-9])" "9") |
| 11:49 | clojurebot | "Hello 95 and 4 and 0993/9999999990" |
| 11:49 | AWizzArd | ,(.replaceAll "Hello 15 and 4 and 0123/1234567890" "[1-9](?=[0-9][0-9])" "9") |
| 11:49 | clojurebot | "Hello 15 and 4 and 0923/9999999990" |
| 11:50 | AWizzArd | there it doesn't catch the 5 of the 15 |
| 11:50 | tomoj | why would it? you're looking for two subsequent digits |
| 11:50 | AWizzArd | yes |
| 11:50 | tomoj | the 5 has none and the 1 has one |
| 11:51 | Chousuke | you want to ignore non-digits for the look-ahead? |
| 11:51 | AWizzArd | "Hello 15 and 4 and 0123/1234567890" ==> "Hello 99 and 4 and 0999/9999999990" |
| 11:52 | tomoj | so.. one subsequent digit? |
| 11:52 | AWizzArd | as soon there are two ore more subsequent digits replace in that sequence all 1-9 with a 9 |
| 11:52 | tomoj | ok, two consecutive digits, I see |
| 11:52 | AWizzArd | oh ok, consecutive is the right word, thanks |
| 11:53 | tomoj | subsequent probably works too I just thought you meant something else |
| 11:53 | AWizzArd | but maybe this should really be a function |
| 11:54 | Chousuke | can't you do [1-9]{2,} or something? |
| 11:54 | AWizzArd | i think it would then replace 12345 with 9 |
| 11:54 | tomoj | problem is you don't know how many 9's to replace it with |
| 11:54 | AWizzArd | and not with 99999 |
| 11:54 | AWizzArd | tomoj: right |
| 11:55 | AWizzArd | if there is just a single digit, not followed by a digit it can stay unchanged |
| 11:55 | AWizzArd | like the 4 |
| 11:55 | tomoj | you could search for a substring of two or more digits, get that match, then just regular .replace that substring with a string of 9's of equal length, and repeat until there are no more substrings of two or more digits |
| 11:57 | kwatz | ,(.replaceAll "Hello 15 and 4 and 0123/1234567890" "(?<=[0-9])[1-9]|[1-9](?=[0-9])" "9") |
| 11:57 | clojurebot | "Hello 99 and 4 and 0999/9999999990" |
| 11:58 | kwatz | though using lookahead or lookbehind usually feels wrong to me |
| 12:01 | stuartsierra | c.c.str-utils2/replace! |
| 12:03 | AWizzArd | kwatz: oh good, that works! stuartsierra: I am already looking at its source. |
| 12:03 | tomoj | oh, my idea obviously doesn't work |
| 12:03 | AWizzArd | tomoj: why not? |
| 12:03 | tomoj | because 99 is also a substring of two consecutive digits |
| 12:03 | tomoj | so it just loops forever |
| 12:04 | AWizzArd | ,(.replaceAll "Hello 15 and 4 and 0123/1234567890" "(?<=\\d)[1-9]|[1-9](?=\\d)" "9") |
| 12:04 | clojurebot | "Hello 99 and 4 and 0999/9999999990" |
| 12:04 | AWizzArd | kwatz: why it feels wrong to you? |
| 12:05 | tomoj | to get my idea to work you'd have to replace the digits with nondigits and build up a list of indices to replace with 9's afterwards.. very ugly |
| 12:06 | kwatz | sometimes they can be hard to reason out, and it's logic that may deserve to be in a function |
| 12:06 | kwatz | mostly a personal preference, i supose |
| 12:08 | kwatz | suppose* |
| 12:17 | lisppaste8 | cgrand pasted "transients VS tail-call position" at http://paste.lisp.org/display/86304 |
| 12:18 | milep | Hello, when extending abstract java class with proxy, can I override the constructor? |
| 12:19 | stuartsierra | milep: no |
| 12:19 | milep | I'm trying to implement pircbot example from http://www.jibble.org/pircbot.php |
| 12:20 | milep | stuartsierra: ok, any tips how to do MyBot.java from that page ^ |
| 12:20 | milep | setName is protected |
| 12:20 | stuartsierra | milep: You can pass arguments to the superclass constructor. |
| 12:21 | stuartsierra | Then you'll have to use gen-class to expose the protected methods, I think. |
| 12:22 | cgrand | rhickey: OoM is an unpleasant side-effect of adding transients to a working fn (see http://paste.lisp.org/display/86304 ) |
| 12:22 | milep | stuartsierra: ok, thanks for the tips, have to study more about that... |
| 12:25 | Chouser | reify can call protected methods, right? |
| 12:25 | stuartsierra | What's reify? |
| 12:26 | cgrand | new new new name |
| 12:26 | stuartsierra | Ha! |
| 12:47 | pluijzer | bin searching a while now, but is there a alternative for lisp's "position" in clojure? |
| 12:52 | stuartsierra | c.c.seq-utils has "positions" which is similar |
| 12:52 | pluijzer | thanks |
| 12:56 | LauJensen | Gents, Im doing a webstat module and for this I need some webservice that I can post IPs to and in return I want information on names, adresses etc. Anyone of you know of such a service available freely? |
| 12:57 | stuartsierra | whois? |
| 12:57 | LauJensen | Something like that |
| 12:58 | stuartsierra | Why not just use whois? |
| 13:00 | LauJensen | Hmm :) |
| 13:00 | LauJensen | I think I will, thanks |
| 13:32 | LauJensen | whois is slow, changing to buffered multithreaded webstatic strategy |
| 13:32 | stuartsierra | true, that |
| 13:35 | LauJensen | Its just the incentive I needed - Flushing webstats straight into sql with every call to the site was a bit too much |
| 13:45 | alinp | ,(reduce + (range 0 9999999)) |
| 13:45 | clojurebot | 49999985000001 |
| 13:45 | alinp | ,(time (reduce + (range 0 9999999))) |
| 13:45 | clojurebot | 49999985000001 |
| 13:45 | clojurebot | "Elapsed time: 3382.242 msecs" |
| 13:46 | alinp | ,*clojure-version* |
| 13:46 | clojurebot | {:interim true, :major 1, :minor 1, :incremental 0, :qualifier "alpha"} |
| 13:49 | alinp | ,(System/getProperty "java.vm.version") |
| 13:49 | clojurebot | java.security.AccessControlException: access denied (java.util.PropertyPermission java.vm.version read) |
| 14:01 | LauJensen | bestinclass.webstats> (country? "213.232.195.196") |
| 14:01 | LauJensen | "RU" |
| 14:01 | LauJensen | Should I be worried? :) |
| 14:19 | rhickey | Chouser: not sure yet about reify and protected - what did you want to call? |
| 14:19 | hiredman | http://groups.google.com/group/clojure/browse_thread/thread/7538a136a39cce48?hl=en <-- |
| 14:19 | hiredman | er |
| 14:19 | hiredman | "Qi in Clojure (Shen) project underway" |
| 14:20 | hiredman | doesn't actually look all that underway |
| 14:23 | stuartsierra | Most Clojure projects are not all that underway. (this morning's blog post http://bit.ly/1Va0o0 ) |
| 14:25 | LauJensen | rhickey: Are you still considering names for new new ? |
| 14:26 | rhickey | LauJensen: have you got one? |
| 14:27 | LauJensen | rhickey: Nope - But if its still open, we could have a rumble right here. You could say in as few words as possible what it does or what the name should express and we could all have a go :) |
| 14:28 | rhickey | The name should be able to express everything that 'refiy' does and somehow be better than reify |
| 14:28 | rhickey | reify |
| 14:28 | LauJensen | hehe, alright |
| 14:29 | LauJensen | And just using 'extend' would be totally wrong? |
| 14:29 | rhickey | Java distinguishes extend and implement, this would normally be used for the latter |
| 14:30 | LauJensen | Alright, it might be my english thats failing me, but leaving Java behind new new still has the feeling of extending a class to me |
| 14:31 | rhickey | except it isn't a class but an interface, normally |
| 14:31 | rhickey | extend means add something to, implement means provide a definition for |
| 14:34 | ataggart | does anyone know of some clojure-to-JMS code? |
| 14:36 | rhickey | ataggart: using thre JMS API directly is pretty easy - what are you looking for? |
| 14:37 | LauJensen | rhickey: alright, I'll think and read my dictionary |
| 14:37 | LauJensen | In the mean time, why does the doc for send say "dispatch and agent", and send-off "dispatch a potentially blocking agent" ? |
| 14:37 | LauJensen | "dispatch and agent" = "dispatch AN agent" |
| 14:40 | ataggart | rhickey: nothing fancy, I just need to write a jms topic subscriber, figured I'd do it in clojure, and just wanted to see if there was some lib that would make it more idiomatic. |
| 14:41 | LauJensen | I mean - Arent both qualified for use with a blocking action? |
| 14:41 | rhickey | ataggart: wrapping queues is on my todo list, so I'm interested in use cases |
| 14:41 | rhickey | LauJensen: no |
| 14:42 | ataggart | rhickey: I'll write bang this out, and send you anything I find useful. |
| 14:42 | rhickey | the send pool is fixed size, you could exhaust it with blocking actions |
| 14:42 | rhickey | ataggart: what queue are you using? |
| 14:42 | ataggart | I believe we have an activemq instance running |
| 14:43 | rhickey | I want to check out HornetQ when I have some time |
| 14:43 | rhickey | I'm particularly interested in a model that scales from an in-process, direct embedded instance to full client-server |
| 14:43 | ataggart | I haven't done any work with jms before, so I'm not clear on how much of a "drop in" different implementations are |
| 14:44 | duncanm | anyone familiar with StAX/XMLEventWriter? |
| 14:44 | duncanm | is there a way to force <foo></foo> be written as <foo /> |
| 14:44 | duncanm | ? |
| 14:47 | LauJensen | Concurrent gentlemen, I have a website which generates an sql statement everytime somebody navigates the site. Every 5 minutes I'll commit all of these statements to the sql-server. Is this a good way to ensure not flushing an uncomitted entry from my buffer ref: (let [sql-statements @*sql-buffer*] (dosync (ref-set! *sql-buffer* []))... ? |
| 14:50 | hiredman | well, for starters ref-set doesn't have an exclamation mark |
| 14:51 | LauJensen | alright, scrap that then :) its just the approach |
| 14:52 | hiredman | hmmm |
| 14:53 | Chouser | rhickey: I was wonder aloud about protected on behalf of someone trying to use http://www.jibble.org/javadocs/pircbot/ |
| 14:53 | LauJensen | technically the question must be, can an entry slip in, between the let-definition and the dosync. Which I actually think it can |
| 14:53 | hiredman | Chouser: for nick setting? |
| 14:54 | Chouser | in general, java interop sometimes requires overriding protected methods, and gen-class is a bit painful. |
| 14:55 | Chouser | I'm fully on board with reify being a native clojure rather than host interop feature, as long as we get something as pleasant to use as proxy with greater capability for interop. |
| 14:56 | Chouser | I'd be ok with proxy growing :exposes and :exposes-methods features |
| 14:57 | hiredman | http://paste.lisp.org/display/83651 |
| 14:59 | stuartsierra | me too |
| 14:59 | cemerick | rhickey: I did a little spelunking and some thinking, and I'm curious as to what you have in mind w.r.t. isa? caching that doesn't run up against the same issues a WeakHashMap memoization of supers is intended to solve (e.g. class unloading). |
| 15:00 | Chouser | hiredman: that was the kind of thing going around when rhickey put in super-proxy |
| 15:00 | cemerick | I actually think the current caching in MultiFn is a bit of a time-bomb w.r.t. retaining otherwise-unloaded Classes. |
| 15:02 | hiredman | Chouser: I was previously just seting clojurebot's nick after it connected, but now I am using that to access the protected setName method |
| 15:02 | LauJensen | Nobodys got a take on how to flush a global ref ? |
| 15:02 | Chouser | hiredman: heh. nice. |
| 15:02 | rhickey | cemerick: I'm not sure class unloading is a solved problem in general |
| 15:02 | rhickey | but specifically to use of WeakHashMap as a cache, what are you doing for concurrency? |
| 15:03 | stuartsierra | LauJensen: I think you want to put your SQL code inside the dosync |
| 15:03 | stuartsierra | no, wait, that's wrong |
| 15:03 | hiredman | stuartsierra: :( |
| 15:03 | stuartsierra | sorry |
| 15:04 | cemerick | rhickey: nothing -- no entries should roll off the map, as long as the caller of isa?/supers is using that class (or an ancestor of it) as an argument |
| 15:05 | LauJensen | stuartsierra: ClojureQL prohibits sql transactions within dosyncs |
| 15:05 | hiredman | good |
| 15:05 | cemerick | Maybe that's too trusting of class GC and WeakReferences, but it's contractually correct, I think. |
| 15:05 | stuartsierra | Right, it should. you shouldn't have side effects inside dosync. |
| 15:05 | rhickey | cemerick: so multiple threads calling supers just access the map without synchronization? |
| 15:05 | LauJensen | stuartsierra: but I should probably let dosync return the let value, that'll solve it |
| 15:06 | stuartsierra | LauJensen: you want to do something like (dosync (let [old-value @my-ref] (ref-set my-ref nil) old-value) |
| 15:06 | LauJensen | exactly |
| 15:06 | stuartsierra | sorry for the confusion |
| 15:06 | hiredman | ~alter |
| 15:06 | clojurebot | alter is always correct |
| 15:07 | cemerick | rhickey: Yes. What's the downside, given that supers is working with entirely static classes? |
| 15:08 | rhickey | cemerick: my question has nothing to do with the caching behavior, has to do with proper use of a collection from multiple threads |
| 15:08 | cemerick | ah |
| 15:08 | cemerick | well, the results of super will be identical given the same argument, so worst case is the results get calculated a few more times than necessary |
| 15:09 | cemerick | and there's never any removes |
| 15:09 | rhickey | no, the worst case is the map will be inconsistent on a lookup and produce garbage |
| 15:09 | rhickey | you start caching modi, I start read, boom |
| 15:10 | rhickey | you'll need to wrap that in a lock which will make supers a serialization point |
| 15:10 | rhickey | "Like most collection classes, this class is not synchronized." |
| 15:11 | cemerick | haha -- been working with clojure too long :-) |
| 15:11 | cemerick | rhickey: yes, sorry, I'm using locking around accesses to the WHM |
| 15:11 | cemerick | I was thinking of transactions and such. :-| |
| 15:11 | hiredman | I suppose you could manually wrap everything in a WeakReference before sticking it in the map |
| 15:12 | rhickey | right, so, I've tried to avoid that kind of bottleneck in Clojure |
| 15:12 | hiredman | :| |
| 15:12 | rhickey | hiredman: then you couldn't look it up |
| 15:12 | hiredman | oh |
| 15:12 | Chouser | cemerick: a persistent map of weak references stored in an atom? |
| 15:12 | hiredman | right you are |
| 15:13 | rhickey | I think there will be a concurrent weak hashmap in Java7 |
| 15:13 | rhickey | ? |
| 15:13 | cemerick | Chouser: yeah, that'd probably be fine |
| 15:13 | rhickey | any map using weak refs as keys won't support lookup without something extra |
| 15:14 | rhickey | since weak refs have identity semantics |
| 15:16 | rhickey | maybe a persistent map of hashes to weak refs of class+cached-value thingies |
| 15:16 | Chouser | in an atom |
| 15:17 | rhickey | lookup on hash, then dig in to check class and find data |
| 15:17 | rhickey | yes, in an atom |
| 15:17 | Chouser | :-) |
| 15:17 | Chouser | could be a new joke template, like adding "in bed" to fortune cookie sayings. |
| 15:19 | rabidsnail | If you ask low-level jvm questions in #java they look at you funny. |
| 15:19 | rhickey | it does highlight how refs to persistent things are usually better than locks around mutable things |
| 15:23 | cemerick | well, that certainly sounds better than trying to cache isa? relationships in general |
| 15:23 | cemerick | there's too much dynamism in the hierarchies for me to be comfortable with the latter (or, with me implementing the latter) |
| 15:24 | rhickey | cemerick: sure, if you want to take a crack at that for supers I'll have a look |
| 15:24 | rhickey | but the same strategy could be used for isa? |
| 15:25 | rhickey | it's just a matter of remembering the hierarchy that was used as your basis, if not identical, toss cache |
| 15:25 | cemerick | Perhaps, but you'd have to get into invalidating relationships when a hierarchy is updated |
| 15:26 | rhickey | no, just toss the cache |
| 15:26 | cemerick | oh, the whole thing? |
| 15:26 | rhickey | yup |
| 15:27 | cemerick | jeez, in that case, we could just use a standard memoization |
| 15:27 | rathore_ | sivajag: he |
| 15:27 | rhickey | cemerick: not with the class unloading |
| 15:28 | cemerick | rhickey: well, MultiFn has the same problem with class unloading |
| 15:41 | rhickey | cemerick: You are the one with a problem with it :) I wonder how real a problem it is |
| 15:42 | cemerick | rhickey: No problem per se, I just see where it *could* be a problem. I actually didn't make the MultiFn dispatch table connection until after I figured using a WHM was most prudent. |
| 15:42 | rhickey | any value you use as a dispatch value can be captured by MultiFns, not just classes |
| 15:43 | cemerick | yeah -- I guess I'm just a little paranoid about being able to drop stuff into osgi environments, etc. |
| 15:43 | rhickey | but if you do this for isa? it could be a model for a better version of the multifn caches |
| 15:43 | cemerick | e.g. I wonder how many times people reload apps in their glassfish containers, etc. |
| 15:49 | ambient | is there a default directory structure i should use in a clojure project? |
| 15:50 | rhickey | cemerick: I agree there (containers) any global cache is tricky. I'd like to get away from a global set of namespaces even. |
| 15:50 | Chouser | ambient: I like: projname/src/com/mydomain/projname.clj for sources, projname/classes/ for compiled .class files, then maybe projname/README |
| 15:51 | ambient | projname/lib for libs? |
| 15:51 | Chouser | ambient: clojure itsel inserts another level under src to separate .java from .clj code, directories named jvm and clj respectively |
| 15:51 | cemerick | rhickey: whoa, sounds like environments :-P |
| 15:51 | rhickey | no |
| 15:51 | Chouser | ambient: for external libs? yeah, I've seen some projects drop jars they need in a projname/lib/ dir |
| 15:52 | cemerick | rhickey: I know, just rocking the boat ;-) |
| 15:52 | cemerick | I'll take a crack at a thread-safe WeakReference approach tomorrow-ish, though. |
| 15:52 | rhickey | cemerick: you want to pass an additional env arg to every fn? |
| 15:52 | rhickey | cemerick: cool |
| 15:52 | ambient | ok ty |
| 15:52 | cemerick | rhickey: no, certainly not |
| 15:53 | cemerick | I just remember us having roughly that conversation about 14-15 months ago. |
| 15:53 | Chouser | cemerick: I think that's what IRC is for. Having the same conversation as often as possible |
| 15:54 | ambient | Chouser I thought IRC was for typing random stuff on caffeine high |
| 15:54 | rhickey | :) |
| 15:55 | cemerick | Chouser: what was remarkable about that conversation was that it was via email! |
| 15:55 | cemerick | Like, who ever does email anymore? :-P |
| 15:57 | hiredman | clojurebot doesn't do email... |
| 15:58 | hiredman | (bugfixes welcome) |
| 16:10 | cemerick | does anyone else get a twitch under their eye when they read "patches welcome"? :-) |
| 16:11 | Chouser | cemerick: no! I feel a surge of adrenaline that nearly masks the vague sinking feeling of my time slipping away... |
| 16:11 | hiredman | well that explains things |
| 16:12 | Chouser | hehe |
| 16:12 | dave[csc] | Does anyone have experience with the Clojure bundle for TextMate? |
| 16:13 | cemerick | Chouser: ha! |
| 16:14 | cemerick | I happened to ask someone about Inkscape providing an OS X-native UI (rather than X11), and got "patches welcome", which was funny. |
| 16:19 | dave[csc] | No one uses TextMate to edit clojure eh? |
| 16:19 | Chousuke | I think emacs is the most popular editor. :) |
| 16:20 | dave[csc] | Humbug :P |
| 16:21 | dave[csc] | I love textmate, and the bundles, but I can never seem to get them to 'just work' |
| 16:21 | Chousuke | Hm, I hate jokes that can't be translated ;( |
| 16:22 | Chousuke | I always get an urge to tell them to people who can't understand the language. |
| 16:23 | Chousuke | But maybe I could make a joke out of translating them |
| 16:24 | Chousuke | and ending up with something that makes no sense to anyone. |
| 16:25 | arbscht | sounds a lot like porting code between computer languages |
| 16:25 | hiredman | ~sense |
| 16:25 | clojurebot | No entiendo |
| 16:26 | Chouser | Chousuke: something like this? http://translationparty.com/#3751265 |
| 16:27 | Chousuke | That's perfect! |
| 16:28 | Chousuke | I suppose it doesn't work with Finnish. :/ |
| 16:30 | ambient | so what's the joke? |
| 16:31 | jensli | Finnish has plentifull of, is it calls 'cases' in english? Maybe that is a bit like polymorfism. |
| 16:31 | ambient | Finnish is like elvish, it mangles words to give them different meanings |
| 16:32 | duncanm | i can't convince XMLEventWriter to write out <foo /> instead of <foo></foo> for me ;-( |
| 16:35 | cemerick | Chousuke: I think we need a preferred-editor-poll :-P |
| 16:35 | ambient | is that an emacs command? |
| 16:35 | Chousuke | :D |
| 16:46 | LauJensen | ~ozzilee |
| 16:46 | clojurebot | It's greek to me. |
| 16:46 | LauJensen | ~seen ozzilee |
| 16:46 | clojurebot | no, I have not seen ozzilee |
| 17:00 | wtetzner_ | how are datetimes usually dealt with in clojure? |
| 17:01 | wtetzner_ | is there an immutable datetime object, or do people just take a java datetime and construct a hash-map from it? |
| 17:03 | stuartsierra | I use Java Date and just don't modify it. |
| 17:03 | wtetzner_ | ok |
| 17:04 | stuartsierra | The Joda library is popular, as well. |
| 17:04 | wtetzner_ | cool, i'll take a look at it |
| 17:04 | wtetzner_ | thanks |
| 17:07 | jensli | You could take a java DateTime and call (bean ...) on it? |
| 17:09 | jensli | Ops, speaking .net here, Date it is |
| 17:09 | stuartsierra | ,(bean (java.util.Date.)) |
| 17:09 | clojurebot | {:seconds 28, :date 31, :class java.util.Date, :minutes 11, :hours 14, :year 109, :timezoneOffset 420, :month 7, :day 1, :time 1251753088119} |
| 17:09 | stuartsierra | that works surprisingly well |
| 17:10 | wtetzner_ | yeah |
| 17:10 | wtetzner_ | jensli: thanks |
| 17:11 | jensli | But how does that struct works with the Date methods? |
| 17:11 | wtetzner_ | why is year 109? |
| 17:12 | Chouser | wtetzner_: years since 1900. yes, really. |
| 17:12 | wtetzner_ | hmm |
| 17:13 | Chouser | people speak highly of Joda Time which apparently has an immutable date/time object. |
| 17:13 | wtetzner_ | Chouser: yeah, stuartsierra mentioned it too |
| 17:13 | wtetzner_ | looks pretty fancy |
| 17:14 | stuartsierra | haven't tried it myself |
| 17:16 | Chouser | oh, sorry, missed that in the join/part noise. |
| 17:17 | technomancy | wtetzner_: I wrote a wrapper for java.util.Date that translates between those and maps, but Date doesn't give correct behaviour for dealing with daylight savings and other time-based edge cases |
| 17:17 | technomancy | so joda is probably better if you need that |
| 17:17 | wtetzner_ | technomancy: ok, thanks |
| 17:17 | wtetzner_ | yeah, i think i'm gonna try going with joda |
| 17:21 | technomancy | the built-in JDK Date stuff is really horrible |
| 17:22 | stuartsierra | eh, it works for simple cases |
| 17:24 | cemerick | yeah, I've never had reason to stray. I know it's fundamentally shoddy, but... *shrug* |
| 17:28 | wavis | i'm experiencing some weirdness with the set function |
| 17:29 | wavis | I have a function that simplifies a common case where I use map, but instead of returning a seq it returns a set |
| 17:30 | hiredman | … |
| 17:32 | wavis | sorry got a call |
| 17:33 | wavis | so if i take out the set function, it doesn't run out of memory, and i can just apply the set outside, to the results of the function |
| 17:33 | wavis | and everything is hunky dorey. is there some weirdness with nesting a map within the #(set %) ? |
| 17:34 | wavis | like (set (map #(awesomeness %) bigseq)) |
| 17:34 | Chouser | ,(set {:a 1 :b 2}) ; like this?? |
| 17:34 | clojurebot | #{[:b 2] [:a 1]} |
| 17:34 | Chouser | oh, that kind of map. |
| 17:34 | wavis | yeah |
| 17:35 | Chouser | how big? I just put a million Integers in a set that way with no problem. |
| 17:36 | wavis | there are a few tens of thousands of entries in bigseq, and each is a map like {:id 100 :name "tommy"} |
| 17:36 | Licenser | isn't map doing lazy sequs so mapping over a huge set would not instantly do much? |
| 17:37 | Chouser | Licenser: yes, but 'set' would force the whole thing. |
| 17:37 | Licenser | ah okay |
| 17:37 | Licenser | again I learned something :) |
| 17:37 | wavis | right. without the enclosing set function it returns instantly |
| 17:37 | ambient | what's the easiest way to transform [2 10 4 59 9 3 ...] sequence into [[2 10] [4 59] [9 3] ...]? |
| 17:37 | stuartsierra | I got memory errors with (count (set (map identity (range 1000000)))) |
| 17:37 | ambient | im sure there's a single function for this |
| 17:37 | Licenser | perhamps pmap would hep here? |
| 17:37 | Chouser | ambient: (partition 2 coll) |
| 17:37 | stuartsierra | Licenser: pmap won't help with the memory problem |
| 17:38 | ambient | Chouser ok ty I need to look that up |
| 17:38 | Licenser | hmm string the data in half bits? *hides* |
| 17:38 | Chouser | ,(count (set (map inc (range 1e6)))) |
| 17:38 | clojurebot | Execution Timed Out |
| 17:38 | Licenser | I try it right now |
| 17:38 | Chouser | hm. inconclusive. :-) |
| 17:38 | Licenser | 1:1 user=> (count (set (map identity (range 1000000)))) |
| 17:38 | Licenser | 1000000 |
| 17:38 | Licenser | works for me |
| 17:39 | stuartsierra | add another zero |
| 17:39 | Chouser | this is why I like the 1e6 notation |
| 17:40 | stuartsierra | Basically, if adding the set around the map is the only thing that causes a problem, it's probably because the set is too big to fit in memory all at once. |
| 17:40 | drewr | ,1e17 |
| 17:40 | clojurebot | 1.0E17 |
| 17:40 | Licenser | I don't really have a problem yet |
| 17:41 | Licenser | ah here we go: |
| 17:41 | Licenser | 1:3 user=> (count (set (map identity (range 10000000)))) |
| 17:41 | Licenser | java.lang.OutOfMemoryError: Java heap space (repl-1:3) |
| 17:41 | Licenser | out of heap space |
| 17:41 | Licenser | it seems to be a problem with recursion to me |
| 17:41 | stuartsierra | map isn't recursive |
| 17:41 | Licenser | kind of semi recursive isn't it? with loop I guess |
| 17:42 | stuartsierra | loop exists to avoid recursion |
| 17:42 | Licenser | it wasn't a memory problem, I hardly noticed the process to use more memory then before |
| 17:42 | hiredman | map doesn't use loop |
| 17:42 | stuartsierra | right, it's lazyor whatever |
| 17:42 | Chouser | Licenser: what happens if you drop the map, and use (set (range ...)) |
| 17:43 | Licenser | spam :P |
| 17:43 | Licenser | I should have put count there |
| 17:43 | Chouser | or set your *print-length* :-) |
| 17:43 | Licenser | 1:1 user=> (count (map identity (range 10000000)) |
| 17:43 | Licenser | ) |
| 17:43 | Licenser | 10000000 |
| 17:44 | Licenser | works with no problem at all |
| 17:44 | stuartsierra | I think the problem is that Licenser had a sequence of maps, which take up considerably more space than an Integer. |
| 17:44 | hiredman | Licenser: that is dropping the set |
| 17:44 | hiredman | not the map |
| 17:44 | Licenser | oops sorry |
| 17:44 | stuartsierra | When you force the entire sequence into memory with "set", it's too big. |
| 17:44 | wavis | there's more to this whole situation for me. the seq is lazily reading a large csv which can't fit in memory. is it possible that the function prevents cleaning up the streamed file? |
| 17:44 | Licenser | let me test |
| 17:45 | Licenser | the problem isn't that the set does not fit in the memory |
| 17:45 | stuartsierra | ok |
| 17:45 | Licenser | it's aheap error not a you don't have enough memory error |
| 17:45 | wavis | no the set fits in memory fine. it's about 70k longs. |
| 17:45 | Licenser | 1:1 user=> (count (set (range 10000000))) |
| 17:45 | Licenser | java.lang.OutOfMemoryError: Java heap space (repl-1:1) |
| 17:45 | Licenser | yap there is something very wrongish |
| 17:45 | wavis | Licenser: oh |
| 17:46 | wavis | well i'll work around it for now with a do |
| 17:46 | Chouser | Is this legit? (let [s (atom! nil)] (defn my-singleton [] (or @s (swap! s #(or @s (MySingleton.)))))) |
| 17:47 | stuartsierra | Chouser: that's more or less what clojure.contrib.singleton does. |
| 17:47 | Licenser | I forgot which sutff I had to (use) to see the soruce code o.O |
| 17:47 | Chouser | oh, maybe I shuould just use delay... |
| 17:48 | stuartsierra | Chouser: that too. c.c.singleton can do per-thread singletons too |
| 17:48 | Chouser | stuartsierra: oh, ok. thanks. |
| 17:48 | stuartsierra | But someone pointed out to me that a global singleton is the same thing as a delay. |
| 17:51 | Licenser | I get the feeling the problem is in the implemetation of set |
| 17:55 | Chouser | it'd be nice if there was a handy conclusive test for a seq-head being held. |
| 17:57 | stuartsierra | Rich suggested recently on the mailing list that more such cases can be avoided, but that doing so would require more sophistication on the part of the compiler. |
| 17:58 | wavis | oh interesting so set just uses apply |
| 17:58 | Licenser | I think to thinn I found teh bad guy |
| 17:58 | Licenser | but I might be wrong |
| 17:59 | Licenser | set uses spread, which is recursive |
| 18:00 | achim | Licenser: recursion wouldn't blow the heap, but the stack, right? |
| 18:00 | Licenser | isn't the stack in the heap? |
| 18:02 | wavis | I just tried with (into #{} (map #(awesomeness %) bigseq)) and it still runs out of heap |
| 18:04 | wavis | and the same for: |
| 18:04 | wavis | (reduce #(conj %1 %2) #{} (map #(awesomeness %) bigseq)) |
| 18:05 | Chouser | is that a chunked bigseq? |
| 18:05 | wavis | hmm chunked? |
| 18:06 | wavis | how do i chunk something? |
| 18:06 | Chouser | (chunked-seq? bigseq) to find out |
| 18:06 | Chousuke | #(awesomeness %) is a bit redundant btw :P |
| 18:06 | Chouser | depends on how the seq is built |
| 18:07 | Chousuke | I bet overuse of #() is the most common anti-idiom |
| 18:07 | wavis | not chunked |
| 18:07 | wavis | k, i'll #() from here out if it's preferred |
| 18:08 | Chouser | and (set bigseq) works ok? |
| 18:08 | jensli | Yeah, (fn ...) is so extremly verbose. Specially compared to Java. |
| 18:08 | Licenser | okay a bad recursion gives a stack overflow not a heap overflow |
| 18:09 | Licenser | good greif |
| 18:09 | Chousuke | wavis: I mean, (map #(awesomeness %) foo) could be expressed as just (map awesomeness foo) :) |
| 18:10 | hiredman | Licenser: the stack contains pointers to objects on the heap, basically |
| 18:11 | Licenser | Okay it seems java by default only uses very little memory o.O that seems to be the real problem |
| 18:11 | Licenser | It does not just use the memory of your computer as much as it wants to |
| 18:12 | Licenser | I added -Xmx200m and now it runs way longer |
| 18:12 | Licenser | no Heap exception yet |
| 18:12 | wavis | (set bigseq) also run out of heap. and although i thought it worked before, now i'm losing on just > (map :stuff bigseq) > (set *1) in the repl |
| 18:12 | wavis | i have -Xmx512 so that's not it i don't think |
| 18:13 | Licenser | it just means that there is something very big ^^ |
| 18:14 | Licenser | wow it still is calculating o.O |
| 18:14 | wavis | oh crap, do i really? *sigh* i set that halfway through and forgot the "m". I need -Xmx512m |
| 18:14 | wavis | so i need to retry some things |
| 18:14 | Licenser | oi |
| 18:14 | Licenser | yes with 512 it really won't go very far |
| 18:15 | Licenser | 1:1 user=> (count (set (range 10000000))) |
| 18:15 | Licenser | java.lang.OutOfMemoryError: Java heap space (repl-1:1) |
| 18:15 | Licenser | :( |
| 18:16 | Licenser | doing (0.. 10000000).to_a in ruby it takes about 200 MB |
| 18:17 | ambient | try forcing a type? im sure 1e10 big ints will take more than 1e10 32 bit integers |
| 18:17 | Licenser | yea it are fixnums not bignums |
| 18:17 | Licenser | but still |
| 18:17 | Licenser | it is a good estimate |
| 18:18 | Licenser | making a set takes about 200MB too |
| 18:18 | Licenser | irb(main):003:0> (s = Set.new((0...10000000))).size |
| 18:18 | Licenser | => 10000000 |
| 18:18 | Licenser | and oddly enough is way faster |
| 18:18 | wavis | ok, nice, so (into #{} (map foo bigseq)) works fine |
| 18:18 | Licenser | something seems very wrong wiht clojure there |
| 18:18 | hiredman | http://books.google.com/books?id=dg7bj_e-SAMC&lpg=PP1&dq=jvm&pg=PA9#v=onepage&q=stack&f=false <-- "Java Stack" |
| 18:18 | Licenser | hiredman: thanks |
| 18:19 | Licenser | (count (set (range 10000000))) is the same as (s = Set.new((0...10000000))).size or am I wrong? |
| 18:20 | Licenser | creating a new set with 10000000 elements and counting it's size |
| 18:22 | hiredman | well, not all sets are the same |
| 18:23 | Licenser | set as in a collection of unique values |
| 18:23 | hiredman | … |
| 18:24 | hiredman | so? |
| 18:24 | hiredman | that is a specification |
| 18:24 | Licenser | I just wonder why it is so sloow and crashes my JVM even with 200 MB of memory |
| 18:24 | hiredman | it leaves a great degree of freedom on the implementation side |
| 18:24 | ambient | premature optimization is premature optimization |
| 18:27 | hiredman | ,(class #{}) |
| 18:27 | clojurebot | clojure.lang.PersistentHashSet |
| 18:27 | Licenser | Well I'm not sure if it is premature to make things somewhat fastish. It is not like a exotic thing to have a set of values. |
| 18:28 | Chousuke | how many values, exactly? |
| 18:28 | Licenser | might the reason be that it can't convert the entire set due to immutability but has to do it element by element? |
| 18:28 | hiredman | no |
| 18:28 | Licenser | please don't get me wrong, I don't want to jump con clojure. I like the language, I just would like to figure out why it is so slow for this task |
| 18:29 | Licenser | here it are 10_000_000 values |
| 18:29 | ambient | (count (set (range foo))) could in princible done lazily |
| 18:29 | Chousuke | hmm, and each value is how big? :/ |
| 18:29 | hiredman | ambient: it would be difficult to do and still retain the nice java interop clojure enjoys |
| 18:30 | Licenser | Chousuke: simple fixnums |
| 18:30 | Licenser | (time (count (set (range 10000000)))) |
| 18:30 | Chousuke | Licenser: don't forget they all need to be boxed though |
| 18:31 | hiredman | ,(first (clojure.lang.PersistentHashSet/create (range 1e6))) |
| 18:31 | clojurebot | Execution Timed Out |
| 18:31 | ambient | using (int 10000000) halves the time used |
| 18:31 | hiredman | the jvm doesn't have fixed nums, btw |
| 18:32 | Licenser | however they call it |
| 18:33 | Licenser | for me it's running 7 minutes now |
| 18:33 | Chousuke | for 10 million it does 10 million allocations |
| 18:33 | Chousuke | I guess that takes some time |
| 18:33 | Chousuke | :P |
| 18:34 | Chousuke | which version of clojure are you using btw? |
| 18:34 | Licenser | Okay if it is 'only' that it can be fixed later with some tricks I guess (like bulkallocating space for the boxes) |
| 18:34 | Licenser | Clojure 1.1.0-alpha-SNAPSHOT |
| 18:35 | Chousuke | hm, interesting |
| 18:35 | Chousuke | user=> (time (count (set (range 1000000)))) |
| 18:35 | Chousuke | java.lang.OutOfMemoryError: GC overhead limit exceeded (NO_SOURCE_FILE:0) |
| 18:35 | ambient | i'd rather have a real problem though. |
| 18:35 | Chousuke | yeah, microbenchmarks are problematic. |
| 18:35 | ambient | this isn't even a microbenchmark |
| 18:35 | Licenser | true, but it's a interesting thing still |
| 18:36 | ambient | this is just bad coding |
| 18:36 | Chousuke | it might be unrealistic behaviour that throws off the optimiser and GC |
| 18:36 | Licenser | well someone actually ran in that problem, that is why it came up. We didn't made it up |
| 18:36 | Licenser | so I think it is kind of real at least |
| 18:37 | ambient | then you need to change your code from (count (set (range 1000000))) to 1000000 |
| 18:37 | ambient | much faster that way |
| 18:37 | Licenser | the count set was just an example to replicate the behaviour of the real code without needing all the dependencies around it |
| 18:38 | ambient | I think I have to take your word on that |
| 18:39 | Licenser | (set (map #(awesomeness %) bigseq)) <- that was the original problem |
| 18:39 | Licenser | about parsing a big log file or something |
| 18:39 | Licenser | it was discussed about an houre ago if you want to look in the scrollback |
| 18:41 | ambient | i see only your solution to the problem, not the problem itself :/ correct me if im wrong |
| 18:41 | Chousuke | hm |
| 18:41 | Chousuke | interesting |
| 18:42 | Chousuke | (into #{} (range 1000000)) works but (set (range 1000000)) fails |
| 18:42 | ambient | but alas, im a total newbie with clojure so im not much help |
| 18:42 | Chousuke | maybe set doesn't use transients |
| 18:42 | wavis | do you need a test case? i can generate a large csv of gobble-de-gook, provide my csv parsing function, et ceteras |
| 18:43 | Chousuke | into performance is not too bad either I guess. |
| 18:43 | wavis | but I'm not sure that's necessary |
| 18:43 | Chousuke | boxing and inserting a million takes 12 seconds here |
| 18:43 | hiredman | Chousuke: should be almost the same |
| 18:43 | Chousuke | hiredman: into works, set fails. the performance is quite different :P |
| 18:43 | hiredman | ipersistenthashset/create is just a for loop that uses cons internally |
| 18:44 | Chousuke | hm, I wonder if I have transient sets yet |
| 18:44 | hiredman | I have actually looked at the clojure set function |
| 18:44 | hiredman | ~def set |
| 18:44 | durka42 | ~def hash-set |
| 18:45 | Chousuke | hm, apparently no transient sets in my clojure |
| 18:46 | achim | set will realize the entire sequence before starting to consume it (args to apply must fit in memory), into doesn't do that |
| 18:47 | hiredman | achim: that is not true |
| 18:47 | durka42 | apply is lazy is it not |
| 18:47 | Chousuke | it is. |
| 18:47 | Chousuke | hiredman: for some reason, (set) still fails and into doesn't. |
| 18:47 | Chousuke | I think it might be the chunkedness. |
| 18:48 | Chousuke | c.l.PHS/create is not chunked, but reduce is |
| 18:48 | Chousuke | so, it does less allocation |
| 18:48 | Chousuke | and since I'm failing with "GC overhead limit exceeded" that might just be it. |
| 18:49 | hiredman | (time (first (set (range 1e7)))) runs fine with -Xmx1000m |
| 18:51 | ambient | how can you take first from set? |
| 18:51 | ambient | isn't that undetermined |
| 18:52 | achim | hiredman: you're right |
| 18:52 | hiredman | ambient: first calls seq which returns a sequence |
| 18:52 | hiredman | ,(first #{1 2 3}) |
| 18:52 | clojurebot | 1 |
| 18:52 | hiredman | ,(first {1 2 3 4}) |
| 18:52 | clojurebot | [1 2] |
| 18:53 | hiredman | ,(first [1 2 3 4]) |
| 18:53 | clojurebot | 1 |
| 18:53 | hiredman | etc |
| 18:53 | hiredman | so first should work on anything that the seq function works on |
| 18:53 | hiredman | achim: about what? |
| 18:53 | ambient | i feel conflicted by that kind of flexibility in data structures |
| 18:53 | Licenser_ | bah internet died |
| 18:54 | achim | hiredman: but retains head, so args to apply really must fit in memory |
| 18:54 | achim | er |
| 18:54 | achim | but "apply" retains head |
| 18:54 | achim | http://groups.google.com/group/clojure/msg/34b9a170d36c5ab5 |
| 18:54 | hiredman | ~def apply |
| 18:56 | Licenser_ | ~def spread |
| 18:57 | hiredman | ,(let [f (fn [x & y] x)] (apply f (interate inc 0))) |
| 18:57 | clojurebot | java.lang.Exception: Unable to resolve symbol: interate in this context |
| 18:57 | hiredman | ,(let [f (fn [x & y] x)] (apply f (iterate inc 0))) |
| 18:57 | clojurebot | 0 |
| 18:57 | hiredman | *tada* |
| 18:58 | drewr | ,(let [f (fn [x & y] x)] (apply f (range 1000000))) |
| 18:58 | clojurebot | 0 |
| 18:58 | drewr | it's pseudo-lazy, but that doesn't mean it doesn't retain head |
| 18:58 | achim | hiredman: that doesn't show it doesn't retain head ... |
| 18:59 | Licenser | when talking about lazy, I've to work tomorrow early :) good night everyone! And good luck with this! |
| 18:59 | drewr | ,(let [f (fn [x & y] x)] (apply f (doall (range 1000000)))) |
| 18:59 | clojurebot | 0 |
| 18:59 | drewr | ,(let [f (fn [x & y] x)] (apply f (doall (range 1e10)))) |
| 18:59 | clojurebot | Execution Timed Out |
| 19:00 | hiredman | ,(let [f (fn [x & y] (first (drop 1e3 y)))] (apply f (iterate inc 0))) |
| 19:00 | clojurebot | 1001 |
| 19:00 | hiredman | ,(let [f (fn [x & y] (first (drop 1e7 y)))] (apply f (iterate inc 0))) |
| 19:00 | clojurebot | Execution Timed Out |
| 19:00 | Chousuke | hm |
| 19:01 | Chousuke | I think you're holding on to y in that case. |
| 19:01 | Chousuke | because you're calling first |
| 19:01 | Chousuke | so when the (drop ...) is completed the reference to y still exists |
| 19:02 | Chousuke | ... I wonder if I'm correct ;/ |
| 19:02 | cemerick | are struct-maps transient-capable in HEAD? |
| 19:18 | wtetzner_ | Chousuke: doesn't first just return the value of the first thing in the sequence? not the front of the sequence? |
| 19:18 | hiredman | … |
| 19:19 | wtetzner_ | so you wouldn't be holding onto y still |
| 19:20 | Chousuke | wtetzner_: well, the drop of course won't return y, but I think it's still there because it's in the function's locals. |
| 19:20 | wtetzner_ | oh, i see |
| 19:20 | Chousuke | and the drop call is not the tail call (at least I think it isn't) |
| 19:20 | Chousuke | I'm not sure how exactly it works |
| 19:21 | hiredman | I'd have to replace it with a call to nth |
| 19:22 | hiredman | I always forget about nth |
| 19:22 | Chousuke | but say you do (fn [x] (let [a (drop 1000000 x)] (first a))) then you will hold on to all of x |
| 19:22 | Chousuke | the compiler is not yet smart enough to optimise that :/ |
| 20:36 | powr-toc | If I have a list of functions '(f1 f2 f3) how would I best execute them? |
| 20:36 | powr-toc | sequentially... |
| 20:40 | wtetzner_ | powr-toc: (doall (map (fn [fucn] (func)) '(f1 f2 f3))) |
| 20:42 | durka42 | (doseq [f (list #(prn 1) #(prn 2) #(prn 3))] (f)) |
| 20:42 | durka42 | ,(doseq [f (list #(prn 1) #(prn 2) #(prn 3))] (f)) |
| 20:42 | clojurebot | 1 2 3 |
| 20:43 | wtetzner_ | durka42's solution is nicer |
| 20:43 | tomoj | why isn't there a "call" function |
| 20:43 | durka42 | because you just call things... |
| 20:43 | durka42 | there's apply |
| 20:44 | wtetzner_ | ,(.invoke #(prn 2)) |
| 20:44 | clojurebot | 2 |
| 20:44 | durka42 | ,(#(prn 2)) |
| 20:44 | clojurebot | 2 |
| 20:45 | tomoj | but .invoke can't be mapped :( |
| 20:45 | wtetzner_ | ,(.call #(prn 2)) |
| 20:45 | clojurebot | 2 |
| 20:45 | durka42 | #(.invoke %) can be mapped |
| 20:45 | durka42 | as can #(%) |
| 20:45 | durka42 | ,(apply #(prn 2)) |
| 20:45 | clojurebot | 2 |
| 20:45 | tomoj | #(%) looks cool but that's just weird |
| 20:45 | tomoj | apply works, it seems |
| 20:46 | tomoj | ,(map apply (list #(prn 1) #(prn 2) #(prn 3))) |
| 20:46 | clojurebot | (nil nil nil) |
| 20:46 | tomoj | well.. yeah |
| 20:46 | durka42 | ,(dorun (map apply (list irc://irc.freenode.net/#(prn 1) irc://irc.freenode.net/#(prn 2) irc://irc.freenode.net/#(prn 3)))) |
| 20:46 | clojurebot | java.lang.ClassNotFoundException: irc://irc.freenode.net |
| 20:46 | wtetzner_ | ,(map #(.invoke %) (list #(prn 1) #(prn 2) #(prn 3))) |
| 20:46 | clojurebot | (nil nil nil) |
| 20:46 | durka42 | ehh... what's that, clojurebot? |
| 20:46 | durka42 | ,(dorun (map apply (list irc://irc.freenode.net/#(prn 1) irc://irc.freenode.net/#(prn 2) irc://irc.freenode.net/#(prn 3)))) |
| 20:46 | clojurebot | java.lang.ClassNotFoundException: irc://irc.freenode.net |
| 20:46 | durka42 | hiredman: ??? |
| 20:47 | hiredman | clojurebot: huh? |
| 20:47 | clojurebot | Gabh mo leithscéal? |
| 20:47 | tomoj | you've got "irc://irc.freenode.net/" in your code |
| 20:47 | hiredman | durka42: what? |
| 20:47 | durka42 | i do? |
| 20:47 | tomoj | from here it looks like you do |
| 20:47 | tomoj | apparently clojurebot thinks so too |
| 20:47 | tomoj | 19:47 <durka42> ,(dorun (map apply (list irc://irc.freenode.net/#(prn 1) irc://irc.freenode.net/#(prn 2) irc://irc.freenode.net/#(prn 3)))) |
| 20:47 | hiredman | ↑ |
| 20:47 | durka42 | ,(dorun (map apply (list #(prn 1) #(prn 2) #(prn 3)))) |
| 20:47 | clojurebot | 1 2 3 |
| 20:48 | durka42 | spooky copy and paste action |
| 23:05 | LauJensen | I've tried to add a threaded buffered logger to www.bestinclass.dk and now the site seems a bit slow to me. Can somebody here try to access it and let me know what they think? |
| 23:19 | LauJensen | Gents? |
| 23:25 | cark | do you mean the first access ? |
| 23:27 | cark | mhh if i was a corporate customer i wouldn't want my web site doing this accordeon thing |
| 23:28 | cark | but it's responsive from belgium |
| 23:37 | scottj | Anyone use visualvm to profile? if my namespace is foo, do I put foo.* in the Settings box labeled "Start profiling from classes:"? |
| 23:38 | scottj | (I guess I wouldn't be asking if that was working for me :) |
| 23:41 | scottj | I start clojure from shell, run slime-connect from emacs, load code, launch visualvm and connect to jline.ConsoleRunner, change Profiler settings as mentioned, click CPU, it instruments 21 methods, then I go back to slime and run a 2 minute operation, and in the profiler nothing shows up except maybe a few things outside my app. |
| 23:42 | LauJensen | cark: Thanks for both testing and giving design advice :) Yes it was first response I was worried about |
| 23:42 | cark | first response isn't very fast |
| 23:42 | LauJensen | not fast or slow ? |
| 23:42 | cark | nah it's ok |
| 23:43 | cark | i guess you have quit e a bit of javascript loading there |
| 23:43 | LauJensen | Its not that bad actually |
| 23:44 | LauJensen | The amount of javascript that is. But I decorated my entire servlet in a buffered logger which keeps a certain amount of records in memory, flushing it every 5 minutes to SQL. I was afraid it was lagging somehow |
| 23:45 | cark | mhh but how could a logger slow things down ? even to an sql database ... do you plan to have millions page views ? |
| 23:45 | cark | that's because you open a connection to the database per request ? |
| 23:47 | cemerick | scottj: I generally just exclude the junk I don't want, and profile everything (I use NetBeans/enclojure -- visualvm is based on the former) |
| 23:48 | cemerick | scottj: with good results, FWIW http://twitter.com/cemerick/status/3615358808 :-D |
| 23:48 | LauJensen | cark: no no, like I said it flushes every 5 minutes, so it keeps a buffer. And I'm not saying I can explain how it would slow things down, it was just a 'feeling' :) |
| 23:48 | cark | ok =) |
| 23:49 | cark | see i'm just pestering you because i'd like to see some connection pooling in clojureql ! |
| 23:49 | LauJensen | hehe, are you following us on lighthouse? because its really quite close to implementation now |
| 23:50 | cark | nope didn't know about it |
| 23:50 | LauJensen | lemme dig up the discussion |
| 23:50 | cark | i've got it thanks |
| 23:52 | LauJensen | http://clojureql.lighthouseapp.com/projects/34981/tickets/4-allow-optional-global-db-conn |
| 23:52 | LauJensen | this one right? We're still not in agreement, so you're welcome to add your wisdom to the equation |
| 23:53 | cark | well i have a library doing generic pooling, i use it to pool memcached connections |
| 23:53 | cark | and was thinking maybe applying this to sql connections as well |
| 23:54 | cark | though memcached is very slow now due to print/read serialization |
| 23:54 | cemerick | actually, one thing I miss from eclipse is *per-line* runtime stats while profiling. That was tremendous. |
| 23:55 | cemerick | Obviously, only available in Java. |
| 23:55 | LauJensen | cark: The discussion is not centered around technology, its more user implementation details. Unless I'm completely misunderstanding you since its 05:56 here :) |
| 23:57 | cark | my fault i was digressing |
| 23:57 | cark | it's 6am here too ... i'm off to bed ! |
| 23:58 | LauJensen | Alright, good night, and thanks |
| 23:58 | cark | good night |