#clojure logs

2010-12-26

06:36maaclLauJensen: Glaedelig jul!
06:37LauJensenmaacl: Tak for tanken, men jeg fejrer ikke jul :)
06:37maaclOk - religioest eller anti-religioest betinget?
06:38maaclLauJensen: ups - religioest eller anti-religioest betinget?
06:40LauJensenYou got pinged in private )
06:40LauJensen:)
06:55zvrbaok, installing clojure in ANY os is a PITA
06:59RaynesDefine 'install'.
06:59RaynesYou don't really 'install' Clojure. All Clojure is is a jar file. The closest thing you get to 'installing' Clojure is just downloading cake or leiningen, which isn't all that difficult at all.
06:59zvrbai write BigInteger/LONG_MASK in REPL, and clojure replies with not being able to find static field.
07:00zvrbaRaynes: install = making it work with emacs. and understanding how to do it in proper order. persuading windows to get paths right. figuring out whether I need ant+maven (didn't install them, still unsure whether I need them - netbeans plugin recommends maven strongly)
07:00zvrbaetc.
07:01zvrbaRaynes: i.e. install = anything more than running a simple REPL in shell prompt.
07:02RaynesGetting it to work with Emacs isn't really part of installing Clojure. You don't really *need* Emacs to use Clojure. There is a plugin for IntelliJ and Eclipse. As for ant and maven, you don't really need them unless you need them to build any particular project. Most Clojure projects use Leiningen or Cake as their build tools.
07:03zvrbai know that I don't *need* emacs.
07:03zvrbagiven that clojure uses a lot of java, i might really be better off using IntelliJ
07:04zvrbalast time I tested plugin for IntelliJ, it didn't work. it was a year ago. i DO NOT want to use eclipse. which leaves me with NetBEans vs. IntelliJ.
07:04zvrbaany thoughts on that one?
07:04RaynesThe act of getting Clojure to work with Emacs is actually a lot easier than it used to be though. It used to be that you'd need to put stuff in .emacs, which was error prone and never really worked for everybody. Now, you can just grab ELPA and use it to install clojure-mode and slime-repl, and then grab cake or leiningen and use them to get a swank server running on your project which you can then connect to via M-x slime-connect
07:04RaynesThe IntelliJ plugin was pretty young a year ago. I bet you'll have much better luck now.
07:04zvrbayeah, i figured that out. the toughest part was realizing that I needed (package-init) to get packages installef with ELPA to load :-)
07:05zvrbaRaynes: any thoughts on IntelliJ vs. NetBeans in general?
07:05zvrba(related to clojure programming)
07:06RaynesI don't use either, so anything I say is based on what I've heard. It looks like Enclojure's development is much slower than the other plugins development, and a lot of people simply cannot get Enclojure to work.
07:06RaynesI've heard good things about the IntelliJ plugin.
07:06Rayneszvrba: Also, it doesn't look like BigInteger has a field called LONG_MASK
07:07Rayneshttp://download.oracle.com/javase/1.4.2/docs/api/java/math/BigInteger.html
07:07zvrbaRaynes: oh, ok. I get it listed with tab-completion in slime.
07:07zvrbahence, confusion.
07:07RaynesHrm.
07:08zvrbaBigInteger/<TAB> lists that field.
07:08zvrbaah. maybe it's protected, or something.
07:08RaynesBizarre.
07:08RaynesYeah, it could be.
07:08zvrbaswank maybe is not aware of access levels.
07:09zvrbaok, i'll give IntelliJ a try then.
08:01zvrbathere doesn't seem to be a way to nicely interact with REPL from intellij
08:01zvrbalike, write something in the source file, send it to REPL and set focus to REPL to test it
09:31RaynesThat was quick.
09:32cemerickRaynes: you rang? :-P
09:32RaynesMerry Christmas cemerick. <3
09:32cemericksame to you, man :-)
09:32RaynesWhat did you get? Looks like Land of Lisp was a popular gift item this year.
09:33cemerickLoL isn't really my style :-)
09:33RaynesI spent all of yesterday morning finishing that initial draft of your book.
09:34cemericklet's see: a bluray player (which I'll probably trade in for a ps3), a very nice watch, and some light reading I'll tweet about later
09:34cemerickRaynes: not too much of a disaster, I hope?
09:34RaynesI was on the interop chapter, which I had plenty of motivation to read, given that I'm actually just getting to that chapter in my own book.
09:35RaynesThe book wasn't bad at all. The CouchDB chapter was also relevant to my interests, given that I'm working on a project that will call for it.
09:35cemerickI'm leaving a fair bit of that undone, given the prim/numerics changes, etc.
09:35RaynesKind of disappointed at how awkward talking to CouchDB is compared to MongoDB.
09:35cemerickthat's interesting
09:36cemerickRaynes: do you have a favorite mongodb/clojure interaction blog post?
09:36RaynesWell, given the nature of a predominately electronic unpublished book, I can change and add things as they become relevant and release new "editions" easily after the book has been finalized.
09:36RaynesI don't think I've read any. Just a look at the congomongo README should be sufficient though.
09:37RaynesNote that just because it seems awkward to me doesn't really say a lot about it.
09:37RaynesIt's obvious that views are a much more powerful concept.
09:37RaynesI just wish they provided for MongoDB-like lookups in simpler cases.
09:38cemerickoh, you mean ad-hoc query interaction?
09:38RaynesRight.
09:38cemerickyeah, tha'ts a whole 'nuther ballgame
09:39Raynes:p
09:39cemerickthe theory there being that it's just not the right tool for that particular job
09:40cemerickThere's actually very slick replication endpoints for pushing couchdb data into mongo, where you would presumably do your ad-hoc querying in the latter.
09:40RaynesI was considering switching to Couch in sexpbot, but I definitely need to do more research on that. I need to know precisely what I would gain by doing so, because it certainly wouldn't be more concise code.
09:42Raynescemerick: I've nearly beat you as far as pages go, by the way. ;)
09:42cemerickRaynes: sexpbot uses mongo on the backend?
09:42Raynes76 in my latest draft. No stub chapters or anything like that.
09:42RaynesRight.
09:43RaynesA lot of plugins call for a database.
09:43cemerickRaynes: tortoise and the hare and all that comes to mind :-)
09:43cemerickI think we're probably aiming at different demographics anyways ;-)
09:43RaynesWell, I don't have a job or much of a life either, so I naturally have more time to write.
09:43cemerickRaynes: I hope you're doing whatever is required to make mongodb do sane things re: durability?
09:44cemerickI assume you're not running a "web scale" cluster over there ;-)
09:45RaynesNot really. I just store stuff in it. I'm not sure how much sexpbot could really bring out the bad in MongoDB at this point. I really just picked a DB and started using it, sans actual research. I don't know much about the general pros and cons regarding the various databases.
09:46RaynesIf I had to store data that was actually important, I might care more.
09:46chouserI think running a single mongodb server without a replication set may very well lead to heartache
09:46chouserI think mysql promises more about durability than an unreplicated mongodb
09:47Rayneschouser: I've been looking all over for you.
09:47cemerickRaynes: insofar as sexpbot may provide logging capability in the future, that may change.
09:47Raynescemerick: Define 'logging'.
09:47cemerickchouser: Well said, sir!
09:47cemerickRaynes: channel logging, etc
09:47cemerickchouser: Merry belated Christmas :-)
09:48Rayneschouser: There is an example in The Joy of Clojure that defines a print-method method for queues: may I steal it for usage in my book?
09:48RaynesGiving credit where credit is due, of course.
09:48Raynescemerick: sexpbot already does channel logging.
09:48Raynescemerick: But that has nothing to do with the database.
09:48Rayneshttp://raynes.me/logs/
09:49cemerickRaynes: it does if the logs are in an unreplicated mongo instance.
09:49RaynesWhy would they be?
09:49RaynesI just write to log files for consumption by a web browser or text editor.
09:49RaynesNo database storage necessary.
09:50cemericknevermind, then :-)
09:50cemerickI figured they'd be in the database to support potentially interesting query capabilities, etc.
09:50chouserheh, the queue fish? Sure, I think quoting text at that scale is clearly within fair use.
09:50RaynesAnyways, I have been planning to discuss database options with amalloy_ very soon, before we *do* have to store important data. You know, get while the getting is still good and all that.
09:51Rayneschouser: The queue fish is adorable.
09:51chousercemerick: I once worked very hard to get sane irc log search working in a mysql database (for a private IRC channel). My best was horrible little joke compared to google indexing the static pages for #clojure.
09:52RaynesMy IRC logs are extremely important. I mean, I just couldn't imagine what I would do if I needed to view the last 15 minutes of logs before chouser's updates. ;)
09:55cemerickchouser: I can imagine that :-)
09:56RaynesCouchDB is gigantic compared to MongoDB.
09:57RaynesI'll probably have to go with something more minimal.
10:03cemerickRaynes: gigantic?
10:04cemericklike, the size of the rpm/deb or something?
10:04Raynescemerick: Including dependencies, of course.
10:05Raynescemerick: The all-in-one installer is over 20 megs, while the binary distribution of MongoDB is less than 2, and on my barebones Ubuntu VPS, I didn't have to install any outside dependencies.
10:06RaynesI couldn't imagine convincing anybody to install a 20MB database in order to run a ~5mb IRC bot.
10:07cemerickRaynes: I'm pretty sure no one gives a moment's thought to what apt or yum installs. *shrug*
10:08Raynescemerick: I guess not, given that there are few people left on this planet that have a connection quite as slow as mine.
10:09RaynesI used the Ubuntu installer from couch.io, because I have this phobia of older versions of things.
10:14cemerickRaynes: that's a wise thing to do, especially given ubuntu/debian's tendency to screw around with packaging, often to the detriment of proper functioning of things.
10:18neilmockwhat is the proper way to use a protocol defined in another namespace? haven't had any luck with use/require/import...
10:21RaynesWhy hasn't clojure-clr moved to the Clojure organization?\
11:11shortlordhow do compare vim and emacs for clojure development? I haven't learned either one yet, but so far vim looks quite nice to me. I have read that clojure/lisp development is better in emacs though (because of better slime/swank-clojure integration) how big are the differences and when would you choose one over the other?
11:12krumholtif i have a function. can i check how many arguments it takes?
11:13Raynes&(:arglists (meta #'clojure.core/range))
11:13sexpbot⟹ ([] [end] [start end] [start end step])
11:14RaynesClojure functions can take different numbers of arguments (arities) and do different things depending on the number passed.
11:14RaynesThe arglists are stored in metadata, and that's how I got it.
11:14krumholtthanks. i can work with that
11:15Raynesshortlord: The majority of Clojure programmers use Emacs for Clojure development. SLIME is mostly unparalleled as far as a Lisp development environment goes. Emacs is also configured in a Lisp, and that is a bit of a plus for Lispers.
11:15RaynesHowever, VimClojure's support isn't bad.
11:16RaynesI'd typically say "go with whichever you're familiar with", but since you aren't familiar with either of them, you might just want to investigate a bit before making your decision.
11:16RaynesI'd focus on what the editors themselves bring to the table rather than just how good their Clojure support is.
11:17RaynesYou'll almost certainly fall in love with whichever editor you choose, and you'll want to use it for more than just Clojure.
11:17RaynesIn any case, Emacs supports SLIME, and it doesn't get much better than that for a Clojure development environment.
11:21shortlordRaynes: ok, thx for the advice. What are the distinguishing features of slime that are not available in other editors? I've read vim supports slime through slimv as well, how far superior is slime in emacs?
11:23Rayneshttp://common-lisp.net/project/slime/ Has some feature highlights and links to a couple of screencasts.
11:24RaynesAnd I haven't heard of slimv before.
11:31shortlordRaynes: ok, thx a lot. I'll look into both editors then
13:26ajazzDoes 1M mean new BigDecimal(1) ?
13:57LauJensen,(class 1M)
13:57clojurebotjava.math.BigDecimal
14:40powr-tochas anyone here used gloss?
15:05tomojpowr-toc: yes
15:36powr-toctomoj: great... I've been getting on pretty good with it, decoding a binary file format... but I'm wondering how to coerce 3 bytes into an integer
15:39powr-toci.e. 20bits
16:05powr-toclooks like BigInteger can do it
19:12ossareh'lo all
19:30tomojpowr-toc: sorry, forgot you asked that
19:30tomojdunno how to do that yet
19:30tomojI need half-byte numbers soon :/
19:30tomojare you using any finite-blocks?
20:03powr-toctomoj: it's pretty easy
20:08powr-toctomoj: not using finite-blocks yet no... but if you need nibble numbers, you can throw the byte (as a byte-array) into BigInteger and then shift left/right on it as you need
20:08powr-tocBigInteger should have almost everything you need
20:41tomojpowr-toc: thanks..
20:41tomojsucks to be using bigints for 4 bit numbers though
20:48krlany clojure emacs mode that properly indents 'extend-type' ?
21:02technomancykrl: will need a patch for that
21:02krltechnomancy is there one already?
21:08technomancynot yet
21:11krli could try it if noone else is doing it
21:12technomancythat'd be great. I don't use protocols myself.
21:18powr-toctomoj, yeah I know
21:48livingstonare there reader macros like the pipe character in lisp so that I can control the characters in a symbol name? or can I only do that with a call to symbol?
21:55krltechnomancy: turns out it was pretty simple: https://github.com/krl/clojure-mode/commit/6cf4f64de92bf2da18d9c8adb7c05cb088d60954
22:00technomancykrl: hm; that seems to be branched from jochu's, which is out of date
22:00technomancyit looks like my fork actually has (put 'extend-type 'clojure-backtracking-indent '(4 (2)))
22:00technomancywhat does the additional 4 signify?
22:00krli'm not sure, it's not documented
22:01krli used the one from proxy
22:01krlbut which are the newest forks?
22:02lghtngclorqle!
22:03krlok looking at the github fork tree
22:07technomancykrl: jochu disappeared over a year ago, so I took over clojure-mode and swank
22:07technomancyI am not very familiar with how indentation works though.
22:09technomancyI guess I should add some message to jochu's that makes it clearer =\
22:11krltechnomancy: ok yeah your fork worked directly too :) jochus version still has better google pagerank
22:12joshua__How would you get the result of a post request with enlive or clojure in general?
22:12technomancypagerank is always throwing people off with outdated material
22:12krlguess it's because it's still being linked to?
22:12technomancykrl: if you install via package.el you'll be sure to get the latest though
22:13krlok, never got around to using elpa
22:13technomancyI don't think any new links are being created for it; google is just not very well-calibrated for high-churn domains like Clojure
22:22joshua__clojure-contrib has http-agent apparently
22:22joshua__it looks like it will work
22:30joshua__Does clojure http-agent actually work?
22:33dnolenjoshua__: I use https://github.com/rnewman/clj-apache-http
22:33dnolenjoshua__: there's also https://github.com/clj-sys/clj-http, haven't used that myself tho.
22:34joshua__dnolen: Copying the examples given in the http-agent documentation crashes slime for me.
22:34joshua__dnolen: so thanks for the alternatives =)
22:50tomoj,(->> (fn [] (future (let [t (Thread/currentThread)] [(.getId t) (.getName t)]))) (repeatedly 10000) (map deref) set)
22:50clojurebot#{[359 "pool-2-thread-4"] [360 "pool-2-thread-5"]}
22:50tomojhow exactly are futures executed?
22:50tomojI see they use soloExecutor from Agent, guess I need to read those javadocs
22:56tomoj,(time (->> (fn [] (future (let [t (Thread/currentThread)] (Thread/sleep 10) [(.getId t) (.getName t)]))) (repeatedly 100) (map deref) set))
22:56clojurebot"Elapsed time: 1100.567 msecs"
22:56clojurebot#{[362 "pool-2-thread-6"]}
22:56tomojwhy?
22:56clojurebothttp://clojure.org/rationale
22:57tomojoh
22:57tomojof course
23:03Guest32627If you have a big pile o' calculations to do, there is no advantage to dividing them up between multiple agents vs having one agent perform them, right? Won't you still end up with an unbounded pool of threads working on (send-off) or a fixed pool working on (send)?
23:11qbgGuest32627: If they go to one agent, they will be done in a serial order
23:12Guest32627qbg: not true. multiple threads will be working the queue concurrently
23:13qbgFunction #'send to an agent are queued up
23:13qbg*Functions
23:13Guest32627qbg: yes they are queued up to be handled by the thread pool assigned to the agent
23:13Guest32627qbg: if you use "send-off" the thread pool is unbounded
23:13qbgThey will still happen serially because the result of the previous function is feed to the next one
23:14Guest32627qbg: that is absolutely not true at all
23:14Guest32627qbg: the result of the function merely sets the current agent state
23:14qbgWhich will be feed into the next function
23:14Guest32627qbg: that is false
23:15Guest32627qbg: i'm sorry, i mean to say it definitely does NOT go to the very next item on the queue, because they will very likely go out of order if you put a lot of i/o messages in the queue, for example
23:16Guest32627the state just becomes whatever the return is from the current thread that finished running your function
23:16Guest32627the items in the queue can be processed out of order, and will be, with slow i/o handling on an unbounded thread pool
23:20technomancyGuest32627: if you have a pile of calculations that aren't meant to be a series of computations against a single identity, you shouldn't be using agents at all
23:20Guest32627technomancy: that's a very good point and i agree
23:21Guest32627technomancy: so let's assume they are not against a single entity
23:21technomancyif it's possible to split your work among multiple agents, then it's likely your work would be better done using futures
23:22Guest32627technomancy: so my suspicion is right that there is no point to doing such work with multiple agents? for example, i can see using an agent for cpu-bound calcs and another for some sort of i/o like logging, but not multiple agents for the same thing
23:22Guest32627is that the right idea?
23:23technomancyGuest32627: the point of agents is that they are a single reference type that can be used to model a given entity that has many values over a period of time
23:23qbgOnly one function will be ran on an agent at a time
23:23Guest32627technomancy: that is true but insufficient. the asynchronous nature of agents is also a key feature. what you describe holds equally well for atoms and refs
23:24qbgAnd they will happen in order
23:24qbghttp://clojure.org/agents
23:24Guest32627qbg, that is false
23:24Guest32627qbg: read the part about thread pools
23:24technomancyGuest32627: sure, but if you're just concerned with asynchronicity, just use a thread or thread pool on its own
23:24qbg"At any point in time, at most one action for each Agent is being executed."
23:24qbg"Actions dispatched to an agent from another single agent or thread will occur in the order they were sent, potentially interleaved with actions dispatched to the same agent from other sources."
23:26Guest32627qbg: then what *exactly* is that unbounded thread pool doing that is handling (send-off)?
23:26qbgAll sends are sent to a bounded thread pool
23:26qbgSo a send that is io-bound will consume resources from other agents
23:26Guest32627qbg: ergo more than 1 thread (# cpu + 1) but send-OFF is unbounded
23:26qbgsend-off uses an unbounded thread pool so other agents will not be starved
23:28Guest32627qbg: so you mean to say you need to create as many agents as you would like to have potentially handling messages?
23:28Guest32627qbg: if i want to fan out to, say, 100 threads, i need to create 100 agents?
23:28qbgYes
23:29Guest32627qbg: ahh... my sincere apologies then! this was not clear from joy of clojure
23:29qbgAgents are likely the wrong tool for what you want to do though
23:30Guest32627qbg: what if i want to asynchronously handle responses in a server app? is that a good use for them?
23:31tomojagents appear nowhere in aleph/lamina, interesting
23:31qbgIf it is simpler to just use a thread, using an agent is likely the wrong thing to do.
23:34qbgThe only thing that I have written that was a good fit for agents was a parking lot simulator
23:34Guest32627qbg: it would be pretty simple to cycle through some max number of agents though in order to bound the upper size on your thread pool
23:34Guest32627qbg: rather than managing some thread pool yourself. why bother when you get it for free?
23:35qbgagents carry state; if you don't need that state, using agents is probably overkill
23:35Guest32627qbg: the example i'm looking at now in Joy of Clojure uses that state as merely a counter for logged messages
23:35qbgjava.util.concurrent gives you easy to use thread pools also
23:36Guest32627qbg: sure but why drop down to that if you don't need to. i get the same functionality by staying entirely within clojure itself
23:36Guest32627qbg: oops, that was a question :)
23:37qbgTastes vary. I don't like to abuse agents/futures in my code for their side effects.
23:39Guest32627qbg: i don't see how leveraging asynchronous processing in an agent is an abuse if it's a fundamental requirement. you mean just because the inherent state of an agent is not terribly important? is that what you mean by abuse?
23:40qbgIf the functions you send to an agent ignore their first argument, you are probably abusing the agent system
23:41qbgLikewise, if you don't care about the return value of a future, why not use a thread instead?
23:41Guest32627qbg: ok, fair enough. joy of clojure is definitely abusing agents then in their example :)
23:42qbgA counter would be fair use
23:42Guest32627qbg: for log messages? he doesn't even use it in the example
23:42qbgA fair use of agents that ignore their first argument is for logging because they integrate with the STM
23:46Guest32627qbg: that's actually kind of interesting when you think about it. because if the transaction fails, you get no log either :)
23:47qbgAnd if it retries but eventually succeeds, you get only one
23:48Guest32627but only if it succeeds. not if there is an uncaught exception, for example
23:48Guest32627well i guess you could have an error handler do some magic at that opint
23:49Guest32627maybe this is why it's so hard to find clear, convincing uses of agents.
23:50Guest32627i've only heard how they are NOT good for various uses. not what they are good for (only a parking lot simulator)
23:50Guest32627sounds like a worthless feature tehn
23:51qbgThe integration with the STM is a good thing
23:52qbgThey are also good for autonomous identities
23:54qbgActually, agents are rather close to smalltalk style OO
23:55Guest32627qbg: but if they aren't doing some work on your behalf then their autonomy and integration with the STM isn't of much value
23:56Guest32627qbg: i was thinking of them as more like MDB, where the asynchronous processing is the key feature. you seem to think that's not too important compared to the state that the agent holds
23:56Guest32627qbg: i can see your point but i'm struggling to think of any good use for them at all. ok, so i can log with them. i'm sort of underwhelmed
23:57qbgI think you are underwhelmed because there aren't many good uses for object either
23:58qbg*objects
23:58Guest32627qbg: when you put it in those terms, sure. i agree! however there are very many good uses indeed for asynchronous processing
23:58Guest32627qbg: but for that you're saying "well yes, but just use regular java threads and don't bother with agents"
23:58qbgIf you need async + state, then agents work. If you only need one of them there are probably better choices
23:59Guest32627you don't need async + state for logging either
23:59qbgAgents are useful there as a side effect of their interaction with the STM