#clojure logs

2016-02-29

03:16prohobo666 nicks..
03:56jonathanjare there any clojure libraries to *produce* markdown from a clojure datastructure?
03:57jonathanjprogrammatically producing markdown is a pretty unpleasant experience, i was hoping to not write code to do it
04:00J_Arcanejonathanj: huh. I don't seem to be able to find one.
04:00J_ArcaneMayhaps that'd be a good library project.
04:01J_ArcaneI've tinkered with some markdown generation in the past (my resume used to be generated from a custom MD generator in Racket). How would that look though?
04:24jonathanjJ_Arcane: I dunno, something like hiccup maybe?
04:25J_ArcaneYeah. I was thinking that, maybe.
04:25J_ArcaneLast time I did it was with composable functions.
04:25J_ArcaneThat just all resolved down into strings of MD-formatted text.
04:26amalloywhy do you want to generate markdown? markdown is sorta intended to be human-editable source that turns into html
04:26jonathanjamalloy: sure, tell that to like everything on the planet
04:26amalloyhuh?
04:26jonathanjamalloy: for example, Slack's "rich" content is some subset of Markdown
04:27jonathanjwhich is how you interact with it both as a human and as a machine
04:27J_Arcaneamalloy: Well, in my last use case, I had a set of data (resume entries) that I wanted in Markdown so I could just host it on Github, which automatically gives you a rendered view of any .md file.
04:28dacechavezI’m modeling a simple friending app and a user is a hash-map {:name “John” :friends []}. How should I add a friend to John’s vector of friends? By using swap! on an atom or by making a pure function that returns a new hash-map with a friend added to the vector?
04:29J_Arcanedacechavez: That would rather depend I think on where the friend lives? Ie. is it being passed as a value, or kept in an atom?
04:29amalloydacechavez: i mean, the former is something you do in additino to the latter
04:29amalloyyou can't really get away from writing that pure function; once you have, you can optionally use it as an atom's swap! function
04:30jonathanjJ_Arcane: with the bunch of functions approach, it seems pretty hard to know when you can't do something, since the context is encoded in the markdown string you're building instead of the data you're building the string from
04:31jonathanjlike when some syntax cannot be present inside some other syntax
04:32dacechavezhmm Ok I see..
04:33jonathanjas a more general complaint, i don't think anyone has solved this problem: html is way too general for most purposes, so you have to build in all kinds of white/black listing to avoid XSS etc; markdown (and most other human-readable markups is awful to produce programmatically
04:39dacechavezJ_Arcane: i don't know.. I kept it as a value until I started testing the add-friend part, where I tried to conj something to the vector but data is immutable obviously. That's when I thought maybe I need to wrap the hash-map in an atom..(?)
05:04Kneiva_,(update {:name "John" :friends []} :friends #(conj % "Bob"))
05:04clojurebot{:name "John", :friends ["Bob"]}
05:12BronsaKneiva_: you can just do:
05:12Bronsa,(update {:name "john" :friends []} :friends conj "bob")
05:12clojurebot{:name "john", :friends ["bob"]}
05:13Kneiva_cool
05:28J_Arcane,(update-in {:name "John" :friends []} [:friends] conj "Steve")
05:28clojurebot{:name "John", :friends ["Steve"]}
05:28J_Arcanedacechavez: Like that?
05:29dacechavezJ_Arcane: Yea I guess. I had this urge to change the original hash-map but maybe thats not the Clojure-way
06:12kungiI f* love clojure
06:12kungiImplementing a "mocking framework" in 33 lines of code ... :-D
06:13mokuso:)
06:16tdammersplot twist: line 33 is this: ))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
06:16kungitdammers: :-) No actually it's )))
06:17kungihttps://gist.github.com/Kungi/1e1b396db629272749b6
06:25jonathanjanyone have opinions on truss?
06:25jonathanjhttps://github.com/ptaoussanis/truss
07:32benjyz1Hi, I'm using timbre for logging. any idea for how to pipe logs from libraries?
07:32benjyz1jetty webserver spits out a lot of lines and I want to filter them
07:33benjyz1I guess .. :ns-whitelist []
08:27kungiWow the cider debugger is great
10:00shemkungi: yes it is! just today i was stepping all over my functions just for the joy of it
10:30justin_smithbenjyz1: the problem with ns-whitelist is that jetty doesn't have any namespaces in it
10:30justin_smithit has packages and classes...
10:30benjyz1trying to filter the logging output. harder than I thought
10:30justin_smithbenjyz1: or is :ns-whitelist just a weird name and it can manage output from java crap too?
10:31benjyz1I have logback and XML filters, but that's just painful
10:31justin_smithbenjyz1: the jvm logging expects filtering by classes / packages right?
10:31benjyz1don't know. log4j and logback are absurdly complex
10:32benjyz1I mean in logback you can write filters which reference classes ... ha :D
10:32tdammersaverage java applications log absurd amounts of uncaught exceptions, too
10:32benjyz1so XML => class filter.. genius
10:40jonathanjis there a name for (first (filter pred xs))?
10:40justin_smithjonathanj: that should be a FAQ if we had a FAQ and the answer is no, not in core
10:41jonathanjwhat does one normally call that?
10:43jonathanj,(some even? [1 1 nil 2 4])
10:43clojurebot#error {\n :cause "Argument must be an integer: "\n :via\n [{:type java.lang.IllegalArgumentException\n :message "Argument must be an integer: "\n :at [clojure.core$even_QMARK_ invokeStatic "core.clj" 1372]}]\n :trace\n [[clojure.core$even_QMARK_ invokeStatic "core.clj" 1372]\n [clojure.core$even_QMARK_ invoke "core.clj" 1367]\n [clojure.core$some invokeStatic "core.clj" 2592]\n [clojure.co...
10:44jonathanjnot quite.
10:54Rick77Hi! Anyone knows how to declare static fields with gen-class? I know how to declare static (or not static) methods, but I found no information about the fields...
11:05Rick77never mind, found a simpler solution involving leiningen and java-source-path: ;-)
11:16justin_smithjonathanj: some is different in that it returns what the predicate returns
11:17jonathanjjustin_smith: true
13:19puredangerjustin_smith: btw, there is now a faq: http://clojure.org/guides/faq
13:19puredangerand would be happy to have a PRs expanding it on https://github.com/clojure/clojure-site
13:35jonathanjnice FAQ
13:46justin_smithpuredanger: oh cool, I was thinking a FAQ for the IRC channel (which would be different in tone / target than the clojure.org FAQ maybe?) but good to know the clojure.org FAQ is there and accepts PRs, thanks
14:18jweissis there a library that tracks changes of data (branching, merging etc) basically the way git does with files?
14:18justin_smithjweiss: not that I know of, though clojure.data.diff/diff is cool
14:18justin_smith,(require 'clojure.data.diff)
14:19clojurebot#error {\n :cause "Could not locate clojure/data/diff__init.class or clojure/data/diff.clj on classpath."\n :via\n [{:type java.io.FileNotFoundException\n :message "Could not locate clojure/data/diff__init.class or clojure/data/diff.clj on classpath."\n :at [clojure.lang.RT load "RT.java" 456]}]\n :trace\n [[clojure.lang.RT load "RT.java" 456]\n [clojure.lang.RT load "RT.java" 419]\n [clojur...
14:19justin_smitherr
14:19jweissjustin_smith: yeah, i've used that a bit before to diff changes using watchers. afaict there's no branching or merging though
14:19justin_smithright, right
14:20justin_smith,(require 'clojure.data)
14:20clojurebotnil
14:20justin_smith,(clojure.data/diff {:a 0 :b 1 :c 2} {:a 1 :b 2 :c 2})
14:20clojurebot({:b 1, :a 0} {:b 2, :a 1} {:c 2})
14:20jweisscould definitely use clojure.data/diff to help write a merge function though
14:21justin_smithyeah, I imagine a thing that only looks at top level differences and then recurs on changed subtrees
14:21justin_smithbut then maybe its easier to do your own one level diff at that point
14:23jweissi think i'd have to make a graph (using loom or something) and traverse it up to the common ancestor, apply all the diffs of each branch after that (my merge conflict resolution is much simpler than git's)
14:24jweissthe only conflict it would resolve is both branches adding different items (they'd be merged like a map does it)
14:25justin_smithjweiss: why walk back up again when clojure does structural equality?
14:25justin_smithand might even be using the same underlying k/v pair to represent that part of the data if they came from the same origin
14:27jweissjustin_smith: hm. i had planned on having each node contain changes to the ancestor node, but i may be able to just store the resulting value and let clojure's persistent data structures deal with the memory management
14:28justin_smithoh, OK, I guess I misunderstood your plan... interesting
14:31jweissjustin_smith: it may be as simple as writing a merge function that takes the common ancestor value, and both branch values, and just returns the merged value. i can't think of any situation where not knowing the intervening history would be a problem.
14:41justin_smithjweiss: in that case you might even be able to do some magic with (or variation on) the existing merge-with
15:35sdegutisYou know how sometimes you end up with a chain of data processing using ->> ?
15:36sdegutisWell, I've got one of those, and I'm trying to figure out a nice way of accessing the data in the middle of the chain while still using it.
15:36sdegutisNormally I'd split the chain into two chains, the first one bound to a variable-name inside a (let), so that I can just access it via that variable.
15:37sdegutisBut I'm betting there's probably a smarter way.
15:38sdegutisWell I guess another way is to extract a custom function that wraps the thing I'm trying to do. But that's just an extended version of the (let) solution.
15:41tolstoyWould the as-> macro help?
15:42kenrestivohow did people check for exceptions in clojure.test prior to 1.8?
15:42kenrestivothere's no thrown? in 1.7 apparently
15:42sdegutistolstoy: that was my first though, but it's not useful in a ->> chain because the argument goes at the end whereas as-> needs it at the beginning.
15:42sdegutistolstoy: in fact Clojure doesn't have an elegant way of accessing or destructuring the "last" argument, whereas it does have it for the first.
15:43tolstoyOh, I thought you could put "name" in any position of subsequent expressions.
15:43sdegutiskenrestivo: I thought it was?
15:43lumakenrestivo, it's been there always
15:43kenrestivoreally?
15:44kenrestivoclojure.lang.Compiler$CompilerException: java.lang.RuntimeException: No such var: clojure.test/thrown?, compiling:(/tmp/form-init2006229078634769762.clj:1:680)
15:44kenrestivo,(use clojure.test)
15:44lumawell, it has never been a var, the is macro just checks if there's a symbol 'thrown? in there
15:44clojurebot#error {\n :cause "clojure.test"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.ClassNotFoundException: clojure.test, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6688]}\n {:type java.lang.ClassNotFoundException\n :message "clojure.test"\n :at [java.net.URLClassLoader$1 run "URLClassLoader.java" 366]}]\n :trace\n [[ja...
15:45kenrestivooh
15:46amalloykenrestivo: there was never a thrown? in any version
15:46amalloynot as a var that exists
15:46amalloyrather, it's a symbol that's recognized by the 'is macro
15:46kenrestivoamalloy: luma pointed out it's not a var, it's just a symbol the macro checks for
15:47amalloyoh, i see. i missed it between your clojurebot chatting
15:47kenrestivosorry for the noize
15:47amalloynote though that it's not actually a special symbol that's hard-coded into 'is: there's a multimethod used by 'is, and 'thrown? is one of the methods
15:47amalloyyou can add your own if you want
15:47kenrestivodispatches on it? great, thanks.
15:48kenrestivoit's fine, just trying to check for a specific exception in a test case.
15:57Shayanjmjustin_smith: when using clj-kafka, how do you test your producer-consumer interactions? It feels like I almost need to have 2 REPLs going since the data is streamed...
15:57ShayanjmI seem to be able to produce just fine, but i'm wrestling with the consumer
16:05justin_smithShayanjm: what kind of managing do you need between your producer and consumer?
16:06Shayanjmjustin_smith: for right now - I'm just trying to test that I can produce messages that my consumer can then parse into some sort of data structure
16:06justin_smithShayanjm: my consumer just runs in a thread, always looping and looking for the next message
16:06justin_smithso you could set this up via core.async (what I used) or just threads and queues
16:07justin_smithShayanjm: but you can also produce, and then consume, one after the other in a repl
16:07justin_smiththat should work with what you are describing just fine
16:07Shayanjmjustin_smith: I actually tried that and didn't get the result I was expecting
16:07Shayanjmlet me try again
16:08justin_smithShayanjm: what I found useful was first figuring out how to make everything work with kafka-console-producer.sh and kafka-console-consumer.sh
16:08justin_smitheg. just the two of those interacting
16:08Shayanjmjustin_smith: Sure, I've got those two interacting nicely
16:08justin_smithand then after that I introduced my clojure stuff on oneside, leaving the shell one on the other end
16:08ShayanjmYeah that's exactly my approach. I've been producing via clojure & consuming via shell
16:09justin_smiththat isolates the errors (did I produce wrong? consume wrong?)
16:09Shayanjmlet me try doing one after the other
16:09justin_smithShayanjm: the most likely error with producing then consuming in one repl is that you set your offset to latest, which explicitly ignores messages from before the start of your consumer, and is usually a default
16:10Shayanjmahhhhhh
16:10justin_smithinstead of latest you can ask for "oldest"
16:10justin_smithiirc
16:10Shayanjmso in that case it's best practice to create a consumer, emit messages, and then (def msgs (messages c "topic"))?
16:10justin_smithShayanjm: well, that would be one way to do it, sure
16:10justin_smithor start the consumer in a loop on a thread putting messages in a data structure
16:10justin_smithwhatever makes sense
16:12Shayanjmjustin_smith: is there a 'best' way to start an ever-consuming background process using core.async?
16:13Shayanjmone that I can preferably manage (start/stop at will)?
16:17justin_smithunless you are already a core.async user, it's likely easier to loop in a future, reading and putting the result in a java.util.concurrent.ArrayBlockingQueue
16:18justin_smithand then if you have a handle to the future, you can use future-cancel and the wait to read from kafka allows that cancellation (it's one of the small set of things future-cancel reliably halts)
16:19justin_smithwith core.async this would be done with alts! and a cancellation channel but no need to go into that if you aren't already using that tool here
16:28Shayanjmawesome thanks! I'm going to play around with it a bit
16:29justin_smithShayanjm: also for testing you could just open two repl connections to the same lein repl actually
16:29justin_smithso you don't need two actual instances of your app running, just two repl connections, one reading and the other producing
17:08dbaschthis is not a clojure question, but please indulge me. I need to give a 20-min talk at work titled “What is functional programming” and the audience is mostly junior devs, many of them without any formal CS training (coding school, Ruby, js). I want to make it fun and interesting, looking for ideas about how to start with a good hook.
17:10amalloydbasch: https://docs.google.com/presentation/d/1mGihUBBIKMQn5Uz-5DvQ1Vu24qSuk3o4yUoPoBZW3tI/edit is a presentation i put together that you might steal from
17:11dbasch(inc amalloy) ;; if this still works
17:12dbaschI was thinking about going “functional?” “programming?”, then going into the mathematical concept of a function, then asking “ok, but what does this have to do with programming”
17:13amalloythat probably takes longer than 20 minutes, especially for junior devs without the theoretical background
17:13amalloyi think i gave this talk in 35-40 minutes, but you could easily cut out half of it
17:14dbaschyeah, I don’t think I want to spend much time on laziness and concurrency. I have to mention them, but it would be too much.
17:14amalloyincidentally, https://cdsmith.wordpress.com/2012/04/18/why-do-monads-matter/ is a cool article on monads that follows the approach you mention
17:19al80what is the best way to do TDD (London style) with Clojure?
17:20al80clojure.test is it ok? Is there a lib that provide mocking, expectations?
17:20al80and is everything runnable from a repl?
18:07justin_smithal80 left but clojure has mocking via with-redefs, and clojure.test works great, though refactoring code slightly for testability is often win-win regardless of which testing setup you use
18:07justin_smithand yes it works from a repl
18:59pilnenow to convince the friend whose project i committed to that clojure and clojurescript are far superior to java for the jvm... *twirls my non-existant villan mustache
19:01pilne(java and javascript, lol... too busy plotting to be smart i guess... lol)
20:06simplesamI have a java class that has a public interface: public class AA { public interface ILog { void log(String logStr); }}
20:06simplesamhow can I implement ILog in Clojure ?
20:06amalloy(reify AA$ILog ...)
20:07justin_smithsimplesam: the jvm doesn't actually have nested classes or interfaces, it just creates a silly name in the bytecode
20:07justin_smiththe name amalloy mentioned
20:07simplesami've tried using deftype without success.
20:07simplesamah ok. thanks. i'll give that a go
20:16cortexmanblow my mind