* ClojureScript * Rationale ** What *** Compiler that compiles (a subset of?) Clojure to Javascript ** Why? *** Because js is the only universally available client target **** and can't be supplanted due to its browser integration and installed base **** yet isn't very good, expressive, concise, or robust **** but JS engines continue to get lots of optimization love, and are now quite capable perf-wise *** Use same skillset and libs for client and server **** only other similar options are: ***** JS native client, JS (e.g. node) server ****** node still much less powerful than JVM, and might be a mere fad ***** Java (GWT) client, Java native server ****** GWT has lots of baggage due to semantic mismatches etc ****** but familiar tooling if already a Java dev ***** esoteric, open question as to skills and libs: ****** [[http://clamato.net/][Clamato Smalltalk]] ****** [[http://www.wescheme.org/][Moby Scheme]] *** Clojure semantics can fit well on JS **** i.e. defn/type/protocol won't fight with js core model *** ClojureJS arguably becomes most powerful lang on client **** Robust, simple Clojure model **** Macros etc *** Might be best way to run Clojure on mobile *** js a possible path to delivering Clojure libs to C-linkage clients **** via embedded Google V8 js engine **** somewhat speculative, but considering for M **** V8 wrappers exist for Python, Ruby, PHP etc ** How? *** Compiler written in Clojure, generates (readable?) JS *** Optionally run that JS through Google Closure js->js compiler **** for minification, dead code elimination etc *** Use Google Closure js library where needed for implementation support **** e.g. goog.math has Long arithmetic **** module system **** use gclosure annotations for stronger type checking or better code gen? **** dependency system *** Macros written in Clojure proper, run by the compiler **** stick to subset in macros if eval supported *** Any runtime support written completely in itself **** make deftype and protocols work early ** Non-objectives *** complete Clojure **** feel free to subset, especially at first **** but try to make anything present in both work identically *** compiling e.g. core.clj as-is **** don't want to touch Clojure itself for this **** bootstrap will differ anyway ** Ancillary benefits *** Analysis component of compiler might serve Clojure-in-Clojure, or other tooling **** maybe - we'll need far less analysis support in js than we do in Java *** Boost for Clojure adoption due to increased reach *** Power tool for exploring next-gen client approach * Implementation ** Primitives *** DONE def *** fn* **** DONE basics **** DONE recur **** DONE variable arity **** DONE arity overloading **** closures shouldn't map directly to js closures? - no, they should ***** they capture entire surrounding environment ****** hearsay, V8 already better ***** premature optimization to avoid that? - yes ***** shouldn't js engines do that for us? - yes ***** try goog.partial? - not for this **** variable arity how? ***** switch on arguments.length *** DONE if **** need to match Clojure semantics (nil/false) ***** must deal with undefined (group with nil/false?) *** DONE let* **** must fix local scoping problem ***** nested fns or renaming? **** let* semantics *** DONE do **** as with Java, not an expression **** doFn(...) -> returns last arg ***** must alloc array for arguments? *** TODO global **** use 'var for this? ***** already Clojure special op ***** but wrong semantics, (var x) is *ns*-relative ***** no true unqualified globals in Clojure *** DONE recur **** to loop **** to fn head ***** can't do in single pass *** DONE invoke *** DONE macros *** DONE ns **** (ns my.ns (:require your.ns ...) (:macros a-clojure-ns ...)) ***** aliases? **** => ***** make a clojure ns? cljs.my.ns? ***** goog.provide('my.ns'); goog.require('your.ns'); ***** ( *** DONE deftype* **** maps to prototype/contructor combo **** deftype macro is deftype* + extend-type ***** extend-type with ::fields meta on param vectors *** TODO reify* **** yes, for one-off protocol impls **** no ctor created, just put impls on object ***** can share code with putting impls on prototype? *** DONE defprotocol* **** not primitive in Clojure proper **** extend, when given ctor, modifies prototype with slot ***** slot is ns-qualified ***** what about core prototypes - Object, Array et al (String, Number, Boolean, Function)? ****** poor citizenship to modify these? ****** Object different in that it is used as map ****** nested window scope issues? **** protocol fns just turn (my.ns/foo x a b c) into x["my.ns/foo"](a, b ,c) ***** foo(x, a, b, c) ****** must pass target ***** better - x.my$ns$foo(a, b ,c) ****** can be minified *** DONE extend-type *** defrecord? **** any way to get (:foo x) => x.foo? ***** beware GClojure renaming *** DONE new **** what to do? ordinary invoke works fine ***** new could be aliased, not special form then ***** not ordinary - first arg not evaluated ****** but should be in JS since new is an operator on a function, not a name **** new itself shouldn't be evaluated, won't pass fnOf **** (my.ns.Blah. x y z) - just macroexpander stuff **** (Blah. x y z) - requires import and registry ***** class aliases a bigger issue, will there be more conflicts? ***** any interpretation will fit only one ns strategy (e.g. gclosure's, and thus ClojureScript's) ***** start without this *** DONE dot **** field/zero-arg-method distinguished how? ***** not, just support scoped var and be done *** DONE set! (assign) **** same binding rules? ***** no **** or just allow assign to scoped 'vars'? *** DONE name munging **** special chars **** js reserved words *** DONE (js* code-string) **** with name escaping *** TODO exceptions **** throw **** try **** catch ***** won't have exception type **** finally *** quote? *** TODO Evaluated collections **** Map **** Vector *** vars? *** case? *** callable non-function types? **** seems not possible portably **** could do with __proto__ (non-standard, all but IE support, even IE9 doesn't) **** how would Clojure feel without callable collections and keywords? **** could do with conditional every invocation: ***** (f instanceof Function?f:f.cljs_lang_invoke)(args) ***** but where to put f (in expr context)? ****** needs helper fn ****** fnOf(f)(args) ******* function fnOf(x){return (f instanceof Function?f:f.cljs_lang_invoke);} ****** i.e. every call is 2 calls ******* tracing jit will inline? ** Translation | Op | JS | Notes | Questions | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (def x 42) | cljs.my.ns['x'] = 42 | Following gclosure module system | No vars? Compilation-time representation of ns? | | | cljs.my.ns.x = 42 | only this one will get minified | but this precludes special chars in names | | | | | def returns var in Clojure, no var here | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (fn [x y] ...) | (function (x, y) {...}) | never do named function, use anon + def | Use for closures too? | | (fn [x y] ... (recur...) | | rewrite as fn + nested loop | require analysis to transmit recur fact up | | | | | rewrite when? | | | | block always in return context | access to this for methods? | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (if test then else) | (test ? then : else) | | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (do e1 e2 e3) | cljs.dofn(e1,e2,e3) | dofn returns last arg, allocs array? | requires js bootstrap file? | | | | no, forces all to be exprs | no fn needed when not expr context | | | (function () {e1;e2;return e3;})() | | | | | | expr context becomes return except when | | | | | single expr | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (let [x 1 y 2] ...) | (function [x,y] {...})(1, 2) | need to create nested functions for let* | how to detect ref to earlier? | | | var x__42 = 1;var y__43 = 2; ... | var numbering | statement/expr dichotomy if inline? | | | (function [] | could wrap in no-arg function always | needed for expr anyhow | | | {var x = 1; var y = 2; ...})() | if always wrapped, don't need numbers? | can we do var x = 42; var x = 43? | | | | might still when nested | yes, but not var x = 42 ...nesting... var x = x | | | | | | | | | expr always becomes return context | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (. x y) | x.y or x.y()? | no type info to distinguish | bigger problem, both calling and retrieving | | | | | fn in slot are viable, Clojure says method wins | | (. x y ...) | x.y(...) | | | | | | | | | (: x y) ? | x.y | | make all calls, add special field accessor | | x.y | x.y | . not used for classes in JS | so not global, but scoped? | | | | can't test from Clojure | but would want resolution of first segment to locals | | | | | what do macros use? | | | | | | | (. x (y)) | already defined for this case | wasn't going to carry this into cljs, but | no arg == field, penalize no-arg methods? | | ((. x y)) | more correct, it's a slot | | rationale, it's not a method, just a slot, | | (-> (. x y) ()) | doesn't currently work, could | | but then why do the arg-taking ones work? | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (set! (. x y) 42) | x.y = 42 | | whither vars and binding?? | | (set! some.global.x 42) | some.global.x = 42 | | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (loop [bindings] | while(true){ | | wrap in function? depends on context | | ... (recur)) | ... rebind-continue | | | | | ret=xxx;break;} | | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (deftype Foo [a b c]) | my.ns.Foo = function(a,b,c) | turn inline defs into explicit extends? | deftype inline methods split out arities | | | {this.a = a;...this.c=c;} | can't access this and fields. | | | | | in locals map, bind a to this.a etc | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (new Foo 1 2 3) | (new Foo(1,2,3)) | | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (defprotocol P | my.ns.foo = function(obj args) | | How to extend built-ins, default, nil, undefined | | (foo [args])) | {obj['my.ns.foo'](obj, args);} | can't minify | | | | | | | | | obj.my$ns$foo(obj, args) | | | | | P.ns = 'my.ns' | this only compile-time need, but compiler | | | | | not in js world, can't see it | | | | | Require fully qualified protocol names? | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (extend Foo my.ns.P | for each fn in map: | if no reified protocols, extend can't be | or use Object.defineProperty to add method to | | {:foo (fn [foo]...)} | Foo.prototype['my.ns.foo'] = fn | a function, unless protocol quoted | prototype? can then set enumerable to false | | | Foo.prototype.my$ns$foo = fn | or string | | | | | if extend is a macro or special, could | | | | | still evaluate fn map, but then can't be | | | | | minified | | | | | evaluated extend requires maps, keywords | | | | | high bar for bootstrap if protocols | | | | | at bottom - extend* unevaluated? | | | | | make extend-type primitive instead? YES | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | constants | | | | | nil | null | | | | "foo", true, false, 42.0 | same | | | | 42 | goog.Long? | | | | 'foo | symbol ctor | | | | :foo | ? | | how to do keyword interning? | | | | | don't want intern every reference | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| | (ns my.ns | | | | | (:require [foo.bar :as fb]...) | | | | | (:macros [my.macros :as mm]...)) | :require-macros? | | | |----------------------------------+------------------------------------+-------------------------------------------+------------------------------------------------------| ** Library *** persistent data structures? **** make base literals create JS base literals? (array, object-as-map) ***** seems a big waste not to leverage js optimization of dynamic properties ****** or, that's what deftype is about, maps have always added overhead ***** we care more about accessors than assignment/modification ****** i.e. we will superimpose copy-on-write ***** string/keyword problem ****** can make {:a 1 :b 2 :c 3} => {a: 1, b: 2, c: 3} ****** but (keys that) => ["a" "b" "c"] ****** could use internal array of keys trick ******* keys used as strings in property map ******* kept intact in internal arrays, which is what is returned by keys fn ******* means keys must be string distinct - ick **** promote only on conj? ***** or on size as well? ** Questions *** equality and hashing *** undefined **** turn into nil? **** can't catch everywhere *** vars **** def should create slots in global ns objects? **** what var semantics matter? *** keywords and symbols **** make separate object types? ***** not many symbols make it into runtime use, but keywords do **** need to make sure {:key val} and (:key obj) are fast **** native maps can have only string keys *** metadata **** just claim a slot? *** namespaces **** tie into gclosure module system? **** compile-time enumerability? *** eval **** runtime compiler? **** would let you develop in the browser, repl etc **** means compiler must self-host ***** and needs runtime reader ***** and syntax-quote **** no macros? ***** can't do without, as so many basic things are macros **** won't have google closure compiler there ***** ok, shouldn't rely on that *** laziness **** not a great fit **** GC probably not as good **** unlikely to be working with bigger-than-memory **** non-lazy mapping/filtering or mapv, filterv ***** can make it back into Clojure *** Immutability **** enforced? ***** or just use safe lib fns to avoid ****** lets you use base types ****** no final on which to base it anyway ****** would need fancy encapsulation techniques ******* how fancy? ***** correct (non-assigning) code does the same thing, but incorrect not caught ****** fair compromise? **** gclosure compiler can do some enforcement ***** given some const hints in comment-based annotations *** Interactive development **** REPL ***** easiest: ***** Clojure read -> cljs analyze -> cljs emits -> embedded Rhino eval+print **** Incompatible constructs ***** for host interop ****** preclude development in Clojure **** Missing JS things ***** e.g. DOM etc ***** headless JS environment with DOM mocks? * Namespaces and macros ** Macro problem? *** syntax quote in the reader uses Clojure namespaces *** hardwired to Compiler.currentNS().getMapping() Compiler.resolveSymbol(), isSpecial() etc *** ::keyword resolution uses Compiler.currentNS(), namespaceFor() *** if it expands to calls to other macros, those need to be in clojure-land **** maybe - they need to be the cljs-compatible versions ***** argument for calling core clojure.core in cljs too? ****** but we can't have 2 clojure.core namespaces during compilation ***** translate/consider clojure.core/xxx => cljs.core/xxx during cljs compilation? ****** doesn't work if we have separate cljs.macros ******* some core things will be in cljs.core, some in cljs.macros ******* put core macros on cljs/core.clj ******* other core code in cljs/core.cljs ******* both contain ns declarations - ok? *** but expansions destined for cljs compilation need to be resolved in cljs-land **** dummy vars in dummy namespaces? ***** no - doesn't cover external nses, cljs aliases ***** just fully qualify everything non-core in macroexpansions **** different specials set a problem ***** e.g. global, ns, defprotocol* not specials ****** could use var for global ****** could make ns a special? ******* probably not ******* but what macro would emit that? ***** install all cljs specials in dummy nses? ****** no, doesn't help macros file *** inline defmacro in cljs? **** calls Clojure eval of defmacro call **** can expand to calls to enclosing cljs ns ***** but expander can't call code in enclosing cljs ns ***** convention - all macro helper fns are local fns in macro **** gets icky for load/require purposes, as must be loaded with cljs compiler ***** better to keep public macros separate - can use ordinary Clojure require then ***** local macros would be ok for same-file consumption though **** convention - portable libs that expose macros package them separately ** Want some equivalent of refer clojure.core *** else practically everything will be qualified **** e.g. core/defn - ick *** but fewer things brought in by default? **** requires selectivity control, or just a smaller core.cljs? *** this is equivalent to a 'use', which we otherwise aren't supporting **** unfair or don't care? ** Any 'use' equivalent (e.g. refer core) means compile-time disambiguation of unqualified references *** if names a referred thing, that thing, else current.ns.name **** like current namespaces **** but if refers are limited to (entirety of) core, just look there first **** so double lookup instead of copying core vars into name table ** Some core things defined in js *** where we don't want to otherwise expose things needed for their impl **** e.g. ==, ===, math ops, instanceOf typeof etc *** how to reserve names? **** declare in core.cljs? *** if in actual .js file, separate ns for deps purposes? **** i.e. it will be a different file than that produced by compiling core.cljs **** or just a wad of js injected into core.cljs? ***** include a (js code-string) primitive for this purpose? ****** yes, much better than js files ****** accept only at top level? - no ****** using in local scope means knowing how locals are represented ****** some sort of escaping construct for getting (local and other) names resolved ******* ~{identifier} ** Are we doing forward reference detection here? *** requires listing of contents of current ns **** like namespaces ** Are we doing extern-ns name validation? *** could do for cljs names, but not others **** e.g. goog.whatever not enumerable in cljs **** can we discern this situation? ***** probably not, when compiling from files ****** since 'require' doesn't load code at compile time **** another reason we can't support 'use' ***** we do want to be able to (:require goog.foo) ****** but not a compile-time enumerable ns ***** or especially: (:require [goog.foo :as gfoo]) ***** means alias map, like namespaces ** Macros written in separate Clojure files *** Clojure code, in regular namespaces *** Means core split into core.cljs, and core-macros.clj **** both need to be auto-referred *** if no use/only for macro ns, then can only get as succinct as (alias/macro ...) **** could allow explicit aliasing of vars instead of use **** extend alias for this? ***** not really extending, alias will do this due to how nses are just vars ***** but need not be used in that pat of resolution ** goog.provide throws called-twice exception *** intended to prevent providing the same ns in more than one file *** actually prevents reloading same file? - aargh *** can't wrap, since deps checkers look for it at top level **** will we need to track at compilation-time? **** will we still need *compile-file* notion? ** Compilation needs *** current ns **** *cljs-ns* ? **** is this a Clojure ns? ***** not a fit ****** map is sym->Var or Class ****** aliases are sym->Namespace *** ns has: **** *cljs-namespaces* - {name->ns} **** {:name "my.ns" :defs {sym qualified.sym} :deps {alias Namepsace-or-qualified.sym}} **** defs ***** just set of names? no map ***** or map to fully qualified self? **** deps ***** can't merge macros and cljs ns in deps ****** same ns might map to both ******* i.e. cljs.core will ***** aliases ****** sym->fully-qualified-sym ****** is this a separate mapping vs macros and requires? ******* if not, fn alias can mask out ns alias ******* that can't happen in Clojure ***** macro nses ****** map of sym->Namespaces? ******* require an alias? ******* (:macros {mm my.macros, ym your.macros}) ****** aliases for these same as others? ***** required libs must have aliases too? ****** (:require [goog.math.Long :as gml]) ****** or new (:require {gml goog.math.Long}) *** lookup 'foo - no ns, no dots **** if special - done **** if local - done **** if found in cljs.macros Namespace, the macro cljs.macros/foo **** if found in cljs.core ns, cljs.core.foo **** whatever 'foo maps to in (-> env :ns :requires) **** no use of deps *** lookup 'foo.bar.baz - no ns, dot(s) **** if foo is a local, foo_nnnn.bar.baz **** if foo has a mapping in (:ns env) - that.mapping.bar.baz - no ***** really? covered by alias/whatever ****** more idiomatic for goog.stuff than goog.stuff/foo ****** but no :as there ***** leave out for now **** else foo.bar.baz *** lookup 'foo/bar - ns with no dots **** get what 'foo maps to in (:ns env) deps ***** if nothing - error "no alias foo" **** if maps to Namespace, the macro 'bar in that ns **** else a symbol, e.g. 'fred.ethel => fred.ethel.bar *** lookup fully.qualified/foo - ns with dots **** would only use this if local shadowed (and no alias)? **** what doesn't have alias? ***** cljs.core, cljs.macros ***** could use cljs.core.foo for former ***** always interpret as macro ns? ****** or check deps vals for Namespace, else not ***** if Namespace, the macro foo in Namespace ***** fully.quallified.foo **** everything might have alias, but macros/syntax-quote need to emit full expansions *** how to refer to true globals? **** e.g. Object, String, goog **** [[https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects][JS Globals]] **** (var Name)? ***** that doesn't match Clojure, where (var x) means 'whatever x means in current ns' ***** there are no unqualified globals in Clojure **** means (extend (var Object) ...) needs to work ***** ok if extend evaluates first arg ***** it does in Clojure, as it is a function * Setup ** V8 svn co http://v8.googlecode.com/svn/trunk v8 cd v8/ scons console=readline d8 ** Closure Library svn checkout http://closure-library.googlecode.com/svn/trunk/ closure-library *** or download? [[http://code.google.com/p/closure-library/downloads/detail%3Fname%3Dclosure-library-20110323-r790.zip&can%3D1&q%3D][Zip download]] ** Closure Compiler http://closure-compiler.googlecode.com/files/compiler-latest.zip *** better(?): [[http://code.google.com/p/closure-compiler/wiki/Maven][Compiler in Maven]]