One of the things this touches on is associating expressions with source code. That's something I tend to neglect to do, but this makes it look easy.

The other thing is it shows a way to normalize expressions to other expressions, just using HOAS (aka putting functions in your expression AST). This makes expressions the only run time data structures needed in the language.

http://smihica.github.io/arc-js/

The creator (smihica, Shin Aoyama) hasn't come to the forum yet, but their work is impressive. Just look at that REPL start up in under a second! On my machine anyway. :)

Thanks to svetlyak40wt for discovering this project 80 days ago (http://arclanguage.org/item?id=18355). It was in active development then, and it's still active now.

---

The primary spark that led me to make Rainbow.js was a desire to respond constructively to threads like this one.

Using Rainbow was a side effect because I wanted to wow people with the execution speed, which conanite had already meticulously worked on in Rainbow. This was still tied into the desire to help those forum threads along: Between speed and portability, what possible excuse could there be not to use it? ^_^

Once I committed to particular technical goals involving consistency with Rainbow, the 20,000 lines of code were basically predetermined. It's kind of unfortunate that it would fail to be helpful in threads like this one, but it's not all a loss.

This changes everything. With arc-js you can start writing client-side apps in the browser without dealing with Javascript. You can add libraries in arc-js that do that (are there any already?) The big advantage is you bypass HTML: you can make a browser and a server send sexps back and forth.

For instance, the basic fizzbuzz example provided on the main page looks very similar, but doesn't actually run in anarki. Maybe that's anarki's fault, but the syntax of the for loop is different.

---

Even without SQL, code injection is something to worry about. The Arc codebase is a breeding ground for exactly this kind of issue, since it rarely does string escaping. Let's see...

HTML injection (XSS attacks): This is the kind of injection news.arc primarily needs to worry about. Almost every string it passes around is used directly as an HTML code snippet. Fortunately, every user input is sanitized thanks to the form-generating utilities in app.arc.

Shell injection: Make sure that any directory paths passed to (ensure-dir ...) are already shell-escaped. (Arc also invokes the shell in a few other places, but those don't need any extra escaping.)

Format string injection: Be careful about file paths passed to (tofile ...). Everything after the last slash must be a valid 0-argument format string. The format string syntax is described at http://docs.racket-lang.org/reference/Writing.html.

I'm pretty sure the answer is no, for now.

With the right use of Racket's 'parameterize and 'current-parameterization, we could build a system that works like Common Lisp's conditions and restarts. (Global variables are an alternative to 'parameterize, but they'll have more awkward interactions with continuations and threads.)

I've only played with CL once or twice, but I seem to recall one of the nicest parts about conditions and restarts is that when a condition isn't caught at the REPL, the user gets a sub-prompt they can use to invoke a restart. Threads might give us something close to this behavior, if we make the REPL's catch-all condition handler block the thread while the user chooses a restart.

Unfortunately, all preexisting Racket/Arc errors will still use the exception system rather than this homemade condition system. In order to make conditions the main error mechanism, we might have to modify the way Arc compiles to Racket so that it traps exceptions and turns them into conditions.

Finally, CL's condition handlers use its subclassing system to find the most specific handler. Arc doesn't have hierarchical dynamic typing like that, or at least not in the main language, so it might not make sense to handle conditions that way.

It's probably not. Restarts (and similar) just seemed like a really useful tool for use in production environments. Anything that makes error handling more powerful, flexible, or understandable is a good thing. But changing arc to support it might be a bit much, at least at this point.

I do think it would be good if we could improve arc's error handling though. Or at least error reporting on the repl.

Arc doesn't have hierarchical dynamic typing like that...

That's not necessary either. If you want an Arc expression that evaluates to a keyword, you can already just do this:

  arc> '#:foo
  #:foo

(Example output from http://tryarc.org/.)

  arc> (= k-idfn ($:lambda (#:k v) v))
  #<procedure:zz>
  arc> (k-idfn '#:k "hello")
  Error: "struct procedure:zz: expects 0 arguments plus an argument with keyword #:k, given 2: #:k "hello""

We pretty much have two ways to call this function successfully.[1] First, we can write the function call itself as Racket code, taking advantage of Racket's function call syntax compiler to parse the keywords:

  arc> (let call-k ($:lambda (f v) (f #:k v)) (call-k k-idfn "hello"))
  "hello"

  arc> ($.keyword-apply k-idfn ($.list '#:k) ($.list "hello") ($.list))
  "hello"

Note that keyword-apply requires the keywords to be given in a specific order based on their UTF-8 bytes.

If we do a lot of this in the code, I'm thinking we could benefit from one of three things:

- An Arc function that takes an Arc table like (obj k (list "hello")) and calls Racket's 'keyword-apply it to the sorted keywords it expects.

- An Arc macro that looks like a function call, e.g. ($kw-call k-ifn #:k "hello"), but finds any occurrences of keywords and generates the appropriate Racket function call.

- A patch to Arc's own function call syntax so that it parses Racket-style keyword arguments.

I think these solutions would be more effective than shaving one or two heiroglyphics off of '#:k.

---

Not even Racket allows #:k as an expression by itself. It has to be written '#:k. After all, even if #:k evaluated to itself, (list #:k "hello") would not give you a list containing #:k and "hello", since it would try to pass a keyword argument instead.

You might wonder why Racket chose a design with such hoops to jump through. Well, I can't speak for them, but personally I'd give several shallow and hand-wavy reasons:

- It aligns with a certain mental model where positional args are an ordered list and keyword args are an orderless map. (This is the mental model I learned a long time ago in Groovy, so I like it.)

- It lets Racket treat (foo #:a 1 #:b #:c 3) as a compile-time error.

- It lets Racket treat missing/extra keyword args and missing/extra positional args as the same kind of error.

- It means keyword args don't have to be described in terms of positional args. It would be possible to design a lisp that only has Racket-style keyword args, with no positional args whatsoever.

- It means when a programmer wants to get a procedure's arity information dynamically, the result can include information about the keyword args it supports, rather than just the positional args it supports.

---

[1] Hmm, maybe there's a third way to call it successfully:

  arc> (= successfully ($:lambda (#:k v) v))

If what you're saying about '#:key vs #:key is true though, then my test cases were incorrect and I was trying to make something work that shouldn't have.

It turns out I'm getting much different results on a local setup than I was getting in tryarc.org. Since Arc 3.1 and Anarki use (require mzscheme), all sorts of things are replaced with doppelgangers that doesn't support keyword args, including the function application syntax #%app. In fact the lack of keyword arguments is one of the only things (require mzscheme) is good for. This is something tryarc.org changes, apparently.

MzScheme docs: http://docs.racket-lang.org/mzscheme/index.html

---

"So, what's the right way to hack on arc to support calling racket functions with keyword args?"

I wouldn't say it's "the right way," but we could put the functionality of Racket's 'keyword-apply in a function that takes an Arc table, and then we'd pretty much never need to worry about keywords other than that.

I have an implementation for this, which I should be able to commit with some unit tests now that I know what's going on with MzScheme.

...Actually it might take me a few days to get around to that, so here's the code if anyone wants to use it right away:

  (def $kw-apply (func kwargs . posargs)
    
    ; Convert the last element of posargs (the first-class list) into a
    ; Racket list. If posargs is completely empty, pretend the last
    ; element is an empty list just like Arc's 'apply already does.
    (zap [rev:aif rev._
           (cons (apply $.list car.it) cdr.it)
           (list:$.list)]
         posargs)
    
    (let (ks vs)
         (apply map list  ; Transpose.
           (sort (compare $.keyword<? !0)
             (map [list ($.string->keyword:+ "" (or _.0 "nil")) _.1.0]
               tablist.kwargs)))
      (apply $.keyword-apply func (apply $.list ks) (apply $.list vs)
        posargs)))

  ($kw-apply my-func
    
    ; keyword args
    (obj a list.1 b list.2 message (list "hello"))
    
    ; positional args
    1 2 3 (list 4 5))

---

"Or would it be better to make a more arc-idiomatic mongo driver, and how?"

I actually have some opinion about "the best way" for this. :)

I think Arc-idiomatic approaches serve no particular purpose, since Arc is a tool for general-purpose computation.[1] I would want a database driver to be idiomatic only for the database itself, so that it serves the more specific purpose of storing data. This can then be accompanied with sugary helper utilities, as long as they're optional.

It seems many ORMs want to bake the sugar into the interface, or they make sugar that has tiny escape hatches for poking at the underlying interface. If sugar is the only thing a programmer (usually) sees, I look at it as though it's a full-on database design of its own... which is rarely favorable since it usually inherits most of the complexity and obligations of the original.

---

Interesting. I'll have to play around with that. And study it a bit more to figure out how it works. If I understand your description correctly, the reason you're using lists is so that 'nil is interpreted as no value, while '(nil) is interpreted as intentionally passing the value 'nil? How does the function know the difference?

---

Interesting opinion. Unless I misunderstood you, I would have felt the opposite way. Normally, I would expect the purpose of the data interaction layer to be separating the implementation details from the code, so that if changes need to be made to what backend storage system you can just trade it out.

There's always a reason, but these days Anarki has broke all my code enough times that that shouldn't be a concern. :)

I think this would be a positive change.

---

"...the reason you're using lists is so that 'nil is interpreted as no value, while '(nil) is interpreted as intentionally passing the value 'nil? How does the function know the difference?"

Arc doesn't support nil as an element of a table. Setting a table entry to nil removes it. Therefore '$kw-apply will only see non-nil values anyway.

As pg says: "In situations where the values you're storing might be nil, you just enclose all the values in lists." http://arclanguage.org/item?id=493

When I was using Arc heavily, I defined a utility (sobj ...) that was just like (obj ...) but it wrapped everything in a singleton list. That may have been the only extra utility I needed.

I could write (each (k (v)) tab ...) in place of (each (k v) tab ...). I could write tab!key.0 in place of tab!key. I could write (iflet (v) tab!key ...) in place of (iflet v tab!key ...).

It was surprisingly unintrusive, even pleasant.

---

"Normally, I would expect the purpose of the data interaction layer to be separating the implementation details from the code, so that if changes need to be made to what backend storage system you can just trade it out."

I can't find links at the moment, but pg has repeatedly said that HN runs on a single extremely beefy server with lots of caching for performance.

Edit: Racket's docs say that "Threads run concurrently in the sense that one thread can preempt another without its cooperation, but threads do not run in parallel in the sense of using multiple hardware processors." (http://docs.racket-lang.org/guide/concurrency.html) So arc's use of atomic wouldn't matter in this case. It does prevent HN from using multiple load-balanced servers.

Well, in JavaScript, I do concurrency by manually using continuation-passing style and building my own arbiter/trampoline... and using it for all my code. If I ever do something an easier way, I have to rewrite it eventually. Whenever I want to try out a different arbiter/trampoline technique, I have to rewrite all my code.

Arc's threading semantics are at least more automatic than that. Naive Arc code is pretty much always usable as a thread, and it just so happens it's especially useful if it doesn't use expensive 'atomic blocks (or the mutators that use them).

"Automatic" doesn't necessarily mean automatic in a good way for all applications. Even if I were working in Arc, I still might resort to building my own arbiters, trampolines, and such, because concurrency experiments are part of what I'm doing.

(Psst, if you say "mzScheme," I feel the need to remind you that Arc works on the latest versions of Racket, which have long since dropped the name "mzscheme".)

Generally, threads are for concurrency, not necessarily parallelism. They're a workaround for an imperative, sequence-of-side-effects model of computation, which would otherwise force us to choose which subcomputation should come first. In Racket, this kind of workaround is their only purpose.

Racket has two features for parallelism, and they're called "futures" and "places":

http://docs.racket-lang.org/reference/concurrency.html

I'm finding out about these for the first time, but I'll summarize anyway.

Futures are a lot like threads, but they're specifically for speculative parallelism. They're allowed to break some invariants that threads would have preserved, and (as per the nature of speculative parallelism) some of their computations may be thrown away.

Places use shared-nothing concurrency with message passing, and each place runs in parallel.

In the language(s) I'm working on, the number type has mainly been a way to represent list indexes and lengths. List processing depends on integer processing, or so I thought. The other day I realized I could just use lists instead.

  ; Scheme-ish code
  
  > (list-ref '(a b c) '())
  a
  
  > (list-ref '(a b c) '(z))
  b
  
  > (list-ref '(a b c) '(y z))
  c
  
  > (len '(a b c))
  (() () ())  ; or even (a b c)

In my designs, I don't just want to make things that are easy for casual consumers. I want to make things people can consume, understand, talk about, implement, upgrade, deprecate, and so on. These are things users do, even if they're not all formal uses of the interface.

I hardly need number types most of the time. If I add them to my language(s), they might just be dead weight that makes the language design more exhausting to talk about and more work to implement.

Still, sometimes I want bigints, or at least numbers big enough to measure time intervals and pixels. I don't think any one notion of "number" will satisfy everyone, so I like the idea of putting numbers in a separate module of their own, where their complexity will have a limited effect on the rest of the language design.

---

"Huh, this is similar to Church numerals"

I'm influenced by dependently typed languages (e.g. Agda, Twelf, Idris) which tend to use a unary encoding of natural numbers in most of their toy examples:

  data Nat = Zero | Succ Nat

A binary encoding might use a technique like this:

  data OneOrMore = One | Double OneOrMore | DoublePlusOne OneOrMore
  data Int = Negative OneOrMore | Zero | Positive OneOrMore

  data Bool = False | True
  data List a = Nil | Cons a (List a)
  data Int = Negative (List Bool) | Zero | Positive (List Bool)

  data Nat = Zero | Succ Nat

  Zero : Nat
  Succ : (Nat -> Nat)

  buildMyNat : Nat -> (Nat -> Nat) -> Nat
  buildMyNat zero succ = succ (succ (succ zero)))

  buildMyNat : a -> (a -> a) -> a

While it's more general in this way, I think sometimes it's a bit less powerful. Dependently typed languages often provide induction and recursion support for ADT definitions, but I think they can't generally do that for Church-encoded types. (I could be wrong.)

For something more interesting, we can go through the same process to build a Church encoding for my binary integer example:

  data OneOrMore = One | Double OneOrMore | DoublePlusOne OneOrMore
  data Int = Negative OneOrMore | Zero | Positive OneOrMore
  
  buildMyInt :
    OneOrMore ->                  -- One
    (OneOrMore -> OneOrMore) ->   -- Double
    (OneOrMore -> OneOrMore) ->   -- DoublePlusOne
    (OneOrMore -> Int) ->         -- Negative
    Int ->                        -- Zero
    (OneOrMore -> Int) ->         -- Positive
      Int
  
  buildMyInt :
    a ->          -- One
    (a -> a) ->   -- Double
    (a -> a) ->   -- DoublePlusOne
    (a -> b) ->   -- Negative
    b ->          -- Zero
    (a -> b) ->   -- Positive
      b