(fn (a b ? c 1 d 2))

Actually... come to think of it, using ? for optionals would work nicely in PyArc, because all arguments are optional. The primary reason I disliked the ? optional syntax was because you needed to do stuff like this:

  (fn (? a nil b nil c nil))

  (fn (a b c))

  (fn (? a 1 b nil c nil))

  (fn (? a 1 b c))
  (fn (a b ? c 1 ? d 2))

  (fn (= a 1 b c))
  (fn (a b = c 1 = d 2))

  (fn (a b |.=| c d))

Is (= a b) good enough? I'm tempted to say yes, for the reasons I gave earlier: stands out better, less likely to conflict, synergy with assignment, and the a=b ssyntax. The only major issue is using = as the first element when destructuring. That's not too bad, I think, considering the alternatives...

Oh, by the way, there is one optional argument system that would work properly all the time... not using optional syntax at all. So you would do this:

  (fn (a b c)
    (or= a 1)
    (or= b 2)
    (or= c 3))

---

(I totally value both those things, but I came to a compromise myself a while ago and henceforth forgot there was a problem: The compromise, in Penknife, is to have fixed alphabets of punctuation and non-punctuation characters, and to treat punctuated things completely differently when parsing parameter lists. (Actually, advanced parameter list parsing isn't a near-future thing for Penknife; I plan to tackle it at the same time as regexes, which will be well after I've reengineered the core.) I don't know how much an alphabet separation would help you, but it's an option.)

Hmm, what about reading a=b as a tagged value? That way the different kinds of parameters nodes can be fully distinguished by calling [only.type _].

So a=b would expand to (annotate 'optional '(a b)) ? I like how a=b has a really simple expansion though: (= a b) And then what if you want a destructuring list that has `annotate` as the first element?! :P

If you're talking about making it an internal detail: I already do. In fact, just last night I changed it so (o a b) evals at run time (like Arc), but (= a b) evals at creation time (like Python).

Not quite. I'm suggesting a=b could expand to #(tagged optional (a b)), which is to say it would be the same as the result of evaluating (annotate 'optional '(a b)).

To illustrate, these expressions would be equivalent (if you don't consider memory consumption or mutation):

  '(def foo (a b c=d e)
     (+ a b c e))
  
  `(def foo (a b ,(annotate 'optional '(c d)) e)
     (+ a b c e))

  (def foo (a b (= c 1) (= d 2)))

So... your approach would work, in the sense that you can then use = while destructuring without causing problems, but then if you don't want to use the a=b ssyntax, you would need to do this:

  (with (c (annotate 'optional '(c 1))
         d (annotate 'optional '(d 2)))
    (def foo (a b c d)))

  (def foo (a b ? c 1 d 2))
  (def foo (a b ? c 1 ? d 2))

  (def foo (a b c=1 ? d 2))

  (with (c (annotate 'optional '(c 1))
         d (annotate 'optional '(d 2)))
    (def foo (a b c d)))

My first impression of that code is that it doesn't do what you intend. Rather than using the tagged values held in the variables 'c and 'd, you're making new variables named 'c and 'd which happen to shadow the first ones.

I think what you're going for would need to look more like this:

  (def foo (a b #(tagged optional (c 1)) #(tagged optional (d 2)))
    ...)

Seeing this sketch, you might still say it's verbose to write #(tagged optional (a b)), and you might continue to say it's verbose if I change the example to use a syntax like {tag = a b}. (I dunno, you might not. ^_^ ) The thing is, we could avoid that syntax in most cases since a=b would work just as well.

---

Oh, I forgot to mention this the first time you brought it up, but the "?" alternatives don't help with this particular issue, since I might want to name a variable "?" too. It's not that I really need to name a variable "?", any more than I really need to name a variable "o" or "=", but it's the same principle.

---

"P.S. I find it amusing that we're coming up with increasingly convolted and complicated systems just so we can allow = as the first element when destructuring. :P"

Which is okay, if you want users to always use a=b for optional arguments, but I don't think that's so great if you want a=b to desugar to S-expressions (like the other ssyntaxes). I guess it's just the eternal battle between simplicity and correctness.

---

Yeah, but it does help with the issue of destructuring. See what I mean about no system being perfect? Anyways, if I used the hybrid approach, I would expect this to work:

  (def foo (a b ?=nil))

That's fine if you don't mind the additional complexity. But I tend to like simplicity, even at the cost of small corner-case issues.

---

In PyArc, I believe you. A syntax like #(...) doesn't work if # is going to be interpreted as prefix syntax. That said, is {tag = a b} not an option? No worries either way, I'm just happy you know what the heck I'm talking about. ^_^

With all this talk about this one idea, I've been meaning to throw yet another idea at you: How about an alternate way to destructure? If (= a b) makes an optional argument 'a, maybe something like (% = a b) could mean three destructuring local variables '=, 'a, and 'b. A macro that wants to destructure using a user-supplied variable could use the (% ...) version of destructuring, rather than letting '= mess things up or jumping through more complicated hoops to deal with it.

In fact, % and = could someday be part of a user-extensible framework of pattern matchers, although that would take a lot more infrastructure.

---

"Since ssyntax is expanded at read time, this would bypass the ? symbol detection."

Whoa, nifty.

---

"But I tend to like simplicity, even at the cost of small corner-case issues."

And the reason I care about corner-case issues is 'cause I like the simplicity of not having to deal with them. ^_^

Well, I could just refuse to deal with them anyway, but then I sacrifice correctness. So it is a simplicity-and-correctness struggle after all.

---

"I'm not necessarily saying it's a bad thing that we're discussing alternatives (even convolted ones), just that I found it amusing."

Yeah, we could go the alternate route: rather than trying to fix optional syntax, we could change destructuring, since it seems to be the major pain point. If I recall, akkartik was talking about using ' for destructuring:

  (def foo (a b '(= c d)) (list a b = c d))

  (foo 1 2 '(3 4 5)) -> (1 2 3 4 5)

In fact, I like that enough that I'm going to implement that in PyArc. You can use --backwards-compat to use the old behavior for destructuring. In case you're worried, it would work even if you want to use quote as a normal argument name or as an argument name when destructuring:

  (def foo (quote '(quote a b)))

And as an added bonus, I can make it so it distinguishes between symbols and lists:

  (def foo ('a '(b c)))

This also fixes one thing I wanted to add: destructuring assignments. In Python, you can use this:

  a, b = (1, 2)

  (= '(a b) '(1 2))

  (def foo () (list 1 2 3))

  (= '(a) (foo)) -> a is now 1

  (= a,b (foo))

[1]: The only syntax I can think of that can't/doesn't expand to S-expressions is "." I could even change PyArc so "" gets expanded to (string ...) and #\ gets expanded to (char ...). I find this one-to-one mapping between syntax and S-expressions really neat.

---

Oh, yeah, and if you had ? and not = you could even do this:

  (def foo (a b ? ? nil))

  (zap idfn '(a b c))    ; define self-evaluating symbols?
  (= '(a b c=4) x)       ; multiple assignment with optionals?

Turns out the idea originated with waterhouse, akkartik tweaked it, you mentioned something similar which reminded me of it, aaand then I added on destructuring assignments. Hurray for collaboration!

Also, why can't we do both? If I understand it correctly, = works based on the name, so we could have (list ...) and (quote ...) do the same thing. I like how (= '(a b)) is short, and it has synergy with destructuring in argument lists, so I'd like to keep that. :P

In fact, if we did that... we could have destructuring destructuring assignments:

  (= '(a '(b c (d)) e)
     '(1  (2 3 (4)) 5))

  a -> 1
  b -> 2
  c -> 3
  d -> 4
  e -> 5

  (= '(a b . c)
     '(1 2 3 4 5))
     
  a -> 1
  b -> 2
  c -> (3 4 5)

  (= '(a b c) nil)

P.S. Destructuring destructuring might be useful for alists:

  (= '(_    '(_ a))
     '((a 1) (b 2))) -> a is now 2

  (= foo '((a 1) (b 2)))
  (= a foo.'b)

You have an example with _, but be careful, '_ is one of the most common Arc variable names. :-p Replacing '(_ a) with '('() a) should make it safer.

In PyArc, I assume (= '() a) and (= nil a) would be distinct cases. Besides the difference of a (quote ...) form, you've already said nil and () were different.

Of course I could change it so it ignores nil when destructuring. That may be a good idea.

Oh, right. Still, at least (= nil 4) and (= 'nil 4) have the (quote ...) difference....

Come to think of it, maybe it doesn't work to consider a rest parameter as a destructuring form, since conflating (fn 'a ...) with (fn a ...) would also conflate (= 'a ...) with (= a ...). (From the future: In fact, there might be trouble integrating setforms and parameter lists at all. See below.)

Here's a full grammar for pg-Arc parameter lists:

  === parameter list ===
  
  ; This matches based on matching the car and cdr of a cons. It raises
  ; an error on other kinds of value, *including nil*.
  (<destructuring form> . <parameter list>)
  
  ; This binds what it's matched to to a new local variable.
  <non-nil symbol>
  
  ; This raises an error if it's matched to something other than nil.
  ()
  
  ; When matched to nil, this evaluates <default> and matches that to
  ; <destructuring atom>, and it matches nil to <parameter list>. When
  ; matched to a cons, this matches <destructuring atom> to the car and
  ; <parameter list> to the cdr. When matched to another kind of value,
  ; this raises an error.
  ((o <destructuring atom> <default>) . <parameter list>)
  
  ; The ((o <> <>) . <>) rule takes precedence over the (<> . <>) rule.
  
  
  === destructuring form ===
  
  ; This matches based on matching the car and cdr of a cons *or nil*.
  ; It raises an error on other kinds of value.
  (<destructuring form> . <destructuring form>)
  
  ((o <destructuring atom> <default>) . <destructuring form>)
  
  <destructuring atom>
  
  ; The ((o <> <>) . <>) rule takes precedence over the (<> . <>) rule.
  
  
  === destructuring atom ===
  
  <non-nil symbol>
  
  ; This ignores what it's matched to altogether.
  ()

  === parameter ===
  
  ; This matches based on matching the car and cdr of a cons or nil. It
  ; raises an error on other kinds of value.
  (quote (<parameter> . <parameter>))
  
  ; When matched to nil, this evaluates <default> and matches that to
  ; <parameter a>, and it matches nil to <parameter b>. When matched to
  ; a cons, this matches <parameter a> to the car and <parameter b> to
  ; the cdr. When matched to another kind of value, this raises an
  ; error.
  ((= <parameter a> <default>) . <parameter b>)
  
  ; This binds what it's matched to to a new local variable.
  <non-nil symbol>
  
  ; This ignores what it's matched to altogether.
  ()
  
  ; This matches as though it's (quote (<parameter a> . <parameter b>)).
  (<parameter a> . <parameter b>)
  
  ; The (<> . <>) syntax has the lowest precedence.

  (let old fn
    (mac fn (names . body)
      (w/uniq a
        `(,old ,a
           (let ,names nil
             (= ',names ,a)
             ,@body)))))


  (fn args (+ 1 5))  -> (<fexpr fn> gs524
                          (let args nil
                            (= 'args gs524)
                            (+ 1 5)))
    
    
  (fn (n x) (+ n x)) -> (<fexpr fn> gs525
                          (let (n x) nil
                            (= '(n x) gs525)
                            (+ n x)))

Derp, I just realized that `let` desugars to `fn`, which doesn't have destructuring. I guess you could replace `let` with `with` and it would still work, albeit it would be clunkier. Something like this:

  (let old fn
    (mac fn (names . body)
      (w/uniq a
        `(,old ,a
           (with ,(if (isa names 'cons)
                        (mappend [list _ nil] names)
                      (list names nil))
             (= ',names ,a)
             ,@body)))))

  (let old fn
    (mac fn (names . body)
      (if (isa names 'cons)
            (w/uniq a
              `(,old ,a
                 (with ,(mappend [list _ nil] names)
                   (= ',names ,a)
                   ,@body)))
          `(,old ,names ,@body))))

  (simple-fn gs525
    (simple-let-nil (a (b c) d=1 e=2)
      (= '(a (b c) d=1 e=2) gs525)
      ...))

...Well, since you're in an interpreter, you may be able to get away with this:

  (simple-let gs1 (argument-list-vars '(a (b c) d=1 e=2))
    (simple-fn gs2
      (interpreted-simple-let-nil gs1
        (= '(a (b c) d=1 e=2) gs2)
        ...)))

  (with (gs204 nil gs205 nil)
    (= a gs204)
    (= b gs205))

  (with (gs206 1
         gs207 2
         gs208 (3 4))
    (= a gs206)
    (= b gs207)
    (= c gs208))

  (do
    (assign a 1)
    (assign b 2))

I thought you meant something like this:

  (= '(a b (c d)) '(1 2 (3 4)))
  -->
  (apply (fn (a b (c d))
           (export-locals-to-parent-scope))
         '(1 2 (3 4)))

It's not a big deal, though. In a worst-case scenario, we can define `=` so it does a single level of destructuring. That should be simple to add, even in pg-Arc, while still providing some usefulness.

P.S. `with` expands to a function, so technically it is desugaring to a function. :P Your idea is quite a bit shorter and simpler, though.

Also... this just reminded me of something: lambdas in JavaScript. If I recall, there was a proposal for "thin" functions in JS, which (among other things) would not have "var" scope:

  function foo() {
    (lambda {
      var bar = 10;
    })()

    return bar;
  }

  foo() -> 10

A similar "thin-fn" form in Arc might be an interesting experiment: any assignments within the thin-fn would always assign to the outer scope. You could still assign locally, by using `let`, since that desugars to a "heavy" function:

  (def foo (a)
    ((thin-fn (a)
      (= a "foo")))
    a)
      
  (foo) -> "foo"
        
        
  (def foo (a)
    ((thin-fn (a)
      (let a nil
        (= a "foo"))))
    a)
        
  (foo) -> nil

You might already know this (search for "AST" at http://arclanguage.org/item?id=13079), but I'm irked by special forms like 'assign or 'if too. I like reducing the places where some message almost always has one meaning, unless it's some particular message which has a completely different meaning. I'm just fine when language designers make knowing sacrifices, but whenever possible, I want to be able to say "the eagle has left the nest" to talk about actual eagles--or at least to be sure I won't ever want to talk about actual eagles.

I think 'o is a place where neither sacrifice is necessary (neither giving up (o b c) destructuring nor giving up 'o as a variable name).

As for 'tagged, that's a symbol you can't observe from Arc unless you drop to Racket, whereas macros see the symbol 'o all the time.

---

"Your suggestion is to match a 'keyphrase': (tagged optional _) or even having to detect then evaluate things like (annotate 'optional 'a) at compile-time. Maybe even read time. Does that ever happen right now?"

Eh? O_o You're all over the map in misinterpreting me (which isn't something I blame you for).

Some parts of what you're talking about "happen right now" in PyArc, as far as I understand: Under default options, there's no distinction between compile time and read time, and macros are expanded in that one step, just like ssyntax is. This isn't necessarily the final reader design, since it causes problems with things like (afn ((= a b)) ...). Keep in mind that I'm talking about solutions for PyArc in this turbulent time, so suggestions I make may be more generalized, more wacky, and more misinterpretable than the same suggestions applied to any given concrete version of PyArc. ^^;

The solution I'm talking about does apply to official Arc too, except that official Arc doesn't have ssyntax in the reader, so the approach isn't as practical. Official Arc does save ssexpansion until surrounding macros have done their work (post-read-time), so we can use an easier solution like '.o instead.

Here's exactly how the tagged value approach would look in official Arc:

  (def foo (a b #(tagged optional (c d)) e)
    (+ a b c e))

  arc> (type '#(ta­gged a b))
  a
  arc> (mac stx-types args `',(map type args))
  #(tagged mac #<procedure: stx-types>)
  arc> (stx-types 4 a (b) () "c" #\d #(tagged e f))
  (int sym cons sym string char e)

By the way, if the #(...) syntax didn't exist, we could still define functions with optional arguments this way:

  (eval `(def foo (a b ,(annotate 'optional '(c d)) e)
           (+ a b c e)))

Yeah, I don't like those either. In fact, I was thinking about making `fn` a macro in PyArc. Also, if I can, I plan to allow programs to shadow things like `assign` and other special forms:

  (def foo (bar assign)
    (= bar "something"))

  (foo) -> error, because = uses assign, but assign is nil

The only way I can see to get rid of if, assign, quote, and quasiquote would be to create a third "function" type: it behaves like a function, so it's evaluated at run time, but it doesn't evaluate it's arguments, so it's basically a function/macro hybrid.

Haha, I just had a crazy idea. Make (fn) desugar into the special hybrid form, but have it evaluate all it's arguments. Then there would only be two function types, and (fn) would be a thin wrapper around one of them. I should be able to get rid of all the special forms then, leaving only functions and macros (which are just annotated functions anyways).

---

"I think 'o is a place where neither sacrifice is necessary (neither giving up (o b c) destructuring nor giving up 'o as a variable name)."

But you do give up (o a b) destructuring:

  (def foo ((o a 1) (o b a)) (list a b))

  (foo '(1 2 3) '(4 5 6)) -> ((1 2 3) (4 5 6))

Thus, it has the same gotcha as (= a b), which is that you can't use o as the first element when destructuring. By the way, I mentioned earlier that I could treat escaped symbols differently:

  (def foo ((|=| a b)) (list = a b))

  (foo '(1 2 3)) -> (1 2 3)

You can definitely implement 'quasiquote and 'if as macros.

That leaves 'quote, 'fn, and 'assign. Those are all pretty fundamental. While you could make them macros, there's pretty much nothing to turn them into, unless you go for the tagged type route.

---

"The only way I can see to get rid of if, assign, quote, and quasiquote would be to create a third "function" type: it behaves like a function, so it's evaluated at run time, but it doesn't evaluate it's arguments, so it's basically a function/macro hybrid."

Hey, look, fexprs. XD Every sufficiently long Arc Forum topic seems to get around to them at some point. :-p

Since you're writing an interpreter, fexprs are a natural thing to do. They have certain elegance advantages over classical macros, since you can avoid special forms altogether, and programmers can (re)define their fexprs after the places they use them. The main disadvantage is that they're much harder to efficiently compile--not an issue for an interpreter.

---

"Haha, I just had a crazy idea. Make (fn) desugar into the special hybrid form, but have it evaluate all it's arguments. Then there would only be two function types, and (fn) would be a thin wrapper around one of them. I should be able to get rid of all the special forms then, leaving only functions and macros (which are just annotated functions anyways)."

Kernel, the most promising design I've seen for an efficient fexpr language (er, with lexical scope; sorry, PicoLisp) does the opposite: Functions are annotated fexprs. All forms are applied in exactly the same way, but functions take the extra step of evaluating their arguments. (FYI, Kernel calls functions "applicatives" and fexprs "operatives.") The one thing about Kernel is that it's kinda vaporware....

There was recently an Arc Forum topic about Kernel: http://arclanguage.org/item?id=13354

Here are a couple of topics about PicoLisp while I'm at it: http://arclanguage.org/item?id=1110 http://arclanguage.org/item?id=10829

Eight's another fexpr language with lexical scope. Like Kernel, it isn't quite complete, but at least it's developed in the open on GitHub. (Hi diiq!) There's a very old topic on Eight here: http://arclanguage.org/item?id=10719

Recently, like a few months ago, diiq came back to Eight and reimplemented it in JavaScript. I've been too busy to bother with it yet, but it oughta be awesome if it's both similar to Arc and runnable in a browser.

---

"Unless of course you're talking about ab/using ssyntax like .o, in which case you're not really using (o a b) anymore, nor would it work if ssyntax is expanded at read time."

That's exactly the kind of thing I'm talking about. :) That's a solution I'm satisfied with for official Arc, but I'm confident there'd be at least some solution for PyArc that I liked better than 'o. It may just be too early to identify it.

---

"By the way, I mentioned earlier that I could treat escaped symbols differently: [...] I think this approach would be better than using ssyntax, because || already means "escape this thing", so we don't need to invent new syntax for it. Not sure if I'll go that route, but it is an option."

I don't see how `if` could be implemented as a macro. Consider this case:

  (def foo (a) (if a 1 2))

---

Yeah, that's what I was talking about: have `fn` be a fexpr that returns functions that are annotated with type 'fn. And since macros are just functions with type 'fn, that would mean that everything in PyArc boils down to fexprs.

I didn't realize that what I described was fexprs, but in hindsight I probably should have. I actually messed around with NewLisp for a bit; they use fexprs. I prefer Arc's macro system, since I don't think NewLisp has quasiquote/unquote, which made certain things more verbose than they should have been.

  (def fn-if (condition then else)
    ((if condition then else)))
  
  (mac if body
    (whenlet (first . rest) body
      (iflet (then . elses) rest
        `(fn-if ,first (fn () ,then) (fn () (if ,@elses)))
        first)))

Okay, that's a very cool idea: using ' to mean "don't evaluate the argument" (http://arclanguage.org/item?id=10745):

  (def foo ('a b) (list a b))

  (foo (+ 1 2) (+ 1 2)) -> ((+ 1 2) 3)

However, I ran into a bit of a problem while parsing arc.arc, so I'll likely be refactoring that system. For instance, I may want it to only expand macros one level, rather than doing them all. The problem is, let's say you have this:

  (mac foo () "something")
  (mac bar () (foo))

  ... (fn nil "something") ...
  ... (fn nil (foo)) ...

  ... (fn () "something") ...
  ... (fn () "something") ...

In any case, the whole tokenize/parse/transform/macro expand stage is handled at the same time, so once that part's done, it can just use eval on the rest.

---

As for handling argument-only ssyntax, I've been wondering about that myself. But here's my current idea: when eval sees an (fn) form, it then creates and returns a (Python) function, which it can then execute later. Arguments and closures are handled when the function is created, so I should be able to do ssyntax expansion then.

There's another approach I thought you might have taken, and it's free of that trouble. You mentioned looking for (fn) forms, and it's probably the same general idea: When you encounter a left paren, read the next expression after that, and use it to determine what happens next (like a reader macro). If it's a symbol, look it up and see if it's bound to a macro, and let the macro (somehow) take care of reading its body and finding the right paren. Otherwise, read elements until you get a right paren, and treat the expreaaions as a function call.

---