Quoting from myself,

Quote syntax (as well as fexprs in general) lets you take code you've written use it as data, but it does little to assure you that any of that computation will happen at compile time. If you want some intensive calculation to happen at compile time, you have to do it in a way you know the compiler (as well as any compiler-like functionality you've defined) will be nice enough to constant-propagate and inline for you.

I've realized "compiler-like functionality you've defined" is much easier to create in a compiled language where the code-walking framework already exists than in an interpreted language where you have to make your own.

If part of a language's goal is to be great at syntax, it has a conflict of interest when it comes to fexprs. They're extremely elegant, but user libraries can't get very far beyond them (at least, without making isolated sublanguages). On the other hand, the dilemma can be resolved by seeing that an fexpr call can compile into a call to the fexpr interpreter. The compiler at the core may be less elegant, but the language code can have the best of both worlds.

This is an approach I hope will work for Penknife. In a way, Penknife's a compiled language in order to support fexpr libraries. I don't actually expect to support fexprs in the core, but I may write a library. Kernel-style fexprs really are elegant. ^_^

Speaking of such Kernel-like libraries, I've thrown together a sketch of a Kernel-like interpreter written in Arc. It's totally untested, but if the stars have aligned, it may only have a few crippling typos and omissions. :-p https://gist.github.com/778492

Hmm... not sure what you mean by this. I can define a local variable at the PicoLisp REPL using let just as I would in Arc:

  : x   
  -> NIL
  : (let x 5
      x)  
  -> 5
  : x     
  -> NIL

Speaking of speaking too soon, I may have said "user libraries can't get very far beyond [an fexpr language's core syntax]," but I want to add the disclaimer that there's no way I actually know that.

> This is a form of lexical scoping - though we still have dynamic binding - of symbols, similar to the static keyword in C. [1]

> "But with dynamic binding I cannot implement closures!" This is not true. Closures are a matter of scope, not of binding. [2]

---

[1] http://software-lab.de/doc/faq.html#problems

-

What happens when I locally bind a symbol which has a function definition?

That's not a good idea. The next time that function gets executed within the dynamic context the system may crash. Therefore we have a convention to use an upper case first letter for locally bound symbols:

  (de findCar (Car List)
     (when (member Car (cdr List))
        (list Car (car List)) ) )

  arc> (some 'c '(a b c d c b a))
  t
  arc> (find 'c '(a b c d c b a))
  c
  arc> (mem 'c '(a b c d c b a))
  (c d c b a)
  arc> (pos 'c '(a b c d c b a))
  2
  arc> (keep 'c '(a b c d c b a))  ; significantly less efficient
  (c c)

  arc> (pos no '(a b c nil e))
  3
  arc> (pos no (list idfn no 72))
  nil
  arc> (pos [is no _] (list idfn no 72))
  1

  arc> (find [do (prn "checking " _) no._] '(1 2 nil 3))
  checking 1
  checking 2
  checking nil
  checking 3
  nil

I can't say I have got a feel for Arc yet, but I think I am starting to. The more I work with it, the more it seems to open up. Not so much that feeling with the HTML definitions, however. But I am not going down that road until I have a much better handle on the language.

Not sure how anyone could would want something besides vi/vim or emacs, but I use emacs with vim key bindings so maybe I'm biased. ;)

vi and emacs are great editors, but each has a mammoth learning curve. I think this is why they often make bad first impressions. I'd be interested to know more about your experience with vi and what you disliked about it.

My experience with vi involved some physics modeling years ago in undergrad. I recall 'zz'? I never stop forgetting to toggle the input mode. I don't know why. Maybe I am a bit right-brained, but I have a sloppy way of working. Something about vi felt so 'tight', and I prefer a canvas feel. Not really the best mindset for programming, no doubt, but I have learned it's ok to be a bit sloppy and get things done rather than be very neat and unproductive.

Pretty much all I want is an editor that loads quickly, shows whitespace, highlights matching parens, and does regex search-and-replace and find-in-files. Usually my choice has been EditPlus, but Notepad++ is a great open-source alternative, and gedit's another good one (although I think I've had to use grep separately with gedit).

I turn to jEdit for editing Unicode documents, NetBeans for editing C/C++, and DrRacket for editing big Racket projects, but those are extremely rare situations for me. Most of the time I avoid all three of them because of how long they take to load; I can almost always get in and out faster with plain Notepad.

https://github.com/aran/arc.tmbundle

If you're on windows, there's a version of textmate called 'e:

http://www.e-texteditor.com/

me: Maybe :y or %y could be the zapper (or some better-suited abstraction over setforms).

evanrmurphy: Maybe something with `=` in it?

Okay, =y it is. ^_^ I've implemented a working example at https://gist.github.com/772283 (and posted this thread as a link to it).

What place.arc provides is a reader macro such that =(a b c) reads as (place (a b c)) and results in a function that either gets or sets that place depending on whether an argument is passed to it.

Here are example implementations of '= and 'zap using this system:

  (mac = args
    `(do ,@(map (fn ((a b)) `(atomic (=,a ,b))) pair.args)))
  
  (def fn-zap (func place . rest)
    (do.place:apply func (do.place) rest))
  
  ; NOTE: The original evaluates the place before the function instead.
  (mac zap (func place . rest)
    `(atomic:fn-zap ,func =,place ,@rest))

The '= macro only provides an occasional benefit too. Most of the time, you can save a character by saying (=foo bar) instead of (= foo bar).

It seems this abstraction over 'setforms not only shortens a lot of arc.arc definitions but also removes the need for some of them altogether.

---

You should know that place.arc is a little hackish in at least two ways:

- First, it still allows the symbol '= to be interpreted in a non-reader-macro way, but it does so using hard-coded special cases, and it doesn't count ssyntax like '=.variable-to-set-to-nil. The hackishness only exists with '=. If another symbol beginning with #\= is in your code, such as '=mc (an example from Lathe), it'll break by design because it'll read as (place mc). Meanwhile, symbols like 'or= and 'expand=list are just fine; I didn't make #\= a symbol-terminating character.

For what it's worth, 'apply could be moved to arc.arc:

  (def apply (self . args)
    (zap rev args)
    (zap [rev:+ (rev:car _) cdr._] args)
    (self . args))

by the time the compiler sees the expression there's nothing there to tell it that a dot was used.

In Racket's syntax there is a way, I think. It makes syntax more complicated than just its conses, though. You'd have to use an Arc variant of syntax-quasiquote (or just syntax-quasiquote) to construct it in macros.

  (def foo xs
    (bar . xs))

  (iflet (first . rest) args
    ...)

Speaking of which, why not just use another character for dotted lists, like (a b c & d)?

Penknife's a programming language. I have a very specific opinion of what programming languages will become (http://arclanguage.org/item?id=12993), and I'm making Penknife in the hope that it's a well-designed first foray into that world—well-designed in the gemlike sense of not being everything to everybody but being as flawless as possible for what it is. Penknife has two very specific purposes:

- To have a customizable syntax that's fine at the start but almost limitlessly excellent once it's been customized.

- To be great at code generation, so that even if you think you want to use another language, you're likely to be more productive if you generate that code using Penknife instead.

The point is for Penknife to be a language useful to as many people as possible for as long as possible, so that it can continue to be more and more useful as its library support improves. This is sort of a world domination plot, but it isn't really (I promise :-p ). Penknife will probably only dominate syntax design (not platform design, where security and performance are hot topics), and even then it'll probably only dominate sequence-of-commands-style syntax design (the kind good at REPLs, as opposed to the landscape-of-declarations style good in IDEs). Even then, there'll be people who can't stand the foundations upon which their code generators are built; you can't please everyone. That being said, I think Penknife does stand a very real chance of monopolizing a big segment of the programming experience, so I think it's important for a gemlike language to get there first, before a hackish one has the benefit of entrenchment.

To minimize the number of fiddly mistakes (and other hackishness) in Penknife, any complexity-introducing feature that's only rarely needed for the above purposes is to be left out. Two examples are reentrant continuations and concurrency. A lack of continuation support may force certain syntaxes to be implemented without the help of continuations for coroutines or backtracking, and a lack of concurrency may hobble some theorem-proving-heavy compilers, but I doubt that'll be everyday Penknife programming. When such a nightmarish situation does come up, it's probably time to use an FFI or system call, and since you can use Penknife to generate the program you're calling that way, it shouldn't be too bad. :-p

It's quite a bit arrogant of me to think Penknife will be gemlike anytime soon, let alone that it'll still be seen as gemlike in the future. I'm still in the process of adding language features I dearly hope won't make it to the final product in their current form. Still, I think I'm the person who best knows where the heck I'm going with this project, and I intend to use Penknife myself all the time, so I'm following my passion. Feedback and help are welcome, of course!

For syntax experimentation, my favorite existing language is Arc. The Arc Forum community's very particular about every little syntactic detail. :-p So it should be no surprise that Penknife's design borrows heavily from things I like about Arc, while fixing things I don't like. For instance:

- Infix syntax is now seamless with the language. Penknife infix operators follow the normal scoping rules, as shown in the example.

- Penknife syntaxes take arbitrary []-balanced textual code, not just s-expressions, as their bodies. This allows for custom syntaxes to behave the same way as built-in "literal" syntaxes like q[...]. Case in point: the example's sym macro.

(There's more information about the above two points at http://arclanguage.org/item?id=13079.)

- Penknife has a hygienic macro system built on top of its more general-purpose syntax system (although only this high-level macro system is exposed to Penknife for now, and I'm pretty sure it's buggy despite it currently passing every test I've come up with :-p ). It's designed to work harmoniously with a module system I've also been working on. I consider both hygiene and modules to be scoping issues, so I aim to tackle them using environments, and given that premise, I figure a module import is going to be a matter of replacing the interaction environment with a local environment that shadows it, so I generally try to minimize the idea that there's a global environment at all when I can. Thus, for hygiene's sake across modules, I have each macro capture its lexical environment like a closure, and this is why [let arcsym arc.sym [mac* ...]] works in the example.

The other reason's a bit more in-depth: I think it's oftentimes good practice to use gensym keys, and especially gensym types. It helps prevent libraries from conflicting with each other. When using this technique, an 'obj that doesn't evaluate its keys is pretty useless, but an 'obj that does evaluate its keys is as convenient as (obj a 1 b 2) again, where 'a and 'b are variables holding the gensyms.

What I really think is best is to just have both. Looking at my site-generator code, I see that I defined (objal a 1 b 2) in terms of (tabal 'a 1 'b 2). :-p I think that's generally why I make so many macros that are merely cheap abbreviations of functions, actually; I like calling the function versions directly every once in a while, but I like saving a bit of line noise in the common cases.

I agree with this. Compare it with having naming and anaphoric macros: iflet and aif, rfn and afn, even fn and []; I'd add awhile and whilet to the list even though awhile isn't in official Arc, and fromfile, tofile, and ontofile[1].

In fact, it is a very general practice in (functional?) programming to first write a function that handles something basic, then write another function on top of it that's more pleasant to work with. It really is ubiquitous. It's good because it separates the primitives from the high-level interfaces; if you want to change the primitives, you don't need to concern yourself with the high-level interface, and if you want to change the high-level interface, you don't need to concern yourself with the primitives. And macros give you flexibility, the ability to construct pleasant interfaces in ways you can't do with pure functions; for example, functions must evaluate all their arguments once, from left to right, while macros don't have to.

In this case, I don't think we have a function that directly does what you're asking for (i.e. it operates like obj except it evaluates the keys). I consider this a problem. Note that in Scheme and CL, we can go (vector x y z) instead of crap like (let u (make-vector 3) (= u.0 x u.1 y u.2 z) u). We should likewise have a good table-constructing function even if it isn't strictly necessary as a primitive. At the moment, this is the closest we have:

  (apply listtab (pair:list 'a 1 'b 2))
  =
  (obj a 1 b 2)

  (def ???? args
    (apply listtab pair.args)) 

  (xdef table (lambda args
                (let ((h (make-hash-table 'equal)))
                  (if (pair? args) ((car args) h))
                  h)))
  --ac.scm

  (table 
     [each ip ips
       (= (_ ip) (dedup (map !by (+ (bads ip) (goods ip)))))]))

  (table [each-loaded-item i
           (awhen (and i!dead (check (sitename i!url) banned-sites*))
             (push i (_ it)))]))

  (let u (table)
     (each ip ips
        (= (u ip) ...))
     u)

  (w/table u
     (each ip ips
        (= (u ip) ...)))

So. Options: (1) Have table do alternating key-value stuff; if there's one argument remaining at the end, apply it to the table; then return the table. (2) Have table do alternating key-value stuff; signal an error if there's an odd number of arguments [and Paul Graham can get used to w/table], or (2a) if there's one argument, apply that function to the table, while if there are an odd number ≥3 of arguments, signal an error. (3) Leave table as is, and find a new name for our function.

I think (1) is the best thing to do at the moment, and (2) is probably a better future goal but I would be fine being overruled on that (its single advantage is error-reporting when you accidentally give table an odd number of arguments). I think (2a) is a stupid hack and (3) is bad. As such, here is an implementation of (1). Replace the old '(xdef table ...' definition in ac.scm with the following:

  (xdef table (lambda args
                (let ((h (make-hash-table 'equal)))
                  (define (slave xs)
                    (cond ((null? xs) 'nonce)
                          ((null? (cdr xs))
                           ((car xs) h))
                          (else (hash-table-put! h (car xs) (cadr xs))
                                (slave (cddr xs)))))
                  (slave args)
                  h)))
  ;Usage
  arc> (table)
  #hash()
  arc> (table [= _!a 1 _!b 2])
  #hash((a . 1) (b . 2))
  arc> (table 'a 1 'b 2)
  #hash((a . 1) (b . 2))
  arc> (table 'a 1 'b 2 [= _!b 3 _!c 4])
  #hash((a . 1) (b . 3) (c . 4))

I gave two ways we can already accomplish the behavior somewhat conveniently even if we don't have something sensible like your version of 'table:

  (copy (table) 'a 1 'b 2)
  (listtab:pair:list 'a 1 'b 2)

  (def - args
     (if (no cdr.args)  ;or no:cdr.args with my recent hack[1]
         (* -1 car.args)
         (+ car.args (* -1 (apply + cdr.args)))))

Taking '- as the example: there are a few non-obvious things about it (how it works with more than 2 arguments, or how it works with 1 argument), and people could reasonably end up with different versions of it (I do think the current version is best, but I wouldn't blame anyone for not thinking about the 3-or-more-variable case or for implementing it differently).

This is not a problem if you use '- as a utility in the definition of your library functions and someone else just imports the exported functions of your library and uses them. But if you intend to, say, paste code on the Arc Forum and expect anyone to understand it, or if you want people to be able to download the source code of your library and make improvements to it, then there's a cost to every non-standard definition you use: the reader has to learn it. If the language doesn't provide "map" and "let" and "with" (which are certainly not primitive; in fact, they're defined in arc.arc), then either you don't use those operators (and your code will suffer for it), or you do use them and that's one more thing the reader needs to learn to understand your code. It's not the end of the world, it's not a game-breaker for all projects, but it's one more obstacle to people understanding your code, and if you can get rid of it, then that is all to the good.

That's a good point. ^_^ I don't use Anarki- or Lathe-specific stuff in my examples if I can help it. And I recently mused on[1] the trend for a language to incorporate a lot of features in the standard so as to improve the common ground, so I should have seen that point to begin with.

[1] http://arclanguage.org/item?id=13139

Namespaces solve the problem flawlessly only if you plan for no one to work on or read anyone else's code, ever.

I've sorta been banking on the idea that everyone programs to their own languages of utilities anyway, and that each of those languages can be talked about, accepted as a common topic, promoted by its proponents, and standardized just like language cores can. Certainly CMSes and game engines have those kinds of communities. :)

Multiple independently developed libraries won't always work together well enough to all be clearly canon components of the language's identity, but I think they are more likely to be interoperable than are things designed from the outset to be entirely new languages.[2] So I don't think it's important for the core libraries to be cutting-edge in practice. The core can fade into the background to a degree.

I haven't thought about this stuff very deeply yet, and it's not rooted in a lot of fact, so please feel free to disillusion me. ^^

Lathe already defines (= global.x desired-value) so that it behaves just like (eval `(= ,x (',(fn () desired-value)))). As an everyday expression, global.x behaves like (bound&eval x), which makes (zap _ global.x) a bit more intuitive. This utility works on Arc 3.1, Anarki, Rainbow, and Jarc.

Finally, it's easier to see "symbol-value" written in the language description and use it than to figure out how to use the 'eval method properly. Just look at how complex your definition is; even I, who do see why it's necessary to go ',desired-value instead of ,desired-value in my version [if objs in code worked properly, you could use the former version for a while without noticing it was wrong], I don't see why it's necessary to go (',(fn () desired-value)), and I certainly wouldn't think of it myself. (I still don't believe it's actually necessary, but I won't press the point.) Like the case of (++ (car (pop xs))), this is one of the things a core language is good for: the implementor has considered and avoided all the subtle pitfalls for you. (Actually, implementing it as a primitive with the "namespace-..." functions, there aren't any pitfalls to avoid.)

By the way, I don't know if the core of the language is supposed to be all that axiomatic. It should be clean, certainly, but by its nature it should provide more than minimal axioms. In an essay[1], Paul Graham talks about "the core language—those operators that are neither primitives like car and cdr, nor special-purpose library functions. I mean operators like CL's mapcar, let, remove-if-not, and so on."

At any rate, here's what I think: (a) There will be people who need the equivalents of 'symbol-value and (setf symbol-value)--people who write module libraries or debugging facilities or for whatever reason[2] want to determine names at runtime. (b) This might be enough by itself to warrant placement in the core language, but (c) it is extremely easy, not to mention efficient, to implement these as primitives, while it is subtle, error-prone, and inefficient to implement it with 'eval. Therefore, it should be built into the language. (Technically, if it's a primitive, then it's not part of the core language by the implicit definition from the essay, but whatever you call it, it should be built in.)

I feel better about rambling/ranting when I also produce something tangible, like working code. So, here you go:

  (def symbol-value (x)
    ($.namespace-variable-value $.ac-global-name.x))
  
  (defset symbol-value (x)
    (w/uniq g
      (list `(,g ,x)
            `(symbol-value ,x)
            `(fn (val) (($ namespace-set-variable-value!)
                        ($.ac-global-name ,x)
                        val)
               val))))

[2] I once did the following in a chess program:

  (each p '(r k n)
    (each a '(a b c d e f g h)
      (each n '(1 2 3 4 5 6 7 8)
        (setf (symbol-function (symb p a n))
              (fn () ...)))))

That's why it's in Lathe, too. ^_^ Since it's there to implement the module system, it's one of only a couple dozen Lathe utilities that isn't in a module, but I find it useful enough that I like it that way.

I don't see why it's necessary to go (',(fn () desired-value))

If you embed a literal vector in an expression (for instance, a tagged value), Racket will evaluate that expression as a copy of the vector. So (let foo (annotate 'a 'b) (= global!x foo) (is global!x foo)) would return nil, which I find frustrating. ^^ (There might be significant mutations-aren't-reflected-in-both-copies gotchas too, but I'm not sure right now.)

If you embed a procedure instead, I don't know if it copies that too, but it doesn't matter 'cause a copy of the procedure should still close over the same bindings. In practice, (let foo (annotate 'a 'b) (= global!x foo) (is global!x foo)) works, so I'm satisfied.

By the way, I don't know if the core of the language is supposed to be all that axiomatic. It should be clean, certainly, but by its nature it should provide more than minimal axioms.

This is a point I just acknowledged but challenged at http://arclanguage.org/item?id=13266. I'm not sure entirely whether and how I (dis)believe it, so a response is welcome. ^_^

it is extremely easy, not to mention efficient, to implement these as primitives, while it is subtle, error-prone, and inefficient to implement it with 'eval.

Sure. Implementing 'symbol-value in terms of 'eval is a pretty big abstraction inversion, and it causes lots of unnecessary consing and compilation at runtime. I'm kinda conviced. ^_^

working code

Incidentally, official Arc allows you to say (assign a.b t) and observe (bound 'a.b). Your 'symbol-value makes it possible to get the value of 'a.b again, whereas there's no non-exploit way to do that with 'eval. ^_^ (You can use the (cdr `(0 . ,_a.b)) bug, lol.)

  (obj 'a 1 'b 2)

Again, as you already pointed out, it boils down to flexibility and consistency. I don't see why I would trade saving one character spot over both 1. being clear & consistent and 2. allowing functions to be passed in:

  (obj (look-up-something) 1 (look-up-something-else) 2)

Using the same heiroglyphics as I posted last time, we can implement 'case entirely as a function:

  ; Hypothetically, this uses the macros 'def, 'whenlet, 'iflet, and
  ; 'do; the functions 'if, 'testify, 'get, and 'apply; the special
  ; forms 'fn, 'zap, and 'quote; and the read macros
  ; #,x for (fn (gs123) (zap gs123 x)), #'x for (fn () x), and
  ; 'x for (quote x).
  
  (def #,case (subject . args)
    (whenlet (first . rest) args
      (iflet (then . rest) rest
        
        ; Check the testified result of (first) against the subject.
        (if (.subject:testify:do.first)
          
          ; Return the result of (then).
          then
          
          #'(apply case-part-2 subject rest))
        
        ; Return the result of (first). It's the last else.
        first)))
  
  ; was (case x y (dosomething))
  (case x #''y dosomething)

Though it reminds me a little on how clojure handles inline anonymous functions, which I really like:

    #{+ %1 %2} ;% being the args, %1 being the first, %2 the second, and so on

(case x #(myfun)(dosomething))

yet still able to:

(case x 'y (dosomething))

All the necessary assumptions are listed in the top comment though (assuming you can intuit my intentions for the 'iflet macro, etc.). It's by no means implemented. ^^

There is at least some precedent of special treatment. In JavaScript's object literals, for example, you can do

  {a : 1, b : 2}

  {'a' : 1, 'b' : 2}

  {:a => 1, :b => 2}