http://www.lispworks.com/documentation/HyperSpec/Body/f_fn_l...

Which does something similar, but in a non-mutable way. It also requires some additional parsing, because it conflates the argument list and the body together. It's also using multi-value return. Assuming `function-lambda-expression` was available in Arc (but renamed to 'get-fn-attributes), the following two would be equivalent:

  ; function-lambda-expression
  (let '((nil arguments . body) environment) (get-fn-attributes foo) ...)

  ; message passing
  (let '(arguments body environment) foo ...)

1) Mutability. If we're destroying information hiding by making the information available in the first place, might as well make it mutable too, right? Better than adding in a `set-fn-attributes` function...

2) It makes it a lot easier if you only want to access one of the bits of information. Observe:

  (let body '((nil nil . body)) (get-fn-attributes foo) ...)

  (let body foo<-body ...)

  (let '((nil a . b) e) (get-fn-attributes foo) ...)

  (with (a foo<-arguments b foo<-body e foo<-environment) ...)

On the other hand, lists are nice because they preserve their order, they're shorter and easier to create than tables, and they're less complex.

But, we're talking about the core here, so I'm gonna stick to tables for the core datatypes, until I see a better idea.

I'm not sure slots for arg lists and body are the right separation. I can't imagine changing the arglist without the body, for example. Except maybe to add or remove defaults for args. In any other situation I'd want to change both at once. Might as well just run def all over again.

I don't know. But at the same time, I tend to follow pg's mentality that if I can't find a logical reason to disallow it, then I should allow it.

So a better question would be, should we not allow this functionality, and if so, why? You can argue that my particular solution is kludgy, hacky, or any other pejorative term, but that doesn't answer the question, "is this a good idea or not?"

Obviously I wouldn't expect this function mutating to be common place. In case it wasn't obvious, I'm mucking around with low-level special-case stuff here. Stuff that just isn't possible in pgArc. A kludgy solution that works is better than no solution, in the situations where you need it.

One of my goals with py-arc is to get as much as (reasonably) possible written in Arc itself. Once you go down far enough, you eventually get to low-level "assembly kludge". Do you have a better idea for a way to get function inspecting/mutating (or something equivalent) in a way that's better than my proposal?

py-arc isn't even at 1.0, and implementations can change. I'm much more interested in knowing whether the idea itself is good or not. Though, if you do have a suggestion for a better implementation, I'd like to hear that too!

---

"I'm not sure slots for arg lists and body are the right separation. I can't imagine changing the arglist without the body, for example. Except maybe to add or remove defaults for args. In any other situation I'd want to change both at once. Might as well just run def all over again."

We're talking about the situation where you're dealing with somebody else's code. Obviously if it's your module, you can just change it yourself, but imagine somebody else writes a library, and you want to add a feature, or fix a bug.

If the library isn't designed well, the only way to do this might be to mutate a function, so calling `def` isn't an option. Or perhaps the library is designed well, but the particular situation the library is trying to solve requires mutating functions.

Ordinary Arc code wouldn't use function mutating, but perhaps somebody has a special use-case for it. Imagine a debugger, or a profiler, or a logger. Something that needs to inject itself into other people's code in a way that works perfectly every time. Mutable functions might be useful in that situation.

A code walker could also make use of the `environment` attribute to gather information about the function's closure. rocketnia mentioned a potential use-case for this[1].

I don't know exactly where this might be useful, but I think there's at least one situation out there where it would be useful. And since this would be really easy to do given py-arc's architecture[2], it just makes sense to expose it to Arc, unless there's a good reason not to[3].

---

* [1]: http://arclanguage.org/item?id=14411

* [2]: py-arc internally uses this idea. Every Arc function has an opts, rest, and body attribute. Calling a function evals the body attribute, and returns the last value (it's a teensy bit more complicated than that, but that's the basic idea). So it's simply a matter of exposing the internal details to Arc, and message passing can do that easily.

When I said 'hack kludgily at', I was thinking, "can we find a better way?" So combining the two I'd rephrase my original statement as:

"Can you give some examples of where this is a problem? Assuming it's a problem, I'm sure we can come up with more elegant solutions."

The reason I didn't give examples is because I didn't have any. It was just a, "oh hey we can make Arc more hackable if we do this" sorta thing. I figured we'd find use-cases for it later, and, well, we did.

As for a more elegant solution... this solution is using message passing. So to get the environment of the function `foo`, you would use this:

  foo<-environment

  foo<-environment<-outer

  (def get-closure (x)
    x<-environment<-outer)

---

"My apologies for the tone."

And because I'm sorta kinda writing an interpreter, not only am I thinking about low-level stuff, but I'm actually implementing it, so my posts and "great ideas" will probably be about low-level stuff for a while.

So, don't worry about it. You're up there in Arc-land thinking about high-level stuff, whereas I'm down here in low-level Python, thinking about low-level stuff. We both need to try to remember that better.

I'd still be interested in hearing if you come up with a better idea.

"If the library isn't designed well, the only way to do this might be to mutate a function, so calling `def` isn't an option. Or perhaps the library is designed well, but the particular situation the library is trying to solve requires mutating functions."

I don't follow this.

Adding features to a function is the whole point of extend. Can you think of an example where it wouldn't suffice?

"if I can't find a logical reason to disallow it, then I should allow it."

That isn't a defense to the question, "is this the right representation?" Even PG isn't allowing non-sexp syntax, as an extreme example. I think "is there a better way to do this" is a fine logical reason.

Hm... yes, actually. Imagine a function that is wrapped in a closure, to hide data:

  (let hidden-data ...
    (def foo ()
      (do-something hidden-data)))

Technically speaking, you wouldn't need mutable functions, just the ability to see a function's closure. For instance, you could use this:

  (let closure foo<-environment<-outer
    (extend foo ...))

  (= foo<-body ...) 

  foo<-bar 
  (get-attribute foo 'bar)

"I think "is there a better way to do this" is a fine logical reason."

Sure, but I don't know of any better ways, nor has anybody presented them, so for now I'm focusing on whether the idea has merit. We can fix the implementation later.

P.S. If you consider this idea to be kludgy, then you probably consider message passing in general to be kludgy. :P How would you like to be able to access the data, then?

---

"But you've gotten me thinking about alternatives :)"

That's a great example, thanks.

"Sure, but I don't know of any better ways, nor has anybody presented them, so for now I'm focusing on whether the idea has merit."

Yeah that makes sense.

- if you wanted to serialize closures, you could easily implement a serialization strategy simply by inspecting the closure's environment yourself.

- writing a debugging or tracing facility becomes so simple that you could just throw together an ad-hoc solution to a particular need whenever you wanted, instead of having to rely on the debugging facilities offered (or not) by the runtime.

- for a programmer who is new to closures, having everything be completely transparent makes it easy to see what's going on.

One thought I've had is that it could be useful to write an Arc interpreter in Arc. It would be slow of course... but the highest level functions in a program (like the ones that we typically want to serialize to persist closures) often don't need to be fast... as long as the operations that they call (such as in particular anything being done in a loop) are themselves fast.

This would give us four levels of implementation:

- Arc interpreter

- Arc compiler

- Racket

- C

Each higher level is more expressive but slower than the level below it. Thus you can write something at a higher level first because that will be the fastest for you (it will take you less time to write the program), but then if it turns out to be too slow, you can rewrite it at a lower level at the cost of doing more work.

1) Slowness. Then again, Arc's goal isn't speed either. As you mentioned, it should be possible to rewrite the slow bits in a lower layer if needed.

2) Lack of information hiding. This is more serious than point 1, but, I can provide a different mechanism for information hiding. The downside of having a different mechanism is that it requires learning a new mechanism... and I love that closures can be used for information hiding. It's just so simple and axiomatic, you know?

---

"One thought I've had is that it could be useful to write an Arc interpreter in Arc."

I've been mulling that over too, ever since your earlier comment[1], but... I don't really like the Racket runtime (the implementation, not the language). For instance, trying to make Unix shell scripting work with Racket is a big pain, but it's incredibly easy in Python.

Of course, I don't really like Racket the language either. :P Too big and bloated for my tastes. Why do you think I like Arc to begin with? On the other hand, programming in Racket would probably be preferable to programming in Python, if I had to choose between the two.

Oh yeah, and Python seems to be a lot more popular than Racket... so chances are, your users will have Python installed, but not Racket. Proooobably not a big deal, since we're a pretty small group, but still an advantage for Python.

---

"Each higher level is more expressive but slower than the level below it. Thus you can write something at a higher level first because that will be the fastest for you (it will take you less time to write the program), but then if it turns out to be too slow, you can rewrite it at a lower level at the cost of doing more work."

Yes! And don't forget: profile, profile, profile first. I think that's the best way to make good clean code that is also fast. Honestly, though, I rarely need to actually optimize my JavaScript code, because it's usually plenty fast as-is. Chrome's V8 engine is amazingly fast, especially given how dynamic JavaScript is.

---

In what way? Running a Racket program as a shell script, or calling shell scripts from Racket? (I haven't found either particularly difficult... though I can easily imagine that the process might be smoother and better documented in Python).

This works in Anarki (at least, I think so, I haven't tried it), because they made an arc.sh file, and used some kludgy stuff to make it work[1], but in Python it Just Works(tm), no fiddling needed. That's actually true for Arubic[2] as well, because it's written in Python and I designed it that way. You can just use #!/path/to/arubic and it'll Just Work(tm).

I'm probably overly-criticizing Racket for that because I ended up partially implementing arc.sh by myself, then found a forum post that gave a better implementation, swiped that, then modified it. I doubt I'd have cared as much if arc.sh had been included with Arc 3.1 (and worked good).

It's not as big of a deal now that Anarki has arc.sh, but it's just one of those rough-around-the-edges areas. Python also makes it easy to parse command line switches, with the optparse module, though that could be implemented as a library in Arc, so it's not really fair to criticize Racket for that. :P

I'm curious, though, how would you write a Racket program that worked as a shell script?

---

"with respect to Racket vs. Python, an advantage of Arc's axiomatic approach is that it makes it really easy to implement Arc on top of different platforms."

Not in my experience. Arc actually has a lot of primitives[3], not to mention the whole thread/continuations/TCO thing. Implementing a simple toy Arc interpreter? Piece of cake. Making it actually good? A lot harder. That's another reason I'm trying to shove as much into Arc as I can... the more stuff in Arc, the less stuff you need to write in the interpreter layer.

Actually, a lot of my time has been spent debugging small bugs... I've had to wade through a lot of tiny bugs to get Arubic working properly in all sorts of weird edge-cases. Unit tests help a lot with that.

---

* [1]: Actually, I just took a look at Anarki's arc.sh and it looks pretty clean. I'm not sure if it behaves correctly, but I don't feel like testing it to see.

* [2]: I'm renaming py-arc to Arubic, because I like that name a lot. This also gives me some more flexibility... I'm no longer tied to Python. I could implement Arubic in Arc, or Ruby, or another language.

* [3]: Compared to most other popular languages, Arc has very few primitives, but it still has a lot more than I'd like. One thing I did like, though, is that it was incredibly easy to write a tokenizer/parser for Arc, because it has such a regular syntax, due to being a Lisp.

Ah, I don't think this is a Racket problem. Racket has good information on how to run a Racket program as a shell script: http://docs.racket-lang.org/guide/scripts.html

By the way, Racket also has a decent command line argument parser: http://docs.racket-lang.org/reference/Command-Line_Parsing.h...

Of course, to say that Arc doesn't make this easy enough is certainly a valid criticism. (But I don't think we can blame Racket for that).

not to mention the whole thread/continuations/TCO thing

Fair enough. I had somewhat assumed that the problem was Racket, because Arubic handles it just fine. My mistake.

That actually looks like a pretty good command line parser. Thanks for the links!

P.S. Once ar gets into a somewhat more complete state, you may want to consider making shell scripts work good with it. For instance, if I'm in the directory /foo and I run a script that's in /usr/bin and the script uses (load "lib/bar.arc"), I would expect it to load /usr/bin/lib/bar.arc, not /foo/lib/bar.arc.

I made this work in my copy of pgArc, but it'd be nice to have it be more standardized. Obviously it should be possible to load relative to the current directory as well, if your script wants to.

Mind if I go in and start hacking on ar? :P If I decide to make an Arc compiler in Arc, I'd probably base it on ar.

---

"The advantage of an axiomatic approach is once you have them implemented, the rest of the system will run on top of them... but the powerful axioms may be hard to implement."

Please do! (that's why I put it on github :)

By the way, the hackinator (https://github.com/awwx/hack) does generate an Arc program that can be run from the shell from any directory (for example https://github.com/awwx/hackbin/blob/master/hack works this way). I just haven't gotten around to getting it into ar yet.

I could still allow for mutating the arguments list and environment, since I don't think that would leak hidden information (if you know of a way, I'd like to hear it!). Of course, if I went that route, then I should give a way for Arc code to "freeze" an attribute, making it read-only.

Side note: if every function had an `arguments` attribute, then the `sig` table in Arc would be redundant, and could be removed.

---

* [1]: Why would the `body` attribute need to be read-only? Consider this simple example, demonstrating information hiding:

  (let a 0
    (def foo ()
      (if (< a 3)
        (++ a))))
        
  (foo) -> 1
  (foo) -> 2
  (foo) -> 3
  (foo) -> nil