Arc Forumnew | comments | leaders | submitlogin
Interpreter Manipulation?
7 points by Bogart 6001 days ago | 16 comments
It might be interesting if the scheme function that interprets arc was made an arc primitive like car and cdr. This way, one could overload it like any other function, and all special evaluation rules such as macros, special syntax and the defcall rules could be treated in a uniform way. Just thinking...


5 points by cchooper 5999 days ago | link

I think this is a great idea, and should be a standard feature for anything that calls itself a 'reprogrammable programming language'. After all, the ultimate in reprogramability is being able to recode the interpreter itself.

I have a different suggestion for implementing it: instead of redefining the interpreter, have a macro that is called on every expression before it's interpreted. The macro would compile the expression down to canonical Arc, which could then be executed by the standard interpreter. Not sure what the pros/cons are of doing it that way, but it sounds to me like an easier hack than redefining the Scheme function.

-----

3 points by almkglor 5999 days ago | link

efficiency concerns! efficiency concerns! yeah yeah we don't care about efficiency, until it makes stuff 100x slower!

As an example, my ssyntaxes -hack severely slows down 'eval. And that's just part of the program. The slowdown appears to be from the arc compiler occasionally adding 'ar-eval at various points, apparently "just in case".

-----

4 points by absz 5999 days ago | link

It's a very important concern. Now, I don't know the details of the subject, but Steve Yegge just gave a talk on optimizing dynamic languages; he uploaded the transcript at http://steve-yegge.blogspot.com/2008/05/dynamic-languages-st... . Thoughts?

-----

3 points by almkglor 5999 days ago | link

Possibly we can hack apart 'ac and fix the parts where 'ar-eval is randomly inserted, even though they are unnecessary (I think).

However the million dollar question of optimizing an Arc with a redefinable 'eval is the fact that Arc 'eval has TWO steps: compilation and then evaluation. Function definitions are always compiled.

For example:

  ; eval to make everything postfix
  (redef eval (e)
    (if (acons e)
        (given fun (last e)
               arg (cut body 0 -1)
          (old `(,fun ,@arg)))
        (old e)))
  (t 0 5 if)
  (nil 0 5 if)
But how do you handle functions?

  (foo ((x)
      (x
        0
        42
        if)
      fn)
    set)
Do you implicitly insert 'eval to the function body, or not?

-----

2 points by absz 5999 days ago | link

This is actually also the million-dollar semantic question. Right now, arc-eval is not inserted around function or macro bodies, so only top-level forms (those at the repl) are affected. But we can't insert it around bodies as is, because (1) Arc-side eval doesn't take an environment, but they need to pass an environment in, and (2) they don't evaluate things immediately, but instead they create the forms that will be evaluated. What is the best way to do things here? We have no other language to guide us, after all.

-----

2 points by skenney26 5998 days ago | link

I'm curious: So arc-eval translates your arc code into scheme code and then passes it to scheme's eval? If so, would it be possible to print the scheme code to a file before it gets evaluated?

-----

1 point by almkglor 5998 days ago | link

> I'm curious: So arc-eval translates your arc code into scheme code and then passes it to scheme's eval?

Yes, I think so.

> If so, would it be possible to print the scheme code to a file before it gets evaluated?

Yes. sacado did some work on this I think.

-----

1 point by almkglor 5998 days ago | link

Personally, I'm rather squeamish about letting 'eval be redefined. Because the redefined 'eval will not be consistently used.

Perhaps it's better to just define a top-level which preprocesses the code before passing it to 'eval?

-----

2 points by eds 5998 days ago | link

That could be accomplished by redefining 'arc-eval rather than 'eval. 'arc-eval is used internally for evaluating top-level forms but not really anything else since it isn't exposed to arc, whereas 'eval could be used by arbitrary arc code. If you want to redefine the top-level evaluation function, than maybe what you want is some way to influence the results of 'arc-eval.

-----

1 point by almkglor 5998 days ago | link

So much easier:

  (def mytl ()
    (pr "arc> ")
    (eval (my-transformation (read)))
    (mytl))

-----

2 points by eds 5998 days ago | link

I guess I misunderstood your intention. I thought you wanted to be able to redefine the existing toplevel used by the arc interpreter itself, not define a completely new toplevel to be used in user code... (and yes, the latter is quite trivial).

-----

1 point by almkglor 5998 days ago | link

> I thought you wanted to be able to redefine the existing toplevel used by the arc interpreter itself

As opposed to emulating the existing toplevel in user code?

Allowing a redefinition of 'eval to work on the REPL will probably be OK, but still, there would be an impedance mismatch with functions that were defined before 'eval was redefined.

-----

2 points by eds 6000 days ago | link

You mean 'eval? :)

-----

3 points by absz 6000 days ago | link

Not quite—from Bogart's post, it sounds like they want to be able to write (= eval my-magic-fn), and then (+ a b) would be passed through the new eval too. This would be interesting; I've been thinking about similar things.

-----

3 points by eds 6000 days ago | link

Ok, so you can't just redefine 'eval, you need to redefine 'eval and have MzScheme use that function on the Scheme side of things as well.

It looks like the 'arc-eval function is basically the one you want to redefine, since that is the one that is called inside the definition of 'tl2. In fact, the definition of 'eval is exactly the same as 'arc-eval except 'eval calls 'ac-denil on its arguments before compiling them. So basically what you would want to do would be to look up the value of the arc-side eval function and call that instead of hard coding a call to 'arc-eval. Right?

-----

6 points by absz 6000 days ago | link

It sounds like that should work: change the xdef of eval to

  (xdef 'eval (lambda (expr) (arc-prim-eval (ac-denil expr))))
, and then change arc-eval to

  (define (arc-prim-eval expr)
    (eval (ac expr '()) (interaction-environment)))
  
  (define (arc-eval expr) 
    ((eval '__eval) expr))
. The xdef should come soon enough to prevent something from calling a non-existent eval. Then, in Arc, you should be able to do

  (redef eval (expr)
    (prn " - debug - " expr)
    (old expr))
or whatever. Let me test that...

...

...yes, this works! Well, almost. I didn't really test it, but (a) ordinary things seem to get evaluated fine, and (b) the message is printed. It doesn't seem to hit everything in a macro-expansion, but I can't tell whether or not it should. For instance, I get (output reformatted for clarity)

  arc> (mac do+ (a b) `(do (prn ,a) (prn ,b) (+ ,a ,b)))
   - debug - (mac do+ (a b) `(do (prn ,a) (prn ,b) (+ ,a ,b)))
   - debug - (input-history-update
               (mac do+ (a b) `(do (prn ,a) (prn ,b) (+ ,a ,b))))
   - debug - (output-history-update
               (quote #(tagged mac #<procedure>)))
  #3(tagged mac #<procedure>)
  arc> (do+ 1 2)
   - debug - (do+ 1 2)
  1
  2
   - debug - (input-history-update (quote (do+ 1 2)))
   - debug - (output-history-update (quote 3))
  3
. (Of course, its output didn't have extra line breaks and had quasiquote instead of `, etc.) I feel like it should print

   - debug - (do (prn 1) (prn 2) (+ 1 2))
too.

<EDIT>

The problem is that ac-mac-call calls ac directly so that it can pass the current environment as a second argument. Perhaps arc-eval should take an optional environment argument too? But that would clutter things unnecessarily on the Arc side... anyway, the point is, after looking at it for all of five minutes, macros are tricky. (And I haven't even looked at ac-macex yet...) For instance, a simple test:

  arc> (nil 1 2)
  Error: "string-ref: index 0 out of range for empty string"
  arc> (redef eval (expr)
         (if (and (cons? expr) (~car expr))
           (cdr expr)
           (old expr)))
  #<procedure: eval>
  arc> (nil 1 2)
  (1 2)
  arc> (mac donil (a b) `(nil ,a ,b))
  #3(tagged mac #<procedure>)
  arc> (donil 1 2) ; Should return (1 2), but:
  Error: "string-ref: index 0 out of range for empty string"
</EDIT>

The downside is that, if it isn't slow now, it might well prohibit certain optimizations (though reading Steve Yegge's recent talk makes me think that I might well be wrong). Is this worth pushing on Anarki?

-----