Your understanding of this is quite correct.

But this is how it's equivalent to, not necessarily how it's implemented. AST-traversal?

The problem with the nested-environment implementation is that mere variable lookup can be O(N) where N is the number of nested environments:

  (def foo (x)
    (fn (y)
      (fn (z)
          ;accessing x takes 2 indirections!
        (+ x y z))))

  (def foo (x)
    (fn (y)
      (let tmp (+ x y)
        (fn (z)
          (+ tmp z)))))

A faster way would be to "flatten" the environment. In effect, what's really done is:

  (def foo (x)
    (fn (y)
      (fn (z)
        (+ x y z))))
  =>
  (def foo (x)
    (closure (list x)
      (fn (y)
        (closure (list (my-environment 0) y)
          (fn (z)
            (+ (my-environment 0) (my-environment 1) z))))))

------- BUT!

Just because environment flattening is how it's usually implemented doesn't mean it's not possible to capture and name an environment.

For example consider that what we call a "function" is really an object constructed by a 'closure very-hidden function. It's really a pair of the environment and the function code. Similarly if Arc had ways of destructuring a function, we could implement a call-with-current-environment function. Suppose we had a 'environment-of function which extracts just the data of a given function:

  (def call-with-current-environment (f)
    (f:environment-of f))

  (mac w/environment (var . body)
    `(call-with-current-environment
       (fn (,var) ,@body)))

This has been implemented. Take a look at lib/module/module1.arc . But aside from having problems with macros, there's also....

> I've been thinking again ^^, and I just realized that though this might be an interesting way to implement packages, I don't think it would fix your typing issue.

Quite right ^^

> Maybe variables should all carry around info of which scope they're from (kind of what you were suggesting)?

> Err. I don't quite understand this. Also, I don't quite understand how this realates to symbols-as-types...

Well, what I had originally thought when I said this was rather silly, which was that in the value of the variable there should be information about which scope it's located in. Doh! Maybe that would work if it looked in the current env to figure out where the "real" var is.

What I now think is something very similar to what you've been saying, except I still think it would be cool if this could be a method of managing general environments, instead of just modules. I don't know if that was what you were thinking, or not; it's more of an implementation detail.

I do like the idea of being able to extract and name the environment at any point in the code. If all that a package was was an environment with a name and we had the ability to access items from each environment directly, it could (possibly?) be useful for a lot more than just modules.

I really like the syntax (env sym), where env is the environment object, and 'var is the symbol we're looking up. This way env.sym works. That's probably a syntax that many people are used to. I just don't know if it would work or not.

For your typing problem, I would do what you recommended. If it made any sense, I would use the syntax env.sym, so that it can be evaluated to figure out what type it is. You would still have to evaluate everything in head position to see if it evaluated to a macro. On the other hand, you could just check the car of the first item, and see if it was an environment. Or you could treat the whole thing as a single symbol and have it bypass the reader? How does arc's typing system work?

Also, how much would changing the type symbol to <env>sym help? How would it help the code decide how to interpret it?

I just don't know enough about scheme's pointer and environment system, or arc's typing system to know if any of my thoughts are even going in the right direction. So I just make more of them to cover the bases, and have you explain why none of them work. ^^

Basically at this point, I'll agree with anything you say, because I don't know enough to disagree with you. I would be very interested in learning/having you teach me all of the details about arc's implementation. As always, I'd love to help, but I don't know nearly enough to be useful.

Maybe we should start a new thread for the module system that you're proposing?

As per one of you previous comments: If symbols weren't global, how would you do them?

Yes, it would definitely be cool to access the environment. For one, it would make serializing functions possible: aside from giving accessors to the environment and to the function code, you can give constructors for functions that accept a serializable environment and a serializable function code.

> If it made any sense, I would use the syntax env.sym, so that it can be evaluated to figure out what type it is.

Typical 'isa pattern:

  (isa foo 'cons)

  (isa foo 'arc.cons)

First things first. We want to make the type arc.cons readable, don't we? So if say 'arc here is a macro, it will expand to a symbol, which is basically "the symbol for cons in the arc package".

Now that symbol can't be a 'uniq symbol, since those are perfectly unreadable.

What we could do is....... oh, just make up a symbol from the package name and the given symbol. Like, say...... <arc>cons. The choice of <> is completely arbitrary.

Now.... if (arc cons) is a macro that expands to <arc>cons, why go through the macro?

> And how is the isa statement going to be updated to work independent of which module it's running in? Does each module type all of it's objects that way from the beginning, or were you going to have the interpreter do something fancy?

Something fancy, of course. Specifically it hinges on the bit about the "contexter". Remember that in my proposal I proposed adding an additional step after the reader, viz. the contexter.

Basically the contexter holds the current package and it puts any unpackaged symbols it finds into the mapping for the current package.

So for example we have:

  (in-package foo)
  (using <arc>v3)
  (interface <foo>v1
    make-a-foo foo-type is-a-foo)
  (def make-a-foo (x)
    (annotate 'foo-type x))
  (def is-a-foo (x)
    (isa x 'foo-type))

Initially the contexter has the package "User".

It encounters:

  (in-package foo)

  t

Then it accepts:

  (using <arc>v3)

  def => <arc>def
  isa => <arc>isa
  annotate => <arc>annotate
  ...etc.

  t

  (interface <foo>v1
    make-a-foo foo-type is-a-foo)

After creating the interface, it then returns:

  t

  (def make-a-foo (x)
    (annotate 'foo-type x))

  (<arc>def <foo>make-a-foo (<foo>x)
    (<arc>annotate '<foo>foo-type <foo>x))

Then the contexter accepts:

  (def is-a-foo (x)
    (isa x 'foo-type))

  (<arc>def <foo>is-a-foo (<foo>x)
    (<arc>isa <foo>x '<foo>foo-type))

So, how hard is it to take what you've made so far, and generalize it to work with any kind of environment, and not just packages? So, things like destructuring functions, naming environments, passing them around, etc. Would this allow us to avoid shadowing variables by naming which level they came from explicitly?

Also, if I type in the symbol '<arc>bar, will the contexter read that and presume I'm looking for the bar in arc, or will it rename it <foo><arc>bar? And which is better?

Hmm. Well, if you want to be technical about things, one Neat Thing (TM) that most compilers of lexical-scope languages (like Scheme and much of CL, as well as Arc) do is "local renaming". Basically, suppose you have the following code:

  (fn (x)
    (let x (+ x 1)
      x))

  (fn (x@1)
    (let x@2 (+ x@1 1)
      x@2))

  (fn (<fn>x)
    (let <fn/let>x (+ <fn>x 1)
      <fn/let>x))

In fact I am reasonably sure that Scheme's hygienic macros work after local renaming, unlike CL's which work before renaming. You'll probably have to refer to some rather turgid papers though IMO, and decompressing turgid papers is always hard ^^.

> Also, if I type in the symbol '<arc>bar, will the contexter read that and presume I'm looking for the bar in arc, or will it rename it <foo><arc>bar? And which is better?

   (let list '(this is a test case)
       (all is-a-foo (map make-a-foo list)))
   

   (sym "make-a-foo")

Assuming you mean in the same file with the example foo package:

  (<arc>let <arc>list (<axiom>quote (<arc>this <arc>is <foo>a <foo>test <arc>case))
    (<arc>all <foo>is-a-foo (<arc>map <foo>make-a-foo <arc>list)))