Okay, so, I gave this some thought. I might be able to do some kind of hacky thing where when a macro expands, it first checks for variables in the macro's scope. Then it checks for variables in the defining module's scope. Then it checks for variables in the importer's scope.
Sounds really hacky. I may need to use name-munging, but if I implement it at the interpreter level, at least ordinary code shouldn't be aware of it. Ideally it would be completely transparent.
Oh yeah, this is an interesting issue. It's one reason I'm going with hygienic macros in Penknife, besides simple cleanliness. :-p The technique I use in Penknife, as it turns out, is an example of syntactic closures, but I didn't know it until well after I had an okay system in place. I found out when I read this from http://lambda-the-ultimate.org/node/4196:
"Other frameworks for lexically-scoped macros, notably syntactic closures (Bawden and Rees 1988) and explicit renaming (Clinger 1991), use a notion of lexical context that more directly maps to the programmer's view of binding scopes. Unfortunately, the more direct representation moves binding information into the expansion environment and, in the case of syntactic closures, tangling the representation of syntax and expansion environments."
I had exactly this entanglement happening in Penknife at one point. At the time a Penknife macro is defined, the local environment is captured, and later on, syntax generated by the macro is wrapped up in a way that refers back to that environment. At one point I was just putting the environment itself in the wrapper.
For your purposes, that might be just fine, especially if you're just going to eval the macro's result right when you get it. Then again...
PyArc's modules sound exactly like I want Penknife's (not yet existing) modules to be, and you seem to value the ability of the library users to configure the libraries to their own purposes, much like I do. I suspect, in our languages, programs will need to load the same library multiple times for different purposes, especially when it comes to diamond dependencies, where two individual libraries might depend on a common third library but configure it in different ways.
With Penknife's approach to extensible syntax, it turns out parsing is really slow, so I've organized the language so that it can parse a library in one pass and then run the resulting code over and over. ("Parse" basically means "compile" in Penknife, but Penknife does one or two additional things that could be called compilation, so I shy away from that word altogether.) That exposes a weakness of tangled syntactic closures: The compiled representation holds the original captured environment, rather than the one that should be used this time around.
So for Penknife I ended up storing the environment in the macro itself and complicating my environments so that variables were referred to based on what sequence of macro-unwrapping it took to get to them. Since the person using the macro obviously has the macro in their namespace, it comes together pretty nicely. I'm happy with the result, but it's not nearly as simple as (= env!foo 4), so I can't expect you to follow the same path. ^^
In PyArc, right now, we just re-parse and re-eval it, so loading the same module twice is not a problem.
Okay, so, I fixed the issue with it thinking a variable was undefined when it wasn't. I then found out something very interesting:
; foo.arc
(assign message* "hello")
(mac ret () 'message*)
; bar.arc
(import foo "foo.arc")
(foo!ret) -> "hello"
(= ret foo!ret)
(ret) -> error: message* is undefined
So... without even trying to, I automagically made macros sorta-hygienic with modules. When you import a module, and then call it's macros using the foo!bar table convention, it works just fine. The macro will use it's defining environment, rather than the caller's environment. But... if you pull the macro into the current namespace, it dies, because it's now being eval'd in the caller's namespace.
I'm pretty okay with this. It's kind of a wart that macros don't work quite right in that situation, but the fact that they do work when using the table is pretty darn nice. This also theoretically gives the caller the choice of whether to eval the macro in the defined environment, or the caller environment. The only issue, I think, is that this behavior may be confusing, at least until you've been bitten by it once or twice; by then you should have it memorized. :P
Obviously this is just a simple test... the real test will be when it's released and we'll have more complicated macros. Then we'll see if there's still any major hygiene problems or not.
Sounds like you're looking for some kind of hygienic macros, and may have rediscovered the logic behind scheme adopting them in the first place.
It is possible that properly handling modules at the core language level requires that either macros are only ever expanded and evaluated in their original context, or hygiene must be used to encapsulate that context in the macro-expansion. Or leave it without restrictions, and hope people follow the guidelines.
Not all hygienic macro systems are cumbersome or complicated, and it's possible that we could create one that permits selective breaking of hygiene as desired. One candidate would be an implementation of SRFI 72: http://srfi.schemers.org/srfi-72/srfi-72.html
Hygiene has been discussed on this forum, and I think a few systems may exist already.
This is all handled by the interpreter, so normal users don't see it. Anyways, now `ret` would expand to this:
(foo@message*)
Which would refer to the correct binding in the correct namespace. Hm... but then what if the macro actually does want to grab a value defined in bar.arc? Tricky.
I think your question is really "Wouldn't name-munging accomplish hygiene?" :)
What you're talking about is pretty similar to how Racket accomplishes hygiene by way of 'read-syntax. Everything's wrapped up so you know what its source file is, and that's all I know for now. :-p Seems Racket's system is pretty cumbersome, but then that's probably because it's very static, with modules revealing what they export before any of the exported values are actually calculated.
What you're talking about also sounds extremely similar to Common Lisp's approach to namespaces. I don't know that approach very well, but it could act as some sort of example. ^^
