The second reason is that the only (AFAIK) example of standardized hygienic macros - scheme - is a hell of a lot harder to use than it really needs to be.
The third reason is that unhygienic macros are more powerful, because they LET you subvert lexical scope if you want to. Sometimes macros want to mess around with variables you didn't explicitly pass in as arguments but which are not available in the scope of the macro's definition. Rare, but it happens.
For example, what if a macro wants to define something that is not passed in as a symbol, and have it available in the scope in which the macro was invoked? All of the anaphoric macros (eg. "aif") are of this type, and they cannot be done in a pure hygienic system.
IMO, the best idea would be to have a system which is hygienic by default, but in which you can explicitly say "hey, I want to use this variable dynamically". In fact, I also think this is the way that scoping should work in general: sometimes you'd like to be able to bind a variable dynamically - for example, you want to debug a function call within a certain period of code execution, so you dynamically bind the symbol to a wrapper function. This can be emulated by setting it beforehand and unsetting it afterward, but this doesn't handle call/cc, threading, etc. correctly.
However, lexical scope only for variables is usually "good enough", and it's simpler to implement than a hybrid system; likewise dynamic scope only for macros is usually "good enough", and it's simpler to implement than either a hybrid or a lexical system. (In fact, it used to be thought that dynamic scope for variables was "good enough", and it is easier to implement [for lisp at least: just keep a running alist-stack of variable bindings] than either lexical or hybrid scope.)
Okay, I think I get what's going on with macros. So the idea of current macros is that they can only insert references to variables in the scope of wherever they're expanding. Therefore they do operate purely on syntax. If they want to insert references to functions, you need to make sure that the functions are bound in the expansion scope, but on the other hand, the macros are conceptually clean and really really easy to implement.
For what you're saying, I suggest the old MIT Scheme macro system as an example (before they implemented Scheme macros correctly): essentially, a macro is a function of three arguments - the form it's applied to, the environment of that form, and the environment that the macro was defined in. There is a procedure to return a "symbol" that refers specifically to a name bound in a given environment, so you can make specific references to it. The macro procedure returns some sort of code (just like normal lisp macros), but it can optionally include some of these special "symbols" that refer to bindings in a specific other environment.
That is much more complicated to implement, though, and requires environments as first-class objects.
Well, I wouldn't say "on syntax", I'd say "on code" - and code only contains symbols, not "references to variables in a specific scope"; scope is determined by context - the surrounding code, which the macro can't alter: you can't substitute code into other code while retaining the same surrounding code, that's a contradiction in terms! But this is just terminology.
The old MIT Scheme macro system seems interesting from what you say - is there any place I could go to find an implementation which has this behavior? Or at least examples of code which use it? It seems like it lets code do precisely what I said it couldn't above: contain "references to variables in a specific scope", which is pretty cool. I don't think you'd need to implement environments as first-class run-time objects, merely second-class compile-time objects, with this technique, unless you also allow macros themselves to be first-class.
Okay, I think I'm starting to see. There is quite a big difference between the way Lisp people think of macros and the way Scheme people think of them.
From the documentation, I think that the current version of MIT scheme has this behavior, so look at http://groups.csail.mit.edu/mac/projects/scheme/. (By the way, in case you run Ubuntu, the version of MIT Scheme in the repositories is broken for me.) Look in the documentation for macros (under "Special Forms"), and it's their non-standard low-level macro system. If you're interested in stuff like that, you should also check out syntax-case, which I don't know much about, but I understand is the new, cool way to write Scheme macros. It includes hygienic and unhygienic functionality. Google search for "syntax case" and you'll get some documentation.
The more I look at it, though, the more I think that Scheme macros solve a different problem than Lisp macros. I don't know what it is yet, but it would be interesting to know.
I think you've hit the nail on the head. Hygenic macros and unhygenic macros are very different things (unlike dynamic vs lexical scoping, which are just different ways to create a function). Lisp macros are 'true' macros (Wikipedia: "Macro: a set of instructions that is represented in an abbreviated format"). Hygenic macros are more like a new abstraction that was inspired by Lisp macros.
Well, I'd rather not argue about what 'true' macros are, but I would point out that your definition is basically data compression for programs (which, by the way, I think is an interesting approach to take to programming language design). I'm pretty sure both types of macros and normal functions would all fall under it.
As for the hygienic vs. unhygienic difference, unhygienic macros are certainly easier to define: they rearrange source code into other source code.
The one thing I can think of that hygienic macros can do that unhygienic ones can't is that while they are rearranging source code, hygienic macros can insert references to things that aren't bound to any variable in the expansion scope. The common example I've seen for this is that it lets you protect against people redefining your variables weirdly. For instance, if you insert a reference to 'car', it means whatever 'car' meant where you defined your hygienic macro, even if 'car' has been redefined to be something crazy in the place where your macro is used. The Scheme hygienic macro system also has a way to break hygiene if you want to, so it can do everything other Lisp macros can do.
I guess the question then is, is it useful to be able to do that?
And if you decide you want to be able to do that, are Scheme-style hygienic macros the right way to go about it?
(One option would be to just let you insert objects straight into forms, instead of symbols that you think should reference those objects. This would be fine unless you wanted to be able to set those things later, in which case you'd need some way to get at the actual underlying variable boxes.)
> That is much more complicated to implement, though, and requires environments as first-class objects.
Given the interpreted nature of Arc, first class environments shouldn't be too hard to implement, but in a compiled implementation it would be a lot more difficult.
They could not be implemented on top of the existing arc-to-scheme translator, because that's what it is: an arc-to-scheme translator, not an interpreter. Scheme doesn't have first-class environments, so we can't shove them into arc without writing a full-fledged arc interpreter.
If the environments are temporaries created only while processing macros, and are discarded afterwards, they don't necessarily have to be difficult for compilers.
For what it's worth, (PLT) Scheme can in fact do what you require. After all, there's an implementation of defmacro that ships with it (though it might be called something else)! And there's a way (if I recall correctly, a very long, verbose way, shoring up your second reason) to implement aif and the like in their system by "requesting a shadower" or something like that.
