Supporting first-class macros is like adding 'eval to a language: it means there are always corner-cases which will be essentially uncompilable (you will always need the interpreter), but it doesn't mean you can't deal with the vast majority of cases efficiently. Type inference, already useful for compiling a latently-typed language like Arc or Scheme, could be extended to cover whether an expression in functional position might produce a macro. If it could, you'd have to fall back on the interpreter, otherwise not. In most cases, objects in functional position are globally-bound symbols or (fn...) expressions, which are trivial cases. The main exception is higher-order functions. Inlining or specializing those can fix this, and is often a good way to increase efficiency anyway.
Certainly a problem, but not an insurmountable one.
If you have a compiler available at runtime, you don't need an interpreter to support eval: you can just compile the code at runtime(once you have the code to eval) and then load the compiled code into the system.
Hmm. I suppose we would need some sort of caching system to handle cases where a macro is modified.
Basically we don't compile sub-expressions until they're actually executed, and if they are, we check for macros in the expected places and then compile.
Then if a sub-expression is entered again we check if the macro code has changed and re-perform the macroexpansion if it has.
Caching seems necessary. A simple memoization could work, but hashing long pieces of code could be pretty slow. Maybe a bit telling if the macro has changed after its last execution would be faster.
It could be done. Still, this feels like bending over to support something that I don't actually think will be really, really necessary. The pasted http://arclanguage.com/item?id=7451 seems good enough for making macros-in-modules, if you don't want to hack off symbols (i.e. the CL packages way).
As an aside, I think we do want to hack off symbols the CL packages way. Why? Well, suppose module1 wants to define an internally-used type called my-type, and module2 wants to define an internally-used type called my-type, too. If module1's symbols are the same as module2's symbols, then their entries in the call* table will conflict. But if module1's my-type is really 'my-type@module1 and module2's my-type is really my-type@module2, then they can both have their own types.
One simple and elegant solution to the latter problem is to use uniq'ed symbols for internal types. Or use an actual data structure, if you want the type to carry information.
Moreover, what happens if module1 depends on module2? Then, unless you have some way of hiding the symbols a module defines, they'll end up being the same symbol anyways. Of course, this (defining exports explicitly) is how CL handles things, but it's not very much in the style of exploratory programming. In fact, that's why I dislike CL's package system in a nutshell: it requires too much attention, attention I should be giving to code.
> One simple and elegant solution to the latter problem is to use uniq'ed symbols for internal types.
But this would have to be an idiom, and the Lisp way is, there is no idiom: something has to encapsulate this away automatically.
If it's an idiom, it needs a macro.
> Of course, this (defining exports explicitly) is how CL handles things,
I would have thought this was safer. Basically, I'd define my code first outside of any package/module, then when I've got it mostly working and want to release then I'd put it in a module, hiding away my internal functions (which I might want to change later, and which I define as being "implementation-specific").
And then, instead of (annotate 'my-internal-type foo), you do (annotate my-internal-type foo).
> I would have thought this was safer. Basically, I'd define my code first outside of any package/module, then when I've got it mostly working and want to release then I'd put it in a module, hiding away my internal functions (which I might want to change later, and which I define as being "implementation-specific").
Naming conventions (eg. preceding dash or underscore for "private") are simpler than export lists, and don't restrict a power user from using "implementation-specific" functions if s/he has unanticipated needs - as someone inevitably will. They might cause name clashes, but unless you're using a symbol-mangling module system, this doesn't cause problems.
I also have various other issues with the CL package system, most of which are CL-specific and unrelated to the "mangling symbols" method of implementing modules in Arc. We could go back and forth with this argument for a long time, but it seems rather fruitless to do so, especially compared to actually implementing the ideas.
> And then, instead of (annotate 'my-internal-type foo), you do (annotate my-internal-type foo).
The problem here is that the new type name is completely unreadable. What if the new module needs several new types? What if the module would very much like to use 'defcall on its types? 'defm?
IF something is being done in a language in a particular way, there's probably a reason why it's done that way. It might not be a good reason, but first we need to analyze it. As far as I can grok there are very good reasons for CL packages to hack on symbols, and they all have to do with the fact that symbols can be used for lots of things: function name identifiers, macro name identifiers, global variables, named enumerations.
> The problem here is that the new type name is completely unreadable. What if the new module needs several new types? What if the module would very much like to use 'defcall on its types? 'defm?
Unreadability could be solved by making a 'uniq function that takes a prefix instead of "gs", which 'uniqtype would use. If the module needs several new types, then it creates them; I don't see the problem here. The fact that 'defcall and 'defm assume static type names can be hacked around via 'eval, if necessary; but it's inelegant to have to do so. The reason why all this is inelegant is because everyone has assumed hardcoded symbols are the way to go for types. Only by layering something on top of that, something that makes hardcoded symbols not what they appear to be, such as a CL-like package system, can you get around this without changing existing conventions.
My conventions are different, probably because I'm partial to languages like Ruby, Python, and Smalltalk, where types are structured objects and not simple names. For example, my instinct on making tagged objects callable would be to use ((tag <object>) 'call) as the caller function. Then tables, functions, or even tagged objects could be used as types. This would eliminate the need for 'call*.
Of course, we have 'defcall rather than the above, so obviously this is not the one and only true way to do things. I honestly don't know what the best way of solving these problems is. My initial guesses seem to differ from yours, but that's all they are - initial guesses. I'm not bound to them. But it's hard to do a comparison when all you have are hypotheticals, and no implementation to play around with. So, are you gonna make a symbol-mangling system, or what?
The main reason I'm more partial to names is SNAP, which is shared-nothing except for really, really static objects, such as code and symbols. Global variables have an overhead in assigning to them, and global structures are simply not mutable (the process effectively gets its own copy of the structure in the global variable, so any mutations occur in its own copy)
In theory it would be possible to add a type object that is dynamically created but is immutable once created, but attaching everything to such a type object would make dynamism a little slower. My plan had been to define polymorphic functions on (probably symbol) types, and replace 'call* with overloading of apply:
(defcall foo (v x)
(do-something-on-foo v x))
=>
(defm <base>apply ((t gs42 foo) x)
(let v (rep gs42)
(do-something-on-foo v x)))
> So, are you gonna make a symbol-mangling system, or what?
Been thinking of that somewhat; basically the most straightforward would be a symbol-conversion macro upon which any symbol mangling system can be built:
speaking of gensyms, is there a reason uniq isn't defined in arc? one less axiom? (on a related note, I don't know why sig is defined in ac.scm, it doesn't seem to be otherwise referenced there)
Because if/when we have real gensyms, where if we have
arc> (= gs (uniq))
gs2003
then (is gs 'gs2003) returns nil, instead of t (which it currently does), we won't be able to define uniq in Arc. It's another datatype (the "uninterned symbol"), and thus it needs an axiom. The axiom could be something besides uniq (string->uninterned-symbol, for instance, or usym) if (isnt (usym "foo") (usym "foo")).
Aha. So uniq in its current form is really 'kinda-uniq. How important is it that gensyms aren't equal to each other? I mean, if "everyone knows" /gs[0-9]+/ is "reserved" for gensyms, then all we need do is not make symbols matching that pattern. Thus is the language minimaler.
I'm just being lazy. It can't be that difficult to implement ...
And does that really make the language more minimal? If we leave uniq as it is, we could move it to arc.arc, but we have the axiom that symbols of the form /gs\d+/ are forbidden; if we change uniq, uniq is an axiom, but we don't have to worry about formats.
Even if Arc were not based on mzscheme the change would be minimal: it takes exactly the same operations as creating a normal symbol, with the difference that it doesn't have to be interned.
"doesn't have to be", or must not be interned? I'm thinking the latter, if the point is to guarantee that no other symbol shall be equal to a gensym ...
