Arc Forum | Ask: Macros, names, and symbols

Arc Forum

Ask: Macros, names, and symbols

1 point by hasenj 5205 days ago | 11 comments

I'm trying to grok macros, and I'm using html templating as my playground.

So I defined a function "tag" so that:

    (tag "div" "Hey")

return:

    "<div>Hey</div>"

Then I made a macro `def-tag` so that:

(def-tag div)

defines a function `div` so I can say

(div "Hey")

And get the same output as above

Now I'm basically at the point where I can write:

(def-tag div) (def-tag span) (def-tag p)

... etc

And this seems to be working so far.

But now I'm trying to take it a step further:

I want to define all these tags in one step without repeating the def-tag command.

As in:

(def-tags div p span html title body head)

You see where I'm going?

So I tried:

    (each name '(html title head body div span p h1 h2 h3 h4 i em strong b)
        (def-tag name))

But it's not working; it simply doesn't define anything!

I'm obviously doing something wrong.

Putting some debug statements inside my 'def-tag' macro reveals that I'm basically passing "name" as the symbol to def-tag, which defines a name tag, not what name is referring to.

I don't want to pass the symbol name, I want to pass the symbol that name is supposed to be referring to from that 'each' block.

Also, the definition becomes local inside that 'each' block and is not accessible from outside, so even I manage to get that names vs symbols thing right, the definition will stay local.

Help?

1 point by rocketnia 5204 days ago | link

I'm assuming your code is a bit like this:

  (def tag (name contents)
    (+ "<" name ">" contents "</" name ">"))
  
  (mac def-tag (name)
    `(def ,name (contents)
       (tag ',name contents))

As you've figured out, (def-tag name) always expands to

  (def name (contents)
    (tag 'name contents))

even if you're not trying to define something named 'name. It's actually somewhat difficult to take a symbol you get at runtime and set a variable with that name. It can be done using (eval `(def-tag ,name)), but 'eval is usually overkill.

What you originally wanted was another macro that let you say (def-tags div span p). The first step to making that macro is figuring out what expression you want it to be an abbreviation for. In this case, it would probably be an expression like this:

  (do (def-tag div)
      (def-tag span)
      (def-tag p))

No 'each necessary. :) Once you've got an idea like that in mind, it's just a matter of picking the right tools for putting that expression together:

  (mac def-tags names
    `(do ,@(map [do `(def-tag ,_)] names)))

~~~

By the way, some of the particular names you're choosing are a bit dangerous. Arc programmers commonly use names like 'p, 'i, and 'b for local variables, specifically because they choose not to define macros with single-letter names. If you define those as macros before loading their code, then some of their function calls will be compiled as macro forms instead, and it won't be pretty.

On top of that, 'tag is already a macro that comes with Arc.

  arc> (tag foo (pr "contents"))
  <foo>contents</foo>"</foo>"
  arc> (macex1 '(tag foo (pr "contents")))
  (do (pr "<foo>") (pr "contents") (pr "</foo>"))

It comes with a bunch of other HTML-generation utilities in html.arc, like these ones...

  arc> (br 3)
  <br><br><br>
  nil
  arc> (underline:pr "contents")
  <u>contents</u>"</u>"
  arc> (hspace 4)
  <img src="s.gif" height=1 width=4>">"
  arc> (vhspace 4 9)
  <img src="s.gif" height=4 width=9>">"
  arc> (zerotable:pr "contents")
  <table border=0 cellpadding=0 cellspacing=0>contents</table>"</table>"

...which generally don't prevent people from making their own HTML libraries anyway. :-p These tools form the foundation of app.arc, blog.arc, news.arc, prompt.arc, and srv.arc, and they're not much more sophisticated than they need to be to serve those purposes.

-----

1 point by hasenj 5204 days ago | link

I know about html.arc, the point of the exercise is not to replace the builtin html macros, but to learn how to write macros (and how to think in Arc).

I have a couple of questions now:

1. Are macros defined in terms of 'eval'?

If there's such a thing as an anonymous function, why shouldn't there be an anonymous macro? Is eval the closest thing to anonymous macros?

2. Why would I want to my def-tags macro to be defined in terms of 'do'?

More generally, what the hell is the point of 'do' anyway? I would never have thought of doing anything inside a 'do' block, it seems rather redundant.

Note: I'm not trying to ridicule 'do', I'm just expressing my utter lack of understanding.

Btw, your guess of my code is pretty accurate, except I used string instead of +

  (def tag (name body)
    (string "<" name ">" body "</" name ">"))

~~~~

P.S. What's with this?

    arc> (underline:pr "contents")
      <u>contents</u>"</u>"

What's the extra "</u>" at the end? And why shouldn't I (or should I?) worry about it?

-----

3 points by fallintothis 5204 days ago | link

1. Not exactly, but it helps to think of it that way.

  (mac foo (x)
    (list 'bar x))

  (foo abcdef)

is conceptually like

  (eval ((fn (x) (list 'bar x)) 'abcdef)) ; = (eval (list 'bar 'abcdef))

Notice how the argument 'abcdef was quoted. Macros don't evaluate their arguments, but the code they generate might (e.g., if bar was a function, it'd try to evaluate abcdef as a variable).

They aren't actually implemented that way. eval operates at run-time and doesn't have access to lexical variables. Macros expand at compile-time, so it's as if you had the expansion there to begin with. E.g.,

  arc> (let y 10
         (eval 'y))
  Error: "reference to undefined identifier: _y"

but

  arc> (mac foo (arg) arg)
  #3(tagged mac #<procedure: foo>)
  arc> (let y 10
         (foo y))
  10

because

  (let y 10
    (foo y))

macroexpands into

  (let y 10
    y)

before it ever runs.

Anonymous macros are plausible, but they might force work to be done at run-time -- essentially, you're right that eval's the closest thing to it. But since macros happen at compile-time, you can't do something like

  ((if (mem 'x stuff) push pull) 'y stuff)

The compiler sees the macros push and pull, but they're not applied to anything, so it doesn't expand them. Then at run-time, you get an error as it tries to evaluate each form. You have a macro trying to act like a function (i.e., at run-time).

This topic comes up every so often: http://arclanguage.org/item?id=11517.

2. do is for when you want to execute a series of expressions, but do it in just one s-expression (i.e., the thing wrapped in do). You see it a lot in macros; e.g., defs in arc.arc:

  (mac defs args
    `(do ,@(map [cons 'def _] (tuples args 3))))

It converts

   (defs f (x) (+ x 1)
         g (y) (- y 1))

into

   (do (def f (x) (+ x 1))
       (def g (y) (- y 1)))

which is just one list. You couldn't return multiple values, so the macro couldn't expand into

  (def f (x) (+ x 1))
  (def g (y) (- y 1))

directly.

Another place you see do a lot is in if statements. Each clause is one expression, but if you want to do multiple things in one expression, you need the do. E.g.,

  (if test
      (do (pr #\t) (pr #\h) (pr #\e) (pr #\n))
      else)

will print "then" if test is true, otherwise it'll do else. This wouldn't work without the do:

  (if test       ; if "test"
       (pr #\t)  ; print #\t
      (pr #\h)   ; else if (pr #\h)
       (pr #\e)  ; then (pr #\e)
      (pr #\n)   ; else if (pr #\n)
       else)     ; then "else"

P.S. That's the return value of the statement.

  arc> (pr "a") ; prints "a" WITHOUT a newline, then returns the first thing it
                ; printed (the string "a")
  a"a"
  arc> (prn "a") ; print "a" WITH a newline, then returns the first thing it
                 ; printed (the string "a")
  a
  "a"

To learn more about macros, my debugger tool might be helpful: http://arclanguage.org/item?id=11806. Let me know if it is!

-----

1 point by hasenj 5202 days ago | link

on a second thought, the anonymous macro is nothing but the manual expansion of the macro :P

-----

1 point by shader 5201 days ago | link

That's not always true. If a programming system supported first class macros, then an anonymous macro could be passed in as an argument to a function, which could then apply it in a more sophisticated way at run time. I.e. apply it to a list of objects that don't even exist at compile time.

However, since arc does not support first-class macros, and neither do most compiled languages that I'm aware of, you're basically correct.

-----

2 points by waterhouse 5204 days ago | link

The most direct transformation from your attempted code to working code is this:

  (each name '(html title head body div span p h1 h2 h3 h4 i em strong b)
     (eval `(def-tag ,name)))

-----

1 point by hasenj 5202 days ago | link

ok, now can that be encapsulated into a macro? say a macro called apply-mac-to-list

  (= tag-list '(html title head body))
  (apply-mac-to-list def-tag tag-list)

I managed to come up with this:

  (mac apply-mac (mac-name args)
      (prn:string "applying " mac-name " to " args)
      `(each arg ,args
          (eval `(,',mac-name ,arg))))

The pattern of commas and quotes is rather confusing and seems arbitrary

I got there by replacing 'eval' with 'prn', and then I kept playing with commas and quotes until it printed the right thing, then I put eval back in.

So it seems to work but I have no idea why.

EDIT:

I'm not sure if that sort of thing is common in lisp/arc? Am I approaching macros in a totally wrong way?

I presume I'm thinking in python, where a lot of meta programming techniques constitute of taking strings and using them to get fields that would normally be accessed using the symbol directly.

  getattr(obj, 'field') # same as obj.field

and so we get into the habit of doing things like:

  def apply_to_fields(obj, field_name_list, fn):
    for field in field_name_list:
        new_value = fn(getattr(obj, field))
        setattr(obj, field, new_value)

-----

1 point by waterhouse 5201 days ago | link

I see you've encountered the strange ,',name idiom. I congratulate you for writing it yourself... I read about it in "On Lisp", in its description of an "abbrev" macro. The whole book is pretty good to read through, and it says a good deal about macros, so I would recommend you take a look; but section 16.1 explains directly how one can come up with such a thing:

  http://www.bookshelf.jp/texi/onlisp/onlisp_17.html#SEC109
    The full book is there, but if you want it as a pdf or
  something, see Paul Graham's web site:
  http://paulgraham.com/onlisptext.html

Also, note that there are functions called macex and macex1. It stands for "macro-expand". They work on any macros, including ones you've just defined; they are very useful for debugging macros. See:

  arc> (macex1 '(let x 1 (+ x 2)))
  (with (x 1) (+ x 2))
  arc> (macex1 that)
  ((fn (x) (+ x 2)) 1)

Meanwhile, regarding the problem at hand. First, note that, since you're already using eval, 'apply-mac may as well be a function, defined and used like this:

  (def apply-mac (mac-name args)
    (each arg args
      (eval `(,mac-name ,arg))))
  ;Usage
  (= tag-list '(html title head body))
  (apply-mac 'def-tag tag-list)

Second, there's a little side issue brought up by your definition of 'apply-mac. It binds the variable 'arg, which... turns out to be harmless, but in general I would use a unique name for 'arg:

     `(each arg ,args
         (eval `(,',mac-name ,arg))))
  ;Rewrite as:
     (w/uniq garg
        `(each ,garg ,args
            (eval `(,',mac-name ,,garg))))

(Oh man, nested backquotes... I haven't dealt with those for a while. Last time was probably when I wrote 'abbrev.) Now that I've given you a link to On Lisp, I'm going to cut myself off and point you to chapter 9 ("Variable Capture") for further discussion. If I said more, I'd just be telling you what you could read about in On Lisp.

Regarding general practices in Lisp (you ask about how you're approaching macros)... I can say a few things. First, know that eval is slow (it performs full syntactic analysis), and you most likely should not be using it at runtime. Second, using eval is almost always unnecessary; you could use a macro or something else, and if you can't see how, then that's probably due to insufficient macro-fu. Exceptions are when the code you want to generate and evaluate actually depends on stuff that happens at runtime (e.g. your program reads Arc expressions, evaluates them, and prints the results).

In this case, this code just sets up a bunch of definitions; it's run once and it takes an imperceptible amount of time anyway. Performance isn't an issue. I would probably write it as the one-liner "(each x tag-list (eval `(def-tag ,x)))" when I was sitting at the REPL just wanting to get code working; then I'd probably write up the 'def-tags macro in my actual code file. Here's my preferred version (paraphrasing aw):

  (mac def-tags tags
    `(do ,@(map (fn (name) `(def-tag ,name)) tags)))

I think it's overkill to create "apply-mac". The concept is complicated enough and rare enough that giving it a name doesn't do much for me. I think it's best to just write def-tags using basic Arc operators.

Finally, you ask whether this sort of thing is common. The thing you said before that was "It seems to work but I have no idea why", so I'll assume that's what "that sort of thing" refers to. My answer: Generally, no. Sometimes I'll get something working by brute force search, as you seem to have done; and sometimes I'll get something working by pasting it in from someone else's code; but in either case, I will think about it and verify that it is correct and should work, and usually I'll come away satisfied in my understanding.

Note:

  (let name 'BOB
     `(meh `(ach ,',name)))
  -->
  (meh `(ach ,'BOB))
  -->
  (meh (ach BOB))

The innermost ",name" gets replaced with the value of the variable 'name; the other comma still exists, and without the quote, you would get "(meh `(ach ,BOB))", which would try to evaluate BOB. That is just how nested backquotes are defined to work; an informal introduction to them usually doesn't cover nested cases. Then, the second time around, the expression 'BOB within the quasiquote gets evaluated (because it's unquoted by the comma), and it evaluates to the symbol BOB.

-----

1 point by hasenj 5199 days ago | link

thanks for the elaborate explanation :)

> I see you've encountered the strange ,',name idiom. I congratulate you for writing it yourself

cool :D

What I meant by whether "things like this" are common, I was referring to confusing (and seemingly arbitrary) pattern of weird symbols that's not in line with the spirit of clean, readable code that's meant for humans to read instead of being meant for machines to execute.

-----

1 point by aw 5201 days ago | link

The pattern (comma quote comma)

  ,',X

turns out to be common in nested quasiquotation expressions, such as when you have macros defining macros.

See http://repository.readscheme.org/ftp/papers/pepm99/bawden.pd... for a description:

"The value of X will appear as a constant in the intermediate quasiquotation and will thus appear unchanged in the final result."

-----

1 point by aw 5204 days ago | link

  (eval `(do ,@(map (fn (name)
                      `(def-tag ,name))
                    '(html title head body ...))))

-----