Even though I think this is a good idea, I'm pretty sure there is better way to do it. I feel like your solution is only a solution to a very specific problem. Your solution is almost the exact same as keywords, only its not. You are just adding extra information into the function call (the type of argument). Instead we could use keywords for a solution to this, and as a solution for many other problems for which this kind of scheme wouldn't work.
I do like the idea of replacing a ton of functions with functions that do different things depending on their arguments.
I thought about it some more. Here's why I think the wrapping approach is superior to a new arg. Let's say you define a function on lists:
(def foo (x)
(... x))
Later you want to use foo, but traversing nested lists like it's a tree. Ok, so we'll add a new version:
(defextend foo (x y) (is y :tree)
(... x y))
But now calls to the base version will stop working; defextend passes both args to the base version which is only expecting one arg. To make things work you have to modify the base case, and teach it about an arg it doesn't need to know about. That seems unnecessarily ugly.
Another problem is the need to sequence args. If you create a new version with yet another arg:
(defextend foo (x z) (...)
(... x z))
Now you have to worry about the second arg being used for wildly different uses, possibly interfering with each other and causing bugs, or having to provide unnecessary args:
(foo a nil :widget) ; don't traverse a like a tree
Edit: ah, I just thought of one counter-argument. If you have multiple tree arguments you have to wrap each of them in my approach. This is more apparent when trying to support dotted lists in map (fallintothis's original complaint: http://arclanguage.org/item?id=18247)
(map + (dotted '(1 2 3)) (dotted '(1 2 . 3)))
---
Now, arc could be more flexible. My wart language (http://akkartik.name/post/wart) is more like javascript: excess args are silently ignored, and missing args are silently passed nil. Also, it has keyword args, so different versions might be called as:
(foo a :traverse-as tree)
(foo a :like widget) ; don't traverse like a tree
In such a situation the difference seems more minor.
The thing is, I don't want a new arg. I think we would be better of using keyword arguments, similar to ones as you mentioned in wart[1]. The reason I'm making such a big deal out of this is why create a new solution (tagging the arguments as they are passed in) when we can already can do this with keywords which are much more flexible. With keywords we can call them with anything we want: a list, a symbol, or even some sort of complex data structure.
(foo :a 'tree)
(bar :b '(1 2 3))
(baz :c (obj a 'b c 'd))
We can use keywords to pass in many different kinds of arguments. If we were to just tag the args, we could pretty much only pass in different types, not much else.
(foo (tree x)) ; how would you change this from a type to a number?
We can also implement them in such a way that when a function is called with extra keyword args, it won't change the behavior of the program (we would probably want it to log some kind warning). This way when we extend foo, it doesn't matter if we pass the original the keyword arg. As far as the original foo knows, the keyword arg doesn't even exist.
Sorry if I confused you. I don't want extra args, I want keyword args.
That makes sense! I've long been a proponent of keyword args as you can see. My love for them goes back all the way to one of my first posts here: http://arclanguage.org/item?id=10692. I think I've implemented them three times so far. I even have a version of arc with them: http://github.com/akkartik/arc. But I haven't updated it lately. I'll spend a few hours and merge in all the new stuff in anarki over the past year.
That's great! I just googled "Paul Graham arc no keywords" and found in one of his early writings on arc, he had a plan to replace keywords with something called "get parameters".
Ok, I've updated http://github.com/akkartik/arc. It was a useful exercise, because it turns out it has more tests than anarki, and I found several bugs in my work yesterday :) I've copied the tests to anarki as well.
It doesn't really need to know about a new arg, but there's no way to pass it in automatically because there's already a rest arg: body. Ok, so let's try that:
(mac each (var expr ? as nil . body)
`(walk ,expr :as as (fn (,var) ,@body)))
The problem with this is that you need to explicitly provide the optional arg, or it gets taken out of body.
I had this old idea that rest params should take precedence over optionals, but it turns out to have fallen by the wayside: http://www.arclanguage.org/item?id=13083. Time to bring it back here.
Well, the problem is you are trying to make keyword args the same as optional args. It might be better if you make make keyword args and optional args two completely separate kinds of arguments (similar to how common lisp has both &optional and &key). This way you would define each as:
(def each (var expr : as nil . body) ; I used : to represent keyword args
`(walk ,expr :as as (fn (,var) ,@body)))
This way calling each like
(each x '(1 2 3) x)
will expand correctly because keywords only take a value when they are passed in explicitly. It would have to be called as
(each x '(1 2 3) :as x)
to get the error you had. If you need to define a function that takes both optional args and keyword args:
(def foo (a b c ? g h i : j k l . rest)
...)
This way a-c are required, g-i are optional, j-l are keyword, and rest is everything else.
As for the problem you had with needing to redefine each (it is also mentioned by rocketnia in the post you linked[1]), each is just an interface for walk, so if a new feature is added to walk, each need to be redesigned. There might be some way that we can define some sort of macro, interface, which can be used to create a function/macro which is a very thin layer over another function/macro. As for how interface is to be designed, I'm not sure.
Well, it seems more general to let any arg be acceptable as a keyword.
rocketnia's concern I addressed by passing through unknown keyword args unchanged:
(def foo (a b)
(list a b))
(def bar args
(apply foo args))
arc> (bar 3 :a 4)
(4 3)
But that doesn't work in this case because each needs to treat as specially. I don't think that's avoidable.
We _could_ include a new category of 'required' keyword args like you suggest, yes. Let me know if you think of a situation where just giving rest args precedence isn't enough. Things are already pretty complicated with rest, destructured, optional, and keyword args, so I'm reluctant to add another delimiter and category.
"rocketnia's concern I addressed by passing through unknown keyword args unchanged"
The reason I'm okay with this approach is because it localizes the complexity to each function. This way I don't dislike the language; I just dislike the idioms currently being used in it.
Consider the use of flat text formats for shell scripts, HTTP headers, CSS properties, XML attributes, etc. They're usually convenient enough to type out manually, and that also makes them convenient to generate in simple cases, but writing a parser for them can be tough due to the same syntactic sugars that made them convenient to generate in the first place. Writing a full-featured generator or parser can also be tough, because sometimes future versions of the text format may allow it to encode more information.
With keyword arguments, the syntax for a function call is more complicated and sugary, putting programmers like me in the position of wondering what kind of headaches they should go through to parse or generate this format. If the headaches are unclear, combining programs becomes hard, because each developer might opt for different headaches, at which point I think they overlap in the form of complex glue code. :)
I have a greatly-enhanced respect for this perspective over the past month. Look at my travails over the past day trying to get optional, keyword and destructured args to play well together: https://github.com/akkartik/arc/compare/342ebf57a8...e347973.... Or my travails over the past month trying to build macex in wart in the presence of optional, keyword and destructured args AND quoted AND aliased params AND already-eval'd args[1] AND incompletely-eval'd args[2]: https://github.com/akkartik/wart/compare/30025ee25c...cabaa0.... Clearly there's some point at which I jump the shark :)
[2] related to partial eval: https://github.com/akkartik/wart/commit/8239ed9d21. I've been struggling intermittently with partial eval for two years now. This now year-long attempt might well be utterly useless.
---
Perhaps the easiest to understand and most serious extant example of the knots I end up tying myself into:
arc> (iso :x :x)
nil
This is because x is one of the params of iso. Even more serious, this happens even if you try to compare a variable that evaluates to :x. Or a variable that evaluates to a large structure containing :x somewhere in it. Ugh! The reason for this is that I'm extracting keyword args at function-call time inside each function. This:
It took me three years to notice yesterday that I'm extracting keywords from post-evaluated arguments. Ugh! Urrrgggh!
Weird thing is, even wart has this problem. Even though I would superficially seem to be extracting keyword args pre-evaluation. I still need to debug this. Or perhaps I shouldn't debug this. Perhaps I should just throw up my hands and delete all my repos and become a hermit.
We should be able to specify the type of the function as part of the function. We shouldn't need to specify it in the arguments. I just think of it as we are walking over a tree as opposed to calling walk on an object that is a tree.
What I'm trying to say is the keyword arg shouldn't apply to a specific argument. It should apply to the function call as a whole.
