What benefits do modules give us? I can think of three:
1. They hide internal definitions that the user shouldn't see.
2. They prevent the namespace from getting polluted.
3. They allow code written by different users to be integrated without names clashing.
1 can be achieved by using a with form to hold all your private stuff. 2 can be achieved by prefixing your variables (and you can use a local non-prefixed version in your with form to make this more convenient). 3 is a bit more tricky, but will work if you can supply a prefix dynamically. A few macros will make all this nice and convenient. Is anything else required?
Maybe I'm being a little optimistic about how well this will work, but I'm sure there'll be some way of doing this correctly. I think it's more Arc-y to address the problems that modules solve directly, rather than importing an onion from someone else's varnish.
Yes, you can do all those stuff in existing primitives. But maybe a little convenience macros will help, like auto-prefixing macros. And another thin macro that expands to 'with', with some bookkeeping features such as specifying the prefix in one place? Then let's call that macro 'module' and voila! You have modules.
Really, it is possible to build module system if you have enough primitives (what's 'enough' depends on what model you use). The point of modules is not whether it is possible or not. The point is that whether everybody agrees on a certain convention or not.
I originally considered writing a macro that did just that, and even called it module, but then I realised I was packing a load of disparate features into one macro just so it would look like modules in other languages.
I don't think the point is conventions (at least not yet). I think the point is that modules may be an onion, and we don't want any onions.
If the module system is an onion, it is an onion with a kernel, which is really a convention.
For example, see SLIB, the portable Scheme library http://www-swiss.ai.mit.edu/~jaffer/SLIB.html . It is designed to work with R5RS Scheme and before, thus it can't use any module systems. Instead, it uses a naming convention, which works pretty well. The problem is that, to make it work, all programmer should agree and follow the convention. The so-called module systems make it easier to follow the rules by making the existence of the convention invisible (automatically adding prefixes effectively, e.g.) or preventing programmers from doing things differently.
In the discussion of R6RS Scheme, where a module system is finally adopted, some people argued that you can roll your own modules system out of given primitives, and it had proven in the production code. But if you use your own module system (or convention) and I use my own, it will be difficult to mix two libraries. Whatever it is, we have to agree on one way. I think that's really the point.
I agree that module systems are conventions. What I don't agree with is that we should uncritically adopt them in Arc. There may be a better way. What that is, I don't know.
Then the important question is this: When you say better, better in what sense? Beyond a certain point, people have different criteria and your better and my better start to diverge.
In Scheme, traditionally such matter has been settled by "agreed to disagree"---instead of choosing one, the spec just leave it unspecified. But R6RS had a goal to allow portable libraries, so it had to choose one; for modules, it is better to have lesser but one module system, rather than having multiple ("better" in various regards) or than having none.
In Arc, ultimately it's up to pg.
[Edit: To be more PC, R6RS module is 'less agreed'. I do understand it is better in its design criteria.]
Good question. I'm sort of inspired by PG's thoughts about object-oriented programming, that it would be better if they offered the features a la carte rather that packaging them up into objects.
This may be related to #2, but I find modules/classes, very helpful in finding functionality. Someone just asked on IRC if there is a function to determine if a file exists. In Ruby, I would expect to find such a function in the File class, and sure enough, there is a File.exist? function.
This can be done in other ways such as partitioning functionality into files (if the code is the spec/doc), or organizing documentation in certain ways, but it's been my experience that a module/class system makes finding functions easier.
At some point the amount of code in arc-wiki.git will cross a threshold, and a solution will be needed. If we wait until then, the way to solve it might be more obvious.
I also wanted to mention a fourth benefit that I think modules can offer, which is:
4. They allow some form of management of dependencies between pieces of code.
This is optional, but I would integrate it in the discussion about what problems modules solve...
I agree with your sentiment. I also agree with pgs sentiment that modules are a cognitive barrier when dealing with code... that doesn't mean it can do without one, but I think it needs to be thought through carefully...
I think it's in the ansi common lisp book- unfortunately, my copy became too dog eared (the paperback is unfortunately a bit flimsy) and I threw it out, otherwise I'd let you know for sure. I think he was complaining about the CL module system being to hard to understand...
"The kind of modularity provided by packages is actually a bit odd. We have modules not of objects, but of names. Every package that uses 'common-lisp' has access to the name 'cons', because 'common-lisp' includes a function with that name. But in consequence a variable called 'cons' would also be visible in every package that used 'common-lisp'. If packages are confusing, this is the main reason why; they're not based on objects, but on their names".
I like the interesting footnote, very relevant to this discussion:
> So perhaps packages will turn out to be a reasonable way of providing modularity. It is prima facie evidence on their side that they resemble the techniques that programmers naturally use in the absence of a formal module system.
Note this is quite clunky because I couldn't find a better way to bind a dynamically determined variable in Arc. Set only takes a symbol as a first argument.