I'm going to ask some seriously basic questions here:
1. So, Mu is a general programming language - right? (in reading the docs I almost wondered if it was a testing language only).
2. It is built on top of Arc - right? (It looks as though it's built with c code, but I see arc in there too so I have to wonder where it fits in).
3. Is the idea that you build your tests for each function inside the actual function? (which the examples seem to illustrate). If that's the case wouldn't the code become really large and hard to navigate? Have you written any substantial programs in it to see how that might look?
4. How does SubX relate to Mu?
I'm probably missing the depth being presented here, but I'm a high-level language guy so I have to start with basic questions :)
Thanks for the questions! Part of the problem is that the repository isn't for a single 'language'. It's for my experiments for a new way to program that makes codebases easier to understand. As the biggest stress test for the new way, I've trying to create a whole stack that is easy to understand. But it's just a prototype, or rather a series of prototypes. I've tried to make each prototype self-contained, with a Readme and instructions for running it that will continue to work to this day.
Prototype 2 was a similar (but not compatible) language built in C++. It had the most work on it, and I also used it for teaching for a couple of years. I stopped working on it sometime this year. It's still available on the top level at https://github.com/akkartik/mu. It's 23kLoC of C code, and 5kLoC of Mu libraries. The most substantial program built in it was a 2-pane programming environment I used for teaching programming: https://github.com/akkartik/mu/tree/master/edit#readme. It's 12k lines of Mu code, about half of which is tests. You can see its sources colorized to be easier to read at the bottom of http://akkartik.github.io/mu.
Now I'm at prototype 3, SubX. It's in a very preliminary state. As above it is not intended to be compatible with existing prototypes. The previous prototypes were kinda-sorta designed to be easy to translate to native code, but they were still simple-minded tree-walking interpreters. I had hazy plans of gradually compiling them to native code from the top down, but that turns out to be beyond my ability. SubX starts from the bottom up, building an almost trivial syntax on top of raw native x86 machine code, and I'm gradually learning how to implement a compiler in it. Most people would say it's too hard to build a compiler in assembly these days when we have so many high level languages available. I'd like to see if it's manageable with the right framework for writing automated tests. So SubX starts out not with improvements to syntax, but to error checking and automated testing.
The most substantial program I've built in SubX so far is a port of the very initial version of Crenshaw's "Let's build a compiler" series[0]. All it does is read a number and emit native code to return that number in the exit status: http://akkartik.github.io/mu/html/subx/apps/crenshaw2-1.subx.... So SubX is still in a very early state.
SubX tests aren't inside the functions they test, they're just interleaved in the same file. Any label that doesn't start with '$' is the start of a new function. Functions that start with 'test-' are tests, and they all run when you run with a 'test' argument on the commandline.
The hope is to one day build a robust, hackable stack culminating in a high-level Lisp atop this infrastructure, a stack that leans into the Lisp tendency to fragment dialects by encouraging people to create incompatible forks -- while also making it easy (but not automatic!) to share code between the incompatible forks. It would still be some amount of work to copy the tests over and then make them pass, copying over bits of code at a time and modifying it as necessary. That's the sort of workflow I want to encourage rather than blindly upgrading software by running a package manager command. But before I can recommend it to others I have to see if I can get it to work. This repo is a test bed for eventually building tools to help people collaborate across incompatible forks.
Thanks for asking these questions! They're very helpful in understanding how others see the mess my repo has turned into. I'm going to try cleaning it up.
See I always move up to the top level directory and work my way down. So as I did this my reference point for understanding got completely mixed up.
I'd make a top-level dir called 'prototypes' with details like [1] you provided in this comment and then branch down.
* Also, I'd probably limit referencing Mu, specifically, in SubX. I don't think its helpful. Just remove this line:
"We'll gradually port ideas for other syscalls from the old Mu VM in the parent
directory."
It doesn't add much value and takes people's attention away as they start trying and understand the relationship. If you need to just add notes at the bottom.
[1] "The hope is to one day build a robust, hackable stack culminating in a high-level Lisp atop this infrastructure, a stack that leans into the Lisp tendency to fragment dialects by encouraging people to create incompatible forks -- while also making it easy (but not automatic!) to share code between the incompatible forks."
Re: [1]
Yeah, if you can build a language that allows someone to build arc and even a clojure version of arc using the same base code, that would be cool. I'd immediately try spinning up a new version of arc with clojure's tables and table functions :)
> I'd make a top-level dir called 'prototypes'... and then branch down.
Yeah, I've been planning a reorganization like that. Unfortunately the reorg is going to break all the links I shared before I thought of this :/ So I'm going to wait a bit before I make the switch.
> if you can build a language that allows someone to build arc and even a clojure version of arc using the same base code, that would be cool.
I'm not aiming quite there. That would be really hard to do, and then it would be impossible to keep in sync with existing language upstreams over time, given the underlying platform will be very different. The way I imagine providing something similar is this: there would be multiple forks of the Mu stack for providing a Clojure-like or Arc-like high-level language. But these languages wouldn't be drop-in replacements for real Arc or real Clojure. Also, each fork would try to minimize the number of languages it relies on to do its work, so you wouldn't immediately be able to run both Clojure and Arc on a single stack. Because every new language used in a codebase multiplies the comprehension load for readers. (I ranted about this before at https://lobste.rs/s/mdmcdi/little_languages_by_jon_bentley_1...)
Basically, I want to commoditize the equivalent of a Lisp Machine for any language. My goal is to help people collaborate across incompatible forks, _but_ the forks have to all have certain characteristics that no existing software has (because it all assumes rigid compatibility requirements).
> Basically, I want to commoditize the equivalent of a Lisp Machine for any language. My goal is to help people collaborate across incompatible forks...
I'm struggling to understand so forgive me, but the reasons why and what you're doing seem to change or at least are many fold and thus hard to unpack. Or maybe the different prototypes are messing me up.
So let me see if I can unpack it (at least for myself :)
Your goals are:
1. To build prototype 'x' compiler/language that's more robust and easier to maintain because it has been built with testing capabilities in mind (I'm imagining a model with convenience features or set requirements to accomplish this).
2. Build prototype 'x' to permit developers to build their own compiler/language(s) that inherit the benefits from prototype 'x', thus making that process more enjoyable and more likely to succeed.
3. Permit/Encourage greater collaboration amongst developers, on separate prototype 'x' projects, because prototype 'x' is robust and developers are working under a shared model that has core concepts/features that act as a bridge for that collaboration to happen.
Does this seem right?
So is this a project that you're doing because you're passionate about it and you think it can change things (i.e. improve the lives of other people)? Or is this a product idea where you have assessed there's a need and you're going to fill it?
Thanks for the probing questions! Yes, I don't mean to move the goalposts on my reasons. I feel like I'm reaching for something fundamental that could end up having lots of different benefits, mostly things I can't anticipate.
I don't actually care that much what the high level language is at the top. I'm biased toward Lisp :) so that's what I'm going to build towards. But if others want a different language I want to make it easy to switch the superficial syntax to suit their taste -- and convert all existing code on the stack so everything is consistent and easy to read. If some others want to add new runtime features, I want to make that easy too. Finally, I want it to be tractable to mix and match syntax and runtime features from different people and still end up with something consistent. Using tests at the bottom-most layers, and building more rigorous type systems and formalisms as necessary higher up.
The key that would make this (and much else) possible is making the global structure of the codebase easier to comprehend so that others can take it in new directions and add expertise I won't ever gain by myself, in a way that I and others can learn from.
Ignore the other prototypes in this repo; they're just details. The goal I'm working toward is a single coherent stack that is easy for others to comprehend and modify.
This isn't a product, in the sense that I can't/won't charge money for it. I'm not really making something others want right now. I'm trying to make something I think the world needs, and I'm trying to make the case for something the world hasn't considered to be a good idea yet. I'm sure I don't have all the details nailed down yet :)
> I'm trying to make something I think the world needs, and I'm trying to make the case for something the world hasn't considered to be a good idea yet. I'm sure I don't have all the details nailed down yet :)
