"Seems that call-w/stdin and similar had to be implemented like protect, which is annoying, but that had to be how it was done to get them to function in the face of continuations and exceptions."
I, for one, appreciate that you're doing things the way they have to be done, lol.
Any plans to generalize 'call-w/stdout and 'call-w/stdin to 'parameterize? Now that I take a look at http://docs.racket-lang.org/reference/parameters.html, there's one more complication I hope is already on your mind: Threads.
I don't tend to care much about threads myself, but since they're on your roadmap, hopefully you have a good plan for them and the way they interact with 'call-w/std{out,in}. XD
If I understand the notion of parameters correctly from yours and Pauan's mention of them (I'd not heard of them before now), they are essentially a way to do dynamic binding in a language like Arc that normally uses static binding. This is in large part exactly what call-w/stdin does with stdin: it sets up a dynamic binding for that function. Apparently, in my attempts at implementing this functionality I've also independently kludged up a special-purpose version of what Racket calls a continuation mark, and obviously doing such a thing bothers me to no end.
Now that I think about it, implementing parameterize might actually not be that difficult, and Pauan's implicit parameters might actually be easier than explicit parameters that have to be applied in order to obtain their values. It would also get rid of the special-purpose "continuation mark" I created to support call-w/std(out|in) and replace it with a more general-purpose structure capable of storing other dynamic bindings as well.
Well, indeed threads are on my mind, but I will keep things simple for now, and make them green threads whose scheduling is controlled entirely by Arcueid's runtime. I had for a time considered using real POSIX or other OS-level threads to be able to take advantage of multiple cores but soon realized that this would introduce quite a bit of complication. Using real threads affects just about every aspect of the implementation. For instance, I am at the moment using an incremental garbage collection algorithm that ought to be amenable to multi-threaded operation in theory but in order to really use it in a multi-threaded context I'd also have to have a good multi-threaded memory allocator, and by the time I'd had a look at all the literature on such algorithms I realized that I was in way over my head.
Green threads simplify matters considerably. This means that call-w/std(out|in) and the more general notion of parameters can be handled without too much trouble. In the plan I have for Arcueid's green threads, a thread is basically a structure that contains everything that the virtual machine needs to run, including all continuations. The only thing directly shared by all threads is the global environment, and then I'd also have to make available a flattened version of the structures I used to store the call-w/std(out|in) bindings from the thread's creator, and the more general dynamic bindings created by parameterize as well.
And no, while Arcueid's main goal is to produce a version of Arc compatible with at least Paul Graham's Arc3.1, I am of course not above introducing improvements and extensions, provided that they do not also break compatibility. I'd like to be able to at least run news.arc unmodified before I release version 1.0.0. :)
"Apparently, in my attempts at implementing this functionality I've also independently kludged up a special-purpose version of what Racket calls a continuation mark, and obviously doing such a thing bothers me to no end."
Why? Are you worried your runtime will be exactly like Racket but less mature? When I looked at your call-w/std(out|in) commits, I liked your approach exactly because I noticed it was in the same vein as continuation marks. :-p
As you've noticed with complex numbers, Arc exposes lots of accidental complexity that it inherits from Racket. In fact, speaking of accidents, Arc 3.1 without modification exposes pretty much all of Racket: http://arclanguage.org/item?id=11838
When it comes to threads and parameters, I'd say Arc pretty much specifies nothing and leaves it up to Racket to provide the meaning and implementation. Arc literally defines 'call-w/stdin and 'call-w/stdout in terms of Racket's 'parameterize. If Arcueid doesn't end up with (internal) functionality equivalent to thread cells and continuation marks, there's a good chance it'll have certain corner-case inconsistencies with Arc, even if there aren't enough inconsistencies to break the programs we actually care about.
But even so, I wouldn't worry about it too much. I personally consider Arc to have shoddy support for threads (just exposing a tiny subset of Racket's functionality and imposing a GIL) and also for reentrant continuations (not defining 'dynamic-wind, implementing loops with mutation), so I don't really blame an Arc implementation for being incompatible in these areas. In some cases, full compatibility might be more harmful than not trying!
---
"I had for a time considered using real POSIX or other OS-level threads to be able to take advantage of multiple cores but soon realized that this would introduce quite a bit of complication. Using real threads affects just about every aspect of the implementation."
If you want to give an Arc program power to take advantage of those, but you're having trouble with multithreaded allocation, an alternate path might be to have the Arc namespace and most data structures be local to an OS thread but then to have other tools to write and read manually-managed shared memory. I dunno, maybe that's not very inspiring. :-p
---
"The only thing directly shared by all threads is the global environment, and then I'd also have to make available a flattened version of the structures I used to store the call-w/std(out|in) bindings from the thread's creator, and the more general dynamic bindings created by parameterize as well."
Er, local scopes and first-class data structures might need to be shared too, right?
(let foo (list nil nil)
(for n 1 10
(thread (push n foo.0) (push n foo.1)))
(def get-foo ()
foo))
"Why? Are you worried your runtime will be exactly like Racket but less mature?"
Not in the slightest. It just bothered me that I had to embed a special-purpose data structure inside Arcueid's continuations just to support one language feature. Now that I see that there is a natural generalization to this feature, that makes me feel a lot better. :)
'protect is implemented with 'dynamic-wind, so the only functionality we lose is the ability to specify a pre-thunk. Are there any areas where that would be useful?
Dynamic-wind gives us most of the ability to implement parameters ourselves. We just mutate a box upon every entry and exit of the expression. Unfortunately, it might take some crazy trampolining to get the last expression of (parameterize ...) in tail position. I'm not even sure if tail position is possible....
I think the last missing piece is thread-friendliness. In the face of threads, we'd need the box to be thread-local like Racket's parameters. But my point here is just that the pre-thunk is useful for something. ^_^
"[...] they are essentially a way to do dynamic binding in a language like Arc that normally uses static binding. This is in large part exactly what call-w/stdin does with stdin: it sets up a dynamic binding for that function."
That is correct. In fact, in Arc 3.1, std{in,out,err} are Racket parameters[1], and call-w/std{in,out} use Racket's parameterize. My point was merely that it is useful to provide parameters to Arc programmers so they can define their own parameters beyond just stdin/stdout/stderr.
* [1]: That's why you need to use (stdin), (stdout), and (stderr) rather than stdin, stdout, and stderr.
---
"And no, while Arcueid's main goal is to produce a version of Arc compatible with at least Paul Graham's Arc3.1, I am of course not above introducing improvements and extensions, provided that they do not also break compatibility."
Glad to hear it. I would just like to note that any changes whatsoever will break compatibility. For instance, if you provide a "parameterize" form, a library written in Arc might also define a "parameterize" global, etc. My feeling on such things is that there should be a social convention for specifying implementation-specific global variables.
Something like, "if a global variable starts with % it is implementation-defined, so portable Arc libraries shouldn't use or define global variables starting with %".
Then your implementation could provide "%parameterize" to Arc and there would be no problems, because Arc libraries aren't supposed to use variables starting with %, so there's no conflict.
This should be solely a social convention, not enforced by the compiler. I may want to write an Arc library that does use/define implementation-specific globals, while
understanding that such a library won't be portable and may break in the future.
"Any plans to generalize 'call-w/stdout and 'call-w/stdin to 'parameterize?"
As a side note to this, I think it would be very preferable to have a `parameterize` form which `call-w/stdin` and `call-w/stdout` would call. It should behave similarly to Racket's parameterize.
This isn't necessary for an implementation of Arc 3.1, but it's very useful in practice: you could provide a way for users to create their own parameters and then call parameterize on them. This is what ar and Nu do, and it's incredibly convenient, especially when you provide a way to make the parameters implicit[1].
It really does depend on your goals, though. Do you intend for this to be just an implementation of Arc 3.1 and nothing more? Or do you intend to provide convenient features that Arc 3.1 doesn't have? Your work on numerical functions seems to suggest that you're not entirely against extending your Arc runtime to do things that Arc 3.1 doesn't.
---
* [1]: By "implicit parameters" I mean parameters that you don't have to call to extract their value. In other words, you can just say `stdin` rather than `(stdin)` for instance.
The main point is that you want something that works, right? :) If updating Racket isn't working for you, maybe for now you can revert to an older version of Anarki. The commit before the racket/unsafe/ops one is https://github.com/nex3/arc/commit/b36f223bfba2e12e518fff955..., which you can get to like this:
$ git checkout b36f223b
If you want to get back to the Anarki tip, do this:
$ git checkout master
Be careful if you have changes in your working copy.
I must have done some mistake then. I admit that I didnt see akkartik's link before posting that reply. I think my downloaded files are older than October- the directory has a timestamp from October and the ac.scm file has a timestamp from August 2009. The diff is here: http://pastebin.com/BLp6p69v
Yes, I would like to use something that works, and something that is as latest as possible :) I seem to be using Racket (in the form of plt-scheme) and it seems to work for an older version of arc. The point of my question was if I needed to do something to keep up with the changes made in Arc.
"The point of my question was if I needed to do something to keep up with the changes made in Arc."
Then making sure Racket is up-to-date is the answer you're looking for, as you already know. :-p Have you been successful running the latest Anarki on your Racket install yet? What do "racket --version" and "mzscheme --version" say now?
Hi,
Thanks a lot to your earlier comment, about running "racket -f as.scm"- I found out that racket couldnt be located by the system. Something must be unclear with the description of Ubuntu/Debian packages- I got an impression that "plt-scheme" contained racket. I manually installed racket by downloading from here: http://racket-lang.org/download/ .
For easier reading, here's the same thing in Arc (untested):
(= here* '(no-befores-or-afters . nil))
(= orig-ccc ccc)
; This overwrites the original.
(def ccc (body)
(let here here*
(orig-ccc:fn (cont)
;; With multiple value return
;(body:fn results
; reroot.here
; (apply cont results))
; Without
(body [do reroot.here
cont._])
)))
(def dynamic-wind (before during after)
(let here here*
(reroot:cons (cons before after) here)
;; With multiple value return
;(call-with-values during
; (fn results
; reroot.here
; (apply values results)))
; Without
(do1 (during)
reroot.here)
))
(def reroot (there)
(unless (is here* there)
(reroot cdr.there)
(let ((before . after) . ignored-parent) there
(= car.here* (cons after before))
(= cdr.here* there)
(= car.there 'no-befores-or-afters)
(= cdr.there nil)
(= here* there)
(before)))))
This is exactly the same technique I mentioned in the other thread. The language originally doesn't support dynamic-wind, but this code hides the original 'ccc and replaces it with one that manually traverses a global stack of 'dynamic-wind handlers. Good to know this is called Hanson-Lamping. :-p
If the language has built-in exceptions, this particular implementation isn't friendly with them. It doesn't do anything to intercept them! ^_^ To repair this in Arc, we would replace (do1 ...) with (after ...) above.
On the other hand, a language that defines 'dynamic-wind this way can implement exceptions in terms of the replaced 'ccc.
Thanks for that... The Scheme version has even more parentheses and was much harder to understand. I'm starting to see how the algorithm works, and I'll see if this comes out cleaner than the kludge I came up with and incorporated into Arcueid 0.0.5. What I plan to do is restore the very simple continuation invocation it used to have, then wrap it up that way. Exceptions are of course simple enough to implement by using ccc, and implementing them on top of the ccc that supports dynamic-wind should provide us with exceptions that support dynamic-wind as well.
By the way, I haven't seen the orig-cc:fn idiom before. So even a special form like fn works with ssyntax. So I suppose it would not do to just expand it into ((compose orig-cc fn) ...), and we have to actually make it a real function composition.
"So I suppose it would not do to just expand it into ((compose orig-cc fn) ...), and we have to actually make it a real function composition."
Not so. If you look at line 29 in ac.scm you'll see this:
; the next three clauses could be removed without changing semantics
; ... except that they work for macros (so prob should do this for
; every elt of s, not just the car)
((eq? (xcar (xcar s)) 'compose) (ac (decompose (cdar s) (cdr s)) env))
((eq? (xcar (xcar s)) 'complement)
(ac (list 'no (cons (cadar s) (cdr s))) env))
((eq? (xcar (xcar s)) 'andf) (ac-andf s env))
For those not familiar with the Arc compiler, what it's doing is basically these transformations:
If you wish for compose, complement, and andf to work on macros and special forms like fn, your compiler will need to do a similar transformation. The catch is that this transformation only works in functional position:
(map no:do (list 1 2 3 nil nil)) ;; doesn't work
It's all very hacky and whatnot, macros aren't very clean at all in Arc. The other catch is that it hardcodes the symbols 'compose, 'complement, and 'andf, but my Nu compiler fixes that.
The complex interactions are actually all natural consequences of a language that doesn't even have dynamic wind and exceptions in the first place. Because with 'ccc, people can and probably will build these systems themselves, multiple times, in ways that break each other and threaten to fragment the community... ultimately leading to their standardization. IMO, a procedural language with 'ccc is a dubious start if the 100-year goal is a simpler language than Scheme.
"I just have to wonder if there's a better way of implementing protect than the rather kludgy way that I wound up doing it."
Based on a quick skim of https://github.com/dido/arcueid/commit/65a252a87fd817ec33f21..., it looks like you're doing it in a similar way as Rainbow, lol. You're collecting protect handlers on your way down and then pushing them all back onto the stack in a particular order.
I think a more natural way to do this might be to stop at the first protect handler, then enact instructions that accomplish "call this handler, pop the frame of (or otherwise exit) the protect body, then make the same continuation call again." In fact, I wonder why you and Conan Dalton didn't do this to begin with. :-p
Just to explore this a bit, to help both of us understand... if this approach were extended to dynamic-wind, if you encountered a dynamic-wind form on your way up the stack, you might stop there and enact instructions of the form "call this handler, push the frame of (or otherwise enter) the dynamic-wind body, then make the same continuation call again." Does this make sense? Part of my concern is to have clear semantics for what happens if a continuation call exits or enters a handler block.
Meanwhile, an alternate (but not necessarily better) way to do it is to define a core language without 'protect and then wrap that core in a standard library that hides the original version of 'ccc and exposes a version that consults a global stack of 'protect handlers. This Ruby library does that: https://github.com/mame/dynamicwind.
Are you at least some other kind of developer? Building or maintaining a website (or anything!) on Arc is a rather code-it-all-yourself process. Here are a few things that might disappoint you along the way:
- The Arc language comes with the ability to host an HN-like website, but unfortunately the current HN has been modified significantly since the most recent release of Arc. For instance, Arc's news.arc doesn't have OpenID support or a search bar.
- Even if someone did want to recreate HN's support for OpenID in news.arc, HN's approach is based on an external service, Clickpass, whose only documentation is "Clickpass is being reworked. Please, don't develop against the current Clickpass implementation." http://www.clickpass.com/docs
- A while back I linked to a few discussions of websites based on news.arc (http://arclanguage.org/item?id=13277)... but since then, every single one of the sites mentioned has either gone down or turned into a different kind of site. ^_^; Except for HN and Arc Forum, that is! These two are the only live news.arc-based sites I know.
On the plus side, if you're plucky enough to climb this mountain, we're happy to help guide you in the right direction. Specific questions and examples of the code you're trying to write are good places for us to start helping you.
If you'd prefer to keep the code you write to a minimum, I'm afraid I'd have to recommend some other platform. Personally I think just setting up a group on Convore is a pretty good way to get an HN-style forum. :-p (Unfortunately, Convore's apparently been terribly slow for months. @_@ ) If you need the freedom to modify the forum code, there might be a good starting point somewhere in the Django world (https://code.djangoproject.com/wiki/ForumAppsComparison) or of course you might be able to use one of the many PHP forum engines. Those languages aren't nearly as nifty as Arc ;) but they will likely have a much higher volume of community support.
Hopefully someone else will come to the rescue with even better options. :-p
Thanks for your detailed response. I have a development background - mostly scripting on mainframe systems, but no web development experience. The HN format seems so clean and simple I figured there would be a lot of support for it somewhere even if it is not written in ARC in the same way that Digg has a bunch of open source options for someone who wants to 'clone' the site. Yes site administration would be a big consideration for me too - Is HN as simple on the backend for the admin as it is for users on the front end. If developing and deploying the site turns out to be out of my depth I would gladly outsource it to someone else but knowing where to start in terms of what open source options are out there that are currently closest to the goal of what I want to achieve comes first. The PHP bbs wont work for me. So, still looking...
Getting a HN clone up and running isn't too bad, and the version of news.arc available is very stable as is. Also, the folk in this forum are very friendly and willing to answer questions. I don't think that getting your HN clone up and running would be much more difficult than any other forum.
Tip: if you are looking to customize news.arc, I would start by working it out of the table formatting and into CSS. If not, it's a pretty rigid beast. It's worth the effort, and would also be a good way to get familiar with the code.
As an FYI aw (aka CatDancer) once provided code for arc/OpenID support. -> http://arclanguage.org/item?id=3333. Link is no longer valid and aws currently published pages[1] don't appear to contain the code. I'm sure if someone needed it they could contact him and he would be willing to help[2].
There is no example in official Arc. If we're talking hypothetically, here's one way to make sense of it:
arc> (= x '(2))
(2)
arc> (= (~car x) t)
t
arc> (~car x)
t
arc> x
(nil)
arc> (= (~x 0) nil)
nil
arc> (~x 0)
nil
arc> x
(t)
Since (~foo ...) only ever returns nil or t, this kind of place can't faithfully store other values:
arc> (= (~x 0) "something unique and special")
Error: Can't assign "something unique and special" as boolean
-- or --
arc> (= (~x 0) "something unique and special")
"something unique and special"
arc> (~x 0)
t
I don't see much purpose to implementing this feature, since every time someone would type (= (~foo bar) baz), they could just type (= foo.bar no.baz) instead.
Like akkartik before me, I'm having some trouble. Here's my story:
Started from a barely used Linux Mint 12 installation.
Installed libgmp-dev, check, and git. (The packages pkg-config,
autoconf, automake, and libtool were already installed.)
Created an SSH key and set it up with GitHub.
mkdir -p ~/mine/prog/repo
cd ~/mine/prog/repo
git clone git://github.com/dido/arcueid.git
cd arcueid
autoreconf -i
./configure
make
Got the following error, among other output:
/bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -MT compiler.lo -MD -MP -MF .deps/compiler.Tpo -c -o compiler.lo compiler.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -MT compiler.lo -MD -MP -MF .deps/compiler.Tpo -c compiler.c -fPIC -DPIC -o .libs/compiler.o
compiler.c: In function 'macex':
compiler.c:81:9: error: 'debug' undeclared (first use in this function)
compiler.c:81:9: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [compiler.lo] Error 1
Do a git pull. Oops, some debugging code I was using temporarily made it into the repo. Will try to ensure this doesn't happen again. Last commit should be: fdeb76087b48ff19d9f2666d05a4429ac4145b24
to the end of /etc/ld.so.conf and then run ldconfig -v. This is a frequent issue when it comes to shared libraries that get installed in /usr/local/lib. I have no idea why most Linux distros don't like to put /usr/local/lib in ld.so.conf by default. It is extremely irritating, since most everything one builds from source installs there by default.
Interestingly, this is what /etc/ld.so.conf looked like before:
include /etc/ld.so.conf.d/*.conf
And this was in /etc/ld.so.conf.d/libc.conf:
# libc default configuration
/usr/local/lib
The "include" line doesn't seem to do what it looks like it's supposed to do. :-p The man page for ldconfig doesn't mention "include" at all (nor #-comments...). It would be funny if Linux Mint came with this kind of ld.so.conf but its version of ldconfig thought "include" was just another directory in the list.
