Cool idea! I used to do something a long time ago: use a little-known option of ctags (-r) to match wikiWords, so that hitting C-] in vim or M-. in emacs with the cursor on say 'databaseSubsystem' would take you to '$<databaseSubsystem>' in the codebase.
The nice thing about it is that a name could potentially belong to multiple different tags/narratives. Multiple overlapping trails is a pretty decent way to narrate an intrinsically non-linear thing like a codebase. See also http://leoeditor.com.
For the connective narrative, how about using something like the existing [[ref]] markers that were in the docstrings to provide 'tags' for each entry? Every entry would then be associated with a set of tags that includes it's own name + any tag mentioned in its docstring.
This gets you two things.
1) You can now provide grouped details pages, like the 'list operations' and
'macros' pages from the arc3.1 docs.
2) You can provide connective narrative associated with a particular tag, in a separate document.
It probably gets you other things too. Maybe we could add the ability on the command line to filter help by tag instead of just name. That would be nice.
I like the examples stuff. The only issue that I have with it is that you are providing explicit, yet non-tested results. It makes sense to me to either provide just the code and have the documentation system generate the results, or to provide test cases instead.
Ideally, the 'results' in the documentation would still be computed dynamically, so you can always see whatever the current output would be, but also have the expected results available for separate regression testing.
Also, I think it would be great if the regression test results could be automatically generated and stored somewhere else, because I don't want to have to manually type in expected results for every case. Some mechanism to 'freeze' the current output as the accepted value for regression purposes would be cool.
Marginalia is clojure specific so I expect it will not help other than to provide ideas.
To create an arc equivalent you probably need build an arc library which provides some code inspection/dissection capabilities and ideally also be able to attach metadata to any given function or macro. With such a library you then build a script to auto generate the docs.
As for GitHub syncing; well no, I'm guessing users would need to trigger the script and then check in the updated docs.
This is still better for a few reasons...1. developers can generate docs, locally, that are in sync with the code base they are actually using (checked out or branched). 2. Even if the online docs gets out of sync for a while you're still only a script trigger away for updating all outstanding changes.
The alternative is what you just went through; having someone remind you to do the work manually as an after-thought, which I've only seen happen once.
The crux is colocating the rendered docs online with the repo. Would marginalia help us use github's hosting with github pages, managing branches, etc? If it does I think I'd be willing to go on a significant undertaking.
Something like marginalia[1] might prove to be better than the arcfn docs. Not only because the docs would be fully integrated with the source code, but because it would also solve the multiple dialects problem. i.e. If some given code can be tagged with a dialect name then automation could also apply a dialect filter.
Of course this would probably be quite a bit of an undertaking.
I've updated the docs. I've also taken out the help/ dir entirely and inlined all the docstrings into arc.arc and elsewhere.
I'm still unsure how to organize the arcfn reference guide at http://arclanguage.github.io/ref so that we remember to update it when we make changes, and so it's convenient to update the website. Another complication is that the arclanguage account contains multiple dialects of arc with subtle differences, and the current organization of documentation is misleadingly monolithic. Any suggestions to fix this most appreciated. (We discussed this previously a year ago: http://www.arclanguage.org/item?id=17774)
Absolutely. I was speaking only in the context of making use of multiple cores.
I see now that I overstated how bad things are when I said it's as if there's only one thread. Since I/O can be done in parallel and accessing in-memory data is super fast, atomic isn't as bad as I thought for the past 5 years.
"The net effect is as if the HN server has only one thread, since most threads will be blocked most of the time."
Well, in JavaScript, I do concurrency by manually using continuation-passing style and building my own arbiter/trampoline... and using it for all my code. If I ever do something an easier way, I have to rewrite it eventually. Whenever I want to try out a different arbiter/trampoline technique, I have to rewrite all my code.
Arc's threading semantics are at least more automatic than that. Naive Arc code is pretty much always usable as a thread, and it just so happens it's especially useful if it doesn't use expensive 'atomic blocks (or the mutators that use them).
"Automatic" doesn't necessarily mean automatic in a good way for all applications. Even if I were working in Arc, I still might resort to building my own arbiters, trampolines, and such, because concurrency experiments are part of what I'm doing.
All in all, what I mean to say is, Arc's threads are sometimes helpful. :-p
Arc does have threads, yes, but it also has a style of mutating in-memory globals willy-nilly. As a result, all its mutator primitives run in atomic sections (http://arclanguage.github.io/ref/atomic.html#atomic) at a deep level. The net effect is as if the HN server has only one thread, since most threads will be blocked most of the time.
I can't find links at the moment, but pg has repeatedly said that HN runs on a single extremely beefy server with lots of caching for performance.
Edit: Racket's docs say that "Threads run concurrently in the sense that one thread can preempt another without its cooperation, but threads do not run in parallel in the sense of using multiple hardware processors." (http://docs.racket-lang.org/guide/concurrency.html) So arc's use of atomic wouldn't matter in this case. It does prevent HN from using multiple load-balanced servers.
Looking back, I see that I did indeed inaccurately answer zck's question about "running single-threaded". I'd like to amend my answer to "No, it runs multi-threaded, but the threads use a single core." Rocketnia is right that arc has concurrency but not parallelism.
I think you're telling fibs. :-p I double-checked srv.arc (which defines 'defop), and the code there opens a thread for every request. This is true in Anarki, in official Arc 3.1, and even way back in Arc0.
Even without parallelism, this would come in handy to prevent I/O operations from pausing the whole server.
"It sounds hard to believe using a single core is a problem of mzScheme."
(Psst, if you say "mzScheme," I feel the need to remind you that Arc works on the latest versions of Racket, which have long since dropped the name "mzscheme".)
Generally, threads are for concurrency, not necessarily parallelism. They're a workaround for an imperative, sequence-of-side-effects model of computation, which would otherwise force us to choose which subcomputation should come first. In Racket, this kind of workaround is their only purpose.
Racket has two features for parallelism, and they're called "futures" and "places":
I'm finding out about these for the first time, but I'll summarize anyway.
Futures are a lot like threads, but they're specifically for speculative parallelism. They're allowed to break some invariants that threads would have preserved, and (as per the nature of speculative parallelism) some of their computations may be thrown away.
Places use shared-nothing concurrency with message passing, and each place runs in parallel.
So although news.arc does spawn a thread to handle every server request, it would probably need to use Racket's futures to take advantage of multi-core systems--and even then, I'm guessing it would need some fine-tuning to avoid wasting resources. If it used Racket's places, that could make its resource usage easier to reason about (but not necessarily better!), but it would require even more substantial refactoring.
You would have to make sure that all the variables used in arc are shared behind locks. That's a big enough problem that it hasn't been attempted, to my knowledge.
I don't think this itself would be called Church numerals, but it's related. The Church encoding takes an ADT definition like this one and looks at it as a polymorphic type. Originally we have two constructors for Nat whose types are as follows:
Zero : Nat
Succ : (Nat -> Nat)
These two constructors are all you need to build whatever natural number you want:
We could make this function more general by abstracting it over any type, not just Nat:
buildMyNat : a -> (a -> a) -> a
This type (a -> (a -> a) -> a) is the type of a Church numeral.
While it's more general in this way, I think sometimes it's a bit less powerful. Dependently typed languages often provide induction and recursion support for ADT definitions, but I think they can't generally do that for Church-encoded types. (I could be wrong.)
For something more interesting, we can go through the same process to build a Church encoding for my binary integer example:
data OneOrMore = One | Double OneOrMore | DoublePlusOne OneOrMore
data Int = Negative OneOrMore | Zero | Positive OneOrMore
buildMyInt :
OneOrMore -> -- One
(OneOrMore -> OneOrMore) -> -- Double
(OneOrMore -> OneOrMore) -> -- DoublePlusOne
(OneOrMore -> Int) -> -- Negative
Int -> -- Zero
(OneOrMore -> Int) -> -- Positive
Int
buildMyInt :
a -> -- One
(a -> a) -> -- Double
(a -> a) -> -- DoublePlusOne
(a -> b) -> -- Negative
b -> -- Zero
(a -> b) -> -- Positive
b
"What was the reason you decided to do it this way? It seems more complicated to work with."
In my designs, I don't just want to make things that are easy for casual consumers. I want to make things people can consume, understand, talk about, implement, upgrade, deprecate, and so on. These are things users do, even if they're not all formal uses of the interface.
I hardly need number types most of the time. If I add them to my language(s), they might just be dead weight that makes the language design more exhausting to talk about and more work to implement.
Still, sometimes I want bigints, or at least numbers big enough to measure time intervals and pixels. I don't think any one notion of "number" will satisfy everyone, so I like the idea of putting numbers in a separate module of their own, where their complexity will have a limited effect on the rest of the language design.
---
"Huh, this is similar to Church numerals"
I'm influenced by dependently typed languages (e.g. Agda, Twelf, Idris) which tend to use a unary encoding of natural numbers in most of their toy examples:
data Nat = Zero | Succ Nat
To be fair, I think they only do this when efficiency isn't as important as the simplicity of the implementation. In toy examples, implementation simplicity is pretty important. :)
A binary encoding might use a technique like this:
data OneOrMore = One | Double OneOrMore | DoublePlusOne OneOrMore
data Int = Negative OneOrMore | Zero | Positive OneOrMore
Or like this:
data Bool = False | True
data List a = Nil | Cons a (List a)
data Int = Negative (List Bool) | Zero | Positive (List Bool)
I get the impression these languages go to the trouble to represent these user-defined binary types as efficient bit strings, at least some of the time. I could be making that up, though.
For what I'm talking about, I don't have the excuse of an optimization-friendly type system. :) I'm just talking about dynamically typed cons cells, but I still think it could be a nifty simplification.
Sorry, I think I was mistaken. If the hypertext reference is generated from docstrings at all, I don't see how. Even if there were a tool, it would require some manual work afterward to place the definitions into appropriate categories.
arc> (help do)
[mac] (do . args)
Evaluates each expression in sequence and returns the result of the
last expression.
See also [[do1]] [[after]]
This is awesome. Yes, worth updating. Let me think about how to sync it with the reference. (I'd also forgotten that the reference is generated from anarki. Is that still true, or is this copy redundant? Need to check..)
It's funny to see you make this change to 'pluralize, 'cause I was thinking of exactly the reverse change for everything else.
In the language(s) I'm working on, the number type has mainly been a way to represent list indexes and lengths. List processing depends on integer processing, or so I thought. The other day I realized I could just use lists instead.
; Scheme-ish code
> (list-ref '(a b c) '())
a
> (list-ref '(a b c) '(z))
b
> (list-ref '(a b c) '(y z))
c
> (len '(a b c))
(() () ()) ; or even (a b c)
This unary encoding won't be efficient for big numbers, but these aren't very big numbers. They're no bigger than other cons lists that I already have in memory, and they might even share the same structure.
So I'm thinking that even if I do support other number types, I'll very rarely need to use them for counting first-class objects. They'll mostly be for counting pixels, milliseconds, etc.
pluralize
" Returns `str' pluralized. If `n' is 1 or a list of length 1, `str' is
returned unchanged; otherwise an `s' is appended.
See also [[plural]] "
plural
" Returns a string \"<n> <x>\" representing `n' of `x' in english, pluralizing
`x' if necessary.
See also [[pluralize]] "
(def pluralize "n str" "Returns str pluralized; if n is 1 or a list of length 1, str is returned unchanged; otherwise an 's' is appended. Renamed from plural in arc3."
(tests (pluralize 2 "fox") (pluralize '() "fish")))
(def plural "n str" "Returns n and str pluralized. New in arc3."
(tests (plural 2 "fox") (plural '() "fish")))
