Arc Forumnew | comments | leaders | submitlogin
Contributing to the Anarki Arc Test Suite
3 points by shlomif 6033 days ago | 4 comments
The Anarki Arc Test Suite is found in the t/.t. A few of which are written in Perl 5 and the rest are written in Arc itself. They all use http://testanything.org/ .

Up to a few hours ago, the test suite required my Test::Run TAP harness, which has quite a few dependencies. However, I've now written a more minimalistic (and less user-friendly) test harness based on TAP::Parser ( http://search.cpan.org/dist/TAP-Parser/ ), which has fewer dependencies. You can find it on Mini-Test.pl and invoke it using "make test" or "make check".

To write automated tests for Arc look at arctap.arc , or take a look at the existing scripts (t/.arc.t) .

It would be a good idea to beef up the test suite to increase the code coverage, and to give more examples, that can also be used as documentation and specification. They are also useful for preventing regressions.

One sub-optimal fact at the moment is that I haven't found a way to query the line numbers of the function call-stack, so one can pinpoint the error only using the description text that needs to be added manually.



2 points by conanite 6033 days ago | link

In arctap.arc

  (def not (c) (if c nil 1))
I think (no c) does what you want here?

Rainbow has a fairly comprehensive test suite for the arc foundation (260 tests) - see http://github.com/conanite/rainbow/tree/master/src/java/arc/... - please feel free to steal tests :)

It has a very simple runner - http://github.com/conanite/rainbow/tree/master/src/java/arc/... - somewhat inspired by junit, it supports the concept of nested tests nested inside test suites (in fact the tests started life in junit)

The test runner is still a bit limited - I haven't thought about how to test sockets or threads yet, for example.

It would be great to have a complete suite for arc so we can play spot-the-difference more easily when the next version comes out.

In fact, it might even be useful, for each test, to declare which implementations and versions it should pass for, so that we can run the suite and say with confidence "this conforms to Anarki 3.14, and to arc17", for example.

-----

1 point by shlomif 6033 days ago | link

(no c) indeed does what I want, but I find its name counter-intuitive and not psychologically sound. Saying (if (no x)) does not sound as natural as (if (not x). I could define not by saying

  (= not no)
Or:

  (def (not x) (no x))
Etc. I recall searching for not in the arc sources and getting nothing because "no" was used instead. It's "not" in Perl 5, Ruby, etc. - not "no", so I assumed Graham would use it, but he didn't.

Regarding the tests - I can integrate them into my own test suite, but I only accept code that is under the MIT X11 licence, so I may not be able to copy-paste code like that. (I still don't know what licence the original Arc code is under).

Regarding your suggestion for implementation/version declaration, this seems interesting and is supported in TAP using TODO and SKIP tests, but will naturally require more logic and complexity.

-----

1 point by conanite 6032 days ago | link

see "copyright" in Arc distribution:

  This software is copyright (c) Paul Graham and Robert Morris.  Permission to use it is granted under the Perl Foundations's Artistic License 2.0.
Rainbow, ditto. In any case, you would need to adapt my tests to fit your framework, so it wouldn't exactly be copy-paste.

-----

1 point by absz 6033 days ago | link

The reason it's called no is that that is, in fact, what you're asking: the value representing false is the empty list, so there is (no c) there.

I'm not convinced I like the choice, but it is consistent, and does make sense.

-----