The other problem then becomes: how do we implement, say, (import ...) or (in-package ...) ? Should the reader silently filter them out? If not, how does 'eval notify the reader that the package is changed? If someone defines foo::in-package and bar::in-package, which one does the reader return, and how will 'eval know if the package should be changed?

Yet another problem would be intrasymbol syntax: should the reader leave foo::bar!nitz, or should it process it into (foo::bar 'foo::nitz) or even (foo::bar 'quux::nitz) if (import quux::nitz foo::nitz) is used?

Yet another problem is the "standard" package, i.e. CL-USER in CL. Obviously all symbols from this package are imported into all packages.... or should they? The problem is a problem which PG ignored in ArcN: backwards compatibility. If I write code today which uses the symbol "convoke" as, say, a table in my function and tomorrow PG decides to create a macro "convoke" which does something else, kaboom! my code dies.

In my opinion we should separate packages into several interfaces.

Basically, suppose I release a package, AlmkglorSuperSupremePanPizza, and I define a "version 1" of this interface:

  (in-package AlmkglorSuperSupremePanPizza)
  (interface v1
    eat drink be-merry)

  (import AlmkglorSuperSupremePanPizza::v1)

  (in-package AlmkglorSuperSupremePanPizza)
  (interface v1
    eat drink be-merry)
  (interface v2
    ;include v1 interface elements
    v1 for-tomorrow-we-die)

Anyway, I've been thinking rather deeply about packages and module systems and the like. One major reason for using a symbol-based package/module system is types: for example, if packageA defines a 'gaz type, and packageB defines a 'gaz type also, obviously the types are incompatible and they shouldn't be the same.

Also, for a potential solution from within the Scheme implementation, consider instead moving the problem from the reader to the evaller.

Instead of having symbol packages be handled by the reader, we might instead define a new builtin type, 'eval-cxt. An 'eval-cxt object is simply an evaluator, but understands packages and the (in-package ...) and (import ...) etc. syntaxes.

The reader simply reads in symbols blindly, without caring about the exact package they should go to. This simplifies the reader and allows us to continue using the reader and writer for saving plain Arc data. Instead, any particular context for evaluation is put into the 'eval-cxt object. 'eval-cxt will understand 'import etc. forms, and will perform the translation of all symbols into their qualified, package-based counterparts, i.e.:

  (= evaller (eval-cxt))
  (= tmp (read))
  user input> '(hello world)
  => (quote (hello world))

  (evaller tmp)
  => (arc-user::hello arc-user::world)

  (def tl ()
    (let my-eval (eval-cxt)
      ((afn ()
         (pr "arc> ")
         (write:my-eval:read)
         (self)))))

  (in-package sample)
  (= private-table
     (table 'foo 42
            'bar 99))
  (mac foo ()
    `(private-table!foo))

Incidentally, the 'eval function can still be expressed rather simply by this manner:

  (def eval (e)
    ((eval-cxt) e))

  (w/uniq no-param
    (def eval (e (o p no-param))
      (let evaller (eval-cxt)
        (unless (is p no-param)
          (evaller `(in-package ,p)))
        (evaller e))))

I think I'll keep your interfaces idea. It's much more elegant than a list of symbols (although it isn't necessarily much different underneath).

I don't really understand the problem with 'in-package. Is this a SNAP-specific problem or does it affect Arc generally? Won't things work the same way as in CL?

Some other thoughts I had about this which may be useful: modules are usually kept in a file, so a 'load-in-package function which takes a package argument might be useful

Also, I thought it would be good to have a read-macro to switch packages. I'll reuse #: as it's not needed in my system:

  #:foo (some expressions)

I like the idea of symbols being read in without a package, but then getting a package at eval time. One way to implement this may be to store all the symbols in a special package when they are read, then eval can move these to a new package when it evaluates them. This makes packages very dynamic.

Which is the point, of course ^^

The other point is disciplining package makers to make package interfaces constant even as newer versions of the package are made. This helps preserve backward compatibility. In fact, if the ac.scm and arc.arc functions are kept in their own package, we can even allow effective backward compatibility of much of Arc by separating them by version, i.e.

  (using arc v3)
  (using arc v4)
  (using arc v5)

  ; tell the reader that package 'foo has a symbol 'in-package
  (= foo::in-package t)
  ; enter package foo
  (in-package foo)
  ; now: does the reader parse this as (in-package ...) or (foo::in-package ...)
  (in-package bar)

It's somewhat SNAP-specific, since we cannot have a stateful, shared reader, but I suspect that any Arc implementation that supports concurrency of any form will have similar problems with having readers keep state across invocations. The alternative would be having a monadic reader.

> Won't things work the same way as in CL?

Not sure: I never grokked anything except the basics of CL packages.

> #:foo (some expressions)

How about in a module file? It might get inconvenient to have to keep typing #:foo for each expression I want to invoke in the foo package, which means we really should think deeply about how in-package should be properly implemented.

> One other thought I've had: package names should be strings. Otherwise, 'foo::bar actually becomes 'foo::foo::bar, which is really 'foo::foo::foo...::bar etc.

If we don't allow packages to have sub-packages, then a name that is at all qualified will quite simply directly belong to that package, i.e. foo::bar is always foo::bar, as long as :: exists in the symbol.

I'm assuming you can use it like this:

  #:foo 
  ((def (x) (+ 1 x))
   (def (y) (expt y))
   (= bar 123))

  #:foo ((load "filename"))

> If we don't allow packages to have sub-packages, then a name that is at all qualified will quite simply directly belong to that package

True, but it will be a bit confusing:

  (import 'foo::bar 'foo::quux)
  (import 'foo::bar 'baz::quux)

  (def load (file (o hook))
    " Reads the expressions in `file' and evaluates them.  Read expressions
      may be preprocessed by `hook'.
      See also [[require]]. "
    (push current-load-file* load-file-stack*)
    (= current-load-file* file)
    (or= hook idfn)
    (after
      (w/infile f file
        (whilet e (read f)
          (eval (hook e))))
      (do (= current-load-file* (pop load-file-stack*)) nil)))

It still looks like a stateful 'read to me, and I don't want a stateful 'read at all, because a file might want to directly use 'read:

  $ cat getconfig.arc
  (= configuration (read "my.cfg"))

For that matter I'd prefer to keep the package definitions in the file itself, rather than have to remember to put the file in a package:

  $ cat mine.arc
  (in-package mine)

  (def mine ()
    (prn "this is my mine!!"))

  - (def load (file (o hook))
  + (def load-in-package (package file (o hook))
  - (whilet e (read f)
  + (whilet e (read-in-package package f)

> (import 'foo::bar 'foo::quux)

Oops, that should be

  (import 'foo::quux 'baz::quux)

We move all state information into a new object type called a "context". It can be constructed without parameters via the 'cxt function:

  (cxt)
  => <implementation-specific>
  (type (cxt))
  => arc::cxt

  (eval-cxt)
  => <implementation-specific>
  (type (eval-cxt))
  => arc::eval-cxt

  (let ob (eval-cxt)
    (ob x))
  ==>
  (let ob (cxt)
    (eval:ob x))

The context object accepts a plain read expression (with unpackaged symbols) and emits an s-expression where all symbols are packaged symbols.

It is the context object which keeps track of the current package, so you might have some accessor functions to manipulate the context object (e.g. destructure it into the current package, etc.).

The read function is stateless and simply emits unpackaged symbols, and emits packaged symbols if and only if the given plaintext specifically includes a package specification.

A package object is a stateful, synchronized (as in safely accessible across different threads, and whose basic operations are assuredly atomic) object. A context is a stateful object intended for thread- and function- local usage.

context objects

===============

A context object is callable (and has an entry in the axiom::call* table) and has the following form:

  (let ob (cxt)
    (ob expression))

1. If the expression is one of the following forms (the first symbol in each form is unpackaged, 'symbol here is a variable symbol):

  (in-package symbol)
  (interface symbol . symbols)
  (using symbol)
  (import symbol symbol)

2. For all other forms, it returns an equivalent expression, but containing only packaged symbols. The state of the context is not changed.

The forms in number 1 above have the following changes in the context or current package of the context:

  (in-package symbol)

  (interface symbol . symbols)

If the interface already exists, it is checked if the lists are the same to the existing list. If it is not the same, the implementation is free to throw an error.

  (using symbol)

For conflicting package interfaces: let us suppose that the context is in package 'User, and there exists two package interfaces, A::v1 and B::v1. A::v1 is composed of (A::foo A::bar) while B::v1 is composed of (B::bar B::quux). If the context receives (using A::v1), the User package contains the mapping {foo => A::foo, bar => A::bar}. Then if the context receives (using B::v1), the User package afterwards contains the mapping {foo => A::foo, bar => B::bar, quux => B::quux}.

  (import symbol symbol)

Continuing the example above, after (import A::bar A-bar), this changes the package to {foo => A::foo, bar => B::bar, A-bar => A::bar, quux => B::quux}

package objects

===============

A package object is callable and has the following form:

  (ob expression)

1. The same symbol, if the given symbol is a packaged symbol; this does not change the state of the package

2. A packaged symbol, if the given symbol is an unpackaged symbol. If the package does not contain a mapping for the unpackaged symbol, the state of the package is changed so that a mapping for the unpackaged symbol to a packaged symbol exists.

The package object also supports an 'sref operation:

  (sref ob v k)

Packages are handled by interface.

Further, we also predefine two packages, axiom and arc.

The axiom package contains the following symbols:

  axiom::t
  axiom::nil
  axiom::fn
  axiom::if
  axiom::quote
  axiom::quasiquote
  axiom::unquote
  axiom::unquote-splicing
  axiom::set
  axiom::call*

The arc package contains all "standard" Arc functions and macros. The arc package is not implicitly imported into all packages.

The arc package contains the interface arc::v3 . This interface is the set of symbols currently defined on Anarki. Future extensions to the arc standard library must be first placed in the interface arc::v3-exp until they are placed into a future arc::v4 interface, and so on.

load

====

The load implementation is thus:

  (def load (file (o hook))
    " Reads the expressions in `file' and evaluates them.  Read expressions
      may be preprocessed by `hook'.
      See also [[require]]. "
    (push current-load-file* load-file-stack*)
    (= current-load-file* file)
    (or= hook idfn)
    (after
      (w/infile f file
        (let evaller (eval-cxt)
          (evaller '(in-package User))
          (whilet e (read f)
            (evaller (hook e)))))
      (do (= current-load-file* (pop load-file-stack*)) nil)))

  (if (eq x 10) (in-package foo) (in-package bar))

Also in CL, the value of package* doesn't always correspond to the actual current package. For example

  (setf x "I'm in the default pacakge!")
  (setf foo::x "I'm in the FOO package!")
  (setf *package* foo)
  (print *package*)
  (print x)

  #<PACKAGE FOO>
  "I'm in the FOO package!"

  #<PACKAGE FOO>
  "I'm in the default pacakge!"

    ;; In package 'a
    (mac afn (parms . body)
      `(rfn self ,parms ,@body))

    ;; In package 'b
    (import 'a::afn 'afn)

    ;; A stupid example function
    (afn () self)

    ;; The translation process
    (b::afn () b::self)
    
    (rfn a::self () b::self)
    
    (let a::self nil
      (set a::self
           (fn () b::self)))