Arc Forumnew | comments | leaders | submitlogin

This came up during ar development, although I don't remember who brought it up. I think aw was about to implement it but was concerned that it would be annoying at the REPL to have all the program's bindings be looked up eagerly (aka for them to not be "hackable"). It could be annoying for mutually recursive macros, too, although those are already tricky enough that it probably wasn't a big concern at the time.

I remember recommending an upgrade: Instead of generating:

  `(,my-func ,a ,b)
Generate something that preserves the late binding of mutable global variables, like this:

  `((',(fn () my-func)) ,a ,b)
I seem to remember aw wasn't convinced this cruft was going to be worth whatever hygiene it gained, even after I recommended a syntactic upgrade:

  `(,late.my-func ,a ,b)
Since aw values concision a lot, perhaps the issue was that most programmers would surely just write ,my-func in the hope that it would be rare (or even incorrect) to ever want to rebind it, and then other programmers would suffer from that decision.

But Pauan followed a design path like this in Nulan and/or Arc/Nu. In Pauan's languages... actually, I think I remember a couple of approaches, and I don't remember which ones were real.

One approach I think I remember is that `(my-func a b (another-func c) d) inserted mutable boxes for my-func and another-func into the result, but it would just implicitly unquote a, b, c, and d, because variables at the beginning of a list are likely to be mutable globals and variables in other positions are likely to refer to gensyms.

There might have even been an auto-gensym system in that quasiquote operator at some point.

---

I liked this approach a lot at the time. That was the same time I was working on Penknife. I was trying to avoid name collision with first-class namespaces in Penknife, but I still wanted macros to work, and I was trying to take a very similar approach. (I couldn't take exactly the same approach because my macroexpansion results were strings; instead, I think I allowed regions of the macroexpansion result to be annotated with a first-class namespace to use for name lookup.)

When Penknife's compile times were abysmally long, I started to realize that even if I found an optimization, this was going to be a problem with macros in general. Anyone can write an inefficient macro, and anyone who can put up with it can build a lot of stuff on top of it that other users won't be able to appreciate. So I started to require separate compilation in my language designs.

With separate compilation in mind as a factor for the language design, it no longer made sense to put unserializable values into the compiled code. Instead, in Penknife's case, I devised a system of namespace paths to replace them. The namespaces were still first-class values, but one thing you could do with a Penknife macro was get hold of its first-class definition-site namespace, so the macroexpanded code could refer to variables in distant namespaces by specifying a chain of names of macros to look them up from. And this kept things hackable, too, since you could mutate a macro's definition-time namespace explicitly (not that many programs or REPL sessions would bother to do that).

Not long after, I set a particular goal to make a language (Era) where the built-in functionality was indistinguishable from libraries. Builtins aren't hackable from within the language, so the module system needs to make it possible (and ideally easy) for people to write non-hackable libraries.

(Technically they only need to be non-hackable to people who don't know the source code, because once you know the source code, you know you're not dealing with builtins. I intend to take advantage of this to make the language hackable after all, but it's going to take essentially a theorem prover in the module system before it's useful to tell the module system you have the source code of a module, as opposed to just using that source code to compile a new module of your own behind the module system's back.)

Anyhow, this means I haven't put hackability at the forefront for a long time.

I think the embedding-first-class-values approach will work, and I think late binding is workable (by using late.my-func) and there's a workable variation of that late binding approach to enable separate compilation too (by using namespace paths made out of chains of macro names). So I like it, but I just have this stuff to recommend for it to make it really tick. :)

---

By the way, it's good to see you! I wondered how you were doing.


Check out John Shutt's thesis on Kernel: https://web.cs.wpi.edu/~jshutt/kernel.html. He takes your observation that quasiquotation is pointless to the limit, dropping it entirely and constructing his macros out of cons and friends.

By the way, "what you'd expect" is apparently not what I expected:

  ; SBCL
  * (defmacro plus (&rest args) (cons #'+ args))
  PLUS
  * (plus 1 2 3)
  ; in: PLUS 1
  ;     (#<FUNCTION +> 1 2 3)
  ;
  ; caught ERROR:
  ;   illegal function call
  
  ; CLISP
  [1]> (defmacro plus (&rest args) (cons #'+ args))
  PLUS
  [2]> (plus 1 2 3)
  *** - EVAL: #<SYSTEM-FUNCTION +> is not a function name; try using a symbol instead
  
  ; Racket
  > (eval (list '+ 1 2))
  3
  > (eval (list + 1 2))
  3
3 points by zck 1 day ago | link | parent | on: About lobste.rs

Thanks! I'll spend some time checking it out. :)
3 points by akkartik 1 day ago | link | parent | on: About lobste.rs

I'll invite you.
3 points by zck 1 day ago | link | parent | on: About lobste.rs

I've never signed up for lobste.rs. Could I get an invite? What do you need to invite me? (email in profile)
2 points by i4cu 1 day ago | link | parent | on: About lobste.rs

> I don't believe so.

I shouldn't post in the middle of the night... of course it's many-to-many... I was attempting to say I don't believe it will be a problem.

The rest of my comment should hold true.

2 points by i4cu 1 day ago | link | parent | on: About lobste.rs

> Tags imply a many-to-many relationship.

I don't believe so, but it depends on what relationships and queries you have in mind.

I think you just need to maintain an index of story ids for each tag then alter the page list code to check each story against chosen tag indexes in order to apply custom filters. And you will also need to bypass the page caching operations for these filtered pages (just given the number of combinations that are possible).

Obviously db capabilities would make it (and everything else) much better, but it's not a showstopper.

4 points by i4cu 2 days ago | link | parent | on: Inline JavaScript

> What's needed is the ability to pass custom headers from the application to srv.arc

There is the possibility of just putting the CSP into a meta tag within the page header, but I didn't suggest that because not all CSP options are available when using the meta tag.

I think you're right in that being able to dynamically add headers is the right way to go. When I moved from arc to clojure I did this by implementing something like arc templates [1] and used them to pass attributes through to the server ops. I ended up with a 'defop' like call that took an options hash-map argument (i.e. a template instance) which then generated the headers dynamically (with built in sane defaults).

1. http://arclanguage.github.io/ref/template.html

> A lot of that can be removed altogether by removing the table layout and just using a basic grid...

Yeah the whole thing should get HTML5-alived. CSS, JS and web-standards have evolved significantly since the app was originally written.

3 points by i4cu 2 days ago | link | parent | on: Inline JavaScript

Strict CSP settings are a form of whitelisting what js, css etc, is valid thus protecting from injection. Inline code for both js and css can't be whitelisted like header items can be so they will fail (unless you use the hash code hack mentioned for js).

Css is vulnerable too (since at least 2009):

https://scarybeastsecurity.blogspot.com/2009/12/generic-cros...

"By controlling a little bit of text in the victim domain, the attacker can inject what appears to be a valid CSS string. It does not matter what proceeds this CSS string: HTML, binary data, JSON, XML. The CSS parser will ruthlessly hunt down any CSS constructs within whatever blob is pulled from the victim's domain...."

Furthermore:

https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP

"A policy needs to include a default-src or script-src directive to prevent inline scripts from running, as well as blocking the use of eval() . A policy needs to include a default-src or style-src directive to restrict inline styles from being applied from a <style> element or a style attribute."

So it's just the 'style' attribute people worry about and strict CSP manages.

2 points by hjek 2 days ago | link | parent | on: Algolia HN Search source

Has anyone used this with Anarki, ever?
3 points by hjek 2 days ago | link | parent | on: Inline JavaScript

> 4. All inline style attributes need to be removed and changes to news.css or news.js will need to be made in order to compensate.

Wat. Wow, browsers today! Is CSS vuln by default? Is that really necessary?

3 points by krapp 2 days ago | link | parent | on: About lobste.rs

Tags imply a many-to-many relationship, don't they? How difficult would that be to do efficiently in News, without a proper relational database?
3 points by krapp 2 days ago | link | parent | on: Inline JavaScript

>srv.arc needs the addition of a Content-Security-Policy header for server ops (with the appropriate settings).

What's needed is the ability to pass custom headers from the application to srv.arc (or maybe app.arc) since CSP headers would be application specific. Unfortunately, unless I'm wrong, it looks like header generation is baked into srv.arc.

> All inline style attributes need to be removed and changes to news.css or news.js will need to be made in order to compensate. i.e. stuff like this:

A lot of that can be removed altogether by removing the table layout and just using a basic grid. There's no reason the forum has to be pixel-perfect. This would have the added benefit of letting us get rid of a lot of hacky one-off table macros in html.arc.

3 points by i4cu 2 days ago | link | parent | on: About lobste.rs

I like the invitation tree concept. I plan to do that with one of my apps, but I think I would have to add that feature well after I obtain a decent user base.
3 points by i4cu 2 days ago | link | parent | on: Inline JavaScript

In order for the News app to have a CSP and be strict about it, you would need to:

1. Remove the inline js. This means the votelink code (votejs) needs to be moved from news.arc and put into an external file (news.js?) that is linked to as a file within the header.

2. The inline onclicks need to change. The onclick values have to be actual function pointers not strings and given Arc has no built in js functionality that likely means removing them completely. Instead you will need to have a js call in the new 'news.js' file that does document.addEventListener with the 'DOMContentLoaded' argument along with a function that finds all the relevant doms for a given page and adds listeners to each that will trigger the votejs code.

3. srv.arc needs the addition of a Content-Security-Policy header for server ops (with the appropriate settings).

4. All inline style attributes need to be removed and changes to news.css or news.js will need to be made in order to compensate. i.e. stuff like this:

   (div style "margin-top:1px;margin-bottom:0px")
edit #1. note that adding the hash code referred to (or even the 'nonce' option) is a hack intended to provide short term relief to production environments until proper changes can be implemented.

edit #2. regarding point 4 I believe (but not absolutely sure of) that all the inline font, color, font-size tags are a problem too. i.e. It's any tagged string value that will be interpreted by the browsers css engine. If anyone can confirm this, please do. Either way, none of that stuff is HTML 5 compliant and probably should be removed anyway.

2 points by hjek 2 days ago | link | parent | on: Self-hosting the Anarki community

Looks like banned IPs are written to the disk even:

    (def set-ip-ban (user ip yesno (o info))
      (= (banned-ips* ip) (and yesno (list user (seconds) info)))
      (todisk banned-ips*))
3 points by i4cu 3 days ago | link | parent | on: Self-hosting the Anarki community

Looks as though it ranks how bad you are and always keeps the baddest of the bad-asses in cache, while never deleting any from disk. In a low volume site like this I doubt you'll get out of it without contacting them.
2 points by akkartik 3 days ago | link | parent | on: Self-hosting the Anarki community

Hmm, I wonder if it stays banned until they restart the server. I'd ping hn@ycombinator.com.
2 points by hjek 3 days ago | link | parent | on: Self-hosting the Anarki community

Amazing you managed to get your IP banned!

Hacker News has an IP unpanning procedure[0] but I don't think Arc Forum has one.

In the Arc 3.1 code there is a function `set-ip-ban` for unbanning users, but no `unban` op.

(Someone should add that to Anarki, actually.)

[0]: https://news.ycombinator.com/item?id=4761102

3 points by shader 3 days ago | link | parent | on: Self-hosting the Anarki community

Anyone know how long an IP stays banned? Or what I can do about it? Rather inconvenient not to be able to check the forum from home...

I suppose I could set up a proxy or something. And I was planning on scraping everything to a new community site anyway, so maybe I should take this as the incentive to do so.

3 points by i4cu 9 days ago | link | parent | on: Advanced search for news.arc

As I understand it - the 'Open Source' movement concerns itself with improving the software by making the code openly accessible, where as the 'Free Software' movement concerns itself with a fighting for users rights (i.e. having the freedom to access, modify and distribute the code in a manner that empowers the user).

And so, an 'Open Source' repository holds code that is openly accessible for the purpose of improving the software. Where as an 'Ethical Repository' holds code that is graded by its' ability to guarantee users rights according to a specific set of morals (established by free software foundation). It so happens that open source repos tend to align well the ethics associated with free-software, but they should not be mistaken for each other. As an example to illustrate: If a repo SaaS were built for open source code, but restricted users from a certain country it wouldn't rank high in ethical repository grading. This is because while having the code openly accessible leans towards a Grade A rating (excellent), the restricting some users part puts it at a Grade F rating (unacceptable).

-- additional info --

https://en.wikipedia.org/wiki/Open-source_software

"Despite initially accepting it,[31] Richard Stallman of the FSF now flatly opposes the term "Open Source" being applied to what they refer to as "free software". Although he agrees that the two terms describe "almost the same category of software", Stallman considers equating the terms incorrect and misleading.[32] Stallman also opposes the professed pragmatism of the Open Source Initiative, as he fears that the free software ideals of freedom and community are threatened by compromising on the FSF's idealistic standards for software freedom.[33] The FSF considers free software to be a subset of open-source software, and Richard Stallman explained that DRM software, for example, can be developed as open source, despite that it does not give its users freedom (it restricts them), and thus doesn't qualify as free software.[34]"

1 point by hjek 9 days ago | link | parent | on: Advanced search for news.arc

I think free (or "open source") and ethical mean the same in most cases.

Exceptions might include something like Facebook, which is technically somehow usable w/o non-free JS when using their basic mobile web page, but where the company is still engaging in other unethical activities, like selling user data to sway elections.

Or something like Amazon, where you might possibly be able to buy something w/o non-free JS (haven't checked), but where the treatment of their employees is unacceptable.

But, I think, when we're talking git hosting sites, there's no difference?

But FSF considers Gitlab ethical enough for hosting GNU packages[0].

[0]: https://www.fsf.org/news/gnu-releases-ethical-evaluations-of...

2 points by i4cu 10 days ago | link | parent | on: Advanced search for news.arc

I'm pretty sure it would be similar to ad-blockers. The initial implementations are trivial and easily circumvented, but as they evolve they become more useful overall.

Plus note that I was just suggesting that it would be more effective than a social movement with 'ethical repositories'. Just imagine if the ad-blocker devs tried the same strategy...

1 point by krapp 10 days ago | link | parent | on: Advanced search for news.arc

It seems like the thesis here is that whether or not "non-trivial" Javascript (which is just about all Javascript in the wild) should be trusted depends on the presence of an explicit GPL license. If so, that doesn't seem like a reliable heuristic for a script blocker to me.
2 points by i4cu 10 days ago | link | parent | on: Advanced search for news.arc

Actually, I got the notion from Stallman's original post 'The Javascript Trap' [1].

"Finally, we need to change free browsers to detect and block nontrivial nonfree JavaScript in web pages. The program LibreJS detects nonfree, nontrivial JavaScript in pages you visit, and blocks it. LibreJS is included in IceCat, and available as an add-on for Firefox."

However I am opposed to that call for action given it's an all-or-none implementation. I feel it's the role of each country to regulate, which is why I expressly suggested it as a configuration option (ideally it could be enforced at the browser level country by country and if not then user by user).

1. https://www.gnu.org/philosophy/javascript-trap.html

2 points by hjek 10 days ago | link | parent | on: Advanced search for news.arc

> IMO; If they really wanted to make a dent, they should push for a regulation requiring that browsers provide functionality that enforces a free-software configuration OPTION.

Sounds interesting. Apart from the regulation part, it sounds a bit like LibreJS[0].

[0]: https://www.gnu.org/software/librejs/

2 points by hjek 10 days ago | link | parent | on: Advanced search for news.arc

Good question. I'm not sure, but I think it may have to do with removing Gravatar support?
3 points by zck 11 days ago | link | parent | on: With and withs

Similarly to this, when I'm writing Java, I use `final`^1 everywhere I can. It's nice to be able to know that anywhere later where the variable declared final is in scope, it will have the same value as at the point it's set. I don't need to look through any code to see if it's rebound; I know it hasn't been.

[1] "final" is kind of like "const", if I understand `const` right. `final int x = 3;` means that it is an error to later have the line of code `x = 4;`.

3 points by prestonbriggs 11 days ago | link | parent | on: With and withs

OK, I get it, thanks. In scheme, I would use letrec for this situation; my intuition for Arc isn't very well developed.
More