Arc Forumnew | comments | leaders | submitlogin
How to handle Racket types?
3 points by krapp 2336 days ago | 31 comments
Racket types like #t and #f seem not to be recognized by Arc, but they can be returned from functions that use Racket libraries. I'm currently stuck parsing a JSON response because of this, getting 'unrecognized type' errors. What's the method for handling this?


2 points by akkartik 2336 days ago | link

Try https://github.com/arclanguage/anarki/blob/master/lib/json.a...?

-----

3 points by hjek 2327 days ago | link

Someone should get rid of json.rkt and json.arc from the Anarki repo.

Racket already has a built-in JSON parser[0] that works just fine with Arc.

[0]: https://docs.racket-lang.org/json/index.html

-----

3 points by krapp 2336 days ago | link

I am, and json.rkt is modified for Arc but the json response is still returning Racket types, which is weird:

    #hash((challenge_ts . 2018-08-03T12:59:44Z)
          (hostname . testkey.google.com) 
          (success . #t))

-----

3 points by i4cu 2336 days ago | link

I think the problem might be that Arc doesn't support some of data types that you are getting from the json response. For example Arc does not support booleans true/false it only has 'nil' or not a nil value. That's why, in the past, I used Andrews library [1].

That said I haven't had a running install of arc in many years so I could be off base.

1. https://github.com/awwx/lib/blob/master/fromjson2.arc

-----

4 points by akkartik 2335 days ago | link

Looking through the Anarki logs I think I'm seeing signs of our first edit war. It's been very gradual so we haven't noticed until now.

Anarki used to use Andrew's library from http://awwx.ws/fromjson (perhaps an older version of your link, i4cu) I switched it out back in 2009 for a faster replacement (https://github.com/arclanguage/anarki/commit/dad4dc6662), not realizing that I was breaking the read side in this rather obvious manner. Since then a bunch of work has happened on emitting JSON, but apparently nobody tried parsing JSON until now.

We could retrofit these bugfixes, but I think it would just give up the speed gains to parse things character by character. May be worth just returning to Andrew's version.

But I'll try working with the existing version first, just in case it's still fast.

-----

4 points by i4cu 2335 days ago | link

Just my two cents, but if I were using Arc today I'd consider doing what could be done to conform to the edn spec [1]. It was born out of Clojure, but is not Clojure specific. Also, as I understand it json is actually a subset of edn thus the two would not collide. But for now I think just getting a json parser running would be a win.

https://github.com/edn-format/edn

-----

4 points by krapp 2335 days ago | link

Possibly a silly question but... could we just add support for those types in Arc?

I mean, as long as Racket interop is going to be a thing, maybe using the boolean type isn't a bad idea.

And by "we" I mean "probably you," of course.

-----

2 points by hjek 2327 days ago | link

Arc already supports Racket booleans:

    arc> (if $.#f 'foo 'bar)
    bar
    arc> (if $.#t 'foo 'bar)
    foo
They work, so what is the problem?

Also, I don't get why the Anarki repository contains a JSON parser implemented in Racket, when the built-in Racket JSON parser works just fine with Arc:

    arc> ($ (require json))
    arc> ($.write-json (obj foo "bar"))
    {"foo":"bar"}#<void>
    arc> (w/instring in "{\"foo\":\"bar\"}" ($.read-json in))
    #hash((foo . "bar"))
I fail to see the problem with the Racket built-in JSON parser here. Would anyone care to explain?

-----

3 points by krapp 2326 days ago | link

I got an error saying #t was an unknown type when I tried to do a boolean check on it, as it was passed from a JSON response. It's the reason I started this thread to begin with.

-----

2 points by hjek 2326 days ago | link

Would you mind preparing a simple code example to show this issue? I'm not sure exactly what you are doing.

-----

2 points by krapp 2326 days ago | link

Sure... the if statement in verify-captcha is where the problem occurs.

It currently works but that's only because json.rkt has been modified.

    (def post-getjson (url params (o port stdin))
      (fromstring (post-url url params) 
        (read-json (port))))

    (def recaptcha-response (s r)
      (post-getjson "https://www.google.com/recaptcha/api/siteverify"
      (list 'secret s 'response r)))

    (def verify-captcha (req)
      (do
        (= resp (recaptcha-response recaptcha-secret* 
        (alref (req 'args) "g-recaptcha-response")))
      (if resp!success 
        (pr "success") 
        (pr "failure"))))

-----

2 points by hjek 2326 days ago | link

> when I tried to do a boolean check on it,

Can I ask how you did a "boolean check" on #t?

-----

2 points by akkartik 2326 days ago | link

Thanks! I think you're right that the existing JSON parser is unnecessary. But there's still an open problem about translating to Arc:

    arc> (w/instring in "{\"foo\": true}" ($.read-json in))
    #hash((foo . #t))
We'd like this to translate to:

    #hash((foo . t))
Perhaps what we're looking for isn't related to JSON at all, but a general shim for translating the output of Racket functions. That could be broadly useful.

-----

2 points by hjek 2326 days ago | link

Why do we need to explicitly translate #t to t? Arc has no problem handling Racket booleans.

I'd like to see perhaps a specific brief example, where using Racket booleans within Arc actually is a problem, because I haven't encountered any myself.

If I try to use the hash from your example in actual Arc code, it works fine:

    arc> (if ((w/instring in "{\"foo\": true}" ($.read-json in)) 'foo) 'yes 'no)
    yes
    arc> (if ((w/instring in "{\"foo\": false}" ($.read-json in)) 'foo) 'yes 'no)
    no

-----

2 points by akkartik 2326 days ago | link

Wow, that's kinda mind-blowing. I had no idea. Thanks!

Ah, here's perhaps the only special-case we need to handle:

    arc> (= x (w/instring in "{\"foo\": null}" ($.read-json in)))
    #hash((foo . null))  # expected: #hash((foo . nil)) or #hash()
    arc> (if x!foo (prn "error"))
    error

-----

2 points by hjek 2326 days ago | link

Nice example. (So much easier to tell what's going on when you have these brief examples)

Actually the Racket docs[0] clarifies the situation with null in JSON:

    (json-null) → any/c

    (json-null jsnull) → void?
      jsnull : any/c
> This parameter determines the default Racket value that corresponds to a JSON “null”. By default, it is the 'null symbol. In some cases a different value may better fit your needs, therefore all functions in this library accept a #:null keyword argument for the value that is used to represent a JSON “null”, and this argument defaults to (json-null).

If you set the JSON null to nil before running your example, it works as you'd expect:

    arc> ($.json-null nil)
    #<void>
    arc> (= x (w/instring in "{\"foo\": null}" ($.read-json in)))
    #hash((foo . nil))
    arc> (if x!foo (prn "error"))
    nil
I think we should get rid of json.rkt and use the Racket built-in. It's way better documented, and we should use that one. (But I'm not going to delete json.rkt myself, particularly when I know someone is working with it.)

[0]: https://docs.racket-lang.org/json/index.html

-----

2 points by rocketnia 2326 days ago | link

For round-tripping between JSON and Arc, I would expect the JSON values {}, {"foo": null}, {"foo": false}, and {"foo": []} to parse as four distinct Arc values.

I recommend (obj), (obj foo (list 'null)), (obj foo (list 'false)), and (obj foo (list (list))). Arc is good at representing optional values as subsingleton lists:

  ; Access a key that's known to be there.
  t!foo.0
  
  ; Access a key that isn't known to be there.
  (iflet (x) t!foo
    (do-something-then x)
    (do-something-else))
Using my `sobj` utility, you can write (obj foo (list 'false)) as (sobj foo 'false).

Meanwhile, I don't think there's any big deal when we don't have Arc-style booleans....

  ; Branch on the symbols 'false and 'true.
  (if (isnt 'false x)
    (do-something-then)
    (do-something-else))
  
  ; Alternative
  (case x true
    (do-something-then)
    (do-something-else))

-----

2 points by hjek 2326 days ago | link

> ; Branch on the symbols 'false and 'true.

We do already have booleans that work in Arc without any conversion. Please see http://arclanguage.org/item?id=20492

There's absolutely no need to convert the booleans to symbols and other hackery.

-----

3 points by rocketnia 2326 days ago | link

I liked it when it was returning #f.

But now that I look closer at the ac.scm history (now ac.rkt in Anarki), I realize I was mistaken to believe Arc treated #f as a different value than nil. Turns out Arc has always equated #f and 'nil with `is`, counted them both as falsy, etc. So this library was already returning nil, from Arc's perspective.

There are some things that slip through the cracks. It looks like (type #f) has always given an "unknown type" error, as opposed to returning 'sym as it does for 'nil and '().

So with that in mind, I think it's a bug if an Arc JSON library returns 'nil or #f for JSON false, unless it returns something other than '() for JSON []. To avoid collision, we could represent JSON arrays using `annotate` values rather than plain Arc lists, but I think representing JSON false and null using Arc symbols like 'false and 'null is easier.

-----

2 points by hjek 2326 days ago | link

Having nil represent both false and the empty list is also what Common Lisp does.

Really, #t and #f are not proper Arc booleans[0], so it makes sense that Arc can't tell what type they are.

You can configure the value Arc chooses for a JSON null with the $.json-null function, which I think is fine as JSON APIs might have differing semantics.

[0]: http://arclanguage.github.io/ref/combining.html

-----

3 points by rocketnia 2325 days ago | link

That documentation may be wrong. On the other hand, it may be correct in the context of someone who is only using Arc, not Racket.

There are a lot of ways to conceive of what Arc "is" outside of the Racket implementations, but I think Arc implementations like Rainbow, Jarc, Arcueid, and so on tend to be inspired first by the rather small set of operations showcased in the tutorial and in arc.arc. (As the tutorial says, "The definitions in arc.arc are also an experiment in another way. They are the language spec.") Since #f isn't part of those, it's not something that an Arc implementation would necessarily focus on supporting, so there's a practical sense in which it's not a part of Arc our Arc code can rely on.

(Not that any other part of Arc is stable either.)

-----

3 points by i4cu 2325 days ago | link

> Really, #t and #f are not proper Arc booleans[0], so it makes sense that Arc can't tell what type they are.

Really, #t and #f are not proper Arc anything, but the language apparently handles them so IMHO Arc should also be able to know what type they are. Otherwise, I fear, this will become a Hodge Podge language that will lose appeal.

Personally I don't care if Arc supports booleans. I only care that it can translate booleans (when need be) to a meaningful Arc semantic. That said, if we're going to support booleans then let's not create partial support.

-----

2 points by i4cu 2327 days ago | link

$.#f means you need to add a handler on a value, that's not handling the value as it's supposed to.

-----

2 points by hjek 2326 days ago | link

Is the $ the handler you are referring to?

You don't need to add any $ handlers to handle Racket booleans. They can be passed directly to Arc functions.

The $ is just there to get some Racket booleans.

-----

2 points by i4cu 2326 days ago | link

sorry I don't have an install. I've been assuming:

  (if #f 'foo 'bar)
 
would error.

edit: and now I have and install... :)

-----

1 point by hjek 2326 days ago | link

> sorry I don't have an install.

http://tryarc.org/

-----

3 points by akkartik 2326 days ago | link

In fairness, tryarc.org it isn't always up lately. evanrmurphy has broached the subject of someone taking it over, but it hasn't happened yet.

-----

2 points by hjek 2326 days ago | link

Yes, looks like it's plain Arc 3.1.

-----

2 points by akkartik 2336 days ago | link

Can you share the input JSON? Let's treat this as a bug and see if we can fix it.

-----

3 points by krapp 2335 days ago | link

It's just the default Recaptcha response found here[0]:

    {"success": true,  
     "challenge_ts": "2018-08-03T22:31:05Z",  
     "hostname": "testkey.google.com"}
[0]https://developers.google.com/recaptcha/docs/verify

edit: I went ahead and pushed my current working branch[1] if anyone wants to see it.

[1]https://github.com/kennethrapp/anarki/tree/dev-test

-----

2 points by hjek 2327 days ago | link

Great, that looks fine. Then you can just access the `success` key like this:

    ($ (require json))
    
    (if
       ((w/instring json "{\"success\":false}"
                          ($.read-json json))
                          'success)
       'success ; proceed to login
       'fail    ; must be a robot then
     )

-----