That doesn't actually address my concern, but this (from the same page) does:
(if (amb #f #t)
1
(amb))
This has to force the first amb to return #t, or the whole expression would fail. I still find it somewhat odd, though, that I can (fail) on the next line, but I suppose there's not a better way to handle it; after all, that's probably just because the interpreter is sequential.