A possible implementation: use the Anarki repository as the website and build the package manager on top of git. I don't know much about gems so I don't know if this would work, but it would save a lot of reinventing the wheel.
To knock down my own suggestion: git isn't suitable for this because it doesn't have any way of partially downloading a repository. At least, none that I can find.