DotCloud [1] is built from the ground up around distributed services. We use a custom rpc mechanism called ZeroRPC. It uses ZeroMQ [2] for transport, msgpack [3] for serialization, and some extra glue for RPC mechanics, discovery, event dispatching, etc.
Among other neat tricks, it allows us to handle synchronous RPC calls and asynchronous message passing in a unified way. It also maps transparently to any ZeroMQ topology.
Combine this with the recent release of gevent support for zeromq [4] and you get a fun, fun playground :)
Oh, awesome, thank you!. I never thought to use ZeroMQ, but that would work perfectly. Ironically, it doesn't seem like DotCloud supports these sorts of architectures very well yet, any plans to fix that? ;)
Absolutely - our job is to make developers happy and productive. Whenever we find a sweet tool that makes our lives easier, we're practically tripping over ourselves to share it with you
Our network team is working on some major goodies which should make you happy. Personally they bring tears to my eyes :)
In the meantime, my recommendation is to start with http. If you keep the RPC semantics simple, you can easily migrate to a different transport later. And you'll need HTTP to authenticate and encrypt at the boundaries anyway.
Among other neat tricks, it allows us to handle synchronous RPC calls and asynchronous message passing in a unified way. It also maps transparently to any ZeroMQ topology.
Combine this with the recent release of gevent support for zeromq [4] and you get a fun, fun playground :)
[1] http://www.dotcloud.com
[2] http://www.zeromq.org
[3] http://msgpack.org
[4] https://github.com/zllak/gevent-zeromq