Foolscap

"Foolscap" is the name for the next-generation RPC protocol, intended to replace Perspective Broker. It has been under development (on and off) by Brian Warner for several years.

The current release is foolscap-0.1.6 (gpg sig).

The two primary responsibilities of an RPC protocol are to:

1. serialize an object graph, and reconstruct a useful equivalent on the remote side
2. allow the holder of a RemoteReference in one process to invoke methods on a Referenceable in a different one

In addition, there are details like object naming, connection establishment, controlling object serialization, reference lifetime, and error reporting.

New features

(relative to oldpb, aka twisted.spread)

• most inert Python types are serializable, including unicode and sets
• clients and servers are implemented as ​Services, which share connections when possible and are easy to shut down
• links are encrypted/authenticated by default (using SSL)
• all objects are accessed through secure/unguessable "FURLs"
  • explicitly published objects can be accessed through well-known FURLs
• you can declare method signatures (with "constraints", either in Interface classes or as method attribute/decorators)
  • this enables defenses against resource-consumption DoS attacks, including stack-frame-exhaustion attacks
  • (TODO) you can ask the deserializer about your current maximum memory/stackframe exposure, based upon what objects are visible and what methods they declare
• serializers for third-party classes can be registered using Adapters
• serializers are more "streaming" than in oldpb
  • serializers can pause themselves, deferring serialization until later
  • serializers can be paused when the network pipe is full
• newpb is architected to make it possible to rewrite serializers/deserializers in C, for speed
• object graph depth is limited by available heap memory, not available stack depth

Show Me The Code

The current release is available here. This is an alpha release, since the protocol has not been finalized, and incompatible changes are being made every few months. Older releases are available here.

The latest Foolscap code lives in a ​mercurial repository, at ​http://foolscap.lothar.com/repos/ . To retrieve a copy of the trunk use the following command:

hg clone http://foolscap.lothar.com/repos/trunk foolscap-trunk

There are two online source-code browsers, the native hgwebdir.cgi at ​http://foolscap.lothar.com/repos/trunk (which also has a ​gitweb-style view), and the Trac Mercurial plugin at the Browse Source link above. Both are pretty handy.

The documentation is in the doc/ directory and includes both the protocol specification and a short user's manual. Rendered HTML from the most recent release lives here, and using-foolscap.html is probably the best place to start.

Some day, Foolscap may be shipped as part of the "Twisted-Core" distribution, but only after the protocol and code have stabilized a bit.

Status

Foolscap is not yet ready for production use. The todo.txt​ and jobs.txt​ files contain some of the things that need finishing before it will be ready. Connection negotiation needs to be changed to use variable-length certificate digests and certificate chains and variable-length object names, various questions about making objects "giftable" by default or not need to be answered and implemented, and we need to acquire operational experience with Foolscap as a whole to think about usability.

Release 0.1.6 (as well as older releases) are available in the releases directory.

Links

• Foolscap takes many ideas from the ​E Language. Foolscap is my attempt to make the concepts of E available to python+twisted programmers.
• The Twisted wiki has ​a page on Foolscap where this information used to live.

Trac Hints

The old Trac starting page was moved to OldWikiStart

For a complete list of local wiki pages, see TitleIndex.