id summary reporter owner description type status priority milestone component version resolution keywords cc 236 pluggable connection hints, plan for Tor Brian Warner "dawuud and I worked out a plan for making it easier to run Foolscap over Tor and other network layers. The first API change will be to make `Tub.listenOn()` accept a new (optional) `advertise=` argument. The idea is that each Listener knows how to figure out what connection-hints will reach it, and these hints are used when `advertise=` is set to its default of `""AUTO""`. Any other value overrides that figure-it-out-yourself logic. The built-in TCP Listener uses IP-address autodetection to compute this. While doing this, we should also backport the improved ipaddr-autodetection changes from Tahoe. Then we introduce ""foolscap connection plugins"". There are two categories: one for Listeners, and a second for outbound connection hints. The Listener plugins are used to translate the string passed into `Tub.listenOn()` into a Twisted server endpoint (`IStreamServerEndpoint`). The basic TCP listener uses `tcp:0` or `tcp:PORT` or `tcp:PORT:interface=IFADDR` as usual, and the plugin is a no-op. These plugins are also responsible for figuring out the right connection-hints (for `advertise=AUTO`). They'll implement an interface vaguely like this: {{{ class IFoolscapServerConnectionHelper(Interface): typename = ""onion"" def how_to_listen(listenspec): return IStreamServerEndpoint # or str def what_to_advertise(IListeningPort): return connection_hint }}} The Connection plugins are used to translate a FURL's connection-hints into actual Twisted client endpoints. Each connection-hint is parsed enough to figure out the hint type (e.g. ""tcp""), then passed into the matching plugin. The interface looks something like: {{{ class IFoolscapClientConnectionHelper(Interface): typename = ""tor"" def make_client_endpoint(connection_hint): # add client-private stuff, like SOCKS port return IStreamClientEndpoint # or str. Endpoint must be able to .startTLS }}} The [https://txtorcon.readthedocs.org/en/latest/ txtorcon] package makes it possible to create a client Endpoint that uses a local Tor daemon to connect to an arbitrary DNS name, IPv4 address, or `.onion` hidden service. By adding a Connection plugin that handles `tor:XYZ.onion:80`, Foolscap becomes capable of using `tor:`-prefixed connection hints in FURLs. Note that Twisted already has functions to convert from string to endpoint: `twisted.internet.endpoints.serverFromString()` and `clientFromString()`. These use a set of *Twisted* plugins (using Twisted's slightly-weird non-pip/setuptools-based plugin mechanism, the one with a .plugins file in $PYTHONPATH). The txtorcon package installs a Twisted plugin so that `endpoints.clientfromString(""tor:HOST:PORT:otheroptions"")` will use Tor to connect to HOST/PORT. The endpoints can (or will, some day) be configured to use a pre-existing Tor relay (with a SOCKS and/or control port), or to launch a new instance (by providing a path to the Tor executable, and maybe a persistent directory for it to store state). txtorcon also makes it pretty easy to launch hidden services. We used to think that it was a good idea to use interpret the FURL's connection hints directly as Twisted client endpoint specification strings, but then came to our senses. The problem is that the ""otheroptions"" fields are powerful: for txtorcon, these fields are used to tell the plugin where to find the local Tor proxy, and control which directory is used for persistent state, etc. These fields must not be controlled by an external party. So the endpoint should either be constructed as a normal python object (combining host/port arguments from the FURL, with locally-defined proxy settings), or the endpoint specification string must be carefully assembled from the same pieces (guarding against attacks like `host=""XYZ.onion:socksProxy=attacker.com:1234`, which would reveal the client's address to the attacker). The server specification string, on the other hand, is entirely controlled by the local admin. So it would nominally be ok to use e.g. `Tub.listenOn(""onion:80:controlPort=9052:hiddenServiceDir=/path"")`. But if the socks-port/control-port must be given to the Foolscap plugin for client purposes, then it probably makes sense to give them to the Foolscap plugin for Listener purposes too. So the next step is to write Tor-for-Foolscap plugins. Application code could then do {{{ tub.registerPlugin(TorListenerPlugin(control_port=XYZ)) tub.registerPlugin(TorConnectionPlugin(control_port=XYZ)) # or TorConnectionPlugin(tor_exe=TORPATH, state_dir=DIR) tub.listenOn(""onion:80"") tub.getReference(furl_with_tor_hints) }}} All information about the Tor configuration is stored in the plugins, and used when constructing the client/server Endpoints. The listener's `what_to_advertise()` method would figure out the hidden-server onion address, and return a connection hint of `tor:FOO.onion:80`. This Tor-for-Foolscap package would include a connection-plugin which used Tor client endpoints to talk to all hosts, not just .onion services. We'll add another API to foolscap to clear the plugin table. Then, an application which wants to *only* use Tor for everything (to hide its own IP address) would do: {{{ tub.removeAllPlugins() tub.registerPlugin(TorListenerPlugin(tor_stuff)) tub.registerPlugin(TorConnectionPlugin(tor_stuff)) tub.registerPlugin(TorForTCPPlugin(tor_stuff)) }}} and all subsequent `.listenOn()` and `.getReference()` calls would use Tor exclusively. Finally, to help Tahoe use this, we'd change Tahoe to install these plugins if it sees a `[tor]` section in tahoe.cfg, and to replace its `Tub.setLocation()` call with a corresponding `Tub.listenOn(.., advertise=)` argument. Tahoe needs to delegate its what-is-my-ip-address code to Foolscap. " enhancement closed major 0.9.0 network 0.7.0 fixed