root/ChangeLog

2008-10-15  Brian Warner  <warner@allmydata.com>

        * misc/*/debian/rules (install/python-foolscap): include
        misc/classify_foolscap.py in the debian package, uncompressed, so
        it goes into /usr/share/doc/python-foolscap/classify_foolscap.py ,
        so you can make a symlink to it from an incident-gatherer.

        * misc/classify_foolscap.py (TUBCON_RE): update to match
        foolscap-tubconnector messages for both old and new versions of
        Foolscap, and for python2.4 and 2.5 .

        * foolscap/logging/incident.py (IncidentClassifier): add --verbose
        option to 'flogtool classify-incident' to show the trigger
        dictionary for any unclassifiable incidents. This is useful when
        developing classification functions.
        * foolscap/test/test_logging.py (Incidents.test_classify): test it

        * Makefile (test-figleaf): oops, fix the test-figleaf-poll target,
        by making the test-figleaf target use $(TRIAL), not hardcoded
        'trial'.

        * foolscap/_version.py: bump revision to 0.3.2+ while between releases
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

2008-10-14  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py: release Foolscap-0.3.2
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

        * MANIFEST.in: add misc/classify_foolscap.py to source distribution

        * NEWS: update for the next release

        * Makefile (test-poll): add a convenience target, for running
        tests on a system with the broken combination of pyopenssl and
        twisted that requires the pollreactor
        (test-figleaf-poll): same

2008-10-14  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/tail.py (LogTail): include the retrieved PID in
        the saved logfile. Part of #80.
        * foolscap/logging/incident.py (IncidentReporter.incident_declared):
        also include the PID in the incident report's header.
        * doc/specifications/logfiles.xhtml: document the ['versions']
        and ['pid'] fields in incident reports and the 'flogtool tail'
        savefile.


        * foolscap/logging/web.py: mark any incident triggers in the
        logfile, and add links to them from the welcome page. Closes #79.

        * foolscap/logging/web.py (Reload): add a 'Reload Logfile' button
        to the web-viewer's Welcome page. Not automatic, but it means you
        don't have to get back to the shell and restart the viewer. Works
        well-enough to say Closes #103.

        * misc/classify_foolscap.py: plugin to classify some
        foolscap-internal incidents

        * foolscap/logging/cli.py: new "flogtool classify-incident"
        subcommand: given an incident, say what categories it falls into.
        Closes #102.
        * foolscap/logging/gatherer.py (IncidentGathererService): factor
        out the classification pieces into IncidentClassifierBase
        * foolscap/logging/incident.py (IncidentClassifierBase): same
        (IncidentClassifier.run): support for the new CLI command
        * foolscap/test/test_logging.py (Incidents.test_classify): test it

        * foolscap/logging/gatherer.py
        (IncidentGathererService.classify_incident): change the classifier
        function signature: now it just takes a single 'trigger' dict.
        * foolscap/test/test_logging.py: update to match
        * doc/logging.xhtml (gatherer): same

        * foolscap/logging/gatherer.py
        (IncidentGathererService.add_classify_files): make the incident
        gatherer look in its base directory for "classify_*.py" files, use
        them as plugins with classification functions.
        (INCIDENT_GATHERER_TACFILE): update example text to match

        * foolscap/test/test_logging.py (IncidentGatherer): test it
        (Gatherer.test_wrongdir): exercise another error case
        * doc/logging.xhtml (gatherer): document it

2008-10-13  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/publish.py (IncidentSubscription.catch_up):
        Don't tell the gatherer about incidents that don't have triggers:
        these are malformed logfiles, such as the zero-length files that
        result from a process being terminated before it manages to write
        anything.

        * foolscap/logging/log.py (Count): replace itertools.count with a
        version that doesn't overflow at 2**31-1 (thanks to Zooko for the
        patch). Closes #99.
        (FoolscapLogger.__init__): use it

        * foolscap/pb.py (Listener.startFactory): add log facility
        identifiers
        (Listener.stopFactory): same
        (Listener.buildProtocol): same
        (Tub.getReference): same

2008-09-20  Brian Warner  <warner@lothar.com>

        * doc/listings/xfer-client.py (_failure): do sys.exit(1) upon
        failure, so the caller can detect it via the exit code

2008-09-10  Brian Warner  <warner@allmydata.com>

        * foolscap/test/test__versions.py (Versions.test_required): check
        for bug #62 (openssl>0.7, twisted<=8.1.0, selectreactor) and print
        a warning if it is likely that the tests would fail, to remind the
        user to re-run with -r poll.
        (Versions.test_required): oops, guard import of OpenSSL on having
        crypto available, otherwise this test gets an error when OpenSSL
        is not installed.

2008-09-08  Brian Warner  <warner@lothar.com>

        * doc/listings/xfer-client.py: use os.path.expanduser() on the
        filenames passed in by the user, to enable things like
        --furlfile ~/.upload.furl

        * foolscap/_version.py: bump revision to 0.3.1+ while between releases
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

2008-09-03  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py: release Foolscap-0.3.1
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

        * NEWS: update for next release

2008-09-02  Brian Warner  <warner@lothar.com>

        * foolscap/negotiate.py (TubConnector.__repr__): add more
        information to the repr, to help track down #81. Made TubConnector
        a new-style class in the process, to make upcalling easier.
        (TubConnector.checkForIdle): improve the log message

        * foolscap/pb.py (Tub._log_gatherer_connected): use callRemoteOnly
        to pass the logport to the gatherer: we don't need to hear about
        any problems it has.

        * foolscap/logging/gatherer.py
        (IncidentGathererService.classify_stored_incidents): reclassify
        everything that isn't already present in one of the classified/*
        files. This makes it a lot easier to iterate over the [start
        gatherer; see what is unknown; update classifiers; remove
        classified/unknown; repeat] loop. Also log classification events
        better. Closes #94.
        * foolscap/test/test_logging.py (IncidentGatherer.test_emit): test it

        * foolscap/test/common.py (PollMixin): replace the use of chained
        Deferreds with a task.LoopingCall-based version, from Tahoe. This
        avoids the weird and annoying maximum-recursion-depth-exceeded
        error that occurs when the check function is called more than
        about 600 times. Closes #95.

2008-08-29  Brian Warner  <warner@lothar.com>

        * foolscap/logging/app_versions.py (versions): move the
        application version dict (which gets reported to remote
        subscribers, and copied into logfiles) to a separate module. It
        was causing circular import problems when it lived in an attribute
        of LogPublisher.
        (add_version): provide a setter method
        * foolscap/logging/publish.py (LogPublisher.versions): same
        (LogPublisher.remote_get_versions): same
        * foolscap/logging/incident.py (IncidentReporter.incident_declared):
        same

        * foolscap/test/test_logging.py (IncidentGatherer.test_emit):
        improve test shutdown a little bit

2008-08-28  Brian Warner  <warner@lothar.com>

        * foolscap/logging/incident.py (IncidentReporter.incident_declared):
        put versions in the incident header too, also for #80.
        * foolscap/test/test_logging.py (Incidents.test_basic): test it

        * foolscap/logging/tail.py (LogPrinter.got_versions): record
        remote application versions to the --save-to file, in the header.
        Part of #80.
        (LogPrinter.got_versions): oops, control where stdout goes
        * foolscap/logging/dumper.py (LogDumper.start): show those
        versions with --verbose
        * foolscap/test/test_logging.py (Dumper.test_dump): update to match
        (Tail.test_logprinter): same, make sure got_versions is called

        * foolscap/logging/gatherer.py (IncidentObserver): only fetch one
        incident at a time, to limit the size of the sender's outbound
        queue. This should help close #85.
        (IncidentGathererService.new_incident): include the classification
        results in the per-incident log messages
        * foolscap/test/test_logging.py (IncidentGatherer.test_emit): tests

        * foolscap/logging/log.py (FoolscapLogger.declare_incident):
        combine overlapping incidents, by passing new triggers to an
        existing reporter instead of creating a new one. Helps with #85.
        * foolscap/logging/incident.py (IncidentReporter.new_trigger): new
        method to add subsequent triggers to an existing reporter. This
        doesn't do anything yet. It should be improved to record the other
        triggers in a trailer (since it's too late to add it to the
        header). Every triggering event will make it into an incident
        somewhere, but the report-file analysis tools may not know how to
        pay attention to the subsequent triggers.
        * foolscap/logging/interfaces.py (IIncidentReporter.new_trigger): same
        * foolscap/test/test_logging.py (Incidents.test_overlapping): test it
        * doc/logging.xhtml: mention incident-coalescing

        * foolscap/logging/log.py: add bridge-foolscap-logs-to-twisted
        functionality, set up by either calling bridgeLogsToTwisted(), or
        by setting the FLOGTOTWISTED environment variable (to anything).
        The default filter will exclude events below the OPERATIONAL
        severity level, and those generated by foolscap internals (i.e.
        facility.startswith("foolscap") ). Closes #93.
        * foolscap/pb.py (Tub.setOption): update to match
        * foolscap/test/test_logging.py (Bridge): tests for it
        (Publish): update to use new APIs
        * doc/logging.xhtml: docs

2008-08-25  Brian Warner  <warner@lothar.com>

        * foolscap/broker.py (Broker.abandonAllRequests): map both
        ConnectionLost and ConnectionDone into DeadReferenceError, so that
        application code only needs to check for one exception type.
        * foolscap/negotiate.py (Negotiation.evaluateNegotiationVersion1):
        when an existing connection is dropped in favor of a new one, drop
        it with DeadReferenceError instead of ConnectionDone. The mapping
        in Broker.abandonAllRequests doesn't seem to quite catch
        everything in unit tests.
        (Negotiation.acceptDecisionVersion1): same, when we're the slave
        * foolscap/test/test_call.py (TestCall.test_connection_lost_is_deadref):
        test it
        (TestCall.test_connection_done_is_deadref): same
        (TestCall.testChoiceOf): switch to use ShouldFailMixin
        * foolscap/test/test_gifts.py (Gifts): remove
        ignoreConnectionDone, just look for DeadReferenceError

2008-08-21  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): bump revision to 0.3.0+ while
        between releases
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

2008-08-04  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): release Foolscap-0.3.0
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:

2008-08-04  Brian Warner  <warner@lothar.com>

        * foolscap/logging/interfaces.py
        (RILogPublisher.subscribe_to_incidents): small edit, make it more
        clear that since= is used for catch_up=True

        * NEWS: update for the upcoming release

        * foolscap/referenceable.py (RemoteReferenceOnly.getRemoteTubID):
        make this secure, by using the broker's .remote_tubref field,
        instead of the remote-side-controlled sturdyref.
        (RemoteReferenceOnly.getSturdyRef): add a note about the
        insecurity of this method
        * foolscap/test/test_pb.py (TestCallable.testGetSturdyRef): add a
        test for getRemoteTubID

        * doc/logging.xhtml: change filenames in the incident-gatherer to
        have fewer files starting with incident*, so tab-completion works
        better.
        * foolscap/logging/gatherer.py (IncidentGathererService): same
        (create_incident_gatherer): same

        * foolscap/test/test_logging.py (Filter): test 'flogtool filter'
        * foolscap/logging/filter.py: control stdout/stderr better

        * foolscap/test/test_logging.py (Dumper): test 'flogtool dump'

        * foolscap/logging/cli.py (dispatch): pass stdout/stderr through
        attributes on the subcommand's Options instance, rather than as
        separate function arguments.. makes testing easier.
        * foolscap/logging/gatherer.py (create_log_gatherer): same
        (create_incident_gatherer): same
        (CreateGatherOptions): same
        (CreateIncidentGatherOptions): same
        * foolscap/logging/dumper.py (DumpOptions): same

2008-08-01  Brian Warner  <warner@lothar.com>

        * foolscap/logging/log.py (LogFileObserver): make these more
        useful, by not doing the addSystemEventTrigger in __init__
        (LogFileObserver.stop_on_shutdown): do it here instead
        (FLOGFILE): when using $FLOGFILE, add call to stop_on_shutdown

        * foolscap/logging/gatherer.py (GathererService.do_rotate): oops,
        implement the precautions claimed by the comment in startService:
        test self._savefile before doing anything.
        (GathererService.__init__): set self._savefile to None
        * foolscap/test/test_logging.py (Gatherer.test_log_gatherer2): add
        a timed rotator, which caught the problem in do_rotate

        * foolscap/test/test_logging.py (IncidentGatherer.test_emit): add
        test coverage for re-classifying existing incidents
        (IncidentGatherer.test_emit._update_classifiers_again): and verify
        that leaving the classified/ directory in place properly inhibits
        reclassification at startup

        * foolscap/referenceable.py (RemoteReferenceOnly.getRemoteTubID):
        new method to extract the remote tubid from a RemoteReference. I'm
        not sure this is a good idea, but it fixes the immediate problem
        I'm dealing with.
        * foolscap/logging/gatherer.py: use "tubid_s" instead of "nodeid_s"
        (IncidentGathererService.remote_logport): use getRemoteTubID()
        instead of trying to sanitize the tubid we receive, since we
        use it as a directory name
        (IncidentGathererService.add_classifier): rename addClassifier to
        add_classifier, I'm more fond of the latter form these days. Also
        remove a pyflakes warning.

        * foolscap/test/test_logging.py (IncidentGatherer.test_emit): add
        test of incident generation, publish, recording, and default
        classification

        * foolscap/logging/gatherer.py (IncidentGathererService): get
        control over stdout, so we can exercise more code during tests
        (IncidentGathererService.startService): oops, this needs to be
        startService instead of start

        * foolscap/test/test_logging.py (IncidentGatherer): basic test of
        an incident gatherer, just setup and connection so far

        * foolscap/test/common.py (StallMixin): factor stall() out into a
        separate class
        (TargetMixin): same

        * foolscap/test/test_logging.py (LogfileReaderMixin): refactor
        (Gatherer.test_log_gatherer): turn on bzip=True, for more coverage

        * foolscap/test/test_logging.py (Gatherer._check_gatherer): ignore
        internal foolscap messages (like connection negotiation), since
        they occur at unpredictable times (specifically in
        test_log_gatherer_furlfile_multiple, which has establishes
        multiple connections)

        * foolscap/test/common.py (GoodEnoughTub): factor this and
        crypto_available out of all the other unit tests, make
        GoodEnoughTub(certFile=) work even if crypto is unavailable and
        we must therefore discard the certFile= argument.
        * foolscap/test/test_crypto.py: same
        * foolscap/test/test_gifts.py: same
        * foolscap/test/test_keepalive.py: same
        * foolscap/test/test_logging.py: same
        * foolscap/test/test_loopback.py: same
        * foolscap/test/test_negotiate.py: same
        * foolscap/test/test_pb.py: same
        * foolscap/test/test_serialize.py: same
        * foolscap/test/test_tub.py: same

        * foolscap/logging/gatherer.py (GatheringBase): rewrite Gatherers
        to make them easier to test: now they are a Service (with a
        subordinate Tub), meant to be run by a .tac file or manually. The
        intermediate class has been removed. All .tac files are unchanged:
        gatherers created by old versions of 'flogtool create-gatherer'
        will continue to work.
        (GathererService.do_rotate): return the name of the logfile that
        was just closed and/or compressed, so tests can know where to
        look.
        * foolscap/test/test_logging.py (Gatherer): update to match
        * foolscap/util.py (get_local_ip_for): update reactor comment

2008-07-30  Brian Warner  <warner@lothar.com>

        * setup.py: update comment about bug #62 (pyopenssl problems)

        * foolscap/test/test_logging.py (CLI.test_create_gatherer):
        improve test coverage a little bit, by recording stdout
        (CLI.test_create_incident_gatherer): exercise the 'flogtool
        create-incident-gatherer' command
        * foolscap/logging/cli.py (run_flogtool): capture stdout+stderr
        (dispatch): same
        * foolscap/logging/gatherer.py (create_log_gatherer): same
        (create_incident_gatherer): same

        * trees/tahoe/Makefile (figleaf-output): exclude foolscap/test/ from
        the HTML results

        * doc/logging.xhtml (Running an Incident Gatherer): describe the
        Incident Gatherer, like the Log Gatherer but it only gathers
        incidents. It also does classification, and will eventually do
        reporting. No unit tests yet, but some manual system-level tests
        have been run.
        * foolscap/logging/gatherer.py (IncidentGatherer): implement it
        * foolscap/logging/cli.py (Options.subCommands): add the CLI
        command, named 'flogtool create-incident-gatherer'
        * foolscap/logging/incident.py
        (IncidentReporter.incident_declared): clean up incident naming
        * foolscap/logging/interfaces.py (RILogPublisher.list_incidents): same
        * foolscap/logging/log.py (FoolscapLogger.incident_recorded): same
        * foolscap/logging/publish.py (LogPublisher.list_incident_names): same
        * foolscap/test/test_logging.py
        (IncidentPublisher.test_list_incident_names): test the
        incident-naming cleanup

2008-07-29  Brian Warner  <warner@lothar.com>

        * foolscap/logging/gatherer.py (GatheringBase): refactor the log
        gatherer to share code with the upcoming incident gatherer
        (LogGatherer.__init__): add a basedir= argument, rather than
        using os.getcwd
        * foolscap/test/test_logging.py (Gatherer): add basedir= argument

        * doc/logging.xhtml (Setting up the logport): New feature (well,
        it didn't work before, and now it does, so make it explicit): the
        log-gatherer FURL can be configured (but will not be connected)
        until after setLocation. This should resolve a crash I've seen in
        Tahoe (which runs a slow /sbin/ifconfig command to figure out the
        addresses to pass to setLocation) in which the app connects to the
        log gatherer before it figures out its own location, and then gets
        an exception during registerReference. Closes #55.
        * foolscap/pb.py (Tub._maybeConnectToGatherer): don't initiate the
        gatherer connection until locationHints is set
        (Tub.setLocation): call _maybeConnectToGatherer after the location
        is set. Also, don't let setLocation be called multiple times.
        * foolscap/test/test_logging.py
        (Publish.test_logport_furlfile2): test it
        (Gatherer.test_log_gatherer2): same
        (Gatherer.test_log_gatherer_furlfile2): same

        * doc/logging.xhtml (Setting up the logport): slight API
        restriction: the logport and its FURL are not available until
        after Tub.setLocation is called. This results in a better error
        message than the usual one inside registerReference.
        * foolscap/pb.py (Tub.getLogPort): enforce the API restriction by
        throwing an exception when it is violated
        (Tub.getLogPortFURL): same
        * foolscap/tokens.py (NoLocationError): new exception for it
        * foolscap/test/test_tub.py (SetLocation.test_set_location): test it

        * doc/logging.xhtml (Configuring a Log Gatherer): allow multiple
        log-gatherer furls in the log-gatherer-furlfile
        * foolscap/pb.py (Tub._maybeConnectToGatherer): same
        * foolscap/logging/gatherer.py (LogGatherer.remote_logport):
        return the subscribe_to_all Deferred, for testing
        * foolscap/test/test_logging.py (Gatherer.test_log_gatherer):
        refactor, to accomodate new test
        (Gatherer.test_log_gatherer_furlfile_multiple): test it

2008-07-28  Brian Warner  <warner@lothar.com>

        * foolscap/logging/interfaces.py (RILogPublisher.list_incidents):
        change signature to remove tubid/incarnation from the response.
        (RILogPublisher.subscribe_to_incidents): add pubsub interface for
        incidents, including catch_up= and since=
        (RILogObserver.new_incident): same
        (RILogObserver.done_with_incident_catchup): same
        * foolscap/logging/publish.py (IncidentSubscription): same
        * foolscap/logging/incident.py
        (IncidentReporter.finished_recording): tell the logger about the
        incident name, so it can publish it.
        * foolscap/logging/log.py
        (FoolscapLogger.addImmediateIncidentObserver): same
        (FoolscapLogger.incident_recorded): same
        * foolscap/test/test_logging.py (IncidentPublisher._check_listed):
        same
        (IncidentPublisher.test_subscribe): test it

        * foolscap/logging/interfaces.py (RISubscription.unsubscribe):
        move the RILogPublisher.unsubscribe() method to the RISubscription
        object, since that's a better place for it.
        (RILogPublisher.unsubscribe): deprecate this one

        * foolscap/logging/publish.py (Subscription.remote_unsubscribe): same

        * foolscap/logging/publish.py (LogPublisher.remote_get_incident):
        reject invalid incident names

        * doc/logging.xhtml: add details about running a Log Gatherer
        (Python 'logging' module): comment out this section, it is wrong

2008-07-09  Brian Warner  <warner@lothar.com>

        * doc/specifications/logfiles.xhtml: fix typos
        * doc/logging.xhtml (Remote log aggregation):  same

2008-07-07  Brian Warner  <warner@lothar.com>

        * foolscap/logging/dumper.py (LogDumper): when dumping an Incident
        Report, mark the triggering event with "[INCIDENT-TRIGGER]"

        * foolscap/_version.py (verstr): bump revision to 0.2.9+ while
        between releases
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

2008-07-02  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): release Foolscap-0.2.9
        * misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
        same

        * NEWS: update for the upcoming release
        * MANIFEST.in: add gutsy/hardy directories

2008-07-02  Brian Warner  <warner@allmydata.com>

        * foolscap/test/test_logging.py
        (Publish.test_logpublisher_overload): change poller condition to
        avoid spurious failures on slow systems

        * misc/*/debian/watch: update to point at foolscap.lothar.com

        * Makefile (debian-gutsy, debian-hardy): add .deb targets for
        gutsy and hardy. The rules are the same as for feisty. Closes #76.
        * misc/gutsy/*, misc/hardy/*: same

        * foolscap/logging/publish.py (Subscription.send): change the
        discard policy to discard-new instead of discard-random: in tests
        with a busy Tahoe node, this seems to give good enough
        behavior (probably since the busyness is bursty), although I can't
        say we fully understand what's really going on. discard-new lets
        us use a faster and simpler deque instead of requiring random
        access to the list.
        * foolscap/test/test_logging.py
        (Publish.test_logpublisher_overload): update to match

2008-07-02  Brian Warner  <warner@lothar.com>

        * foolscap/logging/publish.py (Subscription.send): add a
        size-limited queue for sending messages to subscribers, with a
        default queue size of 2000 entries. This should probably keep the
        memory footprint bounded to perhaps 1MB. When the queue gets full,
        we randomly discard old messages, so recent messages are more
        likely to survive than earlier ones. We allow 10 messages to be
        outstanding on the wire at once, to pipeline them a bit and
        improve network utilization. Any errors during sending will cause
        the subscription to be dropped. This should close #72.
        * foolscap/logging/log.py (FoolscapLogger.addImmediateObserver):
        add a new kind of observer, so that the logport publisher can also
        avoid unboundedness in the eventual-send queue. If we can't throw
        away messages fast enough, callers to log.msg will block, slowing
        down the inlet rate.
        * foolscap/test/test_logging.py
        (Publish.test_logpublisher_overload): test it, by throwing 10k
        messages at log.msg and counting how many make it through.

        * foolscap/test/common.py (PollMixin): add a comment: this
        chained-Deferred pattern will run up against python's recursion
        limit if the check function is called more than about 300 times.

2008-07-01  Brian Warner  <warner@lothar.com>

        * foolscap/logging/publish.py (Subscription): refactor a bit, in
        preparation for #72 limit log-publishing queue size

2008-07-01  Brian Warner  <warner@allmydata.com>

        * doc/logging.xhtml ("That Was Weird" Buttons): provide a simple
        example of giving the user a way to trigger Incident logging.
        Closes #75.

        * foolscap/logging/interfaces.py (RILogPublisher.get_pid): new
        interface, to retrieve the process ID through the logport
        * foolscap/test/test_logging.py
        (Publish.test_logpublisher_catchup._got_logport._check_pid): test it
        * foolscap/logging/publish.py (LogPublisher.remote_get_pid):
        implement it
        * foolscap/logging/tail.py (LogTail._got_logpublisher._announce):
        Use it, but tolerate old logports that don't offer it. Closes #71.

        * foolscap/slicers/decimal_slicer.py: handle Decimal objects,
        serializing them as a string. No constraints yet. Closes #50.
        * foolscap/slicers/allslicers.py: import decimal_slicer
        * foolscap/test/test_banana.py (ThereAndBackAgain.test_decimal):
        test it

        * foolscap/logging/tail.py (LogTail._got_logpublisher): exit if we
        get an error while connecting or subscribing. Closes #63.
        (LogTail._print_versions): print remote versions immediately after
        connecting to the logport. Closes #70.

        * foolscap/logging/log.py (FoolscapLogger.setLogDir): oops, allow
        the incident directory to be re-used: os.makedirs throws an
        exception if the directory already exists.
        * foolscap/test/test_logging.py (Incidents.test_basic): test it


        * foolscap/_version.py (verstr): bump revision to 0.2.8+ while
        between releases
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

2008-06-04  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): release Foolscap-0.2.8
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

        * NEWS: update for the upcoming release

2008-06-04  Brian Warner  <warner@allmydata.com>

        * foolscap/pb.py (Tub._tubsAreNotRestartable): fix args so that
        these methods actually get run (and produce a useful error
        message, instead of TypeError). Thanks to Brian Granger for the
        patch. Closes #65.
        * foolscap/test/test_tub.py (Shutdown.test_doublestop): test it
        * foolscap/test/common.py (ShouldFailMixin.shouldFail): add some
        support code

        * foolscap/referenceable.py (RemoteReferenceOnly.getPeer): new
        method to get the IP address and port number of the other end of a
        connection. This returns a twisted.internet.interfaces.IAddress
        provider. Loopback connections give
        foolscap.broker.LoopbackAddress instances. Real remote connections
        give twisted.internet.address.IPv4Address instances, so you can use
        rref.getPeer().host and rref.getPeer().port on them. Closes #45.

        * setup.py: add an "extras_require" clause to the
        setuptools-specific setup args, to declare that our
        "secure_connections" feature requires pyOpenSSL. This helps other
        packages, which can declare a dependency on
        "Foolscap[secure_connections]", rather than claiming to require
        pyOpenSSL themselves. Addresses #66.

        * foolscap/pb.py (Tub.registerReference): fix an exception that
        occurs if you call this with both name= and furlFile= and the
        furlFile already exists. Also prohibit attempts to change the name
        to something other than what is in the furlFile. Thanks to Brian
        Granger for the patch. Closes #64.

        * foolscap/_version.py (verstr): bump revision to 0.2.7+ while
        between releases
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

2008-05-13  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): release Foolscap-0.2.7
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

        * NEWS: update for new "OMG flogtool is broken" release.

2008-05-12  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/cli.py (run_flogtool): fix use of sys.argv,
        the previous version was completely broken

        * foolscap/_version.py (verstr): bump revision to 0.2.6+ while
        between releases
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

2008-05-06  Brian Warner  <warner@lothar.com>

        * foolscap/_version.py (verstr): release Foolscap-0.2.6
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

        * NEWS: update for the upcoming release

2008-05-05  Brian Warner  <warner@allmydata.com>

        * foolscap/test/test_logging.py (Web): add tests for 'flogtool
        web-viewer'
        * foolscap/logging/web.py (WebViewer.start): make this class more
        amenable to being tested

        * foolscap/logging/interfaces.py: misc cleanups
        (RILogPublisher.list_incidents): provide an interface to retrieve
        stored incident reports from a logport.
        (RILogPublisher.get_incident): same
        * foolscap/logging/publish.py (LogPublisher): same
        * foolscap/test/test_logging.py (IncidentPublisher): test it
        * foolscap/pb.py (Tub.setup): make the Tub's logger a bit easier
        to override, for tests

        * foolscap/logging/incident.py (IncidentReporter): make the
        compressed logfile use a temporary name until we've closed it, so
        later observers don't get tricked into using an incomplete
        compressed logfile (they should use the more-complete uncompressed
        logfile instead).
        (IncidentReporter.incident_declared): make it easier to turn off
        the gather-trailing-events behavior
        (NonTrailingIncidentReporter): convenience subclass that does that

        * doc/logging.xhtml: update to match current reality, remove some
        TODO warnings

        * foolscap/logging/log.py (FoolscapLogger): rename
        setIncidentReporterClass to setIncidentReporterFactory
        * foolscap/test/test_logging.py (Incidents.test_customize): same

        * foolscap/logging/interfaces.py (RISubscription): remove spurious
        'pass', figleaf thought it was real code

        * bin/flogtool: split out the CLI dispatcher to..
        * foolscap/logging/cli.py (run_flogtool): here, to help with #51
        (run_flogtool): make it possible to run with a wrapper, closes #51.
        * foolscap/test/test_logging.py (CLI.test_wrapper): test a wrapper,
        provide an example of how to build one
        * foolscap/logging/gatherer.py (CreateGatherOptions.optFlags): add
        a --quiet flag to make the unit test less noisy

        * misc/testutils/trial_figleaf.py: make this compatible with
        twisted-8.0.x: the earlier version didn't write out any coverage
        data on newer twisteds

        * foolscap/logging/log.py (FoolscapLogger.setLogDir): wire up
        Incident handling. We still need incident publishing. This gets
        us most of the way to #61.
        (setIncidentQualifier): this customizes the qualification function
        (setIncidentReporterClass): this customizes the reporter/recorder
        * foolscap/logging/levels.py: move log.WEIRD and friends here to
        avoid circular import problems elsewhere. They are still available
        from log.py .
        * foolscap/logging/incident.py (IncidentReporter.incident_declared):
        flush the uncompressed logfile just before we switch into
        gather-trailing-events mode
        (IncidentReporter.stop_recording): oops, fix typo
        * foolscap/test/test_logging.py (Incidents): test most
        incident-handling functionality


2008-05-02  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/incident.py: start to implement Incident
        handling, for ticket #61. Not complete yet, might be completely
        broken, needs tests and to be wired up.
        * foolscap/logging/log.py (FoolscapLogger.setIncidentQualifier):
        start to add the new interfaces, not complete yetxo
        * foolscap/logging/interfaces.py (IIncidentReporter): same
        * doc/logging.xhtml (Incidents): document the new features

2008-05-01  Brian Warner  <warner@allmydata.com>

        * doc/logging.xhtml: fix discussion of format=, since the
        instructions and examples were simply wrong

        * doc/specifications/logfiles.xhtml (Logfile Headers): define
        "headers", a separate dictionary at the start of the logfile that
        contains metadata. The specific use for this will be the
        "Triggering Event" that gets put into incident reports, once we
        implement those. This induces a backwards compatibility break:
        logfiles produced after this change will probably not be tolerated
        by tools like 'flogtool dump' from before this point. The other
        direction is ok: newer tools can handle either format.
        * foolscap/logging/gatherer.py (LogGatherer._open_savefile): emit
        a header, with type="gatherer", and a starting timestamp in
        "start". Refactor a bit to make this easier.
        * foolscap/logging/tail.py (LogPrinter): emit a header, with
        type="tail"
        * foolscap/logging/log.py (LogFileObserver): emit a header, with
        type="log-file-observer", and a "threshold" key
        * foolscap/logging/dumper.py (LogDumper.start): tolerate headers
        * foolscap/logging/filter.py (Filter.run): same
        * foolscap/logging/web.py (WebViewer.process_logfiles): same
        * foolscap/test/test_logging.py: update tests to match

        * doc/specifications/logfiles.xhtml: document the current saved
        logfile format (event dictionaries, pickled wrapper dicts)

        * foolscap/referenceable.py (SturdyRef): accept multiple
        connection hints in unauthenticated FURLs. This fixes a test
        failure induced by the #60 changes when pyOpenSSL is unavailable.
        * doc/using-foolscap.xhtml: document the new feature.

        * foolscap/referenceable.py (SturdyRef): tolerate extensions in
        tubid and location-hints fields, change FURL parsing, store
        structured hints in the SturdyRef instead of just strings. This
        should give us some wiggle room in the future to gracefully
        transition applications to using new features while retaining
        backwards compatibilty. Many thanks to Zooko for the suggestion.
        Closes #60.
        * foolscap/negotiate.py (TubConnector): same
        * foolscap/test/test_sturdyref.py (URL.testTubIDExtensions): new
        tests for it
        (URL.testLocationHintExtensions): same
        * foolscap/test/test_tub.py (SetLocation.test_set_location):
        update to match
        * foolscap/test/test_gifts.py (Bad): same
        * foolscap/test/test_negotiate.py
        (Versus.testVersusHTTPServerAuthenticated): same
        * foolscap/base32.py (is_base32): new utility function
        * foolscap/test/test_util.py (Base32.test_is_base32): test it

2008-04-22  Brian Warner  <warner@lothar.com>

        * doc/listings/command-client.py: don't emit extra newlines, use
        /usr/bin/env on shbang line
        * doc/listings/command-server.py: use /usr/bin/env on shbang line

2008-04-22  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/filter.py: add "--above UNUSUAL" option (to
        discard events below the given level", and "--from [TUBID]" (to
        discard events that weren't recorded by the given tubid).

        * foolscap/logging/dumper.py (LogDumper.print_event): use
        --rx-time to show both event-generation time and event-receive
        time in the logs, useful if you suspect the application is getting
        bogged down and events are being delivered slowly.
        (LogDumper.print_event): Use a different format to display failure
        tracebacks.

        * foolscap/logging/web.py (WebViewer.get_events): catch
        ValueError, mention it as a possible truncated pickle file

        * foolscap/logging/log.py (FoolscapLogger.msg): if something goes
        wrong, print both str() and repr() in case it helps figure out the
        problem

        * foolscap/logging/dumper.py (LogDumper.print_event): include
        failure tracebacks in 'flogtool dump' output

2008-04-09  Brian Warner  <warner@allmydata.com>

        * foolscap/constraint.py (Constraint.checkOpentype): always
        accept ('reference',) sequences. The per-token constraint checking
        system is a defense against resource-exhaustion attacks, and
        shared reference don't consume any more memory or stack frames
        than any other object. The check-all-args that CallUnslicer does
        just before delivering the arguments is responsible for making
        sure the final (resolved) types all match the constraint. By
        allowing shared references here, we fix a bug in which a schema
        violation was raised when python combined two equivalent tuples
        into a single object.
        * foolscap/test/test_call.py (TestCall.testMega3): test it
        * foolscap/test/common.py (MegaSchema3): same

        * setup.py: finally remove zip_safe=False

        * foolscap/test/test_interfaces.py (TestInterface.testStack):
        allow the test to pass even if the source code isn't
        available (i.e. it's locked away inside an egg). This ought to
        remove the need for zip_safe=False.

        * foolscap/test/test_logging.py (Tail.test_logprinter): make the
        test pass on python2.5: I think twisted now emits type(exception)
        instead of str(exception), or something: the printed form changed.

        * foolscap/schema.py (PolyConstraint): improve error messages

        * foolscap/test/test_logging.py (Tail.test_logprinter): improve
        error messages, to figure out why this fails on python2.5

        * foolscap/__init__.py (_unused): add __version__, to hush pyflakes

        * foolscap/test/test_logging.py (Publish.test_logpublisher): make
        the test work with twisted-8.0.x, which puts more information in
        log.err Failures than earlier versions did.

        * foolscap/__init__.py (__version__): move version from here..
        * foolscap/_version.py (verstr): .. to here
        * foolscap/setup.py: merge zooko's patch to get foolscap version
        by scanning foolscap/_version.py instead of importing foolscap.
        This makes setuptools a lot happier: it doesn't need to have
        Twisted installed (so foolscap can be imported) while it's 
        building foolscap as part of automatic dependency satisfaction.
        Also change the shebang line to use /usr/bin/env, and import (but
        do not use) setuptools if it is available. We set 'zip_safe=False'
        in the extra setuptools arguments because two of the unit tests
        assert that their stack traces have source code lines in them,
        and that doesn't happen if foolscap is living inside an egg.
        We will probably change those tests soon and allow zipped eggs.

2008-03-31  Brian Warner  <warner@allmydata.com>

        * foolscap/test/test_logging.py (Tail): basic tests for
        logging.tail
        * foolscap/logging/tail.py (LogPrinter.remote_msg): parameterize
        the output filehandle so we can test it

        * foolscap/logging/gatherer.py (LogGatherer): remove the "Z"
        suffix from the from- and to- timestamps that we put into
        filenames, since we're actually using localtime. If and when we
        switch to use UTC, we'll bring the Z back. Also remove the hyphens
        between the time portions: from-2008-03-31-161721 instead of
        from-2008-03-31-16-17-21. I'm still looking for a format that
        feels readable, scannable, and clear.

        * foolscap/logging/tail.py (LogPrinter.formatted_print): remove
        spurious "0" from timestamps: i.e. print "08:09:12.345" instead of
        "08:09:120.345"
        * foolscap/logging/web.py (LogEvent.to_html): same

        * foolscap/__init__.py: bump revision to 0.2.5+ while between releases
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

2008-03-25  Brian Warner  <warner@lothar.com>

        * foolscap/__init__.py: release Foolscap-0.2.5
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same

2008-03-25  Brian Warner  <warner@lothar.com>

        * MANIFEST.in: add LICENSE

        * NEWS: update for the upcoming release

        * bin/flogtool (Options.synopsis): update with all commands.
        There's still something broken with --help output, though.

        * doc/listings/command-server.py:
        * doc/listings/command-client.py: new sample programs, a 
        client/server pair which lets the client trigger a specific
        command to be run on the server. Like xfer-server.py, but for
        running commands instead of transferring files.

        * foolscap/test/common.py (PollMixin): refactor a bit, to extract
        the poll() method for use by other tests
        * foolscap/test/test_logging.py (Publish.test_logpublisher):
        improve the timing a bit, by waiting until the observer has heard
        only silence for a full second, instead of starting the verify
        pass one second after subscribing.
        (Publish.test_logpublisher_catchup): same. It makes a bigger
        difference here, because catch_up=True means that we'll be seeing
        several hundred messages, which may take a non-trivial amount of
        time to receive. I was seeing intermittent test failures with the
        one-second-from-subscribe stall.

2008-03-24  Brian Warner  <warner@lothar.com>

        * foolscap/logging/interfaces.py (RILogPublisher.subscribe_to_all):
        add a catch_up= argument, which causes the publisher to dump all
        its stored messages just after adding the subscriber. Closes #49.
        * foolscap/logging/log.py (FoolscapLogger.get_buffered_events): same
        (LogFileObserver.msg): catch+print exceptions while pickling events
        * foolscap/logging/publish.py (LogPublisher): add catch_up=, make
        sure that any log events arrive after subscribe_to_all() has
        returned, to make event sequencing easier on the subscriber. Also
        make sure that any catch-up events arrive before subsequent log
        events.
        * foolscap/logging/tail.py: add --catch-up option, make sure we
        remain compatible with <=0.2.4 publishers (as long as you don't
        use --catch-up)
        * foolscap/test/test_logging.py (Publish.test_logpublisher_catchup):
        test it
        (Publish.test_logpublisher_catchup): make test more reliable

        * LICENSE: make it clear that Foolscap ships under the MIT license,
        the same as Twisted uses. Closes #47.
        * README: same
        * misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/copyright: same

2008-03-24  Brian Warner  <warner@allmydata.com>

        * foolscap/logging/dumper.py (LogDumper.print_event): event dicts
        store the printable tubid, not a binary form
        * foolscap/logging/gatherer.py (Observer.__init__): same
        (LogGatherer.remote_logport): same
        * foolscap/logging/interfaces.py (TubID): same
        * foolscap/logging/web.py (LogEvent.__init__): same
        * foolscap/logging/gatherer.py (LogGatherer.__init__): make bzip=
        argument optional, for unit tests

        * foolscap/logging/tail.py: fix --save-to, also put exception
        information into the twistd.log when we can't pickle the event

        * foolscap/logging/gatherer.py: add --rotate and --bzip options to
        'flogtool create-gatherer': rotate the logfile every N seconds,
        and optionally compress the results. Each logfile gets named like
        from-2008-03-24-20-31-46Z--to-2008-03-24-20-31-56Z.flog.bz2 , and
        the open one is named from-2008-03-24-20-32-16Z--to-present.flog .
        Sending SIGHUP to the gatherer will force a rotation. Closes #48.

        * foolscap/logging/tail.py: add --save-to option to 'flogtool
        tail', which saves the log events to a file (in addition to
        printing them to stdout). 'flogtool dump' can be used on the saved
        file later. Also refactor things a bit to let us grab the
        tubid from the target furl, since this gets recorded in the
        save file format.
        * foolscap/logging/gatherer.py (LogSaver): move to tail.py

        * foolscap/logging/tail.py (LogPrinter): print formatted event
        lines by default. Use the new --verbose option to dump raw event
        dictionaries. Closes #43.
        * bin/flogtool (dispatch): same

2008-02-17  Brian Warner  <warner@lothar.com>

        * doc/using-foolscap.xhtml: fix href, thanks to Stephen Waterbury
        for the catch. Note that the .xhtml points to .xhtml, and lore
        converts the target of the link to .html in the .html output.
        * doc/copyable.xhtml: same

        * foolscap/logging/gatherer.py: make 'flogtool create-gatherer' to
        build a .tac file (which can be launched with twistd) rather than
        starting a gatherer right away. This is much more useable in
        practice.
        * bin/flogtool: same

2008-01-31  Brian Warner  <warner@allmydata.com>

        * foolscap/test/test_tub.py: hush pyflakes, make it work without
        crypto too

        * doc/listings/xfer-server.py:
        * doc/listings/xfer-client.py: new sample programs, a
        client/server pair which allow the client to put files in the
        server's directory. Useful as a replacement for restricted-command
        passphraseless ssh key arrangements.

        * foolscap/pb.py (Tub.setLocationAutomatically): new method that
        guesses an externally-visible IP address and uses it to call
        setLocation().
        * foolscap/test/test_tub.py (SetLocation.test_set_location): test