While trying to debug a remote exception (which turned out to be in remote_decref, being called on a CLID which had already gone away), I was frustrated to see that the trial output was an extremely terse [ERROR] "KeyError: 1". Asking trial for more detail (by passing it --tbformat=verbose provoked an internal trial error, no attribute ".count" while trying to do printTraceback.
There are several overlapping problems:
- Twisted's Failure object has acquired some new attributes in recent versions. In particular, each Failure instance gets a unique .count attribute, assigned in the constructor, and Failure.printTraceback will include it in the rendered traceback if tbformat==verbose. Foolscap was not sending a copy of .count over the wire.
- the receiving process is getting an actual Failure instance, rather than a CopiedFailure instance. In particular, Failure.printTraceback is being called instead of CopiedFailure.printTraceback, so the lack of .count is more troublesome. I don't understand why this could be the case. I think what we do is instantiate a CopiedFailure and then set its __dict__ to the deserialized attribute dictionary. Perhaps that __dict__ set also manages to set __class__ and thus changes the method lookup behavior. But __class__ isn't supposed to be part of the attribute dictionary that is sent over the wire.
- if CopiedFailure could produce a .frames attribute that looked more like what Failure was expecting, this wouldn't be such a problem.