itertools.count() overflows

[Zooko]

In defiance of the Python documentation, itertools.count() in Python 2.5.1 raises an overflow error if you call next() on it more than PY_SSIZE_T_MAX times (see attached stack trace). Here is a patch which does the same thing but does the normal Python behavior of switching to longs as needed:

2008-10-13 15:38:38.816Z [-] Exception rendering error page:
2008-10-13 15:38:38.817Z [-] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.5/site-packages/foolscap/eventual.py", line 26, in _turn
            cb(*args, **kwargs)
          File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 239, in callback
            self._startRunCallbacks(result)
          File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 304, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 317, in _runCallbacks
            self.result = callback(self.result, *args, **kw)
        --- <exception caught here> ---
          File "/var/lib/python-support/python2.5/nevow/appserver.py", line 82, in processingFailed
            handler.renderHTTP_exception(ctx, reason)
          File "/usr/lib/python2.5/site-packages/allmydata/web/common.py", line 129, in renderHTTP_exception
            return super.renderHTTP_exception(self, ctx, f)
          File "/var/lib/python-support/python2.5/nevow/appserver.py", line 61, in renderHTTP_exception
            request.finishRequest( False )
          File "/var/lib/python-support/python2.5/nevow/appserver.py", line 171, in finishRequest
            server.Request.finish(self)
          File "/usr/lib/python2.5/site-packages/twisted/web/server.py", line 272, in finish
            http.Request.finish(self)
          File "/usr/lib/python2.5/site-packages/twisted/web/http.py", line 667, in finish
            self.channel.factory.log(self)
          File "/var/lib/python-support/python2.5/nevow/appserver.py", line 312, in log
            request._logger()
          File "/usr/lib/python2.5/site-packages/allmydata/webish.py", line 116, in _logger
            level=log.OPERATIONAL,
          File "/usr/lib/python2.5/site-packages/foolscap/logging/log.py", line 137, in msg
            num = self.seqnum.next()
        exceptions.OverflowError: cannot count beyond PY_SSIZE_T_MAX

[Brian Warner]

Drat, I'd just assumed itertools used bigints internally. I'll apply this soon. Thanks!

[Brian Warner]

Applied, in [2a178d07afd74083adae609443df70da0c17a8e9]. Thanks!

[Zooko]

Note that it wasn't a mistake on Brian's part, but a bug in the Python doc, which explicitly claims that itertools.count() is equivalent to:

def count(n=0):
    # count(10) --> 10 11 12 13 14 ...
    while True:
        yield n
        n += 1

​http://www.python.org/doc/2.4.2/lib/itertools-functions.html ​http://docs.python.org/library/itertools.html#itertools.count

Ah, it looks like this has been fixed in python trunk one year ago, October 2007:

​http://svn.python.org/view/python/trunk/Misc/NEWS?rev=58305&view=diff&r1=58305&r2=58304&p1=python/trunk/Misc/NEWS&p2=/python/trunk/Misc/NEWS

Ah, yes the fix was released in Python 2.5.2:

​http://www.python.org/download/releases/2.5.2/NEWS.txt

[Zooko]

Oh I misread that 2.5.2/NEWS.txt. It doesn't mention this issue at all. As far as I see, this fix has been released only in Python 2.6.