WindowsError: [Error 183] from foolscap.test.test_logging.Filter.test_basic on Windows

[davidsarah]

After fixing #179 by replacing all "w" file open modes in foolscap code with "wb", the following error in the same test (foolscap.test.test_logging.Filter.test_basic) is unmasked:

[ERROR]
Traceback (most recent call last):
  File "C:\cygwin\home\David-Sarah\tahoe\trunk\support\Lib\site-packages\foolscap-0.6.1-py2.7.egg\fo
olscap\test\test_logging.py", line 1887, in _check
    (out,err) = cli.run_flogtool(argv[1:], run_by_human=False)
  File "C:\cygwin\home\David-Sarah\tahoe\trunk\support\Lib\site-packages\foolscap-0.6.1-py2.7.egg\fo
olscap\logging\cli.py", line 103, in run_flogtool
    dispatch(command, so)
  File "C:\cygwin\home\David-Sarah\tahoe\trunk\support\Lib\site-packages\foolscap-0.6.1-py2.7.egg\fo
olscap\logging\cli.py", line 67, in dispatch
    f.run(options)
  File "C:\cygwin\home\David-Sarah\tahoe\trunk\support\Lib\site-packages\foolscap-0.6.1-py2.7.egg\fo
olscap\logging\filter.py", line 103, in run
    os.rename(newfilename, options.newfile)
exceptions.WindowsError: [Error 183] Cannot create a file when that file already exists

foolscap.test.test_logging.Filter.test_basic

This seems to be due to a difference in behaviour between os.rename on Windows and Unix. On Linux, the destination directory entry is clobbered:

Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> open("test", "wb").close()
>>> open("test2", "wb").close()
>>> os.rename("test", "test2")
>>> open("test2", "rb")
<open file 'test2', mode 'rb' at 0x7fe56c033db0>
>>> open("test", "rb")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'test'

but on Windows XP an exception is raised:

Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> open("test", "wb").close()
>>> open("test2", "wb").close()
>>> os.rename("test", "test2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 183] Cannot create a file when that file already exists

(The inconsistency is ​documented and ​apparently intentional.)

[davidsarah]

Changing the two lines at foolscap/logging/filter.py@4f28a4f#L102​ to:

        if options.newfile == options.oldfile:
            # On Windows we need to delete the destination file first
            # (this is harmless on other OSes).
            try:
                os.unlink(options.newfile)
            except OSError:
                pass
            os.rename(newfilename, options.newfile)

fixes the problem.

If the rename was intended to be atomic on Unix, this also works:

        if options.newfile == options.oldfile:
            if sys.platform == "win32":
                # Win32 can't do an atomic rename to an existing file.
                try:
                    os.unlink(options.newfile)
                except OSError:
                    pass
            os.rename(newfilename, options.newfile)

[edit: indentation fixed]

[Brian Warner]

Applied the second patch (unlink on win32) in [918e123]. Thanks!

[Brian Warner]

I just went through and replaced some new os.rename calls to fix the same problem, in [161f9b5309].