Incident filename timestamps: TZ, UTC, ISO-8601 please

[Zooko]

I'm looking at some "incident" files, with names like:

incident-2008-08-22-162028-hd4vfua.flog.bz2  incident-2008-12-14-014634-ag6hmpq.flog.bz2  incident-2009-01-03-172732-c27essy.flog.bz2  incident-2009-01-05-111841-2byuega.flog.bz2  incident-2009-01-12-225526-u7ff6pq.flog.bz2

I can't tell when they were recorded because there isn't a time zone indicator in the timestamp.

[Brian Warner]

Merging in nearly-identical tickets #112 and #113.

Zooko wants one of the following:

• incident-2008-08-22_16:20:28-07-hd4vfua.flog.bz2
• incident-2008-08-22_16:20:28Z-hd4vfua.flog.bz2
• incident-2008-08-22_16:20:28+00-hd4vfua.flog.bz2

Which have the following properties:

• unambiguous (either it's clearly in UTC or has a clear timezone indicator)
• formatted like ISO-8601

I don't object to removing the ambiguity.

I must say I don't like the notion of having *five* separate punctuation/delimiter characters in a single formatted string (-_:+.). And I don't like having colons in filenames (shells want to escape them, especially when I use bash autocomplete, and serving them through webservers want to escape them in URLs). I'm not yet convinced of the value of conforming to ISO-8601 here, especially because these are embedded in filenames anyways, and someone who desperately wanted to mechanically parse the filenames would have to regexp off the incident- and the trailing -RANDOM.flog.bz2 anyways.

And I kinda think that someone who is writing code to deal with these things should be using or extending the flogtool code to do it, in which case they should be reading the *contents* of the file and extracting the interesting information (like the seconds-since-epoch and details of the triggering event), rather than trying to extract a smaller set of information from the filename.

In my mind, the filename is strictly useful to a human who is trying to decide which file is worth opening with flogtool dump or to send to somebody else. That plus being a unique filename so two Incidents in the same second (possibly gathered from different machines) don't clobber each other.

Zooko: what about something like these?:

• incident-2008-08-22--16-20-28Z-hd4vfua.flog.bz2
• incident-2008-08-22--16-20-28-0700-hd4vfua.flog.bz2

Zooko and I always go back and forth about UTC versus localtime. #100 was an effective compromise, because it's easy for both of us to set the command-line arguments to our favorite style. But incident filenames aren't as easy to compromise on.. it has to be one way or the other (a flag to foolscap.logging.log.setLogDir() to choose which style will be used certainly doesn't feel right).

[Zooko]

I'm pretty sure that I would prefer having the filenames actually contain an RFC-3339'ish/ISO-8601'ish substring in them, like:

• incident-2008-08-22_16:20:28Z-hd4vfua.flog.bz2
• incident-2008-08-22_16:20:28-0700-hd4vfua.flog.bz2

I don't seem to have the problems you have due to shells or web servers escaping the : chars, and I strongly value consistency with regard to these things. Not just for writing automated parsers (although that too, and usually written in bash, run once, and then thrown away, rather than written as an extension to Foolscap written in Python). I also strongly value consistency just for being easier on my eyeballs and on my brain. Having all the timestamps that I looked at from different sources be syntactically and semantically comparable is a relief. Frankly, I'm really frustrated about this recurring problem that I have with inconsistent timestamps in Tahoe-LAFS:

​http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1077# consistent timestamp format and timezone

However, having said that, I can live with your proposal of:

• incident-2008-08-22--16-20-28Z-hd4vfua.flog.bz2
• incident-2008-08-22--16-20-28-0700-hd4vfua.flog.bz2

Since the information is in there and it is semantically comparable. I would still prefer more syntactic consistency as well, but if I'm not going to get that then fine, let's at least get what I can and as soon as possible.

[Brian Warner]

Fixed by [77d2c68f1fcc4fd0a4651327c5f0495695be47dd] . I decided to use incident-2008-08-22--16:20:28Z-hd4vufa.flog.bz2, which is just like ISO8601 but with a separator of "--" instead of "_". I did some mockups and the underscore was too hard for me to read (it seemed to visually "bind" more tightly than the hyphen, so I kept seeing 2008-08-22_16 or 22_16:20:28).

I also updated the gatherer's output files to match, so they now look like: from-2008-08-22--16:20:28Z---to-2008-08-22--22:20:28Z.flog