PEP: 495 Title: Local Time Disambiguation Version: $Revision$ Last-Modified: $Date$ Author: Alexander Belopolsky <alexander.belopolsky@gmail.com>, Tim Peters <tim.peters@gmail.com> Discussions-To: Datetime-SIG <datetime-sig@python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 02-Aug-2015
This PEP adds a new attribute fold to instances of the datetime.time and datetime.datetime classes that can be used to differentiate between two moments in time for which local times are the same. The allowed values for the fold attribute will be 0 and 1 with 0 corresponding to the earlier and 1 to the later of the two possible readings of an ambiguous local time.
US public service advertisement
In most world locations, there have been and will be times when local clocks are moved back. [1] In those times, intervals are introduced in which local clocks show the same time twice in the same day. In these situations, the information displayed on a local clock (or stored in a Python datetime instance) is insufficient to identify a particular moment in time. The proposed solution is to add an attribute to the datetime instances taking values of 0 and 1 that will enumerate the two ambiguous times.
People who live in locations observing Daylight Saving Time (DST) move their clocks back (usually one hour) every Fall.
It is less common, but occasionally clocks can be moved back for other reasons. For example, Ukraine skipped the spring-forward transition in March 1990 and instead, moved their clocks back on July 1, 1990, switching from Moscow Time to Eastern European Time. In that case, standard (winter) time was in effect before and after the transition.
Both DST and standard time changes may result in time shifts other than an hour.
When clocks are moved back, we say that a fold [2] is created in time. When the clocks are moved forward, a gap is created. A local time that falls in the fold is called ambiguous. A local time that falls in the gap is called missing.
[2] The term "fall-backward fold" was invented in 1990s by Paul Eggert of UCLA who used it in various Internet discussions related to the C language standard that culminated in a Defect Report #139.We propose adding an attribute called fold to instances of the datetime.time and datetime.datetime classes. This attribute should have the value 0 for all instances except those that represent the second (chronologically) moment in time in an ambiguous case. For those instances, the value will be 1. [3]
fold=1 in a non-ambiguous case is said to represent an invalid time (or is invalid for short), but users are not prevented from creating invalid instances by passing fold=1 to a constructor or to a replace() method. This is similar to the current situation with the instances that fall in the spring-forward gap. Such instances don't represent any valid time, but neither the constructors nor the replace() methods check whether the instances that they produce are valid. Moreover, this PEP specifies how various functions should behave when given an invalid instance.
Instances of datetime.time and datetime.datetime classes will get a new attribute fold with two possible values: 0 and 1.
The __new__ methods of the datetime.time and datetime.datetime classes will get a new keyword-only argument called fold with the default value 0. The value of the fold argument will be used to initialize the value of the fold attribute in the returned instance.
The replace() methods of the datetime.time and datetime.datetime classes will get a new keyword-only argument called fold. It will behave similarly to the other replace() arguments: if the fold argument is specified and given a value 0 or 1, the new instance returned by replace() will have its fold attribute set to that value. In CPython, any non-integer value of fold will raise a TypeError, but other implementations may allow the value None to behave the same as when fold is not given. [4] If the fold argument is not specified, the original value of the fold attribute is copied to the result.
None to mean "no change to existing attribute" for all other attributes in replace().
Access macros will be defined to extract the value of fold from PyDateTime_DateTime and PyDateTime_Time objects.
int PyDateTime_GET_FOLD(PyDateTime_DateTime *o)
Return the value of fold as a C int.
int PyDateTime_TIME_GET_FOLD(PyDateTime_Time *o)
Return the value of fold as a C int.
New constructors will be defined that will take an additional argument to specify the value of fold in the created instance:
PyObject* PyDateTime_FromDateAndTimeAndFold(
int year, int month, int day, int hour, int minute,
int second, int usecond, int fold)
Return a datetime.datetime object with the specified year, month, day, hour, minute, second, microsecond and fold.
PyObject* PyTime_FromTimeAndFold(
int hour, int minute, int second, int usecond, int fold)
Return a datetime.time object with the specified hour, minute, second, microsecond and fold.
The datetime.now() method called with no arguments, will set fold=1 when returning the second of the two ambiguous times in a system local time fold. When called with a tzinfo argument, the value of the fold will be determined by the tzinfo.fromutc() implementation. If an instance of the datetime.timezone class (e.g. datetime.timezone.utc) is passed as tzinfo, the returned datetime instance will always have fold=0.
The astimezone() method will now work for naive self. The system local timezone will be assumed in this case and the fold flag will be used to determine which local timezone is in effect in the ambiguous case.
For example, on a system set to US/Eastern timezone:
>>> dt = datetime(2014, 11, 2, 1, 30)
>>> dt.astimezone().strftime('%D %T %Z%z')
'11/02/14 01:30:00 EDT-0400'
>>> dt.replace(fold=1).astimezone().strftime('%D %T %Z%z')
'11/02/14 01:30:00 EST-0500'
Conversion from POSIX seconds from EPOCH
The fromtimestamp() static method of datetime.datetime will set the fold attribute appropriately in the returned object.
For example, on a system set to US/Eastern timezone:
>>> datetime.fromtimestamp(1414906200) datetime.datetime(2014, 11, 2, 1, 30) >>> datetime.fromtimestamp(1414906200 + 3600) datetime.datetime(2014, 11, 2, 1, 30, fold=1)Conversion to POSIX seconds from EPOCH
The timestamp() method of datetime.datetime will return different values for datetime.datetime instances that differ only by the value of their fold attribute if and only if these instances represent an ambiguous or a missing time.
When a datetime.datetime instance dt represents an ambiguous time, there are two values s0 and s1 such that:
datetime.fromtimestamp(s0) == datetime.fromtimestamp(s1) == dt
In this case, dt.timestamp() will return the smaller of s0 and s1 values if dt.fold == 0 and the larger otherwise.
For example, on a system set to US/Eastern timezone:
>>> datetime(2014, 11, 2, 1, 30, fold=0).timestamp() 1414906200.0 >>> datetime(2014, 11, 2, 1, 30, fold=1).timestamp() 1414909800.0
When a datetime.datetime instance dt represents a missing time, there is no value s for which:
datetime.fromtimestamp(s) == dt
but we can form two "nice to know" values of s that differ by the size of the gap in seconds. One is the value of s that would correspond to dt in a timezone where the UTC offset is always the same as the offset right before the gap and the other is the similar value but in a timezone the UTC offset is always the same as the offset right after the gap.
The value returned by dt.timestamp() given a missing dt will be the greater of the two "nice to know" values if dt.fold == 0 and the smaller otherwise.
For example, on a system set to US/Eastern timezone:
>>> datetime(2015, 3, 8, 2, 30, fold=0).timestamp() 1425799800.0 >>> datetime(2015, 3, 8, 2, 30, fold=1).timestamp() 1425796200.0
Users of pre-PEP implementations of tzinfo will not see any changes in the behavior of their aware datetime instances. Two such instances that differ only by the value of the fold attribute will not be distinguishable by any means other than an explicit access to the fold value.
On the other hand, if an object's tzinfo is set to a fold-aware implementation, then the value of fold will affect the result of several methods, but only if the corresponding time is in a fold or in a gap: utcoffset(), dst(), tzname(), astimezone(), strftime() (if "%Z" or "%z" directive is used in the format specification), isoformat(), and timetuple().
The datetime.datetime.combine() method will copy the value of the fold attribute to the resulting datetime.datetime instance.
The datetime.datetime.time() method will copy the value of the fold attribute to the resulting datetime.time instance.
Pickle sizes for the datetime.datetime and datetime.time objects will not change. The fold value will be encoded in the first bit of the 5th byte of the datetime.datetime pickle payload or the 2nd byte of the datetime.time. In the current implementation these bytes are used to store minute value (0-59) and the first bit is always 0. (This change only affects pickle format. In the C implementation, the fold attribute will get a full byte to store its value.)
No new implementations of datetime.tzinfo abstract class are proposed in this PEP. The existing (fixed offset) timezones do not introduce ambiguous local times and their utcoffset() implementation will return the same constant value as they do now regardless of the value of fold.
The basic implementation of fromutc() in the abstract datetime.tzinfo class will not change. It is currently not used anywhere in the stdlib because the only included tzinfo implementation (the datetime.timezone class implementing fixed offset timezones) override fromutc().
Implementors of concrete datetime.tzinfo subclasses who want to support variable UTC offsets (due to DST and other causes) should follow these guidelines.
New implementations of utcoffset(), tzname() and dst() methods should ignore the value of fold unless they are called on the ambiguous or missing times.
New subclasses should override the base-class fromutc() method and implement it so that in all cases where two UTC times u1 and u2 (u1 <u2) correspond to the same local time fromutc(u1) will return an instance with fold=0 and fromutc(u2) will return an instance with fold=1. In all other cases the returned instance should have fold=0.
On an ambiguous time introduced at the end of DST, the values returned by utcoffset() and dst() methods should be as follows
where stdoff is the standard (non-DST) offset, dstoff is the DST correction (typically dstoff = timedelta(hours=1)) and zero = timedelta(0).
On a missing time introduced at the start of DST, the values returned by utcoffset() and dst() methods should be as follows
On ambiguous/missing times introduced by the change in the standard time offset, the dst() method should return the same value regardless of the value of fold and the utcoffset() should return values according to the following table:
where delta is the size of the fold or the gap.
The value of "fold" will be ignored in all operations with naive instances. As a consequence, datetime.datetime or datetime.time instances that differ only by the value of fold will compare as equal. Applications that need to differentiate between such instances should check the value of fold explicitly.
The result of addition (subtraction) of a timedelta to (from) a datetime will always have fold set to 0 even if the original datetime instance had fold=1.
Since one of the goals in this PEP is to preserve backward compatibility, it is helpful to start by explaining how comparison and arithmetic operations are curretly defined.
Let's start with naive datetime instances. It is helpful to think of those as different configurations of hands on an ordinary mechanical clock. For example, a Python expression time(1, 30) corresponds to a "long hand at 6, short hand between 1 and 2". A datetime object is a fancy clock with separate hands for years, months, and so on through microseconds. A timedelta is the knob on the back of the clock that you can turn to change the configuration of the clock hands. The fancy datetime clock has several knobs that allow to change the displayed time by weeks, days, hours and so on through microseconds, but if we don't care about convinences, we can make any change using a single microsecond knob.
The operations defined on the datetimes and timedeltas have nothing to do with the physical time only with the construction of this imaginary clock. For example, addition of two timedeltas tells you how to get the effect of turning a knob twice by two different angles with a single turn by another angle. Timedelta negation is a way to undo the change caused by a given timedelta and timedelta subtraction is the same as addition of a negative delta. (Clockwise turns are considered positive and counterclockwise turns negative.)
The subtraction operation on two datetime instances tells you how to turn the knob to get from the suntrahend configuration to the minuend.
The addition/subtraction of a timedelta to/from a datetime is simply the result of turning the knob clockwise or counterclockwise starting from on configuration to arrive at another.
An aware datetime is simply a clock with the name of a time zone written on it.
The construction of the clock is still the same: same hands, same knobs, same gears connecting them. The only difference is the time zone label which does not affect the results of various operations in any way.
Now, suppose you have two clocks displaying time in two different time zones.
No matter what you do to the knobs on the left clock, it will not magically transform itself into the clock on the right. Moreover, matching the hands configuration on the two clocks does not make much sense either because the same hands display may correspond to different times in Budapest and Newcastle. Python defines the "difference" between the times shown on such clocks as follows. Each clock comes with a manual (utcoffset()) which says how one should turn the knob to display a "universal" time. Once both clocks are adjusted according to their manuals, "equal" times correspond to congruent configurations. The difference between the two times is how far one should turn the knob on the adjusted right clock to make it display a configuration that is congruent to the adjusted left clock.
Note that interzone difference is more complicated and is defined in terms of intrazone operations. Although conceptually, intrazone and interzone comparisons and "difference" operations are completely different, Python uses the same operators == and - to implement them.
This proposal will have little effect on the programs that do not read the fold flag explicitly or use tzinfo implementations that do. The only visible change for such programs will be that conversions to and from POSIX timestamps will now round-trip correctly (up to floating point rounding). Programs that implemented a work-around to the old incorrect behavior may need to be modified.
Pickles produced by older programs will remain fully forward compatible. Only datetime/time instances with fold=1 pickled in the new versions will become unreadable by the older Python versions. Pickles of instances with fold=0 (which is the default) will remain unchanged.
(same characters, an hour later)
While the tm_isdst field of the time.struct_time object can be used to disambiguate local times in the fold, the semantics of such disambiguation are completely different from the proposal in this PEP.
The main problem with the tm_isdst field is that it is impossible to know what value is appropriate for tm_isdst without knowing the details about the time zone that are only available to the tzinfo implementation. Thus while tm_isdst is useful in the output of methods such as time.localtime, it is cumbersome as an input of methods such as time.mktime.
If the programmer misspecified a non-negative value of tm_isdst to time.mktime, the result will be time that is 1 hour off and since there is rarely a way to know anything about DST before a call to time.mktime is made, the only sane choice is usually tm_isdst=-1.
Unlike tm_isdst, the proposed fold attribute has no effect on the interpretation of the datetime instance unless without that attribute two (or no) interpretations are possible.
Since it would be very confusing to have something called isdst that does not have the same semantics as tm_isdst, we need a different name. Moreover, the datetime.datetime class already has a method called dst() and if we called fold "isdst", we would necessarily have situations when "isdst" is zero but dst() is not or the other way around.
Suggested by Guido van Rossum and favored by one (but initially disfavored by another) author. A consensus was reached after the allowed values for the attribute were changed from False/True to 0/1. The noun "fold" has correct connotations and easy mnemonic rules, but at the same time does not invite unbased assumptions.
This was a working name of the attribute chosen initially because the obvious alternative ("second") conflicts with the existing attribute. It was rejected mostly on the grounds that it would make True a default value.
The following alternative names have also been considered:
- later
- A close contender to "fold". One author dislikes it because it is confusable with equally fitting "latter," but in the age of auto-completion everywhere this is a small consideration. A stronger objection may be that in the case of missing time, we will have
later=Trueinstance converted to an earlier time by.astimezone(timezone.utc)that that withlater=False. Yet again, this can be interpreted as a desirable indication that the original time is invalid.- which
- The original placeholder name for the localtime function branch index was independently proposed for the name of the disambiguation attribute and received some support.
- repeated
- Did not receive any support on the mailing list.
- ltdf
- (Local Time Disambiguation Flag) - short and no-one will attempt to guess what it means without reading the docs. (Feel free to use it in discussions with the meaning ltdf=False is the earlier if you don't want to endorse any of the alternatives above.)
Several reasons have been raised to allow a None or -1 value for the fold attribute: backward compatibility, analogy with tm_isdst and strict checking for invalid times.
It has been suggested that backward compatibility can be improved if the default value of the fold flag was None which would signal that pre-PEP behavior is requested. Based on the analysis below, we believe that the proposed changes with the fold=0 default are sufficiently backward compatible.
This PEP provides only three ways for a program to discover that two otherwise identical datetime instances have different values of fold: (1) an explicit check of the fold attribute; (2) if the instances are naive - conversion to another timezone using the astimezone() method; and (3) conversion to float using the timestamp() method.
Since fold is a new attribute, the first option is not available to the existing programs. Note that option (2) only works for naive datetimes that happen to be in a fold or a gap in the system time zone. In all other cases, the value of fold will be ignored in the conversion unless the instances use a fold-aware tzinfo which would not be available in a pre-PEP program. Similarly, the astimezone() called on a naive instance will not be available in such program because astimezone() does not currently work with naive datetimes.
This leaves us with only one situation where an existing program can start producing diferent results after the implementation of this PEP: when a datetime.timestamp() method is called on a naive datetime instance that happen to be in the fold or the gap. In the current implementation, the result is undefined. Depending on the system mktime implementation, the programs can see different results or errors in those cases. With this PEP in place, the value of timestamp will be well-defined in those cases but will depend on the value of the fold flag. We consider the change in datetime.timestamp() method behavior a bug fix enabled by this PEP. The old behavior can still be emulated by the users who depend on it by writing time.mktime(dt.timetuple()) + 1e-6*dt.microsecond instead of dt.timestamp().
The time.mktime interface allows three values for the tm_isdst flag: -1, 0, and 1. As we explained above, -1 (asking mktime to determine whether DST is in effect for the given time from the rest of the fields) is the only choice that is useful in practice.
With the fold flag, however, datetime.timestamp() will return the same value as mktime with tm_isdst=-1 in 99.98% of the time for most time zones with DST transitions. Moreover, tm_isdst=-1-like behavior is specified regardless of the value of fold.
It is only in the 0.02% cases (2 hours per year) that the datetime.timestamp() and mktime with tm_isdst=-1 may disagree. However, even in this case, most of the mktime implementations will return the fold=0 or the fold=1 value even though relevant standards allow mktime to return -1 and set an error code in those cases.
In other words, tm_isdst=-1 behavior is not missing from this PEP. To the contrary, it is the only behavior provided in two different well-defined flavors. The behavior that is missing is when a given local hour is interpreted as a different local hour because of the misspecified tm_isdst.
For example, in the DST-observing time zones in the Northern hemisphere (where DST is in effect in June) one can get
>>> from time import mktime, localtime >>> t = mktime((2015, 6, 1, 12, 0, 0, -1, -1, 0)) >>> localtime(t)[:] (2015, 6, 1, 13, 0, 0, 0, 152, 1)
Note that 12:00 was interpreted as 13:00 by mktime. With the datetime.timestamp, datetime.fromtimestamp, it is currently guaranteed that
>>> t = datetime.datetime(2015, 6, 1, 12).timestamp() >>> datetime.datetime.fromtimestamp(t) datetime.datetime(2015, 6, 1, 12, 0)
This PEP extends the same guarantee to both values of fold:
>>> t = datetime.datetime(2015, 6, 1, 12, fold=0).timestamp() >>> datetime.datetime.fromtimestamp(t) datetime.datetime(2015, 6, 1, 12, 0)
>>> t = datetime.datetime(2015, 6, 1, 12, fold=1).timestamp() >>> datetime.datetime.fromtimestamp(t) datetime.datetime(2015, 6, 1, 12, 0)
Thus one of the suggested uses for fold=-1 -- to match the legacy behavior -- is not needed. Either choice of fold will match the old behavior except in the few cases where the old behavior was undefined.
Another suggestion was to use fold=-1 or fold=None to indicate that the program truly has no means to deal with the folds and gaps and dt.utcoffset() should raise an error whenever dt represents an ambiguous or missing local time.
The main problem with this proposal, is that dt.utcoffset() is used internally in situations where raising an error is not an option: for example, in dictionary lookups or list/set membership checks. So strict gap/fold checking behavior would need to be controlled by a separate flag, say dt.utcoffset(raise_on_gap=True, raise_on_fold=False). However, this functionality can be easily implemented in user code:
def utcoffset(dt, raise_on_gap=True, raise_on_fold=False):
u = dt.utcoffset()
v = dt.replace(fold=not dt.fold).utcoffset()
if u == v:
return u
if (u < v) == dt.fold:
if raise_on_fold:
raise AmbiguousTimeError
else:
if raise_on_gap:
raise MissingTimeError
return u
Moreover, raising an error in the problem cases is only one of many possible solutions. An interactive program can ask the user for additional input, while a server process may log a warning and take an appropriate default action. We cannot possibly provide functions for all possible user requirements, but this PEP provides the means to implement any desired behavior in a few lines of code.
This document has been placed in the public domain.
This image is a work of a U.S. military or Department of Defense employee, taken or made as part of that person's official duties. As a work of the U.S. federal government, the image is in the public domain.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4