Daylight Savings Time Software Bugs
What is it about daylight savings time? DST not only makes it hard to get up on that first, dark Monday morning (it was for me today, that's for sure), but it appears to be a bug-fest for software systems. I may be reading too much into things, but I observed what I believe to be Daylight Savings bugs in the software systems of two different vendors yesterday/today, and, while I don't have any statistical analysis to go with it, our support phones have been ringing off the hook all day (we provide outsourced help desk services for a variety of companies).
A daylight savings-related bug is like an April Fools joke for computers and support people (even the ones in November)--they are usually fairly harmless, but also kind of annoying. For those who live in places where daylight savings time changes are not an issue, the basic problem is one of time shifting--the clock is manually moved either forward or backward in April and November.
Things can get interesting when you have multiple coordinated, yet ultimately autonomous, computer-based systems for which the time shifts are not synchronized--or when, like the clock on your VCR that never gets changed, the time shift does not happen at all for some parts of the system.
Am I imagining all this? Have you seen or heard about any daylight savings bugs today? (see update below)
Do leaps forward in April cause more problems than leaps backward in November?
When was the last time you were involved with a project in which daylight savings-related requirements or design provisions were discussed in advance?
Update: I was just talking to a good friend of mine who is a senior support tech at a software vendor. I asked him if he had any Daylight Savings bugs today or yesterday! I think he jumped about a foot and exclaimed, "Yes! We had one today. The programmers had to fix some code and compile a new DLL. Something like this happens every six months. I made the comment today that we need to start putting this on the calendar."
This was for me a welcome confirmation that I was not imagining things. I think what happens is that because the bugs are usually not too severe and are simple to figure out, we go back to the normal routine and forget that anything happened--and then six months later we're surprised by a DST bug again.
I don't mean to sound preachy here. No doubt I've written my share of DST bugs into software systems, and there's a good chance I'll do it again. Why? Partly it's because the kind of software systems I typically work on (data-oriented software systems for business/commerce, developed using an accelerated methodology on a tight budget) can tolerate these kinds of bugs--what I mean is the nature of the project does not warrant Nth-degree risk analysis, using FMEA/FMECA and the like. In 12 years, I've only worked on one project where we did that sort of thing, and that was a simulation model and long term maintenance predictor for giant machines (I'm being vague for confidentiality's sake).
This reasoning runs close to the making of excuses for bugs that theoretically could have been prevented (or maybe we should have at least had a "Timing Requirements" section in the design document, in which we address DST and other related items), but I think there is a legitimate concept here that there are classes of bugs that we find acceptable, and therefore don't test for or explicitly protect against. Some of these system failure modes within a given domain will not even be spoken of.
If someone spilled coffee on the machine and your program lost some data would your boss hold it against you? Probably not.
If the operating system your program was running on automatically changed the time for DST (or failed to), and your system lost data, would your boss hold it against you? Maybe he'd be justified in doing so.
This is a continuum that becomes more rigorous in areas such as real time systems, life-critical systems, financially sensitive systems, etc. You can get into redundancy and other fault tolerance techniques and really tighten things down, such that a DST bug is childs play, eliminated in the first round of design and testing. I won't presume to tell you where the lines are in your world.
So we get daylight savings time bugs. Apparently it's no big deal, because we keep building new systems that introduce new daylight savings bugs, and we don't follow up DST fire drills with after-the-fact analysis and preventative adjustments. I know I'm generalizing from my own experience here, but I am doing so in part to test whether others can relate. What do you think?
Here's a few more questions I thought of:
Have you ever seen or heard of a post-DST post-mortem discussion, meeting, or analysis?
Have you ever observed a change in attitude or behavior after a DST bug (or rash of them)?
Have you ever been burned by someone *else's* DST bug? Was there anything you could have done to prevent it?
I'm going to start running DST scenarios against all of my designs. For real this time. Seriously.
Dan
P.S. I've been listening for the past couple weeks to this excellent lecture-series-on-tape about WWI, and the lecturer (Vejas Gabriel Liulevicius of the University of Tennessee) explains that the daylight savings time was first instituted during WWI as a way to increase the productivity of workers, which was seen as essential to the war effort. It was "Total War," after all. Germany was apparently the first to implement it, and the other "great powers" followed suit in order to avoid giving anyone else a competitive edge.
Interestingly, the Wikipedia article linked above references energy efficiency as a primary rationalle for daylight savings time (I did not see a reference to WWI), and I wonder whether this was a later development as a justification for *keeping* daylight savings time after the war. In a similar fashion, the lecturer also says that bar closing times were implemented as a productivity measure in the same context--get the workers home to sleep it off before work the next day.
DST 'bugs'
I'm not sure that this really qualifies as a 'bug' but I'll share it anyway :)
Way back in the olden days, I used to work as a trainman (brakeman) on the CPR (Canadian Pacific Railroad). Some of the trains I crewed had to actually use CNR (Canadian National Railroad) tracks for part of the route. This is not really as uncommon as many might think. Anyway, to the 'bug'.
CNRail was operating in the context of what they called the 'Prairie region', which included a tiny chunk of Ontario, all of Manitoba and most of Saskatchewan. CPRail was operating in the context of what they called the 'Mountain Region', which included British Columbia, Alberta, and most of Saskatchewan. Thus we had 3 times to 'track' (sorry, I couldn't resist): 'Manitoba' time, 'Alberta' time, and 'local' time. Normally, local time matched one or the other of Manitoba or Alberta because of Saskatchewan's unique place in Canada, so we seldom used the words 'local' or 'Saskatchewan' when referring to time.
I'm sure you can see where this is going! In order to be allowed to crew a train that was utilizing track owned and operated by the other company, you had to wear (or carry) not just a standard approved timepiece, but a timepiece with 2 hour hands, once of which was coloured red. Train crews from both RR basically lived on the edge when running in a region where you could expect the other company to be sharing track with you, especially in light of the fact that the 'schedules' were simple works of fiction. In fact, this time issue is one of the reasons that many eligible people chose to not pursue a career with VIA Rail, the Canadian passenger service. As a separate 'railroad' that owns no tracks, the passenger trains just run whichever combination of tracks makes most sense to the route. In practice, there isn't a lot of switching from one RR to the other, but some people still find it a cause for concern.
I consider that to be one of the main formative experiences of my relationship with time, the other being my status as an amateur astronomer. Although I 'get' Standard time and appreciate some of the resulting conveniences, I must confess that I think that DST is one of man's stupider inventions.
DST PIA
Yes, DST is a tricky little fellow, but then so is time zone management in an application that must be used in multiple time zones.
One reason for so many DST "bugs" this time around, is that Congress last year saw fit to move up the start date of DST. Also, much of Indiana went to using DST for the first time this year. So if you have an app that was not patched to use the new DST rules, you'd get some real confusion.
Same problem for me also
I also faced a similar problem. It was a project of handling time variables excessively. In between ,it fell into rough patches. But,finally I was able to recover.
I dont understand what was
I dont understand what was the purpose of this whole thing.. Maybe I'm at a wrong link after searching for a solution for implementing DST rules in SQL server (:. Somehow this was the top rated link in google!
Daylight Savings Changes in 2007 and Beyond
It appears that very soon we will have a new class of DST bug to worry about: according to this article on Sun's Java web site, the rules around the timing of Daylight Savings Time in the U.S. are changing in 2007. Sun's article focuses on the effect this will have on the JVM:
The Java Runtime Environment (JRE) stores rules about DST observance all around the globe. Older JREs will have outdated rules that will be superseded by the Energy Policy Act of 2005. As a result, applications running on an older JRE may report incorrect time from March 11, 2007 through April 2, 2007 and from October 29, 2007 through November 4, 2007.
Apparently, these changes could be rippling beyond the U.S. in coming years:
Some countries are still evaluating whether they will adopt the new rules for themselves. You should anticipate more changes in DST and time zone rules for countries that typically align with U.S. DST rules.
I wonder how the automatic DST adjustment rules built into Windows will handle this...and how many other devices, virtual machines, libraries, etc. have hard-coded DST timing rules...this could get interesting. Please post any good tales and info here!
Dan
P.S.
Thanks to Steve Anglin and The Server Side for the links.
Windows DST Updates
Since a lot of people are coming to this post from the search engines looking for information about the March 11/April 11 DST upheaval, I thought this link might be helpful and interesting; the link points to a ZDNet blog post by Mary Jo Foley about the upcoming Windows changes for the early DST switchover.
Best,
Dan
DST
This year (2007) daylight savings time is changing and this is giving a lot of sysadmins a lot of extra work: cf. http://tf.nist.gov/timefreq/general/dst.htm for the new rules, which apply only to the USA.
Here in China, we don't have DST. Also, all of China is on ONE time, which makes for some weirdness in its western provinces but simplifies communication. China is approximately as wide as the United States with its four time zones.
Hong Kong users can manually or with software synchronize their systems with the clock in use at the HK Observatory, and similar systems are available in the USA.
Question is whether DST saves any energy at all. It's fun to have extra hours of daylight after work especially in the northernmost cities of the USA, where you can laze in the dying sun of Seattle. But that day in Spring when an hour is taken out of your life is no fun (especially if you forget), and by fall, when you get it back, the pain is forgotten.
Vista handles the new rules already, but the thought of upgrading to Vista because it handles new DST is a tail most assuredly wagging a big old dog.
Lazing about in the 'extra' sun
I can almost understand the whole DST thing in places where daylight is at a premium. Frankly, I'd be unhappy if SK ever decided to implement DST on our current timekeeping. I already spend a good part of late spring and early summer going to bed while it's still pretty light out. And I'm not a particularly early-to-bed kind of guy; anywhere in the last 2 hours before midnight is actually pretty typical around here. If anything, it might be nice to do DST in winter instead of summer so that we could actually get a chance to see the sun once in a while.
An Aparent Mess with Microsoft DST Patches
The saga continues with the current Windows DST switchover: Mary Jo Foley of the "All About Microsoft" blog relates the tale of user woes with the DST patches.
Dan
DST Embedded Everywhere
I came across this bulletin today about the DST bug that is going to appear in a few days in a Sirius satellite radio model called the Stiletto. This underlines for me how pervasive this problem is, and makes me wonder whether lawmakers really thought this through before passing this law (I suppose that's a stupid question...of course they didn't!).
How long will it take the theorized energy savings from this move to exceed the losses by businesses in implementing this change? It's easy to throw stones at companies that are having trouble implementing and supporting patches, but the sure didn't ask for this.
This, in an ironic way, points back to one of the original points of my DST post (see top of thread), which I wrote long before I had heard any talk about the 2007 DST change that's causing so much uproar as I write this comment: DST bugs seem to bite software shops and enterprises twice a year, year after year. This year's DST issue is just an extreme sort. Next year we'll go back to normal, everyday DST bugs, which (in parts of the world where DST applies) will probably be with us forever.
An excerpt from the Stiletto bulletin:
At 2 a.m. Sunday, March 11, 2007, Daylight Saving Time (DST) starts three weeks earlier than usual in a federally mandated effort to save energy. At this time, the Stiletto 10 and 100 will not be able to automatically compensate for this recent change. If you live in a time zone which observes DST the time on your Stiletto will be one hour behind until April 1st, the previously observed beginning of DST. On April 1st, 2007, the time setting on the unit will be corrected.
In general, your device will continue to operate normally during the brief period the time setting is incorrect. There will be no impact to your ability to receive programming or use the LOVE function. However, if you intend to use timer recordings during this period, you will need to take steps to compensate for this offset.
I wish I had me a LOVE function!
Dan
DST is a problem even in Hong Kong...
If your workday has to coincide with financial markets.
Don't trust the date time control in Windows XP if you are working in Europe or Asia, and you want to call your Mom, and you don't want to wake her up. Go to an online site in the USA that is DST change aware.
The ultimate DST bug?
This may be the ultimate Daylight Savings Time "bug": Kid wrongly imprisoned for bomb threat - error due to Daylight Savings Time.
Dan
12 days in jail for a nonexistent bomb threat owing to DST bugs?
What's really nuts is giving the kid a 12 day jail sentence EVEN IF he had really phoned in a bomb threat. In former times, kids who got in trouble as first time offenders were let off with a warning, but today, grownups, who haven't grown up, are frightened of their own kids.
Terrorism? Give me a break. All this "zero tolerance" failed to prevent the killings at Virginia Tech because it makes human time bombs into things undeserving of help in advance of going off.
Kid probably has a lawsuit for wrongful arrest but unless he has a rich Daddy as did those kids at Duke wrongfully accused, it'll go nowhere.
I am glad to be out of America.
Once I implemented the
Once I implemented the client side of a token-based autenthication system (you remain authenticated from now to now + a few minutes). I supported time zones and DST with the appropriate C standard library functions.
Then the system stopped working for any user because the server neglected DST and tokens expired one hour ago + a few minutes.
Of course, by customer request, the solution was disabling DST support in our new library without fixing the existing server and the old clients.