We Keep Coding

UTC to local time conversion for previously saved datetimes if rules for timezone change

I'm storing a product in db. All dates (sql server datetime) are UTC and along with the dates I store the time zone id for that product. User enters the dates when product is available "from" and "until" in the listing.
So I do something like:
// Convert user's datetime to UTC
var userEnteredDateTime = DateTime.Parse("11/11/2014 9:00:00");
// TimeZoneInfo id will be stored along with the UTC datetime
var tz = TimeZoneInfo.FindSystemTimeZoneById("FLE Standard Time");
// following produces: 9/11/2014 7:00:00 AM (winter time - 1h back)
var utcDateTime = TimeZoneInfo.ConvertTimeToUtc(userEnteredDateTime, tz);
and save the record. Let's assume user did this on the 1st of August, while his time zone offset to UTC is still +03:00, nevertheless the saved date for the future listing has the correct +02:00 value because conversion took into consideration the "winter" time for that period.
Question is what datetime value will I get if I will attempt to convert that product's "from" and "until" date to product's local time zone on 11/11/2014 if, for example, due to some new rules the transition to winter time was abandoned, thus the time zone is still +03:00 instead of +02:00?
// Convert back
var userLocalTime = TimeZoneInfo.ConvertTimeFromUtc(utcDateTime, tz);
will I get 10AM or correct 9AM because OS/.NET patch will handle this?
Thank you!
P.S.: TimeZoneInfo has ToSerializedString() method, if I rather store this value instead of timezone id, will this guarantee that via UTC datetime + serialized timezoneinfo I will always be able to convert to the user's original datetime input?

In the scenario you describe, you would get 10:00 AM. The time zone conversion function would not have any idea that the value was originally entered as 9:00 AM, because you only saved the UTC time of 7:00 AM.
This illustrates one of the cases where the advice "always store UTC" is flawed. When you're working with future events, it doesn't always work. The problem is that governments change their mind about time zones often. Sometimes they give reasonable notice (ex. United States, 2007) but sometimes they don't (ex. Egypt, 2014).
When you made the original conversion from local time to UTC, you intentionally decided to trust that the time zone rules would not change. In other words, you decided that you would assign the event to the universal timeline based solely on the time zone rules as you knew them at that time.
The way to avoid this is simple: Future events should be scheduled in local time. Now, I don't mean "local to your computer", but rather "local to the user", so you will need to know the user's time zone, and you should also store the ID of the time zone somewhere.
You'll also need to decide what you want to do if the event falls into the spring-forward or fall-back transition for daylight saving time. This is especially important for recurrence patterns.
Ultimately though, you'll need to figure out when to run the event. Or in your case, you'll need to decide if the event has passed or not. There are a few different ways you can accomplish this:
Option 1
You can calculate the corresponding UTC value for each local time and keep it in a separate field.
On some cycle (daily, weekly, etc) you can recalculate upcoming UTC values from their local values and your current understanding of the time zone rules. Or, if you apply time zone updates manually, you can choose to recalculate everything at that time.
Option 2
You can store the values as a DateTimeOffset type instead of a DateTime. It will contain the original local time, and the offset that you calculated based on the time zone rules as you knew them at time of entry.
DateTimeOffset values can easily be coerced back to UTC, so they tend to work very well for this. You can read more in DateTime vs DateTimeOffset.
Just like in option 1, you would revisit the values periodically or after time zone data updates, and adjust the offsets to align with the new time zone data.
This is what I usually recommend, especially if you're using a database that has support for DateTimeOffset types, such as SQL Server or RavenDB.
Option 3
You can store the values as a local DateTime.
When querying, you would calculate the current time in the target time zone and compare against that value.
DateTime now = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, targetTZ);
bool passed = now >= eventTime;
The down side of this option is that you may have to make lots queries if you have events in lots of different time zones.
You may also have issues with values close to the fall-back DST transition, so be careful if you use this approach.
I recommend against the idea of serializing the time zone itself. If the time zone has changed, then it has changed. Pretending that it hasn't isn't a good workaround.

Representation of a DateTime as the local to remote user

Hello!
I was confused in the problem of time zones. I am writing a web application that will contain some news with dates of publication, and I want the client to see the date of publication of the news in the form of corresponding local time. However, I do not know in which time zone the client is located.
I have three questions.
I have to ask just in case: does DateTimeOffset.UtcNow always returns the correct UTC date and time, regardless of whether the server is dependent on daylight savings time? For example, if the first time I get the value of this property for two minutes before daylight savings time (or before the transition from daylight saving time back) and the second time in 2 minutes after the transfer, whether the value of properties in all cases differ by only 4 minutes? Or here require any further logic? (Question #1)
Please see the following example and tell me what you think.
I posted the news on the site. I assume that DateTimeOffset.UtcNow takes into account the time zone of the server and the daylight savings time, and so I immediately get the correct UTC server time when pressing the button "Submit". I write this value to a MS SQL database in the field of type datetime2(0).
Then the user opens a page with news and no matter how long after publication. This may occur even after many years. I did not ask him to enter his time zone. Instead, I get the offset of his current local time from UTC using the javascript function following way:
function GetUserTimezoneOffset()
{
var offset = new Date().getTimezoneOffset();
return offset;
}
Next I make the calculation of the date and time of publication, which will show the user:
public static DateTime Get_Publication_Date_In_User_Local_DateTime(
DateTime Publication_Utc_Date_Time_From_DataBase,
int User_Time_Zone_Offset_Returned_by_Javascript)
{
int userTimezoneOffset = User_Time_Zone_Offset_Returned_by_Javascript; // For
// example Javascript returns a value equal to -300, which means the
// current user's time differs from UTC to 300 minutes. Ie offset
// is UTC +6. In this case, it may be the time zone UTC +5 which
// currently operates summer time or UTC +6 which currently operates the
// standard time.
// Right? (Question #2)
DateTimeOffset utcPublicationDateTime =
new DateTimeOffset(Publication_Utc_Date_Time_From_DataBase,
TimeSpan.Zero); // get an instance of type DateTimeOffset for the
// date and time of publication for further calculations
DateTimeOffset publication_DateTime_In_User_Local_DateTime =
utcPublicationDateTime.ToOffset(new TimeSpan(0, - userTimezoneOffset, 0));
return publication_DateTime_In_User_Local_DateTime.DateTime;// return to user
}
Is the value obtained correct? Is this the right approach to solving this problem? (Question #3)
UPDATED Oct 19 at 6:58 (I tried post it as a comment but it's too long by 668 characters)
Matt Johnson, Thank You for such a detailed answer despite that of the fact that you are doing this not the first time. Thank you for taking the time to explain this particular case, and not just provide links to other posts.
I have read the information that you have provided. Maybe I'm still not fully aware of all the details, but if I understand it right, for the right convertion of DateTime (which was written many years ago in the database) from UTC to the same user's moment, I need to know UTC offset which he had at that moment. And it is difficult taking into account that transfer rules for DST change constantly. And even now, though the platform ".NET" contains some TZDB that is used for the TimeZoneInfo type, I can't take advantage of it without the exact position of the user.
But what if I am only interested in the date and time of starting this year, and only in Russia, where DST was canceled in 2011? As I understand it, this means that if properly configured clock on user's computer, located in Russia, this approach will always give the correct result. And since 2011, the offset to UTC of user's clock should always be the same. Accordingly, the shift indicators in different browsers will not be different for the Russian user's.

Answer to Question 1
... does DateTimeOffset.UtcNow always returns the correct UTC date and time, regardless of whether the server is dependent on daylight savings time?
Yes. As long is your clock is set correctly, UtcNow always refers to the UTC time. The time zone settings of the server will not affect it. The value in your example will always be 4 minutes, regardless of DST.
Answer to Question 2
var offset = new Date().getTimezoneOffset();
Since new Date() returns the current date and time, this will return you the current offset. You then proceed to apply this current offset to some past value, which may or may not be the correct offset for that specific time. Please read the timezone tag wiki, especially the section titled "Time Zone != Offset".
Answer to Question 3
Is the value obtained correct? Is this the right approach to solving this problem?
No. This is not the correct approach. You have a few options:
First Option
Just pass the UTC value to JavaScript without modification.
Send it in ISO8601 format, such as 2013-10-18T12:34:56.000Z. You can get this in .Net easily using yourDateTime.ToString("o").
Be sure the DateTime you are starting with has .Kind == DateTimeKind.Utc, otherwise it won't get the Z on the end, which is essential.
If you are targeting older browsers that can't parse ISO8601, then you will need a library such as moment.js to parse it for you.
Alternatively, you could pass the number of milliseconds since 1/1/1970 UTC as a number and load that into a JavaScript Date instead of parsing.
Now you can just display the Date object using JavaScript. Let the browser convert it from UTC to the users local time.
Warning, with this approach, some conversions might be incorrect due to the problem I describe here.
Second Option
Like the first option, pass the UTC timestamp to JavaScript
Use that to get the offset for that timestamp.
Pass the offset back to the server in a postback or ajax call.
Apply the offset on the server
Output the local time zone
I don't particularly like this option because of the round trip. You might as well calculate it in JavaScript like the first option.
Third Option
Get the user's time zone. The best way is to ask them for it, usually on a user setting page.
Use it to convert from the UTC time to their local time completely on the server.
You can use Windows time zones and the TimeZoneInfo class to do the conversions.
Or you can use IANA/Olson time zones and the Noda Time library.
If you do this, you can optionally use a map-based timezone picker.
You can take a guess at the user's time zone with jsTimeZoneDetect.
This approach is more accurate, but requires more user interaction.
Also, please in future posts, ask just one question at a time. And please search before you post - I've written most of this in one form or another many times before.

Converting any date time from server time to user time (dealing with standard and daylight time)

I am working on a web app using .NET MVC 3 and SQL server 2008 (r2).
Anyway, I have a date time object and I want to convert it to user time. It is fairly trivial to convert right now to user time; I have some java script that will get me the users offset from UTC. And I know my date times offset to UTC.
What I realized yesterday, is that my user's offsets will change, if they live in a pesky area of the world. And that would cause old dates and time to be wrong by some amount of time.
Now I know C# has some utilities to convert into a time zones time, but do they really handle all of the intricacies of daylights savings?
For example. If I have a time, 10-10-2001 8:00:00 how do I get that into pacific time? Or if i have 6-6-2004, how do i get that into pacific time?
Since the dates change for the roll overs, it is pretty complex, and I would like a general solution that does not require maintenance to set up date ranges for time zones.
I know this is a classic problem in CS, but I cannot find something that really answers my question 100%. From what I have seen: C# uses the current year's daylights savings dates to change every years date times. Which could cause some mistakes.
Thank you so much for the help.

Now I know C# has some utilities to convert into a time zones time, but do they really handle all of the intricacies of daylights savings?
TimeZoneInfo does, yes1. (It's part of the .NET framework, not part of C# - C# is just the language you happen to be using.) However, I don't think that's what you really want to do.
Why are you storing the DateTime in the server's time zone anyway? It would be more sensible to store it in UTC, in most cases. Aside from anything else, if your server is in a time zone which observes daylight saving time, you will end up with ambiguity for one hour per year, when the clock goes back. (The same local time occurs twice.)
Once you've stored it as UTC, you should give it to your Javascript client as UTC too. While you say that you have "some java script that will get me the users offset from UTC" - that will depend on the exact instant in time. For example, as I'm in the UK, my offset is sometimes 0 and sometimes +1 hour. If you pass the UTC back to the client, that can work out the local time from that UTC time. Your server can't, unless you can get an accurate representation of the time zone from the client to the server, which is generally a tricky thing to do.
From what I have seen: C# uses the current year's daylights savings dates to change every years date times.
Again, C# itself isn't relevant here. It's not clear which part of the .NET framework you mean - TimeZone? TimeZoneInfo? DateTime? TimeZoneInfo has historical data, but only if you're on an operating system version which supports it.
1 Well, as far as you're likely to care. It doesn't have as much historical data as TZDB, and it has some very odd representations for Russia and Namibia, but it does generally have the idea of rules changing.

I have found the following rules to be the best solution:
At server-side use UTC. No need to deal with Time Zones at all. That means that if you store dates in database, use UTC dates, if you generate dates in server side use generally UTC (e.g: var nowDate =DateTime.UtcNow). The server dates should be location agnostic and that way there is no confusion.
At client-side use UTC as well when dealing with raw data, but display the dates in the UI converted to the browser's local time (which will be whatever time is configured in the operating system).
The good news is that this is already a built in capability in browsers! we just need to make it easier for them to handle it.
This means that if we have a javascript client code POSTing/PUTing/PATCHing data to a server and GETting data from the server using JSON, the JSON should use the ISO 8601 (natively supported) in UTC (ended in Z) as if you were doing:
var dateNow = new Date(); //Wed May 11 2016 13:06:21 GMT+1200 (New Zealand Standard Time)
var dateNowIso = dateNow.toISOString(); //"2016-05-11T01:06:21.147Z"
And then, simply rely on the browser capability to convert automatically any UTC date to the current local time and vice versa. The browser's javascript engine will be able to display in the UI the correct local time as long as the format is a Date object.
NOTE on how to implement it for a .NET server & Javascript client scenario: For example, I am using AngularJS (with restangular) in my client javascript code and MVC 6 (ASP.NET Core 1.0) in server side.
The JSON that comes from the server might contain date properties, but in JSON the type is a string such as:
{
"myDateField":"2016-05-11T05:00:00Z"
}
In order to rely on the browser's capability to handle properly UTC times and convert them to local browser's time I need to parse this UTC date string into a real javascript Date object, so I use a regular expression to match any text value that looks like a UTC date in ISO 8601 format (in my case this code is in a restangular response interceptor but it could be anywhere):
const regexIso8601 = /((((\d{4})(-((0[1-9])|(1[012])))(-((0[1-9])|([12]\d)|(3[01]))))(T((([01]\d)|(2[0123]))((:([012345]\d))((:([012345]\d))(\.(\d+))?)?)?)(Z|([\+\-](([01]\d)|(2[0123]))(:([012345]\d))?)))?)|(((\d{4})((0[1-9])|(1[012]))((0[1-9])|([12]\d)|(3[01])))(T((([01]\d)|(2[0123]))(([012345]\d)(([012345]\d)(\d+)?)?)?)(Z|([\+\-](([01]\d)|(2[0123]))([012345]\d)?)))?))/;
function convertDateStringsToDates(input: any): void {
// Ignore things that aren't objects.
if (typeof input !== "object") {
return input;
}
for (var key in input) {
if (!input.hasOwnProperty(key)) {
continue;
}
let value = input[key];
let match: RegExpMatchArray;
// Check for string properties which look like dates.
if (angular.isArray(value)) {
angular.forEach(value, (val, key) => {
convertDateStringsToDates(val);
});
} else if (angular.isString(value) && (match = value.match(regexIso8601))) {
let milliseconds = Date.parse(match[0]);
if (!isNaN(milliseconds)) {
input[key] = new Date(milliseconds);
}
} else if (angular.isObject("object")) {
// Recurse into object
convertDateStringsToDates(value);
}
}
}
That way I can use my javascript object with proper Date objects.
In the server side (either MVC6 or WebAPI2..) I have an instruction to tell my JSON parser to ALWAYS use UTC formats when parsing Dates, otherwise it wouldn't add the Z character at the end, which is very important. Example for MVC6:
services.AddMvc().AddJsonOptions(opt => {
// json dates always in javascript date format with UTC e.g: "2014-01-01T23:28:56.782Z"
opts.SerializerSettings.DateTimeZoneHandling = Newtonsoft.Json.DateTimeZoneHandling.Utc;
});
Inspired by this article
UPDATE: There are scenarios where it might be interesting to store data in local time as stated in the comments below.

Datetime nightmare, any thorough module or control in C# to help ease the pain?

once again I have to create a date module, and once again i live the horror of perfecting it, is it me or are date and time the filthiest animals in programming profession, its the beast lurking behind the door that I wish I never have to deal with :(
does anyone know of a great source I can learn from that deals with dates in the following aspects:
user enters datetime and time zone
system translates to universal time and saves in data source
system retrieves universal time converted to local time chosen by developer (not by server or client location which may not be the right zone to display)
system should consider daylight time saving differences
cannot rely on "DateTime" parsing as it parses bohemiangly with respect to local server time
must give ability to developer to deal in both shapes: datetime and string objects
i looked at blogengine.net to see how they deal with but its too nieve, they save the time difference in hours in the settings datasource, it is absoluteley inaccurate... any sources to help?
i already went far in creating the necessary methods that use CultureInfo, TimeZoneInfo, DateTimeOffset ... yet when i put it to the test, it failed! appreciate the help
EDIT:
After squeezing some more, i narrowed it down to this:
public string PrettyDate(DateTime s, string format)
{
// this one parses to local then returns according to timezone as a string
s = TimeZoneInfo.ConvertTimeBySystemTimeZoneId(s, "AUS Eastern Standard Time");
CultureInfo Culture = CultureInfo.CreateSpecificCulture("en-au");
return s.ToString(format , Culture);
}
problem is, I know the passed date is UTC time because im using
DateTimeOffset.Parse(s, _dtfi).UtcDateTime;
// where dtfi has "yyyy-MM-ddTHH:mmzzz" as its FullDateTimePattern
when i call the function on my datetime, like this:
AuDate.Instance.PrettyDate(el.EventDate,"yyyy-MM-dd HH:mm zzz");
on my machine i get:
2009-11-26 15:01 +11:00
on server I get:
2009-11-26 15:01 -08:00
I find this very peculiar! why is the timezone incorrect? everything else is in place! have i missed something?

My comments for your pointers.
user enters datetime and time zone
# OK no issue
system translates to universal time and saves in data source
# OK no issue
system retrieves universal time converted to local time chosen by developer (not by server or client location which may not be the right zone to display)
# Is this s requirement? Why not just retrieve as universal time
system should consider daylight time saving differences
# Can be handled by DaylightTime Class, TimeZone Class etc
cannot rely on "DateTime" parsing as it parses bohemiangly with respect to local server time
# Then do not rely on DateTime Parsing
must give ability to developer to deal in both shapes: datetime and string objects
# DateTime Class as the basis should be good enough, use TimeZone / TimeZoneInfo / DaylightTime / DateTimeOffset etc to augment it

I feel your pain - which is why I'm part of the Noda Time project to bring a fully-featured date and time API to .NET. However, that's just getting off the ground. If you're still stuck in a year's time, hopefully Noda Time will be the answer :)
The normal .NET situation is better than it was now that we've got DateTimeOffset and TimeZoneInfo, but it's still somewhat lacking.
So long as you use TimeZoneInfo correctly twice, however, it should be fine. I'm not sure that DateTime parsing should be too bad - I think it should parse it as DateTimeKind.Unspecified unless you specify anything else in the data. You can then convert it to UTC using TimeZoneInfo.
Could you provide a short but complete program which shows the problems you're having?

Actually, I find the .NET date/time functionality to be quite nice. I'm puzzled by your troubles with it.
What exactly are you trying to do that DateTimeOffset and TimeZoneInfo can't do for you?
"User enters datetime and timezone" -- Check! Either DateTime or DateTimeOffset would work here.
"System translates to universal time and saves in data source" -- Check! Again, either DateTime or DateTimeOffset would work for you, although most database backends will need some special handling if you want to store timezone offsets. If you're already converting it to UTC, just store it as a datetime field in SQL Server or the equivalent in another RDBMS and don't worry about storing the fact that it's UTC.
"System retrieves universal time converted to local time chosen by the developer" -- Check! Just construct a TimeZoneInfo for your desired local time, and then call TimeZoneInfo.ConvertTime.
"System should consider daylight time saving differences" -- Check! That's what TimeZoneInfo.AdjustmentRule is for.
"Cannot rely on "DateTime" parsing as it parses bohemiangly with respect to local server time" -- ??? First off, "bohemiangly" isn't even a word. And you can customize how the datetime gets parsed with DateTime.ParseExact.
"Must give ability to developer to deal in both shapes: datetime and string objects" -- Why? What's wrong with just keeping one internal representation and then transforming only on input and output? I can't think of any operation on date/time values that would be made easier by doing it against a string.
In short, I think you're just griping about the complexities of handling date/time data in general.

Thanks to Jon Skeet who put me on the right track, i never knew this before but now I know, DateTime object does not hold time zone information in it AT ALL! so whenever i use it i am already losing the datetime offset information, the DateTimeOffset object however, retains the time zone bit, so all my objects should use that, i really thought datetimeoffset object to be a bit limiting, i wanted to post a question about what is different between datetime and datetimeoffset, i should have done that!
now Im using the following code to retrieve the right zone:
string s = "2009-11-26T04:01:00.0000000Z";
DateTimeOffset d = DateTimeOffset.Parse(s);
TimeZoneInfo LocalTimeZoneInfo = TimeZoneInfo.FindSystemTimeZoneById("AUS Eastern Standard Time");
DateTimeOffset newdate = TimeZoneInfo.ConvertTime(d, LocalTimeZoneInfo);
return newdate.ToString("yyyy-MM-dd HH:mm zzz");
thank you all for your input

Datetime timezone adjustments

My database is located in e.g. california.
My user table has all the user's timezone e.g. -0700 UTC
How can I adjust the time from my database server whenever I display a date to the user who lives in e.g. new york? UTC/GMT -4 hours

You should store your data in UTC format and showing it in local timezone format.
DateTime.ToUniversalTime() -> server;
DateTime.ToLocalTime() -> client
You can adjust date/time using AddXXX methods group, but it can be error prone. .NET has support for time zones in System.TimeZoneInfo class.

If you use .Net, you can use TimeZoneInfo. Since you tagged the question with 'c#', I'll assume you do.
The first step is getting the TimeZoneInfo for the time zone in want to convert to. In your example, NY's time zone. Here's a way you can do it:
//This will get EST time zone
TimeZoneInfo clientTimeZone
= TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time");
//This will get the local time zone, might be useful
// if your application is a fat client
TimeZoneInfo clientTimeZone = TimeZoneInfo.Local;
Then, after you read a DateTime from your DB, you need to make sure its Kind is correctly set. Supposing the DateTime's in the DB are in UTC (by the way, that's usually recommended), you can prepare it to be converted like this:
DateTime aDateTime = dataBaseSource.ReadADateTime();
DateTime utcDateTime = DateTime.SpecifyKind(aDateTime, DateTimeKind.Utc);
Finally, in order to convert to a different time zone, simply do this:
DateTime clientTime = TimeZoneInfo.ConvertTime(utcDateTime, clientTimeZone);
Some extra remarks:
TimeZoneInfo can be stored in static fields, if you are only interested in a few specific time zones;
TimeZoneInfo store information about daylight saving. So, you wouldn't have to worry about that;
If your application is web, finding out in which time zone your client is in might be hard. One way is explained here: http://kohari.org/2009/06/15/automagic-time-localization/
I hope this helps. :)

You could use a combination of TimeZoneInfo.GetSystemTimeZones() and then use the TimeZoneInfo.BaseUtcOffset property to offset the time in the database based on the offset difference
Info on System.TimeZoneInfo here

Up until .NET 3.5 (VS 2008), .NET does not have any built-in support for timezones, apart from converting to and from UTC.
If the time difference is always exactly 3 hours all year long (summer and winter), simply use yourDate.AddHours(3) to change it one way, and yourDate.AddHours(-3) to change it back. Be sure to factor this out into a function explaining the reason for adding/substracting these 3 hours.

You know, this is a good question. This year I've done my first DB application and as my input data related to time is an Int64 value, that is what I stored off in the DB. My client applications retrieve it and do DateTime.FromUTC() or FromFileTimeUTC() on that value and do a .LocalTime() to show things in their local time. I've wondered whether this was good/bad/terrible but it has worked well enough for my needs thus far. Of course the work ends up being done by a data access layer library I wrote and not in the DB itself.
Seems to work well enough, but I trust others who have more experience with this sort of thing could point out where this is not the best approach.
Good Luck!