Do not return DATE fields as Javascript Date

[felixfbecker]

A Javascript Date is always parsed in local timezone. A DATE column in Postgres or any other DB doesn't have any timezone associated with it, it is just a date. It begins and ends differently depending on where you are. Parsing this as a JS Date that has timezone associated with it is a huge problem. Let's say your server is located in GMT+2 (Germany). You insert a date into your database, like 2016-08-26. When you query that in Germany, you will get the Date object Fri Aug 26 2016 00:00:00 GMT+0200. But run toUTCString() on it, and you get Thu, 25 Aug 2016 22:00:00 GMT! The date is one day off. This imposes a huge problem for distributed applications.

TL;DR JavaScript objects are not suited to represent a DATE field, because they have a timezone and time associated, while a DATE has not.

postgres-date should not be used to parse it and instead, the string should be returned (just like with a DECIMAL). Users can register their own type parser with their own DateOnly object if they want.

This will be a breaking change and we would like to depend on this in Sequelize 4.

See sequelize/sequelize#4858

[brianc]

I agree it can be clumsy to use JavaScript dates. This particular bike shed we've painted a few times in the past & kinda settled on the current behavior because it's easy enough to register your own type parser to do whatever you want with dates. I would be open to changing this behavior in the next major release for DATE (not timestamp or timestamptz) to have it return the unparsed string...but I'm not sure supplying your own type parser for dates is really that hard? Anyways this would be a pretty big breaking change so let's table this one until pg@7.0?

[felixfbecker]

Would be nice if we could depend on this in Sequelize 4. We have not yet implemented the connection-level type parsers (registering global type parsers should be avoided).

Of course the change should only be for DATE, not timestamp[tz].

[felixfbecker]

Isn't this pg-types? 😄

[brianc]

I am super smart.

[brianc]

Sorry was doing a bunch of node-postgres issues this morning and got confused. I'll delete the milestone & clean up the comments a bit - I'm only 1 of the maintainers here & my opinion is only 1 of several so we'll see what others weigh in on.

[brianc]

okay tried to fix it up! Sorry for the confusion! Toooo many browser tabs! 😁

[bendrucker]

I also don't have strong feelings about this. I don't think returning a date as a Date is wrong, regardless of the time zone issues. The story in Javascript has always been that you have to be careful about time zones. I don't think it's reasonable to run toUTCString() on something you know is a pure date. If I'm using a Date to store info like that I'm using helper functions later to extract the date info in local time.

[felixfbecker]

The example might be a bit misleading. Consider the server is GMT+2. You query your database and then JSON.stringify() your data and send it to the client. The client is done by a company who is using your API, you have no control over it, and the user is sitting in England. Now the client gets the date string Fri Aug 26 2016 00:00:00 GMT+0200 inside its AngularJS app and wants to display this in a localized date format, so he uses a date filter to format it to DD.MM.YYYY. Now what the user sees is 25.08.2016. What happened? A very subtle but major bug got introduced. The client parsed the date string in its own timezone. If it was a simple ISO YYYY-MM-DD string, this would not have been a problem. But because it incorrectly included a time and timezone, the date is one day off in his timezone.

You could of course argue that the API should simply not have used JSON.stringify() to stringify the date, but first convert every single date to a YYYY-MM-DD string. But that is huge boilerplate code for big models and Imo the library shouldn't even use this kind of object to represent the DATE field.

[bendrucker]

You could of course argue that the API should simply not have used JSON.stringify() to stringify the date, but first convert every single date to a YYYY-MM-DD string.

That's my expectation here, despite the acknowledged need for extra boilerplate around serialization. Dates give you language level features like date1 > date2. Returning a string means people who wish to treat the date like a date in their code have extra boilerplate. To me it's not the database driver's responsibility to help avoid programmer error elsewhere in the application.

[felixfbecker]

Thanks for the feedback. So far we only heard from people that a DATE (date-only) should be represented like a date-only and not as a timestamp with timezone, see the issue I linked.

I actually never used this implicit object comparison, but always compared .getTime() values. If you want to compare them, you can easily use Date.parse() on both of them, and then it will not impose any problem with timezones, because you parse both strings in the same statement.

[bendrucker]

That's helpful but there's also a pretty significant confirmation bias in issues like that. You're far more likely to hear from the people who want strings and got tripped up by JS dates than the people who might be relying on a capital-D Date.

The common factor in other types that are not parsed is that they cannot be safely represented by a JS value type, e.g. int types with enough bits to be potentially greater than Number.MAX_SAFE_INTEGER.

This change would make dates inconsistent with other types, including others with potentially treacherous locale behavior (timestamp).

[felixfbecker]

The problem is for a column that only contains date information and nothing about the time or a timezone (therefor representing a timezone-independent 24h range), an object is used for representation that includes date, time and timezone information (therefor representing a millisecond exact point in time in a specific meridian).

Yes you can argue that you will hear more about people who complain, but I don't think that argument is 100% valid. In open source projects there are always also people who express their opinions against a proposed change, the sequelize issue is open for quite some time and so far you are the first one I met who considers this change undesirable.

I think the use case of JSON.stringifying your database object is way greater than any native comparison capability. NodeJS is very often used to build RESTful JSON APIs, and it should be possible to recursively turn objects with data that somewhere comes from the database into a logical JSON representation without having to iterate and correct every single DATE(ONLY) property. In the common API you will have dozens of endpoints that need to send back a DATE(ONLY) in JSON, but I would argue having to compare two DATE(ONLY)s is very rare, and the additional code required to do it is way less than the additional code required to correct DATE(ONLY)s before JSON.stringifying them.

Yes, custom type parsers can be used but to be honest I would feel weird making that decision inside the sequelize library and taking a high-abstraction object from the low-level database driver and converting it into a no-abstraction string inside the high-level ORM. Behavior should be consistent so the user is not confused if he comes from using plain pg. We should decide on one common way to represent DATE(ONLY) columns in Javascript, or at least make the low-level db driver delegate that decision to higher-level modules or the user by not using an abstraction.

[bendrucker]

Ok, fair enough. I do want to throw out a third option here. Date.parse is not 100% consistent across all environments. We could instead return a {year, month, date} object with number values and a toString method similar to intervals.