We Keep Coding

Filtering using a DateTimeOffset as a parameter issue

I was playing with dates using a C# program.
I want to filter on any table that has a DateTime, DateTime2, DateTimeOffset columns.
I store the LastRefreshDate as DateTimeOffSet in UTC and I use it to filter data on those tables. I adjust the offset (using NodaTime) of the LastRefreshDate based on the timezone used to store the dates in those tables. Usually, it is given by the user.
So I created a test sample to explain the problem. Usually, the SQL queries are dynamic and the parameters as well. Here is the sample code:
[TestMethod]
public void Test()
{
using (SqlConnection connection = new SqlConnection("Server=myserver;Database=mydb;User ID=admin;Password=admin"))
{
connection.Open();
using (SqlCommand command = new SqlCommand("SELECT [TimeStamp] FROM [dbo].[DATA] WHERE [TimeStamp] >= #p0", connection))
{
string datestring = "2019-06-18 13:35:20.1133868 -04:00";
// Does not work
// DateTimeOffset p0 = DateTimeOffset.Parse(datestring, CultureInfo.InvariantCulture);
// Does work
DateTime p0 = DateTime.Parse(datestring, CultureInfo.InvariantCulture);
command.Parameters.AddWithValue("#p0", p0);
using (SqlDataReader reader = command.ExecuteReader())
{
var dataTable = new DataTable();
dataTable.Load(reader);
var result = dataTable.Rows.Count == 0;
}
}
}
}
I created 2 SQL fiddles that demonstrate the issue. By the way, I ran the SQL Server Profiler and the generated queries are similar to the queries in the fiddles.
DateTime fiddle: http://sqlfiddle.com/#!18/a06be/1
declare #p0 datetime = '2019-06-18 13:35:20'
SELECT
[TimeStamp]
FROM
[dbo].[DATA]
WHERE
([TimeStamp] >= #p0)
DateTimeOffSet fiddle: http://sqlfiddle.com/#!18/a06be/2
declare #p0 datetimeoffset(7) ='2019-06-18 13:35:20.1133868 -04:00'
SELECT [TimeStamp]
FROM
[dbo].[DATA]
WHERE
([TimeStamp] >= #p0 )
I did even more tests. By applying the cast directly, the SQL query works. It seems that SQL Server implicit conversion is not behaving in the same manner as an explicit cast. Here is the test case:
declare #p0 datetime
set #p0 = '2019-06-18 17:48:00.00'
declare #p1 datetimeoffset(7)
set #p1 = '2019-06-18 17:47:00.5385563 -04:00'
select 1
where #p0 > cast(#p1 as datetime) -- working
--where #p0 > #p1 -- not working

A few things:
In SQL Server, if you use CAST, or CONVERT without specifying a style, the default style is 0, which when converting a datetimeoffset to either a datetime or datetime2 simply takes the date and time value from the datetimeoffset without considering the offset. If you want to take the offset into account, then use CONVERT and pass 1 for the style:
DECLARE #p0 datetimeoffset = '2019-06-18 13:35:20.1133868 -04:00'
SELECT convert(datetime, #p0, 0) as 'A', convert(datetime, #p0, 1) as 'B'
-- A = 2019-06-18T13:35:20.113Z
-- B = 2019-06-18T17:35:20.113Z
When querying a datetime or datetime2 field using a datetimeoffset parameter, the offset is indeed taken into account in the implicit conversion (it is like B above).
On the C# side, be careful about DateTime.Parse. By default, it emits a local time based value when an offset is provided. If you check, you'll see p0.Kind == DateTimeKind.Local. You could pass DateTimeStyles.AdjustToUniversal, but a better idea is to parse as a DateTimeOffset like you showed in your "doesn't work" code. But then instead of passing the full DateTimeOffset, pass the UtcDateTime property:
DateTime p0 = DateTimeOffset.Parse(datestring, CultureInfo.InvariantCulture).UtcDateTime;
For both performance and stability reasons, you might consider using ParseExact or TryParseExact instead of Parse. Or, since you said you already are using Noda Time, you can use its text parsing features with an OffsetDateTimePattern. From there you'd either call .ToDateTimeOffset().UtcDateTime, or .ToInstant().ToDateTimeUtc().
Alternatively, you could just define your SQL database columns as datetimeoffset, then you can pass any DateTimeOffset parameter and it will be normalized to UTC when querying.

PostgreSQL: 42883 Operator does not exist: timestamp without time zone = text

I am using Npgsql 3.0.3.0 and PetaPoco latest version.
When I run this command:
var dateCreated = DateTime.Now; // just an example
var sql = new Sql("WHERE date_created = #0", dateCreated.ToString("yyyy-MM-dd HH:00:00"));
var category = db.SingleOrDefault<Category>(sql);
I get the following error:
Npgsql.NpgsqlException 42883: operator does not exist: timestamp
without time zone = text
I understand the error message is saying I'm trying to compare a timestamp (date) with a text, however for me it's perfectly valid to compare them as I am expecting the following SQL statement to be built:
SELECT * FROM category WHERE date_created = '2017-02-03 15:00:00'
I don't really want to typecast my database column to text for performance reasons.

You need to cast value to timestsamp:
var sql = new Sql("WHERE date_created = #0::timestamp", dateCreated.ToString("yyyy-MM-dd HH:00:00"));

You need to explicit typecast text to timestamp.
Try using :
TO_TIMESTAMP(date_string,'YYYY-MM-DD')

As #roman-tkachuk answered, you can tell PostgreSQL to cast your string into a timestamp.
Or better yet, you can simply send a timestamp to PostgreSQL directly, rather than sending a string representation of a timestamp and having it cast. To do that, simply send dateCreated directly, without the ToString().

DateTime is saved incorrectly into my database

I'm struggling as why my DateTime is saved incorrectly into my database.
I pass it the value 20/12/2015 (as a string) alongside the format (dd/MM/yyyy) and parse it into a DateTime but it always saves into my SQL Server database as 06/12/2015
public ActionResult SaveSettings(ProjectPlan projectPlan)
{
projectPlan.StartDate = DateTime.ParseExact(projectPlan.ShortDateTime, projectPlan.DateFormat, null); //ShortDateTime is 20/12/2015, DateFormat is dd/MM/yyyy
var plan = this._dc.ProjectPlans.Single(a => a.Id == projectPlan.Id);
plan = projectPlan;
this._dc.SaveChanges();
}

Erm, didn't you mean updating the StartDate property of your entity and then saving it back into the database:
public ActionResult SaveSettings(ProjectPlan projectPlan)
{
var plan = this._dc.ProjectPlans.Single(a => a.Id == projectPlan.Id);
plan.StartDate = DateTime.ParseExact(projectPlan.ShortDateTime, projectPlan.DateFormat, null);
this._dc.SaveChanges();
}
On this line you are basically killing every context EF knows about:
plan = projectPlan;
So if you looked at the actual SQL query generated against your SQL database you would have noticed that exactly 0 rows were updated.

The textual representation is not stored in the server, neither is it in .NET when you have a DateTime object. It is just a large number counting the number of "ticks" from a set time.
You will need to format the date when you select the date out back in to a string, the inserting side can't control it (unless the inserting side also stores the formatting string or you don't store it as a date and instead store it as a string)

how to convert string into DateTime in where Clause?

i have to compare a choosen Date from a Calendar (startdat & enddat) with a Date in my SQL 2008 R2 DB. I have to write it with LINQ, but in the view i have, the DateTime is converted to a String (Varchar) so i have to convert it in my LINQ Query back to DateTime. My basic Query looks like this now:
var reportlist = (from r in context.Monthly_Report
where r.CreateDate >= startdat && r.CreateDate <= enddat
select r.Ticketnumber).ToList();
So the CreateDate i get is a String and for comparing i've to convert it. I've tried it with Convert.ToDateTime() but there's is the Problem with L2E.
So how can i convert it like in a SQL Script or that SQL knows what i means?
Thanks for every help i get. (btw i'm not allowed to change the view)

You can only use a reduced set of functions in Linq to Entities. This functions will be transalated to DB functions.
You can use:
canonical functions: they're availabel for all the providers (DB tastes)
entity functions: exposes canonical functions in the EDM
db functions: exposes canonical functions in the EDM
sql functions: exposes SQL Server specific functions
None of these groups includes a function that can convert from string to datetime, so there is no way to do it directly.
You must look for alternatives:
Create a DB view which exposes the "stringified" datetime as a datetimecolumn and query it
Create a stored proc and use it
Convert the datetime to string and compare it, if at all possible (this depends on how the "stringified" datetime looks like)
You can use the previous solution using substrings (which will map to DB functions). This will work for all cases: reorder the y, m, d, of the "stringified" dt date, so that it looks like "yyyymmdd". Then convert your startdat and enddat to the same format, and compare it in string (alphabetic) order.

Fundamentally, you need your view to return a datetime.
However, there are a couple of ways to do this.
1) You could pull your data out as a string into a list object. Then you wouldnt be using L2E.
var temp = (from r in context.Monthly_Report
select new { r.Ticketnumber, r.CreateDate} ).ToList();
var reportList = temp.Where(r =>
Convert.ToDateTime(r.CreateDate) >= startdat &&
Convert.ToDateTime(r.CreateDate) <= enddat)
2) You could convert your datetime to a string value and compare it.
var reportlist = (from r in context.Monthly_Report
where r.CreateDate.CompareTo(startdatasstring) >= 0 &&
r.CreateDate.CompareTo(enddatasstring) <= 0
select r.Ticketnumber).ToList();

DataTable.Select expression fails using DateTime equals filter

I have some data retrieved from a database which I hold in a data table.
What I want to do is extract only records that have a match to the date passed as a parameter only if it's greater than DateTime.MinValue(). This should be easy.
Below is the code snippet of how I build the query string. I know that if I don't have the date filter I get 'x' records but I always get 0 with the date filter.
string query = string.Format("Field_Name IN( 'GENDER','DOB','MARITAL_STATUS','SS') AND DIFF_TYPE = 'PER' AND Person_ID = '{0}'", Person_ID);
if (ChangeDetected > DateTime.MinValue)
{
query += string.Format(" AND ChangeToDT = #{0}#", ChangeDetected);
}
ChangeDataSet.DifferencesRow[] perChanges = this.m_ChangeDS.Differences.Select(query, "ChangeFromDT ASC, Field_Name DESC") as ChangeDataSet.DifferencesRow[];
I have tried all sorts of variations on outputting format for the DateTime in the filter but they all have the same result.
I don't know enough LINQ to do the conditional filter that way either. {:o(

Try specifying the format directly:
query += string.Format(" AND ChangeToDT = #{0}#", ChangeDetected.ToString("MM/dd/yyyy"));
BTW: This has nothing to do with LINQ.