Currently, I'm developing an application that depends on (and thus connects to) various databases via LINQ-to-SQL. For one of the databases, the connection string may vary and is thus configurable - however, the schema of this database is identical for all connection strings.
Because of the configurable connection string, I want to validate the DataContext during the startup of my application, to make sure that all tables and views my application uses, are available.
The Table<T> objects in the DataContext object are always initialized - even if the corresponding SQL table or view doesn't have any records.
So then. Currently, the validation check is performed as follows:
bool valid = _dataContext.Articles.Count() > 0
&& _dataContext.Customers.Count() > 0
&& _dataContext.Orders.Count() > 0;
While this does work, the determination of the value of valid takes quite some time (every record of each Table is touched), which ultimately results in a time out. So, is there a faster, more reliable way to determine whether or not a Table<T> of a certain DataContext really exists as a table in the corresponding database?
Here is an (untested) idea:
Grab the name of your table. You can hard code it in, or you can grab it programmatically via
TableAttribute attribute = (TableAttribute)typeof(MyTableObject)
.GetCustomAttributes(typeof(TableAttribute), true)
.Single();
string name = attribute.Name;
MyTableObject is the LINQ-to-SQL generated object contained in your Table, i.e., the generic parameter T in Table<T>.
(TableAttribute is in System.Data.Linq.Mapping.)
Use the DataContext.ExecuteQuery method as in
var db = new MyDataContext();
var results = db.ExecuteQuery<string>("SELECT name FROM dbo.sysobjects WHERE xtype = 'U'");
bool hasTable = results.Any(s => "dbo." + s == name);
A slight change on Jason's answer (I gave him an upvote :))
public bool TableExistsInDatabase<T>()
{
TableAttribute attribute = (TableAttribute)typeof(T)
.GetCustomAttributes(typeof(TableAttribute), true)
.Single();
var result = ExecuteQuery<bool>(
String.Format(
"IF OBJECT_ID('{0}', 'U') IS NOT NULL
SELECT CAST(1 AS BIT) ELSE
SELECT CAST(0 AS BIT)", attribute.Name));
return result.First();
}
Related
I have a webpage and a gridview connected to a database table. My update queries of the columns are as followed:
if (oldName != NAME && oldCreated == DATE)
{
GeneralDbExecuterService.executeSqlNonQuery(string.Format("UPDATE EXCEPTIONAL_USE_POLICY_PARAM SET NAME = '{0}' WHERE ID = '{1}' ", NAME, ID));
}
// if date was changed alone
if (oldCreated != DATE && oldName == NAME)
{
GeneralDbExecuterService.executeSqlNonQuery(string.Format("UPDATE EXCEPTIONAL_USE_POLICY_PARAM SET CREATED_DATE = to_date('{0}', 'dd/MM/yyyy') WHERE ID = '{1}' ", DATE, ID));
}
// if both values were changed
if (oldName != NAME && oldCreated != DATE)
{
GeneralDbExecuterService.executeSqlNonQuery(string.Format("UPDATE EXCEPTIONAL_USE_POLICY_PARAM SET NAME = '{0}', CREATED_DATE = to_date('{2}', 'dd/MM/yyyy') WHERE ID = '{1}' ", NAME, ID, DATE));
}
My question is, how can I make it more modular?
For example if 2 more columns are added its going to raise my IFs by few if not dozens. What is the best way to achieve that kind of dynamic approach? And is that even possible?. Thanks
edit: my main goal is to be able to detect what/where change has happened, and query the specific columns/values . ( basically its what I did) im just asking if theres a better way because if I were to add 5 more columns, I'd end up adding 40 more if statements..
If you use Entity Framework, rather than raw SQL, then you won't need to have any if statements at all. Your method would take an entity, and your code would just get the existing entity out of the database, and set the properties from the incoming one, irrespective of whether or not they have changed (air code, assumptions made about how you set up your model, etc)...
private async Task Update(PolicyParam p) {
PolicyParam existing = await dbContext.PolicyParams.Single(pp => pp.Id == p.Id);
existing.Date = p.Date;
existing.Name = p.Name;
// Update other properties here
await dbContext.SaveChangesAsync();
}
If you add another column, you just add one more line of code above.
EF has a zillion other benefits, like cleaner code, less chance of SQL injection, etc.
I have a table that contains a LastUpdateOn column, I would like to keep SQL Server current date & time (using GETDATE()) in this column whenever I am doing insert or update operation in my table.
In ADO.NET, we used to call GETDATE() function in insert/update statements, but how we can achieve the same thing using Entity Framework Core?
Reason to store SQL Server date time is we do not want to store the application server date time in LastUpdateOn column as application will be running from different servers and their clock might be not in Sync so we want to store common time from all the places(app servers).
There are two quick solutions for this on top of my head:
Simple put the value as default to GetDate() on this columns and pass null to EF when inserting or update;
Create a function that calls GetDate() before an inserting/update and use this as the value;
Edit:
Create a new EF configuration for DateTime values that get the GetDate() function before inserting/update and automate the process;
So as I moved away from stored procedures (where I used "GetDate()" or "CURRENT_TIMESTAMP" alot)... I embraced the c#/middleware layer.
I do a "null check" and set it on the c# code (in the entity framework data layer code)
Below shows DateTime. DateTime.MinValue means "somebody did not explicitly set it".
myEntityFrameworkEntity.InsertDate = parameterInsertDateValue == DateTime.MinValue ? DateTime.Now : parameterInsertDateValue;
There are other variations, but the idea is the same. Did the "caller" explicitly set the property, or did they not set it, let me write a value on their behalf.
Now, sometimes I don't care what the caller set. If it is a LastUpdatedDate , you can just "force it" as well (always set it to DateTime.Now)....
// i don't care what the parameters are, i always update it in this layer
myEntityFrameworkEntity.LastUpdatedDate = DateTime.Now;
Also, instead of using DateTime(.Now) directly... I use this:
https://github.com/vfabing/Chronos.Net
You "inject" a DateTime-PROVIDER .. that can be mocked for Unit-Tests.
Here is a more complete example to show my EF code (for Rest-Services that needs "disconnected" entities).
public const int ExpectedAddSingleOrUpdateSingleOrDeleteSingleSaveChangesAsyncRowCount = 1;
public async Task<EmployeeEfEntity> UpdateAsync(
EmployeeEfEntity inputEmployee,
CancellationToken token)
{
int saveChangesAsyncValue = 0;
/* find the object in the db-context using the object surrogate-primary-key (since this is an update) */
EmployeeEfEntity foundEntity =
await this.entityDbContext.EmployeeEfEntitys.FirstOrDefaultAsync(
item => item.EmployeeEfEntityKey == inputEmployee.EmployeeEfEntityKey,
token);
if (null != foundEntity)
{
/* update the found-entity with the properties of the input object */
/* the "normal" properties of the employee */
foundEntity.EmployeeBadgeNumber = inputEmployee.EmployeeBadgeNumber;
/* if the insert-date was explicit, use it, otherwise default to date-time.now */
foundEntity.InsertDate = inputEmployee.InsertDate == DateTime.MinValue ? DateTime.Now : inputEmployee.InsertDate;
/* we do not care about the input parameters, we always set it to "now" */
foundEntity.LastUpdated = inputEmployee.LastUpdated == DateTime.Now;
this.entityDbContext.Entry(foundEntity).State = EntityState.Modified;
saveChangesAsyncValue = await this.entityDbContext.SaveChangesAsync(token);
if (ExpectedAddSingleOrUpdateSingleOrDeleteSingleSaveChangesAsyncRowCount != saveChangesAsyncValue)
{
throw new ArgumentOutOfRangeException(
string.Format(ErrorMsgExpectedSaveChangesAsyncRowCount, saveChangesAsyncValue),
(Exception)null);
}
}
else
{
ArgumentOutOfRangeException argEx = new ArgumentOutOfRangeException(
string.Format(ErrorMsgPrimaryEntityNotFound, inputEmployee.EmployeeEfEntityKey),
(Exception)null);
this.logger.LogError(argEx);
}
return foundEntity;
}
My code actually uses the Chronos.Net (injected) date-time-provider, but I removed it to keep the code simpler for this answer.
Since you're using Entity framework, which aimed at allowing database usage to be handled in C# more, I would just use DateTime.Now (DateTime.UtcNow if you want UTC time).
Set LastUpdateOn to DateTime.Now or DateTime.Utc.Now each time you enter/upate a record.
If you really want to use SQL server's GETDATE() method, then write a stored procedure that takes an interger (the id of the record to update) and use it to set LastUpdateOn for the field with the specified id to the value of GETDATE().
I am rather new to SQL in C# and I need some advice on SQL injection.
public System.Linq.IQueryable findBy(List<String> lWhere)
{
string sWhere;
foreach (var (sQueryPart, i) in lWhere.Select((Value, i) => (Value, i)))
{
if (i == 0)
{
sWhere = sQueryPart;
}
else if (i == 1)
{
sWhere += " = " + sQueryPart;
}
else if (i % 2 == 0)
{
sWhere += " and " + sQueryPart;
}
else
{
sWhere += " = " + sQueryPart;
}
}
return this.TABLE.FromSqlRaw("SELECT * FROM TABLE WHERE {0}", sWhere);
}
This method gets a list with entries like {"COLUMN1", "VALUE1" , "COLUMN2", "VALUE2"...}
After that, I build my Where clause using this list and enter it into the select statement.
First of all, the list might get replaced by a dictionary, actually I am pretty sure of that.
Secondly my question, is this safe against SQL injection? There shouldn't be user input other then using the method in program code, but no manual entries after that.
EDIT: it is important that I do not know the number of where clauses used, it could range from 1 to 4
If you are building your SQL query manually by concatenating strings, you are vulnerable to SQL injection. Full stop.
I don't understand why you are even doing this, as your code implies you are using Entity Framework. Which adds methods on your database entities to allow you to dynamically chain as many .Where() clauses as you require, precisely to remove the need for you to write SQL yourself, for example:
var results = dbContext.Table
.Where(t => t.Column1 == "foo")
.Where(t => t.Column2 == 42);
which will generate and execute SQL along the lines of:
select *
from Table
where Column1 = 'foo'
and Column2 = 42;
If you are using Entity Framework properly, you should almost never have to write any SQL yourself. EF will generate it for you, in a way that is not susceptible to SQL injection.
As long as the List of strings is not formed from user input you are safe from SQL Injection; however, this is still bad code:
You should not do select * From. That’s bad because if the table gets a new column that you don’t need -and it may very well be huge binary data- you will end up getting it without needing it, slowing your app considerably.
It’s better to create a stored procedure with parameters for filtering the data. That way, you protect your code from SQL Injection and you can still branch your SQL statement to perform the filtering based on the parameters passed.
If at any point in time your List of Strings is gathered from user input, your code will be vulnerable.
I have a c# console application that is updating a database with about 320,000 records. Basically it is encrypting a password in each record in a loop, then calling DatabaseContext.SubmitChanges(). The "UPDATE" part of the code takes about 20 seconds. I had to CTRL-C the app because it's taking over 15 minutes to do the "SubmitChanges" part: this is part of a time-sensitive system that should not be down for more than a couple minutes.
I ran SQL Profiler and I'm seeing queries like this for each update:
exec sp_executesql N'UPDATE [dbo].[PointRecord]
SET [PtPassword] = #p19
WHERE ([ID] = #p0) AND ([PtLocation] = #p1) AND ([PtIPAddress] = #p2) AND ([PtPort] = #p3) AND ([PtUsername] = #p4) AND ([PtPassword] = #p5) AND ([PtGWParam1] = #p6) AND ([PtGWParam2] = #p7) AND ([PtGWParam3] = #p8) AND ([PtGWParam4] = #p9) AND ([PtTag] = #p10) AND ([PtCapture] = #p11) AND ([PtGroup] = #p12) AND ([PtNumSuccess] = #p13) AND ([PtNumFailure] = #p14) AND ([PtControllerType] = #p15) AND ([PtControllerVersion] = #p16) AND ([PtAssocXMLGroupID] = #p17) AND ([PtErrorType] IS NULL) AND ([PtPollInterval] = #p18)',N'#p0 int,#p1 nvarchar(4000),#p2 nvarchar(4000),#p3 nvarchar(4000),#p4 nvarchar(4000),#p5 nvarchar(4000),#p6 nvarchar(4000),#p7 nvarchar(4000),#p8 nvarchar(4000),#p9 nvarchar(4000),#p10 nvarchar(4000),#p11 int,#p12 nvarchar(4000),#p13 int,#p14 int,#p15 nvarchar(4000),#p16 nvarchar(4000),#p17 int,#p18 int,#p19 nvarchar(4000)',#p0=296987,#p1=N'1234 Anytown USA',#p2=N'10.16.31.20',#p3=N'80',#p4=N'username1',#p5=N'password1',#p6=N'loadmon.htm?PARM2=21',#p7=N'>Operating Mode',#p8=N'',#p9=N'',#p10=N'1234 Anytown USA\HLTH SERVICE LTS\Operating Modeloadmon',#p11=0,#p12=N'1234 Anytown USA',#p13=0,#p14=0,#p15=N'DeviceA',#p16=N'3.5.0.2019.0219',#p17=309,#p18=15,#p19=N'hk+MUoeVMG69pOB3DHYB8g=='
As you can see, the "WHERE" part is asking for EVERY SINGLE FIELD to match, when this is an indexed table, using unique primary key "ID". This is really time-consuming. Is there any way to get this to only use "WHERE ID=[value]"?
I understand now that checking every field is a requirement of concurrency checking in EF. To bypass, methods outside of LINQ are required. I ended up using a variation of what Mr. Petrov and Mr. Harvey suggested, using ExecuteCommand since I am updating the database, not querying for data. Here is sample code, in case it can help others with a similar issue.
It uses LINQ to get the records to update and the record count for user feedback.
It uses ExecuteCommand to update the records. I am actually updating three tables (only one is shown in the sample below), hence the use of a transaction object.
The EncryptPassword method is not shown. It is what I use to update the records. You should replace that with whatever update logic suits your needs.
static void Main(string[] args)
{
DatabaseHelpers.Initialize();
if (DatabaseHelpers.PasswordsEncrypted)
{
Console.WriteLine("DatabaseHelpers indicates that passwords are already encrypted. Exiting.");
return;
}
// Note that the DatabaseHelpers.DbContext is in a helper library,
// it is a copy of the auto-generated EF 'DataClasses1DataContext'.
// It has already been opened using a generated connection string
// (part of DatabaseHelpers.Initialize()).
// I have altered some of the variable names to hide confidential information.
try
{
// show user what's happening
Console.WriteLine("Encrypting passwords...");
// flip switch on encryption methods
DatabaseHelpers.PasswordsEncrypted = true;
int recordCount = 0;
// Note: Using LINQ to update the records causes an unacceptable delay because of the concurrency checking
// where the UPDATE statement (at SubmitChanges) checks EVERY field instead of just the ID
// and we don't care about that!
// We have to set up an explicit transaction in order to use with context.ExecuteCommand statements
// start transaction - all or nothing
DatabaseHelpers.DbContext.Transaction = DatabaseHelpers.DbContext.Connection.BeginTransaction();
// update non-null and non-empty passwords in groups
Console.Write("Updating RecordGroups");
List<RecordGroup> recordGroups = (from p in DatabaseHelpers.DbContext.RecordGroups
where p.RecordPassword != null && p.RecordPassword != string.Empty
select p).ToList();
recordCount = recordGroups.Count;
foreach (RecordGroup rGroup in recordGroups)
{
// bypass LINQ-to-SQL
DatabaseHelpers.DbContext.ExecuteCommand("UPDATE RecordGroup SET RecordPassword={0} WHERE ID={1}", DatabaseHelpers.EncryptPassword(rGroup.RecordPassword), rGroup.ID);
Console.Write('.');
}
// show user what's happening
Console.WriteLine("\nCommitting transaction...");
DatabaseHelpers.DbContext.Transaction.Commit();
// display results
Console.WriteLine($"Updated {recordCount} RecordGroup passwords. Exiting.");
}
catch (Exception ex)
{
Console.WriteLine($"\nThere was an error executing the password encryption process: {ex}");
DatabaseHelpers.DbContext.Transaction.Rollback();
}
}
I am building an ASP.NET MVC 5 application to get read-only access to between 2 and 12 rows at a time. The records are stored in Oracle 11, and accessed via Entity Framework 6, from a view in Oracle. As the title says, it is quick with a hard-coded value, but slow with the string parameter (Example below)
This runs quickly:
public List<VW_COLLISIONS_TS> GetCollisions(string collisionMRN)
{
var collisions = (from c in TSP_Context.VW_COLLISIONS_TS
where c.COLLISION_MASTER_RECORD_NUMBER.Equals("902770479")
select c).AsNoTracking();
return collisions.ToList();
}
But this runs slow:
public List<VW_COLLISIONS_TS> GetCollisions(string collisionMRN)
{
var collisions = (from c in TSP_Context.VW_COLLISIONS_TS
where .COLLISION_MASTER_RECORD_NUMBER.Equals(collisionMRN)
select c).AsNoTracking();
return collisions.ToList();
}
It is taking roughly 30 seconds with the parameter to return the records. Why is this?
UPDATE: I set up a trace, this is what I got:
[#1 - with hard-coded string value]
Opened connection at 2/9/2017 9:48:29 AM -05:00
SELECT
"Extent1"."PERSON_MASTER_RECORD_NUMBER" AS "PERSON_MASTER_RECORD_NUMBER",
"Extent1"."VEHICLE_MODEL" AS "VEHICLE_MODEL",
"Extent1"."PERSON_AGE" AS "PERSON_AGE"
FROM "DWOBDEV"."VW_COLLISIONS_TS" "Extent1"
WHERE ('902770479' = "Extent1"."COLLISION_MASTER_RECORD_NUMBER")
-- Executing at 2/9/2017 9:48:29 AM -05:00
-- Completed in 5 ms with result: OracleDataReader
Closed connection at 2/9/2017 9:44:29 AM -05:00
[2. With string parameter]
Opened connection at 2/9/2017 9:44:17 AM -05:00
SELECT
"Extent1"."PERSON_MASTER_RECORD_NUMBER" AS "PERSON_MASTER_RECORD_NUMBER",
"Extent1"."VEHICLE_MODEL" AS "VEHICLE_MODEL",
"Extent1"."PERSON_AGE" AS "PERSON_AGE"
FROM "DWOBDEV"."VW_COLLISIONS_TS" "Extent1"
WHERE (("Extent1"."COLLISION_MASTER_RECORD_NUMBER" = :p__linq__0) OR (("Extent1"."COLLISION_MASTER_RECORD_NUMBER" IS NULL) AND (:p__linq__0 IS NULL)))
-- p__linq__0: '902770479' (Type = Object)
-- Executing at 2/9/2017 9:44:17 AM -05:00
-- Completed in 12364 ms with result: OracleDataReader
Closed connection at 2/9/2017 9:48:29 AM -05:00
My coworker's take: "I think when you hard-code it is treating as number/integer and when you use string it is treating as varchar/text and then looking for nulls also. Try to use COLLISION_MASTER_RECORD_NUMBER as integer variable, If COLLISION_MASTER_RECORD_NUMBER is of number/int data type."
What do you guys think? [NOTE: Currently, COLLISION_MASTER_RECORD_NUMBER is a varchar2 'number' (no alpha) in the Oracle database]
UPDATE (Answered!):
From what Alexander V posted (including the EF link), I came up with this:
var collisions = (from c in TSP_Context.VW_COLLISIONS_TS
where c.COLLISION_MASTER_RECORD_NUMBER == (DbFunctions.AsNonUnicode(collisionMRN))
select c).AsNoTracking();
And I added this in the Context constructor:
this.Configuration.UseDatabaseNullSemantics = true;
That worked well.
You need to check what sql query generates in both cases.
Try add this before .ToList() call:
var sql = ((System.Data.Entity.Core.Objects.ObjectQuery)collisions)
.ToTraceString();
If you could post sql queries back, we will see what is going on here.
UPDATE: (based on sql query update that #R. Jo posted)
I think performance issue in 2nd query is related to extra NULL checks:
so try to use == "something" instead of .Equals("something")
because Equals could generate extra null checks for nullable types;
and we EF need to work with 2 different data types here when you doing Equals and we want to avoid that also. (see example below).
If code from example wouldn't work try this'
UseDatabaseNullSemantics = true; maybe help.
(see more information here Why is EF generating SQL queries with unnecessary null-checks?.
EDIT:
if COLLISION_MASTER_RECORD_NUMBER then we could do something like
public List<VW_COLLISIONS_TS> GetCollisions(string collisionMRN)
{
int collision = int.Parse(collisionMRN);
var collisions = (from c in TSP_Context.VW_COLLISIONS_TS
where .COLLISION_MASTER_RECORD_NUMBER == collision
select c).AsNoTracking();
return collisions.ToList();
}
Please let me know if it will work for you.