I need to read the result from a stored procedure called from EF Core that includes things intended for spreadsheet formatting purposes such as rows with only null values. My code is mostly working, but there are two columns that cause the query to fail when I include them.
Here's my model class (only a few props included for brevity):
public class StoredProcRecord
{
[Column("Prod Line")]
public string ProductLine { get; set; }
[Column("Current Sales")]
public decimal? CurrentSales { get; set; }
[Column("Current Margin $")]
public decimal? CurrentMargin { get; set; }
}
Here's a snippet of part of the output if I run the stored procedure from SQL Server Management Studio:
Here's how I registered the type as an entity model with EF Core in the db context:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Query<StoredProcRecord>();
}
Here's how I'm calling the stored procedure:
return _context.Query<StoredProcRecord>().FromSql("EXECUTE my_stored_proc").ToList();
In this example, EF Core always throws an exception when trying to call FromSql() with a message stating:
The required column 'Current Margin $' was not present in the results
of a 'FromSql' operation.
According to the EF Core docs, as long as a column is nullable, then it is not required. Yet, I receive that exception that claims "Current Margin $" is required. It also only happens with the CurrentMargin property, not the CurrentSales property, which uses the same type. If I remove CurrentMargin, the whole thing works fine and I'm just missing that one property.
I have tried using [Column(Order = 2)] instead of a named column, thinking maybe the dollar sign in the column name had something to do with it, but that results in:
The required column 'CurrentMargin' was not present in the results of
a 'FromSql' operation.
Including both order and name results in the same type of error.
Worth noting, this is not a stored procedure that I have the option to make updates to.
What is different about the CurrentMargin property compared to CurrentSales that causes it to fail like this? Is there some special way I need to handle the dollar sign in the column name? Am I missing something else completely?
Edit:
Here's a snippet from the very end of the stored procedure where the resulting table is output. I'm including this to show that "Current Margin $" is indeed one of the column names. I actually copied/pasted right from the stored procedure just to make sure I hadn't mis-typed the column name.
BEGIN
-- bulk of stored procedure that builds #tmp_tbl omitted for length
SELECT
'Prod Line' = pl_key,
'Current Sales' = curr_sales,
'Current Margin $' = curr_margin
FROM #tmp_tbl
END
The embarrassing answer to this question is make sure you read your code carefully, especially things like string values.
In this case, I have two stored procedures that will eventually get used by my app, bv_xls_profit_sum_v2 and bv_xls_prod_sum_v2. I was implementing usage of the results of the first stored procedure, but I accidentally provided the name of the second stored procedure to EF. The columns from those two do not match, so the error I was receiving about missing required column wasn't about whether or not the column was nullable, but was actually a case where the column was indeed missing from the result set completely. This wasn't obvious when testing through SQL Server Management Studio because I didn't realize I was running a different stored procedure.
Bonus "Answer":
Prior to figuring out my stupid mistake, I did figure out a workaround that I'll leave here just in case it proves useful for anyone. My app has its own SQL Server instance, but also queries against another SQL Server instance that is not controlled by the app (that's where the stored procedures in question live). To work around the issue I was struggling with, I connected the other SQL Server instance to mine as a linked server. I then duplicated the stored procedure I wanted to call, updated it to have it query the linked server, eliminate unwanted null rows, and return using column names to match my model class. This does work and allows me to manage the stored procedure, though it does likely add additional latency to getting a result set back from querying the modified stored procedure.
Related
I am attempting to use ExecuteSqlInterpolated to update my database. However, it seems that there is a problem with my SQL parameters. Running the following code correctly updates intField1 and intField2, however stringField becomes the string "#p0".
_context.Database.ExecuteSqlInterpolated($"UPDATE table SET stringField='{someString}', intField1={someInt}, intField2={someOtherInt} WHERE id='{id}'");
I have already verified that my variables contain the desired values when the string is passed to the method. I understand that #p0 is what SQL uses to represent the first parameter in the query, but why isn't it being replaced by the string I gave it? I have also tried using ExecuteSqlRaw but ran into the same issue. My knowledge of SQL is limited at best, I know just enough to get by in web dev, so I'm guessing I'm committing some simple error in crafting the query here, but obviously I'm not sure.
I know it's late but just don't use quotation mark with your parameters, specially where you use int data-type
(delete single quotes)
_context.Database.ExecuteSqlInterpolated($"UPDATE table SET stringField={someString}, intField1={someInt}, intField2={someOtherInt} WHERE id={id}");
This is a bit of a puzzle I'm trying to figure out.
I am working on a system where we have a number of company records saved in the database. Some of these records are duplicates and are no longer wanted/required.
However, several external systems are still mapping to these invalid records. If we were to delete them entirely it would cause errors to the systems still wanting to get the detail of that company.
The ideal workflow I would like would be;
The external system looks up Company ID X.
The current system has a table which has a record of all the remapped records, so when the request comes in, the table specifies to redirect Company ID X to Company ID Y.
There are a number of endpoints that could be altered one-by-one to do this - but it would be time-consuming, resulting in lots of repetition too.
My question is, using Entity Framework and .Net - is there a smart way of achieving this workflow?
My initial thoughts were to do something with the constructor for the company object, which repopulates the object from EF if a 'redirect' exists, but I don't know if this will play nice with navigation properties.
Would anyone have an idea?
Thanks very much.
You can create a column with foreign key for the same table to express the single unique valid company.
For example, you can add DuplicateOf column:
ALTER TABLE [Company]
ADD COLUMN [DuplicateOf] bigint NULL,
FOREIGN KEY [DuplicateOf] REFERENCES [Company] ([Id]);
and express this relation in your code:
public class Company
{
// ...
public Company DuplicateOf { get; set; }
// May be useful, hides check for duplicate logic:
public bool IsDuplicate => DuplicateOf != null;
// May be useful as well,
// returns the non-duplicate uniue company, not a duplicate, either linked or current:
public Company EffectiveCompany => DuplicateOf ?? this;
}
You will have to address EffectiveCompany when you want to work with non-duplicate and maintain this column to always point to the correct record. It will also result into additional query, if eager-loaded.
Another idea is to have a stored procedure GetCompany(bigint id) which will return the effective record - if DuplicateOf exists, or record itself otherwise. It will be good for your external systems and will let you hide all this stuff behind abstraction layer of stored procedure. If you decide to change it in future, then you can easily update it without breaking external systems.
However, for you it isn't always convenient to work with stored procedures with EF.
These are just ideas and not the best solutions, anyway.
In my opinion, the best solution would be to get rid of duplicates, update data everywhere and forget forever about this mess of duplicated data.
I'm trying to get an upsert working on a collection of IDs (not the primary key - that's an identity int column) on a table using dapper. This doesn't need to be a dapper function, just including in case that helps.
I'm wondering if it's possible (either through straight SQL or using a dapper function) to run an upsert on a collection of IDs (specifically an IEnumerable of ints).
I really only need a simple example to get me started, so an example would be:
I have three objects of type Foo:
{ "ExternalID" : 1010101, "DescriptorString" : "I am a descriptive string", "OtherStuff" : "This is some other stuff" }
{ "ExternalID" : 1010122, "DescriptorString" : "I am a descriptive string123", "OtherStuff" : "This is some other stuff123" }
{ "ExternalID" : 1033333, "DescriptorString" : "I am a descriptive string555", "OtherStuff" : "This is some other stuff555" }
I have a table called Bar, with those same column names (where only 1033333 exists):
Table Foo
Column ID | ExternalID | DescriptorString | OtherStuff
Value [1]|[1033333] |["I am a descriptive string555"]|["This is some other stuff555"]
Well, since you said that this didn't need to be dapper-based ;-), I will say that the fastest and cleanest way to get this data upserted is to use Table-Valued Parameters (TVPs) which were introduced in SQL Server 2008. You need to create a User-Defined Table Type (one time) to define the structure, and then you can use it in either ad hoc queries or pass to a stored procedure. But this way you don't need to export to a file just to import, nor do you need to convert it to XML just to convert it back to a table.
Rather than copy/paste a large code block, I have noted three links below where I have posted the code to do this (all here on S.O.). The first two links are the full code (SQL and C#) to accomplish this (the 2nd link being the most analogous to what you are trying to do). Each is a slight variation on the theme (which shows the flexibility of using TVPs). The third is another variation but not the full code as it just shows the differences from one of the first two in order to fit that particular situation. But in all 3 cases, the data is streamed from the app into SQL Server. There is no creating of any additional collection or external file; you use what you currently have and only need to duplicate the values of a single row at a time to be sent over. And on the SQL Server side, it all comes through as a populated Table Variable. This is far more efficient than taking data you already have in memory, converting it to a file (takes time and disk space) or XML (takes cpu and memory) or a DataTable (for SqlBulkCopy; takes cpu and memory) or something else, only to rely on an external factor such as the filesystem (the files will need to be cleaned up, right?) or need to parse out of XML.
How can I insert 10 million records in the shortest time possible?
Pass Dictionary<string,int> to Stored Procedure T-SQL
Storing a Dictionary<int,string> or KeyValuePair in a database
Now, there are some issues with the MERGE command (see Use Caution with SQL Server's MERGE Statement) that might be a reason to avoid using it. So, I have posted the "upsert" code that I have been using for years to an answer on DBA.StackExchange:
How to avoid using Merge query when upserting multiple data using xml parameter?
so there are a few other similar type posts about, but none matching the simple task I am attempting.
just running a stored proc through EF6 code first in the following manner
var results = context.DataBase.SqlQuery<MyClass>("exec spGetStuff #param1, #param2", new SqlParameter[] {new SqlParameter("param1",value), new SqlParameter("param2", value2)});
I have used this method on many occasions with no issue.
the class I am mapping the results to is pretty nasty with many properties, but all that need it are marked with the [Column("dbfieldname")] attribute.
all the stored proc is doing is returning some results by using a
SELECT * FROM(
SELECT
ROW_NUMBER() OVER ( PARTITION BY X,Y,Z ORDER BY A) [RowNumber]
,*
FROM
MyTableNAme
WHERE
...) S
WHERE s.RowNumber = 1
not inserting, updating or anything fantastical like that.
The data reader is incompatible with the specified 'MyClass'. A member of the type, 'PropertyNameName', does not have a corresponding column in the data reader with the same name.
if I do change the class properties to the db column names it seems to work fine:I can change the first few properties and it will then fail on other ones in the class... however I do not really want to do that if I can possibly avoid it as most of the columns in the DB are named very badly indeed, so I guess my question is why is it ignoring the [Column()] attributes that have never failed me before.
or is the issue the row_number, which I have tried adding to the class an/or removing from the query resultsset.
We're seeing this issue too since upgrading and refactoring for EF6 from 5. Stored procedure returns and we have a Complex Type defined in our edmx. It seems that everything matches up but we get the same type of error when calling like this
return ((IObjectContextAdapter)this).ObjectContext.ExecuteFunction<GetColumnValueSet_Result>("GetColumnValueSet", param1, param2, param3);
So after trying a few things this is what I found. I opened my edmx and went to the Model Browser within Visual Studio. Under Function Imports I found my sproc. Right clicked and chose Function Import Mapping. It turned out that even though the complex type was correct and it appeared everything should have matched up the Function Import Mapping was thinking the sproc was returning a column called CodeValue instead of Value (which is what it was actually importing).
So for some reason when the sproc columns were gotten it go a wrong name for the mapping.
I have been following the ASP.Net Movie Database Tutorial, and it was all going well.
I have just changed the model and added an attribute. Call me curious, but I didn't want to follow the guide of just drop the database - I wanted to see if I can modify.
The correct error came up about a mismatch - which was expected.
I added the ratings column to the database, and everything worked.
Next, I deleted the ratings column as I wanted to follow the tutorial and learn about DropCreateDatabaseIfModelChanges - however, I just get the error Invalid column name 'Ratings'.
In all this time, the ModelHash entry has not changed, and I have no idea how it know there is a difference between now or before.
So - 1) Have I screwed something up?
2) How can I fix?
3) How did it know before that something has changed, but not now when the hash hasn't changed?
4) Is there any additional advice you can give?
I think you description must be incorrect because correct behavior will throw the exception despite of manually added Rating column.
This behavior works as follows:
When database is created new table called EdmMetadata is added.
EdmMetadata contains two columns - Id and ModelHash.
Part of database creation is storing single row with hash of the current model.
If database was not created by the current context the context executes a query to retrieve stored model hash before the first operation is executed. The retrieved has is compared with context's current hash.
If hash is not the same and database recreation is not allowed by database initializer the exception is thrown.
If hash is the same, required db operation is executed.
Adding manually Rating column will not change stored hash but hash of the model will be different.
This behavior can be completely removed by removing IncludeMetadataConvention convention:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Conventions.Remove(modelBuilder.Conventions
.OfType<IncludeMetadataConvention>().ToArray());
}