I'm trying to retrieve some entities using Entity Framework by querying an XML column. Entity Framework doesn't support this so I had to use raw SQL.
var people = context.People.SqlQuery("SELECT * FROM [People] WHERE [DataXML].value('Properties/Age', 'int') = 21").AsQueryable().AsNoTracking();
My person class:
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
[Column("YearsSinceBirth")]
public int Age { get; set; }
[Column(TypeName = "xml")]
public string DataXML { get; set; }
}
This should work, however, it falls over when trying to map it back to an object. Specifically, it's falling over on the Age property, which has it's column name overridden to "YearsSinceBirth".
'The data reader is incompatible with the specified
'MyProject.CodeBase.DataModel.DbEntities.Person'. A member of the
type, 'Age', does not have a corresponding column in the data reader
with the same name.'
I'm guessing that Entity Framework doesn't map database column names to object property names and therefore is expecting the column to be named 'Age' rather than 'YearsSinceBirth'.
I don't want to have to list each column and their mapping in the SQL query (like SELECT YearsSinceBirth As Age) as the actual project I'm working on which has this column has a lot more columns and that would mean this query would break every time the schema changed (kinda defeating the purpose of Entity Framework).
If this is EF Core, your problem is not that SqlQuery() doesn't support mapping column names (it does). Rather your problem is that your table doesn't contain a column called YearsSinceBirth, and you are returning 'select *'.
If you have a column called YearsSinceBirth, this works fine. Although you will be retrieving the value in the YearsSinceBirth column, not the value in the XML document. EG
using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
//using Microsoft.Samples.EFLogging;
using System.ComponentModel.DataAnnotations.Schema;
using System.ComponentModel.DataAnnotations;
using System.Data.SqlClient;
namespace EFCore2Test
{
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
[Column("YearsSinceBirth")]
public int Age { get; set; }
[Column(TypeName = "xml")]
public string DataXML { get; set; }
}
public class Location
{
public string LocationId { get; set; }
}
public class Db : DbContext
{
public DbSet<Person> People { get; set; }
public DbSet<Location> Locations { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer("Server=(local);Database=EFCoreTest;Trusted_Connection=True;MultipleActiveResultSets=true");
base.OnConfiguring(optionsBuilder);
}
}
class Program
{
static void Main(string[] args)
{
using (var db = new Db())
{
db.Database.EnsureDeleted();
//db.ConfigureLogging(s => Console.WriteLine(s));
db.Database.EnsureCreated();
var p = new Person()
{
Name = "joe",
Age = 2,
DataXML = "<Properties><Age>21</Age></Properties>"
};
db.People.Add(p);
db.SaveChanges();
}
using (var db = new Db())
{
var people = db.People.FromSql("SELECT * FROM [People] WHERE [DataXML].value('(/Properties/Age)[1]', 'int') = 21").AsNoTracking().ToList() ;
Console.WriteLine(people.First().Age);
Console.ReadLine();
}
Console.WriteLine("Hit any key to exit");
Console.ReadKey();
}
}
}
You can use a pattern similar to this to project entity attributes from an XML or JSON column:
public class Person
{
private XDocument xml;
public int Id { get; set; }
public string Name { get; set; }
[NotMapped]
public int Age
{
get
{
return int.Parse(xml.Element("Properties").Element("Age").Value);
}
set
{
xml.Element("Properties").Element("Age").Value = value.ToString();
}
}
[Column(TypeName = "xml")]
public string DataXML
{
get
{
return xml.ToString();
}
set
{
xml = XDocument.Parse(value);
}
}
}
You can dynamically create select query with aliases, if they needed, with the help of reflection and ColumnAttribute checking:
public string SelectQuery<T>() where T : class
{
var selectQuery = new List<string>();
foreach (var prop in typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
var attr = prop.GetAttribute<ColumnAttribute>();
selectQuery.Add(attr != null ? $"{attr.Name} as {prop.Name}" : prop.Name);
}
return string.Join(", ", selectQuery);
}
Usage:
var people = context.People.SqlQuery($"SELECT {SelectQuery<Person>()} FROM [People] WHERE [DataXML].value('Properties/Age', 'int') = 21")
.AsQueryable().AsNoTracking();
Related
I am new to C# development and I am trying to write something that can insert a record in a DB. I have a simple test, which I hoped would insert a record into the database when I run it.
Model:
namespace Users.Models;
using System.ComponentModel.DataAnnotations.Schema;
public class User
{
public int Id { get; set; }
public string Name { get; set; }
public string EmailAddress { get; set; }
public string HashedPassword { get; set; }
public DateTime Created { get; set; }
}
Test:
namespace Database.Tests;
using Users.Models;
using Xunit;
public class ReferrerTests
{
[Fact]
public void TestInsert()
{
User user = new()
{
Name = "Bob",
EmailAddress = "bob#email.com",
HashedPassword = "hgfj",
};
using MyDbContext ctx = new();
ctx.Users.Add(user);
}
}
Database context:
namespace Database;
using Users.Models;
using System.Data.Entity;
using System.Data.Entity.Infrastructure;
using Npgsql;
[DbConfigurationType(typeof(Config))]
[SuppressDbSetInitialization]
public class MyDbContext: DbContext
{
public MyDbContext(): base(MakeConnString()) {}
private static string MakeConnString()
{
// Will be moving these to a common location
string OptEnv(string key, string default_) =>
Environment.GetEnvironmentVariable(key) ?? default_;
string Env(string key) =>
Environment.GetEnvironmentVariable(key) ?? throw new MissingFieldException(key);
NpgsqlConnectionStringBuilder builder = new()
{
Host = Env("PGHOST"),
Port = int.Parse(OptEnv("PGPORT", "5432")),
SslMode = Enum.Parse<SslMode>(OptEnv("PGSSLMODE", "Require")),
TrustServerCertificate = true,
Database = OptEnv("PGDATABASE", "postgres"),
Username = OptEnv("PGUSER", "postgres"),
Password = Env("PGPASSWORD")
};
return builder.ConnectionString;
}
public DbSet<User> Users { get; set; }
}
When running this code I get:
System.NullReferenceException: Object reference not set to an instance of an object.
I think I must have something that is preventing the mapping to my database, but I have been unable to figure it out.
EDIT
I think it's probably important I show the DDL of the table as well:
create table public.user
(
id integer generated always as identity primary key,
name text not null
constraint user_name_check
check (length(name) > 0),
email_address text not null unique
constraint user_email_address_check
check (email_address ~* '^.+#.+\..+$'),
-- Ideally use something like
-- https://www.postgresql.org/docs/13/pgcrypto.html
hash_password text not null
constraint user_password_hash_check
check (length(password_hash) > 0),
created timestamp with time zone default CURRENT_TIMESTAMP not null
constraint user_created_check
check (created <= CURRENT_TIMESTAMP)
);
alter table public."user"
owner to postgres;
EDIT 2:
Suggestions to use annotations to try to get the model to map directly to the DDL - still gives the same error, but this is our new model.
namespace Users.Models;
using System.ComponentModel.DataAnnotations.Schema;
[Table("user", Schema="public")]
public class User
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
[Column("id")]
public int Id { get; set; }
[Column("name")]
public string Name { get; set; }
[Column("email_address")]
public string EmailAddress { get; set; }
[Column("hash_password")]
public string HashedPassword { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[Column("created")]
public DateTime Created { get; set; }
}
I don't know about your DB but your model requires the Id column to have a value (it's not nullable) So you need to proivde a value in order to do that.
If your Id column type is Serial on the DB side, just decorate your Id column with :
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
Following #DubDub advice, removing the following line fixed it:
[SuppressDbSetInitialization]
Currently, I am using ServiceStack.Aws v5.9.0 to communicate with DynamoDB. I have used PutItem for both creating and updating an item without anticipating data loss in case of concurrency handling.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
}
public class CustomerDynamo
{
private readonly IPocoDynamo db;
//Constructor
public CustomerDynamo()
{
var dynamoClient = new AmazonDynamoDBClient(_region);
var entityType = typeof(Customer);
var tableName = entityType.Name;
entityType.AddAttributes(new AliasAttribute(name: tableName));
db = new PocoDynamo(dynamoClient) { ConsistentRead = true }.RegisterTable(tableType: entityType);
}
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
db.PutItem(customer);
return customer;
}
}
The above Update method is called in every service/async task that needs to update the data of the customer.
Refer to this article of AWS I decided to implement the Optimistic Locking to save my life from the issue of concurrency requests.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.VersionSupport.html
Assume that the VersionNumber will be the key for Optimistic Locking. So I added the VersionNumber into the Customer model.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
[DynamoDBVersion]
public int? VersionNumber { get; set; }
}
The result is VersionNumber not updated while it should be automatically incremented. I think it is just because the PutItem will override the whole existing item. Is this correct?
I think I need to change from PutItem to UpdateItem in the Update method. The question is how can I generate the expression dynamically to be used with the UpdateItem?
Thanks in advance for any help!
Updates:
Thanks #mythz for the useful information about DynamoDBVersion attribute. Then I tried to remove the DynamoDBVersion and using the UpdateExpression of PocoDynamo as below
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
var expression = db.UpdateExpression<Customer>(customer.CustomerId).Set(() => customer);
expression.ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#Version", "VersionNumber" }
};
expression.ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":incr", new AttributeValue { N = "1" } },
{ ":zero", new AttributeValue { N = "0" } }
};
expression.UpdateExpression = "SET #Version = if_not_exists(#Version, :zero) + :incr";
if (customer.VersionNumber.HasValue)
{
expression.Condition(c => c.VersionNumber == customer.VersionNumber);
}
var success = db.UpdateItem(expression);
}
But the changes are not saved except the VersionNumber
The [DynamoDBVersion] is an AWS Object Persistence Model attribute for usage with AWS's DynamoDBContext not for PocoDynamo. i.e. the only [DynamoDB*] attributes PocoDynamo utilizes are [DynamoDBHashKey] and [DynamoDBRangeKey] all other [DynamoDB*] attributes are intended for AWS's Object Persistence Model libraries.
When needed you can access AWS's IAmazonDynamoDB with:
var db = new PocoDynamo(awsDb);
var awsDb = db.DynamoDb;
Here are docs on PocoDynamo's UpdateItem APIs that may be relevant.
using SQLite library, I can create a class like
namespace MyApp
{
[Table("mydatatable")]
public class MyData
{
[PrimaryKey, AutoIncrement, Column("_id")]
public int ID { get; set; }
[Unique, NotNull]
public Guid GUID { get; set; }
[MaxLength(256), NotNull]
public string Name { get; set; }
[NotNull]
public DateTime Created { get; set; }
public int MyNumber { get; set; }
}
}
Then I can create a Database class like
namespace MyApp
{
public class SQLiteDatabase
{
protected SQLiteConnection conn;
public SQLiteDatabase()
{
}
public void Attach(string dbName)
{
conn = new SQLiteConnection(dbName);
}
public void CreateTable<T>()
{
conn.CreateTable<T>();
}
public int Insert(Object T)
{
return conn.Insert(T);
}
}
}
This is all wonderful and makes using SQLite very easy. But how could I write my own code to do something different? For example, rather than write the data to SQLite database, let's say I wanted to convert the MyApp instance and send it over the nework.
namespace MyApp
{
public class MyDatabaseHandler
{
public void CreateTable<T>()
{
// How do I get the table name "mydatatable"?
// How do I get the column names and types?
// Once I get that information, I can send a POST to my server and create a table on the backend
}
public int Insert(Object T)
{
// How do I get the table name "mydatatable"?
// How do I get the column names and values of each data member?
// Once I get that information, I can send a POST to my server and insert the record.
}
}
}
https://msdn.microsoft.com/en-us/library/ms973893.aspx?f=255&MSPPError=-2147217396
Look into Serialization in .Net
You can convert your classes down to XML files and then pass the XML file across the network and load them at the new location into your class.
Additionally you could look into using TCP but regardless you're going to have to convert your data to some type of serialized class.
I think you need to use reflection. Try this code:
public class MyDatabaseHandler
{
public void CreateTable<T>()
{
// How do I get the table name "mydatatable"?
var type = typeof(T);
var tableAttribute = (TableAttribute)type.GetCustomAttributes(typeof(TableAttribute), inherit: false).FirstOrDefault();
var tableName = tableAttribute?.Name;
// How do I get the column names and types?
var columns = type.GetProperties()
.Select(p => new {type = p.PropertyType, name = p.Name })
.ToArray();
}
public int Insert(Object T)
{
var type = T.GetType();
// see previous method
}
}
Say I have the following model:
[Table("Record")]
public class RecordModel
{
[Key]
[DatabaseGeneratedAttribute(DatabaseGeneratedOption.Identity)]
[Display(Name = "Record Id")]
public int RecordId { get; set; }
[StringLength(150)]
public string Name { get; set; }
[Required]
[StringLength(15)]
public string IMEI { get; set; }
}
Is it possible to add an index to the IMEI column through using an attribute, data annotation, or something from the model?
EF Core 5
In EF Core 5, the Index attribute should be placed on the class.
See: MSDN
[Index(nameof(Url))]
public class Post
{
public int PostId { get; set; }
public string Url { get; set; }
public string Title { get; set; }
public DateTime PublishedOn { get; set; }
}
or revert to the fluent syntax for more advanced option:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Post>()
.HasIndex(p => p.Url)
.IncludeProperties(
p => new { p.Title, p.PublishedOn });
}
EF 6.1
Since the release of EF 6.1. (March 17th, 2014) there is indeed an [Index] attribute available.
Functionality as:
[Index("IMEIIndex", IsUnique = true)]
public string IMEI { get; set; }
comes out of the box.
PS: other properties are Order and IsClustered.
According to this link: http://blogs.msdn.com/b/adonet/archive/2014/02/11/ef-6-1-0-beta-1-available.aspx
It will be available in EF 6.1 as a standard DataAnnotation attribute.
IndexAttribute allows indexes to be specified by placing an [Index] attribute on a property (or properties) in your Code First model. Code First will then create a corresponding index in the database.
According to this link: Creating Indexes via Data Annotations with Entity Framework 5.0
you should write some kind of extension code:
using System;
[AttributeUsage(AttributeTargets.Property, Inherited = false, AllowMultiple = true)]
public class IndexAttribute : Attribute
{
public IndexAttribute(string name, bool unique = false)
{
this.Name = name;
this.IsUnique = unique;
}
public string Name { get; private set; }
public bool IsUnique { get; private set; }
}
and the second class:
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Linq;
using System.Reflection;
public class IndexInitializer<T> : IDatabaseInitializer<T> where T : DbContext
{
private const string CreateIndexQueryTemplate = "CREATE {unique} INDEX {indexName} ON {tableName} ({columnName})";
public void InitializeDatabase(T context)
{
const BindingFlags PublicInstance = BindingFlags.Public | BindingFlags.Instance;
foreach (var dataSetProperty in typeof(T).GetProperties(PublicInstance).Where(
p => p.PropertyType.Name == typeof(DbSet<>).Name))
{
var entityType = dataSetProperty.PropertyType.GetGenericArguments().Single();
TableAttribute[] tableAttributes = (TableAttribute[])entityType.GetCustomAttributes(typeof(TableAttribute), false);
foreach (var property in entityType.GetProperties(PublicInstance))
{
IndexAttribute[] indexAttributes = (IndexAttribute[])property.GetCustomAttributes(typeof(IndexAttribute), false);
NotMappedAttribute[] notMappedAttributes = (NotMappedAttribute[])property.GetCustomAttributes(typeof(NotMappedAttribute), false);
if (indexAttributes.Length > 0 && notMappedAttributes.Length == 0)
{
ColumnAttribute[] columnAttributes = (ColumnAttribute[])property.GetCustomAttributes(typeof(ColumnAttribute), false);
foreach (var indexAttribute in indexAttributes)
{
string indexName = indexAttribute.Name;
string tableName = tableAttributes.Length != 0 ? tableAttributes[0].Name : dataSetProperty.Name;
string columnName = columnAttributes.Length != 0 ? columnAttributes[0].Name : property.Name;
string query = CreateIndexQueryTemplate.Replace("{indexName}", indexName)
.Replace("{tableName}", tableName)
.Replace("{columnName}", columnName)
.Replace("{unique}", indexAttribute.IsUnique ? "UNIQUE" : string.Empty);
context.Database.CreateIfNotExists();
context.Database.ExecuteSqlCommand(query);
}
}
}
}
}
}
After it you can use your index this way:
[Required]
[Index("IMEIIndex", unique: true)]
[StringLength(15)]
public string IMEI { get; set; }
I am quite certain that questions like this have been answered a number of times before, but I can't get any of the suggestions to work.
I am building a MVC 4 application with Entity Framework 5, where the entities were generated from existing tables. I have entity classes that look like this:
namespace RebuildingModel
{
using System;
using System.Collections.Generic;
public partial class StandardCodeTable
{
public StandardCodeTable()
{
this.StandardCodeTableTexts = new HashSet<StandardCodeTableText>();
}
public int TableCode { get; set; }
public string RefTableName { get; set; }
public virtual ICollection<StandardCodeTableText> StandardCodeTableTexts { get; set; }
}
}
namespace RebuildingModel
{
using System;
using System.Collections.Generic;
public partial class StandardCodeTableText
{
public int TableCode { get; set; }
public string LanguageCode { get; set; }
public string TextVal { get; set; }
public virtual StandardCodeTable StandardCodeTable { get; set; }
}
}
namespace RebuildingSite.Models
{
public class CodeTableJoined
{
public int TableCode { get; set; }
public string ReferenceTableName { get; set; }
public string LanguageCode { get; set; }
public string TextValue { get; set; }
}
}
I have a DAO that looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace RebuildingModel.Dao
{
public class CodeTableDao
{
public CodeTableDao() { }
public ISet<StandardCodeTableText> GetCode(string refTableName)
{
HashSet<StandardCodeTableText> codes = new HashSet<StandardCodeTableText>();
using (var db = new RebuildingTogetherEntities())
{
db.StandardCodeTableTexts.Include("StandardCodeTables");
var query = from c in db.StandardCodeTableTexts
where c.StandardCodeTable.RefTableName == refTableName
orderby c.TableCode
select c;
foreach (var item in query)
{
codes.Add(item);
}
}
return codes;
}
}
I have a controller that looks like this:
namespace RebuildingSite.Controllers
{
public class CodeTableController : Controller
{
public ActionResult Index(string refTableName)
{
CodeTableDao dao = new CodeTableDao();
ICollection<StandardCodeTableText> codes = dao.GetCode(refTableName);
HashSet<CodeTableJoined> joins = new HashSet<CodeTableJoined>();
foreach (var code in codes)
{
CodeTableJoined join = new CodeTableJoined();
join.TableCode = code.TableCode;
join.LanguageCode = code.LanguageCode;
join.TextValue = code.TextVal;
join.ReferenceTableName = code.StandardCodeTable.RefTableName;
joins.Add(join);
}
ISet<string> refTableNames = dao.GetReferenceTables();
ViewBag.RefTableNames = refTableNames;
return View(joins);
}
}
}
When I run the view attached to the controller, an ObjectDisposedException is thrown at this line, where the relationship is used:
join.ReferenceTableName = code.StandardCodeTable.RefTableName;
This has to be something simple. What am I doing wrong? I have tried adding that Include() call in from the context in many different places, even multiple times.
I've also tried adding an explicit join in the Linq query. I can't get EF to fetch that relationship.
Copying my comment to an answer - Put the include be in the actual query
var query = from c in
db.StandardCodeTableTexts.include("StandardCodeTables"). where
c.StandardCodeTable.RefTableName == refTableName orderby c.TableCode
select c;