I'm in charge to migrate our own DAL to a solution based on Entity Framework 4 but, before I can do it, I need to be sure it's possible to translate all our "constructs" to this new technology.
One of the biggest issues I'm having is the possibility to read a field and build a custom type. Valid examples could be a bit mask saved in a BIGINT field, a list of mail addresses saved as a CSV list in a NVARCHAR field or an XML field containing aggregated data not worth to have their own table/entity. Basically the serialization mechanism is not fixed.
Let's take the classic "Address" example.
public class Address
{
public string Street {get; set;}
public string City {get; set;}
public string Zip {get; set;}
public string Country {get; set;}
}
and let's suppose we want to save it in an XML field using this template:
<address>
<street>Abrahamsbergsvägen, 73</street>
<city>Stockholm</city>
<zip>16830</zip>
<country>Sweden</country>
</address>
The question basically is: does exist a method to override how EF4 serializes and deserializes the content of a field mapped to a property of an entity?
I found this solution. It's not as clean as I wished but it seems it's impossible to get anything better.
given this base entity,
public class Institute
{
public int InstituteID { get; set; }
public string Name { get; set; }
// other properties omitted
}
I added in the database an XML field called Data containing some strings using this simple template
<values>
<value>Value 1</value>
<value>Value 2</value>
<value>Value 3</value>
</values>
In the entity I added these properties and I mapped the database field "Data" to the property "DataRaw".
protected string DataRaw
{
get
{
if (_Data == null)
return _DataRaw;
else
return new XElement("values", from s in Data select new XElement("value", s)).ToString();
}
set
{
_DataRaw = value;
}
}
private string _DataRaw;
private string[] _Data;
public string[] Data
{
get
{
if (_Data == null)
{
_Data = (from elem in XDocument.Parse(_DataRaw).Root.Elements("value")
select elem.Value).ToArray();
}
return _Data;
}
set
{
_Data = value;
}
}
This solution works. Here is the sample code:
class Program
{
static void Main(string[] args)
{
var ctx = new ObjectContext("name=TestEntities");
var institute = ctx.CreateObjectSet<Institute>().First();
System.Console.WriteLine("{0}, {1}", institute.InstituteID, institute.Name);
foreach (string data in institute.Data)
System.Console.WriteLine("\t{0}", data);
institute.Data = new string[] {
"New value 1",
"New value 2",
"New value 3"
};
ctx.SaveChanges();
}
}
Does anyone have a better solution?
Entity Framework does NOT serializes or deserializes entities nor it controls how the serialization should take place in other layers or modules of your application.
What you need to do is to simply open your POCO(s) and annotate their Properties with appropriate attributes that will be taken into account at the time of serialization when you want to ship them to your other application layers.
Related
Let me preface by saying I'm very new to C# development so if the solution seems obvious I apologize.
I'm getting a JSON string back from a user and I need to filter a list of C# objects based on what the JSON string contains. The JSON can only have fields that my C# model has but I don't know what fields the JSON string will contain. My C# model looks something like this:
public class Enrollment {
public int Year { get; set; }
public int NumEnrolls { get; set; }
public int DaysIntoEnrollment { get; set; }
public string Name { get; set; }
}
The JSON will have one or more of these properties with values to filter out. It could look like this:
{
"Year": ["2020", "2019"],
"Name": ["CourseA", "CourseB"],
"DaysIntoEnrollment": "20"
}
I need to filter my list of Enrollment objects based on the above JSON. So I would want the end result to have all Enrollment objects that don't contain a Year of 2020 or 2019 for example.
I've gotten a filter to work with linq on a single property but my real model has much more properties that can be filtered and I'm looking for a compact solution that will work regardless of which properties are included in the JSON. This is what I have working
public void GetFilteredData(string filters) {
var enrollList = new List<Enrollments>(); // Pretend this contains a list of valid Enrollment data
var json = JObject.Parse(filters); // filters string is in the json format from above
var propsToFilter =
from p in json["Year"]
select p;
var filtered = enrollList.Where(e => !propsToFilter.Contains(e.Year.ToString())));
}
Is there a simple way to do this without manually going through each property like I did above?
Let's say we have a realm results taken with
RealmDb.All<Entry>();
Then I want to do some search over those results using not yet supported techniques, like StartsWith on a function return or on a property which is not mapped in realm etc, so I get a subset
IEnumerable<Entry> subset = bgHaystack;
var results = subset.Where(entry => entry.Content.ToLower().StartsWith(needle));
To get somehow these as part of RealmResults, I extract the entry ids like this:
List<int> Ids = new List<int>();
foreach (Entry entry in entries)
{
Ids.Add(entry.Id);
}
return Ids;
and finally I want to return a subset of RealmResults (not IEnumerable) of only those Entries that contain those ids, how can I do that? IDE says the Contains method is not supported.
Can I use some kind of predicate or a comparer for that?
Entry is my model class
using System.ComponentModel.DataAnnotations.Schema;
using Realms;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System;
namespace Data.Models
{
[Table("entry")]
public class Entry : RealmObject
{
public class EntryType
{
public const byte Word = 1;
public const byte Phrase = 2;
public const byte Text = 3;
};
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Indexed]
[Column("type")]
public byte Type { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[NotMapped]
public Phrase Phrase { get; set; }
[NotMapped]
public Word Word { get; set; }
[NotMapped]
public Text Text { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
[NotMapped]
public string Content
{
get {
switch (Type)
{
case EntryType.Phrase:
return Phrase?.Content;
case EntryType.Word:
return Word?.Content;
case EntryType.Text:
return Text?.Content;
}
return "";
}
}
}
}
According to the documentation, Realm .NET supports LINQ, so that's promising. In your specific example, you indicate that StartsWith isn't supported, but I see that on the above page, specifically here.
Now, your example makes clear that Entry is a RealmObject, so it's not clear where you'd possibly get a RealmResult from (nor does their documentation on that page mention a RealmResult). Specifically, the home page indicates that you're really only going to ever work with Realm, RealmObject and Transaction, so I'm going to just assume that you meant that you'll need a resulting RealmObject per their examples.
The way you presently have your data object set up, you're rather stuck calling it like you are (though if I could make a recommendation to simplify it a little bit:
var entries = RealmDb.All<Entry>().ToList();
var results = entries.Where(entry => entry.Content.ToLower().StartsWith(needle));
var ids = results.Select(a => a.Id).ToList();
Now, your big issue with just combining the filter predicate in line 2 with the end of line 1: Content itself is marked with a [NotMapped] attribute. Per the documentation again:
As a general rule, you can only create predicates with conditions that
rely on data in Realm. Imagine a class
class Person : RealmObject
{
// Persisted properties
public string FirstName { get; set; }
public string LastName { get; set; }
// Non-persisted property
public string FullName => FirstName + " " + LastName;
}
Given this class, you can create queries with conditions that apply to
the FirstName and LastName properties but not to the FullName
property. Likewise, properties with the [Ignored] attribute cannot be
used.
Because you're using [NotMapped], I've got to believe that's going to behave similarly to [Ignored] and further, because it's just a computed value, it's not something that Realm is going to be able to process as part of the query - it simply doesn't know it because you didn't map it to the information Realm is storing. Rather, you'll have to compute the Content property when you've actually got the instances of your Entry objects to enumerate through.
Similarly, I expect you'll have issues pulling values from Phrase, Word and Text since they're also not mapped, and thus not stored in the record within Realm (unless you're populating those in code you didn't post before executing your Where filter).
As such, you might instead consider storing separate records as a PhraseEntry, WordEntry, and TextEntry so you can indeed perform exactly that filter and execute it on Realm. What if you instead used the following?
public class Entry : RealmObject
{
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[Column("content")]
public string Content { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
}
[Table("wordEntry")]
public class WordEntry : Entry
{
}
[Table("phraseEntry")]
public class PhraseEntry : Entry
{
}
[Table("textEntry")]
public class TextEntry : Entry
{
}
And now, you can offload the filtering to Realm:
var wordEntries = RealmDb.All<WordEntry>.Where(entry =>
entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var phraseEntries = RealmDb.All<PhraseEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var textEntries = RealmDb.All<TextEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var entries = new List<Entry>();
entries.AddRange(wordEntries);
entries.AddRange(phraseEntries);
entries.AddRange(textEntries);
var ids = entries.Select(entry => entry.Id).ToList();
It's not quite as brief as storing it all in one table, but I'm not immediately seeing any Realm documentation that indicates support for executing the same query against multiple tables simultaneously, so at least this would allow you to leave the filtering to the database and work against a more limited subset of values locally.
Finally, so we have all that and I missed your final question up top. You indicate that you want to return a subset of your entries based on some collection of ids you create. In the logic you provide, you're retrieving all the Id properties in all your results, so there's really no further subset to pull.
That said, let's assume you have a separate list of ids that for whatever complicated reason, you were only able to derive after retrieving the list of Entry types from above (themselves all PhraseEntry, WordEntry or TextEntry objects).
At this point, since you've already pulled all the values from Realm and have them locally, just execute another Where statement against them. Because a List implements IEnumerable, you can thus execute the LINQ locally without any of the Realm restrictions:
var myLimitedIdSet = new List<int>()
{
10, 15, 20, 25 //Really complicated logic to narrow these down locally
};
var resultingEntries = entries.Where(entry => myLimitedIdSet.Contains(entry.Id)).ToList();
And you're set. You'll have only those entries that match the IDs listed in myLimitedIdSet.
Edit to address comment
You see this error because of the detail provided at the top of this page in the documentation. Specifically (and adapting to your code):
The first statement gives you a new instance of Entry of a class that implements IQueryable... This is standard LINQ implementation - you get an object representing the query. The query doesn't do anything until you made a further call that needs to iterate or count the results.
Your error is then derived by taking the result from RealmDb.All<Entry>() and trying to cast it to an IEnumerable<Entry> to operate against it as though you have local data. Until you call ToList() onRealmDb.All` you simply have a LINQ representation of what the call will be, not the data itself. As such, when you further refine your results with a Where statement, you're actually adding that to a narrowed version of the IQueryable statement, which will also fail because you lack the appropriate mapping in the Realm dataset.
To skip the optimization I provided above, the following should resolve your issue here:
var bgHaystack = realm.All<Entry>().ToList(); //Now you have local data
var results = bgHaystack.Where(entry => entry.Content.ToLower().StartsWith(needle));
Unfortunately, given your provided code, I don't expect that you'll see any matches here unless needle is an empty string. Not only is your Content property not part of the Realm data and you thus cannot filter on it within Realm, but neither are your Phrase, Word or Text properties mapped either. As a result, you will only ever see an empty string when getting your Content value.
You can further refine the results variable above to yield only those instances with a provided ID as you see fit with normal LINQ (as again, you'll have pulled the data from Realm in the first line).
var limitedIds = new List<int>{10, 20, 30};
var resultsLimitedById = results.Select(a => limitedIds.Contains(a.Id)).ToList();
I've updated my examples above to reflect the use of ToList() in the appropriate places as well.
I've got a small integration service which recieves XML files and parses it.
Also I've created classes from provided XSD for deserializing XML data. During parsing I need to copy properties from those XSD-generated classes to my own that I use in Data Layer. This is an example of my aproach
var supplierInfo = new SupplierInfo();
//coping properties
supplierInfo.Name = supplier.name;
supplierInfo.ShortName = supplier.shortName;
supplierInfo.BrandName = supplier.brandName;
supplierInfo.AdditionalFullName = supplier.additionalFullName;
supplierInfo.AdditionalCode = supplier.additionalCode;
supplierInfo.AdditionalInfo = supplier.additionalInfo;
//lot's of other properties
//...
supplierInfo.Tax = supplier.tax;
supplierInfo.RegistrationDate = supplier.registrationDate;
Some times ammount of properties is very big. Is there more eligant way to copy those properties?
Automapper has been out there since ages ago. Tried and tested. http://automapper.org/
Here's an example:
using System;
using AutoMapper;
public class Program
{
class SupplierInfo
{
public SupplierInfo( string name, string shortName, string brandName ) {
Name = name;
ShortName = shortName;
BrandName = brandName;
}
public string Name {get; private set; }
public string ShortName {get; private set; }
public string BrandName {get; private set; }
}
class Supplier
{
public string name {get; set; }
public string shortName {get; set; }
public string brandName {get; set; }
}
public static void Main()
{
var dto = new Supplier() {
name = "Name 1",
shortName = "Short Name 1",
brandName = "Brand Name 1"
};
//From the tutorial:
//You only need one MapperConfiguration instance typically per AppDomain and should be instantiated during startup.
var config = new MapperConfiguration(cfg => cfg.CreateMap<Supplier, SupplierInfo>());
var mapper = config.CreateMapper();
SupplierInfo info = mapper.Map<SupplierInfo>(dto);
Console.WriteLine( info.Name );
Console.WriteLine( info.ShortName );
Console.WriteLine( info.BrandName );
}
}
The official Getting Started guide can be found at https://github.com/AutoMapper/AutoMapper/wiki/Getting-started
I am happy to be corrected on this but I always find automapper (as per the other answer), which maps property values by name/convention, a little scary to use in production code.
I don't really have a decent alternative but I prefer to do it manually as per your code sample - it's easier to read and debug and if you end up renaming any properties in a class, it will be clear that the copying code is broken (or if you use some IDE tool to rename the property, it'll change the copy code accordingly).
First, install EntityLite.Core:
PM> Install-Package EntityLite.Core
Then use it:
using inercya.EntityLite.Extensions;
...
supplierInfo.AssignPropertiesFrom(supplier);
EntityLite is a micro ORM I developed. It has some little gems :-)
EDIT:
I guess you may not want to install EntityLite.Core just to copy some properties from an object to another. So here you have an implementation of AssignPropertiesFrom extension method that uses Reflection:
public static class ObjectExtensions
{
public static void AssignPropertiesForm(this object target, object source)
{
if (target == null || source == null) throw new ArgumentNullException();
var targetPropertiesDic = target.GetType().GetProperties().Where(p => p.CanWrite).ToDictionary(p => p.Name, StringComparer.CurrentCultureIgnoreCase);
foreach (var sourceProp in source.GetType().GetProperties().Where(p => p.CanRead))
{
PropertyInfo targetProp;
if (targetPropertiesDic.TryGetValue(sourceProp.Name, out targetProp))
{
targetProp.SetValue(target, sourceProp.GetValue(source, null), null);
}
}
}
}
Incidentally, this is not the EntityLite implementation. EntityLite uses dynamic IL generation.
I have a simple database with two tables: Photo and Tag. There is a one-to-many (a photo can have many tags) relationship between the two tables. Here is a diagram:
Now I have made a Photo class and set it up for LINQ-to-SQL using attributes. The code for that class is below:
[Table]
public class Photo
{
[Column(IsDbGenerated = true, IsPrimaryKey = true, CanBeNull = false)]
public int ID { get; set; }
[Column(CanBeNull = false)]
public string Filename { get; set; }
[Column(CanBeNull = false)]
public string Description { get; set; }
[Column(CanBeNull = false)]
public DateTime DateTaken { get; set; }
public List<string> Tags { get; set; }
public override string ToString()
{
string result = String.Format("File: {0}, Desc: {1}, Date: {2}, Tags: ",
Filename, Description, DateTaken);
if (Tags != null)
foreach (string tag in Tags)
result += tag + ", ";
return result;
}
}
You will notice that currently I do not have any attributes for the Tags list. I would like to be able to setup the attributes (associations) for the Tags list so that it would be populated with Name field of the Tag table for all entries in the Tag table of a particular PhotoID. It would be preferable if I could do this directly (i.e. without having to setup a Tag class mimicking/relating to the Tag table). Since I'm only interested in one field (the Name in the Tag table) rather than many fields, I would think there is a way to do this.
Is this possible, and if so how would I further decorate the class with attributes and what would be the syntax for a simiple Select query using LINQ-to-SQL?
If it helps, here is the code I am using to simply add a new photo and then grab all of the photos out of the database (obviously the tag information is not pulled out as the code stands now).
DataContext context = new DataContext(connectionString);
// add new photo
Photo newPhoto = new Photo { Filename = "MyImage1.jpg", Description = "Me", DateTaken = DateTime.Now };
context.GetTable<Photo>().InsertOnSubmit(newPhoto);
context.SubmitChanges();
// print out all photos
var photoQuery = from m in context.GetTable<Photo>() select m;
foreach (Photo myPhoto in photoQuery)
textBox1.Text += Environment.NewLine + myPhoto.ToString();
First I'd suggest you to use a tool to generate your entity classes (the classes that correspond to the database tables). We'r using sqlmetal and it does the job very well.
Next, (if you have a Tag entity) than write a function that fetches the tags for some photos:
void GetTags(IEnumerable<Photo> photos)
{
var ids = photos.Select(p=>p.ID).ToList();
var tagsG = (from tag in context.GetTable<Tag>() where ids.Contains(tag.PhotoID) select new {PhotoID, Name}).GroupBy(tag=>tag.PhotoID);
foreach(ph in photos){
ph.Tags = tagsG[ph.ID].Select(tag=>tag.Name).ToList();
}
}
Note, the code might not compile I've written it in the browser...
You should refer to the Attribute Based Mapping article on msdn.
Also, this article shows how to decorate an EntitySet property with an Association attribute to accomplish the relationship modeling.
It would be preferable if I could do this directly (i.e. without having to setup a Tag class mimicking/relating to the Tag table).
Not possible. LinqToSql needs to know what is or isn't a table, so it can generate the proper sql text.
What you can do instead, is make one set of types for representing database structure, and another set of types for use elsewhere in your program. The second set of types could have a single class representing Photo and Tag data.
Write a query with the first set of types, then use them to construct instances of the second set.
I'm building this app at night after work and have been struggling with this design problem for a week or two now.
I'm building a program that has 44 different types of entries and requires the ability to create a custom type.
Because users might change the fields in a particular type of entry and/or define their own, my first approach of generating an Entity class for each type of entry doesn't seem workable. If users change any of the fields or their version of the schema (subject to validation, of course) then my classes wouldn't really reflect that.
Even if did not allow users to change fields, I want to make sure that data schema changes do not create problems for current data.
In order to build a schema capable of all of this, I have done the following:
dtype
id
datatype
field
id fieldName
fieldDataType (linked by foreign key to dtype)
dataStore
id
dataText
dataString
dataDate
dataDouble
dataInt
fieldID (linked by foreign key to field)
entryID (linked by foreign key to id field of entries)
types
ol>id int
typeName
fields
entries
id
typeid (linked by foreign key to id of types)
Well, the schema is very denormalized but difficult to work with in ASP.NET MVC.
My second crack at it involved making a class with typed properties to store whichever datatype the entry happened to be.
Custom Class for domain level
public class entry
{
public List dataList = new List();
public entry(int id)
{
kleraDataContext s = new kleraDataContext();
var dataSet = s.dataStores.Where(c => c.citationID == id);
foreach (dataStore y in dataSet)
{
dataBlob tempd = new dataBlob();
//We get the data type id
var temp = s.fields.Where(c => c.id == y.fieldID).First();
//Get the fieldname and store the data type id for later
tempd.fieldname = temp.fieldName;
tempd.dataType = temp.dtype.dataType;
switch (tempd.dataType)
{
case "String":
tempd.dString = y.dataString;
break;
case "Text":
tempd.dString = y.dataText;
break;
default:
//Nothing
break;
}
this.dataList.Add(tempd);
}
}
}
public class dataBlob
{
private string _dString;
private DateTime _dDate;
private int _dInt;
private double _dDouble;
private object _data;
private string _fieldname;
private string _dataType;
public string dataType
{
get
{
return _dataType;
}
set
{
_dataType = value;
}
}
public object data
{
get
{
return _data;
}
}
public string dString
{
get
{
return _dString;
}
set
{
_dString = value;
_data = value;
}
}
public string fieldname
{
get
{
return _fieldname;
}
set
{
_fieldname = value;
}
}
public DateTime dDate
{
get
{
return _dDate;
}
set
{
_dDate = value;
_data = value;
}
}
public double dDouble
{
get
{
return _dDouble;
}
set
{
_dDouble = value;
_data = value;
}
}
public int dInt
{
get
{
return _dInt;
}
set
{
_dInt = value;
_data = value;
}
}
}
}
Note several problems with this.
I'm having trouble getting a generic enough property to store the data regardless of what field type it is in the physical structure. Ideally, data's accessor would just retrieve whatever the datatype happened to be.
I still don't have a good enough way to provide ASP.NET MVC's views with a coherent enough model so that the presentation code does not have to do parsing. Ideally, the view would just get an object with a list of fields and with their corresponding data.
Related to #2, I can't seem to figure out an appropriate way of persisting changes. Writing a query and having it return the fields to the view could be done. Because each field is not a strongly typed accessor, I'm not sure how to persist the change from the view to the model. Naively, I've thought of inserting a key in a hidden span and using a Dictionary object in the controller to map edits/creation.
Thoughts?
Ron
While I am not exactly sure of your ultimate goal, I may have an option for you. You need a highly dynamic "entity" which will allow your users to create their own data structures. Imperative languages like C# do not lend themselves well to such a thing...and even with a dynamic language, I think you'll likely run into some difficulties. However, XML is an excellent way to represent dynamic data structures in an ad-hoc, runtime-creatable way.
If you are using SQL Server, I recommend you create a simpler type, as depicted below, and store it in a table that uses the 'xml' data type for one of the columns:
public class DynamicEntity
{
public int ID { get; set; }
public string TypeName { get; set; }
public XDocument DynamicContent { get; set; }
}
The above entity could be stored in the following table:
CREATE TABLE DynamicEntity
(
ID int IDENTITY(1,1) NOT NULL,
NAME varchar(50) NOT NULL,
DynamicContent xml NULL
)
Given SQL Server's xml capabilities, you will still be able to query the data in that xml column. Not only that, if you want your users custom structures to be validated against a schema, you could also put that schema in the database and 'type' your xml column against that schema. Using an XML column in SQL Server does come with come caveats, but it might be a simple solution to your otherwise complicated problem.