What is best approach for data inheritation in elasticsearch? - c#

I have an parent class and two child like these:
public class Parent {
public Guid Id { get; set; }
public string Name { get; set; }
}
public class FirstChild {
public string IdentityCode { get; set; }
}
public class OtherChild {
public string RegistrationCode { get; set; }
}
There is a question: Is it a good approach to store these two inherited classes in the same Index inside ElasticSearch?
I see there is a _type property that is added to my docs after they are stored in DB but it has always "doc" value.
I test this code to fill it but it seems it is not working this way.
await ElasticClient.IndexAsync<FirstChild>(child, m => m.Index(IndexName));
And Also, I found this question on SO for retrieving my entries from DB but it is outdated and the API is changed and no more accessible.
I want to know if it is a good approach to store sibling data in the same index how can I do this properly.

As of ES 6.0, it is not possible anymore to store multiple types inside the same index, i.e. the _type field you're referring to will always be either doc or _doc. In ES 8.0, the _type field will be removed altogether.
However, if it makes sense for your use case, you can still decide to store several types inside a single index using a custom type field that is present in your document.
You should strive to only store in the same index data that share the same (or very similar) mapping, which doesn't seem to be the case for Parent, FirstChild and SecondChild, but if you add a public string type property to your classes you can still do it.

Related

Entity framework adding properties to my class (over my head)

I'm trying to create a datastructure with entity framework to basically store property values of my objects. I want users to add properties to a class at runtime. The properties can be of different datatypes. (string/int/float etc..)
So I thought I needed some tables/classes as defined in the image below.
So my Object class contains a list of properties that are of a type defined in de propertydefinition class.
One hard thing is that values are stored in the table of the datatype of the propertie. (So a conditional foreignKey?)
Please give me some pointers on how to implement this by using Fluent API. Or other ideas on this subject. (I guess I won't be the first ;)
Werner
The EF entity model cannot be changed during Runtime (or at least is not designed for). You could use an infrastructure to store propertyname/propertyvalye with EF but I think is not the right choice (you lose most of the functionalities).
The best choice could be a NoSQL db, ADO.Net or, if only some objects can be personalized and other are fixed you could store the personalizable objects in XML/JSON in a text field.
I found this link
This helped me solve my "Table Per Type" question. I now have:
public abstract class PropertyBase
{
public int PropertyID { get; set; }
public string Name { get; set; }
}
public class TextProperty : PropertyBase
{
public string Value { get; set; }
}
public class IntProperty : PropertyBase
{
public int Value { get; set; }
}
In My Database Context I added:
modelBuilder.Entity<PropertyBase>()
.HasKey(p => p.PropertyID)
.ToTable("Properties");
modelBuilder.Entity<IntProperty>()
.ToTable("IntProperties");
modelBuilder.Entity<TextProperty>()
.ToTable("TextProperties");
The different types of properties (sub classes) are now stored in separate tables. The main abstract class contains all the other info. This worked fine for me.

Entity Framework - code first, remove field without schema change

I am working on a solution that uses the entity framework code first approach:
[XmlRoot(ElementName = "item")]
public class Item
{
[XmlElement("itemId")]
[Index("idx_item_id")]
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public long ItemId{ get; set; }
[XmlElement("otherId")]
[Index("idx_other_id")]
public long OtherId{ get; set; }
[XmlElement("randomStuff")]
public string RandomStuff{ get; set; }
}
And is directly mapped as:
public class DataContext : DbContext
{
public DataContext()
: base("name=DataContext")
{
Database.CommandTimeout = 180;
}
public DbSet<Item> Items{ get; set; }
So basically I have XMLs coming in and these are directly mapped with the model. A HTTP POST comes in, xml is deserialized using the model and then directly inserted to the database. Now I am no longer interested in a field RandomStuff, but I don't want to change the schema, I would simply start inserting NULLs there. Would it make sense to write something like:
private string _randomStuff;
[XmlElement("randomStuff")]
public string RandomStuff
{
get { return _randomStuff; }
set
{
_randomStuff = null;
}
}
Or is there a better way to achieve this?
The best thing of course is to bite the bullet and change the schema. But I know there can be many reasons not to do that (just yet).
A better option (then the null assignment in the setter) is to use a private setter:
public string RandomStuff { get; private set; }
EF can handle private setters and by doing this, EF will never think the property has changed (because it reads a different value than the one it was set with) and fire useless updates. And you will still be able to read old content.
If you don't want to display the content, even if it's still there, I'd replace the mapped property by a new dummy property (again with a private setter) and mark the current property as not mapped.

C# Comparing complex objects returning list of differences

I've been working on a project for a while to parse a list of entries from a csv file and use that data to update a database.
For each entry I create a new user instance that I put in a collection. Now I want to iterate that collection and compare the user entry to the user from the database (if it exists). My question is, how can I compare that user (entry) object to the user (db) object, while returning a list with differences?
For example following classes generated from database:
public class User
{
public int ID { get; set; }
public string EmployeeNumber { get; set; }
public string UserName { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public Nullable<int> OfficeID { get; set; }
public virtual Office Office { get; set; }
}
public class Office
{
public int ID { get; set; }
public string Code { get; set; }
public virtual ICollection<User> Users { get; set; }
}
To save some queries to the database, I only fill the properties that I can retrieve from the csv file, so the ID's (for example) are not available for the equality check.
Is there any way to compare these objects without defining a rule for each property and returning a list of properties that are modified? I know this question seems similar to some earlier posts. I've read a lot of them but as I'm rather inexperienced at programming, I'd appreciate some advice.
From what I've gathered from what I've read, should I be combining 'comparing properties generically' with 'ignoring properties using data annotations' and 'returning a list of CompareResults'?
There are several approaches that you can solve this:
Approach #1 is to create separate DTO-style classes for the contents of the CSV files. Though this involves creating new classes with a lot of similar fields, it decouples the CSV file format from your database and gives you the ability to change them later without influencing the other part. In order to implement the comparison, you could create a Comparer class. As long as the classes are almost identical, the comparison can get all the properties from the DTO class and implement the comparison dynamically (e.g. by creating and evaluating a Lambda expression that contains a BinaryExpression of type Equal).
Approach #2 avoids the DTOs, but uses attributes to mark the properties that are part of the comparison. You'd need to create a custom attribute that you assign to the properties in question. In the compare, you analyze all the properties of the class and filter out the ones that are marked with the attribute. For the comparison of the properties you can use the same approach as in #1. Downside of this approach is that you couple the comparison logic tightly with the data classes. If you'd need to implement several different comparisons, you'd clutter the data classes with the attributes.
Of course, #1 results in a higher effort than #2. I understand that it is not what you are looking for, but maybe having a separate, strongly-typed compared class is also an approach one can think about.
Some more details on a dynamic comparison algorithm: it is based on reflection to get the properties that need to be compared (depending on the approach you get the properties of the DTO or the relevant ones of the data class). Once you have the properties (in case of DTOs, the properties should have the same name and data type), you can create a LamdaExpression and compile and evaluate it dynamically. The following lines show an excerpt of a code sample:
public static bool AreEqual<TDTO, TDATA>(TDTO dto, TDATA data)
{
foreach(var prop in typeof(TDTO).GetProperties())
{
var dataProp = typeof(TDATA).GetProperty(prop.Name);
if (dataProp == null)
throw new InvalidOperationException(string.Format("Property {0} is missing in data class.", prop.Name));
var compExpr = GetComparisonExpression(prop, dataProp);
var del = compExpr.Compile();
if (!(bool)del.DynamicInvoke(dto, data))
return false;
}
return true;
}
private static LambdaExpression GetComparisonExpression(PropertyInfo dtoProp, PropertyInfo dataProp)
{
var dtoParam = Expression.Parameter(dtoProp.DeclaringType, "dto");
var dataParam = Expression.Parameter(dataProp.DeclaringType, "data");
return Expression.Lambda(
Expression.MakeBinary(ExpressionType.Equal,
Expression.MakeMemberAccess(
dtoParam, dtoProp),
Expression.MakeMemberAccess(
dataParam, dataProp)), dtoParam, dataParam);
}
For the full sample, see this link. Please note that this dynamic approach is just an easy implementation that leaves room for improvement (e.g. there is no check for the data type of the properties). It also does only check for equality and does not collect the properties that are not equal; but that should be easy to transfer.
While the dynamic approach is easy to implement, the risk for runtime errors is bigger than in a strongly-typed approach.

An alternative lookup table approach needed to make C# models more generic

I currently have the following Models in my EF Code First MVC project (edited for brevity):
public class Car
{
public int Id { get; set; }
public string Descrip { get; set; }
// Navigation Property.
public virtual CarColour CarColour { get; set; }
... + numerous other navigation properties.
}
public class CarColour
{
public int Id { get; set; }
public string ColourName { get; set; }
}
The CarColour table in the DB contains many rows.
In my project, I have about 10 of these sorts of tables, which are essentially lookup tables.
Rather than have 10 lookup tables (and 10 corresponding 'hard' types in code), I was tasked with implementing a more re-usable approach, instead of having loads of lookup tables, specific to Car (in this example), along the lines of having a couple of tables, one of which may hold the item types (colour, fuel-type etc.) and one which contains the various values for each of the types. The idea being that our model will be able to be re-used by many other projects - some of which will have potentially hundreds of different attributes, and as such, we won't want to create a new Class/Type in code and generate a new lookup table for each.
I am having difficulty in understanding the c# implementation of this sort of approach and hope someone may be able to give me an example of how this can be achieved in code, more specifically, how the above models would need to change, and what additional classes would be required to accomplish this?
your base entity must implement INotifyPropertyChanged and make it generic:
public virtual CarColour CarColour {
Get { return this.carColour; }
Set {
this.Carcolour; = value
OnPropertyChanged("CarColour");
}
}
For more info see :
patterns & practices: Prism in CodePlex.
http://compositewpf.codeplex.com/wikipage?title=Model%20View%20ViewModel%20(MVVM)
Greetings
Bassam
This is not necessarily specific to EF but I've been down this road and didn't really enjoy it.
I wanted to use a single table to represent 'generic' information and while I thought it was smart, it soon showed it's limitations. One of them being the complexity you need to introduce when writing queries to extract this data if you're performing more than just 'get colours for this car'.
I'd say, if your data is simple key/value and the value type is always going to be the same then go for it, it might even be worth having this a mere 'meta-data' for an object:
public class Car
{
public int Id { get; set; }
public string Descrip { get; set; }
public MetaData CarColours { get; set; }
}
public MetaData : Dictionary<int, string>
{
public MetaData(int group){}
}
Hypothetical table:
TableMetaData(int metaGroup, int metaId, string metaValue)
If you're hoping to store different types as your value and may need to perform joining on this data - avoid it and be a bit more specific.

How to make a Attribute aware of the Name of the Proprty it is on?

I want to implement a simple attribute that is used to map Database Columns to Properties.
So what i have so far is something that attached like so:
[DataField("ID")]
public int ID { get; set; }
[DataField("Name")]
public String Name { get; set; }
[DataField("BirD8")]
public DateTime BirthDay { get; set; }
Is there a way that I can make the attribute "aware" of the field it is on, so that for the properties where the name is the same as the ColumnName I can just apply the attribute without the name parameter, or would I have to deal with that at the point where I reflect the properties. I want to end up doing just this:
[DataField]
public int ID { get; set; }
[DataField]
public String Name { get; set; }
[DataField("BirD8")]
public DateTime BirthDay { get; set; }
The attribute itself won't be aware of what it's applied to, but the code processing the attributes is likely to be running through PropertyInfo values etc and finding the attributes associated with them. That code can then use both the property and the attribute appropriately.
To make things simpler, you might want to write a method on the attribute to allow it to merge its information with the information from the property, so you'd call:
DataFieldAttribute dfa = propertyInfo.GetCustomAttributes(...); // As normal
dfa = dfa.MergeWith(propertyInfo);
Note that for the sake of sanity this should create a new instance of the attribute, rather than changing the existing one. Alternatively, you might want a whole separate class to represent "the information about a data field":
DataFieldAttribute dfa = propertyInfo.GetCustomAttributes(...); // As normal
DataFieldInfo info = dfa.MergeWith(propertyInfo);
That way you could also construct DataFieldInfo objects without any reference to attributes, which might be a nice conceptual separation - allowing you to easily load the config from an XML file or something similar if you wanted to.
If you don't mind using postsharp you can look Here, at a previous question I have asked which was close. I ended up using the compile time validate to do what I wanted, although there are other options, like CompileTimeInitalize.
public override void CompileTimeInitialize(object element)
{
PropertyInfo info = element as PropertyInfo;
//....
}

Categories