I want to make a generalized Data Table to Linq Collection
I am a beggginer so if it's not possible please let me know
public void Something(DataTable dt)
{
var data = from row in dt.AsEnumerable()
select new {
Order = row["Order"].ToString(),
Something = row["Something"].ToString(),
Customer = row["Customer"].ToString(),
Address = row["Address"].ToString()
};
}
That is the code for one table
i want something like this:
public static void convertDatatable(DataTable dt)
{
var results = from myRow in dt.AsEnumerable()
select new
{
foreach(DataColumn column in dt.Columns)
column.ColumnName // linq Variable name
= myRow[column.ColumnName];// linq Variable Value
};
}
I know it doesn't work how i wrote it but is there another way ?
Note: the reason i am doing this is because i can't convert Datatable directly to JSON it serializes it to XMl then sends it as a string containing that xml.
If you want to stay with datatables then there is this, mentioned in another SO: What should I use to serialize a DataTable to JSON in ASP.NET 2.0?, which links to What should I use to serialize a DataTable to JSON in ASP.NET 2.0?.
I highly recommend, however, that you consider moving away from DataTables and DataRows, replacing it instead with an ORM such as Entity Framework (EF Quick Start here) or Linq to Sql - there are others, but since you are a beginner these offer the easiest learning curve; not least because of the full designer support in Visual Studio.
For the standard forms of JSON serialization offered by .Net (e.g. WCFs DataContractSerializer or the Asp.Net JSON serializer) then you need concrete types. The ORM solution will create all your table wrapper types at design-time, giving you a concrete type, potentially, for every table in your database.
As for the idea you've specifically outlined above, it is exceptionally difficult to achieve - because the compiler, in the first example, dynamically generates a type whose members match the names and types of the expressions you use. If you open your compiled code in ILSpy and switch to IL instead of C# you'll see what I mean.
Therefore, to reproduce it dynamically you would need to dynamically emit a class, probably using ILGenerator, doing the same thing; and then dynamically emit the expression tree (using the Expression class' static factory methods) to fill it out; and finally compile and execute it.
I would only look at doing something like that if I literally couldn't do it any other way - I'd be more likely to just write a routine to iterate through each column and write the JSON to a StringBuilder and return that! But if I could use an ORM, then I'd do that instead.
Related
I am currently referencing a generic DataSet via an index and would like to convert this to a strongly-typed DataSet of the table type. The issue is that my index to the DataRow is a variable.
Currently I am doing the following where the History_Column value is pulled from the database.
e.Dr[c.History_Column.ToString()] = Entry;
I would like to define the DataRow ('Dr' in the example) as a type of the table so I can do something similar to the following:
e.Dr.COLUMN_NAME = Entry;
How can I use dynamic variables in this fashion?
Thanks!
You could use a dynamic type to get you part of the way there, but your goal sounds kind of contradictory.
dynamic Dr = new ExpandoObject();
Dr.whatever = 6;
Dr.anything = "asdf";
If you use ExpandoObject with dynamic, you can assign any property.
A strongly-typed dataset is a class in your application that derives from the built-in dataset class. You have to add them to your project in visual studio. Here are some directions for creating strongly typed datasets.
However, I suggest that you take the opportunity to refactor and get away from datasets altogether (if you can). Entity Framework provides strongly typed property accessors for each column, a better "unit-of-work" pattern, a graphical database mapping tool, and it is much easier to use.
I have a class that I need to hydrate from a DataTable object. Usually I do this the manual way. (see code snipit). The DataTable object populated using ADO.NET and TSql. I need to transfer the values from the DataTable into my .NET class. Is there a utility method that will do this for me automagically? So that I can avoid repetitive code like the following?
DriverSummary driver = new DriverSummary();
driver.Id = (int)row["Id"];
driver.UserId = row["UserId"] as string;
driver.Name = row["Name"] as string;
driver.TruckType = row["TruckType"] as string;
summaries.Add(driver);
I know that the Entity Framework is a tool that is supposed to fill this gap. I haven't quite made the jump to Entity Framework. For now I'd like to have a method that is similar to MVC's utility method UpdateModel() Which is lightweight and simple and hydrates a class from a list of form-value pairs by matching up key names with property names.
Such a utility method would save me tons of time!
Like mentioned above I believe AutoMapper can do this now. You could also look at a ValueInjecter. ValueInjecter and DataTable
So here's my dilemma. I'm trying to utilize Dynamic LINQ to parse a search filter for retrieving a set of records from an Azure table. Currently, I'm able to get all records by using a GenericEntity object defined as below:
public class GenericEntity
{
public string PartitionKey { get; set; }
public string RowKey { get; set; }
Dictionary<string, object> properties = new Dictionary<string, object>();
/* "Property" property and indexer property omitted here */
}
I'm able to get this completely populated by utilizing the ReadingEntity event of the TableServiceContext object (called OnReadingGenericEvent). The following code is what actually pulls all the records and hopefully filter (once I get it working).
public IEnumerable<T> GetTableRecords(string tableName, int numRecords, string filter)
{
ServiceContext.IgnoreMissingProperties = true;
ServiceContext.ReadingEntity -= LogType.GenericEntity.OnReadingGenericEntity;
ServiceContext.ReadingEntity += LogType.GenericEntity.OnReadingGenericEntity;
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(c => c);
if (!string.IsNullOrEmpty(filter))
{
result = result.Where(filter);
}
var query = result.Take(numRecords).AsTableServiceQuery<GenericEntity>();
IEnumerable<GenericEntity> res = query.Execute().ToList();
return res;
}
I have TableServiceEntity derived types for all the tables that I have defined, so I can get all properties/types using Reflection. The problem with using the GenericEntity class in the Dynamic LINQ Query for filtering is that the GenericEntity object does NOT have any of the properties that I'm trying to filter by, as they're really just dictionary entries (dynamic query errors out). I can parse out the filter for all the property names of that particular type and wrap
"Property[" + propName + "]"
around each property (found by using a type resolver function and reflection). However, that seems a little... overkill. I'm trying to find a more elegant solution, but since I actually have to provide a type in ServiceContext.CreateQuery<>, it makes it somewhat difficult.
So I guess my ultimate question is this: How can I use dynamic classes or generic types with this construct to be able to utilize dynamic queries for filtering? That way I can just take in the filter from a textbox (such as "item_ID > 1023000") and just have the TableServiceEntity types dynamically generated.
There ARE other ways around this that I can utilize, but I figured since I started using Dynamic LINQ, might as well try Dynamic Classes as well.
Edit: So I've got the dynamic class being generated by the initial select using some reflection, but I'm hitting a roadblock in mapping the types of GenericEntity.Properties into the various associated table record classes (TableServiceEntity derived classes) and their property types. The primary issue is still that I have to initially use a specific datatype to even create the query, so I'm using the GenericEntity type which only contains KV pairs. This is ultimately preventing me from filtering, as I'm not able to do comparison operators (>, <, =, etc.) with object types.
Here's the code I have now to do the mapping into the dynamic class:
var properties = newType./* omitted */.GetProperties(
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.Public);
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", Properties[\"{0}\"] as {0}", reflected.Name)).Substring(2) + ")";
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(newSelect);
Maybe I should just modify the properties.Aggregate method to prefix the "Properties[...]" section with the reflected.PropertyType? So the new select string will be made like:
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", ({1})Properties[\"{0}\"] as {0}", reflected.Name, reflected.PropertyType)).Substring(2) + ")";
Edit 2: So now I've hit quite the roadblock. I can generate the anonymous types for all tables to pull all values I need, but LINQ craps out on my no matter what I do for the filter. I've stated the reason above (no comparison operators on objects), but the issue I've been battling with now is trying to specify a type parameter to the Dynamic LINQ extension method to accept the schema of the new object type. Not much luck there, either... I'll keep you all posted.
I've created a simple System.Refection.Emit based solution to create the class you need at runtime.
http://blog.kloud.com.au/2012/09/30/a-better-dynamic-tableserviceentity/
I have run into exactly the same problem (with almost the same code :-)). I have a suspicion that the ADO.NET classes underneath somehow do not cooperate with dynamic types but haven't found exactly where yet.
So I've found a way to do this, but it's not very pretty...
Since I can't really do what I want within the framework itself, I utilized a concept used within the AzureTableQuery project. I pretty much just have a large C# code string that gets compiled on the fly with the exact object I need. If you look at the code of the AzureTableQuery project, you'll see that a separate library is compiled on the fly for whatever table we have, that goes through and builds all the properties and stuff we need as we query the table. Not the most elegant or lightweight solution, but it works, nevertheless.
Seriously wish there was a better way to do this, but unfortunately it's not as easy as I had hoped. Hopefully someone will be able to learn from this experience and possibly find a better solution, but I have what I need already so I'm done working on it (for now).
i'd like to be able to query any number of databases with different table layouts, return a datatable then use that datatable to build a strongly typed object. Is there anything out there that does this or comes close without having to code for each different table layout?
Thank you in advance!
Zac
You could try subsonic.
It does exactly that, using code generation via T4 templates.
Get more information at http://subsonicproject.com/
Cheers,
André
May be you can use LINQ and return Anonymous Types http://msdn.microsoft.com/en-us/library/bb397696.aspx
eg:
var result = (from itm in list where itm.StateID==2 select new {Name = itm.Name, State=Itm.StateID});
I'm not sure this is exactly what you're looking for, but the datatable base type has a method called GetTypedTableSchema() that returns an XmlSchemaSet object. You may be able to use this as a roadmap to translate a typed datatable into a strongly typed object.
Assuming you're trying to get Intellisense benefits - go the code generation route against your tables.
No. This is not possible.
In case of typed-datasets, If you need typed-safety at run-time, the type needs to be defined at design-time or you are going to use plain datasets/datatables for your database operations.
I am able to dynamically train and create my regression model just fine from a string[] of column names. However, when I try to pass in a dynamic object with the same Parameter names as Dictionary Key Pair properties it throw the error:
System.ArgumentOutOfRangeException: 'Could not find input column '<MyColumn>'' Where <MyColumn> is the first parameter that the model is looking for.
private static void TestSinglePrediction(MLContext mlContext, dynamic ratingDataSample, int actual)
{
ITransformer loadedModel;
using (var stream = new FileStream(_modelPath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
loadedModel = mlContext.Model.Load(stream);
}
var predictionFunction = loadedModel.MakePredictionFunction<dynamic, RatingPrediction>(mlContext);
var prediction = predictionFunction.Predict(ratingDataSample);
Console.WriteLine($"**********************************************************************");
Console.WriteLine($"Predicted rating: {prediction.Rating:0.####}, actual rating: {actual}");
Console.WriteLine($"**********************************************************************");
}
I suspect this is because the dynamic object doesn't contain the [Column] attributes that the standard class object I normally would pass in has.
However, I will eventually have hundreds of columns that are auto generated from transposing SQL queries so manually typing each column isn't a feasible approach for the future.
Is there any way I could perhaps apply the attribute at run time? Or any other way I can generically approach this situation? Thanks!
This is a great question. The dynamic objects don't work at runtime because ML.NET needs something called a SchemaDefinition for the objects that you pass in so that it knowns where to get the columns it expects.
The simplest way to solve your problem would be to define an object holding only the columns you need at scoring-time, annotated with Column attributes, and manually cast your dynamic object at runtime. This has the main advantage that since you do the casting to the scoring object yourself, you can handle missing data cases yourself without the ML.NET runtime throwing. While your SQL query may give you a large assortment of columns, you won't need the majority of these columns for scoring your model, and therefor don't need to account for them in the scoring object; you only have to account for the columns the model expects.
See this sample from the ML.NET Cookbook for an example of how to score a single row. Behind the scenes, ML.NET is taking the class you defined, and using attributes like Column to construct the SchemaDefinition.