Im struggling with a following issue:
I have a datatable that contains information retrieved from an SQL Server database. The information retrieved is regarding the price of certain products. So, for example, the datatable has the price of a specific product on an specific price list. One product may be present in several different price list, and today on the datatable, each combination is a different row. For example:
I need to transform the datatable into a datatable that only has one row per product, and the price list information is on columns:
EXAMPLE DATA:
A few notes:
I dont know who many price lists i will have, so i could have the column "cost" (for ex) N amount of times.
The product information (first part) needs to be included.
No more than one row per id_Art.
I been working on the following, but i wanted to step back because i might be going into a rabbit hole and there might be an easier solution.
Currently, i'm creating new datatables filtered by each pricelist. The idea behind this was to join the datatables, but i got stucked.
foreach(var pricelist in pricelists)
{
DataSet dataSetsingle = new DataSet();
dataSetsingle = GetDataSetForMultiTariff(tarifa, dpto, secc, fam, subfam); //This will return a datatable filtered by pricelist
System.Data.DataTable dtnew = dataSetsingle.Tables[0];
var results = from table1 in dtinitial.AsEnumerable()
join table2 in dtnew.AsEnumerable()
on new { A = table1["ID_ART"], B = table1["ID_FORMAT"] } equals new { A = table2["ID_ART"], B = table2["ID_FORMAT"] }
select new
{
table1,table2
};
}
Should i keep moving forward through this approach? I dont get a flattend result the way i'm doing it, and i'm not sure how to solve that.
I have access to the database, so i could potentially change the query.
Pivot tables could work?
Thanks a lot!
If you want to continue with the LINQ join over DataTables, this extension method can help, but I think you would be better off with a pivot. Unfortunately I can't tell from your question what you want to pivot, but I do have a pivot method for a DataTable as well.
public static class DataTableExt {
// ***
// *** T Extensions
// ***
public static IEnumerable<T> AsSingleton<T>(this T first) {
yield return first;
}
// ***
// *** MemberInfo Extensions
// ***
public static Type GetMemberType(this MemberInfo member) {
switch (member) {
case FieldInfo mfi:
return mfi.FieldType;
case PropertyInfo mpi:
return mpi.PropertyType;
case EventInfo mei:
return mei.EventHandlerType;
default:
throw new ArgumentException("MemberInfo must be if type FieldInfo, PropertyInfo or EventInfo", nameof(member));
}
}
public static object GetValue(this MemberInfo member, object srcObject) {
switch (member) {
case FieldInfo mfi:
return mfi.GetValue(srcObject);
case PropertyInfo mpi:
return mpi.GetValue(srcObject);
case MethodInfo mi:
return mi.Invoke(srcObject, null);
default:
throw new ArgumentException("MemberInfo must be of type FieldInfo, PropertyInfo or MethodInfo", nameof(member));
}
}
public static T GetValue<T>(this MemberInfo member, object srcObject) => (T)member.GetValue(srcObject);
// ***
// *** Type Extensions
// ***
public static List<MemberInfo> GetPropertiesOrFields(this Type t, BindingFlags bf = BindingFlags.Public | BindingFlags.Instance) =>
t.GetMembers(bf).Where(mi => mi.MemberType == MemberTypes.Field || mi.MemberType == MemberTypes.Property).ToList();
// ***
// *** DataTable Extensions
// ***
public static IEnumerable<DataColumn> DataColumns(this DataTable aTable) => aTable.Columns.Cast<DataColumn>();
public static IEnumerable<string> ColumnNames(this DataTable aTable) => aTable.DataColumns().Select(dc => dc.ColumnName);
// Create new DataTable from LINQ join results on DataTable
// Expect T to be anonymous object of form new { [DataRow or other] d1, [DataRow or other] d2, ... }
public static DataTable FlattenToDataTable<T>(this IEnumerable<T> src) {
var res = new DataTable();
if (src.Any()) {
var firstRow = src.First();
var memberInfos = typeof(T).GetPropertiesOrFields();
var allDC = memberInfos.SelectMany(mi => (mi.GetMemberType() == typeof(DataRow)) ? mi.GetValue<DataRow>(firstRow).Table.DataColumns() : new DataColumn(mi.Name, mi.GetMemberType()).AsSingleton());
foreach (var dc in allDC) {
var newColumnName = dc.ColumnName;
if (res.ColumnNames().Contains(newColumnName)) {
var suffixNumber = 1;
while (res.ColumnNames().Contains($"{newColumnName}.{suffixNumber}"))
++suffixNumber;
newColumnName = $"{newColumnName}.{suffixNumber}";
}
res.Columns.Add(new DataColumn(newColumnName, dc.DataType));
}
foreach (var objRows in src)
res.Rows.Add(memberInfos.SelectMany(mi => (mi.GetMemberType() == typeof(DataRow)) ? mi.GetValue<DataRow>(objRows).ItemArray : mi.GetValue(objRows).AsSingleton()).ToArray());
}
return res;
}
}
Hmmm... didn't realize how many extension methods that code used :)
Related
I am getting following error while executing simple select statement in my custom code.
Could not find specified column in results
Here is my code-
string queryBuilder="select BASKET_DESCRIPTION from MARKET_BASKET_REQUESTS order by BASKET_DESCRIPTION limit 1";
public T SelectSingle<T>(string queryBuilder) where T : new()//made new
{
T result = new T();
TableScheme dbTable = GetTableSchemeFromType(typeof(T));
IDataReader reader = ExecuteReader(queryBuilder);
result = ParseDataReaderToEntityListtttt<T>(reader,dbTable);
reader.Close();
return result;
}
private T ParseDataReaderToEntityListtttt<T>(IDataReader reader, TableScheme dbTable) where T : new()
{
Type type = typeof(T);
T result = new T();
while (reader.Read())
{
T t = new T();
foreach (var column in dbTable.Columns)
{
type.GetProperty(column.AssociatedPropertyName).SetValue(t, reader[column.ColumnName], null);
}
result = t;
}
return result;
}
Your SELECT statement only selects the BASKET_DESCRIPTION column, while your TableScheme (which is generated by the not shown method GetTableSchemeFromType(), but I guess it uses Type.GetProperties() to get all properties) is requesting other columns by name which aren't present in the result set, which is what the error is trying to tell you.
So either SELECT * (or at least all relevant columns), or stop building your own ORM and use an existing one.
i have my classes as the following:
class SalesInvlice
{
int id {get;set;}
string number {get;set;}
List<InvoiceItems> {get;set}
}
class InvoiceItems
{
id {get;set}
string item_name {get;set}
int price {get;set;}
}
my application is an agent that can connect to any database specified in the config file, and to execute a certain query. in my case it will execute a query on one client DB as the following select id, number, items.id as items_id, item_name, price from transaction left join transaction_details on transactions.id = transaction_details.transaction_id
let say i got the data using SQLDataReader, but i am open in another solutions as well.
what i am looking for here, is to map the results from SQLDataReader to list of SalesInvoice Object.
the issue i am facing that if the transaction has list of transaction_details, this means the datareader will get them to me in different rows.
You have a choice to make you can return the data in one dataset or two. The first will return the transaction data duplicated see bellow. You will need to manage the dr duplicate transactions.
select id,
number,
items.id as items_id,
item_name, price
from transaction
left join transaction_details
on transactions.id = transaction_details.transaction_id
Or return the data in two sets of data ( transaction and transaction detail). I am assuming that you are interested in data for a particular transaction not many different transactions. Ado.net will allow for multiple groups of data to be returned. You would need to cast each data set to is object type. dimly a case of newing up a transaction and assigning properties.
to solve this issue, i have made my simi general mapper.
I have mapper that do the following:
private List<T> mappingFun<T>( IEnumerable<Dictionary<string, object>> args, object propListObj = null) where T: Entity
{
List<T> listObject = new List<T>();
foreach(var arg in args)
{
T returnedObject = Activator.CreateInstance<T>();
PropertyInfo[] modelProperties = returnedObject.GetType().GetProperties();
var addedobject = listObject.FirstOrDefault(x => x.name == arg[returnedObject.GetType().Name + "_name"].ToString());
if (addedobject != null)
{
returnedObject = addedobject;
}
foreach(var prop in modelProperties)
{
var a = prop.PropertyType;
if (prop.PropertyType == typeof(String))
{
prop.SetValue(returnedObject, arg[returnedObject.GetType().Name + "_" + prop.Name].ToString());
}
else
{
var propModuleObj = GetType().GetMethod("mappingFun", BindingFlags.Instance | BindingFlags.NonPublic, null, new Type[] { typeof(IEnumerable<Dictionary<string, object>>), typeof(object) }, null).MakeGenericMethod(prop.PropertyType.GetGenericArguments().Single()).Invoke(this, new object[] { new List<Dictionary<string, object>>() { arg }, prop.GetValue(returnedObject) });
prop.SetValue(returnedObject, propModuleObj);
}
}
listObject.AddIfNotExist(returnedObject);
if(propListObj != null)
listObject.AddRange((List<T>)propListObj);
}
return listObject;
}
This helps 100%
I am using NEST to search my Elasticsearch index:
var result = client.Search<MyObject>(s => s
.From(0)
.Size(10)
// Query here
);
This works and returns a Nest.SearchResponse object. The format returned by result.Hits.ToList() is List<Nest.IHit<MyObject>>().
How can I convert the results returned to a DataSet (or DataTable)?
Any help is appreciated.
You need to loop over your results
DataTable dt = new DataTable();
dt.Columns.Add("Field1", typeof(string));
dt.Columns.Add("Field2", typeof(string));
...
foreach (IHit<JObject> x in result.Hits)
{
dt.Rows.Add(
x.Fields.FieldValuesDictionary["Prop1"] as JArray,
x.Fields.FieldValuesDictionary["Prop2"] as JArray
...
);
}
Read
Retrieve data from elasticsearch results
How do you get search results returned in nest 1.x mapped to an object?
as well as
DocumentsWithMetaData
When you do a search with NEST 0.12, you'd get back a QueryResponse
with two ways to loop over your results. .Documents is an
IEnumerable and .DocumentsWithMetaData is and IEnumerable>
depending on your needs one of them might be easier to use.
Starting from NEST 1.0 .DocumentsWithMetaData is now called simply
.Hits.
http://nest.azurewebsites.net/breaking-changes.html
As proposed in this article, you can use an extension method to convert an IEnumerable<T> to a DataTable:
public static class IEnumerableExtensions
{
/*Converts IEnumerable To DataTable*/
public static DataTable ToDataTable<TSource>(this IEnumerable<TSource> data)
{
DataTable dataTable = new DataTable(typeof(TSource).Name);
PropertyInfo[] props = typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo prop in props)
{
dataTable.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ??
prop.PropertyType);
}
foreach (TSource item in data)
{
var values = new object[props.Length];
for (int i = 0; i < props.Length; i++)
{
values[i] = props[i].GetValue(item, null);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
}
Then, after the search you could do something like this:
var dataTable = result.Documents.ToDataTable();
Where Documents is an IEnumerable<MyObject> that are the documents inside the hits that are returned. Another way to obtain those documents is:
var documents=result.Hits.Select(h => h.Source);
I want a method in my program that returns the resulting of a linq Query, here is the function :
public static IEnumerable<object> ConsultasSQL(int TipoConsulta)
{
SQLDataContext DC = new SQLDataContext();
IEnumerable<object> consulta = (dynamic)null;
switch(TipoConsulta)
{
case 1: // Registro en Linea
DC = SQLDataContext.GetDataContext("Tiendas", componerCS());
consulta = from tienda in DC.Tiendas
where tienda.Cod__Tienda == Globales.Tienda
select tienda;
break;
case 2:// Efectivo_Local
DC = SQLDataContext.GetDataContext("Formas de Pago",componerCS());
consulta = from pagos in DC.Formas_de_Pago
where pagos.Efectivo_Local == 1
select pagos;
break;
case 3: // Productos
DC = SQLDataContext.GetDataContext("Item",componerCS());
consulta = from Productos in DC.Item
select Productos;
break;
}
return consulta.ToList();
}
I called this function in other place of my program like this :
public static void Efectivo_local()
{
var consulta = ConsultasSQL(3);
// Globales.IdPagoLocal = consulta.First().ID_Pago;
//Globales.DesriPagoLocal = consulta.First().ID_Pago;
}
The code works and if I put consulta as a Datagridview datasource it shows me the data , but I have two problems :
1 - if I put datagridview1.datasource = consulta.first() --> Its show me nothing :(
2 - I dont know how to retrieve the value of specific field of consulta.tolist()
As you can see in the picture below , consulta have 144 records and i want to retrieve the field 1 of
the second row , its like and array? [1,1] ??
http://i.imgur.com/5xHe8Hi.jpg)
To get specific value, you have to do casts of value, so change SQLDataContext:
private SQLDataContext DC = new SQLDataContext();
...
public static IEnumerable<object> ConsultasSQL(int TipoConsulta)
{
...
}
And add this code:
var consulta = ConsultasSQL(3) as List<DC.Item.Productos>;
if (consulta != null) {
datagridview1.datasource = consulta.first();
Globales.IdPagoLocal = consulta.Find(c => c.id == 123456).ID_Pago;
}
But I suggest you change the code as Jeff Koch says... :-)
You could add an interface to your data classes. That said, if you know the type you're expecting to get at runtime, why are you using IEnumerable<object> at all? Why not have three separate functions, each of which has a single defined purpose? It would be better OO design, imo.
assuming we use this on a table using "select * from table " ?
i want to create an array of classes
that use filed names (of the datatable) as class field names initalise them with the datatable
fields
is it possible ?
if so please help :)
soory for the spelling
my writen english is not that good
but i hope i was clear
This is a great time to use anonymous types
var query = (from row in DataTable.Rows.Cast<DataRow>()
select new
{
ID = row.Field<int>("ID"),
Name = row.Field<string>("Caption"),
\\etc etc etc
}).ToArray();
If you already have a class to store the results you could just as easily replace the anonymous type with that class (or use a Tuple in 4.0)
If you do this frequently you could write an extension method that uses reflection to match up data columns with the names of properties in the class array you are trying to generate
that use filed names (of the datatable) as class field names initalise them with the datatable fields
The only way to do that would be to generate a class at runtime, which is (a) not easy and (b) usually not very useful because the code that will use this class won't know the names of the class members.
You could use the dynamic features of C# 4 to build a list of ExpandoObjects:
Func<DataRow, dynamic> projection =
r =>
{
IDictionary<string, object> exp = new ExpandoObject();
foreach (DataColumn col in r.Table.Columns)
exp[col.ColumnName] = r[col];
return exp;
};
dynamic[] objects = dataTable.AsEnumerable().Select(projection).ToArray();
...
dynamic o = objects[0];
int id = o.Id;
string name = o.Name;
But then you loose the benefits of strong typing...
I think a more interesting idea would be to project the data rows to existing classes, based on the member names and data column names:
public T DataRowToObject<T>(DataRow row) where T : new()
{
var properties = typeof(T).GetProperties().Where(p => p.CanWrite);
var columns = row.Table.Columns.Cast<DataColumn>();
// Match properties of T with DataTable columns
var common =
from p in properties
from c in columns
where c.ColumnName == p.Name
&& p.PropertyType.IsAssignableFrom(c.DataType)
select p;
T obj = new T();
foreach (var prop in common)
{
if (!row.IsNull(prop.Name))
{
object value = row[prop.Name];
prop.SetValue(obj, value, null);
}
}
return obj;
}
...
MyClass[] objects = dataTable.AsEnumerable().Select(row => DataRowToObject(row)).ToArray();