I'm reading a simple csv file from my program. Here's what my csv file looks like:
NASDQ,O,
OTC,O,
NYSE,N,
TSE,T,
Here's my code to read the csv file:
string csvFile = #"x:\tech\SQL_IntlPricing\ExchangeLookup.csv";
string[] csvLines = File.ReadAllLines(csvFile);
var csvValues = csvLines
.Select(l => new {
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First ()});
So far, everything is fine with the code. I'm using the following LINQ query:
from comp in Companies
where !comp.Coverage_status.Contains("drop")
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
But I'm getting the following error:
NotSupportedException: Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
Basically, I'm trying to get the following results:
AAPL-O
MSFT-O
Is there a way for me to achieve the results I want using my LINQ query?
If Companies are not a lot, then the simple solution would be:
from comp in Companies.Where(c => !c.Coverage_status.Contains("drop")).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
Otherwise you could do the filtering there like;
from comp in Companies.Where( c =>
csvValues.Select(cs => cs.Exchange).Contains(comp.Exchange) &&
!c.Coverage_status.Contains("drop")
).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
Following my comment above, if converting the linq-sql expression into its extension methods syntax form is ok, you could do the following:
I created a list of companies just for the sake of the example. Company was defined as
public class Company
{
public string Coverage_status { get; set; }
public string Exchange { get; set; }
public string Ticker { get; set; }
}
Here's a full sample of how the code will would look like:
List<string> csvLines = new List<string>
{
"NASDQ,O,",
"OTC,O,",
"NYSE,N,",
"TSE,T,"
};
var csvValues = csvLines
.Select(l => new
{
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First()
});
List<Company> companies = new List<Company>
{
new Company { Coverage_status = "aaa", Ticker = "123", Exchange = "NASDQ"},
new Company { Coverage_status = "1521drop422", Ticker = "1251223", Exchange = "aaaaaaaa"},
new Company { Coverage_status = "f2hdjjd", Ticker = "15525221123", Exchange = "TSE"}
};
var result = companies
.Where(c => !c.Coverage_status.Contains("drop"))
.Select(n => new
{
FSTick = string.Format("{0}-{1}", n.Ticker,
csvValues
.Where(v => v.Exchange.Contains(n.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
});
foreach (var r in result)
Console.WriteLine(r.FSTick);
For the record, this code is definitely not performance-wise.
Output:
Related
I'm trying to make some queries using EF-Core and I have the following code
public List<Visitation> GetAllVisitations()
{
return this.hospital.Visitations
.Where(v => v.DoctorId == this.Doctor.Id)
.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
})
.ToList();
}
public List<Visitation> GetVisitationByPatient(int id)
{
var patient = this.GetPatientById(id);
return this.hospital.Visitations
.Where(v => v.PatientId == patient.Id)
.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
})
.ToList();
}
It is pretty obvious that the Select statement is the same in both methods. However I know that EF Core uses Expression<Func>, rather than Func therefore I do not know how to make an Expression, which can be used in both Select statements.
The query won't execute until you call .ToList(). So you may take the partial query up to the .Where() and pass it to a function that adds the Select() portion.
Something like this:
public List<Visitation> GetAllVisitations()
{
var query = this.hospital.Visitations
.Where(v => v.DoctorId == this.Doctor.Id);
return this.addTransformation(query)
.ToList();
}
public List<Visitation> GetVisitationByPatient(int id)
{
var patient = this.GetPatientById(id);
var query = this.hospital.Visitations
.Where(v => v.PatientId == patient.Id)
return this.addTransformation(query)
.ToList();
}
public IQueriable<Visitation> AddTransformation(IQueriable<Visitation> query)
{
return query.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
});
}
I have a foreach with an ordebydescending(), but in one case I need to use an orderby() instead. Depending on the value of articleType how can I use an inline condition inside the foreach to allow this to happen.
This is the condition I need to build into to determine the use of orderbydescending or orderby
if (articleType == BusinessLogic.ArticleType.Webinar)
This is the full function
public static List<Article> GetArticles(BusinessLogic.ArticleType articleType, long languageID)
{
List<Article> articles = new List<Article>();
using (var db = new DatabaseConnection())
{
foreach (var record in db
.GetArticles(BusinessLogic.SystemComponentID, (int) articleType, languageID)
.OrderByDescending(c => c.Date_Time))
{
#region articleTextLanguageID
long articleTextLanguageID = GetArticleTextLanguageID(record.ID, languageID);
string previewImageName = GetArticle(record.ID, articleTextLanguageID).PreviewImageName;
#endregion
Article article = new Article()
{
ID = record.ID,
Title = record.Title,
Summary = record.Summary,
PreviewImageName = previewImageName,
Date = record.Date_Time,
ArticleTextLanguageID = articleTextLanguageID
};
articles.Add(article);
}
}
return articles;
}
Was thinking something along these lines, but its not working
foreach (var record in db
.GetArticles(BusinessLogic.SystemComponentID, (int)articleType, languageID)
.Where(articleType == BusinessLogic.ArticleType.Webinar.ToString()?.OrderByDescending(c => c.Date_Time)) : .OrderBy(c => c.Date_Time)))
The way to do this would be to construct the query in pieces. For example:
var query = db.GetArticles(BusinessLogic.SystemComponentID, (int)articleType, languageID);
if(articleType == BusinessLogic.ArticleType.Webinar)
{
query = query.OrderByDescending(c => c.Date_Time);
}
else
{
query = query.OrderBy(c => c.Date_Time);
}
foreach(var record in query)
{
// process
}
If you required additional sorting, you'd need an extra variable typed as IOrderedEnumerable/IOrderedQueryable (depending on what GetArticles returns) as an intermediate to chain ThenBy/ThemByDescending:
var source = db.GetArticles(BusinessLogic.SystemComponentID, (int)articleType, languageID);
IOrderedEnumerable<Article> query;
if(articleType == BusinessLogic.ArticleType.Webinar)
{
query = source.OrderByDescending(c => c.Date_Time);
}
else
{
query = source.OrderBy(c => c.Date_Time);
}
if(somethingElse)
{
query = query.ThenBy(c => c.OtherProperty);
}
foreach(var record in query)
{
// process
}
Based on your comment below, as notes above the second example would look more like the following (this means that db.GetArticles returns an IQueryable<Article> and not an IEnumerable<Article>):
var source = db.GetArticles(BusinessLogic.SystemComponentID, (int)articleType, languageID);
IOrderedQueryable<Article> query;
if(articleType == BusinessLogic.ArticleType.Webinar)
{
query = source.OrderByDescending(c => c.Date_Time);
}
else
{
query = source.OrderBy(c => c.Date_Time);
}
if(somethingElse)
query = query.ThenBy(c => c.OtherProperty);
foreach(var record in query)
{
// process
}
You could also shorten it to the following:
var source = db.GetArticles(BusinessLogic.SystemComponentID, (int)articleType, languageID);
var query = articleType == BusinessLogic.ArticleType.Webinar
? source.OrderByDescending(c => c.Date_Time)
: source.OrderBy(c => c.Date_Time);
if(somethingElse)
query = query.ThenBy(c => c.OtherProperty);
foreach(var record in query)
{
// process
}
I wrote simple query (issues is when I try to set an Address below ProductCode):
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
//Address = new AddressDTO // THIS WORKS BUT LOOKS UGLY :(
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(Address address)
{
return new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip,
};
}
This code above breaks on this line:
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
It says that linq cannot be translated and advicing me to rewrite a query..
But this commented code here which does almost the same works, so if I remove calling Map method in Select and if I uncomment this code in Select everything would work, but I would like to get rid of this - writing too many OrderBy for each prop, I would like to Order it once and after that I would like to use it.. but unfortunatelly I don't know how.. :
//Address = new AddressDTO
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
Thanks in advance,
Cheers
Presumably Entity Framework is translating this code to SQL, and your custom Map() method is unknown to SQL. Fortunately that method doesn't do much, so you should be able to move its functionality directly to the query.
You can use .Select() to project the collection into a new type (just like you already do for building your ProductDto).
For example:
//...
Address = x.Product.Addresses.OrderBy(x=> x.CreatedDate)
.Select(x=> new AddressDTO
{
Address = x.Address,
Country = x.Country,
Zip = x.Zip
})
.FirstOrDefault()
//...
You can't call the Map function as it is, but you can modify it a bit to work on Queryables instead
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses),
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(IQueryable<Address> addresses)
{
return addresses.OrderBy(x => x.CreatedDate).Select(x => new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip
}).FirstOrDefault();
}
See David's answer above for explanation why it does not work.
For a workaround, you can call .ToList() before the mapping so the query is executed on the server:
_connectDBContext.Products
.Join(...)
.Where(x => x.Id == x.CustomerRelationId)
.ToList() // !!!
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
...
});
I have a query that is currently far too slow.
I am trying to search a Code (a string) on the main page that will bring the user the relevant info.
Eg. The user can search a code from the main page and this will search for the code in Job, Work Phase, Wbs, Work Element, EA, Jobcard and Estimate and return the relevant info.
I make a number of trips to the database to collect the data i need when I believe it can be done in just one.
I have a number of tables that are all linked:
Contracts, Jobs, WorkPhases, Wbss, Engineering Activities, Jobcards and Estimates.
Contracts have a list of Jobs,
Jobs have a list of Workphases,
Workphases have a list of Wbss etc
Is there a quicker way to do this?
public Result Handle(Query query)
{
query.Code = query.Code ?? string.Empty;
var result = new Result();
//result.SetParametersFromPagedQuery(query);
result.Items = new List<Item>();
if (query.SearchPerformed)
{
var contracts = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(contracts.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Contract,
ContractName = x.Name,
Url = string.Format("Admin/Contract/Edit/{0}", x.Id)
})).ToList();
var jobs = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jobs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
ContractName = x.Contract.Name,
Type = MainPageSearchEnum.Job,
Url = string.Format("Admin/Job/Edit/{0}", x.Id)
})).ToList();
//var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code.ToLower() == query.Code.ToLower());
var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code);
result.Items = result.Items.Concat(workPhases.Select(x => new Item()
{
Code = x.ContractPhase.Code,
Id = x.Id,
Name = x.ContractPhase.Name,
Type = MainPageSearchEnum.WorkPhase,
Url = string.Format("Admin/WorkPhase/Edit/{0}", x.Id)
})).ToList();
var wbss = _db.WBSs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(wbss.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.WBS,
Url = string.Format("Admin/WBS/Edit/{0}", x.Id)
})).ToList();
var eas = _db.EngineeringActivities.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(eas.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/EngineeringActivity/Edit/{0}", x.Id)
})).ToList();
var jcs = _db.Jobcards.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jcs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/JobCard/Edit/{0}", x.Id)
})).ToList();
var estimates = _db.Estimates.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(estimates.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Estimate,
Url = string.Format("Estimation/Estimate/Edit/{0}", x.Id)
})).ToList();
}
return result;
}
Disclaimer: I'm the owner of the project Entity Framework Plus
This library has a Query Future feature which allows batching multiple queries in a single roundtrip.
Example:
// using Z.EntityFramework.Plus; // Don't forget to include this.
var ctx = new EntitiesContext();
// CREATE a pending list of future queries
var futureCountries = ctx.Countries.Where(x => x.IsActive).Future();
var futureStates = ctx.States.Where(x => x.IsActive).Future();
// TRIGGER all pending queries in one database round trip
// SELECT * FROM Country WHERE IsActive = true;
// SELECT * FROM State WHERE IsActive = true
var countries = futureCountries.ToList();
// futureStates is already resolved and contains the result
var states = futureStates.ToList();
Wiki: EF+ Query Future
Have you tried the Union / UnionAll operator?
It's purpose is exactly like you wish - combine the identical data from different sources.
Furthermore due to the concept of deferred execution your query will only be executed when you actually iterate over the result (or call a method that does that, for example - .ToList()
var contractsQuery = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new {Code=x.Code, Id=x.Id, ...});
var jobsQuery = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
var workPhasesQuery = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
// and so on
var combinedQuery = contractsQuery.UnionAll(jobsQuery).UnionAll(workPhasesQuery ).UnionAll(...
var result = combinedQuery.ToList();
A similar question is Union in linq entity framework
Another code sample can be found here
Please notice that this is exactly the same concept of manipulating data as in T-SQL union, and under the covers you will get an sql query using a union operator
Yes there most certainly is a way to query multiple tables. You can use the Include() method extension for your query. for instance:
var examplelist = _db.Contracts.(v => v.id == "someid" && v.name == "anotherfilter").Include("theOtherTablesName").ToList();
You can include as many tables as you like this way. This is the recommended method.
You can also use the UnionAll() method but you'd have to define your queries separately for this
Had a class:
class filedate
{
public int id;
public string fname;
}
Fill my list with values:
List<filedate> List = ReadList(sqlFiles);
string[] FolderFiles = System.IO.Directory.GetFiles(path2Copy);
Trying to get results:
var results = List.Where(filedate =>
FolderFiles.Any(x=>Path.GetFileNameWithoutExtension(x) ==
Path.GetFileNameWithoutExtension(filedate.fname)));
I have the same files in List and FolderFiles, but get no results in results. I am a newbie to Linq. Where is the problem?
update:
List: (count) > 1000
for example:
<1023, 'tr_F2opervag_2808_1644.dat'>
FolderFiles example:
"\\domain.corp.dns\share\folder\tr_F2opervag_2808_1644.dat"
Update 2:
found out my mistake! Comment with intersection was helpful! This code is working:
var results = List.Where(
(filedate x) =>
{
return ! FolderFiles.Any(xxx =>
Path.GetFileNameWithoutExtension(xxx) ==
Path.GetFileNameWithoutExtension(x.fname));
});
You're code works fine for me so there's something wrong with the format of your data in the List coming back from the database.
Post an example of an fname value from the filedata object. It needs to be a valid fully qualified path.
This works fine for me.
public class FileData{
public int id;
public string fname;
}
void Main()
{
List<FileData> list = new List<FileData>{
new FileData { id=1, fname="C:\\install.res.1042.dll"},
new FileData { id=2, fname="C:\\install.res.1041.dll" },
new FileData { id=3, fname="C:\\install.res.9999.dll"}
};
string[] FolderFiles = System.IO.Directory.GetFiles("C:\\");
var results = list
.Where(fd =>
FolderFiles.Any(x=>Path.GetFileNameWithoutExtension(x) ==
Path.GetFileNameWithoutExtension(fd.fname)));
Console.WriteLine(results);
}
If you need to find the difference this should work. This is available via Enumerable.Except.
var dbFiles = ReadList(sqlFiles);
var dbFilePaths =
dbFiles.Select(fdate =>
Path.GetFileNameWithoutExtension(fdate.fname).ToLower());
var fsFilePaths =
Directory
.GetFiles(path2Copy)
.Select(filePath =>
Path.GetFileNameWithoutExtension(filePath).ToLower());
var diff =
dbFilePaths
.Except(fsFilePaths)
.Join(dbFiles,
filePath => filePath,
fdate => fdate.fname,
(filePath, fdate) => fdate)
.ToList();