We Keep Coding

LINQ-TO-SQL query not working with CSV file

I'm reading a simple csv file from my program. Here's what my csv file looks like:
NASDQ,O,
OTC,O,
NYSE,N,
TSE,T,
Here's my code to read the csv file:
string csvFile = #"x:\tech\SQL_IntlPricing\ExchangeLookup.csv";
string[] csvLines = File.ReadAllLines(csvFile);
var csvValues = csvLines
.Select(l => new {
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First ()});
So far, everything is fine with the code. I'm using the following LINQ query:
from comp in Companies
where !comp.Coverage_status.Contains("drop")
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
But I'm getting the following error:
NotSupportedException: Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
Basically, I'm trying to get the following results:
AAPL-O
MSFT-O
Is there a way for me to achieve the results I want using my LINQ query?

If Companies are not a lot, then the simple solution would be:
from comp in Companies.Where(c => !c.Coverage_status.Contains("drop")).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
Otherwise you could do the filtering there like;
from comp in Companies.Where( c =>
csvValues.Select(cs => cs.Exchange).Contains(comp.Exchange) &&
!c.Coverage_status.Contains("drop")
).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};

Following my comment above, if converting the linq-sql expression into its extension methods syntax form is ok, you could do the following:
I created a list of companies just for the sake of the example. Company was defined as
public class Company
{
public string Coverage_status { get; set; }
public string Exchange { get; set; }
public string Ticker { get; set; }
}
Here's a full sample of how the code will would look like:
List<string> csvLines = new List<string>
{
"NASDQ,O,",
"OTC,O,",
"NYSE,N,",
"TSE,T,"
};
var csvValues = csvLines
.Select(l => new
{
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First()
});
List<Company> companies = new List<Company>
{
new Company { Coverage_status = "aaa", Ticker = "123", Exchange = "NASDQ"},
new Company { Coverage_status = "1521drop422", Ticker = "1251223", Exchange = "aaaaaaaa"},
new Company { Coverage_status = "f2hdjjd", Ticker = "15525221123", Exchange = "TSE"}
};
var result = companies
.Where(c => !c.Coverage_status.Contains("drop"))
.Select(n => new
{
FSTick = string.Format("{0}-{1}", n.Ticker,
csvValues
.Where(v => v.Exchange.Contains(n.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
});
foreach (var r in result)
Console.WriteLine(r.FSTick);
For the record, this code is definitely not performance-wise.
Output:

C# Use delegate to replace if else code

I know this method is working well. However I reused most of the code and it looks ugly... It should be possible to remove the if else using delegate. Any thoughts?
public IHttpActionResult TimeDatawithUserandServer(string name, string group)
{
List<Model> res = new List<Model>();
//group 1 is Server type
if (group.Equals("1"))
{
var realName = name.Replace("-", ".");
IQueryable<Overalldata> UserData = db.Overalldatas.Where(x => x.Server.Equals(realName));
var groupedUserData = UserData.GroupBy(x => new {x.EventDate, x.User}).ToList();
res.AddRange(groupedUserData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.User
}));
}
//group 2 is User type
else
{
IQueryable<Overalldata> Serverdata = db.Overalldatas.Where(x => x.User.Equals(name));
var groupServerData = Serverdata.GroupBy(x => new {x.EventDate, x.Server}).ToList();
res.AddRange(groupServerData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.Server
}));
}
return Ok(res);
}

You could do it this way:
public IHttpActionResult TimeDatawithUserandServer(string name, string group)
{
var getName = group == "1"
? (Func<Overalldata, string>)(od => od.Server)
: (Func<Overalldata, string>)(od => od.User);
var realName = group == "1"
? name.Replace("-", ".")
: name;
var UserData = group == "1"
? db.Overalldatas.Where(x => x.Server == realName)
: db.Overalldatas.Where(x => x.User == realName);
var groupedData =
UserData
.ToArray()
.GroupBy(x => new { x.EventDate, Name = getName(x) })
.ToArray();
var res =
groupedData
.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.Name
})
.ToList();
return Ok(res);
}

Without Delegates
Your code can simply be refactored to following and relieved from repetition:
public IHttpActionResult TimeDatawithUserandServer(string name, string group)
{
List<Model> res = new List<Model>();
var groupedData = group.Equals("1") ? FetchServerData(name) : FetchUserData(name);
res.AddRange(groupedData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.Server
}));
return Ok(res);
}
private GroupedDataType FetchServerData(string name)
{
var realName = name.Replace("-", ".");
var UserData = db.Overalldatas.Where(x => x.Server.Equals(realName));
return UserData.GroupBy(x => new {x.EventDate, x.User}).ToList();
}
private GroupedDataType FetchUserData(string name)
{
var Serverdata = db.Overalldatas.Where(x => x.User.Equals(name));
return Serverdata.GroupBy(x => new {x.EventDate, x.Server}).ToList();
}
Please replace the GroupedDataType with the appropriate type in the two methods above and in the code below.
With Delegates
Using Reflection we find the correct field (or property) and then compare the value as in the code below:
public IHttpActionResult TimeDatawithUserandServer(string name, string group)
{
List<Model> res = new List<Model>();
var groupedData = GetGroupedList(group, name);
res.AddRange(groupedData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.Server
}));
return Ok(res);
}
private static IEnumerable<GroupedDataType> GetGroupedList(string group, string name)
{
var filteredList = db.Overalldatas.Where(dbData => Filter(dbData, group, name));
return filteredList.GroupBy(x => new {x.EventDate, x.User}).ToList();
}
private static readonly Func<ODType, string, string, bool> Filter = (ode, group, name) =>
{
var objType = overallDataElement.GetType();
var field = objType.GetMember(group)[0];
return (field != null)
&& ((FieldInfo) field).GetValue(ode).ToString().Equals(name);
};
ODType is Type of the elements in of Overalldatas list and ode is overalldata list's one single element.

I agreed with a comment made by lc who suggested running this through a code review.
There are some potential anti-patterns in your code
The group parameter is an example of control coupling
The control coupling is utilized via the if statement. For me, this is a clear sign that the method should be broken into separate methods.
I would also like to call attention to the Overalldatas collection. It's name is suspicious which leads to other questions including "Is this data object trying to do too much?". But I won't really touch and this as it's less relevant to your question and my answer.
That said, my answer is that you should have two methods, one for "Server Type" and one for "User Type". I realize this pushes that "group" check elsewhere, but that can be solved later and today the intent of your methods will be MUCH clearer.
public IHttpActionResult TimeDataByUserName(string userName)
{
List<Model> res = new List<Model>();
IQueryable<Overalldata> Serverdata = db.Overalldatas.Where(x => x.User.Equals(userName));
var groupServerData = Serverdata.GroupBy(x => new {x.EventDate, x.Server}).ToList();
res.AddRange(groupServerData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.Server
}));
return Ok(res);
}
public IHttpActionResult TimeDataByServerName(string serverName)
{
List<Model> res = new List<Model>();
IQueryable<Overalldata> UserData = db.Overalldatas.Where(x => x.Server.Equals(serverName));
var groupedUserData = UserData.GroupBy(x => new {x.EventDate, x.User}).ToList();
res.AddRange(groupedUserData.Select(item => new Model()
{
Count = item.Count(),
Date = item.Key.EventDate.ToString("yyyy-MM-dd HH:mm:ss"),
Name = item.Key.User
}));
}

Efficiently retrieving and filtering files by using filename

i am a newbie hier, i try to retreive files by using filenames, which have the following definition:
Items number + Revision + lot number.pdf
For example:
1109093-A2 (85806S).pdf
1109093-A3 (85806S).pdf
1109092-A1 (85806S).pdf
1109092-A2 (85806S).pdf
for this sample file: 1109093-A2 (85806S).pdf
Items number: 1109093
Revision: -A2
End item number: (85806S)
for my search result, i am supose to have only this files.
1109093-A3 (85806S).pdf
1109092-A2 (85806S).pdf
i must have only files, with the actual Revision like the one up there( A3 ,A2).
But it not, still now i am getting all Files, how can i sort it by Revision please???
A1, A2, A3, A.... (Revisions) represents the selection criteria I should use. I wrote the follow function for this job.
private string[] GetFiles()
{
strSourcePath = textBox1.Text;
strTargetPath = textBox2.Text;
string fileName = string.Empty;
strExtension = "*).pdf";
string[] files = null;
if (Directory.Exists(strSourcePath))
{
files = Directory.GetFiles(strSourcePath, strExtension, SearchOption.AllDirectories);
var Result = "";
string joined = String.Join("# ", Result);
files = null;
Result = joined.Split('#');
files = Result.Where(file => Regex.IsMatch(Path.GetFileName(file), "^[0-9]+")).ToArray();
}
else
{
MessageBox.Show("Source path does not exist!");
}
return files ;
}

After you got the paths, you can parse the filename, extract revision/etc and sort based on your criteria.
This code parse as an anonymous class (for readability) and sort based on ItemNumber, Revision.
The anonymous class contains the path and item number/revision/end number info.
See the demo for complete example
var paths = new [] {
"1109093-A2 (85806S).pdf",
"1109093-A3 (85806S).pdf",
"1109092-A1 (85806S).pdf",
"1109092-A2 (85806S).pdf",
};
var result = paths.Select(x => {
var match = Regex.Match(x, #"(?<ItemsNumber>\d+)-(?<Revision>\w+)\s+\((?<EndItemNumber>\w+)\).pdf");
if (match.Success)
{
return new { ItemNumber = match.Groups[1].Value, Revision = match.Groups[2].Value, EndItemNumber = match.Groups[3].Value, Path = x };
}
else {
return new { ItemNumber = "", Revision = "", EndItemNumber = "", Path = x };
}
})
.OrderBy(x => x.ItemNumber).ThenBy(x => x.Revision);
demo: https://dotnetfiddle.net/47uZni

Using your template I wrote this function, but the return value is always the same - one item, but not a list as I expected. I do not know why. Did you have some idea?
private string[] SortFileName(string []TemP)
{
var paths = GetTheFileName(TemP);
List<string> TheCollection = new List<string>();
var result = paths.Select(x => {
var match = Regex.Match(x, #"(?<ItemsNumber>\d+)-(?<Revision>\w+)\s+\((?<EndItemNumber>\w+)\).pdf");
if (match.Success)
{
return new { ItemNumber = match.Groups[1].Value, Revision = match.Groups[2].Value, EndItemNumber = match.Groups[3].Value, Path = x };
}
else {
return new { ItemNumber = "", Revision = "", EndItemNumber = "", Path = x };
}
})
.GroupBy(x => x.ItemNumber)
.Select(x => x.OrderByDescending(y => y.Revision).First());
foreach (var item in result)
{
TheCollection.Add(item.Path.ToString());
}
return TheCollection.ToArray();
}
PS: GetTheFileName(TemP); return an array with more than 130 items. thanks for the coming help.

Concat or Join list of "names" in Linq

Using WCF RIA I have a query that returns a Query of names
public class WitnessInfo
{
[Key]
public Guid WCFId { get; set; }
public string witnessName { get; set; }
public string AllNames {get; set;}
}
Here's my Linq Query
[Query]
public IQueryable<WitnessInfo> getWitnessInfo(int? id)
{
IQueryable<WitnessInfo> witnessQuery = from witness in this.Context.witness
where witness.DAFile.Id == id
select new WitnessInfo
{
WCFId = Guid.NewGuid(),
witnessName = witness.Person.FirstName,
};
return witnessQuery;
}
I want to take all the names and return them in a single string i.e "John, James, Tim, Jones". Tried taking AllNames and looping through but that didn't work. Any suggestions?

First grab all of the information that you need in a single query, then use String.Join to map the collection of names to a single string:
var firstQuery = from witness in Context.witness
where witness.DAFile.Id == id
select new
{
WCFId = Guid.NewGuid(),
witnessName = witness.Person.FirstName,
Names = Context.witness.Select(w => w.FirstName),
})
.AsEnumerable(); //do the rest in linq to objects
var finalQuery = from witness in firstQuery
//do the string manipulation just once
let allNames = string.Join(", ", witness.Names)
select new WitnessInfo
{
WCFId = witness.WCFId,
witnessName = witness.witnessName,
AllNames = allNames,
});

By having the AllNames property in the WitnessInfo class, it is seems like you want each WitnessInfo object to contain the all of the squence names again and again repeatedly, and if this is your case then do it like that:
var names = (from witness in this.Context.witness
select witness.Person.FirstName).ToArray();
var allNames = string.Join(", ", names);
IQueryable<WitnessInfo> witnessQuery = from witness in this.Context.witness
where witness.DAFile.Id == id
select new WitnessInfo
{
WCFId = Guid.NewGuid(),
witnessName = witness.Person.FirstName,
AllNames = allNames
};

You can concatenate like this:
string.Join(", ", getWithnessInfo(666).Select(wq => wq.witnessName))

this.Context.witness.Select(a => a.Person.Firstname).Aggregate((a, b) => a + ", " + b);

How to compare and select values from string using Linq?

Had a class:
class filedate
{
public int id;
public string fname;
}
Fill my list with values:
List<filedate> List = ReadList(sqlFiles);
string[] FolderFiles = System.IO.Directory.GetFiles(path2Copy);
Trying to get results:
var results = List.Where(filedate =>
FolderFiles.Any(x=>Path.GetFileNameWithoutExtension(x) ==
Path.GetFileNameWithoutExtension(filedate.fname)));
I have the same files in List and FolderFiles, but get no results in results. I am a newbie to Linq. Where is the problem?
update:
List: (count) > 1000
for example:
<1023, 'tr_F2opervag_2808_1644.dat'>
FolderFiles example:
"\\domain.corp.dns\share\folder\tr_F2opervag_2808_1644.dat"
Update 2:
found out my mistake! Comment with intersection was helpful! This code is working:
var results = List.Where(
(filedate x) =>
{
return ! FolderFiles.Any(xxx =>
Path.GetFileNameWithoutExtension(xxx) ==
Path.GetFileNameWithoutExtension(x.fname));
});

You're code works fine for me so there's something wrong with the format of your data in the List coming back from the database.
Post an example of an fname value from the filedata object. It needs to be a valid fully qualified path.
This works fine for me.
public class FileData{
public int id;
public string fname;
}
void Main()
{
List<FileData> list = new List<FileData>{
new FileData { id=1, fname="C:\\install.res.1042.dll"},
new FileData { id=2, fname="C:\\install.res.1041.dll" },
new FileData { id=3, fname="C:\\install.res.9999.dll"}
};
string[] FolderFiles = System.IO.Directory.GetFiles("C:\\");
var results = list
.Where(fd =>
FolderFiles.Any(x=>Path.GetFileNameWithoutExtension(x) ==
Path.GetFileNameWithoutExtension(fd.fname)));
Console.WriteLine(results);
}

If you need to find the difference this should work. This is available via Enumerable.Except.
var dbFiles = ReadList(sqlFiles);
var dbFilePaths =
dbFiles.Select(fdate =>
Path.GetFileNameWithoutExtension(fdate.fname).ToLower());
var fsFilePaths =
Directory
.GetFiles(path2Copy)
.Select(filePath =>
Path.GetFileNameWithoutExtension(filePath).ToLower());
var diff =
dbFilePaths
.Except(fsFilePaths)
.Join(dbFiles,
filePath => filePath,
fdate => fdate.fname,
(filePath, fdate) => fdate)
.ToList();