Linq Lookup to parse a CSV text line - c#

Issue
I had asked this question a while back and the requirements has changed a bit since then.
Now, there is a possibility to have a file with lines as follow:
Bryar22053;ADDPWN;Bryar.Suarez#company.com;ACTIVE
Nicole49927;ADDPWN;Nicole.Acosta#company.com;ACTIVE
Rashad58323;ADDPWN;Rashad.Everett#company.com;ACTIVE
Take first line. The first value Bryar22053 is skipped and the same lookup is used:
var columnCount = dataRow.Skip(1).Count();
var modular = 0;
// Simple Enum
var rightsFileType = new RightsFileType();
if (columnCount % 2 == 0)
{
rightsFileType = RightsFileType.WithoutStatus;
modular = 2;
}
else if (columnCount % 3 == 0)
{
rightsFileType = RightsFileType.WithStatus;
modular = 3;
}
var lookup = dataRow.Skip(1).Select((data, index) => new
{
lookup = index % modular,
index,
data
}).ToLookup(d => d.lookup);
The lookup object now has three groups:
> ? lookup[0].ToList() Count = 1
> [0]: { lookup = 0, index = 0, data = "ADDPWN" } ? lookup[1].ToList() Count = 1
> [0]: { lookup = 1, index = 1, data = "Bryar.Suarez#company.com" } ? lookup[2].ToList() Count = 1
> [0]: { lookup = 2, index = 2, data = "ACTIVE" }
If it was the original case where it would be just System1,User1,System2,User2... the lookup would have two groups and following code would work:
List<RightObjectRetrieved> rights;
rights = lookup[0].Join(lookup[1], system => system.index + 1, username => username.index, (system, username) => new
{
system = system.data,
useraname = username.data
}).Where(d => !string.IsNullOrEmpty(d.system)).Select(d => new RightObjectRetrieved {UserIdentifier = userIdentifier, SystemIdentifer = d.system, Username = d.useraname, RightType = rightsFileType}).ToList();
// rights => Key = System Identifier, Value = Username
But with the third 'status' as System1,User1,Status1,System2,User2,Status2..., I'm having issue trying to Join and get all three. Please help.
Edit
Here is what I have for raw data:
// Method has parameter localReadLine (string) that has this:
// Bryar22053;ADDPWN;Bryar.Suarez#company.com;ACTIVE
// Data line
var dataRow = localReadLine.Split(new[] { ToolSettings.RightsSeperator }, StringSplitOptions.None);
// Trim each element
Array.ForEach(dataRow, x => dataRow[Array.IndexOf(dataRow, x)] = x.Trim());
Tried (failed) so far
rights = lookup[0].Join(lookup[1], system => system.index + 1, username => username.index, status => status.index, (system, username, status) => new
{
system = system.data,
useraname = username.data,
status = status.data
}).Where(d => !string.IsNullOrEmpty(d.system)).Select(d => new RightObjectRetrieved {UserIdentifier = userIdentifier, SystemIdentifer = d.system, Username = d.useraname, RightType = rightsFileType}).ToList();
And
rights = lookup[0].Join(lookup[1], system => system.index + 1, username => username.index, (system, username) => new
{
system = system.data,
useraname = username.data
}).Join(lookup[2], status => status.index, (status) => new
{
status = status.data
}).Where(d => !string.IsNullOrEmpty(d.system)).Select(d => new RightObjectRetrieved {UserIdentifier = userIdentifier, SystemIdentifer = d.system, Username = d.useraname, RightType = rightsFileType, Status = ParseStatus(status)}).ToList();

I think you need to split up a little bit your implementation.
Let's declare a class that will hold the data:
class Data
{
public string System { get; set; }
public string Username { get; set; }
public string Status { get; set; }
}
Now, let's define a couple of parsing functions to parse a line.
The first one will parse a line which includes status:
var withStatus = (IEnumerable<string> line) => line
.Select((token, index) => new { Value = token, Index = index })
.Aggregate(
new List<Data>(),
(list, token) =>
{
if( token.Index % 3 == 0 )
{
list.Add(new Data { System = token.Value });
return list;
}
var data = list.Last();
if( token.Index % 3 == 1 )
data.Username = token.Value;
else
data.Status = token.Value;
return list;
});
The second one will parse a line which doesn't include status:
var withoutStatus = (IEnumerable<string> line) => line
.Select((token, index) => new { Value = token, Index = index })
.Aggregate(new List<Data>(),
(list, token) =>
{
if( token.Index % 2 == 0)
list.Add(new Data { System = token.Value });
else
list.Last().Username = token.Value;
return list;
});
With all that in place, you'll need the following:
Determine the modulus
Iterate the lines of the file and parse each line
Group and aggregate the results
The remaining code would look like this:
var lines = streamReader.ReadAllLines(); // mind the system resources here!
var parser = lines.First().Split(';').Length % 2 == 0 ? withoutStatus : withStatus;
var data = lines.Skip(1) // skip the header
.Select(line =>
{
var parts = line.Split(';');
return new
{
UserId = parts.First(),
Data = parser(parts.Skip(1))
};
})
.GroupBy(x => x.UserId)
.ToDictionary(g => g.Key, g => g.SelectMany(x => x.Data));
Now you have a Dictionary<string, Data> which holds the user id and its info.
Of course, a more elegant solution would be to separate each parsing function into its own class and join those classes under a common interface in case there would be more info to add in the future but the code above should work and give you an idea of what you should do.

If you want to use joins:
var result = lookup[0]
.Join(lookup[1],
system => system.index,
username => username.index - 1,
(system, username) => new {system = system.data, username = username.data, system.index})
.Join(lookup[2],
d => d.index,
status => status.index - 2,
(d, status) => new {d.system, d.username, status = status.data})
.ToList();
Another option to group by records and just select data from it (looks more readable from my point of view):
var result = dataRow
.Skip(1)
.Select((data, index) => new {data, record = index / 3})
.GroupBy(r => r.record)
.Select(r =>
{
var tokens = r.ToArray();
return new
{
system = tokens[0].data,
username = tokens[1].data,
status = tokens[2].data
};
})
.ToList();

Related

How to get two fields with the same Id

I need to send two fields with the same Id in Altair(GraphQl).
mutation{
createGoodsOrder(goodsorder: {
deliveryDate: "2019-10-10"
goodsOrderItems: [
{ orderItemId: 54 quantity: 1 costPerUnit: 1 goodType: INGREDIENT }
{ orderItemId: 54 quantity: 2 costPerUnit: 2 goodType: INGREDIENT }
# { orderItemId: 58 quantity: 2 costPerUnit: 2 goodType: INGREDIENT }
]
}){
id
}
}
When I execute mutation, model contains both fields with the same Id but when I make Fetch, it returns only the first one. If It is not the same, Fetch returns both fields. How can I get both fields with the same Id?
var orderIngredients = _repository.Fetch<Ingredient>(e => model.GoodsOrderItems.Any(g => g.OrderItemId == e.Id)).ToList();
var orderIngredients = _repository.Fetch<Ingredient>(
e => e.IngredientType.PlaceId == model.PlaceId
&& model.GoodsOrderItems.Any(g => g.OrderItemId == e.Id && g.GoodType == GoodsTypes.Ingredient))
.Select(e => new GoodsOrderIngredientCreateModel
{
IngredientId = e.Id,
Quantity = model.GoodsOrderItems.First(i => i.OrderItemId == e.Id).Quantity,
CostPerUnit = model.GoodsOrderItems.First(i => i.OrderItemId == e.Id).CostPerUnit,
TotalPrice = model.GoodsOrderItems.First(i => i.OrderItemId == e.Id).Quantity *
model.GoodsOrderItems.First(i => i.OrderItemId == e.Id).CostPerUnit,
GoodType = GoodsTypes.Ingredient
}).Select(v => new GoodsOrderIngredient
{
Id = v.Id,
IngredientId = v.IngredientId,
Quantity = v.Quantity,
CostPerUnit = v.CostPerUnit,
TotalPrice = v.TotalPrice
}).ToList();
Mutation:
mutation.Field<GoodsOrderType>(
name: "createGoodsOrder",
arguments: new QueryArguments(
new QueryArgument<NonNullGraphType<GoodsOrderCreateInput>> { Name = nameof(GoodsOrder).ToLower() }
),
resolve: context =>
{
if (context.UserContext is GraphQLUserScopedContext userContext)
{
var goodsOrderService = userContext.ServiceScope.ServiceProvider.GetRequiredService<IVendorService>();
var model = context.GetArgument<GoodsOrderCreateModel>(nameof(GoodsOrder).ToLower());
model.PlaceId = userContext.User.PlaceId;
model.NetworkId = userContext.User.NetworkId;
var goodsOrder = goodsOrderService.CreateGoodsOrder(model);
return goodsOrder;
}
else
throw new ExecutionError(Constants.ErrorCodes.WrongUserContext);
}).RequireAuthorization(PermissionsRequirement
.CreateForPermissionSetAll(
new Dictionary<NetworkPermissions, PermissionLevels>
{ {NetworkPermissions.ERP_Cumulative, PermissionLevels.EditCreate} }));
I don't know c# but probably you don't need intermediate types
var orderIngredients = _repository.Fetch<Ingredient>(
e => e.IngredientType.PlaceId == model.PlaceId
&& model.GoodsOrderItems.Any(g => g.OrderItemId == e.Id && g.GoodType == GoodsTypes.Ingredient))
.Select(v => new GoodsOrderIngredient
{
Id = v.Id,
IngredientId = v.IngredientId,
Quantity = v.Quantity,
CostPerUnit = v.CostPerUnit,
TotalPrice = v.Quantity * v.CostPerUnit
}).ToList();
PS. If GoodsOrderIngredientCreateModel (for create mutation?) contains TotalPrice then total calculations are already in DB ?

Returning a LINQ database query from a Method

Hello everyone I have this query I am performing in multiple places. Instead of retyping the query over and over, I would like to be able to call a method that returns the query. I am not sure what to put as the return type for the method or if this is even possible to do. I use the query to write a csv file of the information, and I use the query to add items to my observable collection that is bound to a list view.
using (ProjectTrackingDBEntities context = new ProjectTrackingDBEntities())
{
var result = context.TimeEntries.Where(Entry => Entry.Date >= FilterProjectAfterDate
&& Entry.Date <= FilterProjectBeforerDate
&& (FilterProjectName != null ? Entry.ProjectName.Contains(FilterProjectName) : true))
.GroupBy(m => new { m.ProjectName, m.Phase })
.Join(context.Projects, m => new { m.Key.ProjectName, m.Key.Phase }, w => new { w.ProjectName, w.Phase }, (m, w) => new { te = m, proj = w })
.Select(m => new
{
Name = m.te.Key.ProjectName,
Phase = m.te.Key.Phase,
TimeWorked = m.te.Sum(w => w.TimeWorked),
ProposedCompletionDate = m.proj.ProposedCompletionDate,
ActualCompletionDate = m.proj.ActualCompletionDate,
Active = m.proj.Active,
StartDate = m.proj.StartDate,
Description = m.proj.Description,
EstimatedHours = m.proj.EstimatedHours
});
}
I am able to do both right now by retyping the query and performing the subsequent foreach() loops on the data. I would rather be able to do something like:
var ReturnedQuery = GetProjectsQuery();
foreach(var item in ReturnedQuery)
{
//do stuff
}
Any help would be appreciated.
You want to return IQueryable<T> with a known model that represents what it is you are returning. You should not return an anonymous type. Also you want to pass in the DbContext so it can be disposed of by the caller and not in the method otherwise you will receive an exception that the DbContext has been disposed of.
For example:
public IQueryable<ProjectModel> GetProjectQuery(ProjectTrackingDBEntities context) {
return context.TimeEntries.Where(Entry => Entry.Date >= FilterProjectAfterDate
&& Entry.Date <= FilterProjectBeforerDate
&& (FilterProjectName != null ? Entry.ProjectName.Contains(FilterProjectName) : true))
.GroupBy(m => new { m.ProjectName, m.Phase })
.Join(context.Projects, m => new { m.Key.ProjectName, m.Key.Phase }, w => new { w.ProjectName, w.Phase }, (m, w) => new { te = m, proj = w })
.Select(m => new ProjectModel
{
Name = m.te.Key.ProjectName,
Phase = m.te.Key.Phase,
TimeWorked = m.te.Sum(w => w.TimeWorked),
ProposedCompletionDate = m.proj.ProposedCompletionDate,
ActualCompletionDate = m.proj.ActualCompletionDate,
Active = m.proj.Active,
StartDate = m.proj.StartDate,
Description = m.proj.Description,
EstimatedHours = m.proj.EstimatedHours
});
}
ProjectModel.cs
public class ProjectModel {
public string Name {get;set;}
public string Phase {get;set;}
// rest of properties
}
Calling code
using (ProjectTrackingDBEntities context = new ProjectTrackingDBEntities())
{
var ReturnedQuery = GetProjectsQuery(context);
foreach(var item in ReturnedQuery)
{
//do stuff
}
}
It is easy to return the enumerator, but you can't return an enumerator for an anonymous type, unfortunately. Probably the easiest path forward for you would be to return enumerator over the full row object, like this:
public IEnumerable<TimeEntries> GetTimeEntries()
{
using (ProjectTrackingDBEntities context = new ProjectTrackingDBEntities())
{
return context.TimeEntries
.Where
(
Entry =>
Entry.Date >= FilterProjectAfterDate &&
Entry.Date <= FilterProjectBeforerDate &&
(FilterProjectName != null ? Entry.ProjectName.Contains(FilterProjectName) : true)
)
.GroupBy(m => new { m.ProjectName, m.Phase })
.Join
(
context.Projects,
m => new { m.Key.ProjectName, m.Key.Phase },
w => new { w.ProjectName, w.Phase },
(m, w) => new { te = m, proj = w }
);
}
)
}
And use it like this:
var query = GetTimeEntries();
foreach (var row in query.Select( m => new { Name = row.te.Key.ProjectName })
{
Console.WriteLine(row.Name);
}

Best algorithm to determine added and removed items when comparing to collections

I am looking for the best algorithm to compare 2 collections and determine which element got added and which element got removed.
private string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement, ICollection<UserInvolvement> oldInvolvement)
{
//I defined the new and old dictionary's for you to know what useful data is inside UserInvolvement.
//Both are Dictionary<int, int>, because The Involvement is just a enum flag. Integer. UserId is also Integer.
var newDict = newInvolvement.ToDictionary(x => x.UserID, x => x.Involvement);
var oldDict = oldInvolvement.ToDictionary(x => x.UserID, x => x.Involvement);
//I Want to compare new to old -> and get 2 dictionaries: added and removed.
var usersAdded = new Dictionary<int, Involvement>();
var usersRemoved = new Dictionary<int, Involvement>();
//What is the best algoritm to accomplish this?
return GetInvolvementLogging(usersAdded, usersRemoved);
}
private string GetInvolvementLogging(Dictionary<int, Involvement> usersAdded, Dictionary<int, Involvement> usersRemoved)
{
//TODO: generate a string based on those dictionaries.
return "Change in userinvolvement: ";
}
Added elements are only in newDict removed only in oldDict
var intersection = newDict.Keys.Intersect(oldDict.Keys);
var added = newDict.Keys.Except(intersection);
var removed = oldDict.Keys.Except(intersection);
EDIT
I modify your base function, dictionaries is no neded.
Example UserInvolvement implementation
class UserInvolvement
{
public int UserId;
public string Name;
public string OtherInfo;
public override bool Equals(object obj)
{
if (obj == null)
{
return false;
}
UserInvolvement p = obj as UserInvolvement;
if ((System.Object)p == null)
{
return false;
}
return (UserId == p.UserId) && (Name == p.Name) && (OtherInfo == p.OtherInfo);
}
public override string ToString()
{
return $"{UserId} - {Name} - {OtherInfo}";
}
}
And example function:
private static string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement,
ICollection<UserInvolvement> oldInvolvement)
{
var intersection = newInvolvement.Select(x => x.UserId).Intersect(oldInvolvement.Select(x => x.UserId));
var addedIds = newInvolvement.Select(x => x.UserId).Except(intersection);
var removedIds = oldInvolvement.Select(x => x.UserId).Except(intersection);
List<UserInvolvement> modifiedUI = new List<UserInvolvement>();
foreach (var i in intersection)
{
var ni = newInvolvement.First(a => a.UserId == i);
var oi = oldInvolvement.First(a => a.UserId == i);
if (!ni.Equals(oi))
{
modifiedUI.Add(ni);
}
}
List<UserInvolvement> addedUI = newInvolvement.Where(x => addedIds.Contains(x.UserId)).Select(w => w).ToList();
List<UserInvolvement> removedUI = oldInvolvement.Where(x => removedIds.Contains(x.UserId)).Select(w => w).ToList();
StringBuilder sb = new StringBuilder();
sb.AppendLine("Added");
foreach (var added in addedUI)
{
sb.AppendLine(added.ToString());
}
sb.AppendLine("Removed");
foreach (var removed in removedUI)
{
sb.AppendLine(removed.ToString());
}
sb.AppendLine("Modified");
foreach (var modified in modifiedUI)
{
sb.AppendLine(modified.ToString());
}
return sb.ToString();
}
And my test function:
static void Main(string[] args)
{
List<UserInvolvement> newUI = new List<UserInvolvement>()
{
new UserInvolvement()
{
UserId = 1,
Name = "AAA",
OtherInfo = "QQQ"
},
new UserInvolvement()
{
UserId = 2,
Name = "BBB",
OtherInfo = "123"
},
new UserInvolvement()
{
UserId = 4,
Name = "DDD",
OtherInfo = "123ert"
}
};
List<UserInvolvement> oldUI = new List<UserInvolvement>()
{
new UserInvolvement()
{
UserId = 2,
Name = "BBBC",
OtherInfo = "123"
},
new UserInvolvement()
{
UserId = 3,
Name = "CCC",
OtherInfo = "QQ44"
},
new UserInvolvement()
{
UserId = 4,
Name = "DDD",
OtherInfo = "123ert"
}
};
string resp = GetInvolvementLogging(newUI, oldUI);
WriteLine(resp);
ReadKey();
WriteLine("CU");
}
Result is:
Added
1 - AAA - QQQ
Removed
3 - CCC - QQ44
Modified
2 - BBB - 123
You could try with Linq:
var usersAdded = newDict.Except(oldDict);
var usersRemoved = oldDict.Except(newDict);
If you need dictionaries as a result you can cast:
var usersAdded = newDict.Except(oldDict).ToDictionary(x => x.Key, x => x.Value);
var usersRemoved = oldDict.Except(newDict).ToDictionary(x => x.Key, x => x.Value);
Think best algorithm will be
foreach (var newItem in newDict)
if (!oldDict.ContainsKey(newItem.Key) || oldDict[newItem.Key]!=newItem.Value)
usersAdded.Add(newItem.Key, newItem.Value);
foreach (var oldItem in oldDict)
if (!newDict.ContainsKey(oldItem.Key) || newDict[oldItem.Key]!=oldItem.Value)
usersRemoved.Add(oldItem.Key, oldItem.Value);
Finally this is my implementation of GetInvolvementLogging:
(the implementation of the string builder method is irrelevant for my question here)
private string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement, ICollection<UserInvolvement> oldInvolvement)
{
//I defined the new and old dictionary's to focus on the relevant data inside UserInvolvement.
var newDict = newInvolvement.ToDictionary(x => x.UserID, x => (Involvement)x.Involvement);
var oldDict = oldInvolvement.ToDictionary(x => x.UserID, x => (Involvement)x.Involvement);
var intersection = newDict.Keys.Intersect(oldDict.Keys); //These are the id's of the users that were and remain involved.
var usersAdded = newDict.Keys.Except(intersection);
var usersRemoved = oldDict.Keys.Except(intersection);
var addedInvolvement = newDict.Where(x => usersAdded.Contains(x.Key)).ToDictionary(x => x.Key, x => x.Value);
var removedInvolvement = oldDict.Where(x => usersRemoved.Contains(x.Key)).ToDictionary(x => x.Key, x => x.Value);
//Check if the already involved users have a changed involvement.
foreach(var userId in intersection)
{
var newInvolvementFlags = newDict[userId];
var oldInvolvementFlags = oldDict[userId];
if ((int)newInvolvementFlags != (int)oldInvolvementFlags)
{
var xor = newInvolvementFlags ^ oldInvolvementFlags;
var added = newInvolvementFlags & xor;
var removed = oldInvolvementFlags & xor;
if (added != 0)
{
addedInvolvement.Add(userId, added);
}
if (removed != 0)
{
removedInvolvement.Add(userId, removed);
}
}
}
return GetInvolvementLogging(addedInvolvement, removedInvolvement);
}

How can I select the last digit of an integer in a LINQ .select?

I have this LINQ select:
var extendedPhrases = phrases
.Select(x => new ExtendedPhrase()
{
Ajlpt = x.Ajlpt,
Bjlpt = x.Bjlpt,
Created = x.Created // an int?
});
If I define:
public int? CreatedLast { get; set; }
Then how can I populate that with the last digit of x.Created?
If you are looking for the last digit of the Created property, the use the % operator like this:
var extendedPhrases = phrases
.Select(x => new ExtendedPhrase()
{
Ajlpt = x.Ajlpt,
Bjlpt = x.Bjlpt,
Created = x.Created,
CreatedLast = x.Created % 10
});
The first way to come to mind is to call .ToString().Last():
var extendedPhrases = phrases
.Select(x => new ExtendedPhrase()
{
Ajlpt = x.Ajlpt,
Bjlpt = x.Bjlpt,
Created = x.Created,
CreatedLast = x.Created?.ToString().Last()
});
If you aren't using the latest shiny C#, then null protection can be done with:
var extendedPhrases = phrases
.Select(x => new ExtendedPhrase()
{
Ajlpt = x.Ajlpt,
Bjlpt = x.Bjlpt,
Created = x.Created,
CreatedLast = x.Created.HasValue ? x.Created.ToString().Last() : null
});
And some conversion back to an int? left as an exercise to the reader.

linq group by contiguous blocks

Let's say I have following data:
Time Status
10:00 On
11:00 Off
12:00 Off
13:00 Off
14:00 Off
15:00 On
16:00 On
How could I group that using Linq into something like
[On, [10:00]], [Off, [11:00, 12:00, 13:00, 14:00]], [On, [15:00, 16:00]]
Create a GroupAdjacent extension, such as the one listed here.
And then it's as simple as:
var groups = myData.GroupAdjacent(data => data.OnOffStatus);
You could also do this with one Linq query using a variable to keep track of the changes, like this.
int key = 0;
var query = data.Select(
(n,i) => i == 0 ?
new { Value = n, Key = key } :
new
{
Value = n,
Key = n.OnOffFlag == data[i - 1].OnOffFlag ? key : ++key
})
.GroupBy(a => a.Key, a => a.Value);
Basically it assigns a key for each item that increments when the current item does not equal the previous item. Of course this assumes that your data is in a List or Array, otherwise you'd have to try a different method
Here is a hardcore LINQ solution by using Enumerable.Zip to compare contiguous elements and generate a contiguous key:
var adj = 0;
var t = data.Zip(data.Skip(1).Concat(new TimeStatus[] { null }),
(x, y) => new { x, key = (x == null || y == null || x.Status == y.Status) ? adj : adj++ }
).GroupBy(i => i.key, (k, g) => g.Select(e => e.x));
It can be done as.
Iterate over collection.
Use TakeWhile<Predicate> condition is text of first element of collection On or Off.
Iterate over the subset of from point one and repeat above step and concatenate string.
Hope it helps..
You could parse the list and assign a contiguous key e.g define a class:
public class TimeStatus
{
public int ContiguousKey { get; set; }
public string Time { get; set; }
public string Status { get; set; }
}
You would assign values to the contiguous key by looping through, maintaining a count and detecting when the status changes from On to Off and so forth which would give you a list like this:
List<TimeStatus> timeStatuses = new List<TimeStatus>
{
new TimeStatus { ContiguousKey = 1, Status = "On", Time = "10:00"},
new TimeStatus { ContiguousKey = 1, Status = "On", Time = "11:00"},
new TimeStatus { ContiguousKey = 2, Status = "Off", Time = "12:00"},
new TimeStatus { ContiguousKey = 2, Status = "Off", Time = "13:00"},
new TimeStatus { ContiguousKey = 2, Status = "Off", Time = "14:00"},
new TimeStatus { ContiguousKey = 3, Status = "On", Time = "15:00"},
new TimeStatus { ContiguousKey = 3, Status = "On", Time = "16:00"}
};
Then using the following query you can extract the Status and grouped Times:
var query = timeStatuses.GroupBy(t => t.ContiguousKey)
.Select(g => new { Status = g.First().Status, Times = g });

Categories