How to increase SQL parser speed that using TSqlParser? - c#

In my code, I have several SQL scripts that I want to parse using TSql130Parser in order to refactor them. But this process takes a long time - is there a way to increase the speed of this?
Also I have another question that TSql130Parser can get bulk as parameter? If it is possible, how can I do it?
This is my method that I use for parsing procedures by passing content of procedure as query in method parameter:
public string ParseProcedureQuery(string query, out string procedureName)
{
procedureName= "";
try
{
TSql130Parser parser = new TSql130Parser(true);
IList<ParseError> errors;
var fragments = parser.Parse(new System.IO.StringReader(query), out errors);
var proc = fragments.ScriptTokenStream.Where(x => x.TokenType == TSqlTokenType.Proc || x.TokenType == TSqlTokenType.Procedure).Select(x => x.Text).FirstOrDefault();
if (proc == null)
return string.Empty;
var index = fragments.ScriptTokenStream.FindIndex(c => c.Text == proc);
var name = fragments.ScriptTokenStream
.Select((value, ind) => new { value, index = ind })
.Where(pair => pair.index > index && pair.value.TokenType != TSqlTokenType.WhiteSpace)
.Select((value, ind) => new { value.value.Text, index = ind }).Where(y => y.index < 3).Select(p => p.Text);
procedureName = string.Join(String.Empty, name);
var processedQuery = string.Join(" ", fragments.ScriptTokenStream
.Where(x => x.TokenType != TSqlTokenType.MultilineComment)
.Where(x => x.TokenType != TSqlTokenType.SingleLineComment)
.Where(x => x.TokenType != TSqlTokenType.WhiteSpace)
.Select(x => x.Text));
return processedQuery;
}
catch
{ }
}

Related

in Linq replace null to empty

var ch = _context.xxtu_nintex_emp_data_v
.Where(o => o.LOGIN_USER_NAME ==userId.ToUpper())
.Select(emp => new
{ OTHERMOBILENO = emp.OTHERMOBILENO ?? "" })
.ToList().SingleOrDefault();
the result is
when use toList() I get what I want but it is so slow
var ch = _context.xxtu_nintex_emp_data_v.ToList()
.Where(o => o.LOGIN_USER_NAME ==userId.ToUpper())
.Select(emp => new
{ OTHERMOBILENO = emp.OTHERMOBILENO ?? "" })
.ToList().SingleOrDefault();
the result is
I use code first approach api and Microsoft.EntityFrameworkCore
Some things you could try
Call ToList after the condition, to only fetch a subset of data from the Database
var ch = _context.xxtu_nintex_emp_data_v
.Where(o => o.LOGIN_USER_NAME == userId.ToUpper())
.ToList() //<-- here
.Select(emp => new
{ OTHERMOBILENO = emp.OTHERMOBILENO ?? "" })
.ToList().SingleOrDefault();
Move the Select after the Last ToList call
var ch = _context.xxtu_nintex_emp_data_v
.Where(o => o.LOGIN_USER_NAME == userId.ToUpper())
//<-- re-arrange ToList/Select
.ToList()
.Select(emp => new { OTHERMOBILENO = emp.OTHERMOBILENO ?? "" })
//<--
.SingleOrDefault();
Re-arrange it a bit more
var emp = _context.xxtu_nintex_emp_data_v
.Where(o => o.LOGIN_USER_NAME == userId.ToUpper())
.SingleOrDefault();
var ch = new { OTHERMOBILENO = emp?.OTHERMOBILENO ?? "" };

Summing a decimal field using Linq to datatable

I was able to get this to work after much frustration, but I was hoping someone could explain to me why this works:
var monthCharges = WRVUData.AsEnumerable()
.Where(dr => dr.Field<DateTime>("Posting_Month") == new DateTime(month.Year, i, 1))
.Where(dr => dr.Field<string>("Report_Producer_Name") == provider || dr.Field<string>("Group_Name") == group)
.Where(dr => dr.Field<decimal?>("Charges") != null)
.Sum(dr => dr.Field<decimal>("Charges"));
but this doesn't:
var monthCharges = (from r in WRVUData.AsDataView()
select new
{
charges = r.Field<decimal?>("Charges"),
activityType = r.Field<string>("Activity_Type"),
postingMonth = r.Field<DateTime>("Posting_Month"),
provider = r.Field<string>("Report_Producer_Name"),
groupName = r.Field<string>("Group_Name"),
WRVUs = r.Field<decimal>("W_RVUs")
})
.Where(dr => dr.postingMonth == new DateTime(month.Year, i, 1))
.Where(dr => dr.provider == provider || dr.groupName == group)
.Where(dr => dr.charges != null)
.Sum(dr => dr.charges);
The second version throws a "Specified cast is not valid" error. Aren't the two basically the same thing?

How do I create and populate a dynamic object using a dynamically built lambda expression

I'm trying to create and populate a dynamic object from a dataset that is only known at run time. In the code below, I create my IEnumerable results from my dataset with some known fields (ID, Primary Data, DisplayOrder, IsActive) and one user-define field (Phone Number) which I won't know at design time, and therefore must be built dynamically. The code below works, but only because I've hard-coded the dynamic field Phone Number. How do I build the Lambda expression dynamically to handle those fields only known at runtime. I want the equivalent of
string fieldName = 'PhoneNumber = ';
string fieldSource = 'pd.tbList_DataText';
string expression = 'pd=>new { ID = pd.ID, PrimaryData=pd.PrimaryData';
expression += fieldName;
expression += FieldSource;
expression += '.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First()})';
var results = primaryData.Select(expression);
So how do I make that work? Thanks
// Get base Data
IEnumerable<tbList_Data> primaryData = await _tbList_DataRepository.GetAsync(ld => ld.ListID == listId && (ld.IsActive == includeInactive ? ld.IsActive : true));
// Get Final Results
var results = primaryData.Select(pd => new {
Id = pd.ID,
PrimaryData = pd.PrimaryData,
PhoneNumber = pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First()
});
I see several options.
1) Tuple
var results = primaryData.Select(pd => new {
Id = pd.ID,
PrimaryData = pd.PrimaryData,
Extra = Tuple.Create("PhoneNumber", pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First())
});
// How to access:
foreach (var result in results)
{
Console.WriteLine(result.Id);
Console.WriteLine(result.PrimaryData);
Console.WriteLine(result.Extra.Item1);
Console.WriteLine(result.Extra.Item2);
}
2) Dictionary
// Using C# 6 notation
var results = primaryData.Select(pd => new Dictionary<string, object>{
["Id"] = pd.ID,
["PrimaryData"] = pd.PrimaryData,
["PhoneNumber"] = pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First()
});
// Using C# 5 notation
var results = primaryData.Select(pd => new Dictionary<string, object>{
{"Id", pd.ID},
{"PrimaryData", pd.PrimaryData},
{"PhoneNumber", pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First()}
});
// How to access:
foreach(var result in results)
{
Console.WriteLine(result["Id"]);
Console.WriteLine(result["PrimaryData"]);
Console.WriteLine(result["PhoneNumber"]);
}
3) Dynamic
var results = primaryData.Select(pd => {
dynamic result = new System.Dynamic.ExpandoObject();
result.Id = pd.ID;
result.PrimaryData = pd.PrimaryData;
// Choose one of the following. Since you only "PhoneNumber" at runtime, probably the second one.
result.PhoneNumber = pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First();
((IDictionary<string, object>)result).Add("PhoneNumber", pd.tbList_DataText.Where(ld => ld.DataRowID == pd.ID && ld.ListColumnID == 1).Select(ld => ld.DataField).DefaultIfEmpty("").First());
return result;
});
// How to access:
foreach(var result in results)
{
Console.WriteLine(result.Id);
Console.WriteLine(result.PrimaryData);
// Both work, independently how you created them
Console.WriteLine(result.PhoneNumber);
Console.WriteLine(((IDictionary<string, object>)result)["PhoneNumber"]);
}
EDIT: just realized from question that the field source should be dynamic as well. So, in the above code, replace any occurrence of pb.tbList_DataText by:
((IEnumerable<X>)pb.GetType().GetField("tbList_DataText").GetValue(pb))
Where X should be the type of ld. But carefull! This cast can potentially fail.
Also, if you want a property instead of a field, just use GetProperty instead of GetField.

Union on empty Enumerable

I have function
public async Task<IQueryable<Document>> GetDocuments(...)
in which I search for documents under some given conditions. Some conditions can be skipped. At the end I perform union of these queries.
var documents = await documentService.GetDocuments(this, userId,
roleShowFullNumber, param.OrderColName(), param.SearchValue, filter);
var usersGroupsId = filter.UsersGroupsId;
if (usersGroupsId != null)
{
if (!usersGroupsId.Contains("All"))
{
IQueryable<Document> myDocs = Enumerable.Empty<Document>().AsQueryable();
if (usersGroupsId.Contains("myOrders"))
{
myDocs = documents.Where(x => x.OwnerId == userId || x.UserId == userId);
usersGroupsId = usersGroupsId.Where(x => x != "myOrders").ToArray();
}
IQueryable<Document> wards = Enumerable.Empty<Document>().AsQueryable();
if (usersGroupsId.Contains("wards"))
{
var relatedUserId = _db.Users.Where(x => x.Id == userId).Select(x => x.RelatedUserId).FirstOrDefault();
if (relatedUserId != null)
{
var myWards = _db.kh__Kontrahent.Where(x => x.kh_IdOpiekun == relatedUserId);
var myWardsUsers = _db.Users.Where(x => myWards.Any(w => w.kh_Id == (x.RelatedCustomerId == null ? -1 : x.RelatedCustomerId)));
wards = documents.Where(x => (myWardsUsers.Any(w => x.UserId == w.Id) || myWardsUsers.Any(w => x.OwnerId == w.Id)));
usersGroupsId = usersGroupsId.Where(x => x != "wards").ToArray();
}
}
IQueryable<Document> groups = Enumerable.Empty<Document>().AsQueryable();
if (usersGroupsId.Length > 0)
{
var usersGroups = _db.Groups.Where(x => usersGroupsId.Contains(x.Id.ToString()));
var usersList = usersGroups.Select(x => x.Users);
var users = usersList.SelectMany(x => x);
var usersId = users.Select(x => x.Id);
groups = _db.Documents.Where(x => (usersId.Any(u => u == x.OwnerId) || usersId.Any(u => u == x.UserId)));
}
documents = myDocs.Union(wards).Union(groups);
}
}
But if one of these partial queries is empty (was skipped) when I try obtain these documents in way shown below I got error.
var documentsPaginated = await documents.Skip(param.Start)
.Take(param.Length)
.ToListAsync();
Error: The source IQueryable doesn't implement IDbAsyncEnumerable.
How can I make this function to be able to skip some sub queries and then union all. I would prefer not to change function return value.
Try this, althought your code seems to have a massive amount of code smell...
public async Task<IQueryable<Document>> GetDocuments(...)
var documents = await documentService.GetDocuments(this, userId,
roleShowFullNumber, param.OrderColName(), param.SearchValue, filter);
var usersGroupsId = filter.UsersGroupsId;
if (usersGroupsId != null)
{
if (!usersGroupsId.Contains("All"))
{
IQueryable<Document> myDocs = null;
if (usersGroupsId.Contains("myOrders"))
{
myDocs = documents.Where(x => x.OwnerId == userId || x.UserId == userId);
usersGroupsId = usersGroupsId.Where(x => x != "myOrders").ToArray();
}
IQueryable<Document> wards = null;
if (usersGroupsId.Contains("wards"))
{
var relatedUserId = _db.Users.Where(x => x.Id == userId).Select(x => x.RelatedUserId).FirstOrDefault();
if (relatedUserId != null)
{
var myWards = _db.kh__Kontrahent.Where(x => x.kh_IdOpiekun == relatedUserId);
var myWardsUsers = _db.Users.Where(x => myWards.Any(w => w.kh_Id == (x.RelatedCustomerId == null ? -1 : x.RelatedCustomerId)));
wards = documents.Where(x => (myWardsUsers.Any(w => x.UserId == w.Id) || myWardsUsers.Any(w => x.OwnerId == w.Id)));
usersGroupsId = usersGroupsId.Where(x => x != "wards").ToArray();
}
}
IQueryable<Document> groups = null;
if (usersGroupsId.Length > 0)
{
var usersGroups = _db.Groups.Where(x => usersGroupsId.Contains(x.Id.ToString()));
var usersList = usersGroups.Select(x => x.Users);
var users = usersList.SelectMany(x => x);
var usersId = users.Select(x => x.Id);
groups = _db.Documents.Where(x => (usersId.Any(u => u == x.OwnerId) || usersId.Any(u => u == x.UserId)));
}
if(myDocs != null)
documents = documents.Union(myDocs);
if(wards != null)
documents = documents.Union(wards);
if(groups != null)
documents = documents.Union(groups);
}
}
It appears that ToListAsync can't be used interchangeably with linq, but only in an EF query.
Quote from MSDN
Entity Framework 6 introduced a set of extension methods that can be used to asynchronously execute a query. Examples of these methods include ToListAsync, FirstAsync, ForEachAsync, etc.
Because Entity Framework queries make use of LINQ, the extension methods are defined on IQueryable and IEnumerable. However, because they are only designed to be used with Entity Framework you may receive the following error if you try to use them on a LINQ query that isn’t an Entity Framework query.
The source IQueryable doesn't implement IDbAsyncEnumerable{0}. Only sources that implement IDbAsyncEnumerable can be used for Entity Framework asynchronous operations. For more details see http://go.microsoft.com/fwlink/?LinkId=287068.

How to combine the multiple part linq into one query?

Operator should be ‘AND’ and not a ‘OR’.
I am trying to refactor the following code and i understood the following way of writing linq query may not be the correct way. Can somone advice me how to combine the following into one query.
AllCompany.Where(itm => itm != null).Distinct().ToList();
if (AllCompany.Count > 0)
{
//COMPANY NAME
if (isfldCompanyName)
{
AllCompany = AllCompany.Where(company => company["Company Name"].StartsWith(fldCompanyName)).ToList();
}
//SECTOR
if (isfldSector)
{
AllCompany = AllCompany.Where(company => fldSector.Intersect(company["Sectors"].Split('|')).Any()).ToList();
}
//LOCATION
if (isfldLocation)
{
AllCompany = AllCompany.Where(company => fldLocation.Intersect(company["Location"].Split('|')).Any()).ToList();
}
//CREATED DATE
if (isfldcreatedDate)
{
AllCompany = AllCompany.Where(company => company.Statistics.Created >= createdDate).ToList();
}
//LAST UPDATED DATE
if (isfldUpdatedDate)
{
AllCompany = AllCompany.Where(company => company.Statistics.Updated >= updatedDate).ToList();
}
//Allow Placements
if (isfldEmployerLevel)
{
fldEmployerLevel = (fldEmployerLevel == "Yes") ? "1" : "";
AllCompany = AllCompany.Where(company => company["Allow Placements"].ToString() == fldEmployerLevel).ToList();
}
Firstly, unless AllCompany is of some magic custom type, the first line gives you nothing.
Also I have a doubt that Distinctworks the way You want it to. I don't know the type of AllCompany but I would guess it gives you only reference distinction.
Either way here'w what I think You want:
fldEmployerLevel = (fldEmployerLevel == "Yes") ? "1" : "";
var result = AllCompany.Where(itm => itm != null)
.Where(company => !isfldCompanyName || company["Company Name"].StartsWith(fldCompanyName))
.Where(company => !isfldSector|| fldSector.Intersect(company["Sectors"].Split('|')).Any())
.Where(company => !isfldLocation|| fldLocation.Intersect(company["Location"].Split('|')).Any())
.Where(company => !isfldcreatedDate|| company.Statistics.Created >= createdDate)
.Where(company => !isfldUpdatedDate|| company.Statistics.Updated >= updatedDate)
.Where(company => !isfldEmployerLevel|| company["Allow Placements"].ToString() == fldEmployerLevel)
.Distinct()
.ToList();
Edit:
I moved Distinct to the end of the query to optimize the processing.
How about trying like this;
AllCompany = AllCompany .Where(company => (company => company.Statistics.Created >= createdDate)) && (company.Statistics.Updated >= updatedDate));
If every part of query is optional (like created date, last update date..) then you can build linq query string.
Here's a sneaky trick. If you define the following extension method in its own static class:
public virtual IEnumerable<T> WhereAll(params Expression<Predicate<T> filters)
{
return filters.Aggregate(dbSet, (acc, element) => acc.Where(element));
}
then you can write
var result = AllCompany.WhereAll(itm => itm != null,
company => !isfldCompanyName || company["Company Name"].StartsWith(fldCompanyName),
company => !isfldSectorn || fldSector.Intersect(company["Sectors"].Split('|')).Any(),
company => !isfldLocation || fldLocation.Intersect(company["Location"].Split('|')).Any(),
company => !isfldcreatedDate || company.Statistics.Created >= createdDate,
company => !isfldUpdatedDate || company.Statistics.Updated >= updatedDate,
company => !isfldEmployerLevel || company["Allow Placements"].ToString() == fldEmployerLevel)
.Distinct()
.ToList();

Categories