We Keep Coding

how to optimize the time in executing the query in c# react

I am new to React and new to Web API. I am uploading data in a tabulator in react front end from the value that I am passing through the web API. I am passing value through the getReports function like this:
[HttpPost]
[Route("GetReports")]
public IHttpActionResult GetReports(string jwt, List<object> data)
{
if (!Common.VerificationToken.VerifyJWToken(jwt))
{
return null;
}
var to = data[0];
var from = data[1];
DateTime toDate = Convert.ToDateTime(to);
DateTime fromDate = Convert.ToDateTime(from);
var ReportData = db.T_CQL_COIL_DESC.Where(t => t.CCD_CREATED_ON >= toDate &&
t.CCD_CREATED_ON <= fromDate).ToList();
ReportsDTO dto = new ReportsDTO();
List<ReportsDTO> ReportDTO = new List<ReportsDTO>();
try
{
foreach (var report in ReportData)
{
List<vehicledetail> vehicle = new List<vehicledetail>();
var imgurl = "https://firebasestorage.googleapis.com/v0/b/tsl-coil-qlty-
monitoring-dev.appspot.com/";
dto = new ReportsDTO();
dto.Type = report.CCD_OP_TYPE;
dto.ID = report.CCD_COIL_ID;
vehicle = GetVehicleID(dto.ID);
vehicledetail vehicledetails = vehicle[0];
dto.vehicleno = vehicledetails.vehicleno.ToString();
dto.wagonno = vehicledetails.wagonno.ToString();
dto.Active = report.CCD_ACTIVE;
dto.ImgURL = report.CCD_IMAGE_URL != null ? imgurl + report.CCD_IMAGE_URL : "NA";
dto.Desc = report.CCD_VIEW_DESC != null ? report.CCD_VIEW_DESC : "NA";
ReportDTO.Add(dto);
}
return Ok(ReportDTO);
}
catch (Exception ex)
{
return Content(HttpStatusCode.NoContent, "Something went wrong");
}
}
The data in vehicledetail in vehicledetail vehicledetails = vehicle[0]; is getting populated from this function:
public List<vehicledetail> GetVehicleID(string coilID)
{
List<vehicledetail> vehicle = new List<vehicledetail>();
vehicledetail vehicledetails = new vehicledetail();
string oradb = Utilities.STAR_DB;
OracleConnection conn = new OracleConnection(oradb);
string query = "SELECT a.Vbeln, b.WAGON_NO FROM sapr3.lips a, sapr3.ZVTRRDA b WHERE
a.MANDT='600' AND a.CHARG='" + coilID + "' AND a.LFIMG > 0 AND a.MANDT = b.MANDT AND
a.VBELN = b.VBELN";
OracleDataAdapter da = new OracleDataAdapter(query, conn);
conn.Open();
DataTable dt = new DataTable();
da.Fill(dt);
foreach (DataRow row in dt.Rows)
{
vehicledetails.vehicleno = row["VBELN"].ToString();
vehicledetails.wagonno = row["WAGON_NO"].ToString();
}
conn.Close();
vehicle.Add(vehicledetails);
return (vehicle);
}
It is working fine but it is taking 30 seconds to load the below data:
How do I optimize this . Please help. Note: that it is taking 30 seconds to upload this data

Few other things aside, the major problem seems to be that you are querying database for each vehicle.
In this particular scenario it might very well be better to select all vehicle ids and query them all.
var vehicleIds = reportData.SelectMany(t => t.ID);
You can then form a query that will get all vehicle details together. This will reduce the number of database calls and it can have massive impact on time.
Another thing to check is if there's any index created on the vehicle id column in database as that may also help speed things up.

Update all null values in column .netcore console app

I've got a console app which updates a column in the database called InstagramId based off user input. I want to be able to query the database and if there's a instagramId which is null, to update it with the correct instagramID based off their instagramUsername.
I wasn't sure on the approach if I should create a fake table based off all the null instagramIds and then use those to update my column or if theres another cleaner approach. This console app is only a one time use so just a quick fix up.
class Program
{
static async Task Main(string[] args)
{
// receive a profile name
var tasks = new List<Task<InstagramUser>>();
foreach (var arg in args)
{
if (string.IsNullOrEmpty(arg)) continue;
var profileName = arg;
var url = $"https://instagram.com/{profileName}";
tasks.Add(ScrapeInstagram(url));
}
try
{
var instagramUsers = await Task.WhenAll<InstagramUser>(tasks);
foreach (var iu in instagramUsers)
{
iu.Display();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
public static async Task<InstagramUser> ScrapeInstagram(string url)
{
using (var client = new HttpClient())
{
var response = await client.GetAsync(url);
if (response.IsSuccessStatusCode)
{
// create html document
var htmlBody = await response.Content.ReadAsStringAsync();
var htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(htmlBody);
// select script tags
var scripts = htmlDocument.DocumentNode.SelectNodes("/html/body/script");
// preprocess result
var uselessString = "window._sharedData = ";
var scriptInnerText = scripts[0].InnerText
.Substring(uselessString.Length)
.Replace(";", "");
// serialize objects and fetch the user data
dynamic jsonStuff = JObject.Parse(scriptInnerText);
dynamic userProfile = jsonStuff["entry_data"]["ProfilePage"][0]["graphql"]["user"];
//Update database query
string connectionString = #"Server=mockup-dev-db";
using (SqlConnection con = new SqlConnection(connectionString))
{
SqlCommand cmd = new SqlCommand("Update ApplicationUser Set InstagramId = '" + userProfile.id + "'" + "where Instagram = '" + userProfile.username + "'", con);
cmd.Connection.Open();
cmd.ExecuteNonQuery();
}
// create an InstagramUser
var instagramUser = new InstagramUser
{
FullName = userProfile.full_name,
FollowerCount = userProfile.edge_followed_by.count,
FollowingCount = userProfile.edge_follow.count,
Id = userProfile.id,
url = url
};
return instagramUser;
}
else
{
throw new Exception($"Something wrong happened {response.StatusCode} - {response.ReasonPhrase} - {response.RequestMessage}");
}
}
}
}
}

It seems like there are two problems here
Retrieving a list of username from the DB which have null instagram IDs
For each of these, downloading and populating the instagram ID
It looks like you've already solved 1, so let's look at number 2:
This data can be fetched using an SQL query:
SELECT Instagram FROM ApplicationUser WHERE InstagramId IS NULL
In C# this could look like:
using (SqlConnection con = new SqlConnection(connectionString))
{
SqlCommand cmd = new SqlCommand("SELECT Instagram FROM ApplicationUser WHERE InstagramId IS NULL", con);
cmd.Connection.Open();
var instagramUsernames = new List<string>();
SqlDataReader reader = command.ExecuteReader();
while (reader.Read())
{
instagramUsernames.Add(reader.GetString(0));
}
}
Now you have all the usernames in the instagramUsernames list and can execute your existing code (with minor modifications) in a foreach loop.
Instead of looping through the args you'd now loop through the list of instagramUsernames (so the above code would come at the start of the Main method):
foreach (var arg in args)
Would become foreach (var arg in instagramUsernames)

IBM db2Command SQL SELECT * FROM IN #namedParameter

When using the C# code below to construct a DB2 SQL query the result set only has one row. If I manually construct the "IN" predicate inside the cmdTxt string using string.Join(",", ids) then all of the expected rows are returned. How can I return all of the expected rows using the db2Parameter object instead of building the query as a long string to be sent to the server?
public object[] GetResults(int[] ids)
{
var cmdTxt = "SELECT DISTINCT ID,COL2,COL3 FROM TABLE WHERE ID IN ( #ids ) ";
var db2Command = _DB2Connection.CreateCommand();
db2Command.CommandText = cmdTxt;
var db2Parameter = db2Command.CreateParameter();
db2Parameter.ArrayLength = ids.Length;
db2Parameter.DB2Type = DB2Type.DynArray;
db2Parameter.ParameterName = "#ids";
db2Parameter.Value = ids;
db2Command.Parameters.Add(db2Parameter);
var results = ExecuteQuery(db2Command);
return results.ToArray();
}
private object[] ExecuteQuery(DB2Command db2Command)
{
_DB2Connection.Open();
var resultList = new ArrayList();
var results = db2Command.ExecuteReader();
while (results.Read())
{
var values = new object[results.FieldCount];
results.GetValues(values);
resultList.Add(values);
}
results.Close();
_DB2Connection.Close();
return resultList.ToArray();
}

You cannot send in an array as a parameter. You would have to do something to build out a list of parameters, one for each of your values.
e.g.: SELECT DISTINCT ID,COL2,COL3 FROM TABLE WHERE ID IN ( #id1, #id2, ... #idN )
And then add the values to your parameter collection:
cmd.Parameters.Add("#id1", DB2Type.Integer).Value = your_val;
Additionally, there are a few things I would do to improve your code:
Use using statements around your DB2 objects. This will automatically dispose of the objects correctly when they go out of scope. If you don't do this, eventually you will run into errors. This should be done on DB2Connection, DB2Command, DB2Transaction, and DB2Reader objects especially.
I would recommend that you wrap queries in a transaction object, even for selects. With DB2 (and my experience is with z/OS mainframe, here... it might be different for AS/400), it writes one "accounting" record (basically the work that DB2 did) for each transaction. If you don't have an explicit transaction, DB2 will create one for you, and automatically commit after every statement, which adds up to a lot of backend records that could be combined.
My personal opinion would also be to create a .NET class to hold the data that you are getting back from the database. That would make it easier to work with using IntelliSense, among other things (because you would be able to auto-complete the property name, and .NET would know the type of the object). Right now, with the array of objects, if your column order or data type changes, it may be difficult to find/debug those usages throughout your code.
I've included a version of your code that I re-wrote that has some of these changes in it:
public List<ReturnClass> GetResults(int[] ids)
{
using (var conn = new DB2Connection())
{
conn.Open();
using (var trans = conn.BeginTransaction(IsolationLevel.ReadCommitted))
using (var cmd = conn.CreateCommand())
{
cmd.Transaction = trans;
var parms = new List<string>();
var idCount = 0;
foreach (var id in ids)
{
var parm = "#id" + idCount++;
parms.Add(parm);
cmd.Parameters.Add(parm, DB2Type.Integer).Value = id;
}
cmd.CommandText = "SELECT DISTINCT ID,COL2,COL3 FROM TABLE WHERE ID IN ( " + string.Join(",", parms) + " ) ";
var resultList = new List<ReturnClass>();
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
var values = new ReturnClass();
values.Id = (int)reader["ID"];
values.Col1 = reader["COL1"].ToString();
values.Col2 = reader["COL2"].ToString();
resultList.Add(values);
}
}
return resultList;
}
}
}
public class ReturnClass
{
public int Id;
public string Col1;
public string Col2;
}

Try changing from:
db2Parameter.DB2Type = DB2Type.DynArray;
to:
db2Parameter.DB2Type = DB2Type.Integer;
This is based on the example given here

How to check linq query result against Sql Server Database for data already exist or not?

I am using active directory for getting all the departsment and filtering distinct departments using linq query, below is my code
private static DomainController GetDomainController(string domainpath)
{
var domainContext = new DirectoryContext(DirectoryContextType.Domain, domainpath);
var domain = Domain.GetDomain(domainContext);
var controller = domain.FindDomainController();
return controller;
}
private static MyMethod()
{
var domainController = GetDomainController(ActiveDirectorySettings.DomainPath);
// Lookup the information in AD
var ldapEntry = new DirectoryEntry(string.Format("LDAP://{0}", domainController)) { AuthenticationType = AuthenticationTypes.Secure | AuthenticationTypes.FastBind };
DirectorySearcher ds;
ds = new DirectorySearcher(ldapEntry)
{
SearchScope = SearchScope.Subtree,
Filter = "(&" + "(objectClass=user)" + "(department=" + departmentname + "*))"
};
ds.PropertiesToLoad.Add("department");
if (ds.FindAll().Count >= 1)
{
//DataSet du = DataReader.CheckAdUserExist();
var results = ds.FindAll();
var uniqueSearchResults = results.Cast<SearchResult>().Select(x => GetProperty(x,"department")).Distinct();
addUsersList.AddRange(uniqueSearchResults.Select(departmentName => new UsersAndDepartments
{
UserDepartment = departmentName
}));
}
}
I want to check the linq query result with the database whether department already exist or not, I am not sure how to do that?

If what you want is to create a simple database connection using SqlConnection you just need to query your DB using the department parameter you received from your AD request.
try{
SqlConnection connection = new SqlConnection("YourConnectionstring");
connection.Open();
//Your connection string can be found through your Server DB
//Now you go through your SearchResultCollection populated by SearchResult objects
foreach(SearchResult res in uniqueSearchResult){
SqlCommand cmd = new SqlCommand("Select * from yourTable where department=" +res.Properties["department"][0].ToString() + "", connection);
SqlDataReader reader = cmd.ExecuteReader();
//Here you verify if there are corresponding rows in your table
//with the request you have executed
if(!reader.HasRows()){
//If you have not found corresponding rows, then you add the department to your
//list
addUsersList.Add(res.Properties["department"][0].ToString());
}
}
connection.close();
}catch(Exception e){
Console.WriteLine("Exception caught : \n\n" + e.ToString();
}
This should work.
There are plenty of tutorials for this, but if you are making alot of requests I do not recommend using this connection method you will just lose too much time / organization, maybe try using a persistence Framework like Entity Framework :
https://www.codeproject.com/Articles/4416/Beginners-guide-to-accessing-SQL-Server-through-C
Hope this answers your question!

Here is my solution, I have solved it myself
private static DomainController GetDomainController(string domainpath)
{
var domainContext = new DirectoryContext(DirectoryContextType.Domain, domainpath);
var domain = Domain.GetDomain(domainContext);
var controller = domain.FindDomainController();
return controller;
}
private static MyMethod()
{
var domainController = GetDomainController(ActiveDirectorySettings.DomainPath);
// Lookup the information in AD
var ldapEntry = new DirectoryEntry(string.Format("LDAP://{0}", domainController)) { AuthenticationType = AuthenticationTypes.Secure | AuthenticationTypes.FastBind };
DirectorySearcher ds;
ds = new DirectorySearcher(ldapEntry)
{
SearchScope = SearchScope.Subtree,
Filter = "(&" + "(objectClass=user)" + "(department=" + departmentname + "*))"
};
ds.PropertiesToLoad.Add("department");
if (ds.FindAll().Count >= 1)
{
// getting list of all departments from the database
var departmentsList = AllDepartments();
// getting list of all departments from active directory
var results = ds.FindAll();
// filtering distinct departments from the result
var uniqueSearchResults = results.Cast<SearchResult>().Select(x => GetProperty(x,"department")).Distinct();
// here firstly i am getting the department list from the database and checking it for null, then using linq query i am comparing the result with ad department results
if (departmentsList != null)
{
addUsersList.AddRange(from sResultSet in uniqueSearchResults
where !departmentsList.Exists(u => u.UserDepartment == sResultSet)
select new UsersAndDepartments
{
UserDepartment = sResultSet
});
}
else
{
addUsersList.AddRange(uniqueSearchResults.Select(departmentName => new UsersAndDepartments
{
UserDepartment = departmentName
}));
}
}
}

Convert Function Based on TSQL with Function Based with Entity Framework

I have two functions that each return the same list of objects. But, the one that uses TSQL is much faster than the one using Entity Framework and I do not understand why one would be faster than the other. Is it possible to modify my EF function to work as fast as the TSQL one?
Any help will be appreciated. My code is below:
TSQL:
public static List<ChartHist> ListHistory_PureSQL()
{
List<DataRow> listDataRow = null;
string srtQry = #"Select LoginHistoryID,
LoginDuration as LoginDuration_Pass,
0 as LoginDuration_Fail,
LoginDateTime,
LoginLocationID,
LoginUserEmailID,
LoginApplicationID,
LoginEnvironmentID,
ScriptFrequency,
LoginStatus,
Reason
From LoginHistory
Where LoginStatus = 'Pass'
UNION
Select LoginHistoryID,
0 as LoginDuration_Pass,
LoginDuration as LoginDuration_Fail,
LoginDateTime,
LoginLocationID,
LoginUserEmailID,
LoginApplicationID,
LoginEnvironmentID,
ScriptFrequency,
LoginStatus,
Reason
From LoginHistory
Where LoginStatus = 'Fail'";
using (SqlConnection conn = new SqlConnection(Settings.ConnectionString))
{
using (SqlCommand objCommand = new SqlCommand(srtQry, conn))
{
objCommand.CommandType = CommandType.Text;
DataTable dt = new DataTable();
SqlDataAdapter adp = new SqlDataAdapter(objCommand);
conn.Open();
adp.Fill(dt);
if (dt != null)
{
listDataRow = dt.AsEnumerable().ToList();
}
}
}
var listChartHist = (from p in listDataRow
select new ChartHist
{
LoginHistoryID = p.Field<Int32>("LoginHistoryID"),
LoginDuration_Pass = p.Field<Int32>("LoginDuration_Pass"),
LoginDuration_Fail = p.Field<Int32>("LoginDuration_Fail"),
LoginDateTime = p.Field<DateTime>("LoginDateTime"),
LoginLocationID = p.Field<Int32>("LoginLocationID"),
LoginUserEmailID = p.Field<Int32>("LoginUserEmailID"),
LoginApplicationID = p.Field<Int32>("LoginApplicationID"),
LoginEnvironmentID = p.Field<Int32>("LoginEnvironmentID"),
ScriptFrequency = p.Field<Int32>("ScriptFrequency"),
LoginStatus = p.Field<String>("LoginStatus"),
Reason = p.Field<String>("Reason")
}).ToList();
return listChartHist;
}
EF:
public static List<ChartHist> ListHistory()
{
using (var db = new LatencyDBContext())
{
var loginHist = (from hist in db.LoginHistories
select new { LoginHistory = hist }).ToList();
//PUT LOGIN HISTORY RECORDS INTO A LOCAL LIST
var listHistory = new List<ChartHist>();
foreach (var item in loginHist)
{
var localHistData = new ChartHist();
localHistData.LoginHistoryID = item.LoginHistory.LoginHistoryID;
//split up the duration for pass and fail values
if (item.LoginHistory.LoginStatus.ToUpper() == "PASS")
{
localHistData.LoginDuration_Pass = Convert.ToDouble(item.LoginHistory.LoginDuration);
localHistData.LoginDuration_Fail = 0;
}
else if (item.LoginHistory.LoginStatus.ToUpper() == "FAIL")
{
localHistData.LoginDuration_Pass = 0;
localHistData.LoginDuration_Fail = Convert.ToDouble(item.LoginHistory.LoginDuration);
}
localHistData.LoginDateTime = item.LoginHistory.LoginDateTime;
localHistData.LoginLocationID = item.LoginHistory.LoginLocationID;
localHistData.LoginUserEmailID = item.LoginHistory.LoginUserEmailID;
localHistData.LoginApplicationID = item.LoginHistory.LoginApplicationID;
localHistData.LoginEnvironmentID = item.LoginHistory.LoginEnvironmentID;
localHistData.LoginStatus = item.LoginHistory.LoginStatus;
localHistData.Reason = item.LoginHistory.Reason;
localHistData.ScriptFrequency = item.LoginHistory.ScriptFrequency;
listHistory.Add(localHistData);
}
return listHistory;
}
}

Of course EF will take longer to execute than a plain old SQL query, and there's very little that you can do about it (except write the most optimal LINQ queries that you can).
There's a very simple reason why this is so. Running a direct SQL command will just send back the data, with no muss and no fuss attached to it, waiting for you to do the data manipulations to get it to the point where it fits nicely into whatever data structure you want it in. Running EF, on the other hand, means that not only does it run the SQL command, but it massages the data for you into objects that you can manipulate right away. That extra action of going through ADO.NET and converting the data into the objects automatically means that it will take longer than just doing the plain SQL query.
On the flip side of that coin, however, EF does provide a very nice and simple way to debug and solve whatever problems you might have from a specific query/function (like by any exceptions thrown).

I can't performance test this, but try this solution instead before you remove EF entirely:
var loginHist = db.LoginHistories.Where(item => item.LoginStatus.ToUpper() == "PASS" || item.LoginStatus.ToUpper() == "FAIL")
.Select(item => new ChartHist()
{
LoginHistoryID = item.LoginHistoryID,
LoginDuration_Pass = item.LoginStatus.ToUpper() == "PASS" ? Convert.ToDouble(item.LoginDuration) : 0,
LoginDuration_Fail = item.LoginStatus.ToUpper() == "FAIL" ? Convert.ToDouble(item.LoginDuration) : 0,
LoginDateTime = item.LoginDateTime,
LoginLocationID = item.LoginLocationID,
LoginUserEmailID = item.LoginUserEmailID,
LoginApplicationID = item.LoginApplicationID,
LoginEnvironmentID = item.LoginEnvironmentID,
LoginStatus = item.LoginStatus,
Reason = item.Reason,
ScriptFrequency = item.ScriptFrequency,
});
return loginHist.ToList();
This is the "correct" way to populate a new object from a select. It will only retrieve the data you care about, and will put it directly into the object, rather than converting it into an object and then converting it again, from one object to another.
Note: I prefer the functional calls to the from / select form, but it'd be correct either way.