I've made a function that builds a query to insert into mysql. The upload is blazing fast but for inserting longer values the building takes somewhat longer. Is there A way to speed up a function like this? Because I know that a loop in a loop takes a lot of time for higher amounts of data.
foreach (string[] st in dataToUpload)
{
buildQuery += " ('";
for (int i = 0; i < st.Length; i++)
{
buildQuery += st[i];
if (i < st.Length - 1)
buildQuery += "','";
}
buildQuery += "')";
if (st != dataToUpload[dataToUpload.Count - 1])
buildQuery += ",";
}
This is the query I would like to build for example;
string test = INSERT INTO test (test, test1, test2, test3) values
test = test + " " + buildquery;
so test will be
INSERT INTO test (test, test1, test2, test3)
values ("testvalue1", "testvalue2" , "testvalue3" , "testvalue4"),
("testvalue1", "testvalue2" , "testvalue3" , "testvalue4"),
I can work with INNODB and MYISAM and it's working on a centos server with a 6700k processor with 32gb ram.
So the main question is: How can I make the building of the query faster?
I would recommend to use a StringBuilder which gets Initialized to the right size right from the beginning. This reduces reallocation of memory on every string append.
Assuming that dataToUpload is a List you can try this:
// String Builder Initialization
// Size is calculated by getting the length of all strings and add 3 for each (',').
// Additionally there are 6 chars for " ('" and "')," per array
StringBuilder build = new StringBuilder(dataToUpload.Sum(data => data.Sum(s => s.Length) + data.Length * 3) + 6);
foreach (string[] st in dataToUpload)
{
build.Append(" ('" + string.Join<string>("','", st) + "'),");
}
buildQuery = build.ToString().TrimEnd(',');
Seems that your buildQuery is a String. Try StringBuilder instead. It's probably the best way to do string concatenation.
Feels like need more information here, but I suppose your are building an insert statement like this:
INSERT INTO MyTable ( Column1, Column2 ) VALUES
( Value1, Value2 ), ( Value1, Value2 )
So, perhaps the best way to not do a for inside a foreach is replacing the string[] in the foreach, for a string with the correct values. Something like this:
var count = 0;
foreach (string st in dataToUpload)
{
buildQuery += " ('" + st + "'") "
if (count++!=0 )
buildQuery += ","
}
Maybe this:
var count = dataToUpload.Count;
var i = 0;
foreach (string[] st in dataToUpload)
{
buildQuery += " ('" + string.Join(",", st) + "')";
if (i++ < count - 1)
buildQuery += ",";
}
Instead of comparing the st with the contents of the dataToUpload, use an index variable to speed it up.
And string.Join is a good way to concatenate strings.
Related
I am trying to create a query. Here is the code
string wherequery = "";
int fromcode = Convert.ToInt32(CodeTextBox.Text);
int count = Convert.ToInt32(CountTextBox.Text);
for (int i = 0; i < count; i++)
wherequery += ("'" + (fromcode + i).ToString().PadLeft(8,'0') + "',");
wherequery = wherequery.TrimEnd(",");
I am using for loop to create a IN query. Is it possible via LINQ?
Output should be
'0000000087','0000000088','0000000089'
Second thing, I get the code from a textbox value like 0000000087. When I convert it into int using Convert.ToInt32, preceding 0s get disappeared.
Is it possible that 0s shouldn't get disappear since number of preceding 0s may vary?
No need for .PadLeft(8,'0') which causes unnecessary leading zeros.
Try following.
var whereQueryFormat = "Whre columnName IN ({0})";
var wherequery = string.Join
(",",
Enumerable.Range(1, count)
.Select(i => "'" + fromcode + i + "'")
);
wherequery = string.Format(whereQueryFormat, wherequery);
I am not quite sure if you can use LINQ at this point. Here is an example how I would solve this problem:
Dim whereQuery As String = ""
Dim operatorValues As New List(Of String)
Dim startingNumber As String = CodeTextBox.Text
Dim lengthOfNumber As Integer = startingNumber.Length
Dim count As Integer = Convert.ToInt32(CountTextBox.text)
For i As Integer = CInt(startingNumber) To CInt(startingNumber + count - 1)
operatorValues.Add(i.ToString.PadLeft(lengthOfNumber, "0"))
Next
whereQuery &= "IN ('" & Join(operatorValues.ToArray, "','") & "')"
Anyway: Why is your database-field a string and not a integer? Then you would not have a problem with the leading zeros.
If you want to remove the for operation, you could probably do it this way using LINQ, Enumerable.Range, and string.Join:
int fromcode = Convert.ToInt32(CodeTextBox.Text);
int count = Convert.ToInt32(CountTextBox.Text);
var words = from i in Enumerable.Range(fromcode, count)
select "'" + i.ToString().PadLeft(8, '0') + "'";
string wherequery = string.Join(",", words);
You will still have to use a loop for this. But the below code address your variable padded zeros issue.
var enteredValue = "00000088";//get this from the text box
int enteredCount = 10;//get this from the text box
string wherequery = "";
int fromCode = Convert.ToInt32(enteredValue);
string fromCodeStr = fromCode.ToString(CultureInfo.CurrentCulture);
string padZeros = enteredValue.Split(new[] {fromCodeStr}, StringSplitOptions.None)[0];
List<string> searchList = new List<string>();
for (int i = 1; i <= enteredCount; i++)
{
searchList.Add(padZeros + fromCode + i);
}
var searchString = string.Join(",", searchList);
If you want to use LINQ to get the IN clause you can use this:
var range = Enumerable.Range(0, count).Select(x =>
string.Format("'{0}'", x.ToString("0000000000")));
var inCluase = string.Format(" IN ({0})", string.Join(",", range));
I think the initial code is fine:
SqlCommand param = new SqlCommand();
SqlGeometry point = SqlGeometry.Point(center_lat,center_lng,0);
SqlGeometry poly = SqlGeometry.STPolyFromText(new SqlChars(new SqlString(polygon)),0);
param.CommandText = "INSERT INTO Circle (Center_Point, Circle_Data) VALUES (#point,#poly);";
param.Parameters.Add(new SqlParameter("#point", SqlDbType.Udt));
param.Parameters.Add(new SqlParameter("#poly", SqlDbType.Udt));
param.Parameters["#point"].UdtTypeName = "geometry";
param.Parameters["#poly"].UdtTypeName = "geometry";
param.Parameters["#point"].Value = point;
param.Parameters["#poly"].Value = poly;
However I realised there could be a problem when the polygon string is created.
in javascript - I create it like so:
var Circle_Data = "POLYGON ((";
for (var x = 0; x < pointsToSql.length; x++) { // formatting = 0 0, 150 0, 150 50 etc
if (x == 360) { Circle_Data += pointsToSql[x].lat.toString() + " " + pointsToSql[x].lng.toString() + "))"; }
else { Circle_Data += pointsToSql[x].lat.toString() + " " + pointsToSql[x].lng.toString() + ","; }
}
It is then passed to C#. So is this safe? even though the parametrization has happened in the query?
With the parameter you will be saved from SQL Injection, If some SQL is injected in the POLYGON string, it will error out at SQL Server end.
So for example if you have :
POLYGON(12.33 12.55,13.55; DROP TABLE students;)
SQL server will try to construct a geometry type based on the passed string, and it will fail doing so.
I have 3 arrays. Two are arrays of strings and one is of date/time. I pulled all 3 from user input. Each array is always going to have the same exact amount of entries, so what I want to do is be able to loop through all 3 at once to make a string.
I was trying:
List<string> results = new List<string>();
// select
foreach (string line in array1)
{
foreach (string lines in array2)
{
foreach (DateTime date in datearray1)
{
results.Add("select * from table1 d, table2 c where d.specheader = c.specheader and c.true_false = true and d.number = " + lines.ToString() + " and d.date = '" + date.ToShortDateString() + "' and d.specnum like '%" + line.ToString() + "';");
}
}
}
results.ToArray();
foreach (string line in results)
{
MessageBox.Show(line);
}
The user types in information into 3 boxes and I'm just trying to concatenate sql statements based on the input. However when I tried doing it this way it looped through 6 times when I had only 2 entries. Is there a way to concatenate a string using all 3 arrays at the same time (so like entry 1 of array 1, entry 1 of array 2, entry 1 of array 3 - Then move on to creating the next string, entry 2 of array 1, entry 2 of array 2, entry 2 of array 3, etc.)
Any input would be appreciated. Thank you!
As the first commenter said (Yuck) don't use concatenation of strings into your SQL like that. You will want to setup an SQL Command and then pass in parameters.
That is however beside the point as you are asking about rolling together data from multiple arrays into 1 string.
Iterate through one of the arrays, If they all have the same count you will neatly get the data in one.
for(int i = 0; i < array1.Length; i++)
{
results.Add(string.format("Hello you! {0} , {1}, {2}", array1[i], array2[i], datearray[i])
}
This will get your desired result but your code is open to vulnerabilities as it stands. You need to change your approach.
Because your loops are nested, you're getting every value of array2 combined with every value in array1 (and similarly with datearray1. That's why you get too many results.
Your loops would work as intended like this (I've used similar local variables to avoid retyping the results.Add line, and to make clear how the code differs from yours):
for (int i = 0; i < array1.Length; i++)
{
string line = array1[i];
string lines = array2[i];
DateTime date = datearray1[i];
results.Add("select * from table1 d, table2 c where d.specheader = c.specheader and c.true_false = true and d.number = " + lines.ToString() + " and d.date = '" + date.ToShortDateString() + "' and d.specnum like '%" + line.ToString() + "';");
}
As a side-note: building a database query in this manner is inefficent and very insecure (try reading up on "Sql Injection" to understand why). You would see better results if you used a stored procedure instead.
if number of entries are going to be same for all you can simple do a for loop
for (int 1 = 0; i < datearray1.length; i++)
{
results.Add("select * from table1 d, table2 c
where d.specheader = c.specheader and c.true_false = true
and d.number = " + array2[i].ToString() + "
and d.date = '" + datearray1[i].ToShortDateString() + "'
and d.specnum like '%" + array1[i].ToString() + "';");
}
Originally, I was using a short C# program I wrote to average some numbers. But now I want to do more extensive analysis so I converted my C# code to R. However, I really don't think that I am doing it the proper way in R or taking advantage of the language. I wrote the R in the exact same way I did the C#.
I have a CSV with two columns. The first column identifies the row's type (one of three values: C, E, or P) and the second column has a number. I want to average the numbers grouped on the type (C, E, or P).
My question is, what is the idiomatic way of doing this in R?
C# code:
string path = "data.csv";
string[] lines = File.ReadAllLines(path);
int cntC = 0; int cntE = 0; int cntP = 0; //counts
double totC = 0; double totE = 0; double totP = 0; //totals
foreach (string line in lines)
{
String[] cells = line.Split(',');
if (cells[1] == "NA") continue; //skip missing data
if (cells[0] == "C")
{
totC += Convert.ToDouble(cells[1]);
cntC++;
}
else if (cells[0] == "E")
{
totE += Convert.ToDouble(cells[1]);
cntE++;
}
else if (cells[0] == "P")
{
totP += Convert.ToDouble(cells[1]);
cntP++;
}
}
Console.WriteLine("C found " + cntC + " times with a total of " + totC + " and an average of " + totC / cntC);
Console.WriteLine("E found " + cntE + " times with a total of " + totE + " and an average of " + totE / cntE);
Console.WriteLine("P found " + cntP + " times with a total of " + totP + " and an average of " + totP / cntP);
R code:
dat = read.csv("data.csv", header = TRUE)
cntC = 0; cntE = 0; cntP = 0 # counts
totC = 0; totE = 0; totP = 0 # totals
for(i in 1:nrow(dat))
{
if(is.na(dat[i,2])) # missing data
next
if(dat[i,1] == "C"){
totC = totC + dat[i,2]
cntC = cntC + 1
}
if(dat[i,1] == "E"){
totE = totE + dat[i,2]
cntE = cntE + 1
}
if(dat[i,1] == "P"){
totP = totP + dat[i,2]
cntP = cntP + 1
}
}
sprintf("C found %d times with a total of %f and an average of %f", cntC, totC, (totC / cntC))
sprintf("E found %d times with a total of %f and an average of %f", cntE, totE, (totE / cntE))
sprintf("P found %d times with a total of %f and an average of %f", cntP, totP, (totP / cntP))
I would use the data.table package since it has group by functionality built in.
library(data.table)
dat <- data.table(dat)
dat[, mean(COL_NAME_TO_TAKE_MEAN_OF), by=COL_NAME_TO_GROUP_BY]
# no quotes for the column names
If you would like to take the mean (or perform other function) on multiple columns, still by group, use:
dat[, lapply(.SD, mean), by=COL_NAME_TO_GROUP_BY]
Alternatively, if you want to use Base R, you could use something like
by(dat, dat[, 1], lapply, mean)
# to convert the results to a data.frame, use
do.call(rbind, by(dat, dat[, 1], lapply, mean) )
I would do something like this :
dat = dat[complete.cases(dat),] ## The R way to remove missing data
dat[,2] <- as.numeric(dat[,2]) ## convert to numeric as you do in c#
by(dat[,2],dat[,1],mean) ## compute the mean by group
Of course to aggregate your result in a data.frame you can use the the classic , But I don't think is necessary here since it a list of 3 variables:
do.call(rbind,result)
EDIT1
Another option here is to use the elegant ave :
ave(dat[,2],dat[,1])
But the result is different here. In the sense you will get a vector of the same length as your original data.
EDIT2 To include more results you can elaborate your anonymous function:
by(dat[,2],dat[,1],function(x) c(min(x),max(x),mean(x),sd(x)))
Or returns data.frame more suitable to rbind call and with columns names:
by(dat[,2],dat[,1],function(x)
data.frame(min=min(x),max=max(x),mean=mean(x),sd=sd(x)))
Or use the elegant built-in function ( you can define your's also) summary:
by(dat[,2],dat[,1],summary)
One way:
library(plyr)
ddply(dat, .(columnOneName), summarize, Average = mean(columnTwoName))
I want to display 2 sets of data on the one list box, for example, I would wont to display the 7 times table and the 8 times table on the same listbox. Here is how I get the first set of data displaying:
int awnser = 0;
int z;
z = int.Parse(textBox1.Text);
for (int i = 0; i < 11; i++)
{
awnser = z * i;
listBox6.Items.Add(z + " * " + i + " = " + awnser.ToString());
}
But how do I get a line break or separation so I can put the 8 times table just underneath?
How about this?
EDIT Insert it AFTER your loop
listBox6.Items.Add(z + " * " + i + " = " + awnser.ToString());
}
listBox6.Items.Add("--------------------");
In WPF this is easy to do using a custom template, but in WinForms I think you must do it by rendering the list items yourself.
Look at this example where they override the OnDrawItem method: http://www.syncfusion.com/FAQ/windowsforms/faq_c87c.aspx#q627q