I need some advice regarding updating a large number of records in a MySql database. I am currently storing a large number of words and their suffixes (a suffix array) into a database which results in a row count of approximately 4.3 million. Each record contains the primary key id, the actual word word, the document the word is in document, the offset of the word within the document 'offset', a flag which determines whether the record is a whole word or not flag and a link to the next record with the same value for word. Each record is initialized with a link value of -1.
This is my current code for updating the links in the database:
public void Link(object c)
{
DBConnection conn = (DBConnection)c;
rowcount = conn.GetRowCount();
string word;
int link;
List<Record> recordsList = new List<Record>();
List<Record> recordsMatched = new List<Record>();
for (int i = 0; i < rowcount; i++)
{
recordsList.AddRange(conn.ReadQuery("SELECT * FROM csa2018.words WHERE id = " + i));
word = recordsList[0].Word;
link = recordsList[0].Link;
recordsMatched = conn.ReadQuery("SELECT * FROM csa2018.words WHERE word = '" + word + "'");
for(int j = 0; j < recordsMatched.Count-1; j++)
{
if (recordsMatched[j].Link == -1)
{
conn.WriteQuery("UPDATE csa2018.words SET link = " + recordsMatched[j + 1].Id + " WHERE id = " + recordsMatched[j].Id);
}
else
{
break;
}
linkedRecords++;
}
linkedRecords++;
recordsMatched.Clear();
recordsList.Clear();
}
Form1.linkingFinished = true;
}
Overall, it has good performance when it finds words which are repeated frequently; however at around 60% the performance deteriorates because most of the remaining words are unique.
My guess is that this query:
recordsMatched = conn.ReadQuery(
"SELECT * FROM csa2018.words WHERE word = '" + word + "'");
shouldn't be like this because it is being called once for every row. Are there any better approaches like using stored procedures maybe?
P.S.: the ReadQuery method reads rows using the query supplied and constructs a Record object and adds each record to a List<Record>.
This is what my database looks like :
CREATE TABLE words ( id int(11) NOT NULL, word varchar(45) NOT NULL,
document varchar(45) NOT NULL, offset int(11) NOT NULL, flag int(11) NOT NULL,
link int(11) DEFAULT NULL, PRIMARY KEY (id) ) ENGINE=InnoDB DEFAULT CHARSET=utf8
if I understand your code correctly than this single sql-statement should do the job:
UPDATE csa2018.words as w1
left join
(select w2.id as id, min(w3.id) as linked_to
from csa2018.words w2, csa2018.words w3
where w2.word = w3.word and
w3.id > w2.id limit 1) w4
on (w1.id = w4.id)
SET w1.link = IFNULL(w4.linked_to, -1)
The inner select-statement gives the mapping from one dataset to the linked dataset. You should watch the result of the select-statement to see if everthing is fine.
Related
We are working with forms and in current project we are working with NPOI object but due to word metadata npoi is not returning the first,even and odd page Headers/footers. So we are trying with interop for getting the correct results and we are able to get the results as per the below code. But again we are facing the challenge with reading bodyelements in sequence order which are under headers/footers.
Current Result:
I am able to read the body elements(table and paragraph) of header/footer. But not in sequence order. I can able to read the elements individually like "firstPageFooter .Paragraphs" and "firstPageFooter .Tables"
foreach (Section data in sections)
{
var tempVal = string.Empty;
HeadersFooters headersFooters = data.Footers;
bool isDifferentFirstPageHeaderFooter = Convert.ToBoolean(data.PageSetup.DifferentFirstPageHeaderFooter);
bool isOddAndEvenPagesHeaderFooter = Convert.ToBoolean(data.PageSetup.OddAndEvenPagesHeaderFooter);
if (isDifferentFirstPageHeaderFooter)
firstPageFooter = headersFooters[WdHeaderFooterIndex.wdHeaderFooterFirstPage].Range;
Tables headerTable = firstPageFooter .Tables;
StringBuilder tableBuilder = new StringBuilder();
foreach (Table table in headerTable)
{
foreach (Row row in table.Rows)
{
foreach (Cell cell in row.Cells)
{
tableBuilder.Append(cell.Range.Text);
}
}
}
StringBuilder paraBuilder = new StringBuilder();
var headerPara= firstPageFooter.Paragraphs;
foreach (Paragraph paragraph in headerPara)
{
paraBuilder.Append(paragraph.Range.Text);
}
} // Added by edit!!!
Expected Results:
If the header/Footer body elements contains tables and paragraph. First element is table and second is paragraph. So the word should return the body elements in the same sequence order.
Can some one help me out?
It's possible to loop the Paragraphs collection of the document Story (a Header, for example) and test whether it's in a table. If it is, then pick up the table and process it. Then continue with the paragraph below the table, until the Story has been completed.
Here's a code snippet that demonstrates this approach, based on a single Header.
Word.HeaderFooter hdr = doc.Sections[1].Headers[Word.WdHeaderFooterIndex.wdHeaderFooterPrimary];
Word.Range hdrRange = hdr.Range;
Word.Paragraphs paras = hdrRange.Paragraphs;
for (int counter = 1; counter <= paras.Count; counter++)
{
Word.Paragraph para = paras[counter];
if ((bool)para.Range.get_Information(Word.WdInformation.wdWithInTable))
{
//Get the table that belongs to the first paragraph in the table
Word.Table tbl = para.Range.Tables[1];
//Reset the counter so that it cycles to the paragraph after the table
counter += tbl.Range.Paragraphs.Count - 1;
Debug.Print("In table with " + (counter - 1).ToString() + " paragraphs");
//Process the table
}
else
{
//Process the paragraph
Debug.Print("Paragraph " + counter.ToString() + ": " + para.Range.Text);
}
}
I have a number in a database (e.g 12) and I want to display that number in the dropdown list on a webform, however, not the number itself but a range of numbers from 1 to 12 (so 1,2,3....12). Is there a property I can use or a way to get a list of numbers from SQL Statement?
Read the Number from Database
Display a full range of numbers from 1 to X (X = Number from Database)
Bind dropdown list to:
Enumerable.Range(1, <number from database>);
First create an array or list and then iterate till the number you read from database then add these numbers to your array or list then bind with your dropdown datasource
var numbers = new List<int>();
for(var i = DB_NUMBER; i >= 1; i--)
{
numbers.Add(i);
}
yourDropDown.DataSource = numbers;
yourDropDown.DataBind();
Hope it helps
Since, you wish to return the range of numbers from the SQL Query you might need a complex query, but it will always have some limit to it.
SQL Query as per your requirement:
Select Value from
(
SELECT ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n as Value
FROM (VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) ones(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tens(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) hundreds(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) thousands(n)
) as tbl
Where value between 0 and 12
ORDER BY 1
I Modified above SQL from the Original Source : Answer
C#:
DropDownList.DataSource = DataTable;
DropDownList.DisplayField = Value;
DropDownList.ValueField = Value;
DropDownList.DataBind();
You just read the number from the database and then use a for loop:
for(int i = readNumber; i > 0; i--)
{
//Add i to your dropDown list or do anything you want with it
}
Based on the question
If I assume that you want SQL to generate and return the range then you want to use a recursive cte to build it from the value in your table ...
// construct the db connection and command object
var con = new SqlConnection("Your Connection String");
using(var cmd = new SqlCommand(con) { CommandType = CommandType.Text })
{
// tell the command what SQL query we want to execute
cmd.CommandText = #"
DECLARE #startnum INT=1
DECLARE #endnum INT= SELECT TOP 1 Number FROM ValueTable
;
WITH gen AS (
SELECT #startnum AS num
UNION ALL
SELECT num+1 FROM gen WHERE num+1<=#endnum
)
SELECT * FROM gen
option (maxrecursion 100)
";
// connect to the db and execute the command
con.Open();
using(var reader = cmd.ExecuteReader())
{
// build the range from the values generated by it
var range = new List<int>();
while(reader.Read()) { range.Add(reader.Read()); }
// bind the results to the drop down on the page
DropDownList.DataSource = range
.Select(i = > new { Key = i, Value = i })
.ToArray();
DropDownList.DisplayField = "Key";
DropDownList.ValueField = "Value";
DropDownList.DataBind();
}
con.Close();
}
The simplest approach
ok querying a db is a pretty well documented problem so I won't repeat that here.
But lets assume you have the following ...
// sourced from your db
int start = 1;
int end = 12;
... from there you can build a range of values ...
var range = Enumerable.Range(start, end)
.Select(i = > new { Key = i, Value = i })
.ToArray();
... and then bind that range to your drop down on the page ...
DropDownList.DataSource = range;
DropDownList.DisplayField = "Key";
DropDownList.ValueField = "Value";
DropDownList.DataBind();
Sources of information ...
How to generate a range of numbers between two numbers?
https://msdn.microsoft.com/en-us/library/fksx3b4f.aspx
https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader(v=vs.110).aspx
To achieve this you need to
Take the desire number from database
then you have to write a for loop.
private void InitializeDropDownList(int number)
{
for (int i = 0; i < number; i++)
{
ddlNumberRange.Items.Add(new ListItem { Text = (i + 1).ToString(), Value = (i + 1).ToString() });
}
}
How do I create multiple rows on my table at once?
I want to be able to add non-existent rows, and edit/update existing rows whenever I press submit.
To test this I only created 2 fields in the database, and cannot seem to add more than one row.
Value of numofbuilding = 5.
Only one row is inserted.
Tried:
public ActionResult CreateBuildings(Guid pi, int? numofbuilding)
{
OnboardModel model = new OnboardModel();
List<onboard_BuildingInfo> coms = new List<onboard_BuildingInfo>();
for (int i = 1; i <= (numofbuilding+1); i++)
{
onboard_BuildingInfo f = new onboard_BuildingInfo
{
projectID = pi,
building_ID = i
};
coms.Add(f);
}
context.onboard_BuildingInfos.InsertAllOnSubmit(coms);
context.SubmitChanges();
return View(model);
}
and tried:
public ActionResult CreateBuildings(Guid pi, int? numofbuilding)
{
OnboardModel model = new OnboardModel();
for (int i = 1; i <= numofbuilding; i++)
{
onboard_BuildingInfo coms = new onboard_BuildingInfo
{
projectID = pi,
building_ID = i
};
context.onboard_BuildingInfos.InsertOnSubmit(coms);
context.SubmitChanges();
}
return View(model);
}
BuildingInfo-Table
Create Table onboard_BuildingInfo (
projectID UNIQUEIDENTIFIER DEFAULT NEWID() ,
building_ID int NULL ,
city_building varchar(500) NULL ,
numberofcommon INT NULL
PRIMARY KEY (projectID)
)
Make the projectID not a primary key, or have a composite key on projectID and building_ID. Primary keys must be unique therefore you cannot have duplicate items in the projectID column. A composite key would mean that the combination of (projectID * building_ID) must be unique therefore you may have duplicates in either column, but not both.
I am trying to create a leaderboard based on player's High score.
Each player's scores are entered randomly into DB.
I need to sort them based on their high score and also rank them.
this is my table structure
Here i want to display player_ID, player_name, player_nick and HP in datagridview.
any help is appreciated.
string query1 = "SELECT player_ID'Player ID',player_name'Player
Name',player_nick'Nick Name',HP'High Score' FROM player_profile ORDER
BY HP DESC";
My work so far, i don't know how to rank
You can use SQL to provide the rank with a user variable (I think there are also some Rank() functions). From something like Workbench:
SET #rank=0;
SELECT Name, HP, #rank:=#rank+1 As Rank FROM Demo ORDER BY HP ASC
You can also do it from code, with one small change:
string SQL = #"SET #rank=0;
SELECT Name, HP, StartDate, #rank:=#rank+1 As Rank
FROM Demo ORDER BY HP DESC;";
using (MySqlConnection dbcon = mySqlDB.GetMySQLConnection())
using (MySqlCommand cmd = new MySqlCommand(SQL,dbcon))
{
dbcon.Open();
DataTable dt = new DataTable();
dt.Load(cmd.ExecuteReader());
dgv1.DataSource = dt;
}
Results:
There is no Rank column in the table, that is added via the SQL statement above.
If you have ties, it gets more complicated. You'd have to introduce some other vars to track when the HP/Score changes and increment #rank only then. If you want to skip a rank on ties ({1,2,2,4} vs {1,2,2,3}) you'd have to also add a counter.
The one thing is that you have to allow user vars which can be specified in the connection string:
Server=SvrAddr;Database=myDB;Uid=myUsr;Pwd=myPass;Allow User Variables=True";
According to Connection Strings the option is available as of version 5.2.2
This Great Answer shows how to skip having to initialize the rank var:
string SQL = #"SELECT Name, HP, StartDate, #rank:=#rank+1 As Rank
FROM Demo, (SELECT #rank := 0) r ORDER BY HP DESC;";
Adding (SELECT #rank := 0) r prevents having to explicitly declare it. Very cool.
The following code adds the data to the dataGridView and then answers your question at the end.
// Add columns to the dataGridView
dataGridView1.Columns.Add("player_ID", "player_ID");
dataGridView1.Columns.Add("player_name", "player_name");
dataGridView1.Columns.Add("player_nick", "player_nick");
dataGridView1.Columns.Add("HP", "HP");
// Add some data to the dataGridView
object[] rowData = new object[dataGridView1.Columns.Count];
rowData[0] = 0; // Player_ID
rowData[1] = "Pancho"; // Player_Name
rowData[2] = "Speedy"; // Player Nick
rowData[3] = Convert.ToDecimal("58.7"); // HP
dataGridView1.Rows.Add(rowData);
rowData[0] = 1;
rowData[1] = "Ramon";
rowData[2] = "Sleepy";
rowData[3] = Convert.ToDecimal("39.6"); // HP
dataGridView1.Rows.Add(rowData);
rowData[0] = 2;
rowData[1] = "Cimitrio";
rowData[2] = "Grumpy";
rowData[3] = Convert.ToDecimal("41.2"); // HP
dataGridView1.Rows.Add(rowData);
rowData[0] = 3; // Player_ID
rowData[1] = "Panfilo"; // Player_Name
rowData[2] = "Gummy Bear"; // Player Nick
rowData[3] = Convert.ToDecimal("61.5"); // HP
dataGridView1.Rows.Add(rowData);
// Sort dataGridView by HP
dataGridView1.Sort(dataGridView1.Columns[3], ListSortDirection.Ascending);
// Add rank column
dataGridView1.Columns.Add("Rank", "Rank");
// Rank players
for (int i = 0; i < dataGridView1.Rows.Count-1; i++)
{
dataGridView1.Rows[i].Cells["Rank"].Value = Convert.ToString(i+1);
}
I want to retrieve the number of rows grouped with condition for controlling the number of rows for display. With primary key there is no problem I get the count(*) but when in case of other fields such as date, nom where there is much rows with the same name and date I found a primitive way to find the number of rows wich will be displayed as below:
public static int GetRapportPgeNbr(string Qry, int param)
{
int counter = 0;
int result = 0;
using (MySqlConnection conn = new MySqlConnection(PublicVariables.cs))
{
using (MySqlCommand cmd = new MySqlCommand(Qry,conn))
{
conn.Open();
MySqlDataReader reader = cmd.ExecuteReader();
try
{
while (reader.Read())
{
result = result + Convert.ToInt16(reader["rows"]);
++counter;
}
}
catch(MySqlException e)
{
MessageBox.Show(e.Number.ToString() + " -> " + e.Message.ToString());
return result;
}
}
}
if (param == 1)
return counter;
else
return result;
}
The param variable guides me either I get the sum of rows (sometimes there is 2 or more tables with union) or counter.
Sqlfiddle
In this exemple I have 5 rows but I need only 4 rows so I take the result of the counter.
Is there a better way ?
I think you are looking for COUNT(DISTINCT nom) and remove the GROUP BY to get the total count:
SELECT COUNT(DISTINCT nom) AS rows
FROM ProdMacaron
SQL Fiddle Demo
This will give you 4 not 5.
You can, however, add the GROUP BY nom, but this is useless with COUNT(DISTINCT nom) this will give you ones for any values in your table.