I've some data (row) of some table in a db temp that I want to copy on another db (not temp !) but with the same tablename that has the same number, type and order of columns !
So my idea was to create a function to avoid to repeat myself :
public int copyDB(string **tableName**, int[] **id**)
{
for(int i = 0; i < **id**.Count(); i++)
{
var temp = from t in dbtemp.+(**tableName**)+ where t.student_id == **id**[i]);
}
(...)
}
Is it possible to write i function like this? I will just need to call the function where i want to with the correct args.
I'm still looking also how to copy data, I've found some interesting link like this. I still need to try them.
Thanks for your help
Related
UPDATE
I think I have found what is causing the issue here https://stackoverflow.com/a/5665600/19393524
I believe my issue lies with my use of .DefaultView. The post thinks when you do a sort on it it is technically a write operation to the DataTable object and might not propagate changes made properly or entirely. It is an interesting read and seems to answer my question of why passing valid data to a DataRow is throwing this exception AFTER I make changes to the datatable
UPDATE:
Let me be crystal clear. I have already solved my problem. I would just like to know why it is throwing an error. In my view the code should work and it does.. the first run through.
AFTER I have already deleted the column then added it back (run this code once)
When I debug my code line by line in Visiual studio and stop at the line:
data.Rows[i].SetField(sortColumnNames[k], value);
the row exists
the column exisits
value is not null
sortColumnNames[k] is not null and contains the correct column name
i is 0
Yet it still throws an exception. I would like to know why. What am I missing?
Sorry for the long explanation but this one needs some context unfortunately.
So my problem is this, I have code that sorts data in a DataTable object by column. The user picks the column they want to sort by and then my code sorts it.
I ran into an issue where I needed numbers to sort as numbers not strings (all data in the table is strings). eg (string sorting would result in 1000 coming before 500)
So my solution was to create a temporary column that uses the correct datatype so that numbers get sorted properly and the original string data of the number remains unchanged but is now sorted properly. This worked perfectly. I could sort string numeric data as numeric data without changing the formatting of the number or data type.
I delete the column I used to sort afterwards because I use defaultview to sort and copy data to another DataTable object.
That part all works fine the first time.
The issue is when the user needs to do a different sort on the same column. My code adds back the column. (same name) then tries to add values to the column but then I get a null reference exception "Object not set to an instance of an object"
Here is what I've tried:
I've tried using AcceptChanges() after deleting a column but this did nothing.
I've tried using column index, name, and column object returned by DataTable.Columns.Add() in the first parameter of SetField() in case it was somehow referencing the "old" column object I deleted (this is what I think the problem is more than likely)
I've tried changing the value of the .ItemArray[] directly but this does not work even the first time
Here is the code:
This is the how the column names are passed:
private void SortByColumn()
{
if (cbAscDesc.SelectedIndex != -1)//if the user has selected ASC or DESC order
{
//clears the datatable object that stores the sorted defaultview
sortedData.Clear();
//grabs column names the user has selected to sort by and copies them to a string[]
string[] lbItems = new string[lbColumnsToSortBy.Items.Count];
lbColumnsToSortBy.Items.CopyTo(lbItems, 0);
//adds temp columns to data to sort numerical strings properly
string[] itemsToSort = AddSortColumns(lbItems);
//creates parameters for defaultview sort
string columnsToSortBy = String.Join(",", itemsToSort);
string sortDirection = cbAscDesc.SelectedItem.ToString();
data.DefaultView.Sort = columnsToSortBy + " " + sortDirection;
//copies the defaultview to the sorted table object
sortedData = data.DefaultView.ToTable();
RemoveSortColumns(itemsToSort);//removes temp sorting columns
}
}
This is where the temp columns are added:
private string[] AddSortColumns(string[] items)//adds columns to data that will be used to sort
//(ensures numbers are sorted as numbers and strings are sorted as strings)
{
string[] sortColumnNames = new string[items.Length];
for (int k = 0; k < items.Length; k++)
{
int indexOfOrginialColumn = Array.IndexOf(columns, items[k]);
Type datatype = CheckDataType(indexOfOrginialColumn);
if (datatype == typeof(double))
{
sortColumnNames[k] = items[k] + "Sort";
data.Columns.Add(sortColumnNames[k], typeof(double));
for (int i = 0; i < data.Rows.Count; i++)
{
//these three lines add the values in the original column to the column used to sort formated to the proper datatype
NumberStyles styles = NumberStyles.Any;
double value = double.Parse(data.Rows[i].Field<string>(indexOfOrginialColumn), styles);
bool test = data.Columns.Contains("QtySort");
data.Rows[i].SetField(sortColumnNames[k], value);//this is line that throws a null ref exception
}
}
else
{
sortColumnNames[k] = items[k];
}
}
return sortColumnNames;
}
This is the code that deletes the columns afterward:
private void RemoveSortColumns(string[] columnsToRemove)
{
for (int i = 0; i < columnsToRemove.Length; i++)
{
if (columnsToRemove[i].Contains("Sort"))
{
sortedData.Columns.Remove(columnsToRemove[i]);
}
}
}
NOTE:
I've been able to fix the problem by just keeping the column in data and just deleting the column from sortedData as I use .Clear() on the sorted table which seems to ensure the exception is not thrown.
I would still like an answer though as to why this is throwing an exception. If I use .Contains() on the line right before the one where the exception is thrown is says the column exists and returns true and in case anyone is wondering the params sortColumnNames[k] and value are never null either.
Your problem is probably here:
private void RemoveSortColumns()
{
for (int i = 0; i < data.Columns.Count; i++)
{
if (data.Columns[i].ColumnName.Contains("Sort"))
{
data.Columns.RemoveAt(i);
sortedData.Columns.RemoveAt(i);
}
}
}
If you have 2 columns, and the first one matches the if, you will never look at the second.
This is because it will run:
i = 0
is i < columns.Count which is 2 => yes
is col[0].Contains("sort") true => yes
remove col[0]
i = 1
is i < columns.Count which is 1 => no
The solution is to readjust i after the removal
private void RemoveSortColumns()
{
for (int i = 0; i < data.Columns.Count; i++)
{
if (data.Columns[i].ColumnName.Contains("Sort"))
{
data.Columns.RemoveAt(i);
sortedData.Columns.RemoveAt(i);
i--;//removed 1 element, go back 1
}
}
}
I fixed my original issue by changing a few lines of code in my SortByColumn() method:
private void SortByColumn()
{
if (cbAscDesc.SelectedIndex != -1)//if the user has selected ASC or DESC order
{
//clears the datatable object that stores the sorted defaultview
sortedData.Clear();
//grabs column names the user has selected to sort by and copies them to a string[]
string[] lbItems = new string[lbColumnsToSortBy.Items.Count];
lbColumnsToSortBy.Items.CopyTo(lbItems, 0);
//adds temp columns to data to sort numerical strings properly
string[] itemsToSort = AddSortColumns(lbItems);
//creates parameters for defaultview sort
string columnsToSortBy = String.Join(",", itemsToSort);
string sortDirection = cbAscDesc.SelectedItem.ToString();
DataView userSelectedSort = data.AsDataView();
userSelectedSort.Sort = columnsToSortBy + " " + sortDirection;
//copies the defaultview to the sorted table object
sortedData = userSelectedSort.ToTable();
RemoveSortColumns(itemsToSort);//removes temp sorting columns
}
}
Instead of sorting on data.DefaultView I create a new DataView object and pass data.AsDataView() as it's value then sort on that. Completely gets rid of the issue in my original code. For anyone wondering I still believe it is bug with .DefaultView in the .NET framework that Microsoft will probably never fix. I hope this will help someone with a similar issue in the future.
Here is the link again to where I figured out a solution to my problem.
https://stackoverflow.com/a/5665600
The part of the code I'm working on receives an
IEnumerable<T> items
where each item contains a class with properties reflecting a MSSQL database table.
The database table has a total count of 953664 rows.
The dataset in code is filtered down to a set of 284360 rows.
The following code throws an OutOfMemoryException when the process reaches about 1,5 GB memory allocation.
private static void Save<T>(IEnumerable<T> items, IList<IDataWriter> dataWriters, IEnumerable<PropertyColumn> columns) where T : MyTableClass
{
foreach (var item in items)
{
}
}
The variable items is of type
IQueryable<MyTableClass>
I can't find anyone with the same setup, and other's solutions that I've found doesn't apply here.
I've also tried paging, using Skip and Take with a page size of 500, but that just takes a long time and ends up with the same result. It seems like objects aren't being released after each iteration. How is that?
How can I rewrite this code to cope with a larger collection set?
Well, as Servy has already said you didn't provide your code so I'll try to make some predictions... (Sorry for my english)
If you have an exception in "foreach (var item in items)" when you are using paging then, I guess, something wrong with paging. I wrote a couple of examples to explain my idea.
if first example I suggest you (just for test) put your filter inside the Save function.
private static void Save<T>(IQueryable<T> items, IList<IDataWriter> dataWriters, IEnumerable<PropertyColumn> columns) where T : MyTableClass
{
int pageSize = 500; //Only 500 records will be loaded.
int currentStep = 0;
while (true)
{
//Here we create a new request into the database using our filter.
var tempList = items.Where(yourFilter).Skip(currentStep * pageSize).Take(pageSize);
foreach (var item in tempList)
{
//If you have an exception here maybe something wrong in your dataWriters or columns.
}
currentStep++;
if (tempList.Count() == 0) //No records have been loaded so we can leave.
break;
}
}
The second example show how to use paging without any changes in the Save function
int pageSize = 500;
int currentStep = 0;
while (true)
{
//Here we create a new request into the database using our filter.
var tempList = items.Where(yourFilter).Skip(currentStep * pageSize).Take(pageSize);
Save(tempList, dataWriters, columns); //Calling saving function.
currentStep++;
if (tempList.Count() == 0)
break;
}
Try both of them and you'll either resolve your problem or find another place where an exception is raised.
By the way, another potential place is your dataWriters. I guess there you store all data that your have been received from the database. Maybe you shouldn't save all data? Just calculate memory size that all objects are required.
P.S. And don't use while(true) in your code. It just an example:)
Hi I have a record called Tags in a table called Knowledgebase (KB) in my DB. The words in Tags are separated by commas. Also in the KB table is a field called Title.
I need to match the tags field to the Title so I have replaced all the commas in the tags with spaces like so string removeCommas = commas.Replace(",", " ");.
I think I now need to loop around every record in the KB table to find records? I would like to store the number of matches it finds in a variable called suggestions.
Question is what is the best way to implement the for each loop for a query like this? Is the for each loop the best method for this task?
One way is to store the space seperated strings in a List. Now, use Foreach loop for the whole table and in turn a foreach loop for each record to find the match of Title.
I will correct my answer if this is going the wrong direction, but would something like this complete the operation you are attempting?
System.Data.DataTable KB = new System.Data.DataTable();
string[] tags = "some,foo,bar".Split(',');
System.Collections.Generic.List<string> suggestions = new System.Collections.Generic.List<string>();
for (int i = 0; i < tags.Length; i++)
{
for (int j = 0; j < KB.Rows.Count; j++)
{
string row = KB.Rows[j]["Title"].ToString();
if ((row.Contains(tags[i]))
{
suggestions.Add(row);
}
}
}
Also, avoid foreach whenever humanly possible. Check out THIS post to see why.
Yes Linq can help you to retreive the data from your database and then replacing it after a foreach loop
Code Added
Let say you are using EDM and you have a context kb:
public void ReplaceCommaInTag(KbContext kb)
{
var tags = from t in kb.Titles
Select t.Tags;
foreach(Tag tag in tags)
{
tag = tag.Replace(","," ");
}
try
{
kb.SubmitChanges();
}
catch(Exception e)
{
//Display the error.
}
}
I hope it will help you
I am writing the below structure in to the file using Binary Writer.
File Structure:
Num of Employees {employee num & name, employee1 num & name, employee2 num & name...}.
I will get a command with employee num to delete that particualr employee details and update Num of employees feild accordingly.
What is the best way to do the above operation?
Regards
Raju
The question is a little unclear, but when I'm working with a File I always try to deserialize structued data like that to classes.
In general, you'd get a list of Employee objects from the file, remove the item you want, and then write the list back out to the file (or DB, or any other thing you need to write the data to). Quick example (I haven't tried this, so it might not compile as is:)
//EmpList is a List<Employee>
public void DeleteEmployee(int employeeNumber)
{
//Assumes you will never have multiple entries with the same EE#, or that
//if you do you want to delete all records
for(int i = EmpList.Count -1; i>= 0; i--)
{
if(EmpList[i].EmployeeNumber == employeeNumber)
EmpList.Remove(EmpList[i]);
}
//Call code to reserialize (to file, db, whatever)
}
I am reading my DataTable as follow:
foreach ( DataRow o_DataRow in vco_DataTable.Rows )
{
//Insert More Here
}
It crash; because I insert more records.
How can I read my DataTable without reading the new records? Can I read by RowState?
Thanks
Since I don't know what language you are using I can only give general advice.
In most (all?) languages it's not possible to do a foreach over a collection if you are modifying the collection. There are two common ways to deal with this.
Wild ass guessing pseudo code follows:
// first way uses array notation (if possible)
var no_of_rows = vco_DataTable.Rows.count();
for(var i = 0; i < no_of_rows; i++) {
DataRow o_DataRow = vco_DataTable.Rows[i];
//Insert More Here
}
// The second way copies the data
var my_copy = vco_DataTable.Copy()
foreach ( DataRow o_DataRow in my_copy.Rows )
{
//Insert More into vco_DataTable Here
}
copy.Dispose() // delete/destroy the copy