Mystery Memory Leak when overwriting a static DataSet - c#

I am running a test function to identify a memory leak:
[TestMethod]
public void DatabaseTools_Other_MemoryTest()
{
for (int i = 0; i < 100; i++)
{
try
{
var r = DatabaseTools.GetDataSet(true);
r = null;
}
catch (Exception e)
{
int EndPoint = i;
}
}
}
The goal of this method is to call DatabaseTools.GetDataSet(true) until it hits an OutOfMemoryException, which happens during the 3rd or 4th load. However, as I understand it, this shouldn't actually ever happen- this is DatabaseTools.GetDataSet:
public static DataSet GetDataSet(bool setData, string sqlText = null, string strConnection = null)
{
sqlText = sqlText ?? FullFilemakerQuery;
if (setData)
{
Database = strConnection;
Data = new DataSet();
}
DataSet dataSet = new DataSet();
using (SqlConnection connection = GetConnectionString())
{
using (SqlDataAdapter dataAdapter = new SqlDataAdapter(sqlText, connection))
{
dataAdapter.SelectCommand.CommandType = System.Data.CommandType.Text;
if (setData)
{
dataAdapter.FillSchema(Data, SchemaType.Source);
DisableAutoIncrement(Data);
dataAdapter.Fill(Data);
NameTables(Data, sqlText);
BuildDataKeysAndRelations(Database);
dataSet = null;
}
else
{
dataAdapter.FillSchema(dataSet, SchemaType.Source);
DisableAutoIncrement(dataSet);
dataAdapter.Fill(dataSet);
NameTables(dataSet, sqlText);
}
}
connection.Close();
}
return dataSet ?? Data;
}
public static void NameTables(DataSet dataSet, string sqlText)
{
for (int i = 0; i < dataSet.Tables.Count; i++)
{
dataSet.Tables[i].TableName = sqlText.Split(';')[i].Split(Dividers, StringSplitOptions.None)[1].Trim();
}
}
public static void DisableAutoIncrement(DataSet data)
{
foreach (DataTable T in data.Tables)
{
T.PrimaryKey.Select(c => { c.AutoIncrement = false; return c; }).ToList();
}
}
When passing only 'true' to this function, it sets sqlText equal to a static FullFileMakerQuery which selects everything the program could use from the database, it then gets the default database name (Database has a custom setter such that when given a null or empty value, it sets itself to default), and sets the static Data to a new DataSet. We have tried setting it to null at this point (no change) and we have tried using Data.Dispose() which caused errors. Because this function can also just return the dataset without also setting the global Data to it, we initialize a new DataSet dataSet. Then we do the standard connect, data adapter, fillschema, load data.
The weirdness: by setting breakpoints in the memory test function and saving dumps, loading the dataset once takes some amount of memory, reloading it uses less memory (by about 36,000 bytes in System.Data.Common.StringStorage) and then reloading it again uses more memory (by about 120,000 bytes in the same place as before). If we reload it one more time, it uses even more memory and crashes due to an OutOfMemoryException, and at this point I have no idea what could be causing it.

You never call .Dispose() on your DataSet, neither on the return value, nor on your static variable before you call new again.
As you are using using blocks with other classes already, I guess you know what to do to actually call .Dispose() on it.
Turn on Visual Studio Static Code Analysis, it will give you warnings should you forget to do so.

Related

Select data from SQL Server in WPF app

Having problems with accessing what is inside my list in a wpf application of a mafia game I am creating.
Basically I read from SQL Server 2016, then add it to my user collection list. Later when I use my list in a display, they all are there.
However the moment I use a foreach to loop through it to set a temp user equal to a found username, it only finds hard coded users and not ones added using the data read from SQL Server. Need help.
SQL Server read code:
using (connect = new SqlConnection(connetionString))
{
connect.Open();
string readString = "select * from Users";
SqlCommand readCommand = new SqlCommand(readString, connect);
using (SqlDataReader dataRead = readCommand.ExecuteReader())
{
if (dataRead != null)
{
while (dataRead.Read())
{
tempEmail = dataRead["Email"].ToString();
tempName = dataRead["Name"].ToString();
UserCollection.addUser(tempEmail, tempName);
}
}
}
connect.Close();
}
UserCollection relevant parts
private static List<User> UserList = new List<User>();
// add a user
public static void addUser(string email, string name)
{
UserList.Add(new User(email, name, 0, "unset", false, false, false, false, false, false,
false, "", false, false, false, 0, 0));
}
//return list of users for use elsewhere
public static List<User> ReturnUserList()
{
return UserList;
}
Use of a list to set tempPlayer in a wpf window
PlayersList = UserCollection.ReturnUserList();
// tempPlayer = UserCollection.ReturnAUser(sessionUser);
foreach (var element in PlayersList)
{
if (element.UserName == sessionUser)
{
tempPlayer = element;
}
}
Example of code where the list works.
// set listing of current players
ListOfPlayers = UserCollection.ReturnUserList();
var tempList = from player in ListOfPlayers
where player.UserBlocked == false
select new
{
Name = player.UserName,
Email = player.UserEmail,
};
this.PlayerListBox.ItemsSource = tempList;
hard coded User add that works fine and can be found by foreach statement from my app.xaml.cs
UserCollection.addUser("g", "Tom");
Firstly, is there a reason why you need a static method to add users to a collection? Even if you need access to the list via a static accessor, you are better having a static property on the same class which you're using to read the DB
The following snippet should hopefully be of some help.
public class UserManagement {
//Static property
private static List<User> _users;
public static List<User> Users {
get {
if (_users == null) {
_user = new List<User>();
}
return _users;
}
set { }
}
//Static load method must be called before accessing Users
public static void LoadDBUsers() {
using (SqlConnection connection = new SqlConnection(connetionString)) {
connection.Open();
string readString = "select * from Users";
using (SqlCommand command = new SqlCommand(readString, connection)) {
using (SqlDataReader reader = command.ExecuteReader()) {
while (reader.Read()) {
String tempEmail = reader["Email"].ToString();
String tempName = reader["Name"].ToString();
User user = new User(tempEmail, tempName, 0, "unset", false, false, false, false, false, false, false, "", false, false, false, 0, 0));
users.Add(user);
}
}
}
}
}
}
To use from another class :
UserManagement.LoadDBUsers();
var dbUsers = UserManagement.Users;
If you have another list of users (say from a file), you could add a loader method to your class which will handle that for you.
If at some point you need to clear down the list of users, you can simply set it to a new instance...
UserManagement.Users = new List<User>();
A slightly better way to do this would be to remove the static property Users first, change the method definition to return a List<User> from the LoadDBUsers method - eg.
public static List<User> LoadDBUsers() {
List<User> Users = new List<User>();
//Handle the database logic as in the previous example..
//Then at the end of the method..
return Users;
}
and call as follows
List<User> dbUsers = UserManagement.LoadDBUsers();
Using the latter approach, you don't need to worry about multiple locations in your code maintaining a static property. Just call and assign it to a variable.
It also has the advantage that you will always get the most current list of users from the DB without having to clear down and reload the list before you access it from the static property.
An added advantage of not using a global static property is that it can avoid potential memory issues. Static properties can be difficult to dispose by the garbage collector if a reference to them is held open.
With instance variables, it's quite obvious when one goes out of scope and is not referenced anymore, but with static variables, the reference is sometimes not disposed until the program ends.
In many cases, this isn't an issue, but in larger systems it can be a pain to debug.
tempEmail = dataRead["Email"].ToString();
tempName = dataRead["Name"].ToString();
tempEmail,tempName you are declare above on
using (connect = new SqlConnection(connetionString))
the tempEmail and tempName are reference type so, if loop it's first record add after second time loop tempName and tempEmail values update also previously added so,because it's also pointing same memory. so the data's are duplicate record list so, the users cannot add in user properly.
so, you can change your code
var tempEmail = dataRead["Email"].ToString();
var tempName = dataRead["Name"].ToString();
and before tempEmail,tempName remove the declaration before using statement.
I Hope this is Helpful for you.
I believe your while loop was causing the issue. Since "using" was in effect, the global, assumed, variables "tempEmail" & "tempName" remained null. I tested the code using MySql on my end and this solution was effective.
private List<User> PlayersList;
public Tester()
{
using (SqlConnection connect = new SqlConnection(connectionString))
{
connect.Open();
string readString = "select * from user";
SqlCommand readCommand = new SqlCommand(readString, connect);
using (SqlDataReader dataRead = readCommand.ExecuteReader())
{
if (dataRead != null)
{
while (dataRead.Read())
{
string tempEmail = dataRead["Email"].ToString();
string tempName = dataRead["Name"].ToString();
UserCollection.addUser(tempEmail, tempName);
}
}
}
connect.Close();
}
PlayersList = UserCollection.ReturnUserList();
}
public class User
{
public string email;
public string name;
// A constructor that takes parameter to set the properties
public User(string e, string n)
{
email = e;
name = n;
}
}
public static class UserCollection
{
private static List<User> UserList = new List<User>();
// add a user
public static void addUser(string email, string name)
{
UserList.Add(new User(email, name));
}
//return list of users for use elsewhere
public static List<User> ReturnUserList()
{
return UserList;
}
}
This is the area of concern.
while (dataRead.Read())
{
//Set local variable during while loop
string tempEmail = dataRead["Email"].ToString();
string tempName = dataRead["Name"].ToString();
UserCollection.addUser(tempEmail, tempName);
}

C# Stops Executing When Opening A Connection To MySql

I am writing an application to transfer data from a simulated OPC server to a MySql Database. I am using some of the libraries from OPC Foundation - OPCNETAPI.DLL, OPCNETAPI.COM.DLL.
I have the program reading from the server happily, but it seems as soon as I open a connection inside the loop of reading data, it loops about 5 times and then stops executing.. There are no exceptions thrown and I have tried for hours to work it out and miserably failed.
I've also changed VisualStudio to break when an exception is thrown, and it doesn't
Hopefully some one may know whats going on, but there is nothing obvious, hopefully the following code will spark ideas!
Windows Forms Code
public void readplc()
{
Opc.URL url = new Opc.URL("opcda://localhost/Matrikon.OPC.Simulation.1");
Opc.Da.Server server = null;
OpcCom.Factory fact = new OpcCom.Factory();
server = new Opc.Da.Server(fact, null);
try
{
server.Connect(url, new Opc.ConnectData(new System.Net.NetworkCredential()));
}
catch (Exception ecy)
{
}
// Create a group
Opc.Da.Subscription group;
Opc.Da.SubscriptionState groupState = new Opc.Da.SubscriptionState();
groupState.Name = "Group";
groupState.Active = true;
group = (Opc.Da.Subscription)server.CreateSubscription(groupState);
// add items to the group.
Opc.Da.Item[] items = new Opc.Da.Item[2];
items[0] = new Opc.Da.Item();
items[0].ItemName = "Random.Int1";
items[1] = new Opc.Da.Item();
items[1].ItemName = "Random.Time";
items = group.AddItems(items);
group.DataChanged += new Opc.Da.DataChangedEventHandler(OnTransactionCompleted);
}
public void OnTransactionCompleted(object group, object hReq, Opc.Da.ItemValueResult[] items)
{
for (int i = 0; i < items.GetLength(0); i++)
{
try
{
string a = items[i].Value.ToString();
string b = items[i].ItemName;
string c = items[i].Key;
MySqlConnection conn = new MySqlConnection("server=localhost;user id=localhost;password=localhost;database=localhost;pooling=false");
conn.Open();
conn.Close();
}
catch (Exception ec)
{
MessageBox.Show(ec.Message);
}
}
}
I have set break points throughout the code, stepped through everything possible and still nothing obvious that's causing the issue.

Why does my successfully-read Npgsql data disappear?

I have the following code shape. It seems that I'm misunderstanding the C# method return values. How is it possible that a "full" enumerator gets returned as an empty one?
class ThingDoer
{
public NpgsqlDataReader DoQuery()
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
return dataread; // Debugger confirms that six data are enumerable here.
}
}
...
class OtherThing
{
public void higherLevelFunction()
{
NpgsqlDataReader result = myThingDoer.DoQuery();
result.Read(); // No data! result's enumerable returns nothing!
}
}
You don't detail where your connection is coming from. Assuming it's something like:
public NpgsqlDataReader DoQuery()
{
using(NpgsqlConnection = GetConnectionCode())
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
return dataread;
}//Connection closes at this using-scope being left because that triggers Dispose()
}
Then change it to:
public NpgsqlDataReader DoQuery()
{
bool ownershipPassed = false;
NpgsqlConnection conn = GetConnectionCode();
try
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader(CommandBehavior.CloseConnection);
ownershipPassed = true;
return dataread;
}
finally
{
if(!ownershipPassed)//only if we didn't create the reader than takes charge of the connection
conn.Dispose();
}
}
Then where you use the reader, you have to dispose it to in turn dispose the connection's underlying connection to the database:
public void higherLevelFunction()
{
using(NpgsqlDataReader result = myThingDoer.DoQuery())
result.Read();
}
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
The above lines are very local to the method DoQuery. So as soon as the control comes out of the method, every object created inside to this method loses its scope. Hence you're losing the data because it's a reference type you're referring to in the caller method.

Getting exception: Invalid attempt to read when reader is closed

I am attempting to read a MySQL database from my C# project using the MySQL drivers for .net off the MySQL site.
Though I did a bit of research on this (including this), I am still flummoxed why this is happening. I later ran a spike and I still get the same error. (Prior to running this I populated the database with some default values.) Here's the spike code in toto.
class Program {
static void Main (string[] args) {
Console.WriteLine (GetUserAge ("john")); // o/p's -1
}
static int GetUserAge (string username) {
string sql = "select age from users where name=#username";
int val = -1;
try {
using (MySqlConnection cnn = GetConnectionForReading ()) {
cnn.Open ();
MySqlCommand myCommand = new MySqlCommand (sql, cnn);
myCommand.Parameters.AddWithValue ("#username", username);
using (MySqlDataReader reader = myCommand.ExecuteReader ()) {
DataTable dt = new DataTable ();
dt.Load (reader);
if (reader.Read ()) {
val = reader.GetInt32 (0);
}
}
}
} catch (Exception ex) {
Console.WriteLine (ex.Message);
} finally {
}
return val;
}
private static MySqlConnection GetConnectionForReading () {
string conStr = "Data Source=localhost;Database=MyTestDB;User ID=testuser;Password=password";
return new MySqlConnection (conStr);
}
}
The code above gives me the exception: "Invalid attempt to Read when reader is closed."
Later I modified the if-condition like so:
if (reader.HasRows && reader.Read ()) {
val = reader.GetInt32 (0);
}
And now the o/p is -1. (The data's in there in the table.) If for some reason the result set had zero rows, the reader should not have got into the if-block in the first place. I mean, the whole point of the Read() method is to check if there are any rows in the result set in the first place.
At my wit's end here... just cannot figure out where I'm going wrong.
Thank you for your help! :)
I think using DataTable.Load will "consume" the reader and, at the very least, position it at the end. It may even account for the closed connection (but I'm just guessing here). What if you remove that line? I don't think it makes any sense here.

Automation Error 80004005

I've been beating myself over the head with this for a few days now, searched up and down the internet. I know there is a lot I don't quite get about Com Interop but I've had success building and using simpler DLL's with Excel. Anyways to the point.
I get the above mentioned Error -2147467259 80004005 Automation Error Unspecified Error in Excel VBA when running the following code wrapped in a DLL.
[GuidAttribute("96BE21CD-887B-4770-9FAA-CF395375AEA9")]
[ComVisibleAttribute(true)]
interface QInterface
{
void ConnectionFill(string SQLQuery, string CONStr);
string QueryValue(int QueryKey);
string ConnectionValue(int ConnectionKey);
string outputFile { get; set; }
void ThreadTest();
}
[ClassInterfaceAttribute(ClassInterfaceType.None)]
[ProgIdAttribute("QueryThread")]
class QueryThread : QInterface
{
DataSet QueryReturn = new DataSet();
private ArrayList SQLList = new ArrayList();
private ArrayList ConList = new ArrayList();
private string OutputFile;
public void ConnectionFill(string SQLQuery, string CONStr)
{
SQLList.Add(SQLQuery);
ConList.Add(CONStr);
}
public string QueryValue(int QueryKey)
{
return SQLList[QueryKey].ToString();
}
public string ConnectionValue(int ConnectionKey)
{
return ConList[ConnectionKey].ToString();
}
public string outputFile
{
set { OutputFile = value; }
get { return OutputFile; }
}
public void ThreadTest()
{
int i = 0;
i = SQLList.Count;
Thread[] myThreads;
myThreads = new Thread[i];
for (int t = 0; t != i; t++)
{
myThreads[t] = new Thread(() => ThreadRun(SQLList[t].ToString(), ConList[t].ToString()));
myThreads[t].Name = "Thread " + t;
myThreads[t].IsBackground = true;
myThreads[t].Start();
Thread.Sleep(600);
if (t > 9)
{
myThreads[t - 9].Join();
}
}
for (int t = 0; t != i; t++)
{
while (myThreads[t].IsAlive)
{
Thread.Sleep(20);
}
}
TextWriter tw = new StreamWriter(OutputFile);
for (int t = 0; t < QueryReturn.Tables.Count; t++)
{
DataTableReader DR = QueryReturn.Tables[t].CreateDataReader();
while (DR.Read())
{
tw.WriteLine("{0} : {1}", DR.GetValue(0), DR.GetValue(1));
}
}
tw.Close();
QueryReturn.Dispose();
}
private void ThreadRun(string SQLString, string ConString)
{
try
{
OleDbConnection DBCon = new OleDbConnection(ConString);
DBCon.Open();
Thread.Sleep(200);
OleDbCommand DBCmd = new OleDbCommand(SQLString, DBCon);
OleDbDataAdapter DataAdapter = new OleDbDataAdapter(DBCmd);
Thread.Sleep(200);
DataAdapter.Fill(QueryReturn, Thread.CurrentThread.Name.ToString());
DBCon.Close();
DataAdapter.Dispose();
DBCon.Dispose();
DBCmd.Dispose();
}
finally
{
}
}
}
using this VBA code...
Sub test()
Dim QT As New QueryThreading.QueryThread
Dim MyResults As String
Dim outputfile As String
Dim InputStuff(1, 1) As String
InputStuff(0, 0) = "Select DISTINCT * From TrackingData;"
InputStuff(0, 1) = "Provider =Microsoft.ACE.OLEDB.12.0;Data Source =C:\Users\Nick\Desktop\test.accdb; Persist Security Info =False;Connection Timeout=7;"
InputStuff(1, 0) = "Select DISTINCT * From TrackingData;"
InputStuff(1, 1) = "Provider =Microsoft.ACE.OLEDB.12.0;Data Source =C:\Users\Nick\Desktop\test2.accdb; Persist Security Info =False;Connection Timeout=7;"
QT.ConnectionFill InputStuff(0, 0), InputStuff(0, 1)
QT.ConnectionFill InputStuff(1, 0), InputStuff(1, 1)
outputfile = "C:\Users\Nick\Desktop\testrun.txt"
QT.outputfile = outputfile
QT.ThreadTest
End Sub
It runs fine is a pure C# console application. Works perfect and quick with no problems. But via VBA I get the error.
I'm assuming it's something to do with the calling of the access databases in multiple threads. I know there is a lot of junk code in there, it's not optimized of course I'm still in the "playing around" phase.
I've used RegAsm and enabled com interop and all such stuff, I can read back from the returns just fine. So I know the DLL is working right, just when I fill the threads and run "ThreadTest()" I get the automation error.
If I run it a second time Excel locks up.
Any help would be greatly appreciated.
You are calling QT.DictionaryFill though I can't seem to find a corresponding method in you C# code... what happens if you change the call to QT.ConnectionFill instead ?
Another point: IIRC then VBA runs the object in STA - not sure whether MTA is supported though
EDIT:
According to this rather old post VBA does not support multi-threading...

Categories