We Keep Coding

C# Wait until data received and go to next iteration

I'm using Serial Port to get data from device in for loop iteration.
The problem is in loop iteration i need to get data from serial port, validate it and going to the next iteration.
How i can achieved this?
Here are my code :
private void processData()
{
// Loop Procedure
int x = Int32.Parse(master["Cycle"].ToString());
int y = Int32.Parse(master["MinWeight"].ToString());
// Loop for each line
for (int i = this.CLine; i < 2; i++)
{
this.CLine = i;
if (i == 0)
label15.Text = master["LLINE"].ToString();
else
label15.Text = master["RLINE"].ToString();
IDictionary<string, string> dic = (Dictionary<String, String>)master[i.ToString()];
label18.Text = this.CProcess = dic["PROCESSID"];
int z = Int32.Parse(dic["PRODLANE"].ToString());
// Loop for each sampling session (Cycle)
for (int j = this.CCycle; j <= x; j++)
{
this.CCycle = j;
// Loop for production lane
for (int k = this.CLane; k <= z; k++)
{
this.CLane = k;
label16.Text = k.ToString();
// In this section i want to send command over serial port
// get value from my device
// validate it if current weight bellow standard weight
// do it again (get data from device)
// else we can go to next iteration
while (this.CWeight < y)
{
XApi.l("xxx2 " + this.CWeight + " vs " + y + " " + k.ToString() + " " + this.isDataReady);
SendData("Q");
}
// Commit Transaction
// XDb.CommitTrans(this.CCycle.ToString(), dic["LINEID"].ToString(), this.CLane.ToString(), weight.ToString(), this.isTrialStage == true ? "1" : "0");
}
}
}
}
I've tried this
while (this.CWeight < y)
{
XApi.l("xxx2 " + this.CWeight + " vs " + y + " " + k.ToString() + " " + this.isDataReady);
SendData("Q");
}
but it seems blocked UI thread and make my application solaggy.
Anyone can give me some idea? Thanks in advance.
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
if (e.EventType == System.IO.Ports.SerialData.Eof)
return;
// If the com port has been closed, do nothing
if (!comport.IsOpen)
return;
// Update flag data received
this.isDataReady = true;
// Determain which mode (string or binary) the user is in
if (CurrentDataMode == DataMode.Text)
{
// Read all the data waiting in the buffer
string data = comport.ReadExisting();
// Update result
result += data;
if (result.Length > 16)
{
SetText(result.ToString());
}
// Display the text to the user in the terminal
Log(LogMsgType.Incoming, data);
}
else
{
// Obtain the number of bytes waiting in the port's buffer
int bytes = comport.BytesToRead;
// Create a byte array buffer to hold the incoming data
byte[] buffer = new byte[bytes];
// Read the data from the port and store it in our buffer
comport.Read(buffer, 0, bytes);
// Show the user the incoming data in hex format
Log(LogMsgType.Incoming, ByteArrayToHexString(buffer));
}
}
private void SendData(String msg)
{
this.isDataReady = false;
result = "";
if (CurrentDataMode == DataMode.Text)
{
// Send the user's text straight out the port
comport.Write(msg + "\r\n");
// Show in the terminal window the user's text
Log(LogMsgType.Outgoing, msg + "\n");
}
else
{
try
{
// Convert the user's string of hex digits (ex: B4 CA E2) to a byte array
byte[] data = HexStringToByteArray(txtSendData.Text);
// Send the binary data out the port
comport.Write(data, 0, data.Length);
// Show the hex digits on in the terminal window
Log(LogMsgType.Outgoing, ByteArrayToHexString(data) + "\n");
}
catch (FormatException)
{
// Inform the user if the hex string was not properly formatted
Log(LogMsgType.Error, "Not properly formatted hex string: " + txtSendData.Text + "\n");
}
}
}

Anyone can give me some idea?
You can use async/await in your code not to block your UI by writing an extension method like below. Usage would be:
async void SomeMethod()
{
SerialPort serialPort = .......
while (true)
{
serialPort.Write(.....);
var retval = await serialPort.ReadAsync();
}
}
The keyword here is using TaskCompletionSource class with your events...
static public class SerialPortExtensions
{
public static Task<byte[]> ReadAsync(this SerialPort serialPort)
{
var tcs = new TaskCompletionSource<byte[]>();
SerialDataReceivedEventHandler dataReceived = null;
dataReceived = (s, e) =>
{
serialPort.DataReceived -= dataReceived;
var buf = new byte[serialPort.BytesToRead];
serialPort.Read(buf, 0, buf.Length);
tcs.TrySetResult(buf);
};
serialPort.DataReceived += dataReceived;
return tcs.Task;
}
}

System out of memory exception large txt file into an array [duplicate]

This question already has answers here:
Read Big TXT File, Out of Memory Exception
(6 answers)
Closed 6 years ago.
The following code works fine with small txt files , but if we have large txt files its giving outofmemory exception at string[] array = File.ReadAllLines("hash.txt");
hash.txt file is a 500 mb
I tried few suggestions from internet but i didn't get that worked.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
namespace Hash_Parser
{
internal class Program
{
private static List<string> users = new List<string>();
private static Dictionary<string, int> hash_original = new Dictionary<string, int>();
private static List<string> hash_found = new List<string>();
private static List<string> pass = new List<string>();
private static string hash_path = "split.txt";
private static void split()
{
Console.WriteLine("Splitting...");
StreamWriter streamWriter = new StreamWriter("user.txt");
StreamWriter streamWriter2 = new StreamWriter("hash.txt");
string[] array = File.ReadAllLines(Program.hash_path);
for (int i = 0; i < array.Length; i++)
{
string text = array[i];
string[] array2 = text.Split(new char[]
{
':'
}, 2);
if (array2.Count<string>() >= 2)
{
streamWriter.WriteLine(array2[0]);
streamWriter2.WriteLine(array2[1]);
}
}
streamWriter.Close();
streamWriter2.Close();
Console.WriteLine("Saved as user.txt and hash.txt");
}
private static void populate()
{
Console.WriteLine("Populating lists...");
Program.users.AddRange(File.ReadAllLines("user.txt"));
Program.pass.AddRange(File.ReadAllLines("pass.txt"));
Program.hash_found.AddRange(File.ReadAllLines("found.txt"));
int num = 0;
string[] array = File.ReadAllLines("hash.txt");
for (int i = 0; i < array.Length; i++)
{
string key = array[i];
Program.hash_original.Add(key, num);
num++;
}
}
private static void seek()
{
StreamWriter streamWriter = new StreamWriter("userpass.txt");
int num = 0;
int num2 = 100;
foreach (string current in Program.hash_found)
{
if (Program.hash_original.ContainsKey(current))
{
streamWriter.WriteLine(Program.users[Program.hash_original[current]] + ":" + Program.pass[num]);
}
num++;
if (num >= num2)
{
Console.Title = string.Concat(new object[]
{
"Processed: ",
num,
" : ",
Program.hash_found.Count
});
num2 += 1000;
}
}
Console.Title = string.Concat(new object[]
{
"Processed: ",
num,
" : ",
Program.hash_found.Count
});
streamWriter.Close();
}
private static void Main(string[] args)
{
Console.WriteLine("Split hash /split");
Console.WriteLine("'split.txt'\n");
Console.WriteLine("Parse hashes /parse");
Console.WriteLine("'user.txt' | 'found.txt' | 'hash.txt' | 'pass.txt'");
string a = Console.ReadLine();
if (a == "/split")
{
Program.split();
}
else
{
if (a == "/parse")
{
Program.populate();
Console.WriteLine("Processing...");
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
Program.seek();
stopwatch.Stop();
Console.WriteLine("Saved as userpass.txt");
Console.WriteLine("Time elapsed: " + stopwatch.Elapsed);
Console.ReadKey();
}
}
}
}
}
Thnaks for ur help.

Try this code :
foreach (var line in File.ReadLines(_filePath))
{
//Don't put "line" into a list or collection.
//Just make your processing on it.
}
Quoted Text: Just use File.ReadLines which returns an IEnumerable and doesn't load all the lines at once to the memory.
Quote Link : https://stackoverflow.com/a/13416225/3041974
I hope it helps.

Please be aware of process limits in .NET
http://www.codeproject.com/Articles/483475/Memory-Limits-in-a-NET-Process
For instance, a 32 bit system cannot have more than 4 GB of physical memory. Needless to say that 2^32 will give you a virtual address space with 4.294.967.296 different entries, and that’s precisely where the 4GB limit comes from. But even having those 4GB available on the system, your application will actually be able to see 2GB only. Why?
Because on 32 bits systems, Windows splits the virtual address space
into two equal parts: one for User Mode applications, and another one
for the Kernel (system applications). This behavior can be overridden
by using the "/3gb" flag in the Windows boot.ini config file. If we do
so, the system will then reserve 3GB for user applications, and 1 GB
for the kernel.
What is the process MEM Usage in Task Manager?

Starting up several programs at once

I have written a small program (ProcessSample) to launch another programs that are defined in the .txt file (separated by new line).
The code is mostly taken from MSDN. I am just starting my programming adventure but I want to write something useful.
I don't know the smart way to run for example two programs simultaneously from my ProcessSample program.
In my .txt file I have just paths to programs with .exe. It's all working fine but my program is only running one program at the time. I thought I would run foreach but of course it won't work here as it just runs the first program and it waits till I exit it, then it will run the next one.
So I know the reason why is it not working. I just want to know how could I make it work the way I would like.
My C# code:
using System;
using System.Diagnostics;
using System.Threading;
namespace ProcessSample
{
class ProcessMonitorSample
{
public static void Main()
{
Console.BufferHeight = 25;
// Define variables to track the peak memory usage of the process.
long peakWorkingSet = 0;
string[] Programs = System.IO.File.ReadAllLines(#"C:\Programs\list.txt");
foreach (string Program in Programs)
{
Process myProcess = null;
// Start the process.
myProcess = Process.Start(#Program);
// Display the process statistics until
// the user closes the program.
do
{
if (!myProcess.HasExited)
{
// Refresh the current process property values.
myProcess.Refresh();
// Display current process statistics.
Console.WriteLine();
Console.WriteLine("Path: {0}, RAM: {1}", Program, (myProcess.WorkingSet64 / 1024 / 1024));
// Update the values for the overall peak memory statistics.
peakWorkingSet = myProcess.PeakWorkingSet64;
if (myProcess.Responding)
{
Console.WriteLine("Status: Running");
}
else
{
Console.WriteLine("Status: Not Responding!");
}
// Wait 2 seconds
Thread.Sleep(2000);
} // if
} // do
while (!myProcess.WaitForExit(1000));
Console.WriteLine();
Console.WriteLine("Process exit code: {0}", myProcess.ExitCode);
Console.WriteLine("Peak physical memory usage of the process: {0}", (peakWorkingSet / 1024 / 1024));
Console.WriteLine("Press any key to exit.");
System.Console.ReadKey();
} // foreach
} // public
} //class
} // namespace

The problem is in the inner while loop. There you are getting statistics from the running process and displaying them in the console. As far as I understand from your post, you don't need this feature so you can remove it and you would get:
using System;
using System.Diagnostics;
using System.Threading;
namespace ProcessSample
{
class ProcessMonitorSample
{
public static void Main()
{
Console.BufferHeight = 25;
// Define variables to track the peak memory usage of the process.
long peakWorkingSet = 0;
string[] Programs = System.IO.File.ReadAllLines(#"C:\Programs\list.txt");
foreach (string Program in Programs)
{
Process myProcess = null;
// Start the process.
myProcess = Process.Start(#Program);
Console.WriteLine("Program started: {0}", Program);
}
}
}
}

Actually I have learned about Threads and I have used them for my program like this:
class Program
{
static void Main(string[] args)
{
string[] myApps = { "notepad.exe", "calc.exe", "explorer.exe" };
Thread w;
ParameterizedThreadStart ts = new ParameterizedThreadStart(StartMyApp);
foreach (var myApp in myApps)
{
w = new Thread(ts);
w.Start(myApp);
Thread.Sleep(1000);
}
}
private static void StartMyApp(object myAppPath)
{
ProcessStartInfo myInfoProcess = new ProcessStartInfo();
myInfoProcess.FileName = myAppPath.ToString();
myInfoProcess.WindowStyle = ProcessWindowStyle.Minimized;
Process myProcess = Process.Start(myInfoProcess);
do
{
if (!myProcess.HasExited)
{
myProcess.Refresh(); // Refresh the current process property values.
Console.WriteLine(myProcess.ProcessName+" RAM: "+(myProcess.WorkingSet64 / 1024 / 1024).ToString()+"\n");
Thread.Sleep(1000);
}
}
while (!myProcess.WaitForExit(1000));
}
}

You can replace your foreach with Parallel.ForEach to achieve what you want.
Parallel.ForEach<String>(Programs, Program =>
{
Process myProcess = null;
// Start the process.
myProcess = Process.Start(#Program);
// Display the process statistics until
// the user closes the program.
do
{
if (!myProcess.HasExited)
{
// Refresh the current process property values.
myProcess.Refresh();
// Display current process statistics.
Console.WriteLine();
Console.WriteLine("Path: {0}, RAM: {1}", Program, (myProcess.WorkingSet64 / 1024 / 1024));
// Update the values for the overall peak memory statistics.
peakWorkingSet = myProcess.PeakWorkingSet64;
if (myProcess.Responding)
{
Console.WriteLine("Status: Running");
}
else
{
Console.WriteLine("Status: Not Responding!");
}
// Wait 2 seconds
Thread.Sleep(2000);
}
}
while (!myProcess.WaitForExit(1000));
Console.WriteLine();
Console.WriteLine("Process exit code: {0}", myProcess.ExitCode);
Console.WriteLine("Peak physical memory usage of the process: {0}", (peakWorkingSet / 1024 / 1024));
Console.WriteLine("Press any key to exit.");
System.Console.ReadKey();
});

create one time reference for various sample rates per file

My program reads in multiple files that contain time/value pairs sampled at different rates. I'm trying to use the file with the highest sample rate as the time scale for all sampled rates and output one master file with the unique time values from the highest sample rate file.
Each file contains time/values pairs like:
1,58
1.5,90
2,154
2.5,34
Here is my code so far:
public void ReadAndWrite(string[] fileNames)
{
var stopwatch = Stopwatch.StartNew();
List<StreamReader> readers = fileNames.Select(f => new StreamReader(f)).ToList();
try
{
using (StreamWriter writer = new StreamWriter(tbxOutputFile.Text))
{
string line = null;
// For each measurement in max measurements
for (int measNum = 0; measNum < numOfRows; measNum++)
{
// For each file's reader
for (int i = 0; i < readers.Count; i++)
{
// If line contains something, then add it to line
if ((line = readers[i].ReadLine()) != null)
{
// Process line and then write it to file
line = ProcessLine(line);
writer.Write(line);
}
else
{
writer.Write("");
}
// If it's not the last column, add delimiter
if (i < readers.Count - 1)
writer.Write(",");
}
writer.WriteLine();
// Update labels
int val = ((measNum + 1) * 100) / numOfRows;
string newText = Convert.ToString("Progress: " + val + "% " + " " + "Measurement #: " + (measNum + 1)
+ " out of " + numOfRows); // running on worker thread
this.Invoke((MethodInvoker)delegate
{
// runs on UI thread
lblStatus.Text = newText;
progressBar1.Value = val;
});
}
}
}
catch (Exception)
{
throw;
}
finally
{
foreach (var reader in readers)
{
reader.Close();
}
}
MessageBox.Show("File successfully created! " + '\n' + "Elapsed time: " +
(stopwatch.ElapsedMilliseconds/1000) + " seconds", "Processing Complete");
}
I came up with the pseudo code below (currentTime is the time from each file and uniqueTime is from an array that reads in each time from the highest sampled file):
// if time value from individual file is same as uniqueTime
if currentTime == uniqueTime
{
valueToWrite = curr_value // write the current value
}
else // currentTime is not same as uniqueTime
{
valueToWrite = prev_value // write the previous value
}
timeToWrite = uniqueTime // always write the uniqueTime
What is the best way to execute this pseudo code to make a unique time reference for all the various sample rates? Sorry if my question is confusing, I can elaborate more if need be.

To be clear about this, you do not want the values at the specific time they occurred, but you want to display one value for each source at each timepoint that the highest sampled source has?
That should be pretty straightforward. In pseudocode:
foreach (ValuePair val in highSampleRateValues) {
var aggregatedTimePointData;
aggregatedTimePointData.Add(val.Time, val.Value);
foreach (ValuePair val2 in lowSampleRateValues) {
var value = DetermineLatestEntryBackwardFrom(val.Time);
aggregatedTimePointData.Add(value);
}
}
This way, the sample rate of the higher density sampled signal serves as a clock, but you will have inaccuracies since the values from the other sources are only close, but not exactly on the timepoint of their recording. If you want those inaccuracies reduced, choose a higher sample rate and do the same thing. You can get as close to the actual timepoints as you want.

C# application slows Down after 20-30 mins?

I have c# application which reads data from serial port. I have put serial read handler in timer with interval 1 second , because data coming every 1 second
on timer I calling
delegate void SetTextCallback(string text);
ReceivedText(serialPort1.ReadExisting());
I also showing received data in richtextbox just to check it getting proper data or not.
But after 15-20 mins application slows down wont even respond.
private void ReceivedText(string text)
{
if (this.rtbReceived.InvokeRequired)
{
SetTextCallback x = new SetTextCallback(ReceivedText);
this.Invoke(x, new object[] { (text) });
}
else
{
this.rtbReceived.Text += text;
serialdata = text;
if (serialdata.Length > 0 &&
serialdata.Length < 42 &&
serialdata.Contains("#") ||
serialdata.StartsWith(" #"))
{
serialdata.Trim();
splitdata = serialdata.Split(' ');
try
{
txtBathTemp.Text = splitdata[3];
txtBaroPressure.Text = splitdata[4];
double stemp = double.Parse(splitdata[5]);
txtSampleTemp.Text = (Math.Round(stemp, 2)).ToString();
}
catch (Exception EX)
{
MessageBox.Show(ex.Message);
}
}
}
}

There is a chance that this.rtbReceived.Text grows up after some time. Even if it doesn't use huge amount of memory, constantly manipulating String is not efficent. Have you considered using StringBuilder instead?