We Keep Coding

IOException thrown from Image class or byte array

I'm extracting a ZIP file. This ZIP contains image files and an Excel file with a product list. When articles of different sizes are listed the article refers to the same image. I copy the image file to a local folder and write the (compressed) binary data to SQL server database.
So when it gets to the point where a JPG file shall be processed a second time, I get this exception, although I dispose the image object.
Worksheet ws;
string root = "C:\\images\\";
string file;
string importFolder = "C:\\import\\;
Dictionary <string, object> ins;
Image im;
Image th;
//Worksheet has been opened before
//ZIP has been extracted before to C:\import\
for (i = 2; i <= ws.Dimension.End.Row; i++) {
ins = new Dictionary<string, object>(); //Dictionary to write data to database
file = ws.Cells[i, 4].Text;
System.IO.File.Copy(importFolder + "\\" + file, root + "\\" + file, true); // <-- Here the exception is thrown in the second iteration
im = Image.FromFile(root + "\\" + file);
im = im.GetBetterThumbnail(1024);
byte[] im_data = im.GetJpgByteArray(85);
ins.Add("url", "www.test.de/images/" + file);
ins.Add("image_data", im_data);
ins.Add("image_size", im_data.Length);
//image will be written to database
im.Dispose();
im = null;
im_data = null;
//With these initializations there shouldn't be thrown an exception
} // end for
What am I missing? With resetting the Image object and byte array, there shouldn't be another reference to the image file.
I had a look on this
IOException: The process cannot access the file 'file path' because it is being used by another process
but I couldn't figure out, how to adept to my topic.
Yes, I could store all file names just to copy them once, but I think that's the lazy way.
Kind regards

You assign a value to the variable im two times.
One time you use im = Image.FromFile(root + "\\" + file) and the other time you use im = im.GetBetterThumbnail(1024). Could it be that this opens two handles that need to be closed?
Besides, it's better to use the using statement. Then you don't have to take care of the disposing by yourself.
For example like this:
for (i = 2; i <= ws.Dimension.End.Row; i++)
{
ins = new Dictionary<string, object>(); //Dictionary to write data to database
file = ws.Cells[i, 4].Text;
System.IO.File.Copy(importFolder + "\\" + file, root + "\\" + file, true);
using (im = Image.FromFile(root + "\\" + file))
{
// I guess that this method creates its own handle
// and therefore also needs to be disposed.
using (thumbnail = im.GetBetterThumbnail(1024))
{
byte[] im_data = thumbnail.GetJpgByteArray(85);
ins.Add("url", "www.test.de/images/" + file);
ins.Add("image_data", im_data);
ins.Add("image_size", im_data.Length);
//image will be written to database
}
}
} // end for

I got the issue solved by using a stream. The memory management works really better now.
New code:
//im = Image.FromFile(root + "\\" + file);
im = Image.FromStream(File.Open(root + "\\" + file, FileMode.Open));
So could it be that this is another 'Microsoft feature'?

Can write to csv file but not append

string pathDesktop = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
string filePath = pathDesktop + "\\mycsvfile.csv";
string delimter = ",";
string a = "Enoelpro---[037,033,0,018,030,012,004,021,009,038,035,053,044,050,074,F,018,010,070,000]<Vulpix>[-][037,034,0,022,020,029,002,008,024,036,046,049,041,057,077,F,018,005,070,000]<Vulpix>[-] cual es mejor??";
List<string[]> test = conv(a,"testrainer");
int length = test.Count;
using (TextWriter writer = File.CreateText(filePath))
{
for (int index = 0; index < length; index++)
{
writer.WriteLine(string.Join(delimter, test[index]));
}
}
So, at the momement, this works fine, except it doesn't keep the old data in the csv file. How can I modify this so instead of deleting the data, it simply appends to the data?

Can you please try with StreamWriter class?
If the file exists, it can be either overwritten or appended to. If the file does not exist, this constructor creates a new file.
Instead using
TextWriter writer = File.CreateText(filePath) try to use
TextWriter writer = new StreamWriter(filePath, true);
If you pass true in constructor it should append text to file.

File.CreateText Method (String)
This method is equivalent to the StreamWriter(String, Boolean)
constructor overload with the append parameter set to false. If the
file specified by path does not exist, it is created. If the file does
exist, its contents are overwritten. Additional threads are permitted
to read the file while it is open.
StreamWriter Constructor (String, Boolean)
Here second parameter, true to append data to the file; false to overwrite the file.
If you check the documentation for each method it cleary say the answers for your questions and also there is a suggetion in case of you need to append the file. Use StreamWriter constructor with path and append parameter (true)

How to write at the next line in the next opening of the csv file?

At first run, my program, writes to a csv file in the first line,
But, when I'm running my program at the second.. third.. time, it runs over the first line..
how can i correct it?
I would like to have a CSV file input of all the entering to my program.
The code is as follows:
private void WriteToCsvFile()
{
var us = users.ElementAt(0);
string names = "Number',";
string userAnswer = (us.userName + ",");
foreach (string ss in user)
{
string str = Path.GetFileName(ss);
names = names + str + ",";
}
foreach (string ans in us.answer)
{
userAnswer = userAnswer + ans + ",";
}
using (StreamWriter sw = new StreamWriter("EntranceLog.csv"))
{
sw.WriteLine(names);
sw.WriteLine(userAnswer);
}
this.Close();
}

Add true parameter in the constructor:
using (StreamWriter sw = new StreamWriter("EntranceLog.csv", true))
The second parameter named append controls whether an existing file shall be overwritten or appended. MSDN states:
true to append data to the file; false to overwrite the file. If the specified file does not exist, this parameter has no effect, and the constructor creates a new file.

Each time it is run, you are creating a new file with the same name is overwriting the older file. That is the default behavior of the specific constructor you are using.
You want to use this constructor instead and specify the append parameters as true:
using (StreamWriter sw = new StreamWriter("EntranceLog.csv", true))
{
// write your file as normal
}

Append to file failure when executable not in same folder as data files

Problem is now solved. Mistake by me that I hadn't seen before.
I am pretty new to coding in general and am very new to C# so I am probably missing something simple. I wrote a program to pull data from a login website and save that data to files on the local hard drive. The data is power and energy data for solar modules and each module has its own file. On my main workstation I am running Windows Vista and the program works just fine. When I run the program on the machine running Server 2003, instead of the new data being appended to the files, it just overwrites the data originally in the file.
The data I am downloading is csv format text over a span of 7 days at a time. I run the program once a day to pull the new day's data and append it to the local file. Every time I run the program, the local file is a copy of the newly downloaded data with none of the old data. Since the data on the web site is only updated once a day, I have been testing by removing the last day's data in the local file and/or the first day's data in the local file. Any time I change the file and run the program, the file contains the downloaded data and nothing else.
I just tried something new to test why it wasn't working and think I have found the source of the error. When I ran on my local machine, the "filePath" variable was set to "". On the server and now on my local machine I have changed the "filePath" to #"C:\Solar Yard Data\" and on both machines it catches the file not found exception and creates a new file in the same directory which overwrites the original. Anyone have an idea as to why this happens?
The code is the section that download's each data set and appends any new data to the local file.
int i = 0;
string filePath = "C:/Solar Yard Data/";
string[] filenamesPower = new string[]
{
"inverter121201321745_power",
"inverter121201325108_power",
"inverter121201326383_power",
"inverter121201326218_power",
"inverter121201323111_power",
"inverter121201324916_power",
"inverter121201326328_power",
"inverter121201326031_power",
"inverter121201325003_power",
"inverter121201326714_power",
"inverter121201326351_power",
"inverter121201323205_power",
"inverter121201325349_power",
"inverter121201324856_power",
"inverter121201325047_power",
"inverter121201324954_power",
};
// download and save every module's power data
foreach (string url in modulesPower)
{
// create web request and download data
HttpWebRequest req_csv = (HttpWebRequest)HttpWebRequest.Create(String.Format(url, auth_token));
req_csv.CookieContainer = cookie_container;
HttpWebResponse res_csv = (HttpWebResponse)req_csv.GetResponse();
// save the data to files
using (StreamReader sr = new StreamReader(res_csv.GetResponseStream()))
{
string response = sr.ReadToEnd();
string fileName = filenamesPower[i] + ".csv";
// save the new data to file
try
{
int startIndex = 0; // start index for substring to append to file
int searchResultIndex = 0; // index returned when searching downloaded data for last entry of data on file
string lastEntry; // will hold the last entry in the current data
//open existing file and find last entry
using (StreamReader sr2 = new StreamReader(fileName))
{
//get last line of existing data
string fileContents = sr2.ReadToEnd();
string nl = System.Environment.NewLine; // newline string
int nllen = nl.Length; // length of a newline
if (fileContents.LastIndexOf(nl) == fileContents.Length - nllen)
{
lastEntry = fileContents.Substring(0, fileContents.Length - nllen).Substring(fileContents.Substring(0, fileContents.Length - nllen).LastIndexOf(nl) + nllen);
}
else
{
lastEntry = fileContents.Substring(fileContents.LastIndexOf(nl) + 2);
}
// search the new data for the last existing line
searchResultIndex = response.LastIndexOf(lastEntry);
}
// if the downloaded data contains the last record on file, append the new data
if (searchResultIndex != -1)
{
startIndex = searchResultIndex + lastEntry.Length;
File.AppendAllText(filePath + fileName, response.Substring(startIndex+1));
}
// else append all the data
else
{
Console.WriteLine("The last entry of the existing data was not found\nin the downloaded data. Appending all data.");
File.AppendAllText(filePath + fileName, response.Substring(109)); // the 109 index removes the file header from the new data
}
}
// if there is no file for this module, create the first one
catch (FileNotFoundException e)
{
// write data to file
Console.WriteLine("File does not exist, creating new data file.");
File.WriteAllText(filePath + fileName, response);
//Debug.WriteLine(response);
}
}
Console.WriteLine("Power file " + (i + 1) + " finished.");
//Debug.WriteLine("File " + (i + 1) + " finished.");
i++;
}
Console.WriteLine("\nPower data finished!\n");

Couple of suggestions wich I think will probably resolve the issue
First change your filePath string
string filePath = #"C:\Solar Yard Data\";
create a string with the full path
String fullFilePath = filePath + fileName;
then check to see if it exists and create it if it doesnt
if (!File.Exists(fullFilePath ))
File.Create(fullFilePath );
put the full path to the file in your streamReader
using (StreamReader sr2 = new StreamReader(fullFilePath))

How can I split a big text file into smaller file?

I have a big file with some text, and I want to split it into smaller files.
In this example, What I do:
I open a text file let's say with 10 000 lines into it
I set a number of package=300 here, which means, that's the small file limit, once a small file has 300 lines into it, close it, open a new file for writing for example (package2).
Same, as step 2.
You already know
Here is the code from my function that should do that. The ideea (what I dont' know) is how to close, and open a new file once it has reached the 300 limit (in our case here).
Let me show you what I'm talking about:
int nr = 1;
package=textBox1.Text;//how many lines/file (small file)
string packnr = nr.ToString();
string filer=package+"Pack-"+packnr+"+_"+date2+".txt";//name of small file/s
int packtester = 0;
int package= 300;
StreamReader freader = new StreamReader("bigfile.txt");
StreamWriter pak = new StreamWriter(filer);
while ((line = freader.ReadLine()) != null)
{
if (packtester < package)
{
pak.WriteLine(line);//writing line to small file
packtester++;//increasing the lines of small file
}
else if (packtester == package)//in this example, checking if the lines
//written, got to 300
{
packtester = 0;
pak.Close();//closing the file
nr++;//nr++ -> just for file name to be Pack-2;
packnr = nr.ToString();
StreamWriter pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");
}
}
I get this errors:
Cannot use local variable 'pak' before it is declared
A local variable named 'pak' cannot be declared in this scope because it would give a different meaning to 'pak', which is already used in a 'parent or current' scope to denote something else

Try this:
public void SplitFile()
{
int nr = 1;
int package = 300;
DateTime date2 = DateTime.Now;
int packtester = 0;
using (var freader = new StreamReader("bigfile.txt"))
{
StreamWriter pak = null;
try
{
pak = new StreamWriter(GetPackFilename(package, nr, date2), false);
string line;
while ((line = freader.ReadLine()) != null)
{
if (packtester < package)
{
pak.WriteLine(line); //writing line to small file
packtester++; //increasing the lines of small file
}
else
{
pak.Flush();
pak.Close(); //closing the file
packtester = 0;
nr++; //nr++ -> just for file name to be Pack-2;
pak = new StreamWriter(GetPackFilename(package, nr, date2), false);
}
}
}
finally
{
if(pak != null)
{
pak.Dispose();
}
}
}
}
private string GetPackFilename(int package, int nr, DateTime date2)
{
return string.Format("{0}Pack-{1}+_{2}.txt", package, nr, date2);
}

Logrotate can do this automatically for you. Years have been put into it and it's what people trust to handle their sometimes very large webserver logs.

Note that the code, as written, will not compile because you define the variable pak more than once. It should otherwise function, though it has some room for improvement.
When working with files, my suggestion and the general norm is to wrap your code in a using block, which is basically syntactic sugar built on top of a finally clause:
using (var stream = File.Open("C:\hi.txt"))
{
//write your code here. When this block is exited, stream will be disposed.
}
Is equivalent to:
try
{
var stream = File.Open(#"C:\hi.txt");
}
finally
{
stream.Dispose();
}
In addition, when working with files, always prefer opening file streams using very specific permissions and modes as opposed to using the more sparse constructors that assume some default options. For example:
var stream = new StreamWriter(File.Open(#"c:\hi.txt", FileMode.CreateNew, FileAccess.ReadWrite, FileShare.Read));
This will guarantee, for example, that files should not be overwritten -- instead, we assume that the file we want to open doesn't exist yet.
Oh, and instead of using the check you perform, I suggest using the EndOfStream property of the StreamReader object.

This code looks like it closes the stream and re-opens a new stream when you hit 300 lines. What exactly doesn't work in this code?
One thing you'll want to add is a final close (probably with a check so it doesn't try to close an already closed stream) in case you don't have an even multiple of 300 lines.
EDIT:
Due to your edit I see your problem. You don't need to redeclare pak in the last line of code, simply reinitialize it to another streamwriter.
(I don't remember if that is disposable but if it is you probably should do that before making a new one).
StreamWriter pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");
becomes
pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");