We Keep Coding

c# read and delete file used for messaging

I am writing a payments system which communicates with a very old POS interface through files.
The POS system tries to open/create a file (fixed to a single name). If it opens the file it then writes the payment request and closes the file, if it fails to open the file the POS system will wait and try again (sorry don't know how long between attempts, but very short).
I have to monitor the directory and when I see a new file arrive, then I have to try to open the file and block any access to the POS system from writing any new entries. While I have the file open I read through the records and send them to the payments system, once I receive confirmation I have to close the file and delete it.
My concern/problem is - how do I block the POS system from writing anything between the close and delete statements?
I can set the FileShare to Delete, which allows my program to delete the file before I close it, but this means other processes could also delete the file regardless of whether or not I processed the records successfully.
Option 1
void Main()
{
string filename = #"C:\Temp\CHARGES.TXT";
using (var fs = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.None))
{
var buffer = new byte[1024];
while (true)
{
var bytesRead = fs.Read(buffer, 0, buffer.Length);
if (bytesRead == 0)
break;
var text = ASCIIEncoding.ASCII.GetString(buffer, 0, bytesRead);
Console.Write(text);
}
}
File.Delete(filename);
}
Option 2
void Main()
{
string filename = #"C:\Temp\CHARGES.TXT";
using (var fs = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.Delete))
{
var buffer = new byte[1024];
while (true)
{
var bytesRead = fs.Read(buffer, 0, buffer.Length);
if (bytesRead == 0)
break;
var text = ASCIIEncoding.ASCII.GetString(buffer, 0, bytesRead);
Console.Write(text);
}
File.Delete(filename);
}
}
As this is a payments system any loss of records is not acceptable. Also as I have no control over the POS system I cannot change the way it works.
Thank you in advance

Thank you to #Lasse and #Sam for your helpful comments. I have made the following changes to the code:
void Main()
{
string filename = #"C:\Temp\CHARGES.TXT";
int start = 0;
int end = 0;
using (var fs = new FileStream(filename, FileMode.Open, FileAccess.ReadWrite, FileShare.Read))
{
// get a big enough buffer to hold the whole file and read it in
long fileSize = new FileInfo(filename).Length;
var buffer = new byte[fileSize];
var bytesRead = fs.Read(buffer, 0, buffer.Length);
try
{
// convert each line to ASCII to process through to API
while (end < bytesRead)
{
while (end < bytesRead && buffer[end] != 0x0A && buffer[end] != 0x0D)
{
end++;
}
var text = ASCIIEncoding.ASCII.GetString(buffer, start, end - start);
// Send to API
Console.WriteLine("Start [{0}] End [{1}] Text [{2}]", start, end, text);
if (end < bytesRead)
{
end += 2;
start = end;
}
if (end > bytesRead / 2)
{
throw new NotImplementedException();
}
}
}
catch (Exception e)
{
Console.WriteLine("just to prove the partial");
}
finally
{
// convert any remaining records back to bytes and write back to file
fs.SetLength(0);
fs.Write(buffer, end, bytesRead - end);
}
}
}
The process will now go through the file and write back any failed/unprocessed messages to the file at the end - this covers off my locking concerns as well as covering the individual record failures.
Thank you agin

FileStream returns Length = 0

I'm trying to read a local file and upload it on ftp server. when i read a image file, everything is ok, but when i read a doc or docx file, FileStream returns length = 0. Here is my code:
i checked with some other files, it appears that it only works fine with images and it returns 0 for any other file
if (!ftpClient.FileExists(fileName))
{
try
{
ftpClient.ValidateCertificate += (control, e) => { e.Accept = true; };
const int BUFFER_SIZE = 64 * 1024; // 64KB buffer
byte[] buffer = new byte[BUFFER_SIZE];
using (Stream readStream = new FileStream(tempFilePath, FileMode.Open, FileAccess.Read))
using (Stream writeStream = ftpClient.OpenWrite(fileName))
{
while (readStream.Position < readStream.Length)
{
buffer.Initialize();
int bytesRead = readStream.Read(buffer, 0, BUFFER_SIZE);
writeStream.Write(buffer, 0, bytesRead);
}
readStream.Flush();
readStream.Close();
writeStream.Flush();
writeStream.Close();
DeleteTempFile(tempFilePath);
return true;
}
}
catch (Exception ex)
{
return false;
}
}
I couldn't find whats wrong with it. could you please help me?

While this doesn't answer your specific question, you don't actually need to know the length of your stream. Just keep reading until you hit a zero length read. A zero byte read is guaranteed to indicate the the end of any stream.
Return Value
Type: System.Int32
The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.
while (true)
{
int bytesRead = readStream.Read(buffer, 0, BUFFER_SIZE);
if(bytesRead==0)
{
break;
}
writeStream.Write(buffer, 0, bytesRead);
}
alternatively:
readStream.CopyTo(writeStream);
is probably the most concise method of stating your goal...

it was just a silly mistake, i have two fileupload and i've saved the other fileupload, so it creates a zero length file. as it appears the code works fine.
thanks everyone.

FileResult buffered to memory

I'm trying to return large files via a controller ActionResult and have implemented a custom FileResult class like the following.
public class StreamedFileResult : FileResult
{
private string _FilePath;
public StreamedFileResult(string filePath, string contentType)
: base(contentType)
{
_FilePath = filePath;
}
protected override void WriteFile(System.Web.HttpResponseBase response)
{
using (FileStream fs = new FileStream(_FilePath, FileMode.Open, FileAccess.Read))
{
int bufferLength = 65536;
byte[] buffer = new byte[bufferLength];
int bytesRead = 0;
while (true)
{
bytesRead = fs.Read(buffer, 0, bufferLength);
if (bytesRead == 0)
{
break;
}
response.OutputStream.Write(buffer, 0, bytesRead);
}
}
}
}
However the problem I am having is that entire file appears to be buffered into memory. What would I need to do to prevent this?

You need to flush the response in order to prevent buffering. However if you keep on buffering without setting content-length, user will not see any progress. So in order for users to see proper progress, IIS buffers entire content, calculates content-length, applies compression and then sends the response. We have adopted following procedure to deliver files to client with high performance.
FileInfo path = new FileInfo(filePath);
// user will not see a progress if content-length is not specified
response.AddHeader("Content-Length", path.Length.ToString());
response.Flush();// do not add anymore headers after this...
byte[] buffer = new byte[ 4 * 1024 ]; // 4kb is a good for network chunk
using(FileStream fs = path.OpenRead()){
int count = 0;
while( (count = fs.Read(buffer,0,buffer.Length)) >0 ){
if(!response.IsClientConnected)
{
// network connection broke for some reason..
break;
}
response.OutputStream.Write(buffer,0,count);
response.Flush(); // this will prevent buffering...
}
}
You can change buffer size, but 4kb is ideal as lower level file system also reads buffer in chunks of 4kb.

Akash Kava is partly right and partly wrong. You DO NOT need to add the Content-Length header or do the flush afterward. But you DO, need to periodically flush response.OutputStream and then response. ASP.NET MVC (at least version 5) will automatically convert this into a "Transfer-Encoding: chunked" response.
byte[] buffer = new byte[ 4 * 1024 ]; // 4kb is a good for network chunk
using(FileStream fs = path.OpenRead()){
int count = 0;
while( (count = fs.Read(buffer,0,buffer.Length)) >0 ){
if(!response.IsClientConnected)
{
// network connection broke for some reason..
break;
}
response.OutputStream.Write(buffer,0,count);
response.OutputStream.Flush();
response.Flush(); // this will prevent buffering...
}
}
I tested it and it works.

SharpZipLib - ZipException "System.ArgumentOutOfRangeException" - Why am I getting this exception?

I'm using SharpZipLib to unzip files. My code has been working nicely for all zipfiles except the zip file what i am extracting now...
Got this exception:
System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: length
The exception is being thrown at size = s.Read(data, 0, data.Length);
Hereb is my code...
public static void UnzipFile(string sourcePath, string targetDirectory)
{
try
{
using (ZipInputStream s = new ZipInputStream(File.OpenRead(sourcePath)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
//string directoryName = Path.GetDirectoryName(theEntry.Name);
string fileName = Path.GetFileName(theEntry.Name);
if (targetDirectory.Length > 0)
{
Directory.CreateDirectory(targetDirectory);
}
if (fileName != String.Empty)
{
using (FileStream streamWriter = File.Create(targetDirectory + fileName))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
catch (Exception ex)
{
throw new Exception("Error unzipping file \"" + sourcePath + "\"", ex);
}
}

Looks like a bug to me. Fortunately, you have access to the code, so you should be able to see exactly where it's going wrong. I suggest you build a debug version of SharpZipLib, break on the line which is throwing the exception, and have a look at what it's actually testing.
It should be fine to read into a 2K buffer even if there's not 2K of data left.
(I wouldn't actually write the code quite how you have, but that's a different matter. I'd also move it into its own utility method - the act of copying all the data from one stream to another is pretty common. There's no need to tie it to zip.)

Looking at the code, you are reading the same set of bytes again (and advancing the position).
size = s.Read(data, 0, data.Length);
An example from here shows that the 2nd argument should be a moving position & not a fixed number.

Change your code int size = 2048; to int size = data.Length;. You won't take OutOfRange exception.
using (FileStream streamWriter = File.Create(targetDirectory + fileName))
{
int size = data.Length;
byte[] data = new byte[size];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}

how to split a wmv file

im doing a application in which i split a wmv file and transfer it to otherlocation(in 'x' kbs) .after the transfer gets completed the file doesnt play,it gives a message as the format is not supported.is there anyother way to do it.
sory i will explain what im doing now
i wrote an remote application,i want to transfer a .wmv file from one machine to other,i want to split the .wmv and send it to the remote machine and use it there.if i try to send the complete file means it will take lot of memory that seems very bad.so i want to split it and send it.but the file doesnt gets played it raises an exception the format is not supported.
the following is the code im doing i just done it in the local machine itself(not remoting):
try
{
FileStream fswrite = new FileStream("D:\\Movie.wmv", FileMode.Create);
int pointer = 1;
int bufferlength = 12488;
int RemainingLen = 0;
int AppLen = 0;
FileStream fst = new FileStream("E:\\Movie.wmv", FileMode.Open);
int TotalLen = (int)fst.Length;
fst.Close();
while (pointer != 0)
{
byte[] svid = new byte[bufferlength];
using (FileStream fst1 = new FileStream("E:\\Movie.wmv", FileMode.Open))
{
pointer = fst1.Read(svid, AppLen, bufferlength);
fst1.Close();
}
fswrite.Write(svid, 0, pointer);
AppLen += bufferlength;
RemainingLen = TotalLen-AppLen;
if(RemainingLen < bufferlength)
{
byte[] svid1 = new byte[RemainingLen];
using (FileStream fst2 = new FileStream("E:\\Movie.wmv", FileMode.Open))
{
pointer = fst2.Read(svid1, 0, RemainingLen);
fst2.Close();
}
fswrite.Write(svid, 0, pointer);
break;
}
}
fswrite.Close();
}
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}

You'll probably find Good way to send a large file over a network in C# helpful.

Im going to make the assumtion your spliting the file when your sending it, and not trying to have the wmv in 3 different files on the remote machine.
When your sending the file what you basicly do is this:
Local machine
1) Read 16k bytes ( Or whatever number you prefere )
2) Send those 16k bytes over the network
3) Repeat above steps untill done
Remote machine
1) Listen for a connection
2) Get 16k bytes
3) Write 16k bytes
4) Repeat untill done.
This method will work, but your kind of inventing the wheel again, i would recommend using either something as simple as File.Copy ( Works fine over the network ) or if that does not meet your needs perhaps using a FTP client / server solution ( Plenty of C# examples on the net that can be hosted inside your application ).

i tried this
private void Splitinthread()
{
int bufferlength = 2048;
int pointer = 1;
int offset = 0;
int length = 0;
byte[] buff = new byte[2048];
FileStream fstwrite = new FileStream("D:\\TEST.wmv", FileMode.Create);
FileStream fst2 = new FileStream("E:\\karthi.wmv", FileMode.Open);
int Tot_Len = (int)fst2.Length;
int Remain_Buff = 0;
//Stream fst = File.OpenRead("E:\\karth.wmv");
while (pointer != 0)
{
try
{
fst2.Read(buff, 0, bufferlength);
fstwrite.Write(buff, 0, bufferlength);
offset += bufferlength;
Remain_Buff = Tot_Len - offset;
Fileprogress.Value = CalculateProgress(offset, Tot_Len);
if (Remain_Buff < bufferlength)
{
byte[] buff1 = new byte[Remain_Buff];
pointer = fst2.Read(buff1, 0, Remain_Buff);
fstwrite.Write(buff1, 0, pointer);
Fileprogress.Value = CalculateProgress(offset, Tot_Len);
fstwrite.Close();
fst2.Close();
break;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
MessageBox.Show("Completed");
}