We Keep Coding

can't download from HttpWebRequest more than once

I am new to C# and trying to write a download manager with HttpWebRequest.
my manager calls a new class for breaking file into chunks for every file and that class calls a new downloader class.
my code for download is:
private void StartDownload()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
Console.WriteLine("request");
request.AddRange((int)_startPos, (int)_endPos);
WebResponse res = request.GetResponse();
Console.WriteLine("res: "+res);
RequestContentLength = 0;
try
{
bool isFinished = false;
isDownloading = true;
double speed = 0;
using (var responseStream = res.GetResponseStream())
{
using (FileStream localFileStream = new FileStream(FilePath, FileMode.Append))
{
Console.WriteLine("download started - "+ Url);
var buffer = new byte[4096];
int bytesRead;
DateTime lastDate = DateTime.Now;
int lastDownloaded = 0;
while ((bytesRead = responseStream.Read(buffer, 0, buffer.Length)) > 0 && isDownloading)
{
totalBytesRead += bytesRead;
if ((DateTime.Now - lastDate).TotalSeconds >= 1)
{
lastDate = DateTime.Now;
speed = totalBytesRead - lastDownloaded;
lastDownloaded = totalBytesRead;
}
RequestContentLength += bytesRead;
localFileStream.Write(buffer, 0, bytesRead);
double percentage = totalBytesRead * 100 / FileHeader.FileSize;
onDownloadProgress?.Invoke(ID, fname, percentage, totalBytesRead, FileHeader.FileSize, speed);
}
isFinished = true;
}
}
if (isFinished) {
onDownloadFinished?.Invoke(ID, fname, true, "success");
}
else
{
onDownloadFinished?.Invoke(ID, fname, false, "error");
}
isDownloading = false;
}
catch (Exception ex)
{
isDownloading = false;
onDownloadFinished?.Invoke(ID, fname, false, ex.Message);
}
}
the code works fine for the FIRST download ( I cropped the rest to make it shorter) but when I call it after first download I get Console.WriteLine("request"); to output but the rest stops and after a minute or two I get a timeout error at WebResponse res = request.GetResponse();
I couldn't name my exact problem and couldn't find a solution.
there are some similar posts about this and I tried most of them but doesn't work for me.

C# UWP File output for encryption and decryption code seems to stop working for files over 128 MB

Wrote some C# UWP code to encrypt files using the new encryption engine, and recently added code to make it be able to handle more bulky files by splitting the files up into parts and encrypting each part seperately since the libraries encrypt and decrypt functions won't take gigantic buffers. Now, it seems to be in some cases either outputting nothing to the file or giving a "system.exception" error in the Visual Studio debugger, which as you can imagine, isn't very helpful. Here's the code for this part of the program:
private async System.Threading.Tasks.Task encryptFile(StorageFile file, string key)
{
Loading.IsActive = true;
string HashAlgorithmName = HashAlgorithmNames.Md5;
HashAlgorithmProvider HashProvider = HashAlgorithmProvider.OpenAlgorithm(HashAlgorithmName);
CryptographicHash HashObject = HashProvider.CreateHash();
Windows.Storage.Streams.IBuffer keyAsBuffer = CryptographicBuffer.ConvertStringToBinary(key, BinaryStringEncoding.Utf16BE);
HashObject.Append(keyAsBuffer);
Windows.Storage.Streams.IBuffer hashedKeyAsBuffer = HashObject.GetValueAndReset();
string hashedKey = CryptographicBuffer.EncodeToBase64String(hashedKeyAsBuffer);
key = hashedKey;
string algorithmName = SymmetricAlgorithmNames.AesEcbPkcs7;
SymmetricKeyAlgorithmProvider encryptionProvider = SymmetricKeyAlgorithmProvider.OpenAlgorithm(algorithmName);
Windows.Storage.Streams.IBuffer keyBuffer = CryptographicBuffer.CreateFromByteArray(Encoding.ASCII.GetBytes(key));
CryptographicKey fullKey = encryptionProvider.CreateSymmetricKey(keyBuffer);
Stream fileStream = await file.OpenStreamForReadAsync();
byte[] fileBytes = ReadToEnd(fileStream);
Stream writeFileStream = await file.OpenStreamForWriteAsync();
IEnumerable<byte[]> splitFileBytes = ArraySplit(fileBytes, 100000000);
writeFileStream.Seek(0, SeekOrigin.Begin);
foreach (byte[] fileBytesFor in splitFileBytes)
{
Windows.Storage.Streams.IBuffer fileBuffer = fileBytesFor.AsBuffer();
Windows.Storage.Streams.IBuffer encryptedFileBuffer = CryptographicEngine.Encrypt(fullKey, fileBuffer, null);
Stream encryptedFileStream = encryptedFileBuffer.AsStream();
byte[] encryptedFileBytes = ReadToEnd(encryptedFileStream);
await writeFileStream.WriteAsync(encryptedFileBytes, 0, encryptedFileBytes.Length);
writeFileStream.Seek(0, SeekOrigin.End);
}
fileStream.Dispose();
writeFileStream.Dispose();
Loading.IsActive = false;
}
private async System.Threading.Tasks.Task decryptFile(StorageFile file, string key)
{
string HashAlgorithmName = HashAlgorithmNames.Md5;
HashAlgorithmProvider HashProvider = HashAlgorithmProvider.OpenAlgorithm(HashAlgorithmName);
CryptographicHash HashObject = HashProvider.CreateHash();
Windows.Storage.Streams.IBuffer keyAsBuffer = CryptographicBuffer.ConvertStringToBinary(key, BinaryStringEncoding.Utf16BE);
HashObject.Append(keyAsBuffer);
Windows.Storage.Streams.IBuffer hashedKeyAsBuffer = HashObject.GetValueAndReset();
string hashedKey = CryptographicBuffer.EncodeToBase64String(hashedKeyAsBuffer);
key = hashedKey;
string algorithmName = SymmetricAlgorithmNames.AesEcbPkcs7;
SymmetricKeyAlgorithmProvider encryptionProvider = SymmetricKeyAlgorithmProvider.OpenAlgorithm(algorithmName);
Windows.Storage.Streams.IBuffer keyBuffer = CryptographicBuffer.CreateFromByteArray(Encoding.ASCII.GetBytes(key));
CryptographicKey fullKey = encryptionProvider.CreateSymmetricKey(keyBuffer);
Stream fileStream = await file.OpenStreamForReadAsync();
byte[] fileBytes = ReadToEnd(fileStream);
Stream writeFileStream = await file.OpenStreamForWriteAsync();
Loading.IsActive = true;
writeFileStream.SetLength(0);
IEnumerable<byte[]> splitFileBytes = ArraySplit(fileBytes, 100000000);
writeFileStream.Seek(0, SeekOrigin.Begin);
foreach (byte[] fileBytesFor in splitFileBytes)
{
Windows.Storage.Streams.IBuffer fileBuffer = fileBytesFor.AsBuffer();
Windows.Storage.Streams.IBuffer decryptedFileBuffer = CryptographicEngine.Decrypt(fullKey, fileBuffer, null);
Stream decryptedFileStream = decryptedFileBuffer.AsStream();
byte[] decryptedFileBytes = ReadToEnd(decryptedFileStream);
await writeFileStream.WriteAsync(decryptedFileBytes, 0, decryptedFileBytes.Length);
writeFileStream.Seek(0, SeekOrigin.End);
}
fileStream.Dispose();
writeFileStream.Dispose();
Loading.IsActive = false;
}
public static byte[] StreamToByteArray(Stream inputStream)
{
byte[] bytes = new byte[375000000000000000];
using (MemoryStream memoryStream = new MemoryStream())
{
int count;
while ((count = inputStream.Read(bytes, 0, bytes.Length)) > 0)
{
memoryStream.Write(bytes, 0, count);
}
return memoryStream.ToArray();
}
}
public static byte[] ReadToEnd(System.IO.Stream stream)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = 0;
}
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
private IEnumerable<byte[]> ArraySplit(byte[] bArray, int intBufforLengt)
{
int bArrayLenght = bArray.Length;
byte[] bReturn = null;
int i = 0;
for (; bArrayLenght > (i + 1) * intBufforLengt; i++)
{
bReturn = new byte[intBufforLengt];
Array.Copy(bArray, i * intBufforLengt, bReturn, 0, intBufforLengt);
yield return bReturn;
}
int intBufforLeft = bArrayLenght - i * intBufforLengt;
if (intBufforLeft > 0)
{
bReturn = new byte[intBufforLeft];
Array.Copy(bArray, i * intBufforLengt, bReturn, 0, intBufforLeft);
yield return bReturn;
}
}

How to stream an mp3 file using a temporary file?

I am working an Mp3 streamer. To stream mp3 file from url, I want to use a temporary file. But trying to read and write the same file throws IOException for File.ReadAllBytes because the file is in use. How can I get throught this problem?
long pos = 0;
string path = pathtothetempfile;
MemoryStream ms = new MemoryStream();
FileStream fs = new FileStream(path, FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite);
do
{
bytesRead = responseStream.Read(buffer, 0, buffer.Length);
fs.Write(buffer, 0, bytesRead);
fs.Flush();
byte[] tempBuffer = File.ReadAllBytes(path);
pos = ms.Position;
ms = new MemoryStream(tempBuffer);
ms.Position = pos;
frame = Mp3Frame.LoadFromStream(ms);
//....codes....
}
while(bytesRead > 0)

I have found an answer by myself by searching so much. This answer contains NAudio to control the Mp3 and it is RAM friendly by reading stream partially. I am sharing it for other people who has the same problem.
WaveOut waveOut;
AcmMp3FrameDecompressor decompressor;
BufferedWaveProvider provider;
bool firstPlay = true;
public void Play()
{
Task.Run(() =>
{
#region WebRequest creator
HttpWebResponse response = null;
if (avgbytes < 0)
{
HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
req.AllowAutoRedirect = true;
req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0";
response = req.GetResponse() as HttpWebResponse;
contentLength = response.ContentLength;
}
else
response = Helper.CreateAudioWebRequest(url, currentTime, avgbytes)
.GetResponse() as HttpWebResponse;
Stream responseStream = response.GetResponseStream();
#endregion
#region Local Variables
byte[] buffer = new byte[17 * 1024];
byte[] bigBuffer = new byte[response.ContentLength];
int bytesRead = 0;
long pos = 0;
long postotal = 0;
string path = System.IO.Path.GetTempPath() + Guid.NewGuid().ToString() + ".mp3";
#endregion
Mp3Frame frame;
FileStream fs = new FileStream(path,
FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
do
{
bytesRead = responseStream.Read(buffer, 0, buffer.Length);
fs.Write(buffer, 0, bytesRead);
fs.Flush();
using (MemoryStream ms = new MemoryStream(ReadPartial(fs, postotal, 1024 * 10)))
{
ms.Position = 0;
frame = Mp3Frame.LoadFromStream(ms);
if (frame == null)
{
continue;
}
pos = ms.Position;
postotal += pos;
}
#region First Play
if (firstPlay)
{
avgbytes = new Mp3WaveFormat(frame.SampleRate, frame.ChannelMode == ChannelMode.Mono ? 1 : 2,
frame.FrameLength, frame.BitRate).AverageBytesPerSecond;
duration = (int)(response.ContentLength * 1d / avgbytes);
firstPlay = false;
}
#endregion
#region Decompress Frame
if (decompressor == null)
{
decompressor = CreateFrameDecompressor(frame) as AcmMp3FrameDecompressor;
provider = new BufferedWaveProvider(decompressor.OutputFormat);
provider.BufferDuration = TimeSpan.FromSeconds(20);
}
int decompressed = decompressor.DecompressFrame(frame, buffer, 0);
#endregion
#region BufferedWaveProvider Area
if (IsBufferNearlyFull(provider))
{
Thread.Sleep(500);
}
provider.AddSamples(buffer, 0, decompressed);
#endregion
if (provider.BufferedDuration.TotalSeconds >= 2 && waveOut == null)
{
waveOut = new WaveOut();
waveOut.Init(provider);
waveOut.Play();
}
}
while (postotal != contentLength || bytesRead > 0 || waveOut==null ||
(waveOut != null && waveOut.PlaybackState == PlaybackState.Playing));
});
}
public static byte[] ReadStreamPartially(System.IO.Stream stream, long offset, long count)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = offset;
}
try
{
byte[] readBuffer = new byte[4096];
byte[] total = new byte[count];
int totalBytesRead = 0;
int byteRead;
while ((byteRead = stream.ReadByte()) != -1)
{
Buffer.SetByte(total, totalBytesRead, (byte)byteRead);
totalBytesRead++;
if (totalBytesRead == count)
{
stream.Position = originalPosition;
break;
}
}
if (totalBytesRead < count)
{
byte[] temp = new byte[totalBytesRead];
Buffer.BlockCopy(total, 0, temp, 0, totalBytesRead);
stream.Position = originalPosition;
return temp;
}
return total;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
private bool IsBufferNearlyFull(BufferedWaveProvider bufferedWaveProvider)
{
return bufferedWaveProvider != null &&
bufferedWaveProvider.BufferLength - bufferedWaveProvider.BufferedBytes
< bufferedWaveProvider.WaveFormat.AverageBytesPerSecond / 4;
}
private static IMp3FrameDecompressor CreateFrameDecompressor(Mp3Frame frame)
{
WaveFormat waveFormat = new Mp3WaveFormat(frame.SampleRate, frame.ChannelMode == ChannelMode.Mono ? 1 : 2,
frame.FrameLength, frame.BitRate);
return new AcmMp3FrameDecompressor(waveFormat);
}

C# FileStream reads bytes incorrectly

I am trying to develop an app that will upload large files to a web server running PHP. Almost immediately, I stumbled upon a problem that the file is not split correctly.
Currently I have this piece of code
string adrese = "c:\\directory\\file.jpg";
int garums = 16384;
String ext = Path.GetExtension(adrese);
FileStream file = /*/File.Open(adrese, FileMode.Open);/*/
new FileStream(adrese, FileMode.Open, System.IO.FileAccess.Read);
long fgar = file.Length; //100%
long counter = garums;
first = true;
byte[] chunk = new byte[garums];
while (true)
{
int index = 0;
//long Controll = counter+garums;
while (index < chunk.Length)
{
int bytesRead = file.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
/*byte[] biti = new byte[index];
for (int i = 0; i < index; i++)
{
biti[i] = chunk[i];
}
chunk = new byte[index];
chunk = biti;*/
break;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
byte[] biti = new byte[index];
for (int i = 0; i < index; i++)
{
biti[i] = chunk[i];
}
chunk = new byte[index];
chunk = biti;
// index is the number of bytes in the chunk
sutam(Convert.ToBase64String(chunk),ext);
}
double procentuali = ((counter * 100) / fgar);
if (procentuali > 99)
{
procentuali = 100;
}
progressBar1.Value = (int)Math.Round(procentuali);
label1.Text = "" + procentuali;
counter = counter+garums;
if (index != garums) // We didn't read a full chunk: we're done
{
return;
}
}
file.Close();
Everything works if I set garums to 1, but who will wait for a year or so to upload a file sized multiple GB's.
I would be pleased if you could tell me what is wrong and how to fix this.

Try this instead to upload in chunks:
private void ConvertToChunks()
{
//Open file
string file = MapPath("~/temp/1.xps");
FileStream fileStream = new FileStream(file, FileMode.Open, FileAccess.Read);
//Chunk size that will be sent to Server
int chunkSize = 1024;
// Unique file name
string fileName = Guid.NewGuid() + Path.GetExtension(file);
int totalChunks = (int)Math.Ceiling((double)fileStream.Length / chunkSize);
// Loop through the whole stream and send it chunk by chunk;
for (int i = 0; i < totalChunks; i++)
{
int startIndex = i * chunkSize;
int endIndex = (int)(startIndex + chunkSize > fileStream.Length ? fileStream.Length : startIndex + chunkSize);
int length = endIndex - startIndex;
byte[] bytes = new byte[length];
fileStream.Read(bytes, 0, bytes.Length);
ChunkRequest(fileName, bytes);
}
}
private void ChunkRequest(string fileName,byte[] buffer)
{
//Request url, Method=post Length and data.
string requestURL = "http://localhost:63654/hello.ashx";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestURL);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// Chunk(buffer) is converted to Base64 string that will be convert to Bytes on the handler.
string requestParameters = #"fileName=" + fileName + "&data=" + HttpUtility.UrlEncode( Convert.ToBase64String(buffer) );
// finally whole request will be converted to bytes that will be transferred to HttpHandler
byte[] byteData = Encoding.UTF8.GetBytes(requestParameters);
request.ContentLength = byteData.Length;
Stream writer = request.GetRequestStream();
writer.Write(byteData, 0, byteData.Length);
writer.Close();
// here we will receive the response from HttpHandler
StreamReader stIn = new StreamReader(request.GetResponse().GetResponseStream());
string strResponse = stIn.ReadToEnd();
stIn.Close();
}

Base64 from e.ChosenPhoto apparently corrupted

//convert photo to baos
var memoryStream = new System.IO.MemoryStream();
e.ChosenPhoto.CopyTo(memoryStream);
//string baos = memoryStream.ToString();
byte[] result = memoryStream.ToArray();
String base64 = System.Convert.ToBase64String(result);
String post_data = "&image=" + base64;
...
wc.UploadStringAsync(imgur_api,"POST",post_data);
I am using this code to upload an image to the Imgur API v3 using WebClient. The image being selected is either one of the 7 photos provided by the Windows Phone 7.1 emulator, or the simulated camera images. When I try to load the images, they are a largely-grey corrupted mess. Am I generating the base64 properly and/or do I need to render a Bitmap of the picture first before creating the byte[] and base64?
Thanks in advance!

Use something like Uri.EscapeDataString to escape the data so that special URL characters are not interpreted.

I use this
private void PhotoChooserTaskCompleted(object sender, PhotoResult e)
{
if (e.TaskResult != TaskResult.OK) return;
var bimg = new BitmapImage();
bimg.SetSource(e.ChosenPhoto);
var sbytedata = ReadToEnd(e.ChosenPhoto);
}
public static byte[] ReadToEnd(System.IO.Stream stream)
{
long originalPosition = stream.Position;
stream.Position = 0;
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
stream.Position = originalPosition;
}
}
And upload byte[] to server. Hope it's help