I am developing c# application, in which i am downloading package(zip file) from server machine.It was downloading properly, but recently our package data has got some changes which is flex application.And by using c# we are downloading it into c drive or d drive.
Now with the new package i am facing some problem as
Unable to read data from the transport connection: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
My code is below
byte[] packageData = null;
packageData = touchServerClient.DownloadFile("/packages/" + this.PackageName);
public byte[] DownloadFile(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(remoteSite.Url + url);
try
{
request.Method = "GET";
request.KeepAlive = false;
request.CookieContainer = new CookieContainer();
if (this.Cookies != null && this.Cookies.Count > 0)
request.CookieContainer.Add(this.Cookies);
HttpWebResponse webResponse = (HttpWebResponse)request.GetResponse();
// Console.WriteLine(response.StatusDescription);
Stream responseStream = webResponse.GetResponseStream();
int contentLength = Convert.ToInt32(webResponse.ContentLength);
byte[] fileData = StreamToByteArray(responseStream, contentLength);
return fileData;
}
public static byte[] StreamToByteArray(Stream stream, int initialLength)
{
// If we've been passed an unhelpful initial length, just
// use 32K.
if (initialLength < 1)
{
initialLength = 32768;
}
byte[] buffer = new byte[initialLength];
int read = 0;
int chunk;
while ((chunk = stream.Read(buffer, read, buffer.Length - read)) > 0)
{
read += chunk;
// If we've reached the end of our buffer, check to see if there's
// any more information
if (read == buffer.Length)
{
int nextByte = stream.ReadByte();
// End of stream? If so, we're done
if (nextByte == -1)
{
return buffer;
}
// Nope. Resize the buffer, put in the byte we've just
// read, and continue
byte[] newBuffer = new byte[buffer.Length * 2];
Array.Copy(buffer, newBuffer, buffer.Length);
newBuffer[read] = (byte)nextByte;
buffer = newBuffer;
read++;
}
}
// Buffer is now too big. Shrink it.
byte[] ret = new byte[read];
Array.Copy(buffer, ret, read);
return ret;
}
In the above function(StreamToByteArray) , i am getting error as
Unable to read data from the transport connection: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
Please help me on this, coz i am not supposed to change the code also.
Thanks in advance
Sangita
A few things to try:
Wrap your stream handling in a using statement.
This will clean up that resource.
using (Stream responseStream = webResponse.GetResponseStream())
{
int contentLength = Convert.ToInt32(webResponse.ContentLength);
byte[] fileData = StreamToByteArray(responseStream, contentLength);
return fileData;
}
Make sure there are no other heavy memory processes running on the same box. Particularly if they are making Socket-bound calls.
Try upping the value of the MaxUserPort registry value. Here is the article if you didn't see the link provided in the comments.
Related
I have to upload a large file to the server with the following code snippet:
static async Task LordNoBugAsync(string token, string filePath, string uri)
{
HttpWebRequest fileWebRequest = (HttpWebRequest)WebRequest.Create(uri);
fileWebRequest.Method = "PATCH";
fileWebRequest.AllowWriteStreamBuffering = false; //this line tells to upload by chunks
fileWebRequest.ContentType = "application/x-www-form-urlencoded";
fileWebRequest.Headers["Authorization"] = "PHOENIX-TOKEN " + token;
fileWebRequest.KeepAlive = false;
fileWebRequest.Timeout = System.Threading.Timeout.Infinite;
fileWebRequest.Proxy = null;
using (FileStream fileStream = File.OpenRead(filePath) )
{
fileWebRequest.ContentLength = fileStream.Length; //have to provide length in order to upload by chunks
int bufferSize = 512000;
byte[] buffer = new byte[bufferSize];
int lastBytesRead = 0;
int byteCount = 0;
Stream requestStream = fileWebRequest.GetRequestStream();
requestStream.WriteTimeout = System.Threading.Timeout.Infinite;
while ((lastBytesRead = fileStream.Read(buffer, 0, bufferSize)) != 0)
{
if (lastBytesRead > 0)
{
await requestStream.WriteAsync(buffer, 0, lastBytesRead);
//for some reasons didnt really write to stream, but in fact buffer has content, >60MB
byteCount += bufferSize;
}
}
requestStream.Flush();
try
{
requestStream.Close();
requestStream.Dispose();
}
catch
{
Console.Write("Error");
}
try
{
fileStream.Close();
fileStream.Dispose();
}
catch
{
Console.Write("Error");
}
}
...getting response parts...
}
In the code, I made a HttpWebRequest and push the content to server with buffering. The code works perfectly for any files under 60MB.
I tried a 70MB pdf. The buffer array has different content for each buffering. Yet, the request stream does not seem to be getting written. The bytecount also reached 70M, showing the file is properly read.
Edit (more info): I set the break point at requestStream.Close(). It clearly takes ~2 mins for the request stream to write in 60MB files but only takes 2ms for 70MB files.
My calling:
Task magic = LordNoBugAsync(token, nameofFile, path);
magic.Wait();
I am sure my calling is correct (it works for 0B to 60MB files).
Any advice or suggestion is much appreciated.
I want to download the first N Megabytes of huge XML File, so then I can close the broken tags with HTMLAgilityPack. Unfortunately, I can't use XMLReader.
I tried setting the Range on the HTTP Headers but that didn't seem to work, so now I'm trying this:
public string download(string url, int mb)
{
Int32 bytesToGet = 1048576 * mb;
HttpWebRequest request;
request = WebRequest.Create(url) as HttpWebRequest;
var buffer = new char[bytesToGet];
using (WebResponse response = request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
sr.Read(buffer, 0, bytesToGet);
}
}
return new string(buffer);
}
but this still doesn't work either. I tried it with mb=5 and I get just a few lines of the XML file.
You're only calling Read() once, which doesn't promise to fill your buffer. Keep count of bytes downloaded and keep reading until your buffer is full or the end of the stream is reached:
int offset = 0;
int bytesRead = 0;
do
{
bytesRead = sr.Read(buffer, offset, bytesToGet - offset);
offset += bytesRead;
} while (bytesRead > 0);
My problem is unable to download file from sever
My code work fine for any Google image-url (eg.:-http://blogs.independent.co.uk/wp-content/uploads/2012/12/google-zip.jpg)
but it is not work in case of my server image-url("eg.:-[http://110.15.445.87/xyz/abc/123.jpg]")
i dont know where is problem
my code is :-
//Create a stream for the file
Stream stream = null;
//This controls how many bytes to read at a time and send to the client
int bytesToRead = 10000;
// Buffer to read bytes in chunk size specified above
byte[] buffer = new Byte[bytesToRead];
// The number of bytes read
try
{
String Url = "http://112.196.32.51/ll1/UploadFiles/13062402231250.jpg";
//String Url = "http://blogs.independent.co.uk/wp-content/uploads/2012/12/google-zip.jpg";
//Create a WebRequest to get the file
HttpWebRequest fileReq = (HttpWebRequest)HttpWebRequest.Create(Url);
//Create a response for this request
HttpWebResponse fileResp = (HttpWebResponse)fileReq.GetResponse();
if (fileReq.ContentLength > 0)
fileResp.ContentLength = fileReq.ContentLength;
//Get the Stream returned from the response
stream = fileResp.GetResponseStream();
// prepare the response to the client. resp is the client Response
var resp = HttpContext.Current.Response;
//Indicate the type of data being sent
resp.ContentType = "application/octet-stream";
String fileName = "Rock.jpg";
//Name the file
resp.AddHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString());
int length;
do
{
// Verify that the client is connected.
if (resp.IsClientConnected)
{
// Read data into the buffer.
length = stream.Read(buffer, 0, bytesToRead);
// and write it out to the response's output stream
resp.OutputStream.Write(buffer, 0, length);
// Flush the data
resp.Flush();
//Clear the buffer
buffer = new Byte[bytesToRead];
}
else
{
// cancel the download if client has disconnected
length = -1;
}
} while (length > 0); //Repeat until no data is read
}
finally
{
if (stream != null)
{
//Close the input stream
stream.Close();
}
}
}
I'm developing a web page that needs to take an HTTP Post Request and read it into a byte array for further processing. I'm kind of stuck on how to do this, and I'm stumped on what is the best way to accomplish. Here is my code so far:
public override void ProcessRequest(HttpContext curContext)
{
if (curContext != null)
{
int totalBytes = curContext.Request.TotalBytes;
string encoding = curContext.Request.ContentEncoding.ToString();
int reqLength = curContext.Request.ContentLength;
long inputLength = curContext.Request.InputStream.Length;
Stream str = curContext.Request.InputStream;
}
}
I'm checking the length of the request and its total bytes which equals 128. Now do I just need to use a Stream object to get it into byte[] format? Am I going in the right direction? Not sure how to proceed. Any advice would be great. I need to get the entire HTTP request into byte[] field.
Thanks!
The simplest way is to copy it to a MemoryStream - then call ToArray if you need to.
If you're using .NET 4, that's really easy:
MemoryStream ms = new MemoryStream();
curContext.Request.InputStream.CopyTo(ms);
// If you need it...
byte[] data = ms.ToArray();
EDIT: If you're not using .NET 4, you can create your own implementation of CopyTo. Here's a version which acts as an extension method:
public static void CopyTo(this Stream source, Stream destination)
{
// TODO: Argument validation
byte[] buffer = new byte[16384]; // For example...
int bytesRead;
while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0)
{
destination.Write(buffer, 0, bytesRead);
}
}
You can just use WebClient for that...
WebClient c = new WebClient();
byte [] responseData = c.DownloadData(..)
Where .. is the URL address for the data.
I use MemoryStream and Response.GetResponseStream().CopyTo(stream)
HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(url);
myRequest.Method = "GET";
WebResponse myResponse = myRequest.GetResponse();
MemoryStream ms = new MemoryStream();
myResponse.GetResponseStream().CopyTo(ms);
byte[] data = ms.ToArray();
I have a function that does it, by sending in the response stream:
private byte[] ReadFully(Stream input)
{
try
{
int bytesBuffer = 1024;
byte[] buffer = new byte[bytesBuffer];
using (MemoryStream ms = new MemoryStream())
{
int readBytes;
while ((readBytes = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, readBytes);
}
return ms.ToArray();
}
}
catch (Exception ex)
{
// Exception handling here: Response.Write("Ex.: " + ex.Message);
}
}
Since you have Stream str = curContext.Request.InputStream;, you could then just do:
byte[] bytes = ReadFully(str);
If you had done this:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(someUri);
req.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
You would call it this way:
byte[] bytes = ReadFully(resp.GetResponseStream());
class WebFetch
{
static void Main(string[] args)
{
// used to build entire input
StringBuilder sb = new StringBuilder();
// used on each read operation
byte[] buf = new byte[8192];
// prepare the web page we will be asking for
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(#"http://www.google.com/search?q=google");
// execute the request
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
// we will read data via the response stream
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = 0;
do
{
// fill the buffer with data
count = resStream.Read(buf, 0, buf.Length);
// make sure we read some data
if (count != 0)
{
// translate from bytes to ASCII text
tempString = Encoding.ASCII.GetString(buf, 0, count);
// continue building the string
sb.Append(tempString);
}
}
while (count > 0); // any more data to read?
// print out page source
Console.WriteLine(sb.ToString());
Console.Read();
}
}
For all those cases when your context.Request.ContentLength is greather than zero, you can simply do:
byte[] contentBytes = context.Request.BinaryRead(context.Request.ContentLength);
I need to download images from a remote URL and serialise them to Isolated Storage, but am having trouble with figuring out how to get this to work, I'm currently serialising the URIs to the Images - and can get this to work, but want to download the image and store it in the file system, what is the best way to do this using C# and Silverlight.
I've tried finding some ways to do this, but they are too complicated, if possible I can store Image Data in the XML if this is a solution, but I just want to download the file - .NET on the desktop has many methods to do this sort of thing, but I need a Silverlight Solution.
If there are any examples that demonstrate this sort of thing, then this may help too - it seems like a straightforward issue but I cannot resolve it and have tried many things to make this work simply, so that I can save an Image from a remote URL to IsolatedStorage and do this Asynchronously.
I do have some code that does this at the moment but I can't seem to streamline it, will post it if no suitable alternatives are posted if needed.
Try this class. I hope it'll help you.
To start grabbing use:
ImageGrabber grabber = new ImageGrabber();
grabber.GrabImage(#"http://www.google.com.ua/images/srpr/nav_logo25.png");
BTW in this example I've used very nice method which allow to read bytes from stream (even if this stream doesn't support Seek operation).
public class ImageGrabber
{
public void GrabImage(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.BeginGetRequestStream(RequestCallback, request);
}
private void RequestCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
request.BeginGetResponse(GetResponseCallback, request);
}
private void GetResponseCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
// End the operation
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
Stream streamResponse = response.GetResponseStream();
byte[] bytes = ReadToEnd(streamResponse);
//save image to isolated storage as file.png
IsolatedStorageFile.GetUserStoreForApplication().CreateFile("file.png").Write(bytes, 0, bytes.Count());
}
private static byte[] ReadToEnd(Stream stream)
{
long originalPosition = stream.Position;
stream.Position = 0;
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
stream.Position = originalPosition;
}
}
}
File will be saved in isolated storage:
{System drive}:\Users\{User Name}\AppData\LocalLow\Microsoft\Silverlight\is\{bunch of autogenerated folders}\file.png
I've used the following:
WebClient client = new WebClient();
byte[] bytes = client.DownloadData("http://cdn.sstatic.net/stackoverflow/img/apple-touch-icon.png");