httpclient GetAsync doesn't work for large content - c#

I am trying to download the string content from an endpoint in question using xamarin and http requests. The endpoint I'm supporting with python flask, as well as the content of the page, which is similar to this:
[{"name":"test1","timestamp":12312,"type":"ultrasonic","value":65535},
{"name":"test2","timestamp":3123123,"type":"ultrasonic","value":65535}...]
When there are not many lines in the content of the page, I can access the content and save it to a file inside the android device. When the content is very long, with many lines, the application simply closes.
My code is as follows:
public async Task<string> GetNews()
{
List<string> valores = new List<string>();
var response = await client.GetAsync(get_url);
var responseData = await response.Content.ReadAsStringAsync();
string nome = DateTime.Now.ToString("HH:mm:ss");
//creating the subsequent file
string filename = System.IO.Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.Personal),
"Sensor File - " + nome + ".json");
//inserting the content of the GET into the generated text file
System.IO.File.WriteAllText(filename, responseData);
return null;
}
public async void Download_Clicked(object sender, EventArgs e)
{
try
{
await GetNews();
//informing in the first label the directory of the saved file
lb.Text = "File saved successfully!";
}
catch
{
lb.Text = "Connection not possible, enter the correct URL.";
}
}
I did add .ConfigureAwait(false) to each await line, but it didn't work... What am I doing wrong?
Thank you!

When you used HttpClient to download large amounts of data (50 megabytes or more), then the app should stream those downloads and not use the default buffering. If the default buffering is used the client memory usage will get very large, potentially resulting in substantially reduced performance.
You could try to get the response body and load into memory.
using (var response = await client.GetAsync(get_url))
using (Stream streamToReadFrom =
await response.Content.ReadAsStreamAsync())
{
...........
//string fileToWriteTo = Path.GetTempFileName();
//using (Stream streamToWriteTo = File.Open(fileToWriteTo, FileMode.Create))
//{
// await streamToReadFrom.CopyToAsync(streamToWriteTo);
//}
}

Related

Why do some files become corrupt when written from byte[] array

I am retrieving mp4 clips via a REST API for a security camera. The clips are 2-5 seconds long and are almost always less than 600 KB.
The info for the clips is populated into a DataGridView, and that info includes the URI for the REST API. I get the byte[] array via the API, write it to a file, and then play it through windows media player. About 20% of the clips are corrupt, but they are always exactly 919 bytes when they are. I can't figure out why this is happening.
I know the URI isn't invalid, because Blink.GetClipAsync(uri, user.getToken()); would throw an exception.
Am I doing something wrong that could cause this?
private async void ClipView_CellContentClick(object sender, DataGridViewCellEventArgs e)
{
String uri = "";
int rowIndex = e.RowIndex;
try
{
uri = ClipView.Rows[rowIndex].Tag.ToString();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
String fileName = Path.GetFileName(uri);
byte[] mp4 = await Blink.GetClipAsync(uri, user.getToken());
String path = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
String pathString = System.IO.Path.Combine(path, "Blink Desktop");
String filePathString = System.IO.Path.Combine(pathString, fileName);
if (!System.IO.Directory.Exists(pathString))
System.IO.Directory.CreateDirectory(pathString);
if (!System.IO.File.Exists(filePathString))
{
File.WriteAllBytes(filePathString, mp4);
}
Properties.Settings.Default.currentClipSelection = filePathString;
var ClipPlayer = new Clip_Player();
ClipPlayer.Show();
}
public async Task<byte[]> GetClipAsync(String url, String token)
{
byte[] response;
using (var httpClient = new HttpClient())
{
using (var request = new HttpRequestMessage(new HttpMethod("GET"), Properties.Settings.Default.RegionAPI + url))
{
request.Headers.TryAddWithoutValidation("token-auth", token);
HttpContent content = httpClient.SendAsync(request).Result.Content;
response = await content.ReadAsByteArrayAsync();
}
}
return response;
}

Uploading File from console application to WebAPI

I'm trying to post a file + some info to a WebApi I control. My problem is that I can't access the file on the WebAPI side, all other fields are OK.
Here is my Console Application code
using (HttpClient client = new HttpClient())
{
using (MultipartFormDataContent content = new MultipartFormDataContent())
{
string filename = "my_filename.png";
content.Add(new StringContent(DateTime.Now.ToString("yyyy-MM-dd")), "data");
byte[] file_bytes = webClient.DownloadData($"https://my_url/my_file.png");
content.Add( new ByteArrayContent(file_bytes), "file");
string requestUri = "http://localhost:51114/api/File";
HttpResponseMessage result = client.PostAsync(requestUri, content).Result;
Console.WriteLine("Upload result {0}", result.StatusCode);
}
}
Here is my WebAPI Code
[HttpPost]
public void Post(IFormFile file, [FromForm] DateTime data)
{
if (file == null || file.Length == 0)
{
Response.StatusCode = StatusCodes.Status400BadRequest;
return;
}
// Never reaches this point..... file is null
}
Any pointers on what I might be missing?
If i'm not mistaken, you can submit a file to a WebAPI endpoint sending it as FormData with a Content-Type : multipart/form-data, something like this.
[HttpPost]
[Route("..."]
public void ReceiveFile()
{
System.Web.HttpPostedFile file = HttpContext.Current.Request.Files["keyName"];
System.IO.MemoryStream mem = new System.IO.MemoryStream();
file.InputStream.CopyTo(mem);
byte[] data = mem.ToArray();
// you can replace the MemoryStream with file.saveAs("path") if you want.
}
You can grab out the content and convert it into a byte array in 2 lines of code, assuming you are only sending a single file (Note) its a good idea to use async for file upload so you don't consume as much cpu time:
var provider = await Request.Content.ReadAsMultipartAsync(new MultipartMemoryStreamProvider());
var file = provider.Contents.Single();

Task Was Canceled

I am trying to download PDF from the database by API call..
All PDF are getting downloaded but only for one row I am getting task was canceled exception
public async System.Threading.Tasks.Task<ActionResult> Record(string empNo)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/pdf"));
var response = client.GetAsync("URL?empNo=" + empNo).Result;
if (response.IsSuccessStatusCode)
{
var files = Directory.GetFiles(Server.MapPath(#"~/EmpPDF/"));
foreach (var file in files)
{
try
{
System.IO.File.Delete(file);
}
catch
{ }
}
var bytes = await response.Content.ReadAsByteArrayAsync();
using (Stream writer = System.IO.File.Create(System.Web.HttpContext.Current.Server.MapPath(#"~EmpPDF/" + empNo + ".pdf")))
{
writer.Write(bytes, 0, bytes.Length);
writer.Flush();
responsePDFPath = #"/EmpPDF/" + empNo + ".pdf"; //+ response.Content.Headers.ContentDisposition.FileName;
}
ViewBag.PathToPdf = responsePDFPath;
return View();
}
}
}
I am getting Task was cancelled exception for this code.
Deadlock because of mixing of blocking calls (.Result) and async-await code.
var response = client.GetAsync("URL?empNo=" + empNo).Result;
That line should be awaited as well
var response = await client.GetAsync("URL?empNo=" + empNo);
Try making sure to dispose the response object:
using(var response = await client.GetAsync("URL?empNo=" + empNo))
{
// Rest of function
}
Leaving too many HttpResponseMessage objects un-disposed can tie up network resources, meaning that later requests will block waiting for one of those resources (which will never get freed, because there is no memory pressure to trigger a GC) and eventually time out.
I've had tons of projects that experience this very same deadlock. It's very important to dispose things that are IDiposable as soon as possible, especially HttpResponseMessage objects.

Download any kind of files using WebClient or HttpClient?

I know there are other questions similar to mine. But the problem I face is different.
I want to download any file. It could be an Image, PDF, Video. The thing is I won't be able to know the file that I'm downloading. That means I don't know the file name. So I tried the following:
#1:
private Task<string> DownloadFile(Uri fileUri, string locationToStoreTo)
{
using (var client = new WebClient())
{
client.OpenRead(fileUri);
var headerContentDisposition = client.ResponseHeaders["content-disposition"];
var fileName = new ContentDisposition(headerContentDisposition).FileName;
var generatedName = GeneratePath(locationToStoreTo, fileName);
client.DownloadFile(fileUri, generatedName);
return Task.FromResult<string>(generatedName);
}
}
#2:
private async Task<string> DownloadFile(Uri fileUri, string locationToStoreTo)
{
using (var client = new HttpClient())
using (var response = await client.GetAsync(fileUri))
{
response.EnsureSuccessStatusCode();
var fileName = response.Content.Headers.ContentDisposition.FileName;
var generatedName = GeneratePath(locationToStoreTo, fileName);
var stream = await response.Content.ReadAsStreamAsync();
using (var fileStream = File.Create(generatedName))
{
stream.CopyTo(fileStream);
}
return generatedName;
}
}
But in both ways ContentDisposition is null. When I try to find out if the file name is being sent from one of the URLs, I can see that Firefox knows the file name of the file that I'm trying to download.
What am I missing? Is there another way that I can do this?

Android app Json

I writing android app and faced problem.
I have API and write it to file.
Code of writing
string url2 = "http://new.murakami.ua/?mkapi=getProducts";
JsonValue json = await FetchAsync(url2);
string path = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
string filename = System.IO.Path.Combine(path, "myfile.txt");
using (var streamWriter = new StreamWriter(filename, true))
{
streamWriter.Write(json.ToString());
streamWriter.Close();
}
ParseAndDisplay1(json);
ParseAndDisplay2(json);
ParseAndDisplay3(json);
ParseAndDisplay4(json);
ParseAndDisplay5(json);
ParseAndDisplay6(json);
ParseAndDisplay7(json);
ParseAndDisplay8(json);
}
private async Task<JsonValue> FetchAsync(string url)
{
// Create an HTTP web request using the URL:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.ContentType = "application/json";
request.Method = "GET";
// Send the request to the server and wait for the response:
using (WebResponse response = await request.GetResponseAsync())
{
// Get a stream representation of the HTTP web response:
using (Stream stream = response.GetResponseStream())
{
// Use this stream to build a JSON document object:
JsonValue jsonDoc = await Task.Run(() => JsonObject.Load(stream));
//dynamic data = JObject.Parse(jsonDoc[15].ToString);
Console.Out.WriteLine("Response: {0}", jsonDoc.ToString());
// Return the JSON document:
return jsonDoc;
}
}
}
I want to read information from it and display.
I tried something like this, but it don't works.
private void ParseAndDisplay1(JsonValue json)
{
TextView productname = FindViewById<TextView> (Resource.Id.posttittle);
TextView price = FindViewById<TextView> (Resource.Id.price);
TextView weight = FindViewById<TextView> (Resource.Id.weight);
productname.Click += delegate {
var intent404 = new Intent (this, typeof(SoupesDetailActivity1));
StartActivity (intent404);
};
string path = System.Environment.GetFolderPath (System.Environment.SpecialFolder.Personal);
string filename = System.IO.Path.Combine (path, "myfile.txt");
using (var streamReader = new StreamReader (filename, true)) {
JsonValue firstitem = json [81];
productname.Text = firstitem ["post_title"];
price.Text = firstitem ["price"] + " грн";
weight.Text = firstitem ["weight"] + "г";
}
}
Can you help me with this problem?
There are many, many different ways to read/write data in .NET. This is just one fairly simple approach
// assuming data is System.Json.JsonValue with your data
// write data to file
File.WriteAllText(path, data.ToString());
// read back data
string rawdata = File.ReadAllText(path);
// parse it into a JsonValue
JsonValue json = JsonValue.Parse(rawdata);
Man you are using Xamarin. But here some tips of how I solve it in Android native code.
I saved the file in the Assets folder;
them saved the fale with .json extension not .txt;
I accessed it using this code:
private static String loadJSONFromAsset(Context context, String fileName) {
String json = null;
try {
InputStream is = context.getAssets().open("contactos/" + fileName + ".json");
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
json = new String(buffer);
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return json;
}
After that I parse the JSON with no problem.
And it works perfectly, I hope it will help you.

Categories