I am trying to copy a file from amazon s3 to a memory stream so that I can pass it as a file to be downloaded.
I am getting the file from amazon s3, however when I am trying to copy the stream I am getting a null value.
I have checked to see if the stream is getting closed before copying and it is not.
var ms = new MemoryStream();
try
{
GetObjectRequest getObjectRequest = new GetObjectRequest();
getObjectRequest.BucketName = Bucketname;
getObjectRequest.Key = Keyname;
var getObjectResponse = client.GetObjectAsync(getObjectRequest);
getObjectResponse.Wait();
getObjectResponse.Result.ResponseStream.CopyToAsync(ms);
var len = ms.Length;
return File(ms.ToArray(), System.Net.Mime.MediaTypeNames.Application.Pdf, "filename");
}
Also
var len = ms.Length;
is giving a value of 18.
So Why is no content being read.
Please do point me in the right direction.
Your problem is right here:
getObjectResponse.Result.ResponseStream.CopyToAsync(ms);
Who is waiting for that Task to finish? No one, so you return in the middle of the operation.
Overall, the code should be:
try
{
GetObjectRequest getObjectRequest = new GetObjectRequest();
getObjectRequest.BucketName = Bucketname;
getObjectRequest.Key = Keyname;
var getObjectResponse = await client.GetObjectAsync(getObjectRequest);
await getObjectResponse.ResponseStream.CopyToAsync(ms);
var len = ms.Length;
return File(ms.ToArray(), System.Net.Mime.MediaTypeNames.Application.Pdf, "filename");
}
Note the proper use of await for asynchronous operations, which you should be using, specially in a web framework like ASP.NET Core.
Use await for your GetObjectAsync method like below and also for CopyToAsync as #Camilo answer :
try
{
GetObjectRequest getObjectRequest = new GetObjectRequest();
getObjectRequest.BucketName = Bucketname;
getObjectRequest.Key = Keyname;
var getObjectResponse = await client.GetObjectAsync(getObjectRequest);
getObjectResponse.Wait();
await getObjectResponse.Result.ResponseStream.CopyToAsync(ms);
var len = ms.Length;
return File(ms.ToArray(), System.Net.Mime.MediaTypeNames.Application.Pdf, "filename");
}
or you can do something like:
static async Task ReadStream()
{
try
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucketName,
Key = keyName
};
using (GetObjectResponse response = await client.GetObjectAsync(getObjectRequest))
using (Stream responseStream = response.ResponseStream)
using (MemoryStream reader = new MemoryStream(responseStream))
{
//your codes
}
}
catch (AmazonS3Exception e)
{
//Handle it
}
catch (Exception e)
{
//Handle it
}
}
the responseStream is your stream content
and consuming is something like :
ReadStream().Wait();
You need to wait for your async to return. If your method is not marked as async and you don't want to mark it as such you can do the following:
var ms = Task.Run<byte>(async () => await getObjectResponse.Result.ResponseStream.CopyToAsync(ms)).Result;
This will execute the async method and wait for the return. You also won't need to mark your function as async.
Related
I have one memory stream object in my server-side, this object should be accessible in another party, which I call it a client or a consumer for my API.
In server-side I have a method like this (parameters.Save is related to a third-party library)
public MemoryStream GetSerializedParameters()
{
var parameters = GetParameters();
MemoryStream memory = new MemoryStream();
parameters.Save(memory);
return memory;
}
I'm thinking about sending this memory stream to a client with web API, so my action is something like this:
[HttpGet("parameters")]
public HttpResponseMessage GetParameters()
{
var stream = _server.GetSerializedParameters();
HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentLength = stream.Length;
return result;
}
I'm not sure if it is the right way and this implementation is correct because I am in trouble to consume it:
I do not know which method of httpClient I have to use: ReadAsStreamAsync() or anything else, because I could not find anything to work
Sure you can:
[HttpGet]
public async Task Get()
{
var randomString = "thisIsCool";
var randomStringBytes = Encoding.UTF8.GetBytes(randomString);
using (var ms = new MemoryStream(randomStringBytes))
{
await ms.CopyToAsync(this.Response.Body);
}
Based on my under standing below code may help you:
WEB API:
[HttpGet]
public HttpResponseMessage ReadToStream(HttpRequestMessage requestMessage)
{
var streamObj = _server.GetSerializedParameters();
HttpResponseMessage response = new HttpResponseMessage();
response.Content = new StreamContent(streamObj);
requestMessage.RegisterForDispose(streamObj);
response.StatusCode = HttpStatusCode.OK;
return response;
}
Client Side
public async Task<string> DownloadFile(string guid)
{
var fileInfo = new FileInfo($"{guid}.txt");
var response = await _httpClient.GetAsync($"{url}/api/fileDownloadAPI?guid={guid}");
response.EnsureSuccessStatusCode();
await using var ms = await response.Content.ReadAsStreamAsync();
await using var fs = File.Create(fileInfo.FullName);
ms.Seek(0, SeekOrigin.Begin);
ms.CopyTo(fs);
return fileInfo.FullName;
}
I found the solution like this:
here is in server side:
[HttpGet("parameters")]
public IActionResult GetParameters()
{
var stream = _server.GetSerializedParameters();
stream.Seek(0, SeekOrigin.Begin);
return File(stream, MediaTypeNames.Text.Plain, "parameters.txt");
}
and here is in client-side:
public MemoryStream StoreParameters()
{
var request =new HttpRequestMessage
{
RequestUri = new Uri("https://localhost:44316/api/parameters"),
Method = HttpMethod.Get
};
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var result = _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;
var ms = new MemoryStream();
result.Content.CopyToAsync(ms).Wait();
return result.IsSuccessStatusCode ? ms: null;
}
My ASP.NET Core app uses "out-of-box" external login authentication. What I want to implement - on facebook challenge I want to wrap redirect url and return it as json to consume in jquery frontend. But after request ends I see 500 error in browser and next error in application console:
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HLV651D6KVJC", Request id "0HLV651D6KVJC:00000005": An unhandled exception was thrown by the
application. System.InvalidOperationException: Response Content-Length
mismatch: too few bytes written (0 of 470).
My external login action, nothing special to look at
[HttpPost]
[AllowAnonymous]
[ValidateAntiForgeryToken]
public IActionResult ExternalLogin(string provider, string returnUrl = null)
{
// Request a redirect to the external login provider.
var redirectUrl = Url.Action(nameof(ExternalLoginCallback), "Account", new { returnUrl });
var properties = _signInManager.ConfigureExternalAuthenticationProperties(provider, redirectUrl);
return Challenge(properties, provider);
}
Facebook authentication configuration:
services.AddAuthentication().AddFacebook(facebookOptions =>
{
facebookOptions.AppId = Configuration["Authentication:Facebook:AppId"];
facebookOptions.AppSecret = Configuration["Authentication:Facebook:AppSecret"];
facebookOptions.Events.OnRedirectToAuthorizationEndpoint =
async (x) =>
{
UTF8Encoding encoding = new UTF8Encoding();
var content = JsonConvert.SerializeObject(new { redirect_url = x.RedirectUri });
byte[] bytes = encoding.GetBytes(content);
x.Response.StatusCode = 200;
x.Response.ContentLength = bytes.Length;
x.Response.ContentType = "text/plain";
x.Response.Body = new MemoryStream();
await x.Response.WriteAsync(content);
// at this point I see that x.Response.Body.Length == 470, but message states there are 0 of 470 written
};
});
Is there any way I could make it work?
This can also happen when using new C# using syntax like this:
using var ms = new MemoryStream();
using var writer = new StreamWriter(ms);
writer.WriteLine("my content");
memoryStream.Position = 0;
return File(ms, "text/plain");
in this case, the MemoryStream is accessed before the StreamWriter is flushed. Either use old syntax for the StreamWriter:
using var ms = new MemoryStream();
using (var writer = new StreamWriter(ms, Encoding.UTF8, -1, true))
{
writer.WriteLine("my content");
}
memoryStream.Position = 0;
return File(ms, "text/plain");
or flush the writer:
using var ms = new MemoryStream();
using var writer = new StreamWriter(ms);
writer.WriteLine("my content");
writer.Flush();
memoryStream.Position = 0;
return File(ms, "text/plain");
Changed code to write to original response stream and it works now.
facebookOptions.Events.OnRedirectToAuthorizationEndpoint =
async (x) =>
{
var content = JsonConvert.SerializeObject(new { redirect_url = x.RedirectUri });
x.Response.StatusCode = 200;
x.Response.ContentType = "text/plain";
await x.Response.WriteAsync(content);
};
You can use something like this:
var stream = new MemoryStream();
/// writing to the stream
if (stream.CanSeek)
{
stream.Seek(0, SeekOrigin.Begin);
}
/// then read stream
I'm currently using Pushbullet API and need to upload a file.
I can successfully get an upload url as specified in the docs using this method:
public static async Task<Uploads> GetUploadUrl(string file_name, string file_type)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Access-Token", AccessToken);
var json = new JObject
{
["file_name"] = file_name,
["file_type"] = file_type
};
var result = await client.PostAsync(new Uri(_uploadUrl, UriKind.RelativeOrAbsolute), new HttpStringContent(json.ToString(), UnicodeEncoding.Utf8, "application/json"));
if (result.IsSuccessStatusCode)
{
var textresult = await result.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<Uploads>(textresult);
}
}
return null;
}
The problem is when I try to upload the file. I'm currently using this method:
public static async Task<bool> UploadFile(StorageFile file, string upload_url)
{
try
{
System.Net.Http.HttpClient client = new System.Net.Http.HttpClient();
var content = new MultipartFormDataContent();
if (file != null)
{
var streamData = await file.OpenReadAsync();
var bytes = new byte[streamData.Size];
using (var dataReader = new DataReader(streamData))
{
await dataReader.LoadAsync((uint)streamData.Size);
dataReader.ReadBytes(bytes);
}
var streamContent = new ByteArrayContent(bytes);
content.Add(streamContent);
}
client.DefaultRequestHeaders.Add("Access-Token", AccessToken);
var response = await client.PostAsync(new Uri(upload_url, UriKind.Absolute), content);
if (response.IsSuccessStatusCode)
return true;
}
catch { return false; }
return false;
}
but I get a Http 400 error. What's the right way to upload a file using multipart/form-data in a UWP app?
HTTP 400 error indicates Bad Request, it means the request could not be understood by the server due to malformed syntax. In the other word, the request sent by the client doesn't follow server's rules.
Let's look at the document, and we can find in the example request it uses following parameter:
-F file=#cat.jpg
So in the request, we need to set the name for the uploaded file and the name should be "file". Besides, in this request, there is no need to use access token. So you can change your code like following:
public static async Task<bool> UploadFile(StorageFile file, string upload_url)
{
try
{
System.Net.Http.HttpClient client = new System.Net.Http.HttpClient();
var content = new MultipartFormDataContent();
if (file != null)
{
var streamData = await file.OpenReadAsync();
var bytes = new byte[streamData.Size];
using (var dataReader = new DataReader(streamData))
{
await dataReader.LoadAsync((uint)streamData.Size);
dataReader.ReadBytes(bytes);
}
var streamContent = new ByteArrayContent(bytes);
content.Add(streamContent, "file");
}
//client.DefaultRequestHeaders.Add("Access-Token", AccessToken);
var response = await client.PostAsync(new Uri(upload_url, UriKind.Absolute), content);
if (response.IsSuccessStatusCode)
return true;
}
catch { return false; }
return false;
}
Then your code should be able to work. You will get a 204 No Content response and UploadFile method will return true.
I am putting together a test application using WebApi2 and HttpClient in a win forms app.
I have come accross an issue where my HttpClient request to a WebApi2 controller which returns an HttpResponseMessage doesnt return the ByteArrayContent.
WebApiController Code
[HttpGet]
public HttpResponseMessage DownloadFilePart(string fileName)
{
var path = Server.MapPath("~/App_Data/uploads/" + fileName);
var fileArray = System.IO.File.ReadAllBytes(path);
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new ByteArrayContent(fileArray)
};
response.Content.Headers.ContentType = new MediaTypeHeaderValue(System.Web.MimeMapping.GetMimeMapping(fileName));
response.Content.Headers.ContentLength = fileArray.Length;
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = fileName
};
return response;
}
WinForms Code using HttpClient
static async void GetFilePart(string hostrUri)
{
var httpClient = new HttpClient
{
BaseAddress = new Uri(hostrUri)
};
var request = new HttpRequestMessage(HttpMethod.Get, "/Home/DownloadFilePart/?fileName=Test.txt");
var responseMessage = httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var memoryStream = new MemoryStream();
var stream = await responseMessage.Result.Content.ReadAsByteArrayAsync();
var fileToWriteTo = System.IO.Path.GetDirectoryName(Application.ExecutablePath) + "\\Temp\\Test.txt";
using (var fileStream = new FileStream(fileToWriteTo, FileMode.Create, FileAccess.Write, FileShare.None))
{
//copy the content from response to filestream
fileStream.Write(stream, 0, stream.Length);
}
}
When the request return from the WebApi and I write the bytes to file all that is written into the file is the actual headers from the WebApi response. Has anyone any ideas what the issue could be here?
Thanks
Your problem is here
httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
HttpCompletionOption.ResponseHeadersRead is summarized as
The operation should complete as soon as a response is available and headers are read. The content is not read yet.
This would explain why you only get the headers in your response.
Either remove it completely or change it to HttpCompletionOption.ResponseContentRead
static async void GetFilePart(string hostrUri)
{
var httpClient = new HttpClient
{
BaseAddress = new Uri(hostrUri)
};
var request = new HttpRequestMessage(HttpMethod.Get, "/Home/DownloadFilePart/?fileName=Test.txt");
var responseMessage = await httpClient.SendAsync(request);
var byteArray = await responseMessage.Content.ReadAsByteArrayAsync();
var fileToWriteTo = System.IO.Path.GetDirectoryName(Application.ExecutablePath) + "\\Temp\\Test.txt";
using (var fileStream = new FileStream(fileToWriteTo, FileMode.Create, FileAccess.Write, FileShare.None))
{
//copy the content from response to filestream
fileStream.Write(byteArray, 0, byteArray.Length);
}
}
i want to use catchoom in c# , but not able to find any sample could any one please provide any sample code if have.
i have got this sample of curl
curl -F "image=#CATask1-Correct.png" -F "token=sometoken" https://r.catchoom.com/v1/search
Can some one covert this to c# ?
I tried to convert it as shown below:
Here's my code:
public static async Task<string> Upload(byte[] image)
{
using (var client = new HttpClient())
{
// string boundary = "---XXX---";
using (var content = new MultipartFormDataContent())
{
string token = "sometoken";
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(token);
writer.Flush();
stream.Position = 0;
// adding Token and Image to the request
content.Add(new StreamContent(stream), "token");
content.Add(new StreamContent(new MemoryStream(image)), "bilddatei", "upload.jpg");
using (
var message = await client.PostAsync("https://r.catchoom.com/v1/search", content))
{
var input = await message.Content.ReadAsStringAsync();
return input;
}
}
}
}
then I called the method in the on-click event handler:
private async void mybtn_Click(object sender, RoutedEventArgs e)
{
// converting the image to a byte array
BitmapImage bitmapImage = new BitmapImage(new Uri("http://www.familyfuntwincities.com/wp-content/uploads/2013/09/apple_red_1_clipart.png?s=128&g=1"));
RandomAccessStreamReference rasr = RandomAccessStreamReference.CreateFromUri(bitmapImage.UriSource);
var streamWithContent = await rasr.OpenReadAsync();
byte[] buffer = new byte[streamWithContent.Size];
await streamWithContent.ReadAsync(buffer.AsBuffer(), (uint)streamWithContent.Size, InputStreamOptions.None);
// calling the upload method
string output = await Upload(buffer);
mytext1.Text = output;
}
but I keep getting "image is missing" error from the catchoom server although I managed to upload the image to other servers in the same way ( without the token part of course).
My question is: How to add multipule part content? what's the right boundary between the Token part and the image part in order to be recognized by catchoom?
Here is what worked for me:
public static async Task<string> Test(string token, Stream stream, string fileName)
{
HttpClient client = new HttpClient();
MultipartFormDataContent content = new MultipartFormDataContent();
content.Add(new StringContent(token), "token");
content.Add(new StreamContent(stream), "image", fileName);
var resp = await client.PostAsync("https://r.catchoom.com/v1/search", content);
return await resp.Content.ReadAsStringAsync();
}
If you wanted to use the image you listed in your sample, here is how I would fetch it:
HttpClient client = new HttpClient();
var stream = await client.GetStreamAsync("http://www.familyfuntwincities.com/wp-content/uploads/2013/09/apple_red_1_clipart.png?s=128&g=1");
return await Test("<SOME_TOKEN_VALUE>", stream, "apple.png");
However the API claims that the image is unsupported due to alpha transparency. You can find some other samples that show you how to strip alpha transparency if that is important. The above code worked fine for images that don't have alpha transparency.