.NET multipart data appeared in preamble section - c#

client side is using MultipartFormDataContent with two part of data, one is a file, the other one is some metadata.
In each 3 request, there will be 2 request failed due to FileData empty.
Client:
var client = new HttpClient(new WebRequestHandler());
using (var content = new MultipartFormDataContent())
{
var guid = Guid.NewGuid().ToString();
var tmpFileName = string.Format("{0}{1}", guid, Path.GetExtension(fileName));
var dataContent = new ByteArrayContent(data);
content.Add(dataContent, guid, tmpFileName);
var optionContent = new ByteArrayContent(optionData);
optionContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("parameter") { Name = "optionsStr" };
content.Add(optionContent);
var response = client.PostAsync("http://test.com", content).Result;
}
Server:
[HttpPost]
public async Task<HttpResponseMessage> UploadDocument(string dataStr)
{
string rootPath = System.Web.HttpContext.Current.Server.MapPath("~/App_Data/Documents");
MultipartFormDataStreamProvider provider = new MultipartFormDataStreamProvider(rootPath);
await Request.Content.ReadAsMultipartAsync(provider);
if (provider.FileData == null || provider.FileData.Count == 0)
throw new Exception("There is no file in the current request of httpcontext.");
}
After sniffing using Wiresharks, found out the actual file data was misplaced in the preamble section of the multi-part structure:
error request:
wireshark snapshot1
correct request:
wireshark snapshot2
any clue what may cause such behavior?

Related

How can i save a REST Response to a PDF file? [duplicate]

I have a URL (URL for the live feed from client) which when I hit in browser returns the xml response . I have saved this in text file it`s size is 8 MB.
now my problem is that I need to save this response in xml file on server`s drive. from there I will insert this in database. and request needs to be made using code using http-client or rest-sharp library of c# .net 4.5
I am unsure what should I do for above case. can any body suggest me something
With RestSharp, it's right there in the readme:
var client = new RestClient("http://example.com");
client.DownloadData(request).SaveAs(path);
With HttpClient, it's a bit more involved. Have a look at this blog post.
Another option is Flurl.Http (disclaimer: I'm the author). It uses HttpClient under the hood and provides a fluent interface and lots of convenient helper methods, including:
await "http://example.com".DownloadFileAsync(folderPath, "foo.xml");
Get it on NuGet.
It seems SaveAs was discontinued. You can try this
var client = new RestClient("http://example.com")
byte[] response = client.DownloadData(request);
File.WriteAllBytes(SAVE_PATH, response);
In case you want async version
var request = new RestRequest("/resource/5", Method.GET);
var client = new RestClient("http://example.com");
var response = await client.ExecuteTaskAsync(request);
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception($"Unable to download file");
response.RawBytes.SaveAs(path);
Don't keep the file in memory while reading. Write it directly to the disk.
var tempFile = Path.GetTempFileName();
using var writer = File.OpenWrite(tempFile);
var client = new RestClient(baseUrl);
var request = new RestRequest("Assets/LargeFile.7z");
request.ResponseWriter = responseStream =>
{
using (responseStream)
{
responseStream.CopyTo(writer);
}
};
var response = client.DownloadData(request);
Copied from here https://stackoverflow.com/a/59720610/179017.
Add following NuGet package into the current system
dotnet add package RestSharp
Using Bearer Authentication
// Download file from 3rd party API
[HttpGet("[action]")]
public async Task<IActionResult> Download([FromQuery] string fileUri)
{
// Using rest sharp
RestClient client = new RestClient(fileUri);
client.ClearHandlers();
client.AddHandler("*", () => { return new JsonDeserializer(); });
RestRequest request = new RestRequest(Method.GET);
request.AddParameter("Authorization", string.Format("Bearer " + accessToken),
ParameterType.HttpHeader);
IRestResponse response = await client.ExecuteTaskAsync(request);
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
// Read bytes
byte[] fileBytes = response.RawBytes;
var headervalue = response.Headers.FirstOrDefault(x => x.Name == "Content-Disposition")?.Value;
string contentDispositionString = Convert.ToString(headervalue);
ContentDisposition contentDisposition = new ContentDisposition(contentDispositionString);
string fileName = contentDisposition.FileName;
// you can write a own logic for download file on SFTP,Local local system location
//
// If you to return file object then you can use below code
return File(fileBytes, "application/octet-stream", fileName);
}
}
Using Basic Authentication
// Download file from 3rd party API
[HttpGet("[action]")]
public async Task<IActionResult> Download([FromQuery] string fileUri)
{
RestClient client = new RestClient(fileUri)
{
Authenticator = new HttpBasicAuthenticator("your user name", "your password")
};
client.ClearHandlers();
client.AddHandler("*", () => { return new JsonDeserializer(); });
RestRequest request = new RestRequest(Method.GET);
IRestResponse response = await client.ExecuteTaskAsync(request);
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
// Read bytes
byte[] fileBytes = response.RawBytes;
var headervalue = response.Headers.FirstOrDefault(x => x.Name == "Content-Disposition")?.Value;
string contentDispositionString = Convert.ToString(headervalue);
ContentDisposition contentDisposition = new ContentDisposition(contentDispositionString);
string fileName = contentDisposition.FileName;
// you can write a own logic for download file on SFTP,Local local system location
//
// If you to return file object then you can use below code
return File(fileBytes, "application/octet-stream", fileName);
}
}

How to post a Multipart/mixed:boundary=”MIME-Boundary” with two different document types and HMAC signature in C# [Open Invoice]

As part of an integration to Open Invoice the API is requiring a legacy Multipart/mixed:boundary=”MIME-Boundary” request header. There is very minimal documentation on utilizing this Header for C# across the internet including Microsoft documentation.
The two documents in the POST request include a UTF-8 XML string and a Base64 pdf string. The outbound request requires Content-Type headers (one for each document) which HTTPRequestMessage does not support natively as it assumes you will be delivering a "text/plain" request.
Headers.Add() //throws an exception when setting Content-Type
Headers.TryAddWithoutValidation() //does not fix the problem
Additionally the request needs to be signed with an HMAC hash of the request body.
How do I build this request in C#?
Build the client
X509Certificate2 cert = new X509Certificate2(certificateName, certPassword);
var handler = new WebRequestHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(cert);
HttpClient client = new HttpClient(handler);
client.BaseAddress = new Uri(openInvoiceURL);
Build The Request
//Starting from byte[] for both files
var xmlString = System.Text.Encoding.UTF8.GetString(xmlBytes);
string pdf64 = Convert.ToBase64String(pdfBytes);
string url = "/docp/supply-chain/v1/invoices";
var multiPartContent = new MultipartContent("mixed", "_MIME-Boundary");
var xmlHttpContent = new StringContent(xmlString);
xmlHttpContent.Headers.ContentType = new MediaTypeHeaderValue("application/xml");
multiPartContent.Add(xmlHttpContent);
var pdfHttpContent = new StringContent(pdf64);
pdfHttpContent.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
pdfHttpContent.Headers.Add("content-transfer-encoding", "base64");
pdfHttpContent.Headers.Add("Content-ID", pdfName);
multiPartContent.Add(pdfHttpContent);
var request = new HttpRequestMessage(HttpMethod.Post, url);
request.Content = multiPartContent;
request = AttachHMAC(request, secretKey);
var response = client.SendAsync(request).Result;
var responseString = response.Content.ReadAsStringAsync().Result;
Sign the Request Body (only on POST)
private HttpRequestMessage AttachHMAC(HttpRequestMessage request, string secretKey)
{
var payload = string.Empty;
if(request != null && request.Content != null)
{
payload = request.Content.ReadAsStringAsync().Result;
}
var hashMaker = new HMACSHA256(Encoding.UTF8.GetBytes(secretKey));
var payloadByteArray = Encoding.UTF8.GetBytes(payload);
byte[] hash = hashMaker.ComputeHash(payloadByteArray);
var base64Hash = Convert.ToBase64String(hash);
request.Headers.Add("mac", base64Hash);
return request;
}

How to upload a document via API to watson discovery services. ASP.net

I'm using the following API to upload a file to watson discovery services. I'm getting the response as unsupported file format, even though i have provided a valid type.
API:discovery service api
public async Task<ActionResult> Index()
{
using (var httpClient = new HttpClient())
{
//ADD BASIC AUTH
var authByteArray = Encoding.ASCII.GetBytes("{auth key}");
var authString = Convert.ToBase64String(authByteArray);
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", authString);
var text = string.Empty;
var uri = "https://gateway.watsonplatform.net/discovery/api/v1/environments/{envid}/collections/{collectionid}/documents?version=2017-11-07";
var content = new MultipartFormDataContent();
var bytes = System.IO.File.ReadAllBytes(Server.MapPath("~/Views/UploadDocument/civilwar-api1.html"));
var file = new StreamContent(new MemoryStream(bytes));
content.Headers.ContentType = MediaTypeHeaderValue.Parse("text/html");
content.Add(new StreamContent(new MemoryStream(bytes)), "file");
var response = await httpClient.PostAsync(uri, content);
var text1 = await response.Content.ReadAsStringAsync();
}
return View();
}
api response as :{
"code" : 415,
"error" : "Unsupported Media Type"
}
As you can see in Watson Developer Cloud, you can use the .NET SDK.
In this repository, you can see examples for using each service from Watson.
The method to add a document:
#region Documents
private void AddDocument()
{
Console.WriteLine(string.Format("\nCalling AddDocument()..."));
using (FileStream fs = File.OpenRead(_filepathToIngest))
{
var result = _discovery.AddDocument(_createdEnvironmentId, _createdCollectionId, _createdConfigurationId, fs as Stream, _metadata);
if (result != null)
{
Console.WriteLine(JsonConvert.SerializeObject(result, Formatting.Indented));
_createdDocumentId = result.DocumentId;
}
else
{
Console.WriteLine("result is null.");
}
}
}
See more about Watson Discovery.
try httpClient.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/json"));//ACCEPT header

how to call web api to download document to website directory using webclient

I am struggling with being able to create a file with its data based on the byte array returned from the WebAPI. The following is my code for making the call to the web api
using (var http = new WebClient())
{
string url = string.Format("{0}api/FileUpload/FileServe?FileID=" + fileID, webApiUrl);
http.Headers[HttpRequestHeader.ContentType] = "application/octet-stream";
http.Headers[HttpRequestHeader.Authorization] = "Bearer " + authCookie.Value;
http.DownloadDataCompleted += Http_DownloadDataCompleted;
byte[] json = await http.DownloadDataTaskAsync(url);
}
The api code is
[HttpGet]
[Route("FileServe")]
[Authorize(Roles = "Admin,SuperAdmin,Contractor")]
public async Task<HttpResponseMessage> GetFile(int FileID)
{
using (var repo = new MBHDocRepository())
{
var file = await repo.GetSpecificFile(FileID);
if (file == null)
{
throw new HttpResponseException(HttpStatusCode.BadRequest);
}
var stream = File.Open(file.PathLocator, FileMode.Open);
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(file.FileType);
return response;
}
}
I receive a byte array as a response however am unable to create the corresponding file from that byte array. I have no idea how to convert the byte array into the relevant file type (such as jpg, or pdf based on file type in the web api). any help will be appreciated.
Alright so there are a few ways of solving your problem firstly, on the server side of things you can either simply send the content type and leave it at that or you can also send the complete filename which helps you even further.
I have removed the code that is specific to your stuff with basic test code, please just ignore that stuff and use it in terms of your code.
Some design notes here:
[HttpGet]
[Route("FileServe")]
[Authorize(Roles = "Admin,SuperAdmin,Contractor")]
public async Task<HttpResponseMessage> GetFileAsync(int FileID) //<-- If your method returns Task have it be named with Async in it
{
using (var repo = new MBHDocRepository())
{
var file = await repo.GetSpecificFile(FileID);
if (file == null)
{
throw new HttpResponseException(HttpStatusCode.BadRequest);
}
var stream = File.Open(file.PathLocator, FileMode.Open);
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(file.FileType);
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment") { FileName=Path.GetFileName(file.PathLocator)};
return response;
}
}
Your client side code has two options here:
static void Main(string[] args)
{
using (var http = new WebClient())
{
string url = string.Format("{0}api/FileUpload/FileServe?FileID={1}",webApiUrl, fileId);
http.Headers[HttpRequestHeader.ContentType] = "application/octet-stream";
http.Headers[HttpRequestHeader.Authorization] = "Bearer " + authCookie.Value;
var response = http.OpenRead(url);
var fs = new FileStream(String.Format(#"C:\Users\Bailey Miller\Downloads\{0}", GetName(http.ResponseHeaders)), FileMode.Create);
response.CopyTo(fs); <-- how to move the stream to the actual file, this is not perfect and there are a lot of better examples
fs.Flush();
fs.Close();
}
}
private static object GetName(WebHeaderCollection responseHeaders)
{
var c_type = responseHeaders.GetValues("Content-Type"); //<-- do a switch on this and return a really weird file name with the correct extension for the mime type.
var cd = responseHeaders.GetValues("Content-Disposition")[0].Replace("\"", ""); <-- this gets the attachment type and filename param, also removes illegal character " from filename if present
return cd.Substring(cd.IndexOf("=")+1); <-- extracts the file name
}

How to process multipart/mixed in C#

Fiddler shows that server processed my request successfully. I get back a boundary-separated list of HTTP responses. But processing multipart/mixed response is new to me.
Based upon research, I tried the following:
httpResp = (HttpWebResponse)httpRequest.GetResponse() as HttpWebResponse;
var content = new StreamContent(httpResp.GetResponseStream());
var streamProvider = new MultipartMemoryStreamProvider();
var task = content.ReadAsMultipartAsync(streamProvider).ContinueWith(t =>
{
foreach (HttpContent item in streamProvider.Contents) {
log.Debug("in foreach");
partResStr = item.ReadAsStringAsync().Result;
log.DebugFormat("partResStr = {0}", partResStr);
}
});
But the logging on the foreach doesn't occur.
When I did this, I had to set the ContentType of the StreamContent:
var streamContent = new StreamContent(stream);
streamContent.Headers.ContentType = MediaTypeHeaderValue.Parse(HttpContext.Current.Request.ContentType);
var provider = streamContent.ReadAsMultipartAsync().Result;

Categories