I have an CSV file in memory that I want to upload to a Web API.
If I save the CSV file to disk and upload it, it gets accepted.
However, I want to avoid the extra work and also make the code cleaner by simply uploading the text I have as a MemoryStream Object (I think that's the correct format?).
The following code works for uploading the file:
string webServiceUrl = "XXX";
string filePath = #"C:\test.csv";
string cred = "YYY";
using (var client = new WebClient()){
client.Headers.Add("Authorization", "Basic " + cred);
byte[] rawResponse = client.UploadFile(webServiceUrl, "POST", filePath);
Console.WriteLine(System.Text.Encoding.ASCII.GetString(rawResponse));
}
How would I do if I had a string with all the contents and I want to upload it in the same way without having to save it down to a file?
WebClient.UploadData or WebClient.UploadString perhaps?
Thank you
EDIT:
I tried what you said but by using a local file (in case there was something wrong with the string), but I get the same error.
Here is what I suppose the code would be using your solution
string webServiceUrl = "XXX";
string file = #"C:\test.csv";
string cred = "YYY";
FileStream fs = new FileStream(file, FileMode.Open, FileAccess.Read);
BinaryReader r = new BinaryReader(fs);
byte[] postArray = r.ReadBytes((int)fs.Length);
using (var client = new WebClient())
{
client.Headers.Add("Authorization", "Basic " + cred);
using (var postStream = client.OpenWrite(webServiceUrl, "POST"))
{
postStream.Write(postArray, 0, postArray.Length);
}
}
Any thoughts?
Use OpenWrite() from the WebClient.
using (var postStream = client.OpenWrite(endpointUrl))
{
postStream.Write(memStreamContent, 0, memStream.Length);
}
As documentation mentioned:
The OpenWrite method returns a writable stream that is used to send data to a resource.
Update
Try to set the position of the MemoryStream to 0 before uploading.
memoryStream.Position = 0;
When you copy the file into the MemoryStream, the pointer is moved to the end of the stream, so when you then try to read it, you're getting a null byte instead of your stream data.
MSDN - CopyTo()
Copying begins at the current position in the current stream, and does not reset the position of the destination stream after the copy operation is complete.
I finally managed to solve it.
First I made a request using CURL that worked.
I analyzed the packet data and made an except copy of the packet.
I did a lot of changes, however, the final change was that using the different functions I found online it never closed the packet with a "Last-Boundary" while CURL did.
So by modifying the function, making sure it properly wrote a Last-Boundary it finally worked.
Also, another crucial thing was to set PreAuthenticate to true, the examples online didn't do that.
So, all in all:
1. Make sure that the packet is properly constructed.
2. Make sure you pre authenticate if you need to authenticate.
webrequest.PreAuthenticate = true;
webrequest.Headers[HttpRequestHeader.Authorization] = string.Format("Basic {0}", cred);
Don't forget to add SSL if using a https (which you probably do if you authenticate):
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Ssl3;
Hope this helps someone.
And thanks for the help earlier!
Related
I have a small wav sound file in which I want to get the text of it, so I used Azure speech to text API to test it.
first thing I convert the audio file as they recommended in their documentation to PCM - Mono -16K sample rate.
and I use this code in c# in the documentation example here to upload the file and get the result.
HttpWebRequest request = null;
request = (HttpWebRequest)HttpWebRequest.Create("https://speech.platform.bing.com/speech/recognition/interactive/cognitiveservices/v1?language=en-US&format=detailed");
request.SendChunked = true;
request.Accept = #"application/json;text/xml";
request.Method = "POST";
request.ProtocolVersion = HttpVersion.Version11;
request.ContentType = #"audio/wav; codec=audio/pcm; samplerate=16000";
request.Headers["Ocp-Apim-Subscription-Key"] = "my key";
// Send an audio file by 1024 byte chunks
using (FileStream fs = new FileStream("D:/b.wav", FileMode.Open, FileAccess.Read))
{
/*
* Open a request stream and write 1024 byte chunks in the stream one at a time.
*/
byte[] buffer = null;
int bytesRead = 0;
using (Stream requestStream = request.GetRequestStream())
{
/*
* Read 1024 raw bytes from the input audio file.
*/
buffer = new Byte[checked((uint)Math.Min(1024, (int)fs.Length))];
while ((bytesRead = fs.Read(buffer, 0, buffer.Length)) != 0)
{
requestStream.Write(buffer, 0, bytesRead);
}
// Flush
requestStream.Flush();
}
}
string responseString;
Console.WriteLine("Response:");
using (WebResponse response = request.GetResponse())
{
Console.WriteLine(((HttpWebResponse)response).StatusCode);
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
responseString = sr.ReadToEnd();
}
Console.WriteLine(responseString);
Console.ReadLine();
}
also i tried using cUrl tool and also write it in java as i was thought that maybe it's problem with the programming language I use that i not upload the file correctly.
this the link of the sound file i want to convert it to text here.
so Now i need to help to figure it out if the problem comes from the format of the sound file? or from maybe code that i not upload it correctly? or it's from the API I mean to be not accurate enough?
i tried IBM speech to text and it got all the text with no problem.
iam using now the free trial of Azure speech to text API and I want to figure where the problem comes if anyone has experience with this to see if I will work with this API or not.
Update
I want to clear that iam not got any error i just got incomplete result to my sound file I upload, for example the sound file i upload he said at the end of the sound "What is up with that", the result i got from Azure is just the first sentence only which is "I say that like it's a bad thing.", also I upload another sound file which contains the "What is up with that" only check it here,and it just gives me an empty result like this.
{"RecognitionStatus":"NoMatch","Offset":17300000,"Duration":0}
so all that i want to know if this normal from the Speech to text API Azure or the problem with my code or from the sound file? this what i want to get an answer with it.
when i test another API on those files it worked like IBM for example.
Thanks in advance.
I am trying to re-upload a stream that I just retrieved. It shouldn't really matter that I am using AWS I believe... Maybe my understanding of working with streams is just too limited? :-)
I am using the following method straight from the AWS documentation to download and upload streams:
File upload:
public bool UploadFile(string keyName, Stream stream)
{
using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast1))
{
try
{
TransferUtility fileTransferUtility = new TransferUtility(new AmazonS3Client(Amazon.RegionEndpoint.USEast1));
fileTransferUtility.Upload(stream, bucketName, keyName);
return true;
}
catch (AmazonS3Exception amazonS3Exception)
{
[...]
}
}
}
Getting the file:
public Stream GetFile(string keyName)
{
using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
{
try
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucketName,
Key = keyName
};
GetObjectResponse response = client.GetObject(request);
responseStream = response.ResponseStream;
return responseStream;
}
catch (AmazonS3Exception amazonS3Exception)
{
[...]
}
}
}
Now, I am trying to combine the two methods: I am getting a stream and immediately want to upload it again. However, I am getting the following error message: System.NotSupportedException : HashStream does not support seeking
I am guessing it has something to do that I am getting a stream which is somehow now immediately ready to be uploaded again?
This is how I am trying to get the stream of an existing file (.jpg) and immediately try to upload it with a different filename:
newAWS.UploadFile(newFileName, oldAWS.GetFile(oldFile));
Where newAWS and oldAWS are instances of the AWS class, and newFileName is a string :-)
This is the line I am getting the error message pasted above.
Please let me know if I am missing something obvious here why I would not be able to re-upload a fetched stream. Or could it be that it is related to something else I am not aware of and I am on the wrong track trying to troubleshoot the returned stream?
What I am basically trying to do is to copy one file from an AWS bucket to another using streams. But for some reason I get the error message trying to upload the stream that I just downloaded.
Thank you very much for taking your time digging through my code :-)
As the error indicates, the Hashstream object that Amazon returns does not support seeking.
Something in your code, or a method you're calling, is trying to do seeking on that stream. Most likely it's trying to seek to the beginning of the stream.
So you need to convert Hashstream to a different stream that supports seeking:
using (GetObjectResponse response = client.GetObject(request))
using (Stream responseStream = response.ResponseStream)
using (MemoryStream memStream = new MemoryStream())
{
responseStream.CopyTo(memStream);
memStream.Seek(0, SeekOrigin.Begin);
// now use memStream wherever you were previously using responseStream
}
It looks like that my approach won't work and I am not really sure why it doesn't.
The good news is, that you can simply use the following code to copy from one AWS bucket to another:
static IAmazonS3 client;
client = new AmazonS3Client(Amazon.RegionEndpoint.USEast1);
CopyObjectRequest request = new CopyObjectRequest()
{
SourceBucket = bucketName,
SourceKey = objectKey,
DestinationBucket = bucketName,
DestinationKey = destObjectKey
};
CopyObjectResponse response = client.CopyObject(request);
So I didn't have to try to recreate this functionality using the upload and download methods I wrote but simply use the copy method provided.
HOWEVER, I still would like to know why my approach was failing? I really think I am missing something regarding the streams. Any feedback would be greatly appreciated :-)
I will leave this question as unanswered since I would really like to know what I am missing regarding streams. I would be happy to mark an answer as accepted that explains my missing knowledge :-)
I'm trying to download files from my FTP server - multiples at the same time. When i use the DownloadFileAsync .. random files are returned with a byte[] Length of 0. I can 100% confirm the file exists on the server and has content AND there FTP server (running Filezilla Server) isn't erroring and say's the file has been transferred.
private async Task<IList<FtpDataResult>> DownloadFileAsync(FtpFileName ftpFileName)
{
var address = new Uri(string.Format("ftp://{0}{1}", _server, ftpFileName.FullName));
var webClient = new WebClient
{
Credentials = new NetworkCredential(_username, _password)
};
var bytes = await webClient.DownloadDataTaskAsync(address);
using (var stream = new MemoryStream(bytes))
{
// extract the stream data (either files in a zip OR a file);
return result;
}
}
When I try this code, it's slower (of course) but all the files have content.
private async Task<IList<FtpDataResult>> DownloadFileAsync(FtpFileName ftpFileName)
{
var address = new Uri(string.Format("ftp://{0}{1}", _server, ftpFileName.FullName));
var webClient = new WebClient
{
Credentials = new NetworkCredential(_username, _password)
};
// NOTICE: I've removed the AWAIT and a different method.
var bytes = webClient.DownloadData(address);
using (var stream = new MemoryStream(bytes))
{
// extract the stream data (either files in a zip OR a file);
return result;
}
}
Can anyone see what I'm doing wrong, please? Why would the DownloadFileAsync be randomly returning zero bytes?
Try out FtpWebRequest/FtpWebResponse classes. You have more available to you for debugging purposes.
FtpWebRequest - http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest(v=vs.110).aspx
FtpWebResponse - http://msdn.microsoft.com/en-us/library/system.net.ftpwebresponse(v=vs.110).aspx
Take a look at http://netftp.codeplex.com/. It appears as though almost all methods implement IAsyncResult. There isn't much documentation on how to get started, but I would assume that it is similar to the synchronous FTP classes from the .NET framework. You can install the nuget package here: https://www.nuget.org/packages/System.Net.FtpClient/
I have a need to be able to open a file on disk but pass in parameters to that file via a querystring. It's a .SWF file, so I'm passing in the parameter necessary to get it to load correctly.
The code I'm using to do so is:
FileStream fs = new FileStream(#"C:\test\file.swf?key=value", FileMode.Open, FileAccess.Read);
I'm getting an error opening the file: "Invalid characters in path" because of the "?" in the filename. Is there any way to load a file from disk into a FileStream object using a querystring in the filename?
I think you can't do what you're trying to do. When you load a file from disk the querystring does not exist as a concept. It will only return the bytes contained in the SWF file.
The querystring matters at the execution level.
So I solved this problem by putting my two SWF files on a web server and using the following code. Not exactly production ready code, but it illustrates the concept.
private static FileStream GetFileStream()
{
string url = #"http://www.someurl.com/shell.swf?Filename=actualfile.swf";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
WebResponse response = request.GetResponse();
byte[] result = null;
int byteCount = Convert.ToInt32(response.ContentLength);
using (BinaryReader reader = new BinaryReader(response.GetResponseStream()))
result = reader.ReadBytes(byteCount);
return new FileStream(result);
}
I'm trying to obtain an image to encode to a WordML document. The original version of this function used files, but I needed to change it to get images created on the fly with an aspx page. I've adapted the code to use HttpWebRequest instead of a WebClient. The problem is that I don't think the page request is getting resolved and so the image stream is invalid, generating the error "parameter is not valid" when I invoke Image.FromStream.
public string RenderCitationTableImage(string citation_table_id)
{
string image_content = "";
string _strBaseURL = String.Format("http://{0}",
HttpContext.Current.Request.Url.GetComponents(UriComponents.HostAndPort, UriFormat.Unescaped));
string _strPageURL = String.Format("{0}{1}", _strBaseURL,
ResolveUrl("~/Publication/render_citation_chart.aspx"));
string _staticURL = String.Format("{0}{1}", _strBaseURL,
ResolveUrl("~/Images/table.gif"));
string _fullURL = String.Format("{0}?publication_id={1}&citation_table_layout_id={2}",
_strPageURL, publication_id, citation_table_id);
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_fullURL);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream image_stream = response.GetResponseStream();
// Read the image data
MemoryStream ms = new MemoryStream();
int num_read;
byte[] crlf = System.Text.Encoding.Default.GetBytes("\r\n");
byte[] buffer = new byte[1024];
for (num_read = image_stream.Read(buffer, 0, 1024); num_read > 0; num_read = image_stream.Read(buffer, 0, 1024))
{
ms.Write(buffer, 0, num_read);
}
// Base 64 Encode the image data
byte[] image_bytes = ms.ToArray();
string encodedImage = Convert.ToBase64String(image_bytes);
ms.Position = 0;
System.Drawing.Image image_original = System.Drawing.Image.FromStream(ms); // <---error here: parameter is not valid
image_stream.Close();
image_content = string.Format("<w:p>{4}<w:r><w:pict><w:binData w:name=\"wordml://{0}\">{1}</w:binData>" +
"<v:shape style=\"width:{2}px;height:{3}px\">" +
"<v:imagedata src=\"wordml://{0}\"/>" +
"</v:shape>" +
"</w:pict></w:r></w:p>", _word_image_id, encodedImage, 800, 400, alignment.center);
image_content = "<w:br w:type=\"text-wrapping\"/>" + image_content + "<w:br w:type=\"text-wrapping\"/>";
}
catch (Exception ex)
{
return ex.ToString();
}
return image_content;
Using a static URI it works fine. If I replace "staticURL" with "fullURL" in the WebRequest.Create method I get the error. Any ideas as to why the page request doesn't fully resolve?
And yes, the full URL resolves fine and shows an image if I post it in the address bar.
UPDATE:
Just read your updated question. Since you're running into login issues, try doing this before you execute the request:
request.Credentials = CredentialCache.DefaultCredentials
If this doesn't work, then perhaps the problem is that authentication is not being enforced on static files, but is being enforced on dynamic files. In this case, you'll need to log in first (using your client code) and retain the login cookie (using HttpWebRequest.CookieContainer on the login request as well as on the second request) or turn off authentication on the page you're trying to access.
ORIGINAL:
Since it works with one HTTP URL and doesn't work with another, the place to start diagnosing this is figuring out what's different between the two requests, at the HTTP level, which accounts for the difference in behavior in your code.
To figure out the difference, I'd use Fiddler (http://fiddlertool.com) to compare the two requests. Compare the HTTP headers. Are they the same? In particular, are they the same HTTP content type? If not, that's likely the source of your problem.
If headers are the same, make sure both the static and dynamic image are exactly the same content and file type on the server. (e.g. use File...Save As to save the image in a browser to your disk). Then use Fiddler's Hex View to compare the image content. Can you see any obvious differences?
Finally, I'm sure you've already checked this, but just making sure: /Publication/render_citation_chart.aspx refers to an actual image file, not an HTML wrapper around an IMG element, right? This would account for the behavior you're seeing, where a browser renders the image OK but your code doesn't.