We Keep Coding

C# Uploading a memorystream object to Web API

I have an CSV file in memory that I want to upload to a Web API.
If I save the CSV file to disk and upload it, it gets accepted.
However, I want to avoid the extra work and also make the code cleaner by simply uploading the text I have as a MemoryStream Object (I think that's the correct format?).
The following code works for uploading the file:
string webServiceUrl = "XXX";
string filePath = #"C:\test.csv";
string cred = "YYY";
using (var client = new WebClient()){
client.Headers.Add("Authorization", "Basic " + cred);
byte[] rawResponse = client.UploadFile(webServiceUrl, "POST", filePath);
Console.WriteLine(System.Text.Encoding.ASCII.GetString(rawResponse));
}
How would I do if I had a string with all the contents and I want to upload it in the same way without having to save it down to a file?
WebClient.UploadData or WebClient.UploadString perhaps?
Thank you
EDIT:
I tried what you said but by using a local file (in case there was something wrong with the string), but I get the same error.
Here is what I suppose the code would be using your solution
string webServiceUrl = "XXX";
string file = #"C:\test.csv";
string cred = "YYY";
FileStream fs = new FileStream(file, FileMode.Open, FileAccess.Read);
BinaryReader r = new BinaryReader(fs);
byte[] postArray = r.ReadBytes((int)fs.Length);
using (var client = new WebClient())
{
client.Headers.Add("Authorization", "Basic " + cred);
using (var postStream = client.OpenWrite(webServiceUrl, "POST"))
{
postStream.Write(postArray, 0, postArray.Length);
}
}
Any thoughts?

Use OpenWrite() from the WebClient.
using (var postStream = client.OpenWrite(endpointUrl))
{
postStream.Write(memStreamContent, 0, memStream.Length);
}
As documentation mentioned:
The OpenWrite method returns a writable stream that is used to send data to a resource.
Update
Try to set the position of the MemoryStream to 0 before uploading.
memoryStream.Position = 0;
When you copy the file into the MemoryStream, the pointer is moved to the end of the stream, so when you then try to read it, you're getting a null byte instead of your stream data.
MSDN - CopyTo()
Copying begins at the current position in the current stream, and does not reset the position of the destination stream after the copy operation is complete.

I finally managed to solve it.
First I made a request using CURL that worked.
I analyzed the packet data and made an except copy of the packet.
I did a lot of changes, however, the final change was that using the different functions I found online it never closed the packet with a "Last-Boundary" while CURL did.
So by modifying the function, making sure it properly wrote a Last-Boundary it finally worked.
Also, another crucial thing was to set PreAuthenticate to true, the examples online didn't do that.
So, all in all:
1. Make sure that the packet is properly constructed.
2. Make sure you pre authenticate if you need to authenticate.
webrequest.PreAuthenticate = true;
webrequest.Headers[HttpRequestHeader.Authorization] = string.Format("Basic {0}", cred);
Don't forget to add SSL if using a https (which you probably do if you authenticate):
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Ssl3;
Hope this helps someone.
And thanks for the help earlier!

Download a PDF from a third party using ASP.NET HttpWebRequest/HttpWebResponse

I want to send a url as query string e.g.
localhost/abc.aspx?url=http:/ /www.site.com/report.pdf
and detect if the above URL returns the PDF file. If it will return PDF then it gets saved automatically otherwise it gives error.
There are some pages that uses Handler to fetch the files so in that case also I want to detect and download the same.
localhost/abc.aspx?url=http:/ /www.site.com/page.aspx?fileId=223344
The above may return a pdf file.
What is best way to capture this?
Thanks

You can download a PDF like this
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = req.GetResponse();
//check the filetype returned
string contentType = response.ContentType;
if(contentType!=null)
{
splitString = contentType.Split(';');
fileType = splitString[0];
}
//see if its PDF
if(fileType!=null && fileType=="application/pdf"){
Stream stream = response.GetResponseStream();
//save it
using(FileStream fileStream = File.Create(fileFullPath)){
// Initialize the bytes array with the stream length and then fill it with data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, bytesInStream.Length);
// Use write method to write to the file specified above
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}
response.Close();
The fact that it may come from an .aspx handler doesn't actually matter, it's the mime returned in the server response that is used.
If you are getting a generic mime type, like application/octet-stream then you must use a more heuristical approach.
Assuming you cannot simply use the file extension (eg for .aspx), then you can copy the file to a MemoryStream first (see How to get a MemoryStream from a Stream in .NET?). Once you have a memory stream of the file, you can take a 'cheeky' peek at it (I say cheeky because it's not the correct way to parse a PDF file)
I'm not an expert on PDF format, but I believe reading the first 5 chars with an ASCII reader will yield "%PDF-", so you can identify that with
bool isPDF;
using( StreamReader srAsciiFromStream = new StreamReader(memoryStream,
System.Text.Encoding.ASCII)){
isPDF = srAsciiFromStream.ReadLine().StartsWith("%PDF-");
}
//set the memory stream back to the start so you can save the file
memoryStream.Position = 0;

Upload CSV Google Analytics Data File c#

I'm trying to upload a CSV file to google analytics using C#. Theirs an example on the google developers site here:
https://developers.google.com/analytics/solutions/articles/cost-data-import#custom_data_source
However, it uses python. Im trying to rewrite it using C#, but it isn't turning out as easy as I thought. any help?
Don't know how to implement the media from the code sample from the site either, so "csvToUpload" isn't actually doing anything. (csvToUpload is the csv file directory).
Code Snippet:
public void UploadCSVFile(string csvToUpload, string uploadDate, string accountID, string webPropertyID, string dataSourceID)
{
FileStream stream2 = new FileStream(csvToUpload, FileMode.Open);
ManagementResource.DailyUploadsResource.UploadMediaUpload dailyUploads = service.Management.DailyUploads.Upload(accountID, webPropertyID,
dataSourceID, uploadDate, 1, ManagementResource.DailyUploadsResource.UploadMediaUpload.TypeEnum.Cost, stream2, "Data Test");
dailyUploads.Reset = true;
//dailyUploads.UploadAsync();
dailyUploads.Upload();
stream2.Close();
}
Wanted to update the code, and the post to see if anyone could help me out, here's updated code for this quest.
Any help would be appreciated.

Upload has a second one that lets you send a stream. Load your csv file as a stream and try and send it that way.

I know that it's just too late, but for other people, this code works fine:
// Read file to stream
FileStream dataStream = new FileStream(AppWindow.WindowBag["File"].ToString(), FileMode.Open);
// Upload selected file
ManagementResource.DailyUploadsResource.UploadMediaUpload request = AppWindow.Service.Management.DailyUploads.Upload(AppWindow.WindowBag["Account"].ToString(),
AppWindow.WindowBag["Property"].ToString(),
AppWindow.WindowBag["Dataset"].ToString(),
"2014-04-14",
1,
ManagementResource.DailyUploadsResource.UploadMediaUpload.TypeEnum.Cost,
dataStream, "application/octet-stream");
request.Reset = true;
request.ProgressChanged += request_ProgressChanged;
request.ResponseReceived += request_ResponseReceived;
request.Upload();
dataStream.Close();

Issues with webscraping in C#: Downloading and parsing zipped text files

I am writing an webscraper, to do the download content from a website.
Traversing to the website/URL, triggers the creation of a temporary URL. This new URL has a zipped text file. This zipped file has to be downloaded and parsed.
I have written a scraper in C# using WebClient and its function DownloadFileAsync(). The zipped file is read from the designated location on a trapped DownloadFileCompleted event.
My issue is The Windows Open/Save dialog are triggered. This requires user input and the automation is disrupted.
Can you suggest a way to bypass the issue ? I am cool with rewriting the code using any alternate libraries. :)
Thanks for reading

You can use 'HttpWebRequest' to perform the request and save the streamed bytes to disk.
var request = WebRequest.Create(#"your url here");
request.Method=WebRequestMethods.Http.Get;
var response = request.GetResponse();
using (var writeStream = new FileStream(#"path", FileMode.Create))
{
using (var readStream = response.GetResponseStream())
{
var buffer = new byte[1024];
var readCount = readStream.Read(buffer,0,buffer.Length);
while (readCount > 0)
{
writeStream.Write(buffer,0,buffer.Length);
readCount= readStream.Read(buffer,0,buffer.Length);
}
}
}

Slow performance in reading from stream .NET

I have a monitoring system and I want to save a snapshot from a camera when alarm trigger.
I have tried many methods to do that…and it’s all working fine , stream snapshot from the camera then save it as a jpg in the pc…. picture (jpg format,1280*1024,140KB)..That’s fine
But my problem is in the application performance...
The app need about 20 ~30 seconds to read the steam, that’s not acceptable coz that method will be called every 2 second .I need to know what wrong with that code and how I can get it much faster than that. ?
Many thanks in advance
Code:
string sourceURL = "http://192.168.0.211/cgi-bin/cmd/encoder?SNAPSHOT";
byte[] buffer = new byte[200000];
int read, total = 0;
WebRequest req = (WebRequest)WebRequest.Create(sourceURL);
req.Credentials = new NetworkCredential("admin", "123456");
WebResponse resp = req.GetResponse();
Stream stream = resp.GetResponseStream();
while ((read = stream.Read(buffer, total, 1000)) != 0)
{
total += read;
}
Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(buffer, 0,total));
string path = JPGName.Text+".jpg";
bmp.Save(path);

I very much doubt that this code is the cause of the problem, at least for the first method call (but read further below).
Technically, you could produce the Bitmap without saving to a memory buffer first, or if you don't need to display the image as well, you can save the raw data without ever constructing a Bitmap, but that's not going to help in terms of multiple seconds improved performance. Have you checked how long it takes to download the image from that URL using a browser, wget, curl or whatever tool, because I suspect something is going on with the encoding source.
Something you should do is clean up your resources; close the stream properly. This can potentially cause the problem if you call this method regularly, because .NET will only open a few connections to the same host at any one point.
// Make sure the stream gets closed once we're done with it
using (Stream stream = resp.GetResponseStream())
{
// A larger buffer size would be benefitial, but it's not going
// to make a significant difference.
while ((read = stream.Read(buffer, total, 1000)) != 0)
{
total += read;
}
}

I cannot try the network behavior of the WebResponse stream, but you handle the stream twice (once in your loop and once with your memory stream).
I don't thing that's the whole problem but I'd give it a try:
string sourceURL = "http://192.168.0.211/cgi-bin/cmd/encoder?SNAPSHOT";
WebRequest req = (WebRequest)WebRequest.Create(sourceURL);
req.Credentials = new NetworkCredential("admin", "123456");
WebResponse resp = req.GetResponse();
Stream stream = resp.GetResponseStream();
Bitmap bmp = (Bitmap)Bitmap.FromStream(stream);
string path = JPGName.Text + ".jpg";
bmp.Save(path);

Try to read bigger pieces of data, than 1000 bytes per time. I can see no problem with, for example,
read = stream.Read(buffer, 0, buffer.Length);

Try this to download the file.
using(WebClient webClient = new WebClient())
{
webClient.DownloadFile("http://192.168.0.211/cgi-bin/cmd/encoder?SNAPSHOT", "c:\\Temp\myPic.jpg");
}
You can use a DateTime to put a unique stamp on the shot.