I am trying to add trailer in HttpWebRequest header, but it is not appending that trailer after end of file data.
wreq.ContentType = "application/octet-stream";
wreq.AllowWriteStreamBuffering = false;
wreq.SendChunked = true;
//wreq.Headers.Add(HttpRequestHeader.Te, "trailers");
wreq.Headers.Add(HttpRequestHeader.Trailer, "Test");
wreq.Headers["Test"] = "the-value";
using (Stream POSTstream = wreq.GetRequestStream())
{
//dataByte is file-data in byte[]
POSTstream.Write(dataByte, 0, dataByte.Length);
POSTstream.Flush();
//hashValue is trailer in byte[]
POSTstream.Write(hashValue, 0, hashValue.Length);
POSTstream.Flush();
POSTstream.Close();
}
it should append this trailer "Test" # EOF after blank chunk, but it doesn't append it. when i tried to add trailer programatically it consider it as file data rather than trailer.
Expected request:
POST <URL> HTTP/1.1
Content-Type: application/octet-stream
Trailer: Test
Transfer-Encoding: chunked
5d
File-data
0
Test: the-value
Actual request:
POST <URL> HTTP/1.1
Content-Type: application/octet-stream
Trailer: Test
Transfer-Encoding: chunked
5d
File-data
5A
Test: the-value
0
Why this Test trailer is not getting after blank chunk. This trailer will be used on server to identify end of file.
Please Help.
After more than a week research I came to know that Dotnet does not allow to add trailer in httprequest. To achieve above expected request I have used Node.js. For that,
First I have create app.js file which contains code for creating request with trailer and get response from server:
var http = require('http');
var fs = require('fs');
var options = {hostname: 'ABC',port: 6688,path: 'XYZ/101/-3/test.file',method: 'POST',
headers: {'Content-type' : 'application/octet-stream','Trailer' : 'Test','Transfer-Encoding': 'chunked'}};
var fileContent = fs.readFileSync('C:\\test.file');
var req = http.request(options, function(res) {
res.setEncoding('utf8');
res.on('data', function (chunk) {fs.writeFile('C:\\response.xml', chunk);});});
req.on('error', function(e) {fs.writeFile('C:\\response.xml','Error: ' + e.message);});
var len = fileContent.length;
var bufSize = 4096;
for (var i = 0 ; i < len ; ) {
if (i+bufSize <len){req.write(fileContent.slice(i, i+bufSize));}
else{req.write(fileContent.slice(i,len));req.addTrailers({'Test': 'TESTTEST'});req.end();}i = i +bufSize;
}
And run this app.js file from dotnet as application and get the response:
//build app.js file content
var template = GetAppJSString();
var appjsfile = "C:\\app.js";
using (var sw = new StreamWriter(appjsfile))
{
sw.Write(template);
sw.Close();
}
var psi = new ProcessStartInfo
{
CreateNoWindow = true,
FileName = "C:\\Node.exe",
Arguments = "\"" + appjsfile + "\"",
UseShellExecute = false,
WindowStyle = ProcessWindowStyle.Hidden
};
try
{
_nodeProcess.StartInfo = psi;
_nodeProcess.Start();
_nodeProcess.WaitForExit();
//read and process response
var responseText = string.Empty;
using (var sr = new StreamReader("C:\\response.xml"))
{
responseText = sr.ReadToEnd();
sr.Close();
}
File.Delete("C:\\response.xml");
// process response
}
catch (Exception ex) { }
Hope this will help others and save their time.
Related
I am stuck with reading data from HttpListener. Data arrives, I verify it with request.ContentLength64 that is usually over 8000 and it increases as the server generates more and more data.
The server sends data as HTTP post and the content type is text/plain.
When I try to check whether streamreader got some data via its length attribute I get 0.
The code is a little bit messy as I was trying different ways to make it work but unfortunatelly I had no luck.
Does anyone got an idea what I'm doing wrong?
Thanks!
HttpListener listener2 = new HttpListener();
listener2.Prefixes.Clear();
listener2.Prefixes.Add("http://+:4200/");
listener2.Prefixes.Add("http://XXX.XXX.eu/");
listener2.Start();
LogWriteLine("http listener started listening to: " +listener2.Prefixes);
try
{
while (true)//change to match end check
{
LogWriteLine("http listener waiting");
HttpListenerContext context = listener2.GetContext();
LogWriteLine("http request arrived");
HttpListenerRequest request = context.Request;
// Obtain a response object.
HttpListenerResponse response = context.Response;
System.IO.Stream body = request.InputStream;
System.Text.Encoding encoding = request.ContentEncoding;
System.IO.StreamReader reader = new System.IO.StreamReader(body, encoding);
if (!request.HasEntityBody)
{
LogWriteLine("No client data was sent with the request.");
Thread.Sleep(300);
//return;
}
if (request.ContentType != null)
{
LogWriteLine("Client data content type " + request.ContentType);
}
LogWriteLine("Client data content length " + request.ContentLength64); //Works fine
LogWriteLine("Start of client data:");
// Convert the data to a string and display it on the console.
Console.WriteLine(body.CanSeek);
string s = reader.ReadToEnd();
var ahoj = new StreamReader(context.Request.InputStream).ReadToEnd();
Console.WriteLine("ahoj length " + ahoj.Length); //0
Console.WriteLine(s); //nothing
string text;
var bytes = default(byte[]);
using (var reader1 = new StreamReader(request.InputStream,
request.ContentEncoding))
{
text = reader1.ReadToEnd();
Console.WriteLine(text + text.Length); //output: 0
using (var memstream = new MemoryStream())
{
reader1.BaseStream.CopyTo(memstream);
bytes = memstream.ToArray();
}
Console.WriteLine("bytes:" + bytes.Length); //output: bytes: 0
}
LogWriteLine("End of client data:");
//write to console file
sw.Write(s);
body.Close();
reader.Close();
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
My current code downloads a SharePoint file from an absolute URL of the file and writes to local.
I want to change it to use the folder URL instead and downloads file in the folder base on some filter.
Can it be done?
Below is my current code snippet:
string fullFilePath = DownloadSPFile("http://MySharePointSite.com/sites/Collection1/Folder1/File1.docx/");
public static string DownloadSPFile(string urlPath)
{
string serverTempdocPath = "";
try
{
var request = (HttpWebRequest)WebRequest.Create(urlPath);
var credentials = new NetworkCredential("username", "password", "domain");
request.Credentials = credentials;
request.Timeout = 20000;
request.AllowWriteStreamBuffering = false;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
serverTempdocPath = Path.Combine(AppConfig.EmailSaveFilePath + "_DOWNLOADED.eml");
using (FileStream fs = new FileStream(serverTempdocPath, FileMode.Create))
{
byte[] read = new byte[256];
int count = stream.Read(read, 0, read.Length);
while (count > 0)
{
fs.Write(read, 0, count);
count = stream.Read(read, 0, read.Length);
}
}
}
}
}
catch (Exception ex)
{
AppLogger.LogError(ex, "");
throw ex;
}
return serverTempDocPath;
}
If your sharepoint site enables sharepoint rest api, you can get the details very easy.
Get list of files
url: http://site url/_api/web/GetFolderByServerRelativeUrl('/Folder Name')/Files
method: GET
headers:
Authorization: "Bearer " + accessToken
accept: "application/json;odata=verbose" or "application/atom+xml"
and pass query for that
"?filter=$top=1&$orderby=Created desc"
More information
Has you try to format your urlPath ?
like:
string urlPath = "path/to/folder/";
string fileFilter = "Cat.png";
string finalPath= String.format("{0}{1}", urlPath, fileFilter);
(Or anything like this)
(Hope I understand your question and I was able to help you ^^)
I have some difficulties in requesting an access token from an other server.
A request to get that is:
POST /auth/O2/token HTTP/1.1
Host: api.amazon.com
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
grant_type=client_credentials&scope=messaging:push&client_id=(YOUR_CLIENT_ID)&client_secret=(YOUR_CLIENT_SECRET)
and the response I want to obtain is like:
X-Amzn-RequestId: d917ceac-2245-11e2-a270-0bc161cb589d
Content-Type: application/json
{
"access_token":"Atc|MQEWYJxEnP3I1ND03ZzbY_NxQkA7Kn7Aioev_OfMRcyVQ4NxGzJMEaKJ8f0lSOiV-yW270o6fnkI",
"expires_in":3600,
"scope":"messaging:push",
"token_type":"Bearer"
}
I tried to get it through :
private String getAccessToken(String client_id,String client_secret)
{
HttpWebRequest httpWReq =
(HttpWebRequest)WebRequest.Create("http://api.amazon.com/auth/02/token");
Encoding encoding = new UTF8Encoding();
string postData = "grant_type=client_credentials";
postData += "&scope=messaging:push";
postData += "&client_id=" + client_id;
postData += "&client_secret=" + client_secret;
byte[] data = encoding.GetBytes(postData);
httpWReq.Method = "POST";
httpWReq.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
httpWReq.ContentLength = data.Length;
using (Stream stream = httpWReq.GetRequestStream()) // ***here I get this exception : Unable to connect to the remote server !!!****
{
stream.Write(data, 0, data.Length);
}
HttpWebResponse response = (HttpWebResponse)httpWReq.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
String jsonresponse = "";
String temp = null;
while ((temp = reader.ReadLine()) != null)
{
jsonresponse += temp;
}
var dict = new JavaScriptSerializer().Deserialize<Dictionary<string, string>>(jsonresponse);
access_token = dict["access_token"];
String expires_in = dict["expires_in"];
}
return access_token;
}
I am getting this execption : Unable to connect to the remote server, when I tried to get request stream
Check this ...
its not 02 its O2
in your code , it may be the error
HttpWebRequest httpWReq =
(HttpWebRequest)WebRequest.Create("http://api.amazon.com/auth/**02**/token");
Try this
HttpWebRequest httpWReq =
(HttpWebRequest)WebRequest.Create("https://api.amazon.com/auth/o2/token");
Thank you ...
The first problem is that api.amazon.com/auth/02/token has nothing listening on port 80 (so it won't work); so probably it needs https (but if you provide reference to where this is documented we can better advise)
Secondly I think it'd be cleaner to replace the first part of your code with the following;
using (WebClient client = new WebClient())
{
byte[] response = client.UploadValues("https://api.amazon.com/auth/02/token", new NameValueCollection()
{
{ "scope", "messaging:push" },
{ "client_id", "1123" },
{ "client_secret", "2233"}
});
// handle response...
}
I put url to browser's address bar and it downloads the zip file to HD. The size of zipped file is 386 bytes as written in its properties.
When I use UnZipFiles method to extract the file - it works.
But, I want to download programaticaly and extract it in memory. I use GetResultFromServer method to get zipped content. As shown in headers the size of the content is the same as the size of zipped file saved on HD:
content-disposition: attachment; filename=emaillog-xml.zip
Content-Length: 386
Cache-Control: private
Content-Type: application/zip
Date: Mon, 10 Sep 2012 08:28:28 GMT
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
My question is how to extract the content returned by GetResultFromServer?
I tried the following:
var ms = new MemoryStream(Encoding.Unicode.GetBytes(res))
var s = new ZipInputStream(ms);
but I get Unable to read from this stream.
UPDATED
I tried var zipStream = new System.IO.Compression.GZipStream(response.GetResponseStream(), CompressionMode.Decompress) but I get The magic number in GZip header is not correct error
Code
private string GetResultFromServer(ElasticLogParams elasticLogParams)
{
var webRequest = (HttpWebRequest)WebRequest.Create(url);
var response = webRequest.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
var res = reader.ReadToEnd();
var headers = response.Headers.ToString();
return res;
}
}
public static void UnZipFiles(string zippedFilePath, Stream stream = null)
{
var s = new ZipInputStream(stream ?? File.OpenRead(zippedFilePath));
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
using (var streamWriter = File.Create(#"D:\extractedXML.xml"))
{
var size = 2048;
var data = new byte[size];
while (true)
{
size = s.Read(data, 0, size);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
streamWriter.Close();
}
}
s.Close();
}
Give this a shot:
var response = webRequest.GetResponse() as HttpWebResponse;
var stream = response.GetResponseStream();
var s = new ZipInputStream(stream);
I believe you're very close and that you're using the right approach -- you can use this article to back that up -- their code is very similar.
I'm using http://dotnetzip.codeplex.com/
I've not used it to download stuff, but to extract stuff that people upload to my server. I assume it should work perfectly the other way round too.
I've tried the built-in zip from microsoft too, but also had issues. So I gave it up and switched.
I'm downloading a file, from a webserver, using the code below, and am recieving this error:
The Error:
Error saving file from URL:The server committed a protocol violation.
Section=ResponseHeader Detail='Content-Length' header value is invalid
From running Fiddler while this is running, it says:
Content-Length response header is not a valid unsigned integer
Content-Length: 13312583
The Code:
public static bool SaveFileFromURL(string url, string destinationFileName, int timeoutInSeconds)
{
//SetAllowUnsafeHeaderParsing20();
Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
SettingsSection section = (SettingsSection)config.GetSection("system.net/settings");
section.HttpWebRequest.UseUnsafeHeaderParsing = false;
config.Save();
// Create a web request to the URL
HttpWebRequest MyRequest = (HttpWebRequest)WebRequest.Create(url);
MyRequest.UseDefaultCredentials = true;
MyRequest.ContentLength = 0;
MyRequest.Timeout = timeoutInSeconds * 1000;
try
{
// Get the web response
HttpWebResponse MyResponse = (HttpWebResponse)MyRequest.GetResponse();
// Make sure the response is valid
if (HttpStatusCode.OK == MyResponse.StatusCode)
{
// Open the response stream
using (Stream MyResponseStream = MyResponse.GetResponseStream())
{
// Open the destination file
using (FileStream MyFileStream = new FileStream(destinationFileName, FileMode.OpenOrCreate, FileAccess.Write))
{
// Create a 4K buffer to chunk the file
byte[] MyBuffer = new byte[4096];
int BytesRead;
// Read the chunk of the web response into the buffer
while (0 < (BytesRead = MyResponseStream.Read(MyBuffer, 0, MyBuffer.Length)))
{
// Write the chunk from the buffer to the file
MyFileStream.Write(MyBuffer, 0, BytesRead);
}
}
}
}
}
catch (Exception err)
{
throw new Exception("Error saving file from URL:" + err.Message, err);
}
return true;
}
Update: If I pass the URL straight into a browser, the file is downloaded successfully, and the error is thrown on the GetResponse line.
Update 2: I get the same error with WebClient.Downloadfile:
public static bool DL_Webclient(string url, string destinationFileName)
{
WebClient myWebClient = new WebClient();
myWebClient.UseDefaultCredentials = true;
myWebClient.DownloadFile(url, destinationFileName);
return true;
}
Update 3: Having retrieved the other headers in the message (using Fiddler), they are:
HTTP/1.1 200 OK
Connection: close
Date: Wed, 03 Mar 2010 08:43:06 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 13314320
Content-Type: application/x-evsaveset
Set-Cookie: ASPSESSIONIDQQCQSCRC=CFHHJHADOIBCFAFOHFJCDNEG; path=/
Cache-control: private
Are there other HTTP headers present?
It might have to do with the 'Transfer-Encoding' header or similar rules. For more specific information about these headers and their effect on the Content-Length header can be found at the W3C website and More W3C website
Hope this helps,
Could you not use WebClient.DownloadFile instead?