I want to automate some tasks on a page, and after the user logins and visits the page section, I need to get the request payload, marked in red in the image below:
Screenshot
I was able to get the headers (ex: gwt-permutation), implementing IRequestHandler, like below:
public bool OnResourceResponse(IWebBrowser browserControl, IBrowser browser, IFrame frame, IRequest request, IResponse response)
{
string[] headers = request.Headers.AllKeys;
// enumerate through the header collection.
foreach (string s in headers)
{
Console.WriteLine("OnResourceResponse - HEADERS: " + s);
Console.WriteLine("OnResourceResponse - HEADERSValue: " + request.Headers[s]);
}
return false;
}
But I just can't get the the rest of the data, like it's displayed in Chrome DevTools.
Thanks.
If I put the URL at the browser, my server responds properly (a XML).
Although, if this same URL pass through the WebClient.DownloadingString() method, something in the URL changes, and my server responds properly, but with an access denied message (XML, too), as if something had changed.
"Error message"
<?xml version="1.0" encoding="ISO-8859-1"?><said:service xmlns:said="http:xxx"><said:codigo_erro>8</said:codigo_erro><said:mensagem_erro>Unable</said:mensagem_erro></said:service>
The URL used on request is like this one:
http://...<parameter1>S<%2Fparameter1>%0D%0A++<parameter2>S<%2Fparameter2>%0D%0A++<parameter3>S<%2Fparameter3>%0D%0A<%2Fqueryservice>%0D%0A%09%09
I have already tried change de Encode to UT8, ISO, etc. No one of them worked.
You have to be sure that you're sending all the necessary data, cookies and request headers that the server is expecting.
I advise you to install Fiddler Web Debugger and monitor successful requests from web browser, after that try to recreate such requests in your application.
Maybe server is redirecting you to some error page because WebClient is not handling cookies. You can create your own version of WebClient and add cookie support. Create a class that inhertis from WebClient and override GetWebRequest method, there you have to add CookieContainer. Following is a simple implementation of WebClient that handles cookies:
public class MyWebClient : WebClient
{
public CookieContainer CookieContainer { get; private set; }
public MyWebClient()
{
this.CookieContainer = new CookieContainer();
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = this.CookieContainer;
(request as HttpWebRequest).AllowAutoRedirect = true;
}
return request;
}
}
I am using WebClient to get some info from a page that is sometimes not available [302 Moved Temporarily]. So i want to program to detect whether the page exists
I tried to override the WebClient WebResponse with the following code to only return the page when it's status is OK but it did not worked.
protected override WebResponse GetWebResponse(WebRequest request)
{
var response = base.GetWebResponse(request);
if (response is HttpWebResponse)
return (response as HttpWebResponse).StatusCode == HttpStatusCode.OK ? response : null;
return null;
}
when i used my overriden class to get the page (when it is unavailable) it just redirected and did not returned null
Get code
private async Task<string> Get(string uri)
{
return await Handler.DownloadStringTaskAsync(new Uri(uri));
}
[WHAT I WANT TO ACHIEVE] : i want the web client tried to get the page but it was not found so it has been redirected to another page.
WebClient will follow redirects automatically by default (up to a maximum number).
If you override GetWebRequest to modify the returned HttpWebRequest, setting its AllowAutoRedirect property to false, then I believe it will just give you back the 302 directly - although possibly via an exception...
This won't tell you the status but it can be inferred by the fact that you were redirected.
if(reponse.ResponseUri != request.RequestUri) {
// if you really want to know the status
// set AllowAutoRedirect = false;
// and send another request in here.
}
I have a fairly bog standard .net MVC 4 Web API application.
public class LogsController : ApiController
{
public HttpResponseMessage PostLog(List<LogDto> logs)
{
if (logs != null && logs.Any())
{
var goodLogs = new List<Log>();
var badLogs = new List<LogBad>();
foreach (var logDto in logs)
{
if (logDto.IsValid())
{
goodLogs.Add(logDto.ToLog());
}
else
{
badLogs.Add(logDto.ToLogBad());
}
}
if (goodLogs.Any())
{
_logsRepo.Save(goodLogs);
}
if(badLogs.Any())
{
_logsBadRepo.Save(badLogs);
}
}
return new HttpResponseMessage(HttpStatusCode.OK);
}
}
This all work fine, I have devices that are able to send me their logs and it works well. However now we are starting to have concerns about the size of the data being transferred, and we want to have a look at accepting post that have been compressed using GZIP?
How would I go about do this? Is it setting in IIS or could I user Action Filters?
EDIT 1
Following up from Filip's answer my thinking is that I need to intercept the processing of the request before it gets to my controller. If i can catch the request before the Web api framework attempts to parse the body of the request into my business object, which fails because the body of the request is still compressed. Then I can decompress the body of the request and then pass the request back into the processing chain, and hopefully the Web Api framework will be able to parse the (decompressed) body into my business objects.
It looks Like using the DelagatingHandler is the way to go. It allows me access to the request during the processing, but before my controller. So I tried the following?
public class gZipHandler : DelegatingHandler
{
protected override System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
{
string encodingType = request.Headers.AcceptEncoding.First().Value;
request.Content = new DeCompressedContent(request.Content, encodingType);
return base.SendAsync(request, cancellationToken);
}
}
public class DeCompressedContent : HttpContent
{
private HttpContent originalContent;
private string encodingType;
public DeCompressedContent(HttpContent content, string encodType)
{
originalContent = content;
encodingType = encodType;
}
protected override bool TryComputeLength(out long length)
{
length = -1;
return false;
}
protected override Task<Stream> CreateContentReadStreamAsync()
{
return base.CreateContentReadStreamAsync();
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
Stream compressedStream = null;
if (encodingType == "gzip")
{
compressedStream = new GZipStream(stream, CompressionMode.Decompress, leaveOpen: true);
}
return originalContent.CopyToAsync(compressedStream).ContinueWith(tsk =>
{
if (compressedStream != null)
{
compressedStream.Dispose();
}
});
}
}
}
This seems to be working ok. The SendAsync method is being called before my controller and the constructor for the DecompressedContent is being called. However the SerializeToStreamAsync is never being called so I added the CreateContentReadStreamAsync to see if that's where the decompressing should be happening, but that's not being called either.
I fell like I am close to the solution, but just need a little bit extra to get it over the line.
I had the same requirement to POST gzipped data to a .NET web api controller. I came up with this solution:
public class GZipToJsonHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
CancellationToken cancellationToken)
{
// Handle only if content type is 'application/gzip'
if (request.Content.Headers.ContentType == null ||
request.Content.Headers.ContentType.MediaType != "application/gzip")
{
return base.SendAsync(request, cancellationToken);
}
// Read in the input stream, then decompress in to the outputstream.
// Doing this asynronously, but not really required at this point
// since we end up waiting on it right after this.
Stream outputStream = new MemoryStream();
Task task = request.Content.ReadAsStreamAsync().ContinueWith(t =>
{
Stream inputStream = t.Result;
var gzipStream = new GZipStream(inputStream, CompressionMode.Decompress);
gzipStream.CopyTo(outputStream);
gzipStream.Dispose();
outputStream.Seek(0, SeekOrigin.Begin);
});
// Wait for inputstream and decompression to complete. Would be nice
// to not block here and work async when ready instead, but I couldn't
// figure out how to do it in context of a DelegatingHandler.
task.Wait();
// This next section is the key...
// Save the original content
HttpContent origContent = request.Content;
// Replace request content with the newly decompressed stream
request.Content = new StreamContent(outputStream);
// Copy all headers from original content in to new one
foreach (var header in origContent.Headers)
{
request.Content.Headers.Add(header.Key, header.Value);
}
// Replace the original content-type with content type
// of decompressed data. In our case, we can assume application/json. A
// more generic and reuseable handler would need some other
// way to differentiate the decompressed content type.
request.Content.Headers.Remove("Content-Type");
request.Content.Headers.Add("Content-Type", "application/json");
return base.SendAsync(request, cancellationToken);
}
}
Using this approach, the existing controller, which normally works with JSON content and automatic model binding, continued to work without any changes.
I'm not sure why the other answer was accepted. It provides a solution for handling the responses (which is common), but not requests (which is uncommon). The Accept-Encoding header is used to specify acceptable response encodings, and is not related to request encodings.
I believe the correct answer is Kaliatech's and I would have left this as a comment and voted his up is I had enough reputation points, since I think his is basically correct.
However, my situation called for the need to look at the encoding type type rather than the content type. Using this approach the calling system can still specify that the content type is json/xml/etc in the content type, but specify that the data is encoded using gzip or potentially another encoding/compression mechanism. This prevented me from needing to change the content type after decoding the input and allows any content type information to flow through in its original state.
Here's the code. Again, 99% of this is Kaliatech's answer including the comments, so please vote his post up if this is useful.
public class CompressedRequestHandler : DelegatingHandler
{
protected override System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
{
if (IsRequetCompressed(request))
{
request.Content = DecompressRequestContent(request);
}
return base.SendAsync(request, cancellationToken);
}
private bool IsRequetCompressed(HttpRequestMessage request)
{
if (request.Content.Headers.ContentEncoding != null &&
request.Content.Headers.ContentEncoding.Contains("gzip"))
{
return true;
}
return false;
}
private HttpContent DecompressRequestContent(HttpRequestMessage request)
{
// Read in the input stream, then decompress in to the outputstream.
// Doing this asynronously, but not really required at this point
// since we end up waiting on it right after this.
Stream outputStream = new MemoryStream();
Task task = request.Content.ReadAsStreamAsync().ContinueWith(t =>
{
Stream inputStream = t.Result;
var gzipStream = new GZipStream(inputStream, CompressionMode.Decompress);
gzipStream.CopyTo(outputStream);
gzipStream.Dispose();
outputStream.Seek(0, SeekOrigin.Begin);
});
// Wait for inputstream and decompression to complete. Would be nice
// to not block here and work async when ready instead, but I couldn't
// figure out how to do it in context of a DelegatingHandler.
task.Wait();
// Save the original content
HttpContent origContent = request.Content;
// Replace request content with the newly decompressed stream
HttpContent newContent = new StreamContent(outputStream);
// Copy all headers from original content in to new one
foreach (var header in origContent.Headers)
{
newContent.Headers.Add(header.Key, header.Value);
}
return newContent;
}
I then registered this handler globally, which could be a dicey proposition if you are vulnerable to DoS attacks, but our service is locked down, so it works for us
GlobalConfiguration.Configuration.MessageHandlers.Add(new CompressedRequestHandler());
While Web API doesn't support Accept-Encoding header out of the box, but Kiran has a terrific blog post on how to do that - http://blogs.msdn.com/b/kiranchalla/archive/2012/09/04/handling-compression-accept-encoding-sample.aspx - using a custom MessageHandler
If you implement his solution, all you need to do is issue a request with Accept-Encoding: gzip or Accept-Encoding: deflate header and the Web API response will be compressed in the message handler for you.
try this
public class DeCompressedContent : HttpContent
{
private HttpContent originalContent;
private string encodingType;
/// <summary>
///
/// </summary>
/// <param name="content"></param>
/// <param name="encodingType"></param>
public DeCompressedContent(HttpContent content, string encodingType)
{
if (content == null) throw new ArgumentNullException("content");
if (string.IsNullOrWhiteSpace(encodingType)) throw new ArgumentNullException("encodingType");
this.originalContent = content;
this.encodingType = encodingType.ToLowerInvariant();
if (!this.encodingType.Equals("gzip", StringComparison.CurrentCultureIgnoreCase) && !this.encodingType.Equals("deflate", StringComparison.CurrentCultureIgnoreCase))
{
throw new InvalidOperationException(string.Format("Encoding {0} is not supported. Only supports gzip or deflate encoding", this.encodingType));
}
foreach (KeyValuePair<string, IEnumerable<string>> header in originalContent.Headers)
{
this.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
this.Headers.ContentEncoding.Add(this.encodingType);
}
/// <summary>
///
/// </summary>
/// <param name="stream"></param>
/// <param name="context"></param>
/// <returns></returns>
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
var output = new MemoryStream();
return this.originalContent
.CopyToAsync(output).ContinueWith(task =>
{
// go to start
output.Seek(0, SeekOrigin.Begin);
if (this.encodingType.Equals("gzip", StringComparison.CurrentCultureIgnoreCase))
{
using (var dec = new GZipStream(output, CompressionMode.Decompress))
{
dec.CopyTo(stream);
}
}
else
{
using (var def = new DeflateStream(output, CompressionMode.Decompress))
{
def.CopyTo(stream);
}
}
if (output != null)
output.Dispose();
});
}
/// <summary>
///
/// </summary>
/// <param name="length"></param>
/// <returns></returns>
protected override bool TryComputeLength(out long length)
{
length = -1;
return (false);
}
}
I need to setup a web page that listens for XML document via an HTTP POST. I don't need to POST out, I need to receive that POST. What object does this? Should I use an HTTP handler, web service, webRequest, Stream or something else? I need to use a IIS Server and prefer C#.
I've Tried...
I dont think I can use WebRequest since I'm not sending a request, just waiting for them.
"HttpRequest.InputStream" but I'm not sure how to use it or where to put it. Do i need to use it with a web service or a asp.net application? I put it in
http://forums.asp.net/t/1371873.aspx/1
I've tried a simple web service http://msdn.microsoft.com/en-us/library/bb412178.aspx - But when i try to visit "http://localhost:8000/EchoWithGet?s=Hello, world!", i get a "webpage cannot be found error"
If anyone has any helpful code or links that would be great!
EDIT:
I am trying to receive notifications from another program.
You could write an ASP.NET application that you will host in IIS in which you could either have an .ASPX page or a generic .ASHX handler (depending on how you want the result to be formatted - do you want to return HTML or some other type) and then read the Request.InputStream which will contain the body of the request that comes from the client.
Here's an example of how you could write a generic handler (MyHandler.ashx):
public class MyHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
var stream = context.Request.InputStream;
byte[] buffer = new byte[stream.Length];
stream.Read(buffer, 0, buffer.Length);
string xml = Encoding.UTF8.GetString(buffer);
... do something with the XML
// We only set the HTTP status code to 202 indicating to the
// client that the request has been accepted for processing
// but we leave an empty response body
context.Response.StatusCode = 202;
}
public bool IsReusable
{
get
{
return false;
}
}
}
I'm not sure where to call or use the handler. This is what i have so far...
Default.aspx
<%#Page Inherits="WebApplication1._Default"%>
<%#OutputCache Duration="10" Location="Server" varybyparam="none"%>
<script language="C#" runat="server">
void Page_Init(object sender, EventArgs args) {
}
}
</script>
<html>
<body>
</body>
</html>
Default.aspx.cs
namespace WebApplication1
{
public partial class _Default : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
HttpContext contex = Context;
MyHandler temp = new MyHandler();
temp.ProcessRequest(context);
}
}
public class MyHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
var stream = context.Request.InputStream;
byte[] buffer = new byte[stream.Length];
stream.Read(buffer, 0, buffer.Length);
string xml = Encoding.UTF8.GetString(buffer);
... do something with the XML
// We only set the HTTP status code to 202 indicating to the
// client that the request has been accepted for processing
// but we leave an empty response body
context.Response.StatusCode = 202;
}
public bool IsReusable
{
get
{
return false;
}
}
}
}