I want to be able to log the full HTTP request with values with every error message that i log. Current conversion pattern i have looks like this -
<conversionPattern value="%date{G} [%thread] %-5level
[%aspnet-request{AUTH_USER}] [%aspnet-request{REMOTE_ADDR}]
%newline%message%newline
ALL_RAW::%aspnet-request{ALL_RAW}
%newline%newline%newline" />
Which gives me an output like follows
06/19/2015 11:56:02 [60] INFO [user#email.com] [mo-machine]
MESSAGE :: BOOMMMM
ALL_RAW :: Cache-Control: no-cache
Connection: keep-alive
Content-Length: 0
Content-Type: text/plain;charset=UTF-8
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Host: localhost:43899
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36
CSP: active
Origin: chrome-extension://fhbjghcddcbncdddomop
Postman-Token: 38f023c1-3298-5726-e43f-650bab852cf2
URL :: POST localhost:43899 /controller/test InternalID=10124&MID=0921&Detail&OrgID=14347
I am still missing the post body, How do i get the raw post body in the log as well? Is there a better way to do this and log the whole HTTP request with all the parameters and the body posted logged correctly?
Related
I have a Web API application that we forward the HttpRequestMessage.Content to other methods that process the request. One of the set of values that is needed is the Headers that are sent as part of the request.
When I look at HttpRequestMessage.Headers.headerStore, I see all the headers sent to the request. But when I look at HttpRequestMessage.Content.Headers.headerStore, I only see a few of default headers.
Why doesn't HttpRequestMessage.Content.headerStore contain all the headers associated with the request?
Request headers and content headers have different purposes.
While request headers carry information about request itself and about client (caller), content headers describe "entity" or its metadata.
Have a look at sample http request:
POST /some/url HTTP/1.1
Host: someHost
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
request Accept: application/json, text/plain, */*
headers Accept-Encoding: gzip,deflate
Connection: keep-alive
Referer: url
Content-Type: multipart/form-data; boundary=----------564564546545645
Content-Length: 462560
------------564564546545645
content Content-Disposition: form-data; name="file"; filename="1.png"
headers Content-Type: image/png
.PNG
......................;
------------564564546545645
I'm trying to log into the website of my school using c#. Using the software Fiddler, I've managed to capture the post request my browser makes and the post request I've recreated in c#. They seem to be completly identical, however the response I get is different. Shouldn't the server theoretically respond the same way if the post request is the same, at least with the same status code?
When I use my own client I get response code 200, but with my browser I get 303(which it should be).
Something I notice is that the WebForm content is much bigger in browser, but the values used in the client request are from a prior GET request. Sorry for the long post.
These are the requests captured by Fiddler:
Browser request:
POST https://www.lectio.dk/lectio/31/login.aspx HTTP/1.1
Host: www.lectio.dk
Connection: keep-alive
Content-Length: 881
Cache-Control: max-age=0
Origin: https://www.lectio.dk
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3
Referer: https://www.lectio.dk/lectio/31/login.aspx
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-AS;q=0.9,en-DK;q=0.8,en;q=0.7,da-DK;q=0.6,da;q=0.5,en-US;q=0.4
Cookie: LastLoginExamno=31;
ASP.NET_SessionId=ANTOMFZ7ZZWAYYW52OSZQYRXIBAIEGIPGILQHQ6TEJ2O5XEUTE3CAIBA; isloggedin3=N
time=0&__EVENTTARGET=m%24Content%24submitbtn2&__EVENTARGUMENT=&__SCROLLPOSITION=&__VIEWSTATEX=vQAAAGlpZQk1NDg4MjIyMjVpbAJrAIFsAmhpZGwCZwJpbAJrAWUDb2ZmbASBaWRsAoFpZGwCgWlkbAJoaWpkam4BZQcyMDE5LzIwbgFlBDIwMTl%2BAXFsAWhkZwNpZGwCZwVpZGwCaGlkbAJnB2lkbAaBaWwCawJlFE4mIzIzMDtydW0gR3ltbmFzaXVtZGcFaWRsAoFpZGwCgWlsAmsDZQI1MGRnB2lkbAKBaWRsAoFpamlsAmsEcGRkZGRkBQAAABNWYWxpZGF0ZVJlcXVlc3RNb2RlDGF1dG9jb21wbGV0ZQlpbm5lcmh0bWwJbWF4bGVuZ3RoB0NoZWNrZWQAXXvzIZezMAoPCqv5j%2FZuNIu6H1E%3D&__VIEWSTATEY_KEY=&__VIEWSTATE=&__EVENTVALIDATION=CY6WRLYHybmNGyj0%2FWgw9s%2BaewM2gnWAIUbvnFzgoYWgYPLkQnxHIBIpQPjEa3On7opgtRStMA%2FpLhG9PzzgXNTaWwTZIeSfKwS74n4yMULKRovRw2H%2Fwlg8HRUdJemRR%2FyFEzIkbToD8psr4CO9G3nzX706D25SYrmTc2WygXZ%2B1oYTAMe3FD7ocBdwh%2FrHRBAOaoTzCsCSlpMVce8GxcWKS3lryh8E8yVmy4AgrHpjgj3R0g1ziPKG96%2B1vVr%2B&m%24Content%24username2=username&m%24Content%24passwordHidden=password&LectioPostbackId=
C# client request:
POST https://www.lectio.dk/lectio/31/login.aspx HTTP/1.1
Cache-Control: max-age=0
Accept: text/html, application/xhtml+xml, application/xml; q=0.9, image/webp, image/apng, /; q=0.8, application/signed-exchange; v=b3
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB, en-AS; q=0.9, en-DK; q=0.8, en; q=0.7, da-DK; q=0.6, da; q=0.5, en-US; q=0.4
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36
Referer: https://www.lectio.dk/lectio/31/login.aspx
Origin: https://www.lectio.dk
Upgrade-Insecure-Requests: 1
Connection: keep-alive
Cookie: LastLoginExamno=31;
ASP.NET_SessionId=DBYVNKXMKMO2F7XF7T22TM4RALBLKYPBQT42BGZX6H2UA5TZ2ZHCAIBA
Content-Type: application/x-www-form-urlencoded
Content-Length: 485
Host: www.lectio.dk
time=0&__EVENTTARGET=m%24Content%24submitbtn2&__EVENTARGUMENT=&__SCROLLPOSITION=&__VIEWSTATEX=FAAAAGlpZQotMTY2NzgzMzI1ZGQAAAAAALJLI4R47o%2FDmiaj2f9RNeeWl%2B5K&__VIEWSTATEY_KEY=&__VIEWSTATE=&__EVENTVALIDATION=%2FYOJVoNfWl%2F8olHfFefQyPMhqxIUan914QzoN6cFD0eGMukIp%2B%2FciKjpJU04AgXJhME7LttuUGcP8yOJ7xPSSP%2Bl8AnZk2DAdvMIFLFAAPD2Rf4K1DapTvDcnWnyz%2Bj1tesSoSUmLcmmh2E0ljSNR1Qr%2BUpIWQ91RtL1jS4GDh0%3D&m%24Content%24username2=povl0057&m%24Content%24passwordHidden=jdf59jcx&LectioPostbackId=
Shouldn't the server theoretically respond the same way if the post request is the same?
No
I have the following code that successfully connects to a third party API in C#:
using (var client = new WebClient())
{
client.Credentials = new NetworkCredential(login.Username, login.Password);
var xml = client.DownloadString(url);
Debug.Write(xml);
}
This works fine when connecting directly to the API. However, I'm trying to utilize Azure Traffic Manager to spread the load to multiple endpoints, and I'm getting 401 Unauthorized exceptions when doing this. It appears to work correctly using tools like Postman and configuring Basic Auth in the request.
I tried to convert the code to RestSharp but it appears to have the same symptoms.
Here are the request from Fiddler using a few different techniques:
C#/WebClient directly to API endpoint (Success)
GET <ApiUrl> HTTP/1.1
Host: <ApiHost>
Connection: Keep-Alive
401 Unauthorized
GET <ApiUrl> HTTP/1.1
Authorization: Basic <AuthToken>
Host: <ApiHost>
C#/WebClient to Azure Traffic Manager (401 Unauthorized)
GET <TrafficManagerApiUrl> HTTP/1.1
Host: <TrafficManagerApiHost>
401 Unauthorized
GET <ApiUrl> HTTP/1.1
Host: <ApiHost>
Postman to Azure Traffic Manager (Success)
GET <TrafficManagerApiUrl> HTTP/1.1
Host: <TrafficManagerApiHost>
Connection: keep-alive
Authorization: Basic <AuthToken>
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
Postman-Token: 13396800-33ab-8d7b-664f-68b99e8f4ac1
Accept: */*
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
302 Redirect
GET <ApiUrl> HTTP/1.1
Host: <ApiHost>
Connection: keep-alive
Authorization: Basic <AuthToken>
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
Postman-Token: 13396800-33ab-8d7b-664f-68b99e8f4ac1
Accept: */*
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: JSESSIONID=<jsessionid>
I wasn't properly handling the redirect of the Azure Traffic Manager.
The answer is detailed here:
https://stackoverflow.com/a/28671822/86191
When I look in fiddler RAW data, I see about 15 lines of data that goes to the server. I want to play that back exactly like fiddle does when I say replay request. I don't just want to add headers, but want the full request to go back. That is, I want to push something like what I have below (in c#).
Any suggestions?
POST https://www.smith.com/account/signin?returnurl=%2Faccount%2Fsignin%3Freturnurl%3D%252F HTTP/1.1
Host: www.smith.com
Connection: keep-alive
Content-Length: 81
Cache-Control: max-age=0
Origin: https://www.smith.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Referer: https://www.smith.com/account/signin
Accept-Encoding:
...
Context:
Hi everyone, i am trying to simulate a query on this website, but i am failing to do so.
I am using C# and a custom self developed library to Wrap the WebRequests actions making it easier to simulate Posts and Gets for Strings and Bitmaps.
Also, i'm using Fiddler2 Web Debugger to debug the web requests of the service
How to test the service Yourself:
Link to the service
Use this document on the first white box : 04034872000121
Write the captcha and click at "Consultar"
Thats it.
Problem:
After Debuging the requests with fiddler, and replicated everything on code (Cookies, Origin, Host, Postdata with a huge json and so on).
The request for the query, still not working, it redirects me to the home page again, instead of querying the document. (I am allowing "AutoRedirect" on web request object).
The only parameter that i'm not beeing able to replicate is the : GxAjaxRequest: 1
Here is the Fiddler debug feedback of the request:
POST http://sefaznet.ac.gov.br/sefazonline/servlet/hpfsincon?0898a16d81a4e94896958b17b52f252d,gx-no-cache=1354713117196 HTTP/1.1
Host: sefaznet.ac.gov.br
Connection: keep-alive
Content-Length: 1337
Origin: http://sefaznet.ac.gov.br
GxAjaxRequest: 1 **Weird Parameter. I've never saw it before.**
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11
Content-Type: application/x-www-form-urlencoded
Accept: */*
Referer: http://sefaznet.ac.gov.br/sefazonline/servlet/hpfsincon
Accept-Encoding: gzip,deflate,sdch
Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: GX_SESSION_ID=vSLRLKed3eXJGMBorGepVtQkJOQ1I3o0EBUVzT0g%2BI8%3D; JSESSIONID=af2ba968b7889ec8869caaaba281
vNUMDOC=04034872000121&cfield=chin&BUTTON1=Consultar&BTN_VOLTAR=Retornar&GXState=%7B%22_EventName%22%3A%22E'VISUALIZADADOS'.%22%2C%22_EventGridId%22%3A44%2C%22_EventRowId%22%3Aundefined%2C%22nRC_Duplicados%22%3A%220%22%2C%22CAPTCHA1_Reloadimagetext%22%3A%22Obter%20nova%20imagem!%22%2C%22CAPTCHA1_Validationresult%22%3A1%2C%22GX_FocusControl%22%3A%22vNUMDOC%22%2C%22GX_AJAX_KEY%22%3A%2264FFFF0AFF7A4DFF2655FFFFFF26FF77%22%2C%22AJAX_SECURITY_TOKEN%22%3A%221a9634f566dcd40d12bb8146fd7ff6edca12ae737a3743d79b4b826c3bd4a604%22%2C%22GX_CMP_OBJS%22%3A%7B%7D%2C%22sCallerURL%22%3A%22%2Fsefazonline%2Fservlet%2Fhpfsindado%3FeTlFtl5mBgEOtpLCt8Q02bMjmN3K93hV7i2Uxq_rHv0%3D%22%2C%22GX_RES_PROVIDER%22%3A%22com.genexus.webpanels.GXResourceProvider%22%2C%22GX_THEME%22%3A%22GeneXusX%22%2C%22_MODE%22%3A%22%22%2C%22Mode%22%3A%22%22%2C%22IsModified%22%3A%221%22%2C%22MESSAGE_Width%22%3A%22100%22%2C%22MESSAGE_Height%22%3A%22100%22%2C%22MESSAGE_Show%22%3A%22false%22%2C%22MESSAGE_Title%22%3A%22Title%22%2C%22MESSAGE_Message%22%3A%22This%20is%20the%20message%22%2C%22MESSAGE_Type%22%3A%22alert%22%2C%22MESSAGE_Icon%22%3A%22info%22%2C%22MESSAGE_Cls%22%3A%22%22%2C%22MESSAGE_Position%22%3A%22t%22%2C%22MESSAGE_Duration%22%3A1%2C%22MESSAGE_Visible%22%3A1%2C%22CAPTCHA1_Width%22%3A%22140%22%2C%22CAPTCHA1_Height%22%3A%2239%22%2C%22CAPTCHA1_Visible%22%3A1%7D&
Question:
How do i actually replicate/add this parameter to my webrequest via code ?
Is there any way to do so ?
By the way, the site messes alot with scripts which was hard to "figure out" the origin from most parameters used on the requests.
I hope someone might help me out.
Thanks in advance.
I've figured out.
The problem was that i've had to add a custom header to each request.
webRequest.Headers.Add ("customheadertext and value");
Now fiddler shows correctly my new request, with the added header
POST http://sefaznet.ac.gov.br/sefazonline/servlet/hpfsincon?0898a16d81a4e94896958b17b52f252d,gx-no-cache=1354721123208 HTTP/1.1
GxAjaxRequest: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.107 Safari/535.1
Content-Type: application/x-www-form-urlencoded
Referer: http://sefaznet.ac.gov.br/sefazonline/servlet/hpfsincon
Host: sefaznet.ac.gov.br
Cookie: GX_SESSION_ID=B8w8AQ4W%2FLzLHIpBor3JwJDQAWGy1xRqCYUMnzF14Yk%3D; JSESSIONID=c19564cbebfab1911442fd64a0bb
Content-Length: 1291
Expect: 100-continue
Accept-Encoding: gzip, deflate