I need to create Http POST requests and maybe a few GET requests as strings for some tests I am writing. Currently, my tests build them using a StringBuilder and hardcoded POST requests pulled out from fiddler kinda like this:
var builder = new StringBuilder();
builder.Append("POST https://some.web.pg HTTP/1.1\r\n");
builder.Append("Content-Type: application/x-www-form-urlencoded\r\n");
builder.Append("Referer: https://some.referer.com\r\n");
builder.Append("Accept-Language: en-us\r\n");
builder.Append("Accept-Encoding: gzip, deflate\r\n");
builder.Append("User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)\r\n");
builder.Append("Host: login.yahoo.com\r\n");
// ... other header info
builder.Append("\r\n");
builder.Append("post body......\r\n");
var postData = builder.ToString();
This is quickly making my tests messy and would prefer to have a cleaner way to build these POST requests. I've been looking into HttpWebRequest class hoping that maybe it can create these for me. I figured that behind the sences it must have some way to construct this exact request I am trying to creating in some form or another. But alas, the GetRequestStream is a writable only stream.
Is there a way to read the request stream HttpWebRequest will generate (and then change it to a string)? Or even any ideas on how to generate these POST requests would do.
here an msdn sample to make a Get request:
using System;
using System.Net;
using System.IO;
namespace MakeAGETRequest_charp
{
///
/// Summary description for Class1.
///
class Class1
{
static void Main(string[] args)
{
string sURL;
sURL = "http://www.microsoft.com";
WebRequest wrGETURL;
wrGETURL = WebRequest.Create(sURL);
WebProxy myProxy = new WebProxy("myproxy",80);
myProxy.BypassProxyOnLocal = true;
wrGETURL.Proxy = WebProxy.GetDefaultProxy();
Stream objStream;
objStream = wrGETURL.GetResponse().GetResponseStream();
StreamReader objReader = new StreamReader(objStream);
string sLine = "";
int i = 0;
while (sLine!=null)
{
i++;
sLine = objReader.ReadLine();
if (sLine!=null)
Console.WriteLine("{0}:{1}",i,sLine);
}
Console.ReadLine();
}
}
}
and here an for post request (from HTTP request with post)
HttpWebRequest httpWReq =
(HttpWebRequest)WebRequest.Create("http:\\domain.com\page.asp");
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "username=user";
postData += "&password=pass";
byte[] data = encoding.GetBytes(postData);
httpWReq.Method = "POST";
httpWReq.ContentType = "application/x-www-form-urlencoded";
httpWReq.ContentLength = data.Length;
using (Stream newStream = httpWReq.GetRequestStream())
{
newStream.Write(data,0,data.Length);
}
I advise you to use mocking because it is a best practice on unit test:
see this answer on stack
Unit testing HTTP requests in c#
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(yoururllink);
var c = HttpContext.Current;
//Set values for the request back
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
byte[] param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
string strRequest = Encoding.ASCII.GetString(param);
string strResponse_copy = strRequest; //Save a copy of the initial info sent from your url link
strRequest += "&cmd=_notify-validate";
req.ContentLength = strRequest.Length;
//for proxy
//WebProxy proxy = new WebProxy(new Uri("http://url:port#"));
//req.Proxy = proxy;
//Send the request to PayPal and get the response
StreamWriter streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(strRequest);
streamOut.Close();
StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = streamIn.ReadToEnd();
streamIn.Close();
Related
In C # application, when calling API interface, it often takes 15+ seconds to access. API is deployed in another network segment of intranet and needs to be accessed by proxy. Some one said that it was a DNS problem, try to setting the host, which has no effect.
Environment: Windows Server 2012 R2, IIS v8.5
Code Script:
private string PostHttp(string url, string authHeader, string requestBody)
{
var webRequest = System.Net.WebRequest.Create(url);
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(CheckValidationResult);
byte[] data = System.Text.Encoding.UTF8.GetBytes(requestBody);
webRequest.Method = "POST";
webRequest.Headers.Add("Accept-Language", "zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3");
webRequest.Headers.Add("Accept-Encoding", "gzip, deflate");
webRequest.Headers.Add("Authorization", authHeader);
webRequest.ContentLength = data.Length;
webRequest.ContentType = "application/x-www-form-urlencoded";
System.Net.WebProxy proxy = new System.Net.WebProxy("http://myHttpProxyAddress", false);
proxy.Credentials = new System.Net.NetworkCredential("HttpProxyUser", "HttpProxyPassword");
webRequest.Proxy = proxy;
var writer = webRequest.GetRequestStream();
writer.Write(data, 0, data.Length);
writer.Close();
using (WebResponse webResponse = webRequest.GetResponse())
{
System.IO.StreamReader reader = null;
if (webResponse.Headers["Content-Encoding"] == "gzip")
reader = new System.IO.StreamReader(new GZipStream(webResponse.GetResponseStream(), CompressionMode.Decompress), System.Text.Encoding.UTF8);
else
reader = new System.IO.StreamReader(webResponse.GetResponseStream(), System.Text.Encoding.UTF8);
var result = reader.ReadToEnd();
reader.Close();
return result;
}
}
it's very difficult to point out by just hearing someone said one of the possibility (e.g DNS error) , try to use some http utility (e.g Curl) to check and measure that.
you can wrap the request above with below this guide and measure the timing detail.
How do I measure request and response times at once using cURL?
I have a C# code (it's a Web Application, that's hosted on IIS) where I use HttpWebRequest to get HttpWebResponse. There I make request to any website & get the response as string, then I analysis the response string. But recently I get the response where JavaScript performs data fetching after page is loaded in browser.
I tried to debug this in firebug & saw that at bottom of response there's a JavaScript function that updates the dom elements after pageload. Is there any way that I could do the same in my C# code. I have searched on net about this found not solution till now.
Following is the code I am using:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
foreach (Cookie cook in response.Cookies)
{
Cookie cookie = new Cookie();
cookie.Name = cook.Name;
cookie.Value = cook.Value;
cookie.Domain = cook.Domain;
cookie.Expires = DateTime.Now.AddDays(10);
cookieList.Add(cookie);
}
string postData = string.Format("username=" + txtUserID.Text + "&password=" + txtPwd.Text + "&url=https://example.com/&game=");
byte[] postBytes = Encoding.UTF8.GetBytes(postData);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://login.example.com/Login/authenticate");
req.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0";
req.KeepAlive = true;
req.AutomaticDecompression = DecompressionMethods.GZip;
////set the cookie
req.CookieContainer = new CookieContainer();
foreach (Cookie cook in cookieList)
{
Cookie cookie = new Cookie();
cookie.Name = cook.Name;
cookie.Value = cook.Value;
cookie.Domain = cook.Domain;
cookie.Expires = DateTime.Now.AddDays(10);
req.CookieContainer.Add(cookie);
}
req.Headers.Add("Accept-Encoding", "gzip, deflate");
req.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");//en-GB,en-US;q=0.8,en;q=0.6
req.Method = "POST";
req.Host = "login.example.com";
req.Referer = "https://login.example.com/Login/logout";
req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
req.ContentType = "application/x-www-form-urlencoded;";
req.ContentLength = postBytes.Length;
//getting the request stream and posting data
StreamWriter requestwriter = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
requestwriter.Write(postData);
requestwriter.Close();
HttpWebResponse myHttpWebResponse = (HttpWebResponse)req.GetResponse();
Stream responseStream = myHttpWebResponse.GetResponseStream();
StreamReader myStreamReader = new StreamReader(responseStream, Encoding.ASCII);
string responseString = myStreamReader.ReadToEnd();
myStreamReader.Close();
responseStream.Close();
myHttpWebResponse.Close();
I finally got the easy solution to my need. following is the link that I followed:
Link to tutorial
Following is the code that'll get the results:
First you'll need to import following:
using System.Drawing;
using OpenQA.Selenium;
using OpenQA.Selenium.PhantomJS;
using System.Text.RegularExpressions;
using System.IO;
using HtmlAgilityPack;
Now the code:
var options = new PhantomJSOptions();
var driver = new PhantomJSDriver(options);
driver.Manage().Window.Size = new Size(1360, 728);
var size = driver.Manage().Window.Size;
driver.Navigate().GoToUrl("https://example.com/");
string url = driver.Url;
//the driver can now provide you with what you need (it will execute the script)
//get the source of the page
var source = driver.PageSource;
//fully navigate the dom
var pathElement1 = driver.FindElementByName("username");
var pathElement2 = driver.FindElementByName("password");
var pathElement3 = driver.FindElementByXPath("//button[#class='SubmitButton']");
pathElement1.Clear();
pathElement1.SendKeys("username");
pathElement2.Clear();
pathElement2.SendKeys("password");
pathElement3.Click();
//Now get the response after login button click
source = driver.PageSource;
I'am trying to use example api call in below link please check link
http://sendloop.com/help/article/api-001/getting-started
My account is "code5" so i tried 2 codes to get systemDate.
1. Code
var request = WebRequest.Create("http://code5.sendloop.com/api/v3/System.SystemDate.Get/json");
request.ContentType = "application/json; charset=utf-8";
string text;
var response = (HttpWebResponse)request.GetResponse();
using (var sr = new StreamReader(response.GetResponseStream()))
{
text = sr.ReadToEnd();
}
2.Code
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("http://code5.sendloop.com/api/v3/System.SystemDate.Get/json");
httpWebRequest.Method = WebRequestMethods.Http.Get;
httpWebRequest.Accept = "application/json";
But i don't know that i use correctly api by above codes ?
When i use above codes i don't see any data or anything.
How can i get and post api to Sendloop.And how can i use api by using WebRequest ?
I will use api first time in .net so
any help will be appreciated.
Thanks.
It looks like you need to post your API key to the endpoint when making requests. Otherwise, you will not be authenticated and it will return an empty response.
To send a POST request, you will need to do something like this:
var request = WebRequest.Create("http://code5.sendloop.com/api/v3/System.SystemDate.Get/json");
request.ContentType = "application/json; charset=utf-8";
string postData = "APIKey=xxxx-xxxxx-xxxxx-xxxxx-xxxxx";
request.Method = "POST";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
request.ContentLength = data.Length;
Stream newStream = request.GetRequestStream(); //open connection
newStream.Write(data, 0, data.Length); // Send the data.
newStream.Close();
string text;
var response = (HttpWebResponse)request.GetResponse();
using (var sr = new StreamReader(response.GetResponseStream()))
{
text = sr.ReadToEnd();
}
string userAuthenticationURI =
"https://maps.googleapis.com/maps/api/distancematrix/json?origins="+ originZip +
"&destinations="+ DestinationZip + "&units=imperial&language=en-
EN&sensor=false&key=Your API Key";
if (!string.IsNullOrEmpty(userAuthenticationURI))
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(userAuthenticationURI);
request.Method = "GET";
request.ContentType = "application/json";
WebResponse response = request.GetResponse();
var responseString = new
StreamReader(response.GetResponseStream()).ReadToEnd();
dynamic obj = JsonConvert.DeserializeObject(responseString);
}
im trying to send POST request using HttpWebRequest and fiddler shows me that im sending GET?
any help will be appreciated since im able to see what im doing wrong.
code :
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(LOGIN_API_BASE_URL);
string postString = string.Format("api_email={0}&api_password={1}", EMAIL, PASSWORD);
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postString);
CookieContainer cookies = new CookieContainer();
request.CookieContainer = cookies;
//request.AllowWriteStreamBuffering = true;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
request.Timeout = i_timeout;
//request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1";
//request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
//request.Referer = "https://accounts.craigslist.org";
using (Stream writer = request.GetRequestStream())
{
writer.Write(data, 0, data.Length);
//writer.Close();
}
StreamReader responseReader = new StreamReader(request.GetResponse().GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
request.GetResponse().Close();
ok found the problem, the url im sending the request to is http://domain.com and when i send it to http://www.domain.com it works and send a post. so new question why is that? ( the answer will get the answer to my question as i dont like the idea of voting my self )
I have task of uploading file to java servlet. I installed Fiddler to see where web requests are sent and what post data is sent. After logging into java servlet using HttpWebRequest GET method I receive in cookies SessionId. So I was using this SessionId in headers to create POST request to the web server where servlet is. But in response I receive message that "session is time out. Try to login again." But if I use application through user interface I have the one SessionId for all application which is sent in headers with each request.
This application is running in bank so I was thinking if they have some security against scraping.
Am I thinking in a right way? Any help will be appreciated.
Thanks, Elena
Here is my code
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://go.tkygw.pcnet.smbc.local/MgbTokyoGateway/Mgblinkmenu.aspx");
req.Credentials = new NetworkCredential("GB54326", "elena83", "TKYGW");
req.CookieContainer = cookieContainer;
req.Headers.Add("Pragma", "no-cache");
req.Headers.Add("Accept-Language", "en-gb");
req.ProtocolVersion = HttpVersion.Version10;
req.AllowAutoRedirect = true;
WebResponse resp = req.GetResponse();
//here in cookies I receive ASP.NET_session_Id and tkygw_intra
HttpWebResponse webr = (HttpWebResponse)resp;
StreamReader r = new StreamReader(resp.GetResponseStream(), System.Text.Encoding.UTF8);
string res = r.ReadToEnd();
resp.Close();
NameValueCollection nvc = new NameValueCollection();
nvc.Add("_PAGEID", "MWMAL1000P00");
nvc.Add("_SENDTS", "1296208904759");
nvc.Add("_TRANID", "AL1000T00P01");
nvc.Add("_SUBINDEX", "-1");
nvc.Add("_TARGET", "");
nvc.Add("_FRAMID", "");
nvc.Add("_LUID", "1296208904720");
nvc.Add("_WINID", "root");
nvc.Add("_TARGETWINID", "TIMEOUTW_300000_13");
nvc.Add("CHK_FLG", "0");
nvc.Add("BUTTON_NAME", "Corporate Card");
nvc.Add("TITLE_NAME", "[AL1000]Main Menu");
nvc.Add("DateFormat", "1");
nvc.Add("BIZKEY", "AC");
nvc.Add("H_REG_NUM", "");
nvc.Add("H_TODO_DISP_MODE", "");
nvc.Add("H_VIEW_CHANGE_FLG", "1");
nvc.Add("H_SMVA_FLG", "0");
nvc.Add("H_SWITCH_ID", "8837");
nvc.Add("T_BOOKING", "8802");
nvc.Add("T_CUSTOMER_ID", "109732");
nvc.Add("P_DATE_FM", "1");
nvc.Add("D_BA_CREDIT_MONITORING_DISABLED", "");
nvc.Add("D_BA_CREDIT_APPLICATION_DISABLED", "");
nvc.Add("D_BA_CREDIT_APPLICATION_DISABLED", "");
nvc.Add("P_BLANKET_APPLI", "");
HttpWebRequest req3 = (HttpWebRequest)WebRequest.Create("http://gcm.tkygw.pcnet.smbc.local/gcmv0/WACSServlet");
//here in cookiesContainer are 4 values: ASP.NET_session_Id , tkygw_intra
req3.CookieContainer = cookieContainer;
req3.Method = "POST";
req3.Accept = "*/*";
// req3.Headers.Add("Pragma", "no-cache");
// req3.Headers.Add("Accept-Language", "en-gb");
req3.AllowAutoRedirect = true;
req3.KeepAlive = true;
req3.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)";
req3.ContentType = "application/x-www-form-urlencoded";
req3.ProtocolVersion = HttpVersion.Version10;
var sbPostData = new StringBuilder();
if (nvc != null)
{
foreach (string key in nvc.AllKeys)
{
string[] values = nvc.GetValues(key);
if (values != null)
{
foreach (string value in values)
{
if (!string.IsNullOrEmpty(value))
sbPostData.Append(string.Format("{0}={1}&", HttpUtility.UrlEncode(key), HttpUtility.UrlEncode(value)));
}
}
}
}
var parameterString = Encoding.UTF8.GetBytes(sbPostData.ToString());
req3.Referer = "http://gcm.tkygw.pcnet.smbc.local/gcmv0/WACSServlet?_TRANID=AL0010P01C01";
req3.ContentLength = sbPostData.ToString().Length;
using (Stream requestStream = req3.GetRequestStream())
{
requestStream.Write(parameterString, 0, parameterString.Length);
requestStream.Close();
//nothig is received in cookies. Status of response 200 (OK), but on the web page is error that Session is Time OUT. Please Login again
using (var response = req3.GetResponse() as HttpWebResponse)
{
using (var stIn = new System.IO.StreamReader(response.GetResponseStream()))
{
//here I receive session Time Out. Please login again
string s = stIn.ReadToEnd();
}
}
}
If you are uploading file using HttpWebRequest, you must specify content headers correct.
Content must be specified as Content-Type: multipart/form-data
sample snippet
string DataBoundary = "-----------------------------" + DateTime.Now.Ticks.ToString("x");
string contentType = "multipart/form-data; boundary=" + DataBoundary ;
req.Method = "POST";
req.ContentType = contentType ;
req.UserAgent = userAgent;
req.CookieContainer = new CookieContainer();
req.ContentLength = formData.Length;
Check out this and this posts for more detail explanation