We Keep Coding

ASP.NET MVC C# - How to add headers in http request

I need to extract data from an URL which returns HTML data. I can do it using Postman I just need the correct syntax to do it in ASP.NET MVC C#.
https://www.dsebd.org/ajax/load-news.php
Headers
accept:text/html, */*; q=0.01
accept-language:en-US,en;q=0.9
content-type:application/x-www-form-urlencoded; charset=UTF-8
sec-fetch-dest:empty
sec-fetch-mode:cors
sec-fetch-site:same-origin
x-requested-with:XMLHttpRequest
And the method is Post.
Can anyone please share me the syntax in ASP.NET MVC C#?

Guys I wanted the correct format. I never asked for the rules of web scraping.
var httpWebRequest = (HttpWebRequest)WebRequest.Create(String.Format(url));
httpWebRequest.Accept = "text/html, */*; q=0.01";
httpWebRequest.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
httpWebRequest.Headers.Add("accept-language", "en-US,en;q=0.9");
httpWebRequest.Headers.Add("sec-fetch-dest", "empty");
httpWebRequest.Headers.Add("sec-fetch-mode", "cors");
httpWebRequest.Headers.Add("sec-fetch-site", "same-origin");
httpWebRequest.Headers.Add("x-requested-with", "XMLHttpRequest");
httpWebRequest.Method = "POST";
This is all I asked. Thanks for your help though.

Why does this request works using HttpWebRequest but not with RestSharp?

I am consuming an API that expects a XML in the body request. First i consumed the api via Postman and it worked, then i used that tool of Postman to convert the request to RestCharp C# code, and then using that code the reponse that i was receiving was different compared to postman. After that, i used Fiddler to generate c# code with the postman request, and using that code that fiddler generated i was able to consume the API via code sucessfully. I am just trying to understand what is the difference between the code generated from postman and the code generated from Fiddler.
This is the code that is generated from Fiddler and it works:
HttpWebRequest request =
(HttpWebRequest)WebRequest.Create("http://x.x.x.x.x");
request.Accept = "*/*";
request.KeepAlive = true;
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
byte[] postBytes = System.Text.Encoding.UTF8.GetBytes(body);
request.ContentLength = postBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(postBytes, 0, postBytes.Length);
stream.Close();
response = (HttpWebResponse)request.GetResponse();
This is the code generated from Postman (slightly altered, but still the code that was generated from postman didn't work and i don't think that the changes that was made interfered with the result) using RestSharp that doesn't work:
var client = new RestClient("http://x.x.x.x.x");
client.ConfigureWebRequest((r) =>
{
r.ServicePoint.Expect100Continue = false;
r.KeepAlive = true;
});
var request = new RestRequest();
request.AddXmlBody(body);
IRestResponse response = client.Post(request);
return response;
I tried a lot of things in the RestSharp code, like adding a header with different content-types and encoding, for example
request.AddHeader("Content-Type", "text/xml;charset=utf-8");
but nothing worked. The response from the api when consumed by the RestSharp code says it got an error of NPE, which i believe it means NullPointerException, but since the api is working just fine via postman and the code generated by Fiddler, i don't think the problem is in the API. Btw, the parameter body in the code are the exact same in both codes.

It looks like the request body is not matching with expected content type by API. When the content does not match the content type expected by the API then you may get NPE error.
in your fiddler generated code you are sending XML string as text.
Please add the following code:
request.AddHeader("Content-Type", "text/plain");
request.AddParameter("undefined", "<YourXml></YourXml>", ParameterType.RequestBody);
or
request.AddHeader("Content-Type", "application/xml");
request.AddParameter("undefined", "<YourXml></YourXml>", ParameterType.RequestBody);
Instead of
request.AddXmlBody(body);

Sending HTTP POST request using WebClient

I have tried unseccessfully to send the next header to the Twitter API:
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic eHZ6MWV2R ... o4OERSZHlPZw==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
I searched and found that there's a method called WebClient.UploadData(this method, implicitly sets HTTP POST as the request method) but I dont really
know how to work with it.
I know how to change the current headers using Set method.
But what about the HTTP body message? how can I add some body to the header?(grant_type)
PS: I read the documention.

Not much to it unless you are dealing with multi-part data. Just build a string with the post data (URL encode if the data requires it), get the bytes, set the content-length, and write your data to the request stream.
string postData = String.Format("field1=value1&field2=value2");
byte[] postBytes = System.Text.ASCIIEncoding.UTF8.GetBytes(postData);
HttpWebRequest req = HttpWebRequest.Create("http://myurl.com");
// ... other request setup stuff
req.ContentLength = postBytes.Length;
using (var stream = req.GetRequestStream())
{
stream.Write(postBytes, 0, postBytes.Length);
}

HTTPWebRequest Body Formatting

This is a stupidly trivial question, but I can't seem to find a proper example anywhere with more than one property being set. Basically, I'm trying to send a POST request with C#'s HTTPWebRequest library while specifying two different fields in the body of the request.
So far, I have this:
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "POST";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] byt = encoding.GetBytes("recipient=12345ABC");
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byt.Length;
Stream newStream = request.GetRequestStream();
newStream.Write(byt, 0, byt.Length);
Followed by the usual GetResponse() stuff. This works fine, everything's dandy, but I can't figure out how to specify multiple body elements, such as both of these:
recipient=12345ABC
body="testmessage"
I've tried separating them with a semicolon, an ampersand, and a comma, but the server keeps returning Error 400: Bad Request. Perhaps I'm just misunderstanding how this process works?

API docs were sloppily done. Actual parameter names were "recipients" and "text" - code worked fine after this change.
URL encoding in the body works fine, "recipients=12345ABC&text=This+is+URL+escaped+text" worked like a charm in either the URL itself or the POST body.

Logging in to eBay using HttpWebRequest fails due to 'The browser you are using is rejecting cookies' response

I'm trying to log in to my eBay account using the following code:
string signInURL = "https://signin.ebay.com/ws/eBayISAPI.dll?co_partnerid=2&siteid=0&UsingSSL=1";
string postData = String.Format("MfcISAPICommand=SignInWelcome&userid={0}&pass={1}", "username", "password");
string contentType = "application/x-www-form-urlencoded";
string method = "POST";
string userAgent = "Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)";
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(signInURL);
req.CookieContainer = cookieContainer;
req.Method = method;
req.ContentType = contentType;
req.UserAgent = userAgent;
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] loginDataBytes = encoding.GetBytes(postData);
req.ContentLength = loginDataBytes.Length;
Stream stream = req.GetRequestStream();
stream.Write(loginDataBytes, 0, loginDataBytes.Length);
stream.Close();
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
StreamReader xsr = new StreamReader(res.GetResponseStream());
String responseText = xsr.ReadToEnd();
Obviously substituting my real username and password. When I look at the string responseText, I see that part of the response from eBay is
The browser you are using is rejecting cookies.
Any ideas what I'm doing wrong?
P.S. And yes, I am also using the eBay API, but this is for something slightly different than what I want to do with the API.

You're doing a direct http request. The Ebay site has functionality to talk to a browser (probably to store the session cookie). Unless you make the request code smart enough to use cookies correctly it won't work. You'll probably have to use the internet explorer object instead.

Before doing the POST you need to download the page with the form that you are submitting in your code, take the cookie they give you, put it in your CookieContainer (making sure you get the path right) and post it back up in your request.
To clarify, while you might be POSTing the correct data, you are not sending the cookie that needs to go with it. You will get this cookie from the login page.

You need to intercept the http traffic to see what exactly what had happened. I use Fiddler2. It is the good tools for debugging http. So I can know whos wrong, my application or the remote web server.
Using fiddler, you can see the request header, response header with its cookies as well as response content. It used in the middle of your app and the Ebay.
Based on my experience. I think it is because Ebay cookie sent to you is not send back to Ebay server. Fiddler will prove it whether yes or not.
Another thing, the response cookie you receive should be send back to next request by using the same CookieContainer.
You should notice that CookieContainer has a bug on .Add(Cookie) and .GetCookies(uri) method. You may not using it, but internal codes might use it.
See the details and fix here:
http://dot-net-expertise.blogspot.com/2009/10/cookiecontainer-domain-handling-bug-fix.html
CallMeLaNN