We Keep Coding

Restful API working in postman, C# not Working [Magento Integration]

I am integrating with Magento 2, using RESTful APIs. When I use postman, it works like charm, while in C# code, it returns "Unauthorized 401" exception.
However, It was working in C# code earlier, but suddenly it stopped working.
I have tried every way, I tried (WebRequest, HTTPClient & RESTsharp) the same exception returned.
Also, I am using Fiddler 4 to catch & match the requests, I used Fiddler to C# plugins to extract C# code, also I used the RESTsharp Code of Postman same exception returned.
The remote server returned an error: (401) Unauthorized.
//Calls request functions sequentially.
private string MakeRequests()
{
HttpWebResponse response;
if (Request_hatolna_co(out response))
{
//Success, possibly uses response.
string responseText = ReadResponse(response);
response.Close();
return responseText;
}
else
{
//Failure, cannot use response.
return "";
}
}
private static string ReadResponse(HttpWebResponse response)
{
using (Stream responseStream = response.GetResponseStream())
{
Stream streamToRead = responseStream;
if (response.ContentEncoding.ToLower().Contains("gzip"))
{
streamToRead = new GZipStream(streamToRead, CompressionMode.Decompress);
}
else if (response.ContentEncoding.ToLower().Contains("deflate"))
{
streamToRead = new DeflateStream(streamToRead, CompressionMode.Decompress);
}
using (StreamReader streamReader = new StreamReader(streamToRead, Encoding.UTF8))
{
return streamReader.ReadToEnd();
}
}
}
private bool Request_hatolna_co(out HttpWebResponse response)
{
response = null;
try
{
//Create a request to URL.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://MAGENTO.co/index.php/rest//V1/orders/items?searchCriteria[filter_groups][0][filters][0][field]=item_id&searchCriteria[filter_groups][0][filters][0][value]=1");
//Set request headers.
request.KeepAlive = true;
request.Headers.Set(HttpRequestHeader.Authorization, "Bearer xxxxxxxxxxxxxxxxxxxxxx");
request.Headers.Add("Postman-Token", #"1181fa03-4dda-ae84-fd31-9d6fbd035614");
request.Headers.Set(HttpRequestHeader.CacheControl, "no-cache");
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36";
request.ContentType = "application/json";
request.Accept = "*/*";
request.Headers.Set(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
request.Headers.Set(HttpRequestHeader.AcceptLanguage, "en-US,en;q=0.9,ar;q=0.8,la;q=0.7");
request.Headers.Set(HttpRequestHeader.Cookie, #"store=default; private_content_version=f16533d4f181d42a1b3f386fa6d2cdf1");
//Get response to request.
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException e)
{
//ProtocolError indicates a valid HTTP response, but with a non-200 status code (e.g. 304 Not Modified, 404 Not Found)
if (e.Status == WebExceptionStatus.ProtocolError) response = (HttpWebResponse)e.Response;
else return false;
}
catch (Exception)
{
if (response != null) response.Close();
return false;
}
return true;
}

Why Postman-Token has been set in C# code? Remove it and then try.

The problem was in the URL, where Magento's (Server's) Admin Changed it to [HTTPS] instead of [HTTP].
That concludes the difference between [Postman, Insomnia, or any other API app] & the C# code, that C# doesn't handle the [HTTP vs HTTPs], while the API app can handle it.

Error submitting login form to external website through c#

I'm currently trying to login to a website and submit a form once logged into the website.
I've tried looking through all the input fields and adding it to the code but still no luck.
The website I'm trying to login to is http://sythe.org/login/
Here's what I currently have:
string formUrl = "http://www.sythe.org/login/";
string formParams = string.Format("login={0}&password={1}&_xfToken={2}&register={3}&remember={4}&cookie_check={5}&redirect={6}", "", "", "", "0", "1", "1", "https://sythe.org");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Console.WriteLine(pageSource);
Thanks guys.

You can try this code which is generated using the C# plugin for Fiddler:
private void MakeRequests()
{
HttpWebResponse response;
if (Request_www_sythe_org(out response))
{
response.Close();
}
}
private bool Request_www_sythe_org(out HttpWebResponse response)
{
response = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.sythe.org/login/login");
request.KeepAlive = true;
request.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0");
request.Headers.Add("Origin", #"https://www.sythe.org");
request.Headers.Add("Upgrade-Insecure-Requests", #"1");
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3";
request.Referer = "https://www.sythe.org/";
request.Headers.Set(HttpRequestHeader.AcceptEncoding, "gzip, deflate, br");
request.Headers.Set(HttpRequestHeader.AcceptLanguage, "en-GB,en-US;q=0.9,en;q=0.8");
request.Headers.Set(HttpRequestHeader.Cookie, #"__cfduid=de8826b5a5df064d6f2ccf2d0952ce52d1554285418; _ga=GA1.2.1744286790.1554285420; _gid=GA1.2.1835584013.1554285420; G_ENABLED_IDPS=google; sythe_bug_token=c0FFSCNAJio4MzJnZWEjWXlaV1NCempSdWFrUzRnUHVZUXJKOENicnkwSmZFTXROR1M3MDhUR1ovWTFIVmlvRFZaaVN2MjFHRWtaM2JhT0N1bTJHbEdGSG5VY1dUNjdmVGNZd1RoekNmTmgzaHB4T29OTTZkNytJQWh3cDdjeWlndWpidkJFY0NiRXF4b1ljSVJhN2VGVVd2eEsvK2hrak1pclhvYjRFeWRhT09UUU5seXZDUVlETDlYWUUyazZGajRub3VWV0k5aVhkVVJiWUtxcFRuaTljRElnWEVKd2o4T3ZSZlRheFEzRWRzekgxSXN4d0doMS9YRFR6L1d0OGRONUVLcDZHUEN3eVFRWEQ%3D; _gat=1; xf_session=63943f38ebba0827a4208d363cae5dcb");
request.Headers.Add("AlexaToolbar-ALX_NS_PH", #"AlexaToolbar/alx-4.0.3");
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
string body = #"login=testtest%40testtest.testtest&register=0&password=testtesttesttest&remember=1&cookie_check=1&redirect=%2F&_xfToken=";
byte[] postBytes = System.Text.Encoding.UTF8.GetBytes(body);
request.ContentLength = postBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(postBytes, 0, postBytes.Length);
stream.Close();
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException e)
{
if (e.Status == WebExceptionStatus.ProtocolError) response = (HttpWebResponse)e.Response;
else return false;
}
catch (Exception)
{
if(response != null) response.Close();
return false;
}
return true;
}

try to replace string formUrl = "http://www.sythe.org/login/"; to string formUrl = "http://www.sythe.org/login/login";

How to get request with JSESSIONID c#?

I've a problem with a C# HTTP GET request.
I send two requests to server:
The first request returns the value of JSESSIONID and it's work:
private string GetSessionId(){
{
string id = null;
try
{
var webRequest = WebRequest.Create("http://www.xxxxxxxxxxxxxxx.it/home") as HttpWebRequest;
var response = "";
webRequest.Date = DateTime.UtcNow;
webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
webRequest.Headers.Add("Accept-Encoding", "gzip, deflate");
webRequest.Headers.Add("Accept-Language", "it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7,de-DE;q=0.6,de;q=0.5");
webRequest.Connection = "keep-alive";
webRequest.Headers.Add("Cache-Control", "max-age=0");
webRequest.Host = "www.xxxxxxxxxxxxxxxx.it";
webRequest.Headers.Add("Upgrade-Insecure-Requests", "1");
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36";
if (webRequest != null)
{
webRequest.Method = "GET";
using (WebResponse resp = webRequest.GetResponse())
{
using (Stream s = resp.GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
{
id = resp.Headers["Set-Cookie"];
response = sr.ReadToEnd();
Console.WriteLine("Response1> " + resp.Headers);
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
return id;
}
The second request should return the server response to my search but it return an error "302 Moved Temporarily" or "Attention the session has expired".
private void RequestFromServer(string url)
{
string session = GetSessionId().Split(';')[0];
Console.WriteLine("Session> {0}",session);
try
{
HttpWebRequest webRequest = WebRequest.Create(url) as HttpWebRequest;
var response = "";
if (webRequest != null)
{
Uri target = new Uri(url);
webRequest.Method = "GET";
webRequest.Timeout = 20000;
webRequest.Date = DateTime.UtcNow;
webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
webRequest.Headers.Add("Accept-Encoding", "gzip, deflate");
webRequest.Headers.Add("Accept-Language", "it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7,de-DE;q=0.6,de;q=0.5");
webRequest.Headers.Add("Cache-Control", "max-age=0");
webRequest.Connection = "keep-alive";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Headers.Add("Cookie", session);
webRequest.Host = "www.xxxxxxxxxxxxxx.it";
webRequest.Headers.Add("Origin", "http://www.xxxxxxxxxxxx.it");
webRequest.Referer = "http://www.xxxxxxxxxxxxxxxx.it/ricerca/atto/contratti/originario?reset=true&normativi=false";
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36";
webRequest.Headers.Add("Upgrade-Insecure-Requests", "1");
var res = webRequest.GetResponse();
using (Stream s = res.GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
{
response = sr.ReadToEnd();
Console.Write(res.Headers);
Console.WriteLine(response);
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Can someone help me with this problem?

What you probably need here is a CookieContainer. Instead of
webRequest.Headers.Add("Cookie", session);
you should do something like
webRequest.CookieContainer = new CookieContainer();
Cookie cookie = new new Cookie("JSESSIONID", yourSessionIdHere);
// adjust domain accordingly
cookie.Domain = "www.xxxxxxxxxxxxxxx.it";
httpRequest.CookieContainer.Add(cookie);
Code is untested.
The MSDN site also states:
For security reasons, cookies are disabled by default. If you want to use cookies, use the CookieContainer property to enable cookies.
That means in order to use cookies you have to set the CookieContainer explicitly.

using HtmlWeb causes HttpWebRequest to timeout

So I've got a situation where I'm using HtmlAgilityPack to load web pages in order to scrape the Document contents. I have a number of URLs that I need to load and a few of them require gzip encoding so I catch the exception thrown by HtmlWeb.load(), check that it's a gzip encoding issue, and then process the page load with HttpWebRequest. However this allows the first time through with HttpWebRequest to be successful, but any other attemp with HttpWebRequest will timeout.
Here's a cleaned up version of the code:
HtmlDocument doc = new HtmlDocument();
HtmlWeb web = new HtmlWeb();
try
{
doc = web.Load(uri);
Console.WriteLine("htmlweb and htmldocument success");
}
catch (ArgumentException ae)
{
Console.WriteLine("htmlweb and htmldocument not successful");
if (ae.Message.Contains("\'gzip\'"))
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(uri);
try
{
req.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
req.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
req.Method = "GET";
//req.UserAgent = "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US))";
string source;
req.KeepAlive = false;
//req.Timeout = 100000;
// On the second iteration we never get beyond this line
using (WebResponse webResponse = req.GetResponse())
{
using (HttpWebResponse httpWebResponse = webResponse as HttpWebResponse)
{
using (StreamReader reader = new StreamReader(httpWebResponse.GetResponseStream()))
{
source = reader.ReadToEnd();
}
}
}
req.Abort();
Console.WriteLine("httpwebresponse successfull");
}
catch (WebException we)
{
Console.WriteLine("httpwebresponse not successful");
}
}
}
Is there some cleanup that I'm needing to do? or is there something I'm forgetting?
Any help will be greatly appreciated.

I think that I will have to load via WebRequest first, instead of HtmlWeb. then inspect the response header for gzip, and decompress as needed each time.
System.Net.HttpWebRequest req = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(uri);
//req.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
//req.AutomaticDecompression = System.Net.DecompressionMethods.Deflate | System.Net.DecompressionMethods.GZip;
//req.Method = "GET";
string source = String.Empty;
try
{
using (System.Net.WebResponse webResponse = req.GetResponse())
{
using (HttpWebResponse httpWebResponse = webResponse as HttpWebResponse)
{
StreamReader reader;
if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
{
reader = new StreamReader(new GZipStream(httpWebResponse.GetResponseStream(), CompressionMode.Decompress));
}
else if (httpWebResponse.ContentEncoding.ToLower().Contains("deflate"))
{
reader = new StreamReader(new DeflateStream(httpWebResponse.GetResponseStream(), CompressionMode.Decompress));
}
else
{
reader = new StreamReader(httpWebResponse.GetResponseStream());
}
source = reader.ReadToEnd();
}
}
req.Abort();
}
catch(Exception ex){
//received a 404 Error - apparently one of my links is now dead...
}

Internal server error on webresponse while posting data but works using a browser C#

I am trying to post a form using HttpWebRequest but get an internal server error. I have tried setting useragent in the request but no success.
Here is the url which has got the form.
https://portal.llg.de/tracking/xml/sender.asp?cust=anonym_
and this is my sample code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
namespace postdata
{
public class RequestManager
{
public static string LastResponse { protected set; get; }
public static void Main(string[] args)
{
System.Net.ServicePointManager.Expect100Continue = false;
string content = "XML=<VORGANG><KUNDE>6400320</KUNDE><LIEFADR>..... Tokio, Japan</LIEFADR><POS NUMMER='1'><ARTNR>9000032</ARTNR><MENGE>10</MENGE><BESTAND>25</BESTAND></POS><POS NUMMER='2'><ARTNR>9161161</ARTNR><MENGE>100</MENGE><BESTAND>33</BESTAND><TERMIN>20050401</TERMIN><PREIS>80.00</PREIS></POS><VERSAND>80.00</VERSAND></VORGANG>";
string url = "https://portal.llg.de/tracking/xml/receiver.asp?cust=_anonym_";
var response = SendPOSTRequest(url, content, true);
var responsecontent = GetResponseContent(response);
}
public static string GetResponseContent(HttpWebResponse response)
{
if (response == null)
{
throw new ArgumentNullException("response");
}
Stream dataStream = null;
StreamReader reader = null;
string responseFromServer = null;
try
{
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
reader = new StreamReader(dataStream);
// Read the content.
responseFromServer = reader.ReadToEnd();
// Cleanup the streams and the response.
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if (reader != null)
{
reader.Close();
}
if (dataStream != null)
{
dataStream.Close();
}
response.Close();
}
LastResponse = responseFromServer;
return responseFromServer;
}
public static HttpWebResponse SendPOSTRequest(string uri, string content, bool allowAutoRedirect)
{
HttpWebRequest request = GeneratePOSTRequest(uri, content, allowAutoRedirect);
return GetResponse(request);
}
public static HttpWebRequest GeneratePOSTRequest(string uri, string content, bool allowAutoRedirect)
{
return GenerateRequest(uri, content, "POST", null, null, allowAutoRedirect);
}
internal static HttpWebRequest GenerateRequest(string uri, string content, string method, string login, string password, bool allowAutoRedirect)
{
if (uri == null)
{
throw new ArgumentNullException("uri");
}
// Create a request using a URL that can receive a post.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
// Set the Method property of the request to POST.
request.Method = method;
// Set cookie container to maintain cookies
// request.CookieContainer = cookies;
request.AllowAutoRedirect = false;
// If login is empty use defaul credentials
if (string.IsNullOrEmpty(login))
{
// request.Credentials = CredentialCache.DefaultNetworkCredentials;
request.Credentials = CredentialCache.DefaultCredentials;
}
else
{
request.Credentials = new NetworkCredential(login, password);
}
if (method == "POST")
{
// Convert POST data to a byte array.
byte[] byteArray = Encoding.UTF8.GetBytes(content);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
}
return request;
}
internal static HttpWebResponse GetResponse(HttpWebRequest request)
{
if (request == null)
{
throw new ArgumentNullException("request");
}
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
Console.WriteLine("Web exception occurred. Status code: {0}", ex.Status);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return response;
}
}
}

I debugged with fiddler both posts.
Something is weird in the form post. The xml conatains "+" between the tags.
If you use this string the post works.
content = "XML=<VORGANG><KUNDE>6400320</KUNDE><LIEFADR>.....+Tokio,+Japan</LIEFADR><POS+NUMMER=\"1\">++<ARTNR>9000032</ARTNR>++<MENGE>10</MENGE>++<BESTAND>25</BESTAND></POS><POS+NUMMER=\"2\">++<ARTNR>9161161</ARTNR>++<MENGE>100</MENGE>++<BESTAND>33</BESTAND>++<TERMIN>20050401</TERMIN>++<PREIS>80.00</PREIS></POS><VERSAND>80.00</VERSAND></VORGANG>";
Fiddler Debug:
POST https://portal.llg.de/tracking/xml/receiver.asp?cust=_anonym_ HTTP/1.1
Host: portal.llg.de
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Referer: https://portal.llg.de/tracking/xml/sender.asp?cust=anonym_
Cookie: ASPSESSIONIDSQSACSTQ=CIGFABLBPHPMLBDOPJIKPHJB
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 588
XML=%3CVORGANG%3E%0D%0A%3CKUNDE%3E6400320%3C%2FKUNDE%3E%0D%0A%3CLIEFADR%3E.....+Tokio%2C+Japan%3C%2FLIEFADR%3E%0D%0A%3CPOS+NUMMER%3D%221%22%3E%0D%0A++%3CARTNR%3E9000032%3C%2FARTNR%3E%0D%0A++%3CMENGE%3E10%3C%2FMENGE%3E%0D%0A++%3CBESTAND%3E25%3C%2FBESTAND%3E%0D%0A%3C%2FPOS%3E%0D%0A%3CPOS+NUMMER%3D%222%22%3E%0D%0A++%3CARTNR%3E9161161%3C%2FARTNR%3E%0D%0A++%3CMENGE%3E100%3C%2FMENGE%3E%0D%0A++%3CBESTAND%3E33%3C%2FBESTAND%3E%0D%0A++%3CTERMIN%3E20050401%3C%2FTERMIN%3E%0D%0A++%3CPREIS%3E80.00%3C%2FPREIS%3E%0D%0A%3C%2FPOS%3E%0D%0A%3CVERSAND%3E80.00%3C%2FVERSAND%3E%0D%0A%3C%2FVORGANG%3E%0D%0A
if you use url decode to decode the xml="..:"
you get this:
XML=<VORGANG>
<KUNDE>6400320</KUNDE>
<LIEFADR>.....+Tokio,+Japan</LIEFADR>
<POS+NUMMER="1">
++<ARTNR>9000032</ARTNR>
++<MENGE>10</MENGE>
++<BESTAND>25</BESTAND>
</POS>
<POS+NUMMER="2">
++<ARTNR>9161161</ARTNR>
++<MENGE>100</MENGE>
++<BESTAND>33</BESTAND>
++<TERMIN>20050401</TERMIN>
++<PREIS>80.00</PREIS>
</POS>
<VERSAND>80.00</VERSAND>
</VORGANG>