We Keep Coding

C# Http Request with username

Hello everyone I am currently working on a Post Method of my C# work. As the URL I am going to send request to requires username and password, I have it but I do not know how to include them in my code. How should I insert them into my code?
Here is my request:
Content-Type: multipart/related; boundary="myboundary"
From: 12345678
To: 12345678
Password: my_Password
Username: my_UserName
--myboundary
Content-Type: application/smil
Content-ID: <abc.smil>
Content-Location: abc.smil
Here are my code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://myurl.com");
request.Method = "POST";
request.ContentType = "multipart/related; boundary = myboundary";
WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);

Sharepoint Rest OData Api returns a strange error

I am using the OData Rest Api of Sharepoint 2013 to get and post data in my .Net application. The get works fine but the post throws me a an error. I see this in my fiddler:
<m:error xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<m:code>-1, Microsoft.Data.OData.ODataException</m:code>
<m:message xml:lang="de-DE">Im MIME-Typ fehlt ein Parametername für eine Parameterdefinition.</m:message>
</m:error>
The error message is in german but it means "The Mime-Type is missing a parameter name for a parameter definition"
My send headers are this
POST https://myurl/_api/web/lists(guid'7ED022C7-053F-4057-A141-929DE8AA87A6')/items(3) HTTP/1.1
Accept: application/json;odata=verbose;
X-RequestDigest: 0x0FE934F7228D5D88530162E3E5139D096D2E58E08530600CDF1220B43F7A88B0A4EDE904F5572F4537AFC8383D5FDCD1EA324BF9C3D8F66F9438D58192FA2A85,07 Jun 2017 07:08:45 -0000
Accept-Charset: utf-8
X-HTTP-Method: MERGE
If-Match: *
Content-Type: application/json;odata=verbose;charset=utf-8
Host: intranet.weisseskreuz.bz.it
Content-Length: 54
Expect: 100-continue
{"__metadata":{"type":"SP.ListItem"},"TitleDE":"dddd"}
I use this code to send my post
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(internalUrl);
request.Accept = "application/json;odata=verbose;";
request.Headers.Add("X-RequestDigest", GetFormDigest(baseUrl, credential, cookie));
request.ContentLength=json.Length;
request.Headers.Add("Accept-Charset", "utf-8");
request.Method = "POST";
request.Headers.Add("X-HTTP-Method", "MERGE");
request.Headers.Add("If-Match", "*");
request.ContentType = "application/json;odata=verbose;charset=utf-8";
CredentialCache credentialCache = new CredentialCache();
credentialCache.Add(new Uri(internalUrl), "NTLM", credential);
request.Credentials = credentialCache;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse responseInternal = (HttpWebResponse)request.GetResponse();
Thanks for help I am a little bit desperate.

I've just resolved this type of error. Removed an unnecessary semicolon in the accept header.
Previous:
application/json;odata=verbose;
New:
application/json;odata=verbose

C# .NET - HttpWebRequest - System.Net.WebException - Too many automatic redirections were attempted

I have a problem with HttpWebRequest class.
I am trying to get source code of website:
http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
but I am always getting an error:
System.Net.WebException occurred
HResult=-2146233079
Message=Too many automatic redirections were attempted.
Source=System
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at ProjectName.ClassName.MethodName(String urlAddress)
InnerException:
That is my code:
Uri uri = new Uri(#"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
I have used a Fiddler Web Debugger tool to compare Firefox request with my C# .NET request, but still have no answer.
Firefox:
GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Content-Language: pl-PL
Transfer-Encoding: chunked
Date: Wed, 07 Oct 2015 13:36:31 GMT
X-Cache: HIT from blade110.non.3dart.com
X-Cache-Hits: 116
Server: Apache
C# .NET:
GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci:+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
Connection: Keep-Alive
HTTP/1.1 301 Moved Permanently
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Language: pl-PL
Location: /film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
Content-Length: 0
Accept-Ranges: bytes
Date: Wed, 07 Oct 2015 13:34:51 GMT
X-Cache: MISS from blade712.non.3dart.com
Server: Apache
I have read other posts and update my code by different things, eg.
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.TransferEncoding = "gzip, deflate";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0";
request.Referer = "http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983";
request.KeepAlive = true;
request.AllowAutoRedirect = true;
request.MaximumAutomaticRedirections = 250;
request.Proxy = null;
request.UseDefaultCredentials = true;
CookieContainer cookieContainer = new CookieContainer();
request.CookieContainer = cookieContainer;
but nothing works :-/
Can anybody help me with this problem?

You need to have the initial cookies when the website load before you fetch a deep-link.
The following code works for me:
// cookies
CookieContainer cookieContainer = new CookieContainer();
// make one call to the root of the website
// to get the cookies set
Uri uri = new Uri(#"http://www.filmweb.pl");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using(var s = response.GetResponseStream())
{
using(var sr = new StreamReader(s))
{
// linqpad
sr.ReadToEnd().Dump(); // to check for errors
}
}
// we have cookies now
// do the deep link fetch
uri = new Uri(#"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
response = (HttpWebResponse)request.GetResponse();
//store the result
using(var f = File.Create("C:\\temp\\pl.txt"))
{
response.GetResponseStream().CopyTo(f);
}
Make sure that if you scrape a website that you adhere to their license and usage policies. Don't do anything that goes beyond fair use or against any copy-righted materials.

Reading HTTP file attachment as string

I am using a WebRequest to make a GET and the response includes an attachment.
The attachment is a html file that I want to strip the content out between the tags. I have managed to get the call working with the following code:
string URI = "http://www.sample.com/ReportServer/Pages/ReportViewer.aspx?%2fReports&rs:Command=Render&rs:Format=MHTML&OrganisationID=" + organisationID;
CredentialCache cc = new CredentialCache();
cc.Add(new Uri(URI), "NTLM", new NetworkCredential(userName, userPassword, userDomain));
WebRequest req = WebRequest.Create(URI);
req.Credentials = cc;
WebResponse resp = req.GetResponse();
StreamReader reader = new StreamReader(resp.GetResponseStream());
string response = reader.ReadToEnd().Trim();
The response, when i look in Fiddler is :
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: multipart/related
Expires: Wed, 02 Apr 2014 14:35:15 GMT
Set-Cookie: RSExecutionSession%3a%2fPuborts%2fSecreal%2fClub+Meip+Ret=0yu4f1455xnmznu55; path=/
Server: Microsoft-HTTPAPI/2.0
X-AspNet-Version: 2.0.50727
FileExtension: mhtml
Content-Disposition: attachment; filename="Blah Report.mhtml"
Date: Wed, 02 Apr 2014 14:36:15 GMT
Content-Length: 84215
MIME-Version: 1.0
Content-Type: multipart/related;
boundary="----=_NextPart_01C35DB7.4B204430"
X-MSSQLRS-ProducerVersion: V10.50.4000.0
This is a multi-part message in MIME format.
------=_NextPart_01C35DB7.4B204430
Content-Disposition: inline; filename="Blah Membership Report"
Content-Type: text/html;
name="Club Membership Report";
charset="utf-8"
Content-Transfer-Encoding: base64
PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMDEgVHJhbnNpdGlvbmFs
------=_NextPart_01C35DB7.4B204430--
How can I get hold of just the attachment and read the contents into a string please?

The "attachment" part is just to enforce a save as screen in the browser.
You can use a normal WebClient
var client = new WebClient();
using (client)
{
client.Credentials = blablabla
var result = client.DownloadString("http://blablabla.com");
}

HttpWebRequest and Set-Cookie header in response not parsed (WP7)

I am trying to get the header "Set-Cookie" or access the cookie container, but the Set-Cookie header is not available.
The cookie is in the response header, but it's not there in the client request object.
I am registering the ClientHttp stack using
bool httpResult = WebRequest.RegisterPrefix("http://", WebRequestCreator.ClientHttp);
Here's the response:
HTTP/1.1 200 OK
Content-Type: application/xml; charset=utf-8
Connection: keep-alive
Status: 200
X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.0.pre4
ETag: "39030a9c5a45a24e485e4d2fb06c6389"
Client-Version: 312, 105, 0, 0
X-Runtime: 44
Content-Length: 1232
Set-Cookie: _CWFServer_session=[This is the session data]; path=/; HttpOnly
Cache-Control: private, max-age=0, must-revalidate
Server: nginx/0.7.67 + Phusion Passenger 3.0.0.pre4 (mod_rails/mod_rack)
<?xml version="1.0" encoding="UTF-8"?>
<user>
...
</user>
My callback code contains something like:
var webRequest = (HttpWebRequest)result.AsyncState;
raw = webRequest.EndGetResponse(result) as HttpWebResponse;
foreach (Cookie c in webRequest.CookieContainer.GetCookies(webRequest.RequestUri))
{
Console.WriteLine("Cookie['" + c.Name + "']: " + c.Value);
}
I've also tried looking at the headers but Set-Cookie header isn't present in the response either.
Any suggestions on what may be the problem?

Try explicitly passing a new CookieContainer:
CookieContainer container = new CookieContainer();
container.Add(new Uri("http://yoursite"), new Cookie("name", "value"));
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://yoursite");
request.CookieContainer = container;
request.BeginGetResponse(new AsyncCallback(GetData), request);

You are receiving HttpOnly cookies:
Set-Cookie: _CWFServer_session=[This is the session data]; path=/; HttpOnly
For security reasons, those cookies can't be accessed from code, but you still can use them in your next calls to HttpWebRequest. More on this here : Reading HttpOnly Cookies from Headers of HttpWebResponse in Windows Phone
With WP7.1, I also had problems reading non HttpOnly cookies. I found out that they are not available if the response of the HttpWebRequest comes from the cache. Making the query unique with a random number solved the cache problem :
// The Request
Random random = new Random();
// UniqueQuery is used to defeat the cache system that destroys the cookie.
_uniqueQuery = "http://my-site.somewhere?someparameters=XXX"
+ ";test="+ random.Next();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_uniqueQuery);
request.BeginGetResponse(Response_Completed, request);
Once you get the response, you can fetch the cookie from the response headers:
void Response_Completed(IAsyncResult result)
{
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result);
String header = response.Headers["Set-Cookie"];
I never managed to get the CookieContainer.GetCookies() method to work.

Is the cookie httponly? If so, you won't be able to see it, but if you use the same CookieContainer for your second request, the request will contain the cookie, even though your program won't be able to see it.

You must edit the headers collection directly. Something like this:
request.Headers["Set-Cookie"] = "name=value";
request.BeginGetResponse(myCallback, request);