We Keep Coding

Streamreader returns null for a field that has a value

I'm using a streamreader to get the values from a url. It all works fine for 20+ fields, but I have run into an issue with one particular field that shows as having a value but returns null, as highlighted in the below image.
It's not a permission issue as all the other fields come through fine (the above screenshot only shows a cut down version for ease) Also both the web browser and the c# application run as the same user.
string url = "test.com";
WebRequest myReq = WebRequest.Create(url);
string credentials = "username:pwd";
CredentialCache mycache = new CredentialCache();
myReq.Headers["Authorization"] = "Basic " +
Convert.ToBase64String(Encoding.ASCII.GetBytes(credentials));
myReq.Method = "GET";
myReq.ContentType = "application/json";
WebResponse wr = myReq.GetResponse();
Stream receiveStream = wr.GetResponseStream();
StreamReader reader = new StreamReader(receiveStream, Encoding.UTF8);
string content = reader.ReadToEnd();
I'm really stuck for what the reason could be for it being returned as null. Hoping someone can point me in the right direction!

Basic Auth not working when trying to get raw contents from BitBucket?

I'm trying to download file contents from Bit Bucket, but I keep getting a "Log In" page as a response. I'm providing credentials with Basic Auth. Is this a C# specific issue? Everything works fine if I try it via Postman. Code Below.
var url = "https://[BITBUCKET_DOMAIN]/projects/[ID]/repos/[REPO]/raw/[PATH_TO_MY_FILE]"
var uri = new Uri(url);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
string userName = "user";
string password = "pw";
request.PreAuthenticate = true;
request.Credentials = new NetworkCredential(userName, password);
request.Accept = "text/plain";
string result;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
result = reader.ReadToEnd();
}
return result;

This top answer here worked for me.
HttpWebRequest using Basic authentication
String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(userName + ":" + password));
request.Headers.Add("Authorization", "Basic " + encoded);

Send login details with HttpWebRequest

I'm trying to get this bit of code to pause/unpause my XBMC Player
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(baseUrl+playPause);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream resStream = response.GetResponseStream();
Debug.WriteLine(resStream);
And I need to send my login data with it, lets say its MyUsername and SuperPassword.
How would I add that data intp the request? I tried request.Headers.Add("Authorization", "MyUsername SuperPassword"); but I still get the 401 error from it.
EDIT: Nevermind, i figured it out
String username = "MyUsername";
String password = "SuperPassword";
String encoded = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(username + ":" + password));
request.Headers.Add("Authorization", "Basic " + encoded);

There is a awesome post about Send login details with HttpWebRequest and the link is post link
if You feel brother then you can also try one my Best solve this by using webClient class. example:
var url = #"...";
var mybar= new System.Collections.Specialized.NameValueCollection();
mybar.Add("username", "username");
mybar.Add("password", "password");
var client = new System.Net.WebClient();
var data = client.UploadValues(url, mybar);
var res = System.Text.Encoding.ASCII.GetString(data);
Console.WriteLine(res);
Console.ReadLine();

Use Java to read from a REST service

I am trying to figure out how to read from a REST source that requires authentication and having no luck. I have this working fine using C# as follows:
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(filename);
request.Accept = "application/xml";
request.ContentType = "application/xml";
request.KeepAlive = true;
// this part is not used until after a request is refused, but we add it anyways
CredentialCache myCache = new CredentialCache();
myCache.Add(new Uri(filename), "Basic", new NetworkCredential(username, password));
request.Credentials = myCache;
// this is how we put the uname/pw in the first request
string cre = String.Format("{0}:{1}", username, password);
byte[] bytes = Encoding.ASCII.GetBytes(cre);
string base64 = Convert.ToBase64String(bytes);
request.Headers.Add("Authorization", "Basic " + base64);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
return response.GetResponseStream();
But for Java the following is not working:
URL url = new URL(dsInfo.getFilename());
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/xml");
conn.setRequestProperty("Content-Type", "application/xml");
BASE64Encoder encoder = new BASE64Encoder();
String encodedCredential = encoder.encode( (dsInfo.getUsername() + ":" + dsInfo.getPassword()).getBytes() );
conn.setRequestProperty("Authorization", "BASIC " + encodedCredential);
conn.connect();
InputStream responseBodyStream = conn.getInputStream();
The stream is returning:
Error downloading template
Packet: test_packet
Template: NorthwindXml
Error reading authentication header.
What am I getting wrong?
thanks - dave

In your encoding of username/password:
Java uses UTF-8 encoding, and getBytes() returns the bytes corresponding to the local host encoding (who may be or not ASCII). The javadoc of String gives you more detail.
Print the values of such encoded Strings both in c# and Java and check if they match.

The answer came from Codo in a comment - Basic instead of BASIC.

Using .NET to Post a file to Server HttpWebRequest or WebClient

Okay so here is the deal. As the question states, I'm trying to POST a file to a webserver and am having a few issues.
I've tried posting this same file to the same webserver using Curl.exe and have had no issues. I've posted the flags I used with curl just incase they might point out any potential reasons why I'm having trouble with the .NET classes.
curl.exe --user "myUser:myPass" --header "Content-Type: application/gzip"
--data-binary "#filename.txt.gz" --cookie "data=service; data-ver=2; date=20100212;
time=0900; location=1234" --output "out.txt" --dump-header "header.txt"
http://mysite/receive
I'm trying to use a .NET class like WebClient or HttpWebRequest to do the same thing. Here is a sample of the code I've tried. With the WebClient I get a 505 HTTP Version Not Supported error and with the HttpWebRequest I get a 501 Not Implemented.
When trying it with a WebClient:
public void sendFileClient(string path){
string url = "http://mysite/receive";
WebClient wc = new WebClient();
string USERNAME = "myUser";
string PSSWD = "myPass";
NetworkCredential creds = new NetworkCredential(USERNAME, PSSWD);
wc.Credentials = creds;
wc.Headers.Set(HttpRequestHeader.ContentType, "application/gzip");
wc.Headers.Set("Cookie", "location=1234; date=20100226; time=1630; data=service; data-ver=2");
wc.UploadFile(url, "POST", path);
}
And while using a HttpRequest:
public Stream sendFile(string path)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://myserver/receive");
string USERNAME = "myUser";
string PSSWD = "myPass";
NetworkCredential creds = new NetworkCredential(USERNAME, PSSWD);
request.Credentials = creds;
request.Method = "POST";
request.ContentType = "application/gzip";
request.Headers.Set("Cookie", "location=1234; date=20100226; time=1630; data=service; data-ver=2");
FileInfo fInfo = new FileInfo(path);
long numBytes = fInfo.Length;
FileStream fStream = new FileStream(path, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(fStream);
byte[] data = br.ReadBytes((int)numBytes);
br.Close();
fStream.Close();
fStream.Dispose();
Stream wrStream = request.GetRequestStream();
BinaryWriter bw = new BinaryWriter(wrStream);
bw.Write(data);
bw.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
return response.GetResponseStream();
}

First, use something like fiddler and inspect the requests and responses to see what differs between curl and System.Net.WebClient.
Also, you can try (although inspecting with the debugging proxy should allow you to pinpoint the difference):
Use the credential cache to set your credentials for basic authentication:
var cc= new CredentialCache();
cc.Add(new Uri(url),
"Basic",
new NetworkCredential("USERNAME", "PASSWORD"));
wc.Credentials = cc;
Set a user agent header:
string _UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
wc.Headers.Add(HttpRequestHeader.UserAgent, _UserAgent);
Change the protocol version on the WebRequest:
reqeust.KeepAlive = false;
request.ProtocolVersion=HttpVersion.Version10;

There might be another 2 reasons when a 501 accord.
----------1---------
when the postdate contain some Chinese Chracter or some other character.
e.g.
postDate = "type=user&username=计算机学院&password=123&Submit=+登录+"
in order post the right message,you may also add following 2 line;
Request.SendChunked = true;
Request.TransferEncoding = "GB2312";
this also lead to a 501.
in that occasion,you can delete the 2 line,and modify postDate like so.
postDate = "type=user&username=%BC%C6%CB%E3%BB%FA%D1%A7%D4%BA&password=123&Submit=+%C8%B7%C8%CF+"
maybe this is a solution to modify the postDate,however i havn't test yet.
string str = Encoding.GetEncoding("gb2312").GetString(tmpBytes);
----------2---------
if Response.StatusCode == HttpStatusCode.Redirect Redirect is equals to 302.
following line is a must：
Request.AllowAutoRedirect = false;