We Keep Coding

C# downloading page returns old page

I have problem, i wrote method to get current song on Czech radio. They do not have API so i had to get song from html via html agility.dll
Problem is even though song title changes on page my method downloads old page, usually i have to wait like 20 seconds and have my app closed, then it works.
I thought some cache problem, but i could not fix it.
tried: DownloadString method did not refresh either.
public static string[] GetEV2Songs()
{
List<string> songy = new List<string>();
string urlAddress = "http://www.evropa2.cz/";
string data = "";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
readStream = new StreamReader(receiveStream);
else
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
data = readStream.ReadToEnd();
response.Close();
readStream.Close();
}
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(data);
string temp = "";
foreach (var node in doc.DocumentNode.SelectNodes("//body//h2"))
{
if (node.InnerText.Contains("&ndash"))
{
temp = node.InnerText.Replace("–", "-");
songy.Add(temp);
}
}
return songy.ToArray();
}

Sounds like being a caching problem. Try to replace the 4th line with something like that:
string urlAddress = "http://www.evropa2.cz/?_=" + System.Guid.NewGuid().ToString();

Read data from parse.com does not work

I have this code to read JSON data from the parse.com
protected void Button4_Click(object sender, EventArgs e)
{
string URL = "https://api.parse.com/1/classes/SecondObject";
// string DATA = jsonString;
string text;
var request = WebRequest.Create(URL);
request.Method = "POST";
request.ContentType = "application/json";
request.Headers.Add("X-Parse-Application-Id", "aa");
request.Headers.Add("X-Parse-REST-API-Key", "bb");
WebResponse response = request.GetResponse();
Stream responsestream = response.GetResponseStream();
StreamReader reader = new StreamReader(responsestream);
//IEnumerable<parseo
}
unfortunately i get 400 bad request.
what i want is to read all the json that exist there is my class that called "secondobject".
can anyone help me to figure this issue ?
Thank you.

When you created your object, you got back a full url containing an identifier for your particular instance of SecondObject in the Location header. You need to access that full url.
For example, when you posted your object, you got back the following headers
Status: 201 Created
Location: https://api.parse.com/1/classes/SecondObject/E234jdnsl
Moreover, you also received the following json in the body:
{
"createdAt": [timestamp],
"objectId": "E234jdnsl"
}
That E234jdnsl bit is the object identifier of your particular instance. You then need to point your WebRequest to the url https://api.parse.com/1/classes/SecondObject/E234jdnsl to retrieve it.

As mentioned in the comments, your code is 99% correct. The only problem is you're using a POST command to retrieve data when you should use a GET command instead.
Here's some slightly updated code using a GET command that should work for you. I added some code at the end to read the stream into a StringBuilder and display it in a textbox but you can change that to whatever you need.
private void Button4_Click(object sender, EventArgs e)
{
string URL = "https://api.parse.com/1/classes/SecondObject";
var request = WebRequest.Create(URL);
request.Method = "GET";
request.ContentType = "application/json";
request.Headers.Add("X-Parse-Application-Id", "YOUR_APP_ID");
request.Headers.Add("X-Parse-REST-API-Key", "YOUR_REST_API_KEY");
WebResponse response = request.GetResponse();
Stream responsestream = response.GetResponseStream();
StreamReader reader = new StreamReader(responsestream);
StringBuilder sbTempData = new StringBuilder();
while (reader.Peek() > -1)
{
sbTempData.AppendLine(reader.ReadLine());
}
txtResults.Text = sbTempData.ToString();
}
Note that this will retrieve every object in that class. If you only want to retrieve certain objects in the class, then you can modify your url like https://api.parse.com/1/classes/ClassName/ObjectID
For more information see https://parse.com/docs/rest/guide#queries

I have solve the problem
string URL = "https://api.parse.com/1/classes/SecondObject";
// string DATA = jsonString;
string text;
var request = WebRequest.Create(URL);
request.Method = "GET";
request.ContentType = "application/json";
request.Headers.Add("X-Parse-Application-Id", "aa");
request.Headers.Add("X-Parse-REST-API-Key", "bb");
WebResponse response = request.GetResponse();
Stream responsestream = response.GetResponseStream();
StreamReader reader = new StreamReader(responsestream);
string strResponse = null;
if (!reader.EndOfStream)
{
strResponse = reader.ReadToEnd();
}
Now the strResponse contains all the data that exist in the "SecondObject".
Thank you

Httpwebrequest--POST method using text file

I am trying to replicate a Couch database using .NET classes instead of a curl command line. I have never used WebRequest or Httpwebrequest before, but I am attempting to use them to make a post request with the script below.
Here is the JSON script for couchdb replication(I know this works):
{ ""_id"":"database_replicate8/7/12", "source":sourcedb, ""target"":"targetDB", ""create_target"":true, ""user_ctx"": { ""roles"": ["myrole"] } }
The above script is put into a text file, sourcefile.txt. I want to take this line and put it in a POST web request using .NET functionality.
After looking into it, I chose to use the httpwebrequest class. Below is what I have so far--I got this from http://msdn.microsoft.com/en-us/library/debx8sh9.aspx
HttpWebRequest bob = (HttpWebRequest)WebRequest.Create("sourceDBURL");
bob.Method = "POST";
bob.ContentType = "application/json";
byte[] bytearray = File.ReadAllBytes(#"sourcefile.txt");
Stream datastream = bob.GetRequestStream();
datastream.Write(bytearray, 0, bytearray.Length);
datastream.Close();
Am I going about this correctly? I am relatively new to web technologies and still learning how http calls work.

Here is a method I use for creating POST requests:
private static HttpWebRequest createNewPostRequest(string apikey, string secretkey, string endpoint)
{
HttpWebRequest request = WebRequest.Create(endpoint) as HttpWebRequest;
request.Proxy = null;
request.Method = "POST";
//Specify the xml/Json content types that are acceptable.
request.ContentType = "application/xml";
request.Accept = "application/xml";
//Attach authorization information
request.Headers.Add("Authorization", apikey);
request.Headers.Add("Secretkey", secretkey);
return request;
}
Within my main method I call it like this:
HttpWebRequest request = createNewPostRequest(apikey, secretkey, endpoint);
and I then pass my data to the method like this:
string requestBody = SerializeToString(requestObj);
byte[] byteStr = Encoding.UTF8.GetBytes(requestBody);
request.ContentLength = byteStr.Length;
using (Stream stream = request.GetRequestStream())
{
stream.Write(byteStr, 0, byteStr.Length);
}
//Parse the response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
//Business error
if (response.StatusCode != HttpStatusCode.OK)
{
Console.WriteLine(string.Format("Error: response status code is{0}, at time:{1}", response.StatusCode, DateTime.Now.ToString()));
return "error";
}
else if (response.StatusCode == HttpStatusCode.OK)//Success
{
using (Stream respStream = response.GetResponseStream())
{
XmlSerializer serializer = new XmlSerializer(typeof(SubmitReportResponse));
reportReq = serializer.Deserialize(respStream) as SubmitReportResponse;
}
}
...
In my case I serialize/deserialize my body from a class - you will need to alter this to use your text file. If you want an easy drop in solution, then change the SerializetoString method to a method that loads your text file to a string.

The remote server returned an error: (405) Method Not Allowed. WCF REST Service

This question is already asked elsewhere but those things are not the solutions for my issue.
This is my service
[WebInvoke(UriTemplate = "", Method = "POST")]
public SampleItem Create(SampleItem instance)
{
// TODO: Add the new instance of SampleItem to the collection
// throw new NotImplementedException();
return new SampleItem();
}
I have this code to call the above service
XElement data = new XElement("SampleItem",
new XElement("Id", "2"),
new XElement("StringValue", "sdddsdssd")
);
System.IO.MemoryStream dataSream1 = new MemoryStream();
data.Save(dataSream1);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:2517/Service1/Create");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// You need to know length and it has to be set before you access request stream
request.ContentLength = dataSream1.Length;
using (Stream requestStream = request.GetRequestStream())
{
dataSream1.CopyTo(requestStream);
byte[] bytes = dataSream1.ToArray();
requestStream.Write(bytes, 0, Convert.ToInt16(dataSream1.Length));
requestStream.Close();
}
WebResponse response = request.GetResponse();
I get an exception at the last line:
The remote server returned an error: (405) Method Not Allowed. Not sure why this is happening i tried changing the host from VS Server to IIS also but no change in result. Let me know if u need more information

First thing is to know the exact URL for your REST Service. Since you have specified http://localhost:2517/Service1/Create now just try to open the same URL from IE and you should get method not allowed as your Create Method is defined for WebInvoke and IE does a WebGet.
Now make sure that you have the SampleItem in your client app defined in the same namespace on your server or make sure that the xml string you are building has the appropriate namespace for the service to identify that the xml string of sample object can be deserialized back to the object on server.
I have the SampleItem defined on my server as shown below:
namespace SampleApp
{
public class SampleItem
{
public int Id { get; set; }
public string StringValue { get; set; }
}
}
The xml string corresponding to my SampleItem is as below:
<SampleItem xmlns="http://schemas.datacontract.org/2004/07/SampleApp" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Id>6</Id><StringValue>from client testing</StringValue></SampleItem>
Now i use the below method to perform a POST to the REST service :
private string UseHttpWebApproach<T>(string serviceUrl, string resourceUrl, string method, T requestBody)
{
string responseMessage = null;
var request = WebRequest.Create(string.Concat(serviceUrl, resourceUrl)) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/xml";
request.Method = method;
}
//var objContent = HttpContentExtensions.CreateDataContract(requestBody);
if(method == "POST" && requestBody != null)
{
byte[] requestBodyBytes = ToByteArrayUsingDataContractSer(requestBody);
request.ContentLength = requestBodyBytes.Length;
using (Stream postStream = request.GetRequestStream())
postStream.Write(requestBodyBytes, 0, requestBodyBytes.Length);
}
if (request != null)
{
var response = request.GetResponse() as HttpWebResponse;
if(response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
if (responseStream != null)
{
var reader = new StreamReader(responseStream);
responseMessage = reader.ReadToEnd();
}
}
else
{
responseMessage = response.StatusDescription;
}
}
return responseMessage;
}
private static byte[] ToByteArrayUsingDataContractSer<T>(T requestBody)
{
byte[] bytes = null;
var serializer1 = new DataContractSerializer(typeof(T));
var ms1 = new MemoryStream();
serializer1.WriteObject(ms1, requestBody);
ms1.Position = 0;
var reader = new StreamReader(ms1);
bytes = ms1.ToArray();
return bytes;
}
Now i call the above method as shown:
SampleItem objSample = new SampleItem();
objSample.Id = 7;
objSample.StringValue = "from client testing";
string serviceBaseUrl = "http://localhost:2517/Service1";
string resourceUrl = "/Create";
string method="POST";
UseHttpWebApproach<SampleItem>(serviceBaseUrl, resourceUrl, method, objSample);
I have the SampleItem object defined in the client side as well. If you want to build the xml string on the client and pass then you can use the below method:
private string UseHttpWebApproach(string serviceUrl, string resourceUrl, string method, string xmlRequestBody)
{
string responseMessage = null;
var request = WebRequest.Create(string.Concat(serviceUrl, resourceUrl)) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/xml";
request.Method = method;
}
//var objContent = HttpContentExtensions.CreateDataContract(requestBody);
if(method == "POST" && requestBody != null)
{
byte[] requestBodyBytes = ASCIIEncoding.UTF8.GetBytes(xmlRequestBody.ToString());
request.ContentLength = requestBodyBytes.Length;
using (Stream postStream = request.GetRequestStream())
postStream.Write(requestBodyBytes, 0, requestBodyBytes.Length);
}
if (request != null)
{
var response = request.GetResponse() as HttpWebResponse;
if(response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
if (responseStream != null)
{
var reader = new StreamReader(responseStream);
responseMessage = reader.ReadToEnd();
}
}
else
{
responseMessage = response.StatusDescription;
}
}
return responseMessage;
}
And the call to the above method would be as shown below:
string sample = "<SampleItem xmlns=\"http://schemas.datacontract.org/2004/07/XmlRestService\" xmlns:i=\"http://www.w3.org/2001/XMLSchema-instance\"><Id>6</Id><StringValue>from client testing</StringValue></SampleItem>";
string serviceBaseUrl = "http://localhost:2517/Service1";
string resourceUrl = "/Create";
string method="POST";
UseHttpWebApproach<string>(serviceBaseUrl, resourceUrl, method, sample);
NOTE: Just make sure that your URL is correct

Are you running WCF application for the first time?
run below command to register wcf.
"%WINDIR%\Microsoft.Net\Framework\v3.0\Windows Communication Foundation\ServiceModelReg.exe" -r

After spending 2 days on this, using VS 2010 .NET 4.0, IIS 7.5 WCF and REST with JSON ResponseWrapped, I finally cracked it by reading from "When investigating further..." here https://sites.google.com/site/wcfpandu/useful-links
The Web Service Client code generated file Reference.cs doesn't attribute the GET methods with [WebGet()], so attempts to POST them instead, hence the InvalidProtocol, 405 Method Not Allowed. Problem is though, this file is regenerated when ever you refresh the service reference, and you also need a dll reference to System.ServiceModel.Web, for the WebGet attribute.
So I've decided to manually edit the Reference.cs file, and keep a copy. Next time I refresh it, I'll merge my WebGet()s back in.
The way I see it, it's a bug with svcutil.exe not recognising that some of the service methods are GET and not just POST, even though the WSDL and HELP that the WCF IIS web service publishes, does understand which methods are POST and GET??? I've logged this issue with Microsoft Connect.

When it happened to me, I just simply added the word post
to the function name, and it solved my problem. maybe it will help some of you too.

In the case I came up against, there was yet another cause: the underlying code was attempting to do a WebDAV PUT. (This particular application was configurable to enable this feature if required; the feature was, unbeknownst to me, enabled, but the necessary web server environment was not set up.
Hopefully this may help someone else.

The issue I have fixed, because your service is secured by login credential with user name and password, try set up the user name and password on the request, it will be working. Good luck!

Trying to login to a website clicking a button

I've just started playing around with C# few weeks ago. i got a task i am trying to perform and not sure if the way i do it now is the right approach.
I am trying to login to a website(in my case WordPress website- for lake of better options) and navigating in admin panel using C#
So far what I've done was creating a new project - Windows Form Application.
The following code - send a request to the website with password/username and other parameters as POST
private void button2_Click(object sender, EventArgs e)
{
CookieContainer cookieJar = new CookieContainer();
CookieContainer cookieJar2 = new CookieContainer(); // will use this later
string testaa = ""; // will use this later
string paramaters = "log=xxxx&pwd=xxxx&testcookie=1&redirect_to=http://www.example.com/wp-admin/&wp-submit=Log In";
string strResponse;
HttpWebRequest requestLogin = (HttpWebRequest)WebRequest.Create("http://www.lyndatobin-howes.com/wp-login.php");
requestLogin.Method = "POST";
requestLogin.AllowAutoRedirect = false;
requestLogin.ContentType = "application/x-www-form-urlencoded";
requestLogin.CookieContainer = cookieJar;
requestLogin.ContentLength = paramaters.Length;
StreamWriter stOut = new StreamWriter(requestLogin.GetRequestStream(), Encoding.ASCII);
stOut.Write(paramaters);
stOut.Close();
}
I then have this code to to take the cookie of the response.
HttpWebResponse response = (HttpWebResponse)requestLogin.GetResponse();
foreach (Cookie c in response.Cookies)
{
cookieJar2.Add(new Cookie(c.Name, c.Value, c.Path, c.Domain));
}
then i have this to read the response + close some streams.
StreamReader stIn = new StreamReader(requestLogin.GetResponse().GetResponseStream());
strResponse = stIn.ReadToEnd();
string responseFromServer = stIn.ReadToEnd();
webBrowser1.DocumentText = responseFromServer;
stIn.Close();
response.Close();
And then i try using the above cookie for the page i am trying to access as follows :
HttpWebRequest requestLogin2 = (HttpWebRequest)WebRequest.Create("http://www.example.com/wp-admin/");
requestLogin2.Method = "POST";
requestLogin2.AllowAutoRedirect = false;
requestLogin2.ContentType = "application/x-www-form-urlencoded";
requestLogin2.CookieContainer = cookieJar2;
requestLogin2.ContentLength = paramaters.Length;
StreamWriter stOut2 = new StreamWriter(requestLogin2.GetRequestStream(), System.Text.Encoding.ASCII);
stOut2.Write(paramaters);
stOut2.Close();
StreamReader stIn2 = new StreamReader(requestLogin2.GetResponse().GetResponseStream());
strResponse = stIn2.ReadToEnd();
string responseFromServer2 = stIn2.ReadToEnd();
webBrowser1.DocumentText = responseFromServer2;
richTextBox2.Text += "\n\n\n" + responseFromServer2;
stIn.Close();
Well it doesn't work for some reason I've been trying this for a week now.
I tried displaying the header - after the first request to see what headers i get back. and then looked at the cookie i built (cookieJar2) and it seem they aren't the same..
Anyways any help on the matter would be awesome and highly appreciated. i tried to give as much details as i could.

The first thing I notice about your code is that you call GetResponse() twice in your initial login:
HttpWebResponse response = (HttpWebResponse)requestLogin.GetResponse();
and
StreamReader stIn = new StreamReader(requestLogin.GetResponse().GetResponseStream());
As a result, you're making the login request twice and will be getting back two different cookies. This will be why the cookie in cookieJar2 doesn't match what you're outputting later. Reuse your response object:
StreamReader stIn = new StreamReader(response.GetResponseStream());
Next, when you try to load the admin page, don't POST to it. Your second request should be a GET to retrieve the content of the page.