Downloading html source having SWF content using httpwebrequest or webclient - c#

while downloading the html source code of http://www.orientalcuisines.in/ this site, In result I am getting only script data not whole html content,
I tried using webclient as well as httpwebrequest.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://" + website);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
{
readStream = new StreamReader(receiveStream);
}
else
{
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
}
string data = readStream.ReadToEnd();
response.Close();
readStream.Close();
}
OR
using (WebClient client = new WebClient())
{
client.DownloadFile(website);
}

Related

C# Post-Request Status 400 Bad Request

I am currently working on a C# Programm that is supposed to get Data from a REST API that I host.
The API requires a token for authentication which is returned from a POST request.
When I try to do the POST with C# I get a bad request (Status 400) but the GET request works fine. Now, My question is what I did wrong or what might be the cause for that error. When doing the request with postman both work perfectly.
The POST function:
void POST(string url, string jsonContent) {
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
Byte[] byteArray = encoding.GetBytes(jsonContent);
request.ContentLength = byteArray.Length;
request.ContentType = #"application/json";
using (Stream dataStream = request.GetRequestStream()) {
dataStream.Write(byteArray, 0, byteArray.Length);
}
long length = 0;
try {
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) {
length = response.ContentLength;
Console.WriteLine(response);
}
} catch (WebException ex) {
Console.WriteLine(ex);
}
}
The GET function:
string GET(string url) {
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
try {
WebResponse response = request.GetResponse();
using (Stream responseStream = response.GetResponseStream()) {
StreamReader reader = new StreamReader(responseStream, System.Text.Encoding.UTF8);
return reader.ReadToEnd();
}
} catch (WebException ex) {
WebResponse errorResponse = ex.Response;
using (Stream responseStream = errorResponse.GetResponseStream()) {
StreamReader reader = new StreamReader(responseStream, System.Text.Encoding.GetEncoding("utf-8"));
String errorText = reader.ReadToEnd();
// log errorText
}
throw;
}
}

encoding json using webrequest

i am using this code to get json data
public static string GetAllScores(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
try
{
WebResponse response = request.GetResponse();
using (Stream responseStream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(responseStream, Encoding.UTF8);
return reader.ReadToEnd();
}
}
catch (WebException ex)
{
WebResponse errorResponse = ex.Response;
using (Stream responseStream = errorResponse.GetResponseStream())
{
StreamReader reader = new StreamReader(responseStream, Encoding.UTF8);
String errorText = reader.ReadToEnd();
}
throw;
}
}
the data is coming back and everything ok beside the data returning with é . i try lot of other users solution but nothing so far

How can i get the html of the page? [duplicate]

This question already has answers here:
How to get Url Hash (#) from server side
(6 answers)
Closed 9 years ago.
Here is the code I use to perform a web request. I'm getting all of the HTML except for the comments section in the URL.
HttpWebRequest req = (HttpWebRequest) HttpWebRequest.Create(
"http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d"
);
StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());
htl = reader.ReadToEnd();
Can anyone explain why?
Use this chunk of code. Variable result should have the html code.
System.Net.WebClient webClient = new System.Net.WebClient();
string result = webClient.DownloadString(URL);
Getting HTML code from a website page. You can use code like this.
string urlAddress = "http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
readStream = new StreamReader(receiveStream);
else
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
string data = readStream.ReadToEnd();
response.Close();
readStream.Close();
}
or better to use WebClient
using System.Net;
using (WebClient client = new WebClient())
{
string htmlCode = client.DownloadString("http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d");
}

submit attachment using HttpWebRequest

I'm trying to submit an attachment to a REST API. attachment is not submitted correctly. I believe that i'm doing something wrong with the request
RunQueryimage("http://www.extremetech.com/wp-content/uploads/2012/12/Audi-A1.jpg);
public string RunQueryimage(string imagePath)
{
//do get request
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create("https://iss.ontimenow.com/api/v2/incidents/");
request.ContentType = "application/octet-stream";
request.Method = "POST";
var webClient = new WebClient();
byte[] bytearr = webClient.DownloadData(imagePath);
var filecontent = new ByteArrayContent(bytearr);
// request.ContentLength = 0;
if (filecontent != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(filecontent);
}
}
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
return result;
}
You already have a stream open when you create a web request.
Change this:
byte[] bytearr = webClient.DownloadData(imagePath);
var filecontent = new ByteArrayContent(bytearr);
// request.ContentLength = 0;
if (filecontent != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(filecontent);
}
}
To:
byte[] fileContent = webClient.DownloadData(imagePath);
if (fileContent != null)
{
Stream requestStream = request.GetRequestStream();
requestStream.Write(fileContent, 0, fileContent.Length);
requestStream.Close();
}

Process URL and Get data

I have a URL which returns array of data.
For example if I have the URL http://test.com?id=1 it will return values like 3,5,6,7 etc…
Is there any way that I can process this URL and get the returned values without going to browser (to process the url within the application)?
Thanks
Really easy:
using System.Net;
...
var response = new WebClient().DownloadString("http://test.com?id=1");
string urlAddress = "YOUR URL";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (response.CharacterSet == null)
readStream = new StreamReader(receiveStream);
else
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
string data = readStream.ReadToEnd();
response.Close();
readStream.Close();
}
This should do the trick.
It returns the html source code of the homepage and fills it into the string data.
You can use that string now.
Source: http://www.codeproject.com/Questions/204778/Get-HTML-code-from-a-website-C
This is a simple function I constantly use for similar purposes (VB.NET):
Public Shared Function GetWebData(url As String) As String
Try
Dim request As WebRequest = WebRequest.Create(url)
Dim response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
Dim dataStream As Stream = response.GetResponseStream()
Dim readStream As StreamReader = New StreamReader(dataStream)
Dim data = readStream.ReadToEnd()
readStream.Close()
dataStream.Close()
response.Close()
Return data
Catch ex As Exception
Return ""
End Try
End Function
To use it, pass it the URL and it will return the contents of the URL.

Categories