We Keep Coding

C# Invalid URI: The URI is empty [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
Hello i'm trying to collect all data from website.
On start i collecting all links on that website. After that accessing them.
The problem is when i accessing multiple pages of website the "Invalid URI: The URI is empty" error is showing in some pages. These are some of those failed urls.
http://syayoyu.com/category/zensinyokudiet
http://syayoyu.com/fruitgranola-6759.html
http://syayoyu.com/category/diet/fruitgranola
Accessing from browser and accessing them one by one is working ok. But when i try to access in loop the error is occuring. Would you please tell me what is wrong with it.
This is my accessing code
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.UserAgent = "A .NET Web Crawler";
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string htmlText = reader.ReadToEnd();

List<string> list = new List<string>(){
#"http://syayoyu.com/category/zensinyokudiet",
#"http://syayoyu.com/fruitgranola-6759.html",
#"http://syayoyu.com/category/diet/fruitgranola"
};
foreach (var url in list)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.UserAgent = "A .NET Web Crawler";
WebResponse response = request.GetResponse();
//Stream stream = response.GetResponseStream();
//StreamReader reader = new StreamReader(stream);
//string htmlText = reader.ReadToEnd();
//Console.WriteLine(htmlText);
using (Stream stream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(stream))
{
string htmlText = reader.ReadToEnd();
Console.WriteLine(htmlText);
}
}
}
In my solution, it works fine,so is there any questions you don't know?

Cannot decode JSON returned by HttpWebRequest [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I am trying to decode JSON returned by the website http://wthrcdn.etouch.cn/weather_mini?city=北京. When I look at this URL with Firefox using Unicode encoding, I see the following valid JSON:
{"desc":"OK","status":1000,"data":{"wendu":"20","ganmao":"各项气象条件适宜，发生感冒机率较低。但请避免长期处于空调房间中，以防感冒。","forecast":[{"fengxiang":"无持续风向","fengli":"微风级","high":"高温 27℃","type":"晴","low":"低温 15℃","date":"30日星期四"},{"fengxiang":"南风","fengli":"3-4级","high":"高温 30℃","type":"多云","low":"低温 16℃","date":"1日星期五"},{"fengxiang":"北风","fengli":"3-4级","high":"高温 25℃","type":"小到中雨","low":"低温 14℃","date":"2日星期六"},{"fengxiang":"无持续风向","fengli":"微风级","high":"高温 25℃","type":"多云","low":"低温 14℃","date":"3日星期天"},{"fengxiang":"无持续风向","fengli":"微风级","high":"高温 26℃","type":"多云","low":"低温 13℃","date":"4日星期一"}],"yesterday":{"fl":"微风","fx":"无持续风向","high":"高温 27℃","type":"晴","low":"低温 15℃","date":"29日星期三"},"aqi":"149","city":"北京"}}
But if I try to get and decode this JSON with HttpWebRequest the result is always gibberish:
string http = "http://wthrcdn.etouch.cn/weather_mini?city=北京";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(http); //创建一个请求示例
HttpWebResponse response = (HttpWebResponse)request.GetResponse();　　//获取响应，即发送请求
Stream responseStream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(responseStream, Encoding.UTF8);
string jsonStr = streamReader.ReadToEnd();
Console.WriteLine(jsonStr);
Produces
"�\b\0\0\0\0\0\0\0���JA�_%��`��\"_�� �t5���,�\v̏0�4LRJʲ����]jfֹ�:��-v!�������o��ƑG��ȅ�\ri:�#r�%I�!�1r�QT\rz\"�%��pp�`E�i^b����N�������񜡹�Y��t�n��afw�)�}On�����#��ӭ}~�g�m���5[Ơ�b�}^�'�64!�&/�\r������ªk:r-ƑW\r���8�V:e�\r���gY�/�(�\r����5M���~�2��e�u[S��[:U=���G��\b���\f�\"aw0\a�P$Vj�rM8W*(a�J������i�O`E��i��1���m�B�8��L���Vt'Gw;0n^�pX��޷�c�;\0�$s�j(#�F�4f&�P,��qt�(�dC1U�հ\a���\r|����j����7�=�kb�v���~=6�ĸB��'��L�\0\0"
Decoding with Encoding.Unicode produces:
"謟\b\0\0\0鎭䫝䄂윔╟󉥠⋕ꅟẋ먠琘섵�ጕⰑ௭迌됰䰴䩒닊늈橝홦㨕邃瘭픡컝￙�칯워䞑�藈එ㩩⏖爚◙뉉༡㇖앲呑稍ဢ░灰蜅䕠槳罞鵢캶鏀諸鳱릡如놬赴Ὦꪰ晡ٷ⦯鋲佽鉮췁�폨維�핧洚覤㖑왛鎠뭢幽➺㛝ℴ⚽�鬍蟀슆᮪㩫⵲釆ൗ훺㣽Ϩ嘕攺඙ힳ柦驙蠯밨㗫䴔᳣쮊�鐲旸疙卛㩛唅똽說䜞냷萈ꨌ愢๷〓鄇⑐橖犲㡍⩗愨䪳﷐递뎾榕俇䕠育ᢘㆅ賶涅䋿ꌓ踸䳎隈嚝❴睇〻幮ᢹ瀓ј퇤럞揚뼂;ⓧ끳⡪责㒟♦僑ᨬ톎瑱⢓撱ㅃ퍕냕윇࿄෣ᅼ쎅迺罪趼�蜷ᠽ숅扫皣᳇�㵾밶쐘䊸쒉Ｇ퐃"
How can I decode the JSON from this web site using HttpWebRequest?

The response is compressed so you need to enable automatic decompression. Also, you should wrap your disposables in using statements:
string http = "http://wthrcdn.etouch.cn/weather_mini?city=北京";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(http); //创建一个请求示例
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
using (var response = (HttpWebResponse)request.GetResponse())　　//获取响应，即发送请求
using (var responseStream = response.GetResponseStream())
using (var streamReader = new StreamReader(responseStream, Encoding.UTF8))
{
string jsonStr = streamReader.ReadToEnd();
// Use the JSON string.
// Starts out with {"desc":"OK","status":1000,"data":{"wendu":"20","ganmao":"各项气象条件适宜，...
Console.WriteLine(jsonStr);
}
Note that my console doesn't display Chinese characters. If yours doesn't either, the result of writing to the console will contain many "?" characters.

How to create JSON post to api using C# [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I'm in the process of creating a C# console application which reads text from a text file, turns it into a JSON formatted string (held in a string variable), and needs to POST the JSON request to a web api. I'm using .NET Framework 4.
My struggle is with creating the request and getting the response, using C#. What is the basic code that is necessary? Comments in the code would be helpful. What I've got so far is the below, but I'm not sure if I'm on the right track.
//POST JSON REQUEST TO API
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("POST URL GOES HERE?");
request.Method = "POST";
request.ContentType = "application/json";
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(jsonPOSTString);
request.ContentLength = bytes.Length;
using (Stream requestStream = request.GetRequestStream())
{
// Send the data.
requestStream.Write(bytes, 0, bytes.Length);
}
//RESPONSE HERE

Have you tried using the WebClient class?
you should be able to use
string result = "";
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/json";
result = client.UploadString(url, "POST", json);
}
Console.WriteLine(result);
Documentation at
http://msdn.microsoft.com/en-us/library/system.net.webclient%28v=vs.110%29.aspx
http://msdn.microsoft.com/en-us/library/d0d3595k%28v=vs.110%29.aspx

Try using Web API HttpClient
static async Task RunAsync()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://domain.com/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// HTTP POST
var obj = new MyObject() { Str = "MyString"};
response = await client.PostAsJsonAsync("POST URL GOES HERE?", obj );
if (response.IsSuccessStatusCode)
{
response.//.. Contains the returned content.
}
}
}
You can find more details here Web API Clients

How To Read an HTML File from another url using webrequest using c# [closed]

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
I need to use the below call first. I will get
WebRequest request = WebRequest.Create("https://cli-cert.emdeon.com/servlet/DxLogin?userid={userid}&PW={password}0&hdnBusiness={facilityid}&target=jsp/lab/results/FDC.jsp&apiLogin=true&apiuserid={userid}&actionCommand=startDownload&autoPrint=true&batchDownload=true&reportTypes=LABRES&CreationDateFrom=02/01/2013&CreationDateTo=04/24/2013");
WebResponse resp = request.GetResponse();
Stream responseStream = resp.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
WebRequest request1 = WebRequest.Create("https://cli-cert.emdeon.com/servlet/lab.results.fdcServlet?actionCommand=NextFile&batchDownload=true&fdcuserid={userid}");
WebResponse resp1 = request.GetResponse();
Stream responseStream1 = resp.GetResponseStream();
StreamReader responseReader1 = new StreamReader(responseStream1);
string responseString1 = responseReader1.ReadToEnd();
WebRequest request2 = WebRequest.Create("https://cli-cert.emdeon.com/servlet/lab.results.fdcServlet?actionCommand=GetFileInfo&fdcuserid={userid}");
WebResponse resp2 = request.GetResponse();
Stream responseStream2 = resp.GetResponseStream();
StreamReader responseReader2 = new StreamReader(responseStream2);
string responseString2 = responseReader2.ReadToEnd();
WebRequest request3 = WebRequest.Create("https://cli-cert.emdeon.com/servlet/lab.results.fdcServlet?actionCommand=DownloadFile&mmi=true&fdcuserid={userid}");
WebResponse resp3 = request.GetResponse();
Stream responseStream3 = resp3.GetResponseStream();
StreamReader responseReader3 = new StreamReader(responseStream3);
string responseString3 = responseReader3.ReadToEnd();
WebRequest request4 = WebRequest.Create("https://cli-cert.emdeon.com/servlet/lab.results.fdcServlet?actionCommand=MarkAsDownloaded&fdcuserid={userid}");
WebResponse resp4 = request.GetResponse();
Stream responseStream4 = resp4.GetResponseStream();
StreamReader responseReader4 = new StreamReader(responseStream3);
string responseString4 = responseReader4.ReadToEnd();
resp.Close();
resp1.Close();
resp2.Close();
resp3.Close();
resp4.Close();
Note: {userid},{password},{facilityId} are credentials to login.
I need to call all the 5 request and parse it and save. I need to call each calls by listed order only.
1) First call I am getting the expected output.
2) Call expected is "<--BEGIN NUMBER OF DOCS>2;warning;123345;asdf;sdf;234243;18872_TEST_TC3;<--END NUMBER OF DOCS>" but my stream reader getting empty.
3) all the other calls getting empty
Any suggestion to read and solutions are welcome. I am new bee to development and c#.
Thanks in advance.
Thanks
Rajesh.k

You are using the same responseStream "resp" on all your calls instead of each subsequent resp1, resp2, etc, variables, and the same with response and other vars:
Stream responseStream2 = resp.GetResponseStream();
Check your code before posting, please...

WebRequest to connect to the Wikipedia API

This may be a pathetically simple problem, but I cannot seem to format the post webrequest/response to get data from the Wikipedia API. I have posted my code below if anyone can help me see my problem.
string pgTitle = txtPageTitle.Text;
Uri address = new Uri("http://en.wikipedia.org/w/api.php");
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
string action = "query";
string query = pgTitle;
StringBuilder data = new StringBuilder();
data.Append("action=" + HttpUtility.UrlEncode(action));
data.Append("&query=" + HttpUtility.UrlEncode(query));
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
request.ContentLength = byteData.Length;
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream.
StreamReader reader = new StreamReader(response.GetResponseStream());
divWikiData.InnerText = reader.ReadToEnd();
}

You might want to try a GET request first because it's a little simpler (you will only need to POST for wikipedia login). For example, try to simulate this request:
http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page
Here's the code:
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create("http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page");
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
{
string ResponseText;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
ResponseText = reader.ReadToEnd();
}
}
Edit: The other problem he was experiencing on the POST request was, The exception is : The remote server returned an error: (417) Expectation failed. It can be solved by setting:
System.Net.ServicePointManager.Expect100Continue = false;
(This is from: HTTP POST Returns Error: 417 "Expectation Failed.")

I'm currently in the final stages of implementing an C# MediaWiki API which allows the easy scripting of most MediaWiki viewing and editing actions.
The main API is here: http://o2platform.googlecode.com/svn/trunk/O2%20-%20All%20Active%20Projects/O2_XRules_Database/_Rules/APIs/OwaspAPI.cs and here is an example of the API in use:
var wiki = new O2MediaWikiAPI("http://www.o2platform.com/api.php");
wiki.login(userName, password);
var page = "Test"; // "Main_Page";
wiki.editPage(page,"Test content2");
var rawWikiText = wiki.raw(page);
var htmlText = wiki.html(page);
return rawWikiText.line().line() + htmlText;

You seem to be pushing the input data on HTTP POST, but it seems you should use HTTP GET.
From the MediaWiki API docs:
The API takes its input through
parameters in the query string. Every
module (and every action=query
submodule) has its own set of
parameters, which is listed in the
documentation and in action=help, and
can be retrieved through
action=paraminfo.
http://www.mediawiki.org/wiki/API:Data_formats