The following code should log user in, but it does not work. The code uses 9gag as an example, but the general idea should work elsewhere too. The user does not get access his/hers profile.
What is wrong with the code?
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
namespace ttp_B
{
internal class Program
{
private static void Main(string[] args)
{
bool flag = false;
//string word_to_stop = "profile";
string urltosite = "https://9gag.com/login"; // site that i'm trying to log in
string emailaddress = "";
string password = "";
var coo = new System.Net.CookieContainer();
try
{
var request = System.Net.WebRequest.Create(urltosite) as System.Net.HttpWebRequest;
request.CookieContainer = coo;
request.Method = "POST";
request.Proxy = new WebProxy("127.0.0.1", 8888); // fiddler
//some extra headers
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0";
using (var stream = request.GetRequestStream())
{
byte[] buffer =
Encoding.UTF8.GetBytes(
string.Format(
"csrftoken=&next=http%3A%2F%2F9gag.com%2F&location=1&username={0}&password={1}", emailaddress, password));
//this text of request is correct I guess. Got it from fiddler btw.
stream.Write(buffer, 0, buffer.Length);
stream.Close();
}
using (var response = request.GetResponse() as HttpWebResponse)
{
coo.Add(response.Cookies); // adding cookies, just to make this working properly
using (var sr = new System.IO.StreamReader(response.GetResponseStream()))
{
string http_code = sr.ReadToEnd(); // gettin' that new html document
//flag = (http_code.Contains(word_to_stop)); // looking for word that I'm sure exist after succesfull loggin' in
//if(flag == true)
//{
// console.writeline("Works");
//}
}
}
}
catch (WebException e)
{
Console.Write(e.ToString());
}
}
}
}
As far as I can see, the request stream isn't closed before you go for the response.
simplified it should look like this:
var stream = request.GetRequestStream();
tStream(buffer, 0, buffer.Length);
//close the stream
tStream.Close();
//go for the response
request.GetResponse();
Related
I have a project containing data scraping process from a password protected website. The data scraping process is set to a quartz-scheduler job, which is triggered once an hour. When I deploy the project to web, the process works fine but after 1 hour, the next trigger time, it cannot be executed.
I am sure quartz-scheduler usage is true, because a have another job done with no problem. I think problem is about socket issues which I am not good at.
Here is the code:
using Quartz;
using SMSGonder;
using System;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
namespace TMiddaa.Quartz
{
public class Scout : IJob
{
static string baseUrl = "https://TargetWebSite.com";
static string email = "xx#yy.com";
static string password = "12345";
static string requesttype = "login";
static CookieContainer cookieContainer;
public void Execute(IJobExecutionContext context)
{
cookieContainer = new CookieContainer();
MakeWebRequest();
requesttype = "download";
MakeWebRequest();
}
public void MakeWebRequest()
{
StringBuilder postData = new StringBuilder();
string url = "";
string method = "GET";
if (requesttype == "login")
{
postData.Append(String.Format("user={0}&", email));
postData.Append(String.Format("password={0}&", password));
postData.Append(String.Format("type=&"));
postData.Append(String.Format("remember=1&"));
postData.Append(String.Format("captcha="));
method = "POST";
url = "/ajax/login.ajax.php";
}
else if (requesttype == "download")
{
url = "/somePage";
}
ASCIIEncoding ascii = new ASCIIEncoding();
byte[] postBytes = ascii.GetBytes(postData.ToString());
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(baseUrl + url);
request.ServicePoint.BindIPEndPointDelegate = new BindIPEndPoint(BindIPEndPointCallback);
request.Method = method;
if (method == "POST")
{
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
}
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
request.CookieContainer = cookieContainer;
if (method == "POST")
{
Stream postStream = request.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
postStream.Flush();
postStream.Close();
}
else if (method == "GET")
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
//sayfanın htmlini responseString'ten alabilirsiniz.
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
response.Close();
//Do stuff with responseString
}
}
public int m_LastBindPortUsed = 5000;
public IPEndPoint BindIPEndPointCallback(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
{
int port = Interlocked.Increment(ref m_LastBindPortUsed); //increment
Interlocked.CompareExchange(ref m_LastBindPortUsed, 5001, 65534);
if (remoteEndPoint.AddressFamily == AddressFamily.InterNetwork)
{
return new IPEndPoint(IPAddress.Any, port);
}
else
{
return new IPEndPoint(IPAddress.IPv6Any, port);
}
}
}
}
I get help to create this code, so I dont know some part of this code very well. I suspect the socket part. I will be very happy if someone can help.
Quartz has a logger included. You just need to plug to it to retrieve any exception that might occur in your code
If you're using .Net Core, head to https://www.quartz-scheduler.net/documentation/quartz-3.x/quick-start.html
I need a simple Google Speech API example for .NET in which I upload a voice file and receive the text.
Thanks.
Just worked for me, you can create a class as follow:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
namespace GoogleAPI{
public class GoogleSpeech
{
public string URL { get; set; }
public void GetSpeechTranscript(string filePath)
{
try
{
FileStream fileStream = File.OpenRead(filePath);
MemoryStream memoryStream = new MemoryStream();
memoryStream.SetLength(fileStream.Length);
fileStream.Read(memoryStream.GetBuffer(), 0, (int)fileStream.Length);
byte[] resBytes = memoryStream.GetBuffer();
HttpWebRequest request = null;
request = (HttpWebRequest)HttpWebRequest.Create(this.URL + "&key=YOUR_API_KEY");
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = "POST";
request.ContentType = "audio/x-flac; rate=44100";
request.ContentLength = resBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(resBytes, 0, resBytes.Length);
stream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
StreamReader reqStream = new StreamReader(response.GetResponseStream());
var result = reqStream.ReadToEnd();
Console.WriteLine(result);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
Console.ReadLine();
}
}
}
Hope this help!
i am integrating rightmove real time data feed (rtdf) in my property site for listing my properties on rightmove website. i am using asp.net web api to post data on rightmove listing.
they have provide me with these SSL Files [.p12,.pem,.jks]. i have imported .p12 certificate in my local machine personal store and sending it in my http request
to rightmove test api link provide by rightmove.
i am getting the following error from server.
The remote server returned an error: 403 forbidden.
i checked my certificate loaded successfully in the request, below is my code
public static string PostData(string data, string url)
{
String result = "";
try
{
byte[] bytebuffer = Encoding.UTF8.GetBytes(data);
HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
objRequest.Method = "POST";
objRequest.ContentLength = bytebuffer.Length;
objRequest.ContentType = "application/json";
objRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0";
objRequest.PreAuthenticate = true;
objRequest.Accept = "application/json";
objRequest.ClientCertificates.Add(CertificateHelper.GetRightmoveApiX509Certificate());
using (Stream stream = objRequest.GetRequestStream())
{
stream.Write(bytebuffer, 0, bytebuffer.Length);
stream.Close();
}
HttpWebResponse objResponse = (HttpWebResponse)objRequest.GetResponse();
using (StreamReader streamReader = new StreamReader(objResponse.GetResponseStream()))
{
result = streamReader.ReadToEnd();
// Close and clean up the StreamReader
streamReader.Close();
}
}
catch (Exception e)
{
result = "Exception: " + e.Message;
}
return result;
}
help me to get rid from 403 forbidden error.
Use the following.
I have tested it and it's working fine in my case.
// Grab Certificate
X509Certificate2 cert2 = new X509Certificate2(
AppDomain.CurrentDomain.BaseDirectory + "CertificateName.p12",
CertificatePasswordHere,
X509KeyStorageFlags.MachineKeySet);
var httpWebRequest = (HttpWebRequest)WebRequest.Create("https://adfapi.adftest.rightmove.com/v1/property/sendpropertydetails");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
httpWebRequest.ClientCertificates.Clear();
httpWebRequest.ClientCertificates.Add(cert2);
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(data);
streamWriter.Flush();
streamWriter.Close();
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}
I need have some site manipulation from C# programming language (Microsoft Visual Studio 2010). It is site "http://m.vk.com" . I authorized on this site. It was everything normal. First web-page after authorization it was my profile page. But when I passed to another page this site I loosed my authorization. Another site page was displayed but in non authorization mode. I saved and wrote cookies files to another query. I write next my C# programming code:
I used these directives:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Net;
using System.IO;
using HtmlAgilityPack;
It is main function code which started after main button pushing:
private void button1_Click(object sender, EventArgs e)
{
string login = textBox1.Text;
string pass = textBox2.Text;
bool avt = http_auth_vk(login, pass);
if (avt == true)
{
toolStripStatusLabel1.Text = "Succes authorization !";
}
else
{
toolStripStatusLabel1.Text = "Authorization data incorrect !";
}
}
This is the function "http_auth_vk" :
public bool http_auth_vk(string login, string pass)
{
//*****************************
//Получаем action_url
//*****************************
System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
System.Net.WebRequest reqGET = System.Net.WebRequest.Create("http://m.vk.com/");
System.Net.WebResponse resp = reqGET.GetResponse();
System.IO.Stream stream = resp.GetResponseStream();
System.IO.StreamReader sr = new System.IO.StreamReader(stream);
string s = sr.ReadToEnd();
//*****************************
//Парсим
//*****************************
// Создаём экземпляр класса
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
// Загружаем в класс (парсер) наш html
doc.LoadHtml(s);
// Извлекаем значения
HtmlNode bodyNode = doc.DocumentNode.SelectSingleNode("//div[#class='cont']/form");
// Выводим на экран значиение атрибута src
// у изображения, которое находилось
// в теге <div> в слассом bla
string result1 = bodyNode.Attributes["action"].Value;
//*****************************
//POST запрос
//*****************************
var cookies = new CookieContainer();
ServicePointManager.Expect100Continue = false;
var request = (HttpWebRequest)WebRequest.Create(result1);
request.CookieContainer = cookies;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
using (var requestStream = request.GetRequestStream())
using (var writer = new StreamWriter(requestStream))
{
writer.Write("email=" + login + "&pass=" + pass);
}
using (var responseStream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(responseStream))
{
var result = reader.ReadToEnd();
//*****************************
//Парсим, поиск ID
//*****************************
HtmlAgilityPack.HtmlDocument doc2 = new HtmlAgilityPack.HtmlDocument();
doc2.LoadHtml(result);
string result2;
try
{
//textBox3.Text = result;
HtmlNode bodyNode2 = doc2.DocumentNode.SelectSingleNode("//div[#class='user_wrap']/a");
result2 = bodyNode2.Attributes["href"].Value.Substring(3);
//Если ID найден, то авторизация удалась
//ПЕРЕХОД НА СТРАНИЦУ ГРУППЫ
//string sLocation = myHttpWebResponse.Headers["Location"];
// получам cookie
string sCookies = "";
if (!String.IsNullOrEmpty(resp.Headers["Set-Cookie"]))
{
sCookies = resp.Headers["Set-Cookie"];
}
MessageBox.Show(sCookies);
// формируем запрос
HttpWebRequest myHttpWebRequest = (HttpWebRequest)HttpWebRequest.Create("http://m.vk.com/id100669061");
//myHttpWebRequest.Proxy = new WebProxy("127.0.0.1", 8888);
myHttpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MyIE2;";
myHttpWebRequest.Accept = "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
myHttpWebRequest.Headers.Add("Accept-Language", "ru");
myHttpWebRequest.ContentType = "text/plain";
if (!String.IsNullOrEmpty(sCookies))
{
myHttpWebRequest.Headers.Add(HttpRequestHeader.Cookie, sCookies);
}
// выполняем запрос
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
StreamReader myStreamReader = new StreamReader(myHttpWebResponse.GetResponseStream(), Encoding.GetEncoding(1251));
textBox3.Text = myStreamReader.ReadToEnd();
MessageBox.Show(myStreamReader.ReadToEnd());
//Console.WriteLine(myStreamReader.ReadToEnd());
//Console.ReadKey();
return true;
}
catch
{
//textBox3.Text = result;
//Если ID не науден, то авторизация не удалась
MessageBox.Show("Authorization error !");
return false;
}
}
}
How I can save authorization, after next page pass?
After a little investigation I believe I found your problem. The .NET framework will take care the cookies for you, you don't need to worry about setting the cookie or save the cookie for later use, but you do need to use the same CookieContainer for all requests.
So first set the cookie container you would like to use:
var cookies = new CookieContainer();
You can remove this, as well, all piece of code that uses sCookies.
string sCookies = "";
if (!String.IsNullOrEmpty(resp.Headers["Set-Cookie"]))
{
sCookies = resp.Headers["Set-Cookie"];
}
MessageBox.Show(sCookies);
And in the second/third request instead of having this:
if (!String.IsNullOrEmpty(sCookies))
{
myHttpWebRequest.Headers.Add(HttpRequestHeader.Cookie, sCookies);
}
You need to have this (CookieContainer)
myHttpWebRequest.CookieContainer = cookies;
I am trying to get the result of the following json webservice https://mtgox.com/code/data/getDepth.php into a string using the following code.
using (WebClient client = new WebClient())
{
string data = client.DownloadString("https://mtgox.com/code/data/getDepth.php");
}
but it always returns a timeout exception and no data. I plan to use fastjson to turn the response into objects and expected that to be that hard part not the returning of the content of the page.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://mtgox.com/code/data/getDepth.php");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
string data = sr.ReadToEnd();
}
}
Also resulted in the same error. Can anyone point out what i am doing wrong?
Hmm, strage, this works great for me:
class Program
{
static void Main()
{
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
var result = client.DownloadString("https://mtgox.com/code/data/getDepth.php");
Console.WriteLine(result);
}
}
}
Notice that I am specifying a User Agent HTTP header as it seems that the site is expecting it.
I had similar issue before. request.KeepAlive = false solved my problem. Try this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://mtgox.com/code/data/getDepth.php");
request.KeepAlive = false;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
string data = sr.ReadToEnd();
}
}