We Keep Coding

Bot work in local but not in azure

I'm working on a chat bot that help users of a SharePoint on premise network to upload a file. The bot works on local but returns code 500 when tested in Azure.
I'm using Csom library to navigate in the site tree and the SharePoint _api to get all site collections. I have done some test and I don't think that is the Csom that causes this bug, but rather it is the NetworkCredential that doesn't work in Azure.
So can I use credential in Azure ?
I know that the problem comes from this function
public void GetAllSiteCollections(string url)
{
HttpWebRequest endpointRequest = (HttpWebRequest)HttpWebRequest.Create(url + "/_api/search/query?querytext='contentclass:sts_site'&trimduplicates=false&rowlimit=100");
endpointRequest.Method = "GET";
endpointRequest.Accept = "application/json;odata=verbose";
NetworkCredential cred = new NetworkCredential(Login, Mdp, DomaineUser);
endpointRequest.Credentials = cred;
HttpWebResponse endpointResponse = (HttpWebResponse)endpointRequest.GetResponse();
WebResponse webResponse = endpointRequest.GetResponse();
Stream webStream = webResponse.GetResponseStream();
StreamReader responseReader = new StreamReader(webStream);
string response = responseReader.ReadToEnd();
JObject jobj = JObject.Parse(response);
for (int ind = 0; ind < jobj["d"]["query"]["PrimaryQueryResult"]["RelevantResults"]["Table"]["Rows"]["results"].Count(); ind++)
{
string urlCollection = jobj["d"]["query"]["PrimaryQueryResult"]["RelevantResults"]["Table"]["Rows"]["results"][ind]["Cells"]["results"][6]["Value"].ToString();
string nomCollection = jobj["d"]["query"]["PrimaryQueryResult"]["RelevantResults"]["Table"]["Rows"]["results"][ind]["Cells"]["results"][3]["Value"].ToString();
if (urlCollection.Contains("myLocalDomain/sites/") == true)
{
string[] split = urlCollection.Split('/');
ClientCtx = new ClientContext(Domaine + "/sites/" + split[4]);
using (ClientCtx = new ClientContext(ClientCtx.Url))
{
ClientCtx.Credentials = new NetworkCredential(Login, Mdp, DomaineUser);
Web rootWeb = ClientCtx.Site.RootWeb;
ClientCtx.Load(rootWeb);
BasePermissions bp = new BasePermissions();
bp.Set(PermissionKind.AddListItems);
ClientResult<bool> viewListItems = rootWeb.DoesUserHavePermissions(bp);
ClientCtx.ExecuteQuery();
if (viewListItems.Value)
{
ListDesSiteCollections.Add(nomCollection, split[4]);
}
}
}
}
responseReader.Close();
}
When I check the logs at http://botName.azurewebsites.net/api/messages I get the response "The requested resource does not support http method 'GET'"

Unable to call onesignal api within office but working from external wifi

I am using C# to call OneSignal Create Notifications API. Its working from my home but when i try to send it from my office, it gives a 400 BAD REQUEST.
The error that i get is
The error that comes is Please include a case-sensitive header of
Authorization: Basic <YOUR-REST-API-KEY-HERE> with a valid REST API key."],
"reference":["https://documentation.onesignal.com/docs/accounts-and-keys#section-keys-ids"]
Also, in office network, when i send it from OneSignal Dashboard it works.
Is it a firewall issue?
Tested with POSTMAN and got the same results. POSTMAN working outside office network and giving 400 Bad Request inside office network.
using System.IO;
using System.Net;
using System.Text;
var request = WebRequest.Create("https://onesignal.com/api/v1/notifications") as HttpWebRequest;
request.KeepAlive = true;
request.Method = "POST";
request.ContentType = "application/json; charset=utf-8";
request.Headers.Add("authorization", "Basic NGEwMGZmMjItY2NkNy0xMWUzLTk5ZDUtMDAwYzI5NDBlNjxx");
var serializer = new JavaScriptSerializer();
var obj = new { app_id = "5eb5a37e-b458-11e3-ac11-000c2940e6xx",
contents = new { en = "English Message" },
include_player_ids = new string[] {"all"} };
var param = serializer.Serialize(obj);
byte[] byteArray = Encoding.UTF8.GetBytes(param);
string responseContent = null;
try {
using (var writer = request.GetRequestStream()) {
writer.Write(byteArray, 0, byteArray.Length);
}
using (var response = request.GetResponse() as HttpWebResponse) {
using (var reader = new StreamReader(response.GetResponseStream())) {
responseContent = reader.ReadToEnd();
}
}
}
catch (WebException ex) {
System.Diagnostics.Debug.WriteLine(ex.Message);
System.Diagnostics.Debug.WriteLine(new StreamReader(ex.Response.GetResponseStream()).ReadToEnd());
}
System.Diagnostics.Debug.WriteLine(responseContent);

try following
System.Net.CredentialCache credentialCache = new System.Net.CredentialCache();
credentialCache.Add(
new System.Uri("http://www.yoururl.com/"),
"Basic",
new System.Net.NetworkCredential("username", "password")
);
...
...
httpWebRequest.Credentials = credentialCache;
but i think you should use test RESTFUL tools to check what parameters can return correct value at all first.

The issue is solved by adding the onesignal host to our firewall as a trusted site.

HtmlAgilityPack don't get xpath in c#

before, I use this code, it can get xpath of website. But, today I debug code, I see, it don't get data html from website: webtruyen.com. I try to check website.com/robots.txt. but it don't suspect. And I try to add proxy to get data, but return data null. I don't know how to get xpath from website webtruyen.com. Who help me? I want to know how to read data from website http://webtruyen.com.
My code:
string url = "http://webtruyen.com";
var web = new HtmlWeb();
var doc = web.Load(url);
String temps = "";
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//a"))
{
temps = node.InnerHtml;
}
I debug, return:
InnerHtml 'doc.DocumentNode.InnerHtml' threw an exception of type 'System.NullReferenceException' string {System.NullReferenceException}
My code use proxy:
string url = "http://webtruyen.com";
var web = new HtmlWeb();
webGet.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)";
var doc = web.Load(url);
String temps = "";
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//a"))
{
temps = node.InnerHtml;
}

I have the same error using HtmlWeb.Load(), but I can easily solve your issue using HttpWebRequest (TLDR: See #3 for the working code).
Step 1) Using the following code:
HttpWebRequest hwr = (HttpWebRequest)WebRequest.Create("http://webtruyen.com");
using (Stream s = hwr.GetResponse().GetResponseStream())
{ }
You see that you actually get a 403 Forbidden error (WebException).
Step 2)
HttpWebRequest hwr = (HttpWebRequest)WebRequest.Create("http://webtruyen.com");
HtmlDocument doc = new HtmlDocument();
try
{
using (Stream s = hwr.GetResponse().GetResponseStream())
{ }
}
catch (WebException wx)
{
doc.LoadHtml(new StreamReader(wx.Response.GetResponseStream()).ReadToEnd());
}
on doc.DocumentNode.OuterHtml, you see the HTML of the forbidden error with the JavaScript that sets the cookie on your browser and refreshes it.
3) So in order to load the page outside of a manual browser, you have to manually set that cookie and re-access it. Meaning, with:
string cookie = string.Empty;
HttpWebRequest hwr = (HttpWebRequest)WebRequest.Create("http://webtruyen.com");
try
{
using (Stream s = hwr.GetResponse().GetResponseStream())
{ }
}
catch (WebException wx)
{
cookie = Regex.Match(new StreamReader(wx.Response.GetResponseStream()).ReadToEnd(), "document.cookie = '(.*?)';").Groups[1].Value;
}
hwr = (HttpWebRequest)WebRequest.Create("http://webtruyen.com");
hwr.Headers.Add("Cookie", cookie);
HtmlDocument doc = new HtmlDocument();
using (Stream s = hwr.GetResponse().GetResponseStream())
using (StreamReader sr = new StreamReader(s))
{
doc.LoadHtml(sr.ReadToEnd());
}
You get the page :)
Moral of the story, if your browser can do it, so can you.

Pass in callback url when seeking request

I'm adding twitter integration to my companies web application but I'm hitting a snag on it.
The issue currently is with the callback url.
In twitter it seems that it wants to have a set value but the way our application is setup, each customer that access the site has their own url.
ideally I'd like to set the callback url on the fly, but I'm having a hard time finding any information on that.
EDIT:
Think I may need to add some more details here to better help you help me
The issue is I setup the callback url parameter to https://api.twitter.com/oauth/request_token but when I go to https://api.twitter.com/oauth/authorize to get permission, it will go to the callback setup on the api, not what I set from the previous request.
here is what my code looks like:
public string oAuthToken(string callbackUrl)
{
Uri oauthUrl = new Uri("https://api.twitter.com/oauth/request_token");
var oauthNonce = Convert.ToBase64String(new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
var oauthTimestamp = this.GenerateTimeStamp();
var oauthSignature = "";
var authSignature = string.Format("oauth_callback=\"{0}\"&oauth_consumer_key=\"{1}\"&oauth_nonce=\"{2}\"&oauth_signature_method=\"{3}\"&oauth_timestamp=\"{4}\"&oauth_version=\"{5}\"",
Uri.EscapeDataString(callbackUrl),
Uri.EscapeDataString(this.ConsumerKey),
Uri.EscapeDataString(oauthNonce),
Uri.EscapeDataString(oauthSignatureMethod),
Uri.EscapeDataString(oauthTimestamp),
Uri.EscapeDataString(oauthVersion));
var baseString = string.Format("POST&{0}&{1}", Uri.EscapeDataString(oauthUrl.AbsoluteUri), Uri.EscapeDataString(authSignature));
var compositeKey = string.Concat(Uri.EscapeDataString(this.ConsumerSecret), "&");
using (HMACSHA1 hasher = new HMACSHA1(Encoding.ASCII.GetBytes(compositeKey)))
{
oauthSignature = Convert.ToBase64String(hasher.ComputeHash(Encoding.ASCII.GetBytes(baseString)));
}
var authHeader = string.Format("oauth oauth_callback=\"{0}\", oauth_consumer_key=\"{1}\", oauth_nonce=\"{2}\", oauth_signature=\"{3}\", oauth_signature_method=\"{4}\", oauth_timestamp=\"{5}\", oauth_version=\"{6}\"",
Uri.EscapeDataString(callbackUrl),
Uri.EscapeDataString(this.ConsumerKey),
Uri.EscapeDataString(oauthNonce),
Uri.EscapeDataString(oauthSignature),
Uri.EscapeDataString(oauthSignatureMethod),
Uri.EscapeDataString(oauthTimestamp),
Uri.EscapeDataString(oauthVersion));
ServicePointManager.Expect100Continue = false;
HttpWebRequest authRequest = (HttpWebRequest)System.Net.WebRequest.Create(oauthUrl);
authRequest.Method = "POST";
authRequest.Headers.Add("Authorization", authHeader);
authRequest.Headers.Add("Accept-Encoding", "gzip");
authRequest.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
authRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
try
{
using (var response = authRequest.GetResponse() as HttpWebResponse)
{
if (response.StatusCode != HttpStatusCode.OK)
{
throw new Exception(String.Format("Server error (HTTP {0}: {1}).", response.StatusCode, response.StatusDescription));
}
var stream = response.GetResponseStream();
var sr = new StreamReader(stream);
return sr.ReadToEnd();
}
}
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}
}
}
I always get a 401 error back.

While in the application set-up at https://apps.twitter.com/ a callback URL is required, the OAuth 1.0A spec mandates that a callback URL be passed with the request token. You can pass any valid URL to with that request, effectively having each of your callbacks go to the unique URL for each of your users. So, you're not locked into the callback URL in your application's set-up.

The remote server returned an error: (405) Method Not Allowed. WCF REST Service

This question is already asked elsewhere but those things are not the solutions for my issue.
This is my service
[WebInvoke(UriTemplate = "", Method = "POST")]
public SampleItem Create(SampleItem instance)
{
// TODO: Add the new instance of SampleItem to the collection
// throw new NotImplementedException();
return new SampleItem();
}
I have this code to call the above service
XElement data = new XElement("SampleItem",
new XElement("Id", "2"),
new XElement("StringValue", "sdddsdssd")
);
System.IO.MemoryStream dataSream1 = new MemoryStream();
data.Save(dataSream1);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:2517/Service1/Create");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// You need to know length and it has to be set before you access request stream
request.ContentLength = dataSream1.Length;
using (Stream requestStream = request.GetRequestStream())
{
dataSream1.CopyTo(requestStream);
byte[] bytes = dataSream1.ToArray();
requestStream.Write(bytes, 0, Convert.ToInt16(dataSream1.Length));
requestStream.Close();
}
WebResponse response = request.GetResponse();
I get an exception at the last line:
The remote server returned an error: (405) Method Not Allowed. Not sure why this is happening i tried changing the host from VS Server to IIS also but no change in result. Let me know if u need more information

First thing is to know the exact URL for your REST Service. Since you have specified http://localhost:2517/Service1/Create now just try to open the same URL from IE and you should get method not allowed as your Create Method is defined for WebInvoke and IE does a WebGet.
Now make sure that you have the SampleItem in your client app defined in the same namespace on your server or make sure that the xml string you are building has the appropriate namespace for the service to identify that the xml string of sample object can be deserialized back to the object on server.
I have the SampleItem defined on my server as shown below:
namespace SampleApp
{
public class SampleItem
{
public int Id { get; set; }
public string StringValue { get; set; }
}
}
The xml string corresponding to my SampleItem is as below:
<SampleItem xmlns="http://schemas.datacontract.org/2004/07/SampleApp" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Id>6</Id><StringValue>from client testing</StringValue></SampleItem>
Now i use the below method to perform a POST to the REST service :
private string UseHttpWebApproach<T>(string serviceUrl, string resourceUrl, string method, T requestBody)
{
string responseMessage = null;
var request = WebRequest.Create(string.Concat(serviceUrl, resourceUrl)) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/xml";
request.Method = method;
}
//var objContent = HttpContentExtensions.CreateDataContract(requestBody);
if(method == "POST" && requestBody != null)
{
byte[] requestBodyBytes = ToByteArrayUsingDataContractSer(requestBody);
request.ContentLength = requestBodyBytes.Length;
using (Stream postStream = request.GetRequestStream())
postStream.Write(requestBodyBytes, 0, requestBodyBytes.Length);
}
if (request != null)
{
var response = request.GetResponse() as HttpWebResponse;
if(response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
if (responseStream != null)
{
var reader = new StreamReader(responseStream);
responseMessage = reader.ReadToEnd();
}
}
else
{
responseMessage = response.StatusDescription;
}
}
return responseMessage;
}
private static byte[] ToByteArrayUsingDataContractSer<T>(T requestBody)
{
byte[] bytes = null;
var serializer1 = new DataContractSerializer(typeof(T));
var ms1 = new MemoryStream();
serializer1.WriteObject(ms1, requestBody);
ms1.Position = 0;
var reader = new StreamReader(ms1);
bytes = ms1.ToArray();
return bytes;
}
Now i call the above method as shown:
SampleItem objSample = new SampleItem();
objSample.Id = 7;
objSample.StringValue = "from client testing";
string serviceBaseUrl = "http://localhost:2517/Service1";
string resourceUrl = "/Create";
string method="POST";
UseHttpWebApproach<SampleItem>(serviceBaseUrl, resourceUrl, method, objSample);
I have the SampleItem object defined in the client side as well. If you want to build the xml string on the client and pass then you can use the below method:
private string UseHttpWebApproach(string serviceUrl, string resourceUrl, string method, string xmlRequestBody)
{
string responseMessage = null;
var request = WebRequest.Create(string.Concat(serviceUrl, resourceUrl)) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/xml";
request.Method = method;
}
//var objContent = HttpContentExtensions.CreateDataContract(requestBody);
if(method == "POST" && requestBody != null)
{
byte[] requestBodyBytes = ASCIIEncoding.UTF8.GetBytes(xmlRequestBody.ToString());
request.ContentLength = requestBodyBytes.Length;
using (Stream postStream = request.GetRequestStream())
postStream.Write(requestBodyBytes, 0, requestBodyBytes.Length);
}
if (request != null)
{
var response = request.GetResponse() as HttpWebResponse;
if(response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
if (responseStream != null)
{
var reader = new StreamReader(responseStream);
responseMessage = reader.ReadToEnd();
}
}
else
{
responseMessage = response.StatusDescription;
}
}
return responseMessage;
}
And the call to the above method would be as shown below:
string sample = "<SampleItem xmlns=\"http://schemas.datacontract.org/2004/07/XmlRestService\" xmlns:i=\"http://www.w3.org/2001/XMLSchema-instance\"><Id>6</Id><StringValue>from client testing</StringValue></SampleItem>";
string serviceBaseUrl = "http://localhost:2517/Service1";
string resourceUrl = "/Create";
string method="POST";
UseHttpWebApproach<string>(serviceBaseUrl, resourceUrl, method, sample);
NOTE: Just make sure that your URL is correct

Are you running WCF application for the first time?
run below command to register wcf.
"%WINDIR%\Microsoft.Net\Framework\v3.0\Windows Communication Foundation\ServiceModelReg.exe" -r

After spending 2 days on this, using VS 2010 .NET 4.0, IIS 7.5 WCF and REST with JSON ResponseWrapped, I finally cracked it by reading from "When investigating further..." here https://sites.google.com/site/wcfpandu/useful-links
The Web Service Client code generated file Reference.cs doesn't attribute the GET methods with [WebGet()], so attempts to POST them instead, hence the InvalidProtocol, 405 Method Not Allowed. Problem is though, this file is regenerated when ever you refresh the service reference, and you also need a dll reference to System.ServiceModel.Web, for the WebGet attribute.
So I've decided to manually edit the Reference.cs file, and keep a copy. Next time I refresh it, I'll merge my WebGet()s back in.
The way I see it, it's a bug with svcutil.exe not recognising that some of the service methods are GET and not just POST, even though the WSDL and HELP that the WCF IIS web service publishes, does understand which methods are POST and GET??? I've logged this issue with Microsoft Connect.

When it happened to me, I just simply added the word post
to the function name, and it solved my problem. maybe it will help some of you too.

In the case I came up against, there was yet another cause: the underlying code was attempting to do a WebDAV PUT. (This particular application was configurable to enable this feature if required; the feature was, unbeknownst to me, enabled, but the necessary web server environment was not set up.
Hopefully this may help someone else.

The issue I have fixed, because your service is secured by login credential with user name and password, try set up the user name and password on the request, it will be working. Good luck!