I'm trying to perform a query that retrieves information from a CRM Dynamics with the Simple OData Client library like this (C#):
var settings = new ODataClientSettings(resource + "/api/data/v8.0/");
settings.BeforeRequest = (request) =>
{
request.Headers.Clear();
request.Headers.Add("Authorization", accesstoken.AccessTokenType + " " + accesstoken.AccessToken);
};
settings.PayloadFormat = ODataPayloadFormat.Json;
var client = new ODataClient(settings);
var annotations = new ODataFeedAnnotations();
var transactions = await client.For("mss_transaccions").FindEntriesAsync(annotations);
while (annotations.NextPageLink != null)
{
transactions = transactions.Union(await client.For("mss_transaccions").FindEntriesAsync(annotations.NextPageLink, annotations));
}
While this works, it is extremely slow because my query on the mss_transaccions table has 7200 entities. I'm looking at the output in Fiddler and I can see that it is trying to download about 20 MB of information.
I tried to run the same query on Google Chrome, and I could see that by default the obtained response is compressed in gzip format, going from 20MB to some mere 500KB. So I'm deducing that Simple OData Client is not doing any kind of compression, and that's why it is so tremendously slow.
In addition, the request from OData Simple Client asks for metadata information, which adds another 4MB, while Chrome or a simple HttpClient request do not need to make that call.
Is there anything I can do to improve that and enable compression?
Thank you.
In your BeforeRequest action, add the Accept-Encoding header as follows:
settings.BeforeRequest = (request) =>
{
// ... other headers as above
request.Headers.Add("Accept-Encoding", "gzip");
};
I've been able to finally enable compression and speed up the overall process. The whole discussion can be found here: https://github.com/object/Simple.OData.Client/issues/238
To lay it down simply and quickly, you just need to modify the message handler in the ODataSettings instance with the following piece of code:
settings.OnApplyClientHandler = handler =>
{
handler.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
};
Now the httpRequests are sent as gzip,deflate and decompressed on response properly.
If you inspect which headers is sending Chrome and try to replicate them in C#?
Also, If I have to access CRM from C#, I would use the Microsoft.Xrm.Sdk as opposed to OData. You have loads of proxy types and requests that will allow you to write code much more cleaner. OData has other limitations which QueryExpressions / CRM LINQ / FetchXml don't have too.
OData would make more sense for JS code (i.e. requests from a CRM form).
Related
I am breaking my head trying to upload a feed to walmart, after many times trying i used postman to generate C# restsharp code for me, in postman it works, but when using the c# restsharp code it returns a mysterious error. like this:
"No message body writer has been found for response class FeedAcknowledgement"
what does that mean?
here is my code:
string requestUrl = "";
requestUrl = string.Format("https://marketplace.walmartapis.com/v2/feeds?feedType=inventory");
string method = "POST";
// string[] sig = getSig(method, requestUrl).Replace("\r", "").Split('\n');
var mySig = new Signature(ConsumerID, SecretKEY, requestUrl, method);
var s = mySig.TimeStamp;
var returendSigniture = mySig.GetSignature(s);
var client = new RestClient("https://marketplace.walmartapis.com/v2/feeds?feedType=inventory");
var request = new RestRequest(Method.POST);
//request.AddHeader("postman-token", "c325ba5f-813a-f990-7899-6bfc4b14aa1b");
request.AddHeader("cache-control", "no-cache");
request.AddHeader("content-type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
request.AddHeader("accept", "application/xml");
request.AddHeader("wm_consumer.id", "--");
request.AddHeader("wm_sec.auth_signature", returendSigniture);
request.AddHeader("wm_sec.timestamp", mySig.TimeStamp);
request.AddHeader("wm_qos.correlation_id", "123456abcdef");
request.AddHeader("wm_svc.name", "Walmart Marketplace");
request.AddParameter("multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW", "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"BOUNDERY\"\r\n\r\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<wm:inventory xmlns:wm=\"http://walmart.com/\">\n <wm:sku>PP00500-2PC</wm:sku>\n <wm:quantity>\n <wm:unit>EACH</wm:unit>\n <wm:amount>120</wm:amount>\n </wm:quantity>\n <wm:fulfillmentLagTime>1</wm:fulfillmentLagTime>\n</wm:inventory>\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
I spent all day in figuring out how to request Walmart v3. I propose you the following two steps:
Use Walmart signer in order to generate signed token.
You will need to use HttpWebRequest for getting response from Walmart in a way similar to what is described here.
I have not been able to get this to work natively in C#, but I do have a work around.
The Java SDK can successfully submit multi-part requests to Walmart. I wrote a wrapper around the SDK functions that can accept basic command line input to read a text file and send the appropriate call with attached files. From here, you can just call the .jar file (I do it via dynamically generated batch file) from your C# program and receive responses back via text file. This is a sub-optimal system, but it works reliably and when the choice was between updating inventory on 2000 items every day and using some dirty code, I went with the Java wrapper method. This will be replaced as soon as the C# SDK comes out, but I believe this is one of the reasons why the C# SDK may be being delayed.
This solution was used, only after spending about a week trying to get boundaries / streams / attachments to work in C# and having zero success. Cases were also submitted to walmart and I was able to work with some of their top tier engineering support staff and this problem completely stumped them. I was able to trace the Java SDK execution all the way down to a built in Maven / Java function that constructed the web request so there's something under the hood that Java is doing with a multi-part request that isn't immediately clear in C#.
I have started to get some issues where I am over the limit with my headers.
I have a MVC controller, calling a WebApi Controller/Service.
I know the trigger, it is a saml-token (xml) that I've converted to base64.
No, I don't have control of the SecurityToken service...so JWT is not an option at this time. Trust me, I've raised my concerns several times.
We use the saml to secure the WebApi Controller(s) using a custom delegating handler that reads the custom-header and transforms it to a ClaimPrincipal...
I have seen gzip code examples for dealing with the Response, but after hours of googling, I haven't found if there is a way to compress my custom header (or all of them if that's the only way)...for the ~Request.
Ideally I would be able to compress the
"X-My-Custom-Header"
and deal with uncompressing it on the webapi side....
So I'm at a loss to know if this is even possible. This is the first time I've ever had to deal with a way too big header issue.
Sample MVC code below. As an FYI, the windows-credentials are sent over, but that contains the Identity that runs the AppPool that runs the MVC.
My custom header is the saml that is associated with the specific logged in User. Thus why I need to send it over and consider it separately from the windows-identity.
using (var client = new HttpClient(HttpClientHandlerFactory.GetWindowsAuthenticationHttpClientHandler()))
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
string base64SerializedToken = "SuperDuperLongBase64String_IMeanSuperDuper";
client.DefaultRequestHeaders.Add("X-My-Custom-Header", base64SerializedToken);
Uri baseUri = new Uri("http:www.mywebapiservice.com");
Uri destinationUri = new Uri(baseUri, "doSomething");
HttpResponseMessage response = client.PostAsJsonAsync(new Uri(new Uri(this._baseUri), destinationUri.ToString()).ToString(), accountName).Result;
if (response.IsSuccessStatusCode)
{
returnItem = response.Content.ReadAsAsync<MyCustomReturnObject>().Result;
}
else
{
string errorMessage = response.Content.ReadAsStringAsync().Result;
throw new InvalidOperationException(errorMessage);
}
}
public static class HttpClientHandlerFactory
{
public static HttpClientHandler GetWindowsAuthenticationHttpClientHandler()
{
HttpClientHandler returnHandler = new HttpClientHandler()
{
UseDefaultCredentials = true,
PreAuthenticate = true
};
return returnHandler;
}
}
Considering that a SAML token can be several kilobytes in size, depending on the number of claims, sending it as a header is probably a bad idea. Even if you can get it to work now there's no guarantee that it will continue to work if the claim count grows.
Since there is no standard for header compression you will have to modify both ends of the conversation to do something about it. That being the case, why not simply add the SAML token as part of the request body in your API?
If that's really not going to fly (I get that project managers are often painful when it comes to things like this) then you'll have to look into using something like GZipStream to pack the XML, but at some point you're still going to run into problems. This is a bandaid, not a solution.
No, there's no standard for header compression in HTTP. This might have something to do with the fact that you'd need to read the headers to know if (and how) the headers are compressed.
If you don't have a way to decompress whatever manual compression you figure out on the other side, you're out of luck.
I want to call an API called Cloud Sight that provides image recognition.
I want to get a response that basically describes the image from a URL of an image provided from the API Cloud Sight.
This is the code that I Have thus far
var client = new RestClient ("http://api.cloudsightapi.com/image_request");
var request = new RestRequest("http://cdn.head-fi.org/c/c8/1000x500px-c8c39533_beats-by-dre-studio.jpg", Method.POST);
request.AddHeader ("CloudSight", [API KEY HERE]);
IRestResponse response = client.Execute(request);
var content = response.Content;
Console.WriteLine (content);
I get an error that says
{"status":"404","error":"Not Found"}
The documentation for Cloud Sight is not very insightful for each individual language, so I am unsure if I am calling it correctly, particularly, the AddHeader part.
It may also be an error with not waiting for a response. My code executes immediately and the API example that Cloud Sight provides on their website takes 10-15 seconds.
Any ideas for how to go about getting this API working with RestSharp?
Just a guess, but have you tried Method.GET instead of Method.POST? It'd be highly unusual to fetch an image via a POST.
++++++++++++++
Actual Scenario: I am working on a site (ASP.Net using C#) where the system will have 3 different subscription plans i.e. monthly, quarterly and yearly. All the subscription plans have their own cost and pricing. Now, if the system/admin wants to give any discounts to any subscribed user (regardless of subscription plan) on their on-going subscription based on some business logic (for example, for some user it may be $4 and for other it may be $25). How can I achieve this goal. I tried PayPal and Recurly, but stuck in-between.
++++++++++++++
I have to create a coupon and redeem the same using Recurly dynamically in C#. But, as per the code mentioned in "https://docs.recurly.com/api/v1/subscription-plans", we have to use Recurly API v2, but we don't have the code to create and redeem the coupon. So, please help me on how can I create coupons and redeem the same.
When we are using below code in mentioned URL "Recurly PUT request working but returning server error", it causes error while getting response.
uri = "https://" + subdomain + ".recurly.com/v2/subscriptions/" + uuid + "/reactivate";
try
{
string xml = "<subscription><timeframe>now</timeframe></subscription>"; //also tried with blank string.
byte[] arr = System.Text.Encoding.UTF8.GetBytes(xml);
HttpWebRequest renewRequest = (HttpWebRequest)WebRequest.Create(uri);
renewRequest.Headers.Add("Authorization", "Basic " + encodeB64);
renewRequest.Method = "PUT";
renewRequest.ContentType = "text/XML";
renewRequest.ContentLength = arr.Length;
Stream datastream = renewRequest.GetRequestStream();
datastream.Write(arr, 0, arr.Length);
datastream.Close();
HttpWebResponse renewResponse = (HttpWebResponse)renewRequest.GetResponse();
}
Looking for kind response and help...
We (recurly.com) just made available a release candidate of an all new API client for C# compatible with Recurly APIv2 that we highly recommend using. The client API is stable and this release will shortly become the final release pending new show-stopping bugs.
Here's how to get started using it.
Be sure set up your configuration.
Here's how to create a coupon.
Here's how to redeem coupons.
More examples are available here.
If you have further questions please don't hesitate to ask our support team! support#recurly.com.
When using the System.Net.WebClient.DownloadData() method I'm getting an unreasonably slow response time.
When fetching an url using the WebClient class in .NET it takes around 10 sec before I get a response, while the same page is fetched by my browser in under 1 sec.
And this is with data that's 0.5kB or smaller in size.
The request involves POST/GET parameters and a user agent header if perhaps that could cause problems.
I haven't (yet) tried if other ways to download data in .NET gives me the same problems, but I'm suspecting I might get similar results. (I've always had a feeling web requests in .NET are unusually slow...)
What could be the cause of this?
Edit:
I tried doing the exact thing using System.Net.HttpWebRequest instead, using the following method, and all requests finish in under 1 sec.
public static string DownloadText(string url)
var request = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)request.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
While this (old) method using System.Net.WebClient takes 15-30s for each request to finish:
public static string DownloadText(string url)
{
var client = new WebClient();
byte[] data = client.DownloadData(url);
return client.Encoding.GetString(data);
}
I had that problem with WebRequest. Try setting Proxy = null;
WebClient wc = new WebClient();
wc.Proxy = null;
By default WebClient, WebRequest try to determine what proxy to use from IE settings, sometimes it results in like 5 sec delay before the actual request is sent.
This applies to all classes that use WebRequest, including WCF services with HTTP binding.
In general you can use this static code at application startup:
WebRequest.DefaultWebProxy = null;
Download Wireshark here http://www.wireshark.org/
Capture the network packets and filter the "http" packets.
It should give you the answer right away.
There is nothing inherently slow about .NET web requests; that code should be fine. I regularly use WebClient and it works very quickly.
How big is the payload in each direction? Silly question maybe, but is it simply bandwidth limitations?
IMO the most likely thing is that your web-site has spun down, and when you hit the URL the web-site is slow to respond. This is then not the fault of the client. It is also possible that DNS is slow for some reason (in which case you could hard-code the IP into your "hosts" file), or that some proxy server in the middle is slow.
If the web-site isn't yours, it is also possible that they are detecting atypical usage and deliberately injecting a delay to annoy scrapers.
I would grab Fiddler (a free, simple web inspector) and look at the timings.
WebClient may be slow on some workstations when Automatic Proxy Settings in checked in the IE settings (Connections tab - LAN Settings).
Setting WebRequest.DefaultWebProxy = null; or client.Proxy = null didn't do anything for me, using Xamarin on iOS.
I did two things to fix this:
I wrote a downloadString function which does not use WebRequest and System.Net:
public static async Task<string> FnDownloadStringWithoutWebRequest(string url)
{
using (var client = new HttpClient())
{
//Define Headers
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response = await client.GetAsync(url);
if (response.IsSuccessStatusCode)
{
string responseContent = await response.Content.ReadAsStringAsync();
//dynamic json = Newtonsoft.Json.JsonConvert.DeserializeObject(responseContent);
return responseContent;
}
Logger.DefaultLogger.LogError(LogLevel.NORMAL, "GoogleLoginManager.FnDownloadString", "error fetching string, code: " + response.StatusCode);
return "";
}
}
This is however still slow with Managed HttpClient.
So secondly, in Visual Studio Community for Mac, right click on your Project in the Solution -> Options -> set HttpClient implementation to NSUrlSession, instead of Managed.
Screenshot: Set HttpClient implementation to NSUrlSession instead of Managed
Managed is not fully integrated into iOS, doesn't support TLS 1.2, and thus does not support the ATS standards set as default in iOS9+, see here:
https://learn.microsoft.com/en-us/xamarin/ios/app-fundamentals/ats
With both these changes, string downloads are always very fast (<<1s).
Without both of these changes, on every second or third try, downloadString took over a minute.
Just FYI, there's one more thing you could try, though it shouldn't be necessary anymore:
//var authgoogle = new OAuth2Authenticator(...);
//authgoogle.Completed...
if (authgoogle.IsUsingNativeUI)
{
// Step 2.1 Creating Login UI
// In order to access SFSafariViewController API the cast is neccessary
SafariServices.SFSafariViewController c = null;
c = (SafariServices.SFSafariViewController)ui_object;
PresentViewController(c, true, null);
}
else
{
PresentViewController(ui_object, true, null);
}
Though in my experience, you probably don't need the SafariController.
Another alternative (also free) to Wireshark is Microsoft Network Monitor.
What browser are you using to test?
Try using the default IE install. System.Net.WebClient uses the local IE settings, proxy etc. Maybe that has been mangled?
Another cause for extremely slow WebClient downloads is the destination media to which you are downloading. If it is a slow device like a USB key, this can massively impact download speed. To my HDD I could download at 6MB/s, to my USB key, only 700kb/s, even though I can copy files to this USB at 5MB/s from another drive. wget shows the same behavior. This is also reported here:
https://superuser.com/questions/413750/why-is-downloading-over-usb-so-slow
So if this is your scenario, an alternative solution is to download to HDD first and then copy files to the slow medium after download completes.