I was hoping one of you could help me with the use of AWS Kinesis. I have been pouring over the documentation and I am still unable to post a "blob" of data to a Kinesis stream.
In the API the standard POST request is as follows.
POST / HTTP/1.1
Host: kinesis.<region>.<domain>
x-amz-Date: <Date>
Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=contenttype;
date;host;user-agent;x-amz-date;x-amz-target;x-amzn-requestid, Signature=<Signature>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Connection: Keep-Alive
X-Amz-Target: Kinesis_20131202.PutRecord
{
"StreamName": "exampleStreamName",
"Data": "XzxkYXRhPl8x",
"PartitionKey": "partitionKey"
}
Using "Postman" a google chrome app to make Http requests and filling in the appropriate information for the above fields I cannot for the life of me figure out how to make a successful data post to a stream. I was unable to do this successfully so i went on to do use the .NET SDK for visual studio.
I made a quick command line c# console application I am still having some issues.
My code:
http://pastebin.com/cyJeC0vU
The error message, System.Xml.XmlException was unhandled, appears on line 61 of the code: http://pastebin.com/HEG7DmMw
Has anyone had a successful experience using AWS Kinesis. I would love to pick your brain / repay you somehow for a bit of tutoring.
Thanks again for all of your help!
The error you're getting indicates that the response from the service was not able to be parsed. There is a clue in that the SDK switched from the JSON parser to the XML parser because the response looked like XML. This usually indicates that you are behind a proxy which requires authentication, and the proxy is giving you an HTML error message.
You can verify this by firing up a protocol analyzer like Fiddler and watching the request traffic. If your proxy requires credentials, here is some documentation about using the AWS SDK for .NET with proxies:
Configuring Credentials for Your AWS SDK for .NET Application
Related
I am working on a Chatbot in Slack that sends a POST request to http://localhost:44331/values/api an .NET Core API that i built in C#. In the Post request is a response_url in the body I can use to send back the needed information.
So I have been trying to make this work for about two weeks now and used a fiddler and to mimic the request so I can make some changes on the body and the headers to see if that makes a difference.
So after a lot of errors I have come to two specific errors that haven't changed for a long while.
sent with the Slack Chatbot: curl_error_56
There really isnt much I can change in this matter except the url I want to send the request to.
In fact this request has never even reached the post method in my API.
Thats what the Slackbot answers
sent with fiddler: HTTP error 400
I used Requestbin to get the information that has been sent by the bot and copied it into the composer in fiddler.
I am a total novice to Web programming in any kind of way so I really don't know what they have in common.
Are those errors coming because I am using localhost?
What am I missing?
here is the request so you can copy it if needed
host: localhost:44331
Accept: application/json,*/*
Accept-Encoding: gzip,deflate
Content-Type: application/x-www-form-urlencoded
User-Agent: Slackbot 1.0 (+https://api.slack.com/robots)
X-Slack-Request-Timestamp: 1569238196
X-Slack-Signature: v0=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Content-Length: 381
Connection: keep-alive
Alright I got some help by a friend and he explained to me that Slack is unable to access my localhost, because it is not a local app. It is taking its information from the web and making it send a request to my localhost is useless because it is not a static IP.
What i need is an Endpoint.
I can get one by requesting it from my ISP (Internet Service Provider) or getting myself a server which already has an static IP.
Thanks for any help you wanted to provide.
I'm posting telemetry data to the Application Insights data collector endpoint at https://dc.services.visualstudio.com/v2/track using HttpClient.
This works fine when using .NET Core 2.0. However, after upgrading to .NET Core 2.1 all requests fail with HTTP status 500 Internal Server Error and an HTML response body that looks like this:
(It's an IIS 8.5 Detailed Error page)
In .NET Core 2.1 they have changed how HttpClientHandler works and added an AppContext switch that allows us to keep the old behavior:
AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);
Indeed, setting this switch fixes the problem!
But I'd like to avoid using that switch:
Firstly, because I'd like to use the new stuff. It must be better :-)
Secondly, because that switch changes behavior for all new HttpClientHandler instances. Not just the one I'm using to post data to the Application Insights endpoint.
I've tried to figure out what's causing this by looking at the traffic using Wireshark. Trying to interpret HTTPS traffic on that low level is beyond me. I did find something suspicious in the TLS handshake though.
Both the SocketsHttpHandler (that doesn't work) and the legacy WinHttpHandler (that do work) use TLS 1.2. However, only the WinHttpHandler uses the TLS Certificate Status Request extension. Could that be the reason why requests are failing?
In any case, the TLS connection is setup a request is sent and a response is retrieved. And apparently there is an error on the server side while processing the request.
I'd like to figure out what's causing this problem. Any help is appreciated!
As suggested by #mjwills I changed my code so that it uses HTTP (not SSL) and post to a local program that allows me to capture the request. (I didn't figure out how to do this in a clean way in Wireshark, so I ended up writing my own little capture tool based on TcpListener):
Capture when using the new SocketsHttpHandler:
POST / HTTP/1.1
Transfer-Encoding: chunked
Content-Type: application/x-json-stream
Content-Encoding: gzip
Host: localhost:12345
A
[...10 bytes of data...]
F4
[...244 bytes of data...]
0
Capture when using the legacy WinHttpHandler:
POST / HTTP/1.1
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/x-json-stream
Content-Encoding: gzip
Host: localhost:12345
a
[...10 bytes of data...]
f4
[...244 bytes of data...]
0
These are the differences:
WinHttpHandler wants to keep the connection alive while SocketsHttpHandler does not.
WinHttpHandler use lower case hex for chunk lengths while SocketsHttpHandler use upper case.
The first 10 bytes contains the gzip header. The remaining 244 bytes contains the gzip encoded json telemetry data.
I am going to develop a C# application using web service but the service is developed by Apache Axis2 and I can not handle the service to integrate adding service reference in C#. I get no response and only format exception.
I don't have service code, it's like ready-to-use web service, just give service reference and get started to use it.
The web service wsdl address is:
https://pttws.ptt.gov.tr/PttBilgi/services/Sorgu?wsdl
I also tried to run the service with SOAP UI but the XML response I get is like below:
<faultstring>The endpoint reference (EPR) for the Operation not found is /PttBilgi/services/Sorgu.SorguHttpSoap11Endpoint/ and the WSA Action = null. If this EPR was previously reachable, please contact the server administrator.</faultstring>
The raw response is like below, too which is but with some technical detail:
HTTP/1.1 500 Internal Server Error
Date: Fri, 14 Apr 2017 07:20:28 GMT
Server: gizli gizli gizli gizli
X-OPNET-Transaction-Trace: a2_5cc44d1b-0e0c-48a5-a44f-942377e9ab70
X-Powered-By: Servlet/2.5 JSP/2.1
Vary: Accept-Encoding,User-Agent
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml; charset=UTF-8
<faultstring>The endpoint reference (EPR) for the Operation not found is /PttBilgi/services/Sorgu.SorguHttpSoap11Endpoint/ and the WSA Action = null. If this EPR was previously reachable, please contact the server administrator.</faultstring>
It seems that content type has to be text/html to use it in C# apps but being of the service's content type is application/xml, so it can not convert to read and process the result.
How can I use the service with success, what do I have to do?
Thanks guys!
As described in this article at http://wso2.com/library/176/
If Axis2 engine cannot find a service and an operation for a message, it immediately fails, sending a fault to the sender.
If service not found - "Service Not found EPR is "
If service found but not an operation- "Operation Not found EPR is and WSA Action = "
I'm hosting a magento webshop on my local machine using IIS 7.5 and PHP 5.6 (testing). The shop is working just fine, but now I want to create a separate application using visual studio 2013. These are the steps that I've taken:
I've added the domain name "www.domain.com.local" to my hosts file directing to my localhost (www.domain.com.local -> 127.0.0.1)
I've created a new website on my IIS and added a new binding (www.domain.com.local - see 1)
I've added WinCache extension with the PHP Manager and set the PHP version to 5.6
Enable WS-I Compliance in the magento backend (System > Configuration > Magento Core Api)
Create a SOAP Role and User (Resource Access = All)
Open visual studio 2013 and create a new Console Application
Adding a new Service Reference (http://www.domain.com.local/index.php/api/v2_soap/?wsdl)
Trying to login - This is not working like it should be
Here is my piece of code:
class Program
{
static void Main(string[] args)
{
MainAsync(args).Wait();
}
static async Task MainAsync(string[] args)
{
using (var proxy = new Mage_Api_Model_Server_Wsi_HandlerPortTypeClient())
{
try
{
var loginResponse = await proxy.loginAsync("soap-admin", "xxxxxxxxx"); // api key
var sessionId = loginResponse.result;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Console.ReadLine();
}
}
}
And this is the error I'm getting:
The content type text/xml; charset=utf-8,text/xml; charset=utf-8 of the response message does not match the content type of the binding (text/xml; charset=utf-8). If using a custom encoder, be sure that the IsContentTypeSupported method is implemented properly. The first 297 bytes of the response were: '<?xml version="1.0" encoding="utf-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="urn:Magento">
<SOAP-ENV:Body>
<ns1:loginResponseParam>
<result>13fa067676759c3ce8ddd61c386b6d5c</result>
</ns1:loginResponseParam>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
'.
So as you can see, I'm getting my sessionId but keep getting this error. I've also used fiddler to investigate and getting a correct response: HTTP 200 OK. Does someone knows what the problem could be? Is it IIS related? localhost related?
(When I add the url as web reference it works just fine - old webservice method).
Related topics I've read and tried (without success):
C# SOAP - Error in deserializing body of reply message (Magento API)
C#+Magento API V2:The content type text/xml; charset=utf-8,text/xml; charset=UTF-8 of the response message does not match
If got this answer from Rian at Magento Stack Exchange. All credits go to Rian
The solution:
The problem you're experiencing is that .NET/C# is having trouble parsing the content type Magento is sending along with it's response. SOAP is notoriously finicky about receiving just the right stuff in just the right format. Couple that with PHP's rather poor implementation of the protocol and you're in for a lot of fun.
I'm looking at a Magento 1.9 for the following information:
After some digging I found that the header for the SOAP calls are set in app/code/core/Mage/Api/Model/Server/V2/Adapter/Soap.php on line 52.
51. ->clearHeaders()
52. ->setHeader('Content-Type','text/xml; charset='.$apiConfigCharset)
53. ->setBod...
Note that that Content-Type header matches your text/xml; charset=utf-8 desired charset. Adjusting line 52 to:
52. ->setHeader('Content-Type','text/xml; charset='.$apiConfigCharset, true)
tells Magento to force overwriting that header if it's already set.
Make sure to make a copy of the file with it's full path to the app/code/local/Mage/... to avoid overwriting core files. You'll thank me when you want to upgrade Magento at some point.
Also, make sure to look carefully, there's two setHeader() calls in that file.
And finally, there's also a WS-I compliant SOAP adapter available, the same fix applies to that file. You can find it in app/code/core/Mage/Api/Model/Server/Wsi/Adapter/Soap.php.
I'm attempting to post JSON data to a web API and get the resulting json object back. I'm using HttpWebRequest to post to an https address using basic authentication. On the request.GetResponse() I'm getting an error of
The underlying connection was closed: An unexpected error occurred on a send.
and an inner exception of
Received an unexpected EOF or 0 bytes from the transport stream.
My test application is compiled against .NET 4.0 The application works on machines running 4.5, however fails to run on several machines running 4.0. Upgrading to 4.5 on those machines is not a simple task. While pouring through the trace log, I've noticed only 1 difference before the failure occurs which I've included below. Any help would be appreciated.
System.Net Information: 0 : [5876] ConnectStream#9956401 - Sending headers
{
Content-Type: application/json
Authorization: Basic <encoded base64 auth string is here>
Host: <host name is here>
Content-Length: 130
Expect: 100-continue
Connection: Keep-Alive
}.
While trying to research this error, I came across two recurring issues, neither of which seem to apply to my error, but I tried the fixes anyway. I've tried forcing SSL3 (though authentication isn't the issue) and also turning off Keep-Alive. Neither work.
I had a similar problem. I was trying to download a file via .net WebClient class.
It turned out that Apache server had wrong name in VirtualHost::ServerName.