We Keep Coding

Custom 404 http error in web.config is suppressing 400 errors in API C# code

I am developing an API using C# and .net 4.5.2; The API methods can return a handled BadRequest error or OK with an object response as per the below example:
[Authorize]
[RoutePrefix("api/Test")]
public class TestController : ApiController
{
[HttpGet]
[Route("TestMethod")]
public IHttpActionResult TestMethod()
{
MyProvider op = new MyProvider();
var lstResults = new List<Result>();
try
{
lstResults = op.TestMethod();
}
catch (Exception ex)
{
return BadRequest(ParseErrorMessage(ex.Message.ToString()));
}
return Ok(lstResults);
}
}
All errors are returned in a message object as below JSON:
{
Message: "Username or password is incorrect!"
}
The above was working perfectly until we added the below new configuration to redirect all 404 errors to a custom page for security issues. Now anybody (more specifically a hacker) who tries to call the API randomly, will be redirected to a custom 404 error page instead of the original .NET 404 page.
Web.config:
<httpErrors errorMode="Custom" existingResponse="Replace">
<remove statusCode="404" />
<error statusCode="404" path="404.html" responseMode="File"/>
</httpErrors>
The problem is:
BadRequest errors are not handled anymore as it was mentioned in the beginning, the custom JSON structure is not returned anymore, custom messages like "Username or password is incorrect!" are not taken into consideration, just the same simple text is always returned: Bad Request as per below screenshot:
The solution should be running on windows server 2016 IIS version 10.
How to solve the issue by keeping both working?
Update:
If I remove existingResponse="Replace", the badrequest message is returned, but the 404 custom error is not working anymore as per below screenshot
If I set errorMode="Detailed" the 404 custom error won't work anymore, and HTML description is returned for a bad request as you can see here:

I ended up using the below to solve the issue; Thus I won't mark it as the perfect answer since it didn't solve the above issue and I didn't know yet why the configuration did not work properly as excepted. So any answer that can solve the issue is more than welcomed, below is what worked as excepted for me:
Remove the httpErrors configuration from web.config
Use Global.asax file and add the below method to handle any error not handled in the API solution:
protected void Application_Error(object sender, EventArgs e)
{
try
{
try
{
Response.Filter = null;
}
catch { }
Exception serverException = Server.GetLastError();
//WebErrorHandler errorHandler = null;
//Try to log the inner Exception since that's what
//contains the 'real' error.
if (serverException.InnerException != null)
serverException = serverException.InnerException;
// Custom logging and notification for this application
//AppUtils.LogAndNotify(new WebErrorHandler(serverException));
Response.TrySkipIisCustomErrors = true;
string StockMessage =
"The Server Administrator has been notified and the error logged.<p>" +
"Please continue on by either clicking the back button or by returning to the home page.<p>";
// Handle some stock errors that may require special error pages
var HEx = serverException as HttpException;
if (HEx != null)
{
int HttpCode = HEx.GetHttpCode();
Server.ClearError();
if (HttpCode == 404) // Page Not Found
{
Response.StatusCode = 404;
//Response.Write("Page not found; You've accessed an invalid page on this Web server. " + StockMessage);
Response.Redirect("404.html");
return;
}
}
Server.ClearError();
Response.StatusCode = 500;
// generate a custom error page
Response.Write("Application Error; We're sorry, but an unhandled error occurred on the server." + StockMessage);
return;
}
catch (Exception ex)
{
Server.ClearError();
Response.TrySkipIisCustomErrors = true;
Response.StatusCode = 500;
Response.Write("Application Error Handler Failed; The application Error Handler failed with an exception.");
}
}
It worked like a charm:
user is redirected to 404.html custom page
Any 400 error is being thrown properly with the JSON format and proper message

Actions on Google Request Sync returning 403

I'm working on an Actions on Google smart home device.
I'm using a service account key with the "Service Account Token Creator" role, getting a token with the official NuGet package.
This is the code I've got to create the token for the request. It's the same code for both the Report State call and Request Sync.
var oauth = global::Google.Apis.Auth.OAuth2.GoogleCredential.FromJson(jsonContent).CreateScoped("https://www.googleapis.com/auth/homegraph");
System.Diagnostics.Trace.WriteLine(".. Awaiting token");
string token = await oauth.UnderlyingCredential.GetAccessTokenForRequestAsync();
The call to the Report State API is working fine, and I can see the homegraph data updating using the Smart Home Dashboard, but I'm getting a 403 "The caller does not have permission" when trying to use it with the Request Sync API.
I'm sure I've followed the instructions at https://developers.google.com/assistant/smarthome/develop/request-sync#http-post correctly but I'm at a loss as to why this isn't working as expected.
I have read that if the actual SYNC request fails, the RequestSync will fail the same way - that is, the 403 could be actually coming from my fulfillment endpoint, however, if that were the case I should also be seeing that request in my logs, so it looks like it's not generating a SYNC request.
Also, I don't have the logs, but when I first set this up it was actually returning a success response. The problem at that time was that despite getting a success code from the homegraph, the test suite was still failing.
The rest of the code is as follows
using Google.Apis.Auth.OAuth2;
using Google.Apis.HomeGraphService.v1;
using Google.Apis.HomeGraphService.v1.Data;
var homeGraphServiceService = new HomeGraphServiceService();
var devicesResource = new DevicesResource(homeGraphServiceService);
var i = 1;
foreach (var key in keys)
{
System.Diagnostics.Trace.WriteLine(".. Building request for key " + i + " of " + keys.Length);
i++;
var requestSync = devicesResource.RequestSync(
new RequestSyncDevicesRequest
{
AgentUserId = key
});
requestSync.AccessToken = token;
try
{
System.Diagnostics.Trace.WriteLine(".. Sending request");
var _ = await requestSync.ExecuteAsync();
}
catch (GoogleApiException e)
{
System.Diagnostics.Trace.WriteLine(".. .. Request Sync returned error code: " + e.HttpStatusCode + "\n" + e.Message + "\n\n");
}
}
I also left a comment on an IssueTracker issue I found, but I'm not sure if I should be holding my breath on that one.

Bot Framework Unauthorized when creating a conversation

I'm trying to resume a conversation between a bot and a user from a Web Job, and I'm getting an Unauthorized Exception.
I can successfully reply to a conversation in my MessagesController class, but when I try to run the following code in my Web Job, I get the following exception:
private static async Task SendAlertToUser(ChannelAccount botAccount, LipSenseUser user, AlertFlags alert, string extra)
{
Console.WriteLine($"Sending alert to user: {user.Name}");
var sb = new StringBuilder(GetTextFromAlert(alert));
if (!string.IsNullOrEmpty(extra))
{
sb.AppendLine();
sb.Append(extra);
}
var userAccount = new ChannelAccount(user.ChannelId, user.Name);
var connector = new ConnectorClient(new Uri(user.ChannelUri));
var message = Activity.CreateMessageActivity();
message.From = botAccount;
message.Recipient = userAccount;
var conversation = await connector.Conversations.CreateDirectConversationAsync(botAccount, userAccount);
message.Conversation = new ConversationAccount(id: conversation.Id);
message.Locale = "en-Us";
message.Text = sb.ToString();
await connector.Conversations.SendToConversationAsync((Activity)message);
}
And the exception is:
Exception:System.UnauthorizedAccessException
Authorization for Microsoft App ID 58c04dd1-1234-5678-9123-456789012345 failed with status code Unauthorized and reason phrase 'Unauthorized'
When I inspect the connector's Credentials, I see that everything is correct.  I've set a breakpoint in my MessagesController and inspected the connector's Credentials from there, and everything is identical.
Also, when I look at the IntelliTrace, I see the following messages:
My user.ChannelUri is "https://facebook.botframework.com", which I pulled off of the user when they initialized the conversation.  Is that not the correct Uri?
Is there anything else I need to do to send a message?  My App.config appSettings looks like this:
 
 
<appSettings>
    <add key="BotId" value="MyBotName" />
    <add key="MicrosoftAppId" value="58c04dd1-1234-5678-9123-456789012345" />
    <add key="MicrosoftAppPassword" value="5xyadfasdfaweraeFAKEasdfad" />
    <add key="AzureWebJobsStorage" value="DefaultEndpointsProtocol=https;AccountName=BLAH;AccountKey=THIS IS A KEY" />
</appSettings>

Answer from Bot Framework Team on a different channel:
You need to add a call to:
MicrosoftAppCredentials.TrustServiceUrl(serviceUrl);
This is done automatically when you are replying to a message, but for
proactive messages from another process you need to do this.

HttpClient returns The server committed a protocol violation - Browser returns correct data

I am trying to setup github with membership reboot.
If I try calling the github api from a browser like
https://api.github.com/user?access_token=XXXXXXX
I can see all the valid json data however if I try from .net
public ActionResult Index()
{
var url = "https://api.github.com/user?access_token=XXXXXXXX";
////add additional params
//if (additionalParams != null)
//{
// foreach (string key in additionalParams)
// {
// url += string.Format("&{0}={1}", key, additionalParams[key]);
// }
//}
HttpClient client = new HttpClient();
var result = client.GetAsync(url).Result;
if (result.IsSuccessStatusCode)
{
var json = result.Content.ReadAsStringAsync().Result;
var profile = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, object>>(json);
//return GetClaimsFromProfile(profile);
}
return View();
}
I get an error of
The server committed a protocol violation. Section=ResponseStatusLine
What doesn't HttpClient like about what I am trying? Do I need to provide extra details that the browser is doing for me?
Any help would be appreciated!

Adding
<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>
to my web.config seems to solve the problem...I dont really understand why I needed to add this but it works.

Not sure if it is relevant here, but I got the same issue on the https://api.github.com/user endpoint. My solution was simpler than I thought, but nevertheless annoying as it took me several hours to find out.
The User-Agent should be passed to the API Endpoint, maybe it works here also by specifying a User-Agent.

APNS Input string was not in a correct format. Disconnected

I am trying to set up apple push notifications, i am writing my own code for the server and as far as i know i have set up the app correctly. How ever i keep getting the following error come back from the log:
Payload queue received. Connecting to apple server. Creating SSL
connection. Conected. Payload generated for
"Token goes here i deleted it :
{"aps":{"alert":"test","badge":1,"sound":"default"}} Notification
successfully sent to APNS server for Device Toekn :
"Token here I've deleted it" An
error occurred while reading Apple response for token
"Token here I've deleted it" -
Input string was not in a correct format. Disconnected
This is my code.
var push = new PushNotification(true, #"C:\wwwroot\UltraNet\PushService\bin\Debug\206dist.p12", "ultrait");
var payload = new NotificationPayload("devicetoken here ive deleted it", "test", 1, "default");
var p = new List<NotificationPayload> { payload };
var result = push.SendToApple(p);
Console.ReadLine();
I have made sure that the certificates etc are set up correctly.
I am testing it as a adhoc app at the moment because it takes so long for a new version to be able to go live.
I really don't know where I'm going wrong if any one could help it would be brilliant thank you.
I also don't know what i need to do with the PEM files that i have created.
Edit***
I have the correct token this is another error that i receive
Payload generated for
df99286a1cb993cecba86b2e21f3fc4c04d214fcf7e0cf35a668fc822bdaa053 :
{"aps":{"alert":"test","badge":1,"sound":"default"}} Notification
successfully sent to APNS server for Device Toekn :
df99286a1cb993cecba86b2e21f3fc4c04d214fcf7e0cf35a668fc822bdaa053
Disconnected. An error occurred while reading Apple response for token
df99286a1cb993cecba86b2e21f3fc4c04d214fcf7e0cf35a668fc822bdaa053 -
Safe handle has been closed

Based on the code of ReadResponse (see below), the Input string was not in a correct format error message refers to the response received from Apple, and not to the notification you sent.
The code failed to properly read the error response from Apple.
Had it succeeded in reading the response, you would have known what the exact failure was and which message failed. Since you don't have the error response, it's a safe bet to assume the problem is your device token. That's the most common failure. If you can isolate the device token for which the error occurs, you should simply delete that token from your DB. Invalid Device Token error often occurs when you try to use sandbox tokens when pushing to production environment or vica versa.
private void ReadResponse(IAsyncResult ar)
{
if (!_conected)
return;
string payLoadId = "";
int payLoadIndex = 0;
try
{
var info = ar.AsyncState as MyAsyncInfo;
info.MyStream.ReadTimeout = 100;
if (_apnsStream.CanRead)
{
var command = Convert.ToInt16(info.ByteArray[0]);
var status = Convert.ToInt16(info.ByteArray[1]);
var ID = new byte[4];
Array.Copy(info.ByteArray, 2, ID, 0, 4);
payLoadId = Encoding.Default.GetString(ID);
payLoadIndex = ((int.Parse(payLoadId)) - 1000);
Logger.Error("Apple rejected palyload for device token : " + _notifications[payLoadIndex].DeviceToken);
Logger.Error("Apple Error code : " + _errorList[status]);
Logger.Error("Connection terminated by Apple.");
_rejected.Add(_notifications[payLoadIndex].DeviceToken);
_conected = false;
}
}
catch (Exception ex)
{
Logger.Error("An error occurred while reading Apple response for token {0} - {1}", _notifications[payLoadIndex].DeviceToken, ex.Message);
}
}

It was all to do with my certificates.
Because i hadn't turnt my combined PEM certificate back to a p12 file.