I have added the ability for us to log errors to our database for all HTTPExceptions. Once this was pushed live we logged about 10,000 in the span of around 15 minutes. I have to assume these errors were happening all the time, but since they were not critical errors the user didn't notice anything.
Here is the code in my Global.asax
protected void Application_Error(object sender, EventArgs e)
{
Exception exception = Server.GetLastError();
if (exception is HttpException)
{
var httpException = exception as HttpException;
if (httpException.GetHttpCode() == 404)
{
// Page not found error
Response.StatusCode = 404;
using (MyEntities db = new MyEntities())
{
ErrorLog error = new ErrorLog();
error.Date = DateTime.Now;
error.Type = httpException.GetHttpCode().ToString();
error.Message = httpException.Message;
error.Source = httpException.Source;
error.StackTrace = httpException.StackTrace;
error.TargetSite = Request.Url.ToString();
error.Browser = Request.Browser.Browser + "(" + Request.Browser.Type + ")" + "v" + Request.Browser.Version;
error.IP = Request.ServerVariables["REMOTE_ADDR"];
//db.ErrorLogs.Add(error);
//db.SaveChanges();
}
}
else if (httpException.GetHttpCode() == 500)
{
// Server Error
Response.StatusCode = 500;
using (MyEntities db = new MyEntities ())
{
ErrorLog error = new ErrorLog();
error.Date = DateTime.Now;
error.Type = httpException.GetHttpCode().ToString();
error.Message = httpException.Message;
error.Source = httpException.Source;
error.StackTrace = httpException.StackTrace;
error.TargetSite = Request.Url.ToString();
error.Browser = Request.Browser.Browser + "(" + Request.Browser.Type + ")" + "v" + Request.Browser.Version;
error.IP = Request.ServerVariables["REMOTE_ADDR"];
//db.ErrorLogs.Add(error);
//db.SaveChanges();
}
}
else
{
// All Other Errors
Response.StatusCode = 500;
using (MyEntities db = new MyEntities ())
{
ErrorLog error = new ErrorLog();
error.Date = DateTime.Now;
error.Type = "999";
error.Message = httpException.Message;
error.Source = httpException.Source;
error.StackTrace = httpException.StackTrace;
error.TargetSite = Request.Url.ToString();
error.Browser = Request.Browser.Browser + "(" + Request.Browser.Type + ")" + "v" + Request.Browser.Version;
error.IP = Request.ServerVariables["REMOTE_ADDR"];
//db.ErrorLogs.Add(error);
//db.SaveChanges();
}
}
}
Out of the 10,000 errors we have logged only 50 of them were unique.
Here are examples of some of them
type Message
999 A potentially dangerous Request.Path value was detected from the client (:).
999 A potentially dangerous Request.Path value was detected from the client (>).
404 The controller for path '/images/favicon.ico' was not found or does not implement IController.
404 The controller for path '/Scripts/jquery-1.4.2.min.js' was not found or does not implement IController.
404 The controller for path '/Scripts/jquery-ui-1.8.5.custom.min.js' was not found or does not implement IController.
Now these seem like pretty silly errors. And from a user stand point this does not throw them to our custom 404 error page. So I am guessing they hit this error and are later redirected so they never see or notice they hit an error unless something is wrong with the code.
Anyone able to point me in the right direction to fix these if they are legitimate errors?
The last three 404 errors don't even make sense to me because they are not controllers and should not be treated like controllers ever.
These do look pretty harmless, or at least they're problably harmless. There are 2 separate issues here:
The first 2 errors come from someone having a ":" or ">" as part of their request's path. Possibly a url like "http://yoursite.com/stuff:here". This is something asp.net does to help prevent XSS attacks. Unless you have valid URLs that allow this, you probably want to leave it alone. However, if you want to check the errors, you can catch this exception by adding a check for HttpRequestValidationException. There are also some web.config changes you can make to allow these characters if they are necessary to your site's operation:
Getting "A potentially dangerous Request.Path value was detected from the client (&)"
As for the last 3, "The controller for path xxx ...". These are routing issues that only happen on IIS 7 and earlier (as well as the VS WebServer)
One that might work well for you is:
routes.IgnoreRoute("{*staticfile}", new { staticfile = #".*\.(css|js|gif|jpg|ico)(/.*)?" });
Related
I'm trying using gRPC dynamically typed values but with the little information about their usefulness, It's almost impossible to do this... So I will show the image/code that I have problems and the questions that are eating my brain
gRPC Method I'm doing:
public override Task<HelloReply2> TestObject(Status request, ServerCallContext context) {
//The part I may have problems
var status = new Status();
//here I want a User that corresponds to my request.Data
//example -> request.Data = User1 (gives me null if User1 don`t exist in db)
// request.Data = 14 (gives me null if 14 don`t exist in db)
// request.Data = true (gives me null if true don`t exist in db)
var a1 = _context.Users_5.FirstOrDefault(x => x.Username.Equals(request.Data));
var b1 = _context.Users_5.FirstOrDefault(x => x.Email.Equals(request.Data));
var c1 = _context.Users_5.FirstOrDefault(x => x.Age.Equals(request.Data));
var d1 = _context.Users_5.FirstOrDefault(x => x.Test.Equals(request.Data));
//is a bool
//here i want too Create dynamic values
status.Data = Value.ForStruct(new Struct {
Fields =
{
["Integer"] = Value.ForNumber(c1!.Age),
["StringName"] = Value.ForString(a1!.Username),
["StringEmail"] = Value.ForString(b1!.Email),
["Boolean"] = Value.ForBool(d1!.Test)
}
});
//Below is just a simple string who gives different string (depending on the
//data Status (also how to read the message from the status.Data ?)
HelloReply2 hello = new();
if(a1 != null)
{
hello.Message = "There is a User with the Username " + request.Data + ". His Email is " + a1.Email;
} else if (b1 != null) {
hello.Message = "There is a User with the Email " + request.Data + ". His Username is " + b1.Username;
}
else if (c1 != null)
{
hello.Message = "There is at least one User with that Age of " + request.Data + ". His Username is " + c1.Username;
}
else if (d1 != null)
{
if(d1.Test == true)
{
hello.Message = "There is at least one User who dislikes chocolate: " + request.Data + ". His Username is " + d1.Username;
} else
{
hello.Message = hello.Message = "There is at least one User who likes chocolate: " + request.Data + ". His Username is " + d1.Username;
}
}
else
{
hello.Message = "We didn't find something with the value that the User put in. Value:" + request.Data;
}
return Task.FromResult(hello);
}
Questions: How to Get the one Value from my gRPC? How to convert a "Object" in c# (one string, one integer or one List) into a ONE value of google.protobuf.Value (so it not give me errors like this Controller from a Web Api below)? Is something wrong with my gRPC Service Method (is something wrong reading the dynamic values? Can I do that calls for getting a User for a DB? How to read dynamic values?)
// I try using Google.Protobuf.WellKnownTypes.Value obj but
//not workings because gives me a lot of values to put
[HttpGet("TypeObject/{obj}")]
public async Task<ActionResult<HelloReply2>> TypeObject([FromRoute] Object obj){
Status objRequest = new Status { Data = (Google.Protobuf.WellKnownTypes.Value)
obj };
//cannot do this (gives me error of casting but the ideia is doing something
//similar to this)
var hello = await _greetClient.TestObjectAsync(objRequest);
return Ok(hello);
}
Any help on how to resolve this error of using Value gRPC or if is something wrong with the code is always welcome.
Edit:
One day after this question I don't have any solutions / progress. I was think of doing Any or OneOf for testing but it also gives me errors (who don't make sense at all). This code from Microsoft (C# Format part is not recognize) doesn't work in my project with the protos reload (the problem is not in the protos)
Link: https://learn.microsoft.com/en-us/dotnet/architecture/grpc-for-wcf-developers/protobuf-any-oneof
How I can use Any / OneOf without give me error in the Formating? What is the difference between Value and this two? Can the three (Value, Any, OneOf) be dynamic/Object values (if yes how to convert the types)?
Edit 2:
Still have problems, I'm trying using gRPC Any , and maybe have some progress (not all).
So with Any I have my method in the server gRPC and it is like this
public override Task<HelloReply2> TestObject3(AnyMessage request, ServerCallContext context){
HelloReply2 anyMessageResponse;
var y = request.TypeUrl;
switch (request.TypeUrl)
{
case "type.googleapis.com/any.HelloRequest":
var string_1 = request.Unpack<HelloRequest>();
anyMessageResponse = new HelloReply2{
Message = "You type String: " + $"{string_1.Name}"
};
break;
case "type.googleapis.com/any.TestInteger1":
var integer_1 = request.Unpack<TestInteger1>();
anyMessageResponse = new HelloReply2{
Message = "You type Integer: " + $"{integer_1.Message}"
};
break;
case "type.googleapis.com/any.TestBool1":
var bool_1 = request.Unpack<TestInteger1>();
anyMessageResponse = new HelloReply2{
Message = "You type Bool: " + $"{bool_1.Message}"
};
break;
default:
throw new InvalidOperationException("Unexpected type URL.");}
return Task.FromResult(anyMessageResponse);
}
This ideia comes from here (https://github.com/grpc/grpc-dotnet/issues/917), but the client part their don't have any much info or I don't understand that part
This is what I did in the WebApi (who is my client and the code is similar to the above one)
using AnyMessage = Google.Protobuf.WellKnownTypes.Any;
[HttpGet("TypeObject3/{obj3}")]
public async Task<ActionResult<HelloReply2>> TypeObject3([FromRoute] string obj3)
{
AnyMessage objRequest = new() { TypeUrl = obj3 };
var hello = await _greetClient.TestObject3Async(objRequest);
var l = hello.Message;
return Ok(hello);
}
First I had the variable Any declared in the method instead of string but as you can only put string and stringBytes so I preferred to put it like this (with the string as an obj3 variable) but my goal is to see if the variable is of type TestBool1 or TestInteger1 as I have declared in the protos and not be a string that I will be able to see, and the biggest problem was if I had more variables inside the messages how to proceed? So my secondary question is how to use Any on the client side via the Web-Api? I forgot to say but I'm using .Net 6 Core and for testing I'm using Swagger, where at this moment my error hits the Exception dictated by the Server method.
Questions: Why TypeUrl is a string and not object? How to fix my problem? How to test the object type (or string) for more values if the messages was with 1 more types?
Also I will show my test proto too show how I'm doing this
import "google/protobuf/struct.proto";
import "google/protobuf/any.proto";
package greet;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayNormalHello (Empty_2) returns (HelloReply);
rpc SayHello (HelloRequest) returns (HelloReply2);
rpc TestInt (TestInteger1) returns (HelloReply2);
rpc TestBoolean (TestBool1) returns (HelloReply2);
rpc TestObject (Status) returns (HelloReply2); //Not working
rpc TestObject2 (Status2) returns (HelloReply2); //Not working
rpc TestObject3 (google.protobuf.Any) returns (HelloReply2); //Also
//Not working
}
message Empty_2{
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings.
message HelloReply {
string message = 1;
}
// The response message containing the greetings.
message HelloReply2 {
string message = 1;
}
message TestInteger1 {
int32 message = 1;
}
message TestBool1 {
bool message = 1;
}
message Status {
google.protobuf.Value data = 1;
}
message Status2 {
google.protobuf.Any data = 1;
}
Any help is welcome.
In the project I developed, I needed an infrastructure where I could perform dynamic operations such as REST service using gRPC. As I understand it, what you want is something similar to this.
I have developed a solution for this. It might work for you too.
You can gain some flexibility by making a definition with a single String field for Requests and Responses that you want to be Dynamic, and then using this field to hold JSON data as strings.
For example, instead of defining different responses for different types, you can solve it by making one definition like this.
message HelloReply {
// Stringified JSON Data
string data = 1;
}
I made a very simple console application which will hit MWS service and return XML document containing list of financial events. As it starts, an exception hits at line 14 'client.ListFinancialEvents(request)' and it just show null and no other information as to why its not working.
string accessKey = "AccessKey";
string secretKey = "SecretKey";
string appName = "AppName";
string appVersion = "1.0";
string serviceURL = "http://mws.amazonservices.com/Finances/2015-05-01/";
try
{
MWSFinancesServiceConfig config = new MWSFinancesServiceConfig();
config.ServiceURL = serviceURL;
MWSFinancesServiceClient client = new MWSFinancesServiceClient(accessKey, secretKey, appName, appVersion, config);
ListFinancialEventsRequest request = new ListFinancialEventsRequest();
request.SellerId = "SellerID";
request.AmazonOrderId = "111-111111111-111111111";
ListFinancialEventsResponse response = client.ListFinancialEvents(request);
Console.WriteLine("Response:");
ResponseHeaderMetadata rhmd = response.ResponseHeaderMetadata;
Console.WriteLine("RequestId: " + rhmd.RequestId);
Console.WriteLine("Timestamp: " + rhmd.Timestamp);
string responseXml = response.ToXML();
Console.WriteLine(response.ResponseHeaderMetadata);
}
catch (Exception e)
{
Console.WriteLine("Error: " + e.Message);
}
Console.ReadLine();
The DLLs used as reference were downloaded from here. I have already tried MWS Scratchpad and values are working fine. What could be the possible issues due to which this exception occurred and how to solve this issue?
I am posting an answer to my own question in case someone faces similar issue in the future.
I was making a very stupid mistake and it took me a week to figure it out. In service URL, instead of "http://..." it should be "https://..." and it starts working perfectly. It return complete XML as response in 'responseXml' variable.
i'm trying to get a stream of tweets using LinqToTwitter library and the below c# code, but i get this error:
Error 401 Unauthorized
public static SingleUserAuthorizer auth;
static void Main(string[] args)
{
Task task = new Task(getStreamOfTweets);
task.Start();
task.Wait();
Console.ReadLine();
}
static async void getStreamOfTweets()
{
auth = new SingleUserAuthorizer
{
CredentialStore = new SingleUserInMemoryCredentialStore
{
ConsumerKey = CUSTOMER_KEY,
ConsumerSecret = CUSTOMER_SECRET,
AccessToken = ACCESS_TOKEN,
AccessTokenSecret = ACCESS_TOKEN_SECRET
}
};
var context = new TwitterContext(auth);
int count = 0;
await (from strm in context.Streaming
where strm.Type == StreamingType.Filter
&& strm.Track == "federer"
select strm)
.StartAsync(async strm =>
{
string message =
string.IsNullOrEmpty(strm.Content) ?
"Keep-Alive" : strm.Content;
Console.WriteLine(
(count + 1).ToString() +
". " + DateTime.Now +
": " + message + "\n");
if (count++ == 5)
strm.CloseStream();
});
}
notes:
the permission in twitter app is "Read, Write and Access direct messages"
i can get tweet by REST API correctly
Please review the LINQ to Twitter FAQ, which has an extensive section on resolving 401 errors. That said, if it's working for the REST API, but not Streaming, that might narrow the options to try. Here are a couple things to try first:
Double check the keys to make sure you didn't accidentally add a space or lose a character on the ends.
Give a little time before trying again because sometimes too many accesses or failed attempts might cause them to deny your connection for a certain period of time.
Re: #2, try another stream, like Sample.
There are a lot of moving parts in play for using OAuth, so work through that list in case you might have missed something.
this issue happens because the time of windows was wrong.
If you copied & pasted from the Linq"Twitter sample code, make sure you have set properly all the keys:
ConsumerKey
ConsumerSecret
AccessToken
AccessTokenSecret
Then dont use "Application only" auth, use "user auth" instead for streaming which uses all of the above keys.
the issue was because the timing in PC was wrong
I am desperately in need of debugging help, been stuck at this error for 6 days now.....
the error I get in my ASp.net app is :
Microsoft JScript runtime error: Sys.WebForms.PageRequestManagerParserErrorException: The message received from the server could not be parsed. Common causes for this error are when the response is modified by calls to Response.Write(), response filters, HttpModules, or server trace is enabled.
Details: Error parsing near '<script type='text/j'.
Below is the relevant code snippet,
CollyDataExchangeWebService Col_ValSub = new CollyDataExchangeWebService();
CollyReportServiceRequest ServiceReq = new CollyReportServiceRequest();
CollyReportServiceRequestData ServiceReqData = new CollyReportServiceRequestData();
ServiceReqData.AmendmentIndicatorSpecified = true;
ServiceReqData.AmendmentIndicator = false;
ServiceReqData.CollyReport = ColRep;
ServiceReq.ServiceRequestData = ServiceReqData;
ServiceReq.ServiceRequestHeader = ServiceHeader;
errValidate = null;
//btnOK.OnClientClick = "MSGShow()";
bool Valid = true;
string ErrMsgs = "";
if (((System.Web.UI.WebControls.Button)(sender)).CommandArgument == "Validate")
{
CollyReportServiceResponse ValResponse = Col_ValSub.validateReport(ServiceReq);
switch (ValResponse.ServiceResponseHeader.ServiceStatus)
{
case ServiceStatus.Successful:
btnOK.OnClientClick = "";
valHeader.Text = "Validation is Completed. No errors were found";
mlValPopup.Show();
break;
case ServiceStatus.ValidationErrors:
Valid = false;
ErrMsgs = ErrMsgs + _ValidationError(ValResponse);
ValBTN.Update();
mlValPopup.Show();
break;
case ServiceStatus.SystemError:
btnOK.OnClientClick = "";
Valid = false;
ErrMsgs = ErrMsgs + _SystemError(ValResponse);
ValBTN.Update();
mlValPopup.Show();
break;
}
After hours of debugging I found this line to be causing the error:
CollyReportServiceResponse ValResponse = Col_ValSub.validateReport(ServiceReq);
After 6 days of debugging and frustration I found that SOME records cause this issue and others dont in OLDER versions of the code but in new version ALL of the records lead to this error so it has to do something with the data in the DB which means SOME method in the code behaves differently to nulls but I cant find out exactly what the issue is because my app is 30k lines of code
after searching around and trying various solutions, the below 2 are not the solutions to my issue.
forums.asp.net/t/1357862.aspx
http://www.vbforums.com/showthread.php?t=656246
I want to mention that I am already having a difficult time dealing with this application because it was written by other programmers that are now long gone leaving behind non-documented or commented spaghetti code.
I did not code this but other programmers from past have put Response.Write in code:
private void MessageBox(string msg)
{
if (!string.IsNullOrEmpty(msg))
{
Global.tmpmsg = msg;
msg = null;
}
Response.Write("<script type=\"text/javascript\" language=\"javascript\">");
Response.Write("window.open('ErrorPage.aspx?msg=" + "','PopUp','screenX=0,screenY=0,width=700,height=340,resizable=1,status=no,scrollbars=yes,toolbars=no');");
Response.Write("</script>");
}
This one is in another method:
Response.Write("<script type=\"text/javascript\" language=\"javascript\">");
Response.Write("alert('No search resuls were found');");
Response.Write("</script>");
Or This:
if (!string.IsNullOrEmpty(msg))
{
Global.tmpmsg = msg;
Response.Write("<script type=\"text/javascript\" language=\"javascript\">");
Response.Write("window.open('ErrorPage.aspx?msg=" + "','PopUp','screenX=0,screenY=0,width=700,height=340,resizable=1,status=no,scrollbars=yes,toolbars=no');");
Response.Write("</script>");
}
After Jrummel`s comment I added this to code and then nothing at all happened.
private void MessageBox(string msg)
{/*
if (!string.IsNullOrEmpty(msg))
{
Global.tmpmsg = msg;
msg = null;
}
Response.Write("<script type=\"text/javascript\" language=\"javascript\">");
Response.Write("window.open('ErrorPage.aspx?msg=" + "','PopUp','screenX=0,screenY=0,width=700,height=340,resizable=1,status=no,scrollbars=yes,toolbars=no');");
Response.Write("</script>");
*/
// Define the name and type of the client scripts on the page.
String csname1 = "PopupScript";
Type cstype = this.GetType();
// Get a ClientScriptManager reference from the Page class.
ClientScriptManager cs = Page.ClientScript;
// Check to see if the startup script is already registered.
if (!cs.IsStartupScriptRegistered(cstype, csname1))
{
String cstext1 = "<script type=\"text/javascript\" language=\"javascript\">" + " " + "window.open('ErrorPage.aspx?msg=" + "','PopUp','screenX=0,screenY=0,width=700,height=340,resizable=1,status=no,scrollbars=yes,toolbars=no');" + " " + "</script>";
cs.RegisterStartupScript(cstype, csname1, cstext1, false);
}
}
I have found the error after 2 weeks of debugging and 2 days of Brute Forcing:
In one of the 800 DB columns that I have there was a null/improper value. This value reacted with one of the 150 methods in my ASP.NET code in such a way as to present a JavaScript error even though Response.Write() was NOT the issue. I have not found which method it was that reacted to this value but I have found the solution which is to simply input a valid value on the column record..
How a programmer can brute force to find the issue:
In my case after long days of debugging I took a sample of one working record and another sample of an error leading record. Once I had achieved this, I used
DELETE FROM tablename WHERE UniqueColID= unique identifier for the error causing record
Then I did:
INSERT INTO tablename ([uniqueIdentifier],[ column 2],[column 3]) SELECT #UniqueIdentifierofErrorCausingRecord, column2, [column3] FROM TableName WHERE [uniqueIdentifier]=#UniqueIdentifierForWorkingRecord;
What the first statement does is delete the non working record then the 2nd statement reinserts that record with identical column values of the working record but with the UniqueIdentifier of the Non working record. This way I can go through each table to find which table is causing the error and then I can pinpoint which column of that table is the issue.
The specific issue in my case was DateTime.TryParse() because a table column value was inserted in improper format.. The code performed field population in one of the methods without a try and catch using the DateTime.Parse method.... After some testing it seems even a try/catch is not able to pick this error up as it is a javascript error..
Don't use Response.Write().
Instead, create a LiteralControl and add it to the page.
Use ClientScriptManager to add scripts to the page. Here's an example from MSDN:
// Define the name and type of the client scripts on the page.
String csname1 = "PopupScript";
Type cstype = this.GetType();
// Get a ClientScriptManager reference from the Page class.
ClientScriptManager cs = Page.ClientScript;
// Check to see if the startup script is already registered.
if (!cs.IsStartupScriptRegistered(cstype, csname1))
{
String cstext1 = "alert('Hello World');";
cs.RegisterStartupScript(cstype, csname1, cstext1, true);
}
I have created a web browser for windows phone 7. In that am saving the history of browsing also with title of that page. But due to this codes when i tried to load a somewhat big website like thinkdigit,etc. are not loading. At the same time if i delete that particular codes then no history is being recorded in the history page. It shows error for the below highlighted code - "An unknown error has occurred. Error: 80020006". I think its due to big site or else do i need to put milliseconds or what i have to do???
Thanks in advance for your hard work!!!
Below is the codes i have used for it-
private void browsers_Navigated(object sender,System.Windows.Navigation.NavigationEventArgs e)
{
pbarWebClient.Visibility = System.Windows.Visibility.Collapsed;
if (!fromHistory)
{
if (HistoryStack_Index < HistoryStack.Count)
{
HistoryStack.RemoveRange(HistoryStack_Index, HistoryStack.Count - HistoryStack_Index);
}
HistoryStack.Add(e.Uri);
if (!app.incognito)
{
********string title = (string)browsers[this.currentIndex].InvokeScript("eval", "document.title.toString()");********----->This is the error.
stream.WriteLine(title + ";" + e.Uri.ToString());
}
HistoryStack_Index += 1;
}
fromHistory = false;
navigationcancelled = false;
Stop.Visibility = Visibility.Collapsed;
}
I would expect that eval has been overridden or in some other way suppressed on the site(s) in question.
"80020006" is a javascript invocation error code. Simply assuming that a javascript method will be available and work in the same way on any site is not always going to be true.
Finally i got the suitable answer for my own question. Just replace the above codes with this-
if (!app.incognito)
{
Thread.Sleep(100);
Dispatcher.BeginInvoke(() =>
{
string title = (string)browsers[this.currentIndex].InvokeScript("eval", "document.title.toString()");
stream.WriteLine(title + ";" + e.Uri.ToString());
});
}