VTest is failing to start a OWASP ZAP process with code 1 - c#

I have been trying to get the OWASP ZAP tool to run against my code in an automated fashion. I have been successful in writing a method that works consistently through the Text Explorer interface of Visual Studio 2017 on my local machine. However, whenever I try to have one of my automated test boxes run it using the remote testing command line tool triggered by TFS, then it always fails with Code 1, and no other error text.
I am running the same version of windows 10 locally and on the test box. I have the same version of visual studio installed on each, so I would expect them to be using the same version of VTest to run the tests. I'm not actually sure VTest is involved since I have converted to Selenium, and this is being run as a unit test anyway without a GUI component.
I have tried setting security on the .bat file and the .exe file it calls to Authenticated Users full access because I thought it could be a permissions issue.
I have looked in the event log, but there is are no details showing up in there. It is like it never happened.
I watch the test box while it is attempting to run this particular test, and it never even tries to start up a separate command window for the new zap process, it fails before that ever happens. However, it is definitely finding the correct file to run on the test box because I have tried altering the URL and get a specific error saying it can no longer be found.
I have also tried manually executing the command I am trying to run via command line and when I do it manually on the test box then it works perfectly. The TFS box is logged in and running as the user I tried the manual test as, and that user is a local admin on the box.
Because this works locally on my machine, and because I am able to successfully run the command manually on the test box, the problem component seems to be the TFS remote agent. The remote agent is successful in launching two other processes for Selenium testing, one of which is the IE driver, and the other is the WinAppDriver; the WinAppDriver has a separate command window associate with it when it launches. However, both of these processes are launched by the Selenium wrapper, I am not launching the processes myself directly.
You can see an excerpt below of the code I am using to launch the zap process. Does anyone know why it might be failing on my best box?
string ZapBatLocation = #"C:\Program Files\OWASP\Zed Attack Proxy\zap.bat";
var last_idx = ZapBatLocation.LastIndexOf('\\');
var _ZapProc = new System.Diagnostics.Process();
_ZapProc.StartInfo = new System.Diagnostics.ProcessStartInfo(ZapBatLocation.Substring(last_idx + 1), "-daemon") { WorkingDirectory = ZapBatLocation.Substring(0, last_idx), UseShellExecute = true, CreateNoWindow = false };
_ZapProc.ErrorDataReceived += ErrOutputHandler;
_ZapProc.Start();
System.Threading.Thread.Sleep(2000); // give it time to launch and die if it is going to
if (_ZapProc.HasExited)
throw new Exception($"Unable to launch ZAP process, exited with code {_ZapProc.ExitCode}.");
EDIT:
The log file seems to indicate some sort of config corruption. perhaps a security issue on a config file somewhere? Here is an extract of what appears to be the main error that appears frequently in the log:
2019-05-30 16:26:34,178 [AWT-EventQueue-0] INFO Control - New Session
2019-05-30 16:26:34,209 [AWT-EventQueue-0] INFO Control - Create and
Open Untitled Db 2019-05-30 16:26:34,240 [AWT-EventQueue-0] INFO
ENGINE - dataFileCache commit start 2019-05-30 16:26:34,240
[AWT-EventQueue-0] INFO ENGINE - dataFileCache commit end 2019-05-30
16:26:34,256 [AWT-EventQueue-0] INFO ENGINE - Database closed
2019-05-30 16:26:34,396 [AWT-EventQueue-0] INFO ENGINE - open start -
state not modified 2019-05-30 16:26:34,459 [AWT-EventQueue-0] INFO
ENGINE - dataFileCache open start 2019-05-30 16:26:34,475
[AWT-EventQueue-0] INFO ENGINE - dataFileCache open end 2019-05-30
16:26:35,615 [ZAP-cfu] ERROR ExtensionAutoUpdate - Unable to load the
configuration org.apache.commons.configuration.ConfigurationException:
Unable to load the configuration at
org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:955)
at
org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:908)
at
org.apache.commons.configuration.XMLConfiguration$XMLFileConfigurationDelegate.load(XMLConfiguration.java:1583)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:324)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:261)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:184)
at
org.zaproxy.zap.utils.ZapXmlConfiguration.(ZapXmlConfiguration.java:88)
at
org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.getPreviousVersionInfo(ExtensionAutoUpdate.java:883)
at
org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.getNewAddOns(ExtensionAutoUpdate.java:902)
at
org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate.gotLatestData(ExtensionAutoUpdate.java:1139)
at
org.zaproxy.zap.extension.autoupdate.ExtensionAutoUpdate$8.run(ExtensionAutoUpdate.java:983)
Caused by:
com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException:
Invalid byte 2 of 3-byte UTF-8 sequence. at
com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.invalidByte(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown
Source) at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source) at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown
Source) at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown
Source) at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown
Source) at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown
Source) at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown
Source) at
org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:942)
... 11 more 2019-05-30 16:26:45,052 [Thread-10] INFO ENGINE -
dataFileCache commit start 2019-05-30 16:26:45,052 [Thread-10] INFO
ENGINE - dataFileCache commit end 2019-05-30 16:26:45,084 [Thread-10]
INFO ENGINE - Database closed 2019-05-30 16:26:45,193 [Thread-10]
INFO Control - OWASP ZAP 2.7.0 terminated.
I cleared out the log file, and tried to run the automated test, it failed as expected but did not put anything in the log file. I then tried manually running ZAP from the command prompt, and it filled the log file after I exited the program, including the above error.
So it looks like the ZAP log file is not actually created until after a successful start, or at least more of a start than I am getting in the automation.

I'm not really sure this can be called an answer, but the problem no longer exists, so here goes.
After a week of fighting that rather odd issue, I got pulled away on other tasks and could no longer focus on it. After two weeks I finally had time to resume looking into the problem, and to my surprise it was no longer an issue. The .bat file is now launching correctly every time, with no changes on my part.
My best guess is that windows may have gotten stuck and some windows update caused a reboot sometime over the past two weeks causing things to start working. Why I never tried a basic system reboot during all of my troubleshooting I don't know, chalk that up to stupidity.
On a side note, if anyone else is struggling with a similar issue, my next trouble shooting step was to create a custom .bat file with these contents:
echo 'hi'
pause
Then insert a call to it in code right above the call to start the ZAP .bat file. Just to see if the code could start anything at all, or if it were something specific to the ZAP .bat file.
var bt = new System.Diagnostics.Process();
bt.StartInfo = new System.Diagnostics.ProcessStartInfo("C:\\selenium\\test.bat");
bt.Start();

Related

Getting "Can not access a closed Stream" error message when trying to extract a database using sqlpackage.exe with powershell

So I have an automation tool that uses c# to invoke a Powershell script that will use sqlpackage.exe to extract a database on a remote server into a .dacpac and save it into my file system.
We use asynchronous calls to the powershell process/pipeline to get the current result of the script execution and output them to the screen for the user. This tool has worked fine for ages but suddenly is stopping halfway through the extraction process and throwing the following error message "Error extracting database:Could not export schema and data from database.Can not access a closed Stream." At this point I have no idea what is causing the stream to get interupted and the extraction process to come to sudden halt.
Just wondering if anyone has experienced anything similar or if anyone can recommend some steps to debug this error?
Here is the portion of my script that is performing the extraction:
sqlpackage `
/action:extract `
/SourceConnectionString:"$endpoint" `
/TargetFile:"$dacPacFile" `
/p:ExtractAllTableData=True `
/p:ExtractReferencedServerScopedElements=False `
/p:IgnoreUserLoginMappings=True
The "can not access a closed stream" error message can unfortunately obscure the actual issue. You can enable SqlPackage tracing, which will capture all errors to an ETL file that can be opened using the event viewer. The steps to gather an event log are:
Open a new command prompt as Administrator and run the following command:
logman create trace -n DacFxDebug -p "Microsoft-SQLServerDataTools" 0x800 -o "%LOCALAPPDATA%\DacFxDebug.etl" -ets
Logging is now enabled, so retry the operation, reproducing the issue.
Disable logging by returning to the command prompt and running the following command:
logman stop DacFxDebug -ets
The resulting ETL file will be located at %LOCALAPPDATA%\DacFxDebug.etl.
DacFxDebug.etl can be opened with Event Viewer's Open Saved Log... button
i believe it is due to update issue, me to facing similar issue

ASP.NET Process Start

I made new project (ASP.NET MVC Web Application) using Visual Studio Web 2013 Express Edition. Loading index page, redirects and everything works fine, except that in some part of my code I need to create Process and execute it, wait for it to finish, parse output of that external application and show that output to the user. However I get nothing. While debugging the code line by line:
ProcessStartInfo info = new ProcessStartInfo();
info.FileName = Tool;
info.Arguments = filename + " " + avArray;
info.CreateNoWindow = true;
info.UseShellExecute = false;
info.RedirectStandardOutput = true;
Process p = new Process();
p.StartInfo = info;
p.Start();
p.WaitForExit();
Process exits immediately, someone would say it is pretty fast program, but application needs at least 5 seconds (sometimes 20 seconds) to complete.
What I think is happening is that process is not created in the first place, as my IIS settings or whatever other settings are not allowing me to run this EXE. What do I do to change this?
I've been trying to this for the past 2 weeks, I've read probably every post about it, and tried every solution suggested. I am choosing to write on this post since it is the most recent.
First of all, I am using VS2013, testing on IIS Express, and then on IIS 8.5., the webpages version is 3, and the target framework 4.5. And I am assuming that (as with my case) the goal is to launch the process on the server.
What I learned:
Using IIS Express should allow you to run the process without problems. You can try to allow it to launch the window, or start your process through the shell just to be sure that it is actually launching (a shell window should popup and close). If this happens, verify that you are giving the correct path to to your program, etc. So far so good.
Going into the IIS 8.5 is a totally different matter. Theoretically, the bad way of doing it should be preatty straight-forward: you create an app-pool for that server, give it a high privileged account (local system or network system, for instance); go to AdministrativeTools->Services->IIS, LogOn tab, and allow interaction with the desktop. You should be able to do the same as in IISExpress, but in my case is just the same as with a low privilege account.
I actually started by trying just to use impersonation, but also was not able to launch the process with IIS8.5. EDIT: I retried this again today, and the api won't even run if I enable impersonation.
I've spent more time than I would like with this, any input on why I can't get this to work properly with IIS8.5 (#clzola: IIS Express should do the job) is very welcome.

MK_E_UNAVAILABLE in Marshal.GetActiveObject("Word.Application")

0x800401E3 (MK_E_UNAVAILABLE) error occurs in my case when UAC (User Account Control) isn't set to the un-restrictive "Never Notify Me".
Microsoft.Office.Interop.Word.Application wd =
(Microsoft.Office.Interop.Word.Application)
System.Runtime.InteropServices.Marshal.GetActiveObject("Word.Application");
The error is thrown when the code is run after publishing and installing the project. While debugging in the editor instead, everything is fine.
Is this due to security settings or credentials ? How to write such code correctly pls ?
Win Word is open and a document is open too, of course, and this code has always worked fine with UAC set to "Never Notify Me".
Running word as a service is nasty business see for instance here . I went through quite a few problems making it work. The way I made it work was to run it in separate process which launches it only once. The main program communicates with the process by sending commands as strings to stdin of the process and waiting for response on the stdout. If the response does not come in time allotted the process is killed and restarted

Start executable that uses python

I'm new to programming and C#, I am attempting to write a program that detects errors or server crashes from my battlefield 2 server, it will then shut the server down (if the server is not already down) and start it back up but I have ran in to a small bump. When I try to start battlefield 2 back up again I receive this error from bf2:
Debug assertion failed! Version 1.0.2446.12 Build date:
Module: Python
File:
c:\dice\Projects\BF2Branches\BF2Demo\Code\BF2\Game\Python\PythonHost.Cpp
Line:243
Text: couldn't import the bf2 module:
Current confile:
The path in the error doesn't exist.
I have researched a couple different ways to start up an application but they all have the same results. These are a couple I have tried so far:
Process proc = new Process();
proc.StartInfo.FileName = #"txtServerPath.Text";
proc.Start();
and
Process.Start(txtServerPath.Text);
When I manually start the bf2 Server it starts fine. So what am I doing wrong?
My program will be ran from both Windows Server 2003 and Windows 7 if that matters.
Debug assertion failed! Version 1.0.2446.12 Build date: Module: Python File:
c:\dice\Projects\BF2Branches\BF2Demo\Code\BF2\Game\Python\PythonHost.Cpp
Line:243
Text: couldn't import the bf2 module:
Current confile:
The path in the error does not exist because it is the path on the machine on which PythonHost.cpp was written. 'Debug assertion' means that the developer of the code put a condition check at the specified line number in the cpp file to check for a certain condition to be sure that 'everything works fine.' but obviously it isn't. It seems strange though that BF2 has installed a debug version.
Anyhow, your process does start but it errors out. And the problem, it seems is that it is unable to find a python module named 'bf2' when you start the process from within C#.
So first and foremost verify that {BF2 Install folder}\python\bf2\ __init__.py exist.
In order to gain more insight on why this might be happening, try to start BF2 like this (assuming your C# application is a console application):
Process bf2 = new Process();
bf2.StartInfo.FileName = #"C:\Program Files\EA Games\Battlefield 2\BF2.exe";
bf2.StartInfo.Arguments = "+debugOutput 1";
bf2.StartInfo.UseShellExecute = false;
bf2.StartInfo.RedirectStandardOutput = true;
bf2.StartInfo.RedirectStandardError = true;
bf2.Start();
Console.WriteLine(bf2.StandardOutput.ReadToEnd());
Console.WriteLine(bf2.StandardError.ReadToEnd());
This should print some logs on the console going through which, I suspect, something useful can be deduced.
By the way can you post the remaining error message (i.e. after 'Current confile:' line)

Mathematica .Net/Link in an Asp.Net application

I am using the Mathematica .Net/Link platform to create a web service to format and calculate math problems. However I am unable to get it working.
I create it using this code:
_Log.IpDebug("Starting the Kernel Link");
if (string.IsNullOrEmpty(_MathLinkArguments))
_InternelKernel = MathLinkFactory.CreateKernelLink();
else
_InternelKernel = MathLinkFactory.CreateKernelLink(_MathLinkArguments);
_Log.IpDebug("Kernel Link Started");
_InternelKernel.WaitAndDiscardAnswer();
The value of _MathLinkArguments is -linkmode launch -linkname \"C:\\Program Files\\Wolfram Research\\Mathematica\\7.0\\Math.exe\".
This piece of code is called from the Application_Start method of the global.asax.cs file.
When it gets to the WaitAndDiscardAnswer() call it gives the server error:
Error code: 11. Connected MathLink program has closed the link, but there might still be data underway.
Note: The SampleCode given with the .NET/Link package (both a console app and a WinForms app) works.
Edit:
I copied the console app sample code given with Mathematica into an asp.net page and it gave me the same error the first load and then on subsequent loads it gave me:
Error code: 1. MathLink connection was lost.
Edit2:
I forgot to mention that when I have procmon and task manager open while running my app, I can tell that Math.exe starts but it immediately exits, which makes those error code make complete sense...but doesn't explain why that happened.
To allow the .Net/Link to work in Asp.net (at least in IIS 7.5) you need to enable the property loadUserProfile on the app pool for the web site.
I am not entirely sure why this is the case, but from what I found while trying to debug this, there are some things that are gotten from the user's profile. I know for a fact that the default location of the kernel is, which explains why I couldn't use it with no arguments, and so I can only assume that other things are needed as well and without the profile it couldn't determine that.
But whatever the reason is this is required, it is, or at least it is a fix if you are getting similar problems like this in your own application.
I got the same error in a .Net WinForm application.
mathKernel = new MathKernel();
mathKernel.Compute("<< XYZ`XYZGraphs`");
The error occurred on loading the package straight after instantiating the MathKernel.
To resolve it you can wait a couple of seconds and then instantiating the MathKernel works fine. During this state where there might still be data underway the following conditions are both false:
if (!MathKernel.IsConnected)
{
MathKernel.Connect();
}
if (MathKernel.IsComputing)
{
MathKernel.Abort();
}
Edit:
I've recieved the error again and this time was able to determine the problem.
Using a command line open the MathKernel.exe and view the error message:

Categories