I am trying to run a process on a web page that will return its output in realtime. For example if I run 'ping' process it should update my page every time it returns a new line (right now, when I use exec(command, output) I am forced to use -c option and wait until process finishes to see the output on my web page). Is it possible to do this in php?
I am also wondering what is a correct way to kill this kind of process when someone is leaving the page. In case of 'ping' process I am still able to see the process running in the system monitor (what makes sense).
This worked for me:
$cmd = "ping 127.0.0.1";
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("pipe", "w") // stderr is a pipe that the child will write to
);
flush();
$process = proc_open($cmd, $descriptorspec, $pipes, realpath('./'), array());
echo "<pre>";
if (is_resource($process)) {
while ($s = fgets($pipes[1])) {
print $s;
flush();
}
}
echo "</pre>";
This is a nice way to show real time output of your shell commands:
<?php
header("Content-type: text/plain");
// tell php to automatically flush after every output
// including lines of output produced by shell commands
disable_ob();
$command = 'rsync -avz /your/directory1 /your/directory2';
system($command);
You will need this function to prevent output buffering:
function disable_ob() {
// Turn off output buffering
ini_set('output_buffering', 'off');
// Turn off PHP output compression
ini_set('zlib.output_compression', false);
// Implicitly flush the buffer(s)
ini_set('implicit_flush', true);
ob_implicit_flush(true);
// Clear, and turn off output buffering
while (ob_get_level() > 0) {
// Get the curent level
$level = ob_get_level();
// End the buffering
ob_end_clean();
// If the current level has not changed, abort
if (ob_get_level() == $level) break;
}
// Disable apache output buffering/compression
if (function_exists('apache_setenv')) {
apache_setenv('no-gzip', '1');
apache_setenv('dont-vary', '1');
}
}
It doesn't work on every server I have tried it on though, I wish I could offer advice on what to look for in your php configuration to determine whether or not you should pull your hair out trying to get this type of behavior to work on your server! Anyone else know?
Here's a dummy example in plain PHP:
<?php
header("Content-type: text/plain");
disable_ob();
for($i=0;$i<10;$i++)
{
echo $i . "\n";
usleep(300000);
}
I hope this helps others who have googled their way here.
Checked all answers, nothing works...
Found solution Here
It works on windows (i think this answer is helpful for users searching over there)
<?php
$a = popen('ping www.google.com', 'r');
while($b = fgets($a, 2048)) {
echo $b."<br>\n";
ob_flush();flush();
}
pclose($a);
?>
A better solution to this old problem using modern HTML5 Server Side Events is described here:
http://www.w3schools.com/html/html5_serversentevents.asp
Example:
http://sink.agiletoolkit.org/realtime/console
Code: https://github.com/atk4/sink/blob/master/admin/page/realtime/console.php#L40
(Implemented as a module in Agile Toolkit framework)
For command-line usage:
function execute($cmd) {
$proc = proc_open($cmd, [['pipe','r'],['pipe','w'],['pipe','w']], $pipes);
while(($line = fgets($pipes[1])) !== false) {
fwrite(STDOUT,$line);
}
while(($line = fgets($pipes[2])) !== false) {
fwrite(STDERR,$line);
}
fclose($pipes[0]);
fclose($pipes[1]);
fclose($pipes[2]);
return proc_close($proc);
}
If you're trying to run a file, you may need to give it execute permissions first:
chmod('/path/to/script',0755);
try this (tested on Windows machine + wamp server)
header('Content-Encoding: none;');
set_time_limit(0);
$handle = popen("<<< Your Shell Command >>>", "r");
if (ob_get_level() == 0)
ob_start();
while(!feof($handle)) {
$buffer = fgets($handle);
$buffer = trim(htmlspecialchars($buffer));
echo $buffer . "<br />";
echo str_pad('', 4096);
ob_flush();
flush();
sleep(1);
}
pclose($handle);
ob_end_flush();
I've tried various PHP execution commands on Windows and found that they differ quite a lot.
Don't work for streaming: shell_exec, exec, passthru
Kind of works: proc_open, popen -- "kind of" because you cannot pass arguments to your command (i.e. wont' work with my.exe --something, will work with _my_something.bat).
The best (easiest) approach is:
You must make sure your exe is flushing commands (see printf flushing problem). Without this you will most likely receive batches of about 4096 bytes of text whatever you do.
If you can, use header('Content-Type: text/event-stream'); (instead of header('Content-Type: text/plain; charset=...');). This will not work in all browsers/clients though! Streaming will work without this, but at least first lines will be buffered by the browser.
You also might want to disable cache header('Cache-Control: no-cache');.
Turn off output buffering (either in php.ini or with ini_set('output_buffering', 'off');). This might also have to be done in Apache/Nginx/whatever server you use in front.
Turn of compression (either in php.ini or with ini_set('zlib.output_compression', false);). This might also have to be done in Apache/Nginx/whatever server you use in front.
So in your C++ program you do something like (again, for other solutions see printf flushing problem):
Logger::log(...) {
printf (text);
fflush(stdout);
}
In PHP you do something like:
function setupStreaming() {
// Turn off output buffering
ini_set('output_buffering', 'off');
// Turn off PHP output compression
ini_set('zlib.output_compression', false);
// Disable Apache output buffering/compression
if (function_exists('apache_setenv')) {
apache_setenv('no-gzip', '1');
apache_setenv('dont-vary', '1');
}
}
function runStreamingCommand($cmd){
echo "\nrunning $cmd\n";
system($cmd);
}
...
setupStreaming();
runStreamingCommand($cmd);
First check whether flush() works for you. If it does, good, if it doesn't it probably means the web server is buffering for some reason, for example mod_gzip is enabled.
For something like ping, the easiest technique is to loop within PHP, running "ping -c 1" multiple times, and calling flush() after each output. Assuming PHP is configured to abort when the HTTP connection is closed by the user (which is usually the default, or you can call ignore_user_abort(false) to make sure), then you don't need to worry about run-away ping processes either.
If it's really necessary that you only run the child process once and display its output continuously, that may be more difficult -- you'd probably have to run it in the background, redirect output to a stream, and then have PHP echo that stream back to the user, interspersed with regular flush() calls.
If you're looking to run system commands via PHP look into, the exec documentation.
I wouldn't recommend doing this on a high traffic site though, forking a process for each request is quite a hefty process. Some programs provide the option of writing their process id to a file such that you could check for, and terminate the process at will, but for commands like ping, I'm not sure that's possible, check the man pages.
You may be better served by simply opening a socket on the port you expect to be listening (IE: port 80 for HTTP) on the remote host, that way you know everything is going well in userland, as well as on the network.
If you're attempting to output binary data look into php's header function, and ensure you set the proper content-type, and content-disposition. Review the documentation, for more information on using/disabling the output buffer.
Try changing the php.ini file set "output_buffering = Off". You should be able to get the real time output on the page
Use system command instead of exec.. system command will flush the output
why not just pipe the output into a log file and then use that file to return content to the client. not quite real time but perhaps good enough?
I had the same problem only could do it using Symfony Process Components ( https://symfony.com/doc/current/components/process.html )
Quick example:
<?php
use Symfony\Component\Process\Process;
$process = new Process(['ls', '-lsa']);
$process->run(function ($type, $buffer) {
if (Process::ERR === $type) {
echo 'ERR > '.$buffer;
} else {
echo 'OUT > '.$buffer;
}
});
?>
I finally managed to get compression working with NetTcpBinding in WCF in .net 4.0. But it seems quite nasty to me, so maybe someone else has a better idea.
Some general Information:
I know this is working with .net 4.5 out of the box - but we are stuck to 4.0
I found several examples with CustomBinding - but we want to stick with NetTcpBinding because of the binary enconding (it is simply faster than text enconding)
We are doing all the configuration in code (except server address), so for the customer it is just plug and play and no chance to change anything - but made it also sometimes difficult to get an example working (most are provided in config files)
My first approach was implementing a message dispatcher on client and server side which does the compression:
http://dotnetlombardia.org/b/tonyexpo/archive/2011/03/09/compressione-in-wcf.aspx
But whenever I changed (or replaced) the original message by any means, the AfterReceiveReply on client side was never executed.
Though in WCF traces I could see that the client received the message and did even send an ACK to the server which the server received! But the client went in timeout?!
Then I found the MessageEncoderFactory for compression by Microsoft:
http://msdn.microsoft.com/en-us/library/ms751458%28v=vs.110%29.aspx
And applied the bug fix provided here:
http://blogs.msdn.com/b/dmetzgar/archive/2011/03/14/gzipmessageencoder-part-two.aspx
Finally I did inherit the NetTcpBinding and applied the new message encoder in the CreateBindingElements function:
public class CompressedNetTcpBinding : NetTcpBinding
{
MyCompressionMessageEncodingBindingElement compressionEncoding;
public CompressedNetTcpBinding()
: base()
{
FieldInfo fi = typeof(NetTcpBinding).GetField("encoding", BindingFlags.Instance | BindingFlags.NonPublic);
BinaryMessageEncodingBindingElement binaryEncoding = (BinaryMessageEncodingBindingElement)fi.GetValue(this);
compressionEncoding = new MyCompressionMessageEncodingBindingElement(binaryEncoding, CompressionAlgorithm.Deflate);
}
/// <summary>
/// Exchange <see cref="BinaryMessageEncodingBindingElement"/> and use compressed encoding binding element.
/// </summary>
/// <returns>binding elements with compressed message binding element</returns>
public override BindingElementCollection CreateBindingElements()
{
BindingElementCollection bec = base.CreateBindingElements();
BinaryMessageEncodingBindingElement enc = null;
foreach (BindingElement be in bec)
{
if (be is BinaryMessageEncodingBindingElement)
{
enc = (BinaryMessageEncodingBindingElement)be;
break;
}
}
bec.Remove(enc);
bec.Insert(2, compressionEncoding);
return bec;
}
}
I did forward the ordinary BinaryMessageEncoder to the compression encoding, so when I change any settings of the NetTcpBinding, e.g. ReaderQuotas they are applied correctly.
I know the enconding member in NetTcpBinding is also used in the private function IsBindingElementsMatch but so far that did not cause any problems.
On my local machine, the (startup) performance penalty is insignificant (100ms - 250ms). But on LAN and WAN there is a significat performance increase (up to several seconds).
So what do you think:
Is that a (the) way to go?
Are there any better solutions?
I got the following snippet (SomeName/SomeDomain contains real values in my code)
var entry = new DirectoryEntry("LDAP://CN=SomeName,OU=All Groups,dc=SomeDomain,dc=com");
foreach (object property in entry.Properties)
{
Console.WriteLine(property);
}
It prints OK for the first 21 properties, but then fail with:
COMException {"Unknown error (0x8000500c)"}
at System.DirectoryServices.PropertyValueCollection.PopulateList()
at System.DirectoryServices.PropertyValueCollection..ctor(DirectoryEntry entry, String propertyName)
at System.DirectoryServices.PropertyCollection.PropertyEnumerator.get_Entry()
at System.DirectoryServices.PropertyCollection.PropertyEnumerator.get_Current()
at ActiveDirectory.Tests.IntegrationTests.ObjectFactoryTests.TestMethod1() in MyTests.cs:line 22
Why? How can I prevent it?
Update
It's a custom attribute that fails.
I've tried to use entry.RefreshCache() and entry.RefreshCache(new[]{"theAttributeName"}) before enumerating the properties (which didn't help).
Update2
entry.InvokeGet("theAttributeName") works (and without RefreshCache).
Can someone explain why?
Update3
It works if I supply the FQDN to the item: LDAP://srv00014.ssab.com/CN=SomeName,xxxx
Bounty
I'm looking for an answer which addresses the following:
Why entry.Properties["customAttributeName"] fails with the mentioned exception
Why entry.InvokeGet("customAttributeName") works
The cause of the exception
How to get both working
If one wants to access a custom attribute from a machine that is not
part of the domain where the custom attribute resides (the credentials
of the logged in user don't matter) one needs to pass the fully
qualified name of the object is trying to access otherwise the schema
cache on the client machine is not properly refreshed, nevermind all
the schema.refresh() calls you make
Found here. This sounds like your problem, given the updates made to the question.
Using the Err.exe tool here
http://www.microsoft.com/download/en/details.aspx?id=985
It spits out:
for hex 0x8000500c / decimal -2147463156 :
E_ADS_CANT_CONVERT_DATATYPE adserr.h
The directory datatype cannot be converted to/from a native
DS datatype
1 matches found for "0x8000500c"
Googled "The directory datatype cannot be converted to/from a native" and found this KB:
http://support.microsoft.com/kb/907462
I have the same failure. I´m read and saw a lot of questions about the error 0x8000500c by listing attribute from a DirectoryEntry.
I could see, with the Process Monitor (Sysinternals), that my process has read a schema file. This schema file is saved under
C:\Users\xxxx\AppData\Local\Microsoft\Windows\SchCache\xyz.sch.
Remove this file and the program works fine :)
I just encountered the issue and mine was with a web application.
I had this bit of code which pulls the user out of windows authentication in IIS and pulls their info from AD.
using (var context = new PrincipalContext(ContextType.Domain))
{
var name = UserPrincipal.Current.DisplayName;
var principal = UserPrincipal.FindByIdentity(context, this.user.Identity.Name);
if (principal != null)
{
this.fullName = principal.GivenName + " " + principal.Surname;
}
else
{
this.fullName = string.Empty;
}
}
This worked fine in my tests, but when I published the website it would come up with this error on FindByIdentity call.
I fixed the issue by using correct user for the app-pool of the website. As soon as I fixed that, this started working.
I had the same problem with a custom attribute of a weird data type. I had a utility program that would extract the value, but some more structured code in a service that would not.
The utility was working directly with a SearchResult object, while the service was using a DirectoryEntry.
It distilled out to this.
SearchResult result;
result.Properties[customProp]; // might work for you
result.Properties[customProp][0]; // works for me. see below
using (DirectoryEntry entry = result.GetDirectoryEntry())
{
entry.Properties[customProp]; // fails
entry.InvokeGet(customProp); // fails as well for the weird data
}
My gut feel is that the SearchResult is a little less of an enforcer and returns back whatever it has.
When this is converted to a DirectoryEntry, this code munges the weird data type so that even InvokeGet fails.
My actual extraction code with the extra [0] looks like:
byte[] bytes = (byte[])((result.Properties[customProp][0]));
String customValue = System.Text.Encoding.UTF8.GetString(bytes);
I picked up the second line from another posting on the site.
UPDATE2: I got it working completely now! Scroll way down to find out how...
UPDATE: I got it working! Well... partially. Scroll down for the answer...
I'm trying to get my FO file to show an external image upon transforming it to PDF (or RTF for that matter, but I'm not sure whether RTFs are even capable of displaying images (they are)) with FOP, but I can't seem to get it working. (The question asked here is different than mine.)
I am using IKVM 0.46.0.1 and have compiled a FOP 1.0 dll to put in .NET; this code worked fine when I didn't try to add images:
private void convertFoByMimetype(java.io.File fo, java.io.File outfile, string mimetype)
{
OutputStream output = null;
try
{
FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
// configure foUserAgent as desired
// Setup outputput stream. Note: Using BufferedOutputStream
// for performance reasons (helpful with FileOutputStreams).
output = new FileOutputStream(outfile);
output = new BufferedOutputStream(output);
// Construct fop with desired output format
Fop fop = fopFactory.newFop(mimetype, foUserAgent, output);
// Setup JAXP using identity transformer
TransformerFactory factory = TransformerFactory.newInstance();
Transformer transformer = factory.newTransformer(); // identity transformer
// Setup input stream
Source src = new StreamSource(fo);
// Resulting SAX events (the generated FO) must be piped through to FOP
Result res = new SAXResult(fop.getDefaultHandler());
// Start XSLT transformation and FOP processing
transformer.transform(src, res);
}
catch (Exception ex)
...
}
However, when I (or rather a DocBook2FO transformation) added the following code:
<fo:external-graphic src="url(images/interface.png)" width="auto" height="auto" content-width="auto" content-height="auto" content-type="content-type:image/png"></fo:external-graphic>
into the FO file, the image did not show. I read through a bit of the FAQ on Apache's site, which says:
3.3. Why is my graphic not rendered?
Most commonly, the external file is not being found by FOP. Check the
following:
Empty or wrong baseDir setting.
Spelling errors in the file name (including using the wrong case).
...
Other options did not seem to be my case (mainly for the reason below - "The Weird Part"). I tried this:
...
try
{
fopFactory.setBaseURL(fo.getParent());
FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
foUserAgent.setBaseURL(fo.getParent());
FOURIResolver fourir = fopFactory.getFOURIResolver();
foUserAgent.setURIResolver(fourir);
// configure foUserAgent as desired
...
with no avail.
The Weird Part
When I use the command-line implementation of FOP, it works fine and displays my image with no problem. (I don't want to go the run-command-line-from-program route, because I don't want to force the users to install Java AND the .NET framework when they want to use my program.)
The png file is generated from GDI+ from within my application (using Bitmap.Save). I also tried different png files, but none of them worked for me.
Is there anything I might be missing?
Thanks a bunch for getting this far
UPDATE and possible answer
So I might have figured out why it didn't work. I put some time into studying the code (before I basically just copypasted it without thinking about it much). The problem is indeed in the wrong basedir setting.
The key is in this chunk of code:
// Setup JAXP using identity transformer
TransformerFactory factory = TransformerFactory.newInstance();
Transformer transformer = factory.newTransformer(); // identity transformer
// Setup input stream
Source src = new StreamSource(fo);
// Resulting SAX events (the generated FO) must be piped through to FOP
Result res = new SAXResult(fop.getDefaultHandler());
// Start XSLT transformation and FOP processing
transformer.transform(src, res);
What happens here is an identity transformation, which routes its own result into an instance of FOP I've created before. This effectively changes the basedir of the routed FO into that of the application's executable. I have yet to figure out how to do this without a transformation and route my input directly into FOP, but for the moment I worked around this by copying my images into the executable's directory.
Which is where another problem came in. Now whenever I try to execute the code, I get an exception at the line that says transformer.transform(src, res);, which confuses the pants out of me, because it doesn't say anything. The ExceptionHelper says:
java.lang.ExceptionInInitializerError was caught
and there is no inner exception or exception message. I know this is hard to debug just from what I wrote, but I'm hoping there might be an easy fix.
Also, this e-mail seems vaguely related but there is no answer to it.
UPDATE2
Finally, after a few sleepless nights, I managed to get it working with one of the simplest ways possible.
I updated IKVM, compiled fop with the new version and replaced the IKVM references with the new dlls. The error no longer occurs and my image renders fine.
I hope this helps someone someday
I'm using very similar code, although without the FOUserAgent, Resolvers, etc. and it works perfectly.
Did you try setting the src attribute in the XSLT without the url() function?
What might help you diagnose the problem further are the following statements:
java.lang.System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog")
java.lang.System.setErr(New java.io.PrintStream(New TraceStream(TraceStream.Level.Error)))
java.lang.System.setOut(New java.io.PrintStream(New TraceStream(TraceStream.Level.Info)))
Where TraceStream is a .NET implementation of a java.io.OutputStream which writes to your favorite logger.
I posted a version for the Common.Logging package at http://pastebin.com/XH1Wg7jn.
Here's a post not to leave the question unanswered, see "Update 1" and "Update 2" in the original post for the solution.
I use TextWriterTraceListener (System.Diagnostics) in my application to trace several things like exceptions,...
The application is running on a terminal server and if there are many users using it simultaneously the listener starts to create many tracefiles with random GUIDs in the filename.
Are there possibilities or workarounds to avoid this behaviour ?
I've just taken a look at the documentation for TextWriterTraceListener and there's a note about 1/3 of the way down the page
If an attempt is made to write to a file that is in use or unavailable, the file name is automatically prefixed by a GUID
So, this would appear to be by design. If the file is indeed unavailable then there's nothing that can be done about it with the current implementation. What you could try doing is writing a custom implementation of TextWriterTraceListener that overrides the relevant Write/WriteLine methods so that the output goes to a file, per user, with a name that better suits your needs.
If what you want is for ALL logging from ALL users on the Terminal Server to go to a single file, then you'll almost certainly need to have some kind of "3rd party" process running that "owns" the file and synchronises writes to it, such as a Windows Service that is then called by your custom TextWriterTraceListener
Was the fix calling the Trace.Listeners.Add(xxx listener) multiple times on accident?
Because if you have multiple listeners added they write too all listeners when you call the Trace.writeline();
Also local IIS might be continueing to have the file in use when you shut down the application.
I am currently testing the addition of System.Diagnostics.Trace.Listeners.Clear() in my output method...
// Upon a new day re-create the TextWriterTraceListener to update our file name...
if (_date?.Day != DateTime.Now.Day) { _listener = null; }
if (_listener == null)
{
System.Diagnostics.Trace.Listeners.Clear();
_fileName = $"{DateTime.Now.ToString("yyyy-MM-dd")}_Trace.json";
// Add a writer that appends to the trace.log file:
_listener = new System.Diagnostics.TextWriterTraceListener(_fileName);
_listener.IndentSize = 4;
_listener.TraceOutputOptions = System.Diagnostics.TraceOptions.None; // TraceOptions.DateTime | TraceOptions.ThreadId;
System.Diagnostics.Trace.AutoFlush = true;
System.Diagnostics.Trace.Listeners.Add(_listener);
// Obtain the Console's output stream, then add that as a listener...
System.Diagnostics.Trace.Listeners.Add(new System.Diagnostics.TextWriterTraceListener(Console.Out));
}