Comments on: Running multiple processes in PHP /2010/05/running-multiple-processes-in-php/ Sat, 29 Sep 2012 12:14:38 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: bee /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-64512 bee Mon, 05 Dec 2011 10:49:21 +0000 /?p=11#comment-64512 Hey cool threads.php class, Works great. I got a question anyway, what is the maximum number of scripts that can be multitasked? Thanks, bee. Hey cool threads.php class, Works great. I got a question anyway, what is the maximum number of scripts that can be multitasked?

Thanks,
bee.

]]>
By: carlos Pimentel /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-43145 carlos Pimentel Tue, 02 Aug 2011 10:19:44 +0000 /?p=11#comment-43145 Hi Nickolas Wood, could you please post the code that you've created in pastebin.com or similar website ? This code isn't working for me, sometime I receive the output from the process some time I don't and I almost crazy debugging this... Thanks ! Hi Nickolas Wood,
could you please post the code that you’ve created in pastebin.com or similar website ?
This code isn’t working for me, sometime I receive the output from the process some time I don’t and I almost crazy debugging this…
Thanks !

]]>
By: Silviu /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-35876 Silviu Sun, 12 Jun 2011 08:06:32 +0000 /?p=11#comment-35876 Hello. I've tried your code and I get the following error. Please help :) "Error 'tasks' is not recognized as an internal or external command, operable program or batch file." Hello. I’ve tried your code and I get the following error. Please help :)

“Error ‘tasks’ is not recognized as an internal or external command, operable program or batch file.”

]]>
By: Nickolas Wood /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-5401 Nickolas Wood Thu, 06 Jan 2011 17:41:20 +0000 /?p=11#comment-5401 Umm, let try this again removing the initial php tag: START CODE BLOCK: class Thread { var $pref ; // process reference var $pipes; // stdio var $buffer; // output buffer var $output; var $error; var $timeout; var $start_time; function Thread($timeout) { $this-> END CODE BLOCK: This will be my last attempt so that I don't over post here. Umm, let try this again removing the initial php tag:

START CODE BLOCK:

class Thread {
var $pref ; // process reference
var $pipes; // stdio
var $buffer; // output buffer
var $output;
var $error;
var $timeout;
var $start_time;

function Thread($timeout) {
$this->

END CODE BLOCK:

This will be my last attempt so that I don’t over post here.

]]>
By: Nickolas Wood /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-5400 Nickolas Wood Thu, 06 Jan 2011 17:40:10 +0000 /?p=11#comment-5400 I apologize, I missed a couple lines of code at the very top; at the first START CODE BLOCK. This codes should be inserted there: START CODE BLOCK: END CODE BLOCK: I apologize, I missed a couple lines of code at the very top; at the first START CODE BLOCK. This codes should be inserted there:

START CODE BLOCK:

END CODE BLOCK:

]]>
By: Nickolas Wood /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-5399 Nickolas Wood Thu, 06 Jan 2011 17:37:01 +0000 /?p=11#comment-5399 Hi, I have modified your class a little bit and fixed a couple bugs for you; figured I would contribute back. First off, line 107 and 108 of your threads.php file return undefined reference notices when ini_set('display_errors', 'On'); and error_reporting(E_ALL); are used. Secondly, the thread timeout of 0 on line 20 of the threads.php file (with no dynamic way of changing it) causes weird behavior when you start checking each thread for null output and resubmitting the command to a new multithread object on such null output. If a thread takes longer then 0 seconds to complete, the isbusy routine kills it. Most commands take longer then 0 seconds to return anything so I kept getting null output. This threw me for a loop for awhile. These three problems have been fixed in the below code. I have also made a couple of minor changes to better handle output. Look for lines with #MODIFIED in them, those I changed. I hope someone else finds it useful. Great work on your multithread class by the way. It helped me alot. START CODE BLOCK: pref = 0; $this->buffer = ""; $this->pipes = (array)NULL; $this->output = ""; $this->error=""; $this->start_time = time(); $this->timeout = $timeout; #MODIFIED, used $timeout } function Create ($command, $timeout) { #MODIFIED, added $timeout $t = new Thread($timeout); #MODIFIED, used timeout $descriptor = array (0 => array ("pipe", "r"), 1 => array ("pipe", "w"), 2 => array ("pipe", "w")); //Open the resource to execute $command $t->pref = proc_open($command,$descriptor,$t->pipes); //Set STDOUT and STDERR to non-blocking stream_set_blocking ($t->pipes[1], 0); stream_set_blocking ($t->pipes[2], 0); return $t; } //See if the command is still active function isActive () { $this->buffer .= $this->listen(); $f = stream_get_meta_data ($this->pipes[1]); return !$f["eof"]; } //Close the process function close () { $r = proc_close ($this->pref); $this->pref = NULL; return $r; } //Send a message to the command running function tell ($thought) { fwrite ($this->pipes[0], $thought); } //Get the command output produced so far function listen () { $buffer = $this->buffer; $this->buffer = ""; while ($r = stream_get_contents($this->pipes[1])) { #MODIFIED, replaced fgets with stream_get_contents #MODIFIED, removed $this->output appending (unneeded) $buffer .= $r; } return $buffer; } //Get the status of the current runing process function getStatus(){ return proc_get_status($this->pref); } //See if the command is taking too long to run (more than $this->timeout seconds) function isBusy(){ return ($this->start_time>0) && ($this->start_time+$this->timeoutpipes[2])) { #MODIFIED, replaced fgets with stream_get_contents $buffer .= $r; } return $buffer; } } //Wrapper for Thread class class Multithread{ var $output; var $error; var $thread; var $commands = array(); function __construct($commands, $timeout){ #MODIFED, added $timeout $this->commands = $commands; foreach ($this->commands as $key=>$command){ $this->thread[$key]=Thread::create($command, $timeout); #MODIFIED, used $timeout $this->output[$key] = ""; #MODIFIED, added this line to remove undefined reference notices $this->error[$key] = ""; #MODIFIED, added this line to remove undefined reference notices } } function run(){ $commands = $this->commands; //Cycle through commands while (count($commands)>0){ foreach ($commands as $key=>$command){ //Check if command is still active if ($this->thread[$key]->isActive()){ //Get the output and the errors $this->output[$key].=$this->thread[$key]->listen(); #I removed this line on my server as I don't need it, I want all output or nothing. I could see where it is useful however $this->error[$key].=$this->thread[$key]->getError(); #I removed this line on my server as I don't need it, I want all output or nothing. I could see where it is useful however //Check if command is busy if ($this->thread[$key]->isBusy()){ #MODIFIED, removed $this-output (unneeded, if we kill a process then its output is unusable and there is no need to update it further) $this->thread[$key]->close(); unset($commands[$key]); } } else { //Close the command, gather results and free resources #MODIFIED, added, gather results comment $this->output[$key].=$this->thread[$key]->listen(); #MODIFIED, added this line (want to make sure we have all output before we close a thread) $this->error[$key].=$this->thread[$key]->getError(); #MODIFIED, added this line (want to make sure we have all output before we close a thread) $this->thread[$key]->close(); unset($commands[$key]); } } } return $this->output; } } ?> END CODE BLOCK: I know, the formatting sucks but I couldn't find a nice way of doing it. Now, I call this class like so: START CODE BLOCK: $threads = new Multithread($command_list_exec, $execution_time); $threads->run(); END CODE BLOCK: As I mentioned earlier, I check each process's output against several strpos instances for known conditions. The most common problem I saw was no output at all. If this was the case then it was killed before anything had a chance to be reported and it needs to be run again. It is vitally important that set_time_limit($execution_time); is used in php pages that call the multithread class so that a rogue process doesn't cripple a with endless thread creations/kills. Hope this helps. Thanks again!! Hi,
I have modified your class a little bit and fixed a couple bugs for you; figured I would contribute back. First off, line 107 and 108 of your threads.php file return undefined reference notices when ini_set(‘display_errors’, ‘On’); and error_reporting(E_ALL); are used. Secondly, the thread timeout of 0 on line 20 of the threads.php file (with no dynamic way of changing it) causes weird behavior when you start checking each thread for null output and resubmitting the command to a new multithread object on such null output. If a thread takes longer then 0 seconds to complete, the isbusy routine kills it. Most commands take longer then 0 seconds to return anything so I kept getting null output. This threw me for a loop for awhile.

These three problems have been fixed in the below code. I have also made a couple of minor changes to better handle output. Look for lines with #MODIFIED in them, those I changed. I hope someone else finds it useful.

Great work on your multithread class by the way. It helped me alot.

START CODE BLOCK:

pref = 0;
$this->buffer = “”;
$this->pipes = (array)NULL;
$this->output = “”;
$this->error=”";

$this->start_time = time();
$this->timeout = $timeout; #MODIFIED, used $timeout
}

function Create ($command, $timeout) { #MODIFIED, added $timeout
$t = new Thread($timeout); #MODIFIED, used timeout
$descriptor = array (0 => array (“pipe”, “r”), 1 => array (“pipe”, “w”), 2 => array (“pipe”, “w”));
//Open the resource to execute $command
$t->pref = proc_open($command,$descriptor,$t->pipes);
//Set STDOUT and STDERR to non-blocking
stream_set_blocking ($t->pipes[1], 0);
stream_set_blocking ($t->pipes[2], 0);
return $t;
}

//See if the command is still active
function isActive () {
$this->buffer .= $this->listen();
$f = stream_get_meta_data ($this->pipes[1]);
return !$f["eof"];
}

//Close the process
function close () {
$r = proc_close ($this->pref);
$this->pref = NULL;
return $r;
}

//Send a message to the command running
function tell ($thought) {
fwrite ($this->pipes[0], $thought);
}

//Get the command output produced so far
function listen () {
$buffer = $this->buffer;
$this->buffer = “”;
while ($r = stream_get_contents($this->pipes[1])) { #MODIFIED, replaced fgets with stream_get_contents
#MODIFIED, removed $this->output appending (unneeded)
$buffer .= $r;
}
return $buffer;
}

//Get the status of the current runing process
function getStatus(){
return proc_get_status($this->pref);
}

//See if the command is taking too long to run (more than $this->timeout seconds)
function isBusy(){
return ($this->start_time>0) && ($this->start_time+$this->timeoutpipes[2])) { #MODIFIED, replaced fgets with stream_get_contents
$buffer .= $r;
}
return $buffer;
}
}

//Wrapper for Thread class
class Multithread{
var $output;
var $error;
var $thread;
var $commands = array();

function __construct($commands, $timeout){ #MODIFED, added $timeout
$this->commands = $commands;

foreach ($this->commands as $key=>$command){
$this->thread[$key]=Thread::create($command, $timeout); #MODIFIED, used $timeout
$this->output[$key] = “”; #MODIFIED, added this line to remove undefined reference notices
$this->error[$key] = “”; #MODIFIED, added this line to remove undefined reference notices
}
}

function run(){
$commands = $this->commands;
//Cycle through commands
while (count($commands)>0){
foreach ($commands as $key=>$command){
//Check if command is still active
if ($this->thread[$key]->isActive()){
//Get the output and the errors
$this->output[$key].=$this->thread[$key]->listen(); #I removed this line on my server as I don’t need it, I want all output or nothing. I could see where it is useful however
$this->error[$key].=$this->thread[$key]->getError(); #I removed this line on my server as I don’t need it, I want all output or nothing. I could see where it is useful however
//Check if command is busy
if ($this->thread[$key]->isBusy()){
#MODIFIED, removed $this-output (unneeded, if we kill a process then its output is unusable and there is no need to update it further)
$this->thread[$key]->close();
unset($commands[$key]);
}
} else {
//Close the command, gather results and free resources #MODIFIED, added, gather results comment
$this->output[$key].=$this->thread[$key]->listen(); #MODIFIED, added this line (want to make sure we have all output before we close a thread)
$this->error[$key].=$this->thread[$key]->getError(); #MODIFIED, added this line (want to make sure we have all output before we close a thread)
$this->thread[$key]->close();
unset($commands[$key]);
}
}
}
return $this->output;
}
}
?>

END CODE BLOCK:

I know, the formatting sucks but I couldn’t find a nice way of doing it. Now, I call this class like so:

START CODE BLOCK:

$threads = new Multithread($command_list_exec, $execution_time);
$threads->run();

END CODE BLOCK:

As I mentioned earlier, I check each process’s output against several strpos instances for known conditions. The most common problem I saw was no output at all. If this was the case then it was killed before anything had a chance to be reported and it needs to be run again.

It is vitally important that set_time_limit($execution_time); is used in php pages that call the multithread class so that a rogue process doesn’t cripple a with endless thread creations/kills.

Hope this helps.

Thanks again!!

]]>
By: masterb /2010/05/running-multiple-processes-in-php/comment-page-1/#comment-3698 masterb Fri, 03 Dec 2010 21:00:45 +0000 /?p=11#comment-3698 Great script, works perfect! Great script, works perfect!

]]>