PHP bot resource hog
From Botdom Wiki
MAJOR MEMORY LEAK / CPU HOG DISCOVERED in PHP bots: xbot, futurism, dante, amphino, gyn.
Brought to you by infinity0.
Details:
Contents |
microtime(): CPU hog
Note: this bug may affect many other bots (including non-PHP ones), depending on how they implement their main run loop. I only know PHP, so I only checked these bots.
This affect all bots which call microtime() on every iteration of their main run loop. It affects bots especially when the usleep time is unnecessarily short. 50ms is the recommended length, and adequate for normal use, especially since network latency far overrides this. Apart from xbot, all the other bots listed here have far shorter usleep times than this, which is incredibly wasteful and pointless.
Suggested fix: increment a counter every iteration, and estimate the time based on counter * usleep time. Then, get the actual time every 256 iterations, say (but do not check counter % 256; instead, have a separate counter which resets when it gets to 256).
How I verified this bug: After I removed all microtime() calls, CPU usage decreased from 15% to 8% (er, ignore the numbers, it's probably inaccurate. Xorg takes up wildly varying amounts of CPU and this was probably mostly because of that. but calling microtime() 50 times a second is still a bad idea).
event.php: Memory leak and CPU hog
This affects all bots which retains old code from the xbot modules system. This is caused by the "run" event triggering on every iteration of the main run loop, calling the event-handler, which is in a separate file and uses the PHP include construct to run. This is hugely inefficient.
Suggested fix: remove the run event. Or, better, recode this part of the modules system to not use includes. A fix for xbot is provided below; or you can ask me for xbot-2 (see the last paragraph of xbot#Latest for details).
How I verified this bug: After I removed the run event, memory usage stayed at 6MB even overnight, instead of increasing from 6MB to 100MB, and CPU usage decreased from 8% to negligible (er, ignore the numbers, it's probably inaccurate. Xorg takes up wildly varying amounts of CPU and this was probably mostly because of that. but calling include() 50 times a second is still a bad idea).
Affected code
Note: I have not attempted to fix specific cases, apart from xbot, because I don't know how the code here affects the other parts of the bot, and I cba working it out.
xbot
xbot-1.2-20080130-0000:~/system/run.php
3: list($mt1,)=args(microtime(),2);
4: usleep(50000); // time in microseconds
5: list($mt2,)=args(microtime(),2);
6: $usleeptime=$mt2-$mt1; if($usleeptime<0){$usleeptime+=1;}
Fix: comment lines 3, 5, 6. This will break the "tick" command in the devel module. No other xbot modules (that I know of) require this functionality.
Btw, it's probably better to do microtime(true) as in gyn below, which returns a float containing the time() as well. The default PHP settings outputs floats to only 12sf, which gives time-resolution to 0.01s, but actually more accuracy is stored internally, and you can squeeze a further 2sf (to 14sf*) out without getting into round errors. I didn't know this at the time I coded above, which is why I did it that shit way.
* which is the setting in php.ini-recommended.
xbot-1.2-20080130-0000:~/system/class.damn.php
130: require($rd.'system/process_event.php');
Fix: copy and paste the contents of that file over this line. Whilst you're at it, you can optionally copy and paste the contents of the other included files to their respective functions.
futurism
futurism-0.3.5:~/main.php:
in futurism (and dante, below) the "run" event is called the "loop" event.
125: if($loopTime <= microtime(true)){ // For efficency, the rest of the stuff is only called every second.
126: $loopTime = microtime(true) + 1;
127: $event = "loop"; include f("internal/includes/event.php");
160: usleep(10000);
dante
dante-0.9_03032007:~/main.php
192: usleep( 10000 ); // Delay for the main loop
197: if ( $loopTime <= microtime( true ) ) {
dante has the "loop" event disabled by default. thanks for not giving us a heads-up about it, SubjectX52873M. :| oh wait lol judging by the comment below ("decrease") he didn't realise how hoggy it was...
210: //loop event 211: //Uncomment the three lines below to use. It is recommended you decrease the main loop delay to 7500 if you are going to use this event. 212: //$from="!"; 213: //$event="loop"; // Sweet loop event 214: //include f( "system/callables/event.php" );
amphino
amphino-0.7a-beta:~/core.php
compare the below with the one in xbot above, lololol... although for some reason the usleep time is decreased :|
101: list($mt1,) = explode(' ',microtime(),2);
102: usleep(5000); // time in microseconds
103: list($mt2,) = explode(' ',microtime(),2);
104: self::$sleep = $mt2-$mt1;
105: if(self::$sleep < 0) {
106: self::$sleep += 1;
107: }
108: $UTS = time(); // Current time used for timestamps in logging.
109: $_MST = microtime(true); // Um... microtime timestamp... needed for system ping time, which isn't needed...
gyn
gyn-0.9.12:~/gyn.php
35: if(!(mode & MODE_SPEED)) 36: usleep(2500 / $_SPEED['last']);
41: $_SPEED['last'] = microtime(true) - $_SPEED['time']; 42: $_SPEED['time'] = microtime(true);
