[luci] LuCId daemon explicit garbage collection

Bryan Mayland bmayland at leoninedev.com
Wed Jul 20 19:59:05 CEST 2011


I'm sorry to be such a pain in the butt, pointing out things that aren't 
working properly, but I have seen something today that I thought 
warranted a bit more discussion.

In the main lucid daemon process, there's a bit of code that wants to 
perform a full garbage collection if the poll times out:
lucid.lua:137 (run function)
elseif stat == 0 then
     ifaddrs = nixio.getifaddrs()
     collectgarbage("collect")
end

The problem here being that nixio.poll hasn't returned 0 on timeout for 
a valid set of FDs since [4437] back in the nixio 0.2 merge Apr-2009.  
Return value is:
0 - If poll() was called with a nil first parameter or a blank table (to 
use it like a sleep())
 > 0 - The number of signaled FDs on success
boolean(false) - If timeout occurred
throws an error - If there was an error on the native poll() call.

So the result here is that it will only do the garbage collect if it 
isn't polling any descriptors.  I don't think this is the desired 
behavior.  I would suggest just making it a straight "else" branch.

That got me thinking about the forked processes serving luci web pages.  
They share the same LUA state as the parent process by definition.  
Wouldn't it be a better idea to explicitly garbage collect before a 
fork() instead / in addition?  It would add a small amount of delay to 
the servicing of each request but I think it serves a benefit because 
then the child process receives a gc()ed LUA state and therefore will 
have less of a chance of undergoing its own gc cycle.

Consider the case where the parent is close to a gc step right as a 
typical browser 4-connection HTTP request comes in.  Each child http 
handler will inherit the parent's "almost full" memory manager.  Now as 
each starts to serve their respective file, each needs to undergo a gc 
sweep which causes the Linux memory manager to now copy the parent data 
pages containing the LUA memory manager into each child as each collects 
its copies of the parent LUA state.  Each child could potentially 
inherit a large number of private VM pages just due to this garbage 
collection.

(Also, I'm still looking at the speed of the indexcache thing pruning 
unrequested nodes.  I got a little sidetracked when my router kept 
running out of memory)



More information about the luci mailing list