[luci] LuCId daemon explicit garbage collection
Bryan Mayland
bmayland at leoninedev.com
Wed Jul 20 19:59:05 CEST 2011
I'm sorry to be such a pain in the butt, pointing out things that aren't
working properly, but I have seen something today that I thought
warranted a bit more discussion.
In the main lucid daemon process, there's a bit of code that wants to
perform a full garbage collection if the poll times out:
lucid.lua:137 (run function)
elseif stat == 0 then
ifaddrs = nixio.getifaddrs()
collectgarbage("collect")
end
The problem here being that nixio.poll hasn't returned 0 on timeout for
a valid set of FDs since [4437] back in the nixio 0.2 merge Apr-2009.
Return value is:
0 - If poll() was called with a nil first parameter or a blank table (to
use it like a sleep())
> 0 - The number of signaled FDs on success
boolean(false) - If timeout occurred
throws an error - If there was an error on the native poll() call.
So the result here is that it will only do the garbage collect if it
isn't polling any descriptors. I don't think this is the desired
behavior. I would suggest just making it a straight "else" branch.
That got me thinking about the forked processes serving luci web pages.
They share the same LUA state as the parent process by definition.
Wouldn't it be a better idea to explicitly garbage collect before a
fork() instead / in addition? It would add a small amount of delay to
the servicing of each request but I think it serves a benefit because
then the child process receives a gc()ed LUA state and therefore will
have less of a chance of undergoing its own gc cycle.
Consider the case where the parent is close to a gc step right as a
typical browser 4-connection HTTP request comes in. Each child http
handler will inherit the parent's "almost full" memory manager. Now as
each starts to serve their respective file, each needs to undergo a gc
sweep which causes the Linux memory manager to now copy the parent data
pages containing the LUA memory manager into each child as each collects
its copies of the parent LUA state. Each child could potentially
inherit a large number of private VM pages just due to this garbage
collection.
(Also, I'm still looking at the speed of the indexcache thing pruning
unrequested nodes. I got a little sidetracked when my router kept
running out of memory)
More information about the luci
mailing list