[luci] Lua memory usage

[Jo-Philipp Wich]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey.

> I'm running the lua intrepreter in the uhttpd process. I have
> adjusted the original lua plugin to not fork. This gives a good
> improvement to initial page appearance time. It does not change the
> actual page production time. It is a trade-off of memory for
> response time.

Maybe we can make it configurable in the existing plugin, then you can
reuse the same codebase.

> But I started seeing the uhttpd memory rising too much. I added a
> lua garbage collection call after each request which made an
> obvious improvement.

This as well.

> But there was still a real leak. I looked at the lua heap info and 
> tracked it to
> 
> +--- a/libs/web/luasrc/dispatcher.lua ++++
> b/libs/web/luasrc/dispatcher.lua +@@ -435,7 +435,7 @@ function
> dispatch(request) + + 	if type(target) == "function" then +
> util.copcall(function() +-			local oldenv = getfenv(target) ++
> local oldenv = getfenv() + 			local module = require(c.module) +
> local env = setmetatable({}, {__index=
> 
> 'target' in my case, has history. This code would build up a chain
> of tables for env.
> 
> I'm still not 100% sure this fix is correct. I don't fully
> understand which elements of env are important for the target
> functions, beyond the standard environment. My testing has not
> found a problem, yet.

That code is there to preserve the original environment of the
referenced functions, e.g. if they want to access stuff declared in
the module they're originating from.

I think the change above would break at least
http://luci.subsignal.org/trac/browser/luci/trunk/applications/luci-commands/luasrc/controller/commands.lua#L168
- - maybe you can test it, I believe it will fail to find the local
parse_cmdline() function.

Not sure how to fix that properly, maybe by moving the metatable
binding inside the call() function somehow.

> But, there was still a leak, visible in the uclibc heap, not the
> lua memory report. Playing with the uclibc malloc_trim, and also
> restarting the lua interpreter did not help. I have resorted to
> re-starting the uhttpd process.

The leak could happen inside lmo.so or parser.so, maybe take a closer
look there.

> I suspect the process memory is getting fragmented, so that a
> better allocator is the solution. Lua provides a hook for a custom
> allocator.
> 
> Thoughts?

Uhm, no, not at the moment but I would be interested in your findings
in case you try it with another allocator.


~ Jow
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlC2Hc0ACgkQdputYINPTPOumgCdFxf8p23JlDzDhvX5nA6iThyA
RN0An2K9im/Mzd5n3x4AP2B2NAP9NIAF
=YI/V
-----END PGP SIGNATURE-----