Linux memory over-commit. Previously I've only read about it, but now I am bitten by it. When cl-test-grid agent (a CCL program) starts ECL and ECL starts gcc, linux on my VPS kills someone of them (usually CCL). Sometimes it just doesn't allow to fork new process. Why do CCL and other Lisps allocate so large heap at startup?
Did you disable overcommit?
ReplyDeleteCCL reserves a large address space but doesn't actually touch it unless needed.
If that bothers you a LOT, you could replace that by mmap'ing /dev/zero with mprotect and remapping to anonymous pages on demand - hopefully Linux is clever enough to merge the remapped areas.
Yes, I've tried to disable overcommit, but then certain things doesn't work even before I start ECL - namely CMUCL doesn't work.
ReplyDeleteI am just a little bit curious, why CCL and other programs allocate that large heap at risk of being killed, instead of growing heap incrementally during runtime and garbage collecting when memory is not available (instead of being killed).
BTW, I have worked around the issue by restricting CCL heap size using --heap-reserve 58M option. With that option it seems to survive together with ECL and gcc running in parallel.
ReplyDelete