Search code examples
moduleluamodularityreadabilitycode-readability

Lua Spaghetti Modules

I am currently developing my own programming language. The codebase (in Lua) is composed of several modules, as follows:

  • The first, error.lua, has no dependancies;
  • lexer.lua depends only on error.lua;
  • prototypes.lua also has no dependancies;
  • parser.lua, instead, depends on all the modules above;
  • interpreter.lua is the fulcrum of the whole codebase. It depends on error.lua, parser.lua, and memory.lua;
  • memory.lua depends on functions.lua;
  • finally, functions.lua depends on memory.lua and interpreter.lua. It is required from inside memory.lua, so we can say that memory.lua also depends on interpreter.lua.

With "A depends on B" I mean that the functions declared in A need those declared in B.

The real problem, though, is when A depends on B which depends on A, which, as you can understand from the list above, happens quite frequently in my code.

To give a concrete example of my problem, here's how interpreter.lua looks like:

--first, I require the modules that DON'T depend on interpreter.lua
local parser, Error = table.unpack(require("parser"))
--(since error.lua is needed both in the lexer, parser and interpreter module,
--I only actually require it once in lexer.lua and then pass its result around)

--Then, I should require memory.lua. But since memory.lua and
--functions.lua need some functions from interpreter.lua to work, I just
--forward declare the variables needed from those functions and then those functions themself:

--forward declaration
local globals, new_memory, my_nil, interpret_statement

--functions I need to declare before requiring memory.lua
local function interpret_block()
    --uses interpret_statement and new_memory
end
local function interpret_expresion()
    --uses new_memory, Error and my_nil
end

--Now I can safely require memory.lua:
globals, new_memory, my_nil = require("memory.lua")(interpret_block, interpret_espression)
--(I'll explain why it returns a function to call later)

--Then I have to fulfill the forward declaration of interpret_executement:
function interpret_executement()
    --uses interpret_expression, new_memory and Error
end

--finally, the result is a function
return function()
    --uses parser, new_fuction and globals
end

The memory.lua module returns a function so that it can receive interpret_block and interpret_expression as arguments, like this:

--memory.lua
return function(interpret_block, interpret_expression)
    --declaration of globals, new_memory, my_nil
    return globals, new_memory, my_nil
end

Now, I got the idea of the forward declarations here and that of the functions-as-modules (like in memory.lua, to pass some functions from the requiring module to the required module) here. They're all great ideas, and I must say that they work greatly. But you pay in readability.

In fact, breaking in smaller pieces the code this time made my work harder that it would have been if I coded everything in a single file, which is impossible for me because it's over than 1000 lines of code and I'm coding from a smartphone.

The feeling I have is that of working with spaghetti code, only on a larger scale.

So how could I solve the problem of my code being ununderstandable because of some modules needing each other to work (which doesn't involve making all the variables global, of course)? How would programmers in other languages solve this problem? How should I reorganize my modules? Are there any standard rules in using Lua modules that could also help me with this problem?

Solution

  • If we look at your lua files as a directed graph, where a vertice points from a dependency to its usage, the goal is to modify your graph to be a tree or forest, as you intend to get rid of the cycles.

    A cycle is a set of nodes, which, traversed in the direction of the vertices can reach the starting node.

    Now, the question is how to get rid of cycles?

    The answer looks like this:

    Let's consider node N and let's consider {D1, D2, ..., Dm} as its direct dependencies. If there is no Di in that set that depends on N either directly or indirectly, then you can leave N as it is. In that case, the set of problematic dependencies looks like this: {}

    However, what if you have a non-empty set, like this: {PD1, ..., PDk} ?

    You then need to analyze PDi for i between 1 and k along with N and see what is the subset in each PDi that does not depend on N and what is the subset of N which does not depend on any PDi. This way you can define N_base and N, PDi_base and PDi. N depends on N_base, just like all PDi elements and PDi depends on PDi_base along with N_base.

    This approach minimalizes circles in the dependency tree. However, it is quite possible that a function set of {f1, ..., fl} exists in this group which cannot be migrated into _base as discussed due to dependencies and there are still cycles. In this case you need to give a name to the group in question, create a module for it and migrate all to functions into that group.