autogenerate lunatik config from makefile flags #400

abhijeetw035 · 2026-01-18T16:28:45Z

Closes #396

This PR removes the hardcoded module list in bin/lunatik and replaces it with an auto-generated configuration file created during the build process.

Changes

New Script (genconfig.lua): Parses CONFIG_LUNATIK_* flags to generate autogen/lunatik/config.lua containing the modules table and kernel release version.
Makefile: Added a config target to execute the generator. The configuration flags are passed explicitly to the script to avoid modifying the all target variables
Updated bin/lunatik to require("lunatik.config") instead of using a static list.
Git Handling: Added autogen/lunatik/.gitignore to ensure the directory structure exists on fresh clones while ignoring generated artifacts.

I included an ignore list in genconfig.lua to filter out non-module flags (e.g., INSTALL_EXAMPLES, INSTALL_TESTS) introduced in pending PR #387.

lneto · 2026-01-18T17:10:53Z

Makefile

+	"CONFIG_LUNATIK=m CONFIG_LUNATIK_RUN=m CONFIG_LUNATIK_RUNTIME=y CONFIG_LUNATIK_DEVICE=m \
+	CONFIG_LUNATIK_LINUX=m CONFIG_LUNATIK_NOTIFIER=m CONFIG_LUNATIK_SOCKET=m \
+	CONFIG_LUNATIK_RCU=m CONFIG_LUNATIK_THREAD=m CONFIG_LUNATIK_FIB=m \
+	CONFIG_LUNATIK_DATA=m CONFIG_LUNATIK_PROBE=m CONFIG_LUNATIK_SYSCALL=m \
+	CONFIG_LUNATIK_XDP=m CONFIG_LUNATIK_FIFO=m CONFIG_LUNATIK_XTABLE=m \
+	CONFIG_LUNATIK_NETFILTER=m CONFIG_LUNATIK_COMPLETION=m \
+	CONFIG_LUNATIK_CRYPTO_SHASH=m CONFIG_LUNATIK_CRYPTO_SKCIPHER=m \
+	CONFIG_LUNATIK_CRYPTO_AEAD=m CONFIG_LUNATIK_CRYPTO_RNG=m \
+	CONFIG_LUNATIK_CRYPTO_COMP=m CONFIG_LUNATIK_CPU=m CONFIG_LUNATIK_HID=m \
+	CONFIG_LUNATIK_SIGNAL=m CONFIG_LUNATIK_BYTEORDER=m"


you shouldn't redefine them here, but access this flags from your lua script.. perhaps we could use CPP for this..

lneto · 2026-01-18T17:11:05Z

genconfig.lua

+    file:write(string.format('\t"%s",\n', mod))
+end
+file:write("}\n\n")
+file:write("return config\n")


please.. mind \n

autogen/lunatik/.gitignore

lneto · 2026-01-18T17:12:44Z

genconfig.lua

@@ -0,0 +1,61 @@
+-- SPDX-FileCopyrightText: (c) 2023-2026 Ring Zero Desenvolvimento de Software LTDA


it should be your copywrite.. if you are borrowing code from the other script, I recommend you properly modularize it, moving common code to another file and just require it..

abhijeetw035 · 2026-01-18T17:43:08Z

Thanks for the feedback @lneto !

I will update the header to use my name in genconfig.lua.
Modularization: I checked PR Auto generate network constants for lua (cont) #395. Since gendefines.lua doesn't exist in my branch yet, I can't easily extract common code (like the exit function) into a shared utility file right now. I suggest we merge one of these PRs first, and then do a quick follow-up PR to refactor/deduplicate the common logic?
CPP / Duplication: I see that gendefines.lua uses cpp -dM to parse C headers. However, for this task, the input data is the list of Makefile variables (CONFIG_LUNATIK_*), which don't exist in any C header file until the compiler command runs.

Since we can't move these flags to a Makefile variable (to avoid Kconfig breakage), and the Preprocessor can't read the Makefile directly ..... like both ways not possible ... I'm unsure how to use CPP here to avoid the duplication.

so.... would you prefer I stick to the current approach (explicitly passing the flags) as a temporary solution until the Kconfig PR merges (which will give us a .config file to parse)? or do you have a specific way in mind to pipe these Makefile vars to CPP?

lneto · 2026-01-18T18:24:58Z

The problem here is not only to avoid duplication, but redefinition of the flags values.. think about the case you disable some module on make menuconfig, it should reflect the autogen. For using CPP, you could have an "artificial" header using such flags.. however, ideally, we should simply export this flags from the build system to your script.. I'm not sure, but perhaps we can access a .config file or something.. or access such values as a environment var.. it needs more investigation, I think

abhijeetw035 · 2026-01-19T13:47:08Z

The problem here is not only to avoid duplication, but redefinition of the flags values.. think about the case you disable some module on make menuconfig, it should reflect the autogen. For using CPP, you could have an "artificial" header using such flags.. however, ideally, we should simply export this flags from the build system to your script.. I'm not sure, but perhaps we can access a .config file or something.. or access such values as a environment var.. it needs more investigation, I think

I dug into this a bit. I checked for artifacts, but no .config or auto.conf files are generated in the directory. It seems the flags only exist transiently on the command line while make is running and aren't saved anywhere.

So right now, there isn't really a persistent file for the script to read from.

lneto · 2026-01-19T15:16:02Z

hey @abhijeetw035 I took a look at this.. I think we should have two separate approaches.. one for the off-tree build, that is more or less what you already doing.. but assigning the value of the Makefile variable (instead of redefining it).. e.g.,

CONFIG_LUNATIK_LINUX=$(CONFIG_LUNATIK_LINUX), instead of CONFIG_LUNATIK_LINUX=m. I hope that the difference is clear..

Another think is the Kconfig, for this, we should assume it will be build on-tree and leverage /lib/modules/$(uname -r)/build/include/generated/autoconf.h, which should take precedence over the Makefile flags. I would split this issue in two and left Kconfig/autoconf as a next step. What do you think?

abhijeetw035 · 2026-01-20T05:02:15Z

That sounds good ! will handle the off-tree logic now, and we can look at the Kconfig / autoconf side in a follow-up once there’s some support for that.

I will refactor the Makefile to use variable references as u suggested so it correctly respects user overrides

Pushing those changes shortly!

abhijeetw035 · 2026-01-20T05:30:55Z

I've refactored the Makefile

Moved all flags to top-level variables using ?= m and confirmed that running make CONFIG_LUNATIK_SIGNAL=n correctly removes the module from the generated config

Makefile

lneto · 2026-01-23T22:36:51Z

hey @abhijeetw035 can you test this along with our OpenWRT feed? My main concern now is that these autogen script don't break it. Thanks! Let me know if you need help.

lneto · 2026-01-24T12:09:58Z

hey @abhijeetw035 can you test this along with our OpenWRT feed? My main concern now is that these autogen script don't break it. Thanks! Let me know if you need help.

most specifically, you should call your autogen script here https://github.com/luainkernel/openwrt_feed/blob/openwrt-23.05/kernel/lunatik/Makefile#L127.

abhijeetw035 · 2026-01-24T16:11:27Z

I just simulated the OpenWRT build locally by feeding the exact EXTRA_KCONFIG string from the feed into genconfig.lua.
It works perfectly
modules set to =n (like PROBE and SYSCALL in your example) were correctly excluded from the generated list.

So, once this is merged, you can safely update the OpenWRT feed to call this script in Build/Prepare (line 127) passing $(EXTRA_KCONFIG), and it will generate the correct config.lua.

lneto · 2026-01-24T16:40:40Z

I just simulated the OpenWRT build locally by feeding the exact `EXTRA_KCONFIG` string from the feed into genconfig.lua. It works perfectly modules set to =n (like `PROBE` and `SYSCALL` in your example) were correctly excluded from the generated list.
So, once this is merged, you can safely update the OpenWRT feed to call this script in Build/Prepare (line 127) passing $(EXTRA_KCONFIG), and it will generate the correct config.lua.

it's not enough, we should run it using the OpenWRT build

lneto · 2026-01-24T16:42:41Z

...
it's not enough, we should run it using the OpenWRT build

see this discussion, for instance, #387

abhijeetw035 · 2026-01-24T18:46:17Z

...
it's not enough, we should run it using the OpenWRT build

see this discussion, for instance, #387

Suree will go through it and update you.

lneto · 2026-01-24T18:56:33Z

...
it's not enough, we should run it using the OpenWRT build

see this discussion, for instance, #387

Suree will go through it and update you.

Well, I will need to work on this for #395. Perhaps you want to put this issue on hold until we merge the other and work on another issue for a while. It might be better usage of both of our time. What do you think?

abhijeetw035 · 2026-01-24T19:01:20Z

...
it's not enough, we should run it using the OpenWRT build

see this discussion, for instance, #387

Suree will go through it and update you.

Well, I will need to work on this for #395. Perhaps you want to put this issue on hold until we merge the other and work on another issue for a while. It might be better usage of both of our time. What do you think?

yes suree ....it makes sense to synchronize the OpenWRT work since both PRs touch the same integration points.

lneto · 2026-01-25T23:32:05Z

genconfig.lua

+local KERNEL_RELEASE = arg[1]
+local OUTPUT_DIR = arg[2]
+local CONFIGS = arg[3]
+
+if not KERNEL_RELEASE or not OUTPUT_DIR or not CONFIGS then
+	exit("usage: lua genconfig.lua <KERNEL_RELEASE> <OUTPUT_DIR> <CONFIGS>")
+end


the other PR is merged.. let's avoid replicate code and sharing the common code

mabe a better approach is just to insert your logic on gendefines.lua, se we will have a single entry point on the Makefile

@abhijeetw035 it's weird, it looks like you are re-adding the changes that are already in the base.. did you rebase your branch on master?

@abhijeetw035 it's weird, it looks like you are re-adding the changes that are already in the base.. did you rebase your branch on master?

that was an artifact from squashing the merge commit. I have rebased onto origin/master to filter out the upstream changes. The diff should be clean now!

Please, see this #395 (comment).

We don't use merge commits.. --ff-only

@abhijeetw035 it's weird, it looks like you are re-adding the changes that are already in the base.. did you rebase your branch on master?

fixing it ...

lneto · 2026-01-26T16:20:36Z

gendefines.lua

@@ -1,5 +1,6 @@
 --
 -- SPDX-FileCopyrightText: (c) 2026 Ashwani Kumar Kamal <ashwanikamal.im421@gmail.com>
+-- SPDX-FileCopyrightText: (c) 2026 Abhijeet Waghmare <abhijeetw035@gmail.com>


We usually don't do this, we keep the original copyright unless there's very significant change in the original, such as major refactor. Which is not the case of adding new features. If you want to modularize it and move the config portion to a sublib, it's alright to add your copyright to the new file.

We could have autogen/{init,defines,config}.lua for instance

I've rebased onto the latest master and squashed everything into a single commit to keep the history linear.
Changes:
copyright: removed my copyright header from gendefines.lua .
runtime: updated bin/lunatik to load the generated config dynamically (added /lib/modules/lua to package path).

Regarding the modularization (autogen/{defines,config}.lua): I agree that is a cleaner architecture. however, to keep this PR focused and safe, i think we should merge this functional version first. I can then submit a follow-up PR dedicated to splitting the scripts and organizing the autogen/ structure properly. what do you think?

lneto · 2026-01-26T21:57:31Z

I noticed the CI build is failing with make: /bin/sh: 1: lua5.4: not found.

It seems the GitHub Actions runner doesn't have lua5.4 installed. what can we do for now ?

I don't think it's the problem as master (and other branches are alright with CI build)..

Makefile

lneto · 2026-01-26T21:59:37Z

Makefile

+
+defines:
+	@mkdir -p autogen/lunatik autogen/linux
+	CC='${CC}' lua gendefines.lua "$(KERNEL_RELEASE)" "$(INCLUDE_PATH)" "autogen/linux" "$(LUNATIK_CONFIG_FLAGS)"


this should be lua5.4

yes i was checking if the lua works .like before ci build was failing due to that
/bin/sh: 1: lua5.4: not found

Makefile

bin/lunatik

gendefines.lua

lneto · 2026-01-26T22:46:38Z

gendefines.lua

+        exit("cannot open " .. filepath .. ": " .. err)
+    end
+
+    file:write("-- auto-generated, do not edit\n")


we should avoid repeat this, instead we should proper modularize the code base.

yes.... I will submit a follow-up PR immediately after this merges to split gendefines.lua into proper modules. I kept it inline here to keep this PR focused on the feature.

my point is that we should at least avoid to repeat strings that can be added to a local var or very simple patterns that can be added to a local function..

abhijeetw035 · 2026-01-27T05:58:52Z

I noticed the CI build is failing with make: /bin/sh: 1: lua5.4: not found.
It seems the GitHub Actions runner doesn't have lua5.4 installed. what can we do for now ?

I don't think it's the problem as master (and other branches are alright with CI build)..

I checked the build.yaml file and I see the issue.

The workflow installs lua5.4-dev, which provides the headers for compiling the C module (which is why master works). However, it does not explicitly install the lua5.4 package, which provides the CLI interpreter.

Since my PR requires running a Lua script during the build, the runner fails to find the binary.

lneto · 2026-01-27T11:20:11Z

Makefile


 defines:
-	${shell CC='$(CC)' lua5.4 gendefines.lua $(KERNEL_RELEASE) $(INCLUDE_PATH) autogen/linux}
+	@mkdir -p autogen/lunatik autogen/linux


we shouldn't do this.. it's handled by the .gitignore

lneto · 2026-01-27T11:25:41Z

Makefile

 defines:
-	${shell CC='$(CC)' lua5.4 gendefines.lua $(KERNEL_RELEASE) $(INCLUDE_PATH) autogen/linux}
+	@mkdir -p autogen/lunatik autogen/linux
+	CC='${CC}' lua5.4 gendefines.lua "$(KERNEL_RELEASE)" "$(INCLUDE_PATH)" "autogen/linux" "$(LUNATIK_CONFIG_FLAGS)"


why did you remove ${shell ...}? this seems to be CI issue to me.

lneto · 2026-01-27T11:30:10Z

gendefines.lua

+			if key == "CONFIG_LUNATIK" then
+				name = "lunatik"
+			elseif key == "CONFIG_LUNATIK_RUN" then
+				name = "lunatik_run"


you can use the same approach of ignore_list here.. having an overlay_list (couldn't figure a better name)..

lneto · 2026-01-27T11:30:43Z

gendefines.lua

+	local modules = {}
+
+	for token in configs:gmatch("%S+") do
+		local key = token:match("(CONFIG_[%w_]+)=[my]")


let's avoid use key as a variable name and have more meaningful names instead..

lneto · 2026-01-27T11:32:44Z

I noticed the CI build is failing with make: /bin/sh: 1: lua5.4: not found.
It seems the GitHub Actions runner doesn't have lua5.4 installed. what can we do for now ?

I don't think it's the problem as master (and other branches are alright with CI build)..

I checked the build.yaml file and I see the issue.

The workflow installs lua5.4-dev, which provides the headers for compiling the C module (which is why master works). However, it does not explicitly install the lua5.4 package, which provides the CLI interpreter.

Since my PR requires running a Lua script during the build, the runner fails to find the binary.

the defines

I noticed the CI build is failing with make: /bin/sh: 1: lua5.4: not found.
It seems the GitHub Actions runner doesn't have lua5.4 installed. what can we do for now ?

I don't think it's the problem as master (and other branches are alright with CI build)..

I checked the build.yaml file and I see the issue.

The workflow installs lua5.4-dev, which provides the headers for compiling the C module (which is why master works). However, it does not explicitly install the lua5.4 package, which provides the CLI interpreter.

Since my PR requires running a Lua script during the build, the runner fails to find the binary.

master is already running gendefines.lua on build.. I think the problem is to suppress ${shell ...}; please, be more careful when removing or moving code around.

lneto · 2026-01-27T23:07:24Z

gendefines.lua

+		end
+	end
+
+	local filepath = "autogen/lunatik/config.lua"


This should come from the args, like on defines. You coul pass autogen dir and concat subdir on both cases.

lneto · 2026-01-27T23:08:51Z

gendefines.lua

+local CONFIG_FLAGS = arg[4]

 if not KERNEL or not INCLUDE_PATH or not OUTPUT_DIR then
 	exit("usage: lua gendefines.lua <KERNEL> <INCLUDE_PATH> <OUTPUT_DIR>")


It should be updated with CONFIG_FLAGS

lneto · 2026-01-27T23:20:20Z

gendefines.lua

 	write_module(spec, constants, macro_names)
 end

+if CONFIG_FLAGS then


it should be checked along with the other args.. it should be a requirement..

lneto · 2026-01-28T21:45:18Z

@sneaky-potato can you take a look at this as well? thanks!

sneaky-potato · 2026-01-29T11:37:02Z

gendefines.lua

+	file:write("return config\n")
+end
+
+local linux_dir = OUTPUT_DIR .. "/linux"


Suggested change

local linux_dir = OUTPUT_DIR .. "/linux"

sneaky-potato · 2026-01-29T11:37:20Z

gendefines.lua

 	local cpp_output = preprocess(spec.header)
 	local constants, macro_names = collect_constants(cpp_output, spec.prefix)
-	write_module(spec, constants, macro_names)
+	write_module(spec, constants, macro_names, linux_dir)


Suggested change

write_module(spec, constants, macro_names, linux_dir)

write_module(spec, constants, macro_names, OUTPUT_DIR)

To keep the script uniform, let's pass the same base directory to both write_module and generate_build_config

sneaky-potato · 2026-01-29T11:40:02Z

gendefines.lua

+local function write_module(spec, constants, macro_names, dir)
+	local filepath = string.format("%s/%s.lua", dir, spec.module_name)


Suggested change

local function write_module(spec, constants, macro_names, dir)

local filepath = string.format("%s/%s.lua", dir, spec.module_name)

local function write_module(spec, constants, macro_names, output_dir)

local filepath = string.format("%s/linux/%s.lua", output_dir, spec.module_name)

see https://groups.google.com/g/lua-l/c/baGBk9MiXrA/m/VzKLZYO6AgAJ

lneto · 2026-01-29T22:11:35Z

@sneaky-potato, it seems @abhijeetw035 has committed your suggestions. Please, let me know if we can merge it. Thanks!

sneaky-potato

LGTM, we can merge this

abhijeetw035 marked this pull request as ready for review January 18, 2026 16:28

lneto reviewed Jan 18, 2026

View reviewed changes

autogen/lunatik/.gitignore Show resolved Hide resolved

lneto reviewed Jan 18, 2026

View reviewed changes

abhijeetw035 force-pushed the autogen-config branch from c6f286f to edb2722 Compare January 20, 2026 05:28

lneto reviewed Jan 20, 2026

View reviewed changes

Makefile Show resolved Hide resolved

abhijeetw035 force-pushed the autogen-config branch 4 times, most recently from 1f27a66 to 7e4df5f Compare January 20, 2026 17:33

abhijeetw035 requested a review from lneto January 21, 2026 11:13

lneto reviewed Jan 25, 2026

View reviewed changes

abhijeetw035 force-pushed the autogen-config branch 2 times, most recently from 7c23a62 to 6c1535f Compare January 26, 2026 16:13