Makefile.base: cleanup non selected source object files [backport 2021.10]

[leandrolanzieri]

Backport of #16945 and #16981

Contribution description

As explained in #16942, since #14754 we select all object files when linking, which is a problem when the selection of module sources change between two builds. This adds a cleanup for non selected source object files on modules.

Testing procedure

A minimal example that shows the current issue:

diff --git a/tests/external_module_dirs/Makefile b/tests/external_module_dirs/Makefile
index f44d1e3e8e..f4a82e6c16 100644
--- a/tests/external_module_dirs/Makefile
+++ b/tests/external_module_dirs/Makefile
@@ -4,4 +4,6 @@ USEMODULE += random
 USEMODULE += external_module
 EXTERNAL_MODULE_DIRS += external_modules
 
+USEMODULE += external_module_implementation_a
+
 include $(RIOTBASE)/Makefile.include
diff --git a/tests/external_module_dirs/external_modules/external_module/Makefile b/tests/external_module_dirs/external_modules/external_module/Makefile
index 48422e909a..a0fd02f4b4 100644
--- a/tests/external_module_dirs/external_modules/external_module/Makefile
+++ b/tests/external_module_dirs/external_modules/external_module/Makefile
@@ -1 +1,9 @@
+# enable submodules
+SUBMODULES := 1
+
+# base module name
+BASE_MODULE := external_module
+
+SRC += external_module.c
+
 include $(RIOTBASE)/Makefile.base
diff --git a/tests/external_module_dirs/external_modules/external_module/Makefile.include b/tests/external_module_dirs/external_modules/external_module/Makefile.include
index b6d95d22b5..d7225f8d78 100644
--- a/tests/external_module_dirs/external_modules/external_module/Makefile.include
+++ b/tests/external_module_dirs/external_modules/external_module/Makefile.include
@@ -1,3 +1,6 @@
 # Use an immediate variable to evaluate `MAKEFILE_LIST` now
 USEMODULE_INCLUDES_external_module := $(LAST_MAKEFILEDIR)/include
 USEMODULE_INCLUDES += $(USEMODULE_INCLUDES_external_module)
+
+PSEUDOMODULES += external_module_implementation_a
+PSEUDOMODULES += external_module_implementation_b
diff --git a/tests/external_module_dirs/external_modules/external_module/implementation_a.c b/tests/external_module_dirs/external_modules/external_module/implementation_a.c
new file mode 100644
index 0000000000..19c48d0020
--- /dev/null
+++ b/tests/external_module_dirs/external_modules/external_module/implementation_a.c
@@ -0,0 +1,2 @@
+
+char *message_implementation = "Linked implementation A";
diff --git a/tests/external_module_dirs/external_modules/external_module/implementation_b.c b/tests/external_module_dirs/external_modules/external_module/implementation_b.c
new file mode 100644
index 0000000000..8b126371e8
--- /dev/null
+++ b/tests/external_module_dirs/external_modules/external_module/implementation_b.c
@@ -0,0 +1,2 @@
+
+char *message_implementation = "Linked implementation B";
diff --git a/tests/external_module_dirs/external_modules/external_module/include/external_module.h b/tests/external_module_dirs/external_modules/external_module/include/external_module.h
index f358ef4786..aefd12679e 100644
--- a/tests/external_module_dirs/external_modules/external_module/include/external_module.h
+++ b/tests/external_module_dirs/external_modules/external_module/include/external_module.h
@@ -29,6 +29,8 @@ extern "C" {
  */
 extern char *external_module_message;
 
+extern char *message_implementation;
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tests/external_module_dirs/main.c b/tests/external_module_dirs/main.c
index 9695fb6826..421438a0c3 100644
--- a/tests/external_module_dirs/main.c
+++ b/tests/external_module_dirs/main.c
@@ -34,5 +34,6 @@ int main(void)
 {
     puts("If it compiles, it works!");
     printf("Message: %s\n", external_module_message);
+    printf("Submodule message: %s\n", message_implementation);
     return 0;
 }

Compile once with USEMODULE += external_module_implementation_a and once with USEMODULE += external_module_implementation_b, message_implementation will have two implementations at link time on master. With this PR this should be fixed.

Issues/PRs references

#16942
Issue introduced by #14754

---

Contribution description

Since #16945 we cleanup unneeded object files. The list of "needed object files" is defined based on present source files, but bindist is a special case, as bindist objects should always be preserved. As described in #16977, the bindist example fails to link due to the cleanup. This PR avoids cleaning bindist modules object files when building.

Testing procedure

• examples/bindist should work (note that the binary check does not pass for native at least since the linking was fixed in build system: link object files #14754)
• The testing procedure from Makefile.base: cleanup non selected source object files #16945 should pass

Issues/PRs references

Fixes #16977
Issue introduced in #16945

[benpicco]

This needs a rebase

[leandrolanzieri]

Rebased

[benpicco]

Oh sorry, the branch has become out of date again.
Please rebase yet again - feel free to hit merge as soon as you see the green button.

[leandrolanzieri]

Let's wait until #16977 is sorted.

[leandrolanzieri]

@benpicco I rebased and cherry-picked the commit from #16981