Gdb ide integration

Makefile

@@ -2,7 +2,7 @@
 all: all-examples
 
 EXAMPLES = ex/example1 ex/example2 ex/example3 ex/example4 \
-	ex/example5 ex/example6 ex/example7 ex/example8 \
+	ex/example5 ex/example6 ex/example7 ex/example8 ex/example11 ex/main-wrap \
 	'ex ws/example1 ws' 'ex ws/example2 ws' 'ex ws/example3 ws' 'ex ws/example4 ws' \
 	'ex ws/example5 ws' 'ex ws/example6 ws' 'ex ws/example7 ws' 'ex ws/example8 ws'
 

TODO.md

@@ -0,0 +1,148 @@
+# Todo (nick)
+
+- [x] Look if possible to invoke task from launch.json, and shows in button.
+- > conclusion: Not possible to add button, or custom "Run" task: possible to just run it 
+- [x] Determine if file is implementation or test (in tests directory)
+- [x] CLion debugging
+
+## 19.04
+
+- [/] investigate files appearing when running run configurations
+- [/] analyze problem of shell scripts (when/where interpreter is chosen)
+- [/] look workings with shell/IO tests
+
+- If time, generate automatically tests
+  - ~ CLI to generate folder and file structure
+  - lookup Unity C
+
+-> Ideas: m4 to fill up boilerplate
+  ```m4
+  include(bt.m4)      // implicit
+
+  Suite(name)         // sets current suite/file to generate
+  Test_compile()
+  Test_alloc_ok(type) // type is the type declaration
+
+  ```
+-> Nope
+
+## 26.04
+
+- [ ] look workings with shell/IO tests
+  - Run can just run everything
+  - Debug has to make choiches, so better to have a debug target handling it?:
+    - make debug file.ext
+    - > if ext == c; then compile, cp to debugme, and run it (ez, done);
+      > if ext == in | expected; then needs some input redirection (hard);
+      > if ext == sh; then needs to run the program as in the shell script?? (Hardest);
+- [x] investigate files appearing when running run configurations
+  - ? touch only with `if [[ -f ... ]]; then touch ... fi;`?
+  - touch $fileName$ can create file in projectDir directory;
+  -> TODO: should all be from same dir as makefile, instead of projectDir?
+- [ ] analyze problem of shell scripts (when/where interpreter is chosen)
+- [x] VScode not showning tasks anymore (ex/example11)
+  -> .vscode or .idea folder must be in root of the project 
+
+## 10.05
+
+- Will there ever be test sets with more than one of IO / sh / c-cxx? 
+  - (y/n): (__**Kinda, shell could also be IO, but all confined in .sh file**__)
+  - Specific run configuration file for each type of set?
+  - makefile to make all decisions, based on `TESTS_(IO|SH|C|CXX)` variables?
+
+- Types
+  - Shell: Used to test/run binary implementation with specific arguments (e.g cli), (!?!)or other uses?
+  - I/O: Used to test/run binary implementation, with specific stdin, and expected output
+  - c/cpp src files: Used to test library implementations
+
+- Debugging problems:
+  - Library: None, just copy the test
+  - Binary programs: 
+    - how to select the correct compiled binary, to copy into a file named `debugme`?
+      - Need to open src file anyway?
+      - Makefile script to choose when project is library, or test suite, so which compiled binary to copy. 
+    - usually I/O or shell, so see below problems
+  - Shell: 
+    - how to get the arguments present in the shell script, to pass to the binary?
+  - I/O: choose which .in file to run
+    - Manual prompt?
+
+
+- Proposal:
+  - Common `Run all` configuration
+    - CLion: Run configuration (Button)
+    - VScode: Run task, instructions needed for keyboard shortcut
+      - <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>B</kbd>
+  - Specific debug configuration, for each test suite type (either)
+    - Library: currently open `test.c` or `test.cc` file
+    - IO: prompt to pickup file for stdin redirect
+      - Clion: either file, or name prompt
+      - VScode: can be configurable/generetable, by adding a specific field at the end of the file
+
+### TODO
+
+- [ ] Stabilize current solution
+  - Check which files are necessary to make everything work
+  - check that current stuff works
+- [ ] explore debugging for sh files
+- [x] complete section 1 of report, start section 2
+
+## 17.05
+
+- Linker: wrap (dll-library to substitute symbols)
+  - anche per test IO
+    - studente crea un main
+    - altro programma con "vero" main, va a chiamare main altro!
+    - linker:`--wrap = main`
+    - Take every undefined reference to symbol, and transform in symbol `__wrap_symbol_`
+    - => define wrapper 
+  - IO: intercept stodut of program: more difficult
+  - New version: capture IO functions (mostly O), for pinpoint comparison!
+  From LD manual:
+```txt --wrap=symbol
+Use a wrapper function for symbol.  Any undefined reference to
+symbol will be resolved to "__wrap_symbol".  Any undefined
+reference to "__real_symbol" will be resolved to symbol.
+
+This can be used to provide a wrapper for a system function.  The
+wrapper function should be called "__wrap_symbol".  If it wishes to
+call the system function, it should call "__real_symbol".
+
+Here is a trivial example:
+
+  void *
+  __wrap_malloc (size_t c)
+  {
+    printf ("malloc called with %zu\n", c);
+    return __real_malloc (c);
+  }
+
+If you link other code with this file using --wrap malloc, then all
+calls to "malloc" will call the function "__wrap_malloc" instead.
+The call to "__real_malloc" in "__wrap_malloc" will call the real
+"malloc" function.
+
+You may wish to provide a "__real_malloc" function as well, so that
+links without the --wrap option will succeed.  If you do this, you
+should not put the definition of "__real_malloc" in the same file
+as "__wrap_malloc"; if you do, the assembler may resolve the call
+before the linker has a chance to wrap it to "malloc".
+
+Only undefined references are replaced by the linker.  So,
+translation unit internal references to symbol are not resolved to
+"__wrap_symbol".  In the next example, the call to "f" in "g" is
+not resolved to "__wrap_f".
+
+  int
+  f (void)
+  {
+    return 123;
+  }
+
+  int
+  g (void)
+  {
+    return f();
+  }
+```
+Order of compilation may matter.

basic_testing.h

@@ -448,4 +448,26 @@ int bt_test_driver(int argc, char * argv[]) {
     }
 #endif
 
+// IO tests macros
+#define FILENAME(ext) \
+   char ext##_filename[128] = {0}; \
+   strncat(ext##_filename, __FILE__, strlen(__FILE__)-1); \
+   strcat(ext##_filename, #ext)
+
+#define RUN_PROGRAM(...)             \
+    extern int __real_main(int, char*[]);   \
+    int __wrap_main() {                     \
+        FILENAME(in);                       \
+        FILENAME(out);                      \
+        FILENAME(expected);                 \
+        BT_POSSIBLY_UNUSED FILE* in_file = freopen(in_filename, "r", stdin);    \
+        BT_POSSIBLY_UNUSED FILE* out_file = freopen(out_filename, "w", stdout); \
+        int result = __real_main(2, (char*[]) { __VA_ARGS__ });  \
+        if (in_file != NULL) fclose(in_file);    \
+        fclose(out_file);   \
+		if (result != 0) {	\
+			fprintf(stderr, "Implementation exited with value: %d", result);} \
+        return result;      	\
+    }
+
 #endif /* BASIC_TESTING_H_INCLUDED */

basic_testing.mk

@@ -21,6 +21,8 @@ TESTS_BIN:=$(patsubst $(TESTS_DIR)/%.c, $(TESTS_DIR)/%, $(TESTS_C)) \
 	   $(patsubst $(TESTS_DIR)/%.cc, $(TESTS_DIR)/%, $(TESTS_CXX))
 TESTS_BIN_NAMES:=$(sort $(patsubst $(TESTS_DIR)/%.c, %, $(TESTS_C)) $(patsubst $(TESTS_DIR)/%.cc, %, $(TESTS_CXX)))
 
+TESTS_HYBRID:=$(wildcard $(TESTS_DIR)/*.expected)
+
 .PHONY: all
 all: compile check
 
@@ -49,7 +51,12 @@ SUPPRESS_DIAGNOSTICS=yes
 else
 SUPPRESS_DIAGNOSTICS=
 endif
+
 TEST_COLORS=yes
+# $TERM variable not always defined, but required by tput (Eg, CLion run console).
+ifeq ($(TERM),)
+TEST_COLORS=no
+endif
 ifeq ($(TEST_COLORS),no)
 COLOR_RED :=
 COLOR_GREEN :=
@@ -171,7 +178,7 @@ check-io-sh: compile $(TESTS_IO) $(TESTS_SH) $(PROGRAMS_DRIVERS)
 			else \
 				test_ko FAIL ;\
 				test_diag "see $(TESTS_DIR)/$$t.sh" ;\
-				test_diag "run diff $$t.out $(TESTS_DIR)/$$t.expected";\
+				test_diag "run diff -c $$t.out $(TESTS_DIR)/$$t.expected";\
 				test_diag "to see the difference between the actual and expected output";\
 			fi; \
 		fi; \
@@ -199,19 +206,50 @@ check-bin: $(TESTS_BIN)
 			"$(TESTS_DIR)/$$t" -q &\
 		fi; \
 		$(SCRIPT_GET_TEST_RESULT); \
+		if test $$res = KO || test "$(TESTS_HYBRID)" != "" && ! cmp -s "$(TESTS_DIR)/$$t.out" "$(TESTS_DIR)/$$t.expected"; then \
+			if test "$(TESTS_HYBRID)" != ""; then \
+				test_ko FAIL ; \
+				test_diag "run 'diff -c $(TESTS_DIR)/$$t.out $(TESTS_DIR)/$$t.expected' to see the mistakes"; \
+			else \
+				test_diag "run '$(TESTS_DIR)/$$t' to see what went wrong"; \
+			fi; \
+			test_diag "run '$(TESTS_DIR)/$$t -d' with a debugger" ; \
+		else \
+			test_ok PASS; \
+			if test ! -z $$TESTS_HYBRID; then rm -f "$(TESTS_DIR)/$$t.out"; fi; \
+		fi; \
+	done; \
+	test_summary 'Summary: PASS '
+
+.PHONY: check-single-bin
+check-single-bin: $(BIN_NAME)
+	@exec 2> /dev/null; \
+	if test -z $$BIN_NAME; then \
+		echo "Error: Missing test to run, please set BIN_NAME=<path/to/binary>";  \
+		echo "  example: \'make check-single-bin BIN_NAME=tests/test0\'"; \
+		exit 1; fi; \
+	$(SCRIPT_INIT); \
+	t=$$BIN_NAME; \
+	test_start "$$t"; \
+		if test -n "$(WITH_VALGRIND)"; then \
+			echo ;\
+			valgrind $(VALGRIND_FLAGS) "$(TESTS_DIR)/$$t" 2>&1 &\
+		else \
+			"$(TESTS_DIR)/$$t" $(BIN_FLAGS) &\
+		fi; \
+		$(SCRIPT_GET_TEST_RESULT); \
 		if test $$res = KO; then \
 			test_diag "run '$(TESTS_DIR)/$$t' to see what went wrong" ; \
 			test_diag "run '$(TESTS_DIR)/$$t -d' with a debugger" ; \
 		else \
 			test_ok PASS; \
 		fi; \
-	done; \
 	test_summary 'Summary: PASS '
 
 .PHONY: clean
 clean:
 	rm -f $(PROGRAMS) *-valgrind $(OBJECTS) tests/*.o $(TESTS_BIN) \
-		*.gcov *.gcda  *.gcno tests/*.gcov tests/*.gcda  tests/*.gcno
+		tests/debugme *.gcov *.gcda  *.gcno tests/*.gcov tests/*.gcda  tests/*.gcno tests/*.out
 
 .PHONY: coverage
 coverage:

ex/.vscode/README.md

@@ -0,0 +1,14 @@
+# Basic testing VScode config
+
+## Setup
+
+- ensure that the `.vscode` folder is located in the root of the project opened in VScode; this can either be the folder of the single exercise, or a folder containing all other exercise's folders.
+
+## Files
+
+Contains 2 json files:
+- `tasks.json`: build and run tasks, to run either all tests, or a single one (C test file opened in the editor).
+  - Ctrl + Shift + B, to run default build
+  - Ctrl + P to open command panel, type "task " to show all tasks
+- `launch.json`: debug binary tests or implementation, and I/O tests (name of input file selectable from the dropdown). 
+  - Ctrl + Shift + D to open Debug panel