Change development flow to allow removing generated files, using proj…

[FredrikAtIBM]

…ect level binaries and commit hooks

• Change how we manage tools as previously we could rely on a version in the GOBIN instead of running a project controlled version of the tool. Now the GOBIN is changed to being always at the repository, so that no discrepancies between versions can occur.
• Change naming of targets in the makefile to indicate their contexts better, as targets like file generation and kubectl shortcuts missed the naming.
• Update the help command structure by providing more sections and descriptions of commands where logical.
• Running the project should follow a rather self explanatory pattern now from development point of view. install -> build -> run.
• Updated linter and git hooks. Now the pre-commit hook will notify about linting issues before you hit the pipelines. Added a install-githooks target that will be automatically called on install.
• Removed target chaining from the targets, as they will naturally be uncontrollable in favor of target chaining directly from the commandline.
• Removed generated copy of the deepcopy when it can be generated. Run make generate-deepcopies to generate said files.
• Since the repository no longer contains generated content, the boilerplate file is made redundant.
• Updated README guide as some of the targets don't exist anymore.
• Removed obvious .gitignore rules as they're no longer needed.
• The purge kubectl target is so effective, that it has taken over the delete target completely
• Change Dockerfile to Containerfile to indicate that it is no longer explicitly runnable by Docker
• Chage the approach how the image is being built by forcing the user to specify builder and target values so that one can use Podman for this file as well as Docker.
• Change the approach how the image is being built by allowing different architectures to be used for BUILDTARGET so that ARM can now be used to build
• Move microdnf from golangInstall to the Containerfile to have just one microdnf command
• Add gitkeep to the bin directory to avoid any situations where it might not exist
• Rename yml-files to yaml-files for consistency
• Add the new pipeline ARGs to facilitate the changes made to the Containerfile/Dockerfile
• Add ability to load dotenv files in code, so that the e2e tests don't need to be manually inserted

[konrad-ohms]

Looks like removing the dependencies in the Makefile failed the build, I think the deepcopy file is missing, could you take a second look please?

[FredrikAtIBM]

Looks like removing the dependencies in the Makefile failed the build, I think the deepcopy file is missing, could you take a second look please?

Thanks. The generated files no longer exist. This is intended behavior. The pipelines will just have to facilitate the new changes.

[konrad-ohms]

Suggested change

	container-build: ## Build docker image with the manager.
	container-build: buildctl ## Build docker image with the manager.

If you don't define the dependency to check the service is running, you can run into errors when the buildkitd container is not running.

Example:

$ make container-build
/home/ohmsk/git/instana-public/instana-agent-operator/bin/buildctl --addr=podman-container://buildkitd build \
        --frontend=dockerfile.v0 \
        --local context=. \
        --local dockerfile=. \
        --opt build-arg:VERSION=0.0.1 \
        --opt build-arg:GIT_COMMIT=faa67b0  \
        --opt build-arg:DATE="$(date)" \
        --opt build-arg:BUILDPLATFORM=linux/amd64 \
        --opt build-arg:TARGETPLATFORM= \
        --opt filename=Containerfile \
        --output type=oci,name=delivery.instana.io/dev-sandbox-docker-all/konrad/instana-agent-operator:konrad | podman load
[+] Building 0.0s (0/0)
error: listing workers for Build: failed to list workers: Unavailable: connection error: desc = "error reading server preface: command [podman exec -i buildkitd buildctl dial-stdio] has exited with exit status 255, make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Error: can only create exec sessions on running containers: container state improper\n"
Error: payload does not match any of the supported image formats:
 * oci: open /var/tmp/podman3981212783/index.json: not a directory
 * oci-archive: loading index: open /var/tmp/container_images_oci1704363624/index.json: no such file or directory
 * docker-archive: loading tar component "manifest.json": file does not exist
 * dir: open /var/tmp/podman3981212783/manifest.json: not a directory
make: *** [Makefile:160: container-build] Error 125
ohmsk:~/git/instana-public/instana-agent-operator (devenv-changes)$ podman ps
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES
ohmsk:~/git/instana-public/instana-agent-operator (devenv-changes)$ podman ps -a
CONTAINER ID  IMAGE                            COMMAND     CREATED      STATUS                   PORTS       NAMES
80fbc41aea46  docker.io/moby/buildkit:v0.16.0              4 weeks ago  Exited (1) 43 hours ago              buildkitd

[konrad-ohms]

Please check operator-olm-build, I think you need to install kustomize here to work, probably with your make target.
This is probably also needed on the main pipeline for operator-olm-github-release