Add better language documentation

LICENSE.md

@@ -0,0 +1,7 @@
+Copyright 2021 Luuk Verweij
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -1,90 +1,21 @@
 # Aaa
-Stack-based language, like forth.
-
-### Contents
-The following tools for the Aaa language can be found is this repo
-* A [tokenizer](./aaa/tokenizer/) and [parser](./aaa/parser/) for Aaa.
-* A [type checker](./aaa/type_checker/)
-* A [transpiler to Rust](./aaa/transpiler/)
-* A [VS Code extension](./aaa-vscode-extension/README.md) for the Aaa language.
-* A lot of tests, written both in python and Aaa
-
-### Examples
-```sh
-# Run the obligatory hello world example
-pdm run ./manage.py run 'fn main { "Hello world\n" . }'
-
-# Run code from a file. Implements the famous fizzbuzz interview question.
-pdm run ./manage.py run examples/fizzbuzz.aaa
-
-# Run bare-bones HTTP server in Aaa
-pdm run ./manage.py run examples/http_server.aaa
-
-# Send request from different shell
-curl http://localhost:8080
-```
-
-### Running tests
-
-```sh
-# Run tests written in Aaa
-pdm run ./manage.py test .
-
-# Run tests written in Python
-pdm run pytest
-```
-
-
-### Aaa features
-- elementary types (`int`, `bool`, `str`)
-- container types (`vec`, `map`)
-- type checking
-- branching, loops (`if`, `else`, `while`, `foreach`)
-- multi-file support (`import`)
-- [upcoming](https://github.com/lk16/aaa/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement)
-
-### Name
-The name of this language is just the first letter of the Latin alphabet repeated three times. When code in this language doesn't work, its meaning becomes an [abbreviation](https://en.uncyclopedia.co/wiki/AAAAAAAAA!).
-
-### Setup
-All these commands should be run from the root of this repository.
-
-This project requires python 3.12.1 or newer. Consider using [pyenv](https://github.com/pyenv/pyenv).
-
-```sh
-# Download python 3.12.1
-pyenv install 3.12.1
-
-# Use it in this project
-pyenv local 3.12.1
-```
-
-This project also requires rust, see instructions [here](https://www.rust-lang.org/tools/install) on how to install.
-
-After you setup rust and python, run the following commands.
-
-```sh
-# Install dependencies
-pdm install
-
-# Tell Aaa where the standard library lives
-export AAA_STDLIB_PATH=$(pwd)/stdlib
-
-# Run hello world program
-pdm run ./manage.py run 'fn main { "Hello world\n" . }'
-
-# Run tests
-pdm run pytest
-pdm run ./manage.py test .
-
-# Setup pre-commit hooks
-pdm run pre-commit install
-```
-
-Now you can start running code in Aaa or develop the language!
-
-To enable syntax highlighting for VS Code, enable the [Aaa language extension](./aaa-vscode-extension/README.md)
-
-
-### Aaa and porth
-After watching part of the [Youtube series](https://www.youtube.com/playlist?list=PLpM-Dvs8t0VbMZA7wW9aR3EtBqe2kinu4) on [porth](https://gitlab.com/tsoding/porth), I wanted to make my own stack-based language. Aaa and porth have some similarities, but obviously are not compatible with each other. No code was copied over from the porth repo.
+A stack-based programming language.
+
+### Language features and goals
+This language was created with as main goals:
+* Simplicity
+* Type-safety
+* Testability
+* Becoming [selfhosted](https://en.wikipedia.org/wiki/Self-hosting_(compilers))
+
+See also the [FAQ](./docs/faq.md) section.
+
+### How to get started
+* Follow the [setup and installation](./docs/setup.md) guide.
+* See [how to run tests and examples](./docs/examples.md).
+
+### Documentation
+* Find out about [the stack](./docs/stack.md).
+* Learn about [how types work](./docs/typing.md).
+* Read about [the Aaa implementation](./docs/implementation.md).
+* Read the [FAQ](./docs/faq.md).

docs/direnv.md

@@ -0,0 +1,56 @@
+
+### About `direnv`
+
+Direnv is a tool to load environments and settings per folder. This is saves time and sometimes makes commands more readable (example: omit `pdm run` prefix).
+
+Files used by `direnv`:
+* `.envrc` in the root of your project, to setup environment variables and use shell functions.
+* `.direnvrc` in your home folder, to define shell functions that can be used by `.envrc` in projects.
+
+For more details on `direnv`, see their [official site](https://direnv.net/).
+
+Note that the files below can also be found in a [howto section in my dotfiles](https://github.com/lk16/dotfiles/blob/master/howto/direnv.md).
+
+---
+
+### Setup `direnv`
+
+* On debian-based linux, run `sudo apt-get install direnv`, for other systems see [this page](https://direnv.net/docs/installation.html).
+* Follow [instructions](https://direnv.net/docs/hook.html) for your shell
+* Be sure to reload your shell config (for bash run `. ~/.bashrc`)
+* Save this bash script as `.direnvrc` in your home folder. It defines the `layout_pdm` function which we use below.
+
+```sh
+layout_pdm() {
+    PYPROJECT_TOML="${PYPROJECT_TOML:-pyproject.toml}"
+    if [ ! -f "$PYPROJECT_TOML" ]; then
+        log_status "No pyproject.toml found. Executing \`pmd init\` to create a \`$PYPROJECT_TOML\` first."
+        pdm init --non-interactive --python "$(python3 --version 2>/dev/null | cut -d' ' -f2 | cut -d. -f1-2)"
+    fi
+
+    VIRTUAL_ENV=$(pdm venv list | grep "^\*"  | awk -F" " '{print $3}')
+
+    if [ -z "$VIRTUAL_ENV" ] || [ ! -d "$VIRTUAL_ENV" ]; then
+        log_status "No virtual environment exists. Executing \`pdm info\` to create one."
+	pdm info
+        VIRTUAL_ENV=$(pdm venv list | grep "^\*"  | awk -F" " '{print $3}')
+    fi
+
+    PATH_add "$VIRTUAL_ENV/bin"
+    export PDM_ACTIVE=1
+    export VIRTUAL_ENV
+}
+```
+
+* Next: save this bash script as `.envrc` in the root of this project. It calls the `layout_pdm` function, to set relevant environment variables. It also sets `AAA_STDLIB_PATH`, which is used by this project. The `.envrc` file is not tracked by git, [it is](../.gitignore) in `.gitignore`.
+```sh
+layout_pdm
+export AAA_STDLIB_PATH=$(pwd)/stdlib
+```
+* Run `direnv allow` to tell `direnv` that this file is safe to run.
+* You should be good to go. Try running this (note: no `pdm run` prefix):
+```sh
+./manage.py run 'fn main { "Hello world\n" . }'
+```
+* You can now omit `pdm run` prefix for all commands.
+* Happy coding!

docs/examples.md

@@ -0,0 +1,27 @@
+### How to run examples
+
+```sh
+# Run the obligatory hello world example
+pdm run ./manage.py run 'fn main { "Hello world\n" . }'
+
+# Run code from a file. Implements the famous fizzbuzz interview question.
+pdm run ./manage.py run examples/fizzbuzz.aaa
+
+# Run bare-bones HTTP server in Aaa
+pdm run ./manage.py run examples/http_server.aaa
+
+# Send request from different shell
+curl http://localhost:8080
+```
+
+More examples can be found in the [examples folder](./../examples/)
+
+### How to run tests
+
+```sh
+# Run tests written in Aaa
+pdm run ./manage.py test .
+
+# Run tests written in Python
+pdm run pytest
+```

docs/faq.md

@@ -0,0 +1,105 @@
+
+### Frequently Asked Questions
+
+##### What does the name "Aaa" mean?
+Naming things is notoriously hard and naming a made up programming language is no exception. The name of this language is just the first letter of the Latin alphabet repeated three times. When code in this language doesn't work, its meaning becomes an [abbreviation](https://en.uncyclopedia.co/wiki/AAAAAAAAA!).
+
+##### How did you get the idea to make a language?
+After watching part of the [Youtube series](https://www.youtube.com/playlist?list=PLpM-Dvs8t0VbMZA7wW9aR3EtBqe2kinu4) on [porth](https://gitlab.com/tsoding/porth), I wanted to make my own stack-based language. Aaa and porth have some similarities, but obviously are not compatible with each other. No code was copied over from the porth repo.
+
+##### What are the key features of Aaa?
+See [language features and goals](./../README.md#language-features-and-goals).
+
+##### How do I get started with Aaa?
+See the [setup page](./setup.md).
+
+##### What are the system requirements for Aaa?
+Any machine that runs Python 3.11.3 and Rust should be able to run Aaa. If you run into issues, be sure to [open a ticket](https://github.com/lk16/aaa/issues/new).
+
+##### Is Aaa open source or proprietary?
+This project is open source and is distributed under the MIT license. This effectively means: do whatever you want with the code, but you can't sue me.
+
+##### How do I report bugs or request features?
+Both bug reports and feature requests can be sent to developers by opening a ticket.
+Please first check [existing tickets](https://github.com/lk16/aaa/issues), before opening a new one.
+
+##### What is the latest version of Aaa?
+At time of writing there are no releases. Checking out the `main` branch and running `git pull` will get you the latest version.
+
+##### Can I use Aaa on different operating systems?
+Yes, see `What are the system requirements` question above.
+
+##### What kind of applications can I build with Aaa?
+Anything that:
+* does not depend on other libraries (such as GUI's)
+* is serious production / non-"pet project" software
+
+That being said, various tools have been written using Aaa:
+* a [HTTP server](../examples/http_server.aaa)
+* a [HTTP client](../examples/http_client.aaa)
+* a [shell](../examples/shell.aaa)
+* a [sudoku solver](../examples/sudoku_solver.aaa)
+* a [game of life](../examples/game_of_life.aaa) implementation ([What is this?](https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life))
+
+##### Are there any code examples or tutorials available for Aaa?
+Yes, see the [examples documentation](./examples.md)
+
+##### How do I create a new project or file in Aaa?
+At this stage of the language:
+* Create a new branch from `main`
+* Create a new folder in `examples`
+* Create a file `main.aaa` having a `main` function
+* Start typing code
+
+##### What is the package manager for Aaa?
+There is none. There is not even a well-implemented import-system at time of writing.
+
+##### How do I install and manage packages or dependencies?
+Good one.
+
+##### How can I handle errors or exceptions in Aaa?
+We don't have exceptions in this language. Functions can return an enum, with one or more error variants.
+
+##### What is the syntax for ... in Aaa?
+See the [syntax documentation](./syntax.md)
+
+##### How do I work with classes and objects in Aaa?
+It mostly revolves around [the stack](./stack.md).
+
+##### What are the available data structures and collections in Aaa?
+At time of writing:
+* `vec`, sometimes called "dynamic array" or "list"
+* `map`, sometimes called "hash table", "hash map" or "dictionary"
+* `set`
+
+##### How can I work with files and file I/O?
+Use the [builtin](./../stdlib/builtins.aaa) system calls.
+
+##### Is there a standard coding style or best practices guide for Aaa?
+Yes, see the [best practices](./best_practices.md) article.
+
+##### How do I optimize and debug code in Aaa?
+
+
+##### Are there any known limitations or caveats in Aaa?
+Well, this is an enormous "pet project", so nothing is considered stable or production-ready. That being said, a lot of tests are in place to prevent major issues. The main limitations have more to do with features that are not implemented, rather than existing bugs.
+
+Main things that are not implemented:
+* interfaces (as in golang, [issue](https://github.com/lk16/aaa/issues/16))
+* dealing with concurrency
+* having a standard library
+* having a well-implemented import system
+* having a package manager
+
+##### Can I contribute to the development of Aaa?
+* Clone this repo and write some code in Aaa.
+* Find bugs, suggest features on the [issues](https://github.com/lk16/aaa/issues) page.
+
+##### How do I secure my Aaa applications?
+Please don't use Aaa applications with any sensitive data. Do NOT use Aaa applications in a production setting.
+
+##### What is the support and community around Aaa?
+The [issues section](https://github.com/lk16/aaa/issues) on the GithHub repository.
+
+##### Was ChatGPT or other AI involved in the development?
+ChatGPT was not used for writing code, but sometimes used to assist in refactoring. Generally speaking all code in this repo was written by humans. ChatGPT did help coming up with some of the questions on this page.

docs/implementation.md

@@ -0,0 +1,6 @@
+
+# Implementation of Aaa
+
+TODO talk about tokenizer, parser, cross-referencer, type checker, transpiler, compiler
+
+TODO link to this article

docs/setup.md

@@ -0,0 +1,43 @@
+
+### How to setup this project
+
+Start by cloning this repository.
+
+All these commands should be run from the root of this repository.
+
+This project requires Python 3.12.1 or later, consider using [pyenv](https://github.com/pyenv/pyenv) to select a python version.
+
+```sh
+# Download python 3.12.1
+pyenv install 3.12.1
+
+# Use it in this project
+pyenv local 3.12.1
+```
+
+This project also requires Rust, see instructions [here](https://www.rust-lang.org/tools/install) on how to install.
+
+After you setup Rust and Python, run the following commands.
+
+```sh
+# Install dependencies
+pdm install
+
+# Tell Aaa where the standard library lives
+export AAA_STDLIB_PATH=$(pwd)/stdlib
+
+# Run hello world program
+pdm run ./manage.py run 'fn main { "Hello world\n" . }'
+
+# Run tests
+pdm run pytest
+
+# Setup pre-commit hooks
+pdm run pre-commit install
+```
+
+Now you can start running code in Aaa or develop the language!
+
+Tips to make your life better:
+* Enable syntax highlighting for VS Code, enable the [Aaa language extension](./aaa-vscode-extension/README.md)
+* Use `direnv` as described [here](./direnv.md) so you don't have to type `pdm run` all the time.