VScode testing tree doesn't list additional sections

[cidrblock]

I don't see this documented anywhere, so I'm not sure if it is intended to work.

Give a tox.ini in a directory, the testing tree will show specific entries in the ini but not everything environment available (e.f. tox -l)

Additionally, when the specific entry is used, the command line generated is not correct.

For instance, using the p27 entry the following command is run in the terminal:

tox -e py27 /home/bthornto/github/tox_bug

Can the test tree be generated by a tox -l command ?
Can the command line run simply be tox -e <envname>?

Being able to run any tox environment from the test tree would be incredibly powerful and useful.

Thanks!

Brad (cidrblock)

[cidrblock]

(venv) m910x ➜  tox_bug tox -l
py27
py36

sample output from tox -l for the above

[s-hemer]

With a little more "complex" envlist like

[tox]
envlist = {py38,py39,py310}-{lint,test,doc,requirements}

, the testing tree is completely empty (beside the expandable tox.ini of course), although tox -l/tox -a lists them correctly:

py38-lint
py38-test
py38-doc
py38-requirements
py39-lint
py39-test
py39-doc
py39-requirements
py310-lint
py310-test
py310-doc
py310-requirements

Command palette yet lists this correctly.

[ssbarnea]

@The-Compiler Any chance you could avoid parsing the tox.ini file yourself and use tox "api" (tox -la) to gather the list of available environments? The way expansion works makes it close to impossible to re-implement by anyone else than tox itself, as plugins can add their own contributions.

[The-Compiler]

That's what we already do in general:

vscode-python-tox/src/run.ts

Lines 9 to 12 in 34b8f0d

	export async function getToxEnvs(projDir: string) {
	const { stdout } = await exec('tox -a', { cwd: projDir });
	return stdout.trim().split(os.EOL);
	}

(except with tox -a, because -l would only list the ones under envlist, but that's just the default list of environments).

Though for the annotations inside the file, the annotation needs to be with the [sections] in the ini file, so it does need to be parsed manually. Things like expansions indeed are not expanded there. It's unclear to me if we could generate additional test items without a location inside the file for other environments.

However, personally I don't use this part of the extension (it was contributed by @starlord-daniel), and my time to spend on this project is currently rather limited, so it'll take a while (if it's fixable at all).

---

As for the second part, that's a duplicate of #31.

[s-hemer]

That's what we already do in general:

vscode-python-tox/src/run.ts

Lines 9 to 12 in 34b8f0d

	export async function getToxEnvs(projDir: string) {
	const { stdout } = await exec('tox -a', { cwd: projDir });
	return stdout.trim().split(os.EOL);
	}

(except with tox -a, because -l would only list the ones under envlist...

I'd suggest to use the newer tox list command (with --no-desc) here. -a and -l are legacy. tox list --no-desc -d would be the replacement for -l

[s-hemer]

Any chance you could avoid parsing the tox.ini file yourself

That's what we already do in general

vscode-python-tox/src/testController.ts

Lines 111 to 120 in 34b8f0d

	async function parseTestsInFileContents(file: vscode.TestItem, contents?: string): Promise<vscode.TestItem[]> {
	if (contents === undefined) {
	const rawContent = await vscode.workspace.fs.readFile(file.uri!);
	contents = new util.TextDecoder().decode(rawContent);
	}
	let listOfChildren: vscode.TestItem[] = [];
	const testRegex = /^(\[testenv):(.*)\]/gm; // made with https://regex101.com
	let lines = contents.split('\n');

Here, the own parsing happens and this does explicitly

vscode-python-tox/src/testController.ts

Line 131 in 34b8f0d

// Excluding tox permutations for now

Maybe, instead of a separate parsing it should instead try to locate the tests from tox list output? And if it can't locate them, at least add them to the test view.

[The-Compiler]

I'd suggest to use the newer tox list command (with --no-desc) here. -a and -l are legacy. tox list --no-desc -d would be the replacement for -l

As long as it still works in tox 4, I'll continue to use -a. Everything else would just drop support for tox 3 with no benefits, while tox 3 still is rather widespread (currently still e.g. the default even on Archlinux, and certainly on things like Debian/Ubuntu too).

[s-hemer]

Don't know if it works on tox 4, I am on Ubuntu 22.04 with tox 3.21.4 and the new variants work.

[The-Compiler]

You're not running tox 3 then, because this quite simply didn't exist with tox 3:

$ tox --version
3.26.0 imported from /usr/lib/python3.10/site-packages/tox/__init__.py

$ tox list --no-desc
usage: tox [--version] [-h] [--help-ini] [-v] [-q] [--showconfig] [-l] [-a] [-c CONFIGFILE] [-e envlist] [--devenv ENVDIR] [--notest] [--sdistonly] [--skip-pkg-install] [-p [VAL]] [-o]
           [--parallel--safe-build] [--installpkg PATH] [--develop] [-i URL] [--pre] [-r] [--result-json PATH] [--discover PATH [PATH ...]] [--hashseed SEED] [--force-dep REQ]
           [--sitepackages] [--alwayscopy] [--no-provision [REQUIRES_JSON]] [-s [val]] [--workdir PATH]
           [args ...]
tox: error: unrecognized arguments: --no-desc

$ tox list          
GLOB sdist-make: ...
python inst-nodeps: ...
python installed: ...
python run-test-pre: ...
python run-test: commands[0] | pytest

[s-hemer]

As I wrote, no venv active:

$ tox list --no-desc
py38-lint
py38-test
py38-doc
py38-requirements
py39-lint
py39-test
py39-doc
py39-requirements
py310-lint
py310-test
py310-doc
py310-requirements
$ tox --version
3.21.4 imported from /usr/lib/python3/dist-packages/tox/__init__.py
$ apt-cache madison tox
       tox | 3.21.4-1ubuntu1 | http://de.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages

But indeed, while trying to solve the issue, I stumbled upon a strange behavior of the plugin within VSCode (need to investigate deeper): some calls seem to work fine and some calls seem to use another, old version of tox (error messages listing the help similar to yours).

[s-hemer]

Ok, forget about what I just wrote:
There exists a .tox/.tox/lib/python3.10/site-packages/tox/ which looks like of version 4.4.11. And a short test in a naked folder shows that due to the minversion = v4.0 in my tox.ini, tox just installs the newest version itself (in every folder, I tried to get over it but I hate this auto-dependency drawing/venv/dependency hell ... Debian package/system install or die DIY)