Add a test suite. by lucc · Pull Request #194 · rkitover/vimpager

lucc · 2016-06-09T12:24:58Z

This is an attempt to finish #180.

When reviewing #193 it suddenly came to me that script could be used to capture the help and version output at least. Like this: script -c './vimpager -h' -e -q. This pattern is used with the bats test framework.

The decision between bats and roundup is rather arbitrary. I went ahead and used bats because roundup is not available in the Arch repos and that is what I use vor development. If that is unadaptable I can also convert these tests to another test suite.

PS: I don't really understand why travis fails to install bats. Can we somehow force the build to restart without pushing to this branch?

lucc · 2016-06-16T08:46:02Z

I managed to install bats but the git tags of the vimpager repository are not available on travis and they are needed to get the correct version output.

@rkitover any ideas?

Add a comment to the top of vimpager and vimcat with the current release version, and use it in --version output when git describe has failed.

rkitover · 2016-06-18T07:34:10Z

@lucc sorry things have been difficult here, but I'm slowly getting back into things.

Does the above commit help you?

Adjust valid version glob pattern from `[0-9]*` to `[0-9].`, this makes sure that git describe picked up an actual tag.

rkitover · 2016-06-18T07:42:02Z

@lucc made a minor adjustment to make sure a tag is picked up above.

lucc · 2016-06-18T10:00:33Z

@rkitover I don't really like the solution. If we really need to supply a fallback version number I would suggest something like 5438492. (But the makefile is not yet updated with that commit.)

In my opinion it would be better to solve this issue on Travis. Like this we introduce additional code paths just for the test. That means we don't test the normal code paths at all.

lucc · 2016-06-18T10:04:00Z

Also: The first question we should answer here is if we want to use bats and keep these tests? Otherwise we have to decide a new test framework and re-implement these tests.

rkitover · 2016-06-18T20:12:25Z

@lucc check this out: https://travis-ci.org/rkitover/vimpager/builds/138622907

so all you need to do is stick git fetch --unshallow somehwere in the script:.

re: 5438492

I would still like the last release tag somewhere at the top of the file, it can be a variable assignment instead of a comment as well, that's fine. That's even better.
Of course it would be nice for the makefile to update the release tag, but also fine without this, since we can update it in the release commit.
I see now that git describe works ONLY with tags, so some of my code was definitely unnecessary.
Don't call it (fallback) something like (checkout) would be better.

re: this PR

I haven't had a chance to look at the code yet, BATS is what I wanted to use, so that's good, I'll try to read the commits later today. I'm dealing with an eye infection, back and knee pain and lots of other horrible shit right now.

lucc · 2016-06-20T04:33:25Z

I think I fixed the travis build. The core problem was that make install-deb did internally run our make test so I had to move it to the script: step in .travis.yml.

Please do not merge yet, I am still cleaning up the git history and testing more stuff.

lucc · 2016-06-20T23:15:58Z

@rkitover The history is still not clean but that is only because there is a question that came up: The file debian/source/lintian-overrides tells lintian to ignore newer standards versions but we still update it when building on travis.

We could either replace the call to scripts/update_lintian with sudo apt-get update in the travis file or remove this lintian override. I tested both and the build on travis pass.

What do you think? Are the new version of lintian or the override needed for some other reason?

Other than that and adding however many tests this PR is done (from my point of view).

lucc · 2016-06-21T22:09:18Z

I also pushed https://github.com/rkitover/vimpager/tree/test2 . I will open a PR for that as soon as this one is merged, as it is a child of this one. But you can already have a look.

Add support for `make install-deb CLEAN_BUILD_DEPS=0` to not run `apt-get autoremove` so that build dependencies are not removed.

rkitover · 2016-06-23T18:49:41Z

@lucc the reason for update_lintian is to use the latest version of lintian, the travis VM is trusty which is old.

If the new lintian is complaining that the standards version is too old, we should update the standards version in debian/ . If there are incompatibilies with the latest standards version then I will fix them. Also why am I not seeing this issue on master?

I added support for not removing build deps in make install-deb above, while debhelper has support for not running tests, so change the sudo make install-deb command in .travis.yml to:

sudo make install-deb CLEAN_BUILD_DEPS=0 DEB_BUILD_OPTIONS=nocheck

then you can run make test after the debian package is installed and the build deps will not have been removed and debhelper will not run make test. This will make manually installing sharutils unnecessary.

- Move `make install-deb` to the script step as it runs the tests. - Install uuencode (package sharutils) explicitly as it is needed during testing. Formally it was implicitly installed by `make install-deb`. - Install bats manually as there is no package for it on trusty. - Remove bats building directory as it is reported by lintian. - Fetch git tags as they are needed to run the tests. - Run `make all`. - Add a modeline.

lucc · 2016-06-23T23:24:28Z

Thanks for the info. I added that and do now consider ths PR done. Feel free to merge if you don't have further comments.

rkitover · 2016-06-26T12:43:05Z

@lucc finally read the actual tests, awesome stuff! Merging.

Replace script -q -e -c invocation in bats tests with the run_cmd helper which uses a more portable invocation of script(1) which works on OS X, Ubuntu and OpenBSD.

rkitover · 2016-06-26T16:54:20Z

@lucc I made some changes to call script(1) in a more portable way.

Tests now pass on OS X, Ubuntu, OpenBSD and FreeBSD.

lucc · 2016-06-29T10:09:43Z

Nice, I opend #196 as a followup.

lucc force-pushed the test branch 2 times, most recently from 2b7b8cd to ea7620d Compare June 9, 2016 12:32

lucc mentioned this pull request Jun 10, 2016

allow a relative symlink to vimpager to work again #193

Merged

lucc force-pushed the test branch from 65ab102 to ff995f1 Compare June 16, 2016 07:46

rkitover added a commit that referenced this pull request Jun 18, 2016

use version from top comment if not in git #194

d384df3

Add a comment to the top of vimpager and vimcat with the current release version, and use it in --version output when git describe has failed.

rkitover added a commit that referenced this pull request Jun 18, 2016

check that git describe for ver. returns tag #194

4aef79c

Adjust valid version glob pattern from `[0-9]*` to `[0-9].`, this makes sure that git describe picked up an actual tag.

lucc force-pushed the test branch 4 times, most recently from 4b3714f to dd83f2b Compare June 20, 2016 04:24

lucc force-pushed the test branch 6 times, most recently from 8bfcdba to 50f8c2e Compare June 20, 2016 09:21

lucc mentioned this pull request Jun 20, 2016

Version tag at top of progs #195

Merged

lucc force-pushed the test branch 5 times, most recently from d210bc3 to 87d4810 Compare June 20, 2016 23:09

lucc force-pushed the test branch 2 times, most recently from 742e8eb to b383415 Compare June 22, 2016 07:51

rkitover added a commit that referenced this pull request Jun 23, 2016

CLEAN_BUILD_DEPS=0 option on make install-deb #194

2573495

Add support for `make install-deb CLEAN_BUILD_DEPS=0` to not run `apt-get autoremove` so that build dependencies are not removed.

lucc added 6 commits June 24, 2016 00:38

Add some basic tests.

e83724a

Build standalone scripts before running tests.

e9c7f1f

Add test with symlinks for #193.

1c92187

Add tests to copyright info

a3b0b94

Use env variables to improve build on travis

683bf68

lucc force-pushed the test branch from b383415 to 683bf68 Compare June 23, 2016 22:57

Silence apt-get in the makefile and script

5c08279

lucc force-pushed the test branch from 913abe1 to 5c08279 Compare June 23, 2016 23:19

rkitover merged commit 26745c7 into master Jun 26, 2016

rkitover deleted the test branch June 26, 2016 12:43

rkitover added a commit that referenced this pull request Jun 26, 2016

more portable script(1) invocation in tests #194

f51fc21

Replace script -q -e -c invocation in bats tests with the run_cmd helper which uses a more portable invocation of script(1) which works on OS X, Ubuntu and OpenBSD.

Conversation

lucc commented Jun 9, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucc commented Jun 16, 2016

Uh oh!

rkitover commented Jun 18, 2016

Uh oh!

rkitover commented Jun 18, 2016

Uh oh!

lucc commented Jun 18, 2016

Uh oh!

lucc commented Jun 18, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rkitover commented Jun 18, 2016

Uh oh!

lucc commented Jun 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucc commented Jun 20, 2016

Uh oh!

lucc commented Jun 21, 2016

Uh oh!

rkitover commented Jun 23, 2016

Uh oh!

lucc commented Jun 23, 2016

Uh oh!

rkitover commented Jun 26, 2016

Uh oh!

rkitover commented Jun 26, 2016

Uh oh!

lucc commented Jun 29, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lucc commented Jun 9, 2016 •

edited

Loading

lucc commented Jun 18, 2016 •

edited

Loading

lucc commented Jun 20, 2016 •

edited

Loading