Self-verification fails?

[urmilparikh]

A good idea and a good tool!
I tried verifying tbv itself and shasums did not match??

$ tbv verify tbv
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

[CompileNix]

I saw this, too.
I would assume that this is not working, yet.
The project is pretty new, too ;)

[CompileNix]

Unitl now i've not seen any package verification succeeding

[skonves]

Interesting 🤔

Which version of node, npm, and os are you using?

[CompileNix]

$ nvm i 10
Downloading and installing node v10.15.0...
Downloading https://nodejs.org/dist/v10.15.0/node-v10.15.0-linux-x64.tar.xz...
######################################################################################################################################################################################################################################## 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v10.15.0 (npm v6.4.1)
$ npm i -g tbv
/home/compilenix/.nvm/versions/node/v10.15.0/bin/tbv -> /home/compilenix/.nvm/versions/node/v10.15.0/lib/node_modules/tbv/lib/index.js
+ tbv@0.0.3
added 6 packages from 8 contributors in 0.922s
$ tbv verify tbv
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

[skonves]

Interesting. I am on MacOS 10.14.2 running the same versions of node and npm. Which distro/version are you using?

Also, what is your output for:
tbv verify express (no pre-pack step)
tbv verify redux (pre-pack step, but works on my machine)

[CompileNix]

Gentoo

tbv

$ tbv verify tbv
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

express

$ tbv verify express
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

redux

$ tbv verify redux
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

Fedora 29

tbv

$ tbv verify tbv
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

express

$ tbv verify express
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

redux

$ tbv verify redux
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

[skonves]

It looks like the when theCreate package step runs on some (all?) Linux distros, TBV isn't grabbing the full console output (it only contains the tarball name). When the regex is evaluated, it fails. (This should fail the step, but for some reason doesn't)

I think that if I get that part working, we should be able to get validation working on most if not all platforms.

Here is the relevant code:
https://github.com/skonves/tbv/blob/70cf97928ee1421144b1dc44684ddf691054b38e/src/verifier.ts#L280-L293

[skonves]

I just pushed a commit that allows support for running TBV within a docker container: (d6663f6). I was able to reproduce the failures described in this issue.

[skonves]

v0.2.0 compares package contents instead of tarball shasums. I have tbv and express passing in Fedora 29 (docker). redux errors out in Fedora, but passes in Alpine and Jessie.

I'll leave this issue open for now but may close it in the near future in favor of more specific failures.

And thank you to everyone for the feedback! 🎉

[CompileNix]

Thanks!
I think running this via docker is a good idea, to provide a consistent enviroment. atm. i don't have docker configured on my machines.

I aggre for closing this issue, too.

results using v0.2.0 and node 10.15.0:

Gentoo

tbv

$ tbv verify tbv               
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✓ Compare package contents

PASSED

express

$ tbv verify express
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✓ Compare package contents

PASSED

redux

$ tbv verify redux
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✓ Compare package contents

PASSED

Fedora 29

tbv

$ tbv verify tbv
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

express

$ tbv verify express
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
- Install npm packages [SKIPPED]
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED

redux

$ tbv verify redux
✓ Fetch package data from registry
✓ Version contains repository URL
✓ Version contains gitHead
✓ Shallow checkout
✓ Install npm packages
✓ Create package
✗ Compare shasums >>>> Shasums do not match

FAILED