Install local version from branch "specs-fmt-checks-etc" and test

[agstephens]

• Clone repo
• Checkout branch: "specs-fmt-checks-etc"
• Install dependencies: e.g. pip install -r requirements.txt
• Install locally in develop mode:
  • Instead of installing the local package under the system site-packages directory, it just leaves a link file
  • Simple way to do this: pip install -e . --no-deps or python setup.py develop
• Run the tests locally:
  • ./run-cli-tests.sh

NOTE: they expect to find some files in the CEDA archive and/or JASMIN GWS - so need to identify and maybe comment them out. (We should reinstate and maybe have some example files in a separate repo).

[joshua-hampton]

Issue with installing dependency cf through pip install -r requirements.txt:

• cannot install python-cloudfiles, from requirements.txt in line 3 in cf module

Full error:

Collecting python-cloudfiles (from cf->-r requirements.txt (line 3))
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/b5/6d/11d1166c0d1a48eb15daf1b6c748e99c057ebac92e38fa0c19bdad2bc6a0/python-cloudfiles-1.7.11.tar.gz (330kB)
    100% |████████████████████████████████| 337kB 3.3MB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-337trxab/python-cloudfiles/setup.py", line 6, in <module>
        from cloudfiles.consts import __version__
      File "/tmp/pip-build-337trxab/python-cloudfiles/cloudfiles/__init__.py", line 82, in <module>
        from cloudfiles.connection     import Connection, ConnectionPool
      File "/tmp/pip-build-337trxab/python-cloudfiles/cloudfiles/connection.py", line 13, in <module>
        from    urllib    import urlencode
    ImportError: cannot import name 'urlencode'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-337trxab/python-cloudfiles/

I think the important part is ImportError: cannot import name 'urlencode' in /tmp/pip-build-337trxab/python-cloudfiles/cloudfiles/connection.py - using different versions of pip, or having conda activated or not, different messages are printed out but the same error occurs.

cf module is only used in the pp reader - readers.pp.py line 19.

[agstephens]

Sorry @joshua-hampton , the requirements.txt file was wrong. I have fixed it and pushed to github (on same branch).

[joshua-hampton]

1. pandas is used in checksit summary but is not in the requirements.txt file, I've added this in a new branch and will be included in a future pull request to specs-fmt-checks-etc branch.
2. I believe all files looked for when running ./run-cli-tests.sh are on CEDA, a JASMIN GWS, or @agstephens home directory. The ones that I have access to work as currently expected.