Add tests for API

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,6 +1,6 @@
 ---
 name: Feature request
-about: Idea or feature request  
+about: Idea or feature request
 title: ''
 labels: 'kind:feature'
 assignees: ''

.github/workflows/ci.yaml

@@ -0,0 +1,30 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+---
+name: CI
+on:
+  - push
+  - pull_request
+
+jobs:
+  statics:
+    name: Static Checks
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v1
+      - uses: pre-commit/action@v1.0.1

.pre-commit-config.yaml

@@ -28,4 +28,6 @@ repos:
     hooks:
       - id: check-yaml
       - id: end-of-file-fixer
+        exclude: ^ui/vendors/.*$
       - id: trailing-whitespace
+        exclude: ^ui/vendors/.*$

CODE_OF_CONDUCT.md

@@ -1,7 +1,7 @@
 # Code of Conduct
 
-The Apache Kibble project follows the 
+The Apache Kibble project follows the
 [Apache Software Foundation code of conduct](https://www.apache.org/foundation/policies/conduct.html).
 
-If you observe behavior that violates those rules please follow the 
+If you observe behavior that violates those rules please follow the
 [ASF reporting guidelines](https://www.apache.org/foundation/policies/conduct#reporting-guidelines).

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing to Kibble #
 
-## Community 
+## Community
 
 The main development and design discussion happens on our mailing lists.
 We have a list specifically for development, and one for future user questions and feedback.
@@ -15,5 +15,31 @@ We also have:
 
 ## Development installation
 
-This project requires Python in higher version than 3.3.
+This project requires Python in higher version than 3.4.
 More information will come soon!
+
+## Code Quality
+
+Apache Kibble project is using [pre-commits](https://pre-commit.com) to ensure the quality of the code.
+We encourage you to use pre-commits, but it's not required in order to contribute. Every change is checked
+on CI and if it does not pass the tests it cannot be accepted. If you want to check locally then
+you should install Python3.6 or newer together and run:
+```bash
+pip install pre-commit
+# or
+brew install pre-commit
+```
+For more installation options visit the [pre-commits](https://pre-commit.com).
+
+To turn on pre-commit checks for commit operations in git, run:
+```bash
+pre-commit install
+```
+To run all checks on your staged files, run:
+```bash
+pre-commit run
+```
+To run all checks on all files, run:
+```bash
+pre-commit run --all-files
+```

NOTICE

@@ -119,7 +119,7 @@ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 Metis Dashboard (MIT License)
 ------------------------------------------------------------------------
 
-Copyright (c) 2015 onokumus 
+Copyright (c) 2015 onokumus
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in
 the Software without restriction, including without limitation the rights to
@@ -186,4 +186,3 @@ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-

README.md

@@ -1,14 +1,16 @@
 <p align="center"><img src="/ui/images/kibble-logo.png" width="300"/></p>
 
 # Apache Kibble
+![CI](https://github.com/apache/kibble/workflows/CI/badge.svg)
+[![License](http://img.shields.io/:license-Apache%202-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)
 
 Apache Kibble is a tool to collect, aggregate and visualize data about any software project that uses commonly known tools. It consists of two components:
 
 - **Kibble Server** (this repository) - main database and UI Server. It serves as the hub
  for the scanners to connect to, and provides the overall management of sources as well as the
  visualizations and API end points.
 - **Kibble scanners** ([kibble-scanners](https://github.com/apache/kibble-scanners)) - a collection of
- scanning applications each designed to work with a specific type of resource (git repo, mailing list, 
+ scanning applications each designed to work with a specific type of resource (git repo, mailing list,
  JIRA, etc) and push compiled data objects to the Kibble Server.
 
 ## Documentation

api/handler.py

@@ -56,7 +56,7 @@ class KibbleHTTPError(Exception):
     def __init__(self, code, message):
         self.code = code
         self.message = message
-        
+
 
 class KibbleAPIWrapper:
     """
@@ -67,7 +67,7 @@ def __init__(self, path, func):
         self.API = KibbleOpenAPI
         self.path = path
         self.exception = KibbleHTTPError
-     
+
     def __call__(self, environ, start_response, session):
         """Run the function, return response OR return stacktrace"""
         response = None
@@ -90,7 +90,7 @@ def __call__(self, environ, start_response, session):
                         "reason": "Invalid JSON: %s" % err
                     })
                     return
-                
+
             # Validate URL against OpenAPI specs
             try:
                 self.API.validate(environ['REQUEST_METHOD'], self.path, formdata)
@@ -102,7 +102,7 @@ def __call__(self, environ, start_response, session):
                     "reason": err.message
                 })
                 return
-            
+
             # Call page with env, SR and form data
             try:
                 response = self.func(self, environ, formdata, session)
@@ -124,7 +124,7 @@ def __call__(self, environ, start_response, session):
                     "reason": err.message
                 }, indent = 4) + "\n"
                 return
-            
+
         except:
             err_type, err_value, tb = sys.exc_info()
             traceback_output = ['API traceback:']
@@ -140,8 +140,8 @@ def __call__(self, environ, start_response, session):
                 "code": "500",
                 "reason": '\n'.join(traceback_output)
             })
-    
-        
+
+
 def fourohfour(environ, start_response):
     """A very simple 404 handler"""
     start_response("404 Not Found", [
@@ -181,7 +181,7 @@ def application(environ, start_response):
                 elif isinstance(bucket, bytes):
                     yield bucket
             return
-            
+
     for bucket in fourohfour(environ, start_response):
         yield bytes(bucket, encoding = 'utf-8')
 

api/pages/__init__.py

@@ -42,5 +42,5 @@ def loadPage(path):
                 xp = p.replace('.', '/')
                 print("Loading endpoint pages.%s as %s" % (p, xp))
                 handlers[xp] = importlib.import_module("pages.%s" % p)
-    
-loadPage(rootpath)
+
+loadPage(rootpath)

api/pages/account.py

@@ -85,7 +85,7 @@
 #             $ref: '#/components/schemas/Error'
 #       description: unexpected error
 #   summary: Create a new account
-# 
+#
 ########################################################################
 
 
@@ -125,40 +125,40 @@ def sendCode(session, addr, code):
     s.quit()
 
 def run(API, environ, indata, session):
-    
+
     method = environ['REQUEST_METHOD']
 
     # Add a new account??
     if method == "PUT":
         u = indata['email']
         p = indata['password']
         d = indata['displayname']
-        
+
         # Are new accounts allowed? (admin can always make accounts, of course)
         if not session.config['accounts'].get('allowSignup', False):
             if not (session.user and session.user['level'] == 'admin'):
                 raise API.exception(403, "New account requests have been administratively disabled.")
-        
+
         # Check if we already have that username in use
         if session.DB.ES.exists(index=session.DB.dbname, doc_type='useraccount', id = u):
             raise API.exception(403, "Username already in use")
-        
+
         # We require a username, displayName password of at least 3 chars each
         if len(p) < 3 or len(u) < 3 or len(d) < 3:
             raise API.exception(400, "Username, display-name and password must each be at elast 3 characters long.")
-        
+
         # We loosely check that the email is an email
         if not re.match(r"^\S+@\S+\.\S+$", u):
             raise API.exception(400, "Invalid email address presented.")
-        
+
         # Okay, let's make an account...I guess
         salt = bcrypt.gensalt()
         pwd = bcrypt.hashpw(p.encode('utf-8'), salt).decode('ascii')
-        
+
         # Verification code, if needed
         vsalt = bcrypt.gensalt()
         vcode = hashlib.sha1(vsalt).hexdigest()
-        
+
         # Auto-verify unless verification is enabled.
         # This is so previously unverified accounts don'thave to verify
         # if we later turn verification on.
@@ -167,7 +167,7 @@ def run(API, environ, indata, session):
             verified = False
             sendCode(session, u, vcode) # Send verification email
             # If verification email fails, skip account creation.
-        
+
         doc = {
             'email': u,                         # Username (email)
             'password': pwd,                    # Hashed password
@@ -179,24 +179,24 @@ def run(API, environ, indata, session):
             'vcode': vcode,                     # Verification code
             'userlevel': "user"                 # User level (user/admin)
         }
-        
-        
+
+
         # If we have auto-invite on, check if there are orgs to invite to
         if 'autoInvite' in session.config['accounts']:
             dom = u.split('@')[-1].lower()
             for ai in session.config['accounts']['autoInvite']:
                 if ai['domain'] == dom:
                     doc['organisations'].append(ai['organisation'])
-                
+
         session.DB.ES.index(index=session.DB.dbname, doc_type='useraccount', id = u, body = doc)
         yield json.dumps({"message": "Account created!", "verified": verified})
         return
-    
+
     # We need to be logged in for the rest of this!
     if not session.user:
         raise API.exception(403, "You must be logged in to use this API endpoint! %s")
-    
-    
+
+
     # Patch (edit) an account
     if method == "PATCH":
         userid = session.user['email']
@@ -217,4 +217,3 @@ def run(API, environ, indata, session):
         session.DB.ES.index(index=session.DB.dbname, doc_type='useraccount', id = userid, body = udoc)
         yield json.dumps({"message": "Account updated!"})
         return
-