Skip to content
Closed

D->M #180

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
233708b
Update controller.py
bio-boris Oct 7, 2022
7661ea9
Create pr_build.yml
bio-boris Oct 7, 2022
cd61d0c
Create release-main.yml
bio-boris Oct 7, 2022
c701a39
Create manual-build.yml
bio-boris Oct 7, 2022
f7df4ac
Update db.py
bio-boris Oct 7, 2022
83d5d04
Merge pull request #126 from kbase/add_logs
bio-boris Oct 7, 2022
7d47f2f
Update controller.py
bio-boris Oct 10, 2022
21139c6
Update db.py
bio-boris Oct 10, 2022
4b7def5
Merge pull request #130 from kbase/remove-logging
bio-boris Oct 10, 2022
7642a0c
Update pr_build.yml
bio-boris Dec 27, 2022
75769b3
Update controller.py
bio-boris Mar 2, 2024
2054b0d
Update controller.py
bio-boris Mar 2, 2024
923f87a
Merge conflict
Mar 2, 2024
dd630ce
add GHA test.yml, codeql, and dependabot.yml (#136)
Xiangs18 Dec 17, 2024
6ec3d63
mongo7 upgrade
Xiangs18 Dec 17, 2024
26b7d9e
upgrade pymongo to 4.7.2
Xiangs18 Jan 6, 2025
c5f1294
update python to 3.9.19
Xiangs18 Jan 6, 2025
1e57094
update Makefile and run_tests.sh scripts
Xiangs18 Jan 9, 2025
47ff558
update submodules
Xiangs18 Jan 9, 2025
4e1f8bf
fix gradle not found
Xiangs18 Jan 9, 2025
4e0c2dc
start nms service with docker-compose
Xiangs18 Jan 11, 2025
8095c7f
update nms submodule to the latest
Xiangs18 Jan 11, 2025
6f22c74
install docker-compose in GHA
Xiangs18 Jan 11, 2025
dff0b20
switch catalog mongo7 port to 27018
Xiangs18 Jan 11, 2025
76bff46
fix mongo port
Xiangs18 Jan 11, 2025
0b7cd2a
fix mongo port mapping
Xiangs18 Jan 11, 2025
66f2485
set admin_users
Xiangs18 Jan 11, 2025
85c65af
check admin users
Xiangs18 Jan 13, 2025
9fda21f
shut down NMS service properly
Xiangs18 Jan 13, 2025
91c72f7
fix nms curl
Xiangs18 Jan 13, 2025
dd1d2da
update mock_auth service
Xiangs18 Jan 14, 2025
2e84037
update the commit
Xiangs18 Jan 27, 2025
727753c
Update test.cfg.example
bio-boris Jan 27, 2025
77d161b
Update test.cfg.example
bio-boris Jan 27, 2025
bca78a9
Update core_registration_test.py
bio-boris Jan 27, 2025
178a63c
update message
Xiangs18 Jan 27, 2025
a3b1038
Merge branch 'dev-mongo7_upgrade' of github.com:kbase/catalog into de…
Xiangs18 Jan 27, 2025
5379082
test method_spec_admin_users
Xiangs18 Jan 28, 2025
631b52f
test simple_good_repo old commit
Xiangs18 Jan 28, 2025
b412672
test register hack
Xiangs18 Jan 28, 2025
80be977
remove jars nms submodules
Xiangs18 Jan 28, 2025
7ead4f7
update Dockerfile
Xiangs18 Jan 28, 2025
01f1539
remove kbapi_common submodule
Xiangs18 Jan 29, 2025
35a18c4
fix deprecated mongo auth
Xiangs18 Jan 29, 2025
c228438
remove all submodules in catalog
Xiangs18 Jan 29, 2025
9153235
create MongoClient through passing in params
Xiangs18 Jan 31, 2025
2344d64
add a comment for mock_auth server.py
Xiangs18 Jan 31, 2025
f147120
simplify code
Xiangs18 Jan 31, 2025
7a5b354
adjust params in MongoClient
Xiangs18 Feb 3, 2025
3e3673f
add debug log messages in list_user_favorites func
Xiangs18 Feb 4, 2025
093510e
try reinitiate mongoclient
Xiangs18 Feb 4, 2025
ec3064b
update __init__ in db.py file
Xiangs18 Feb 5, 2025
4aabfb9
fix typo
Xiangs18 Feb 5, 2025
fe87438
add check_db_schema
Xiangs18 Feb 5, 2025
5af0ccd
add boolean flag for schema check
Xiangs18 Feb 5, 2025
93e0033
collection direct call
Xiangs18 Feb 5, 2025
37f083d
remove debugging message
Xiangs18 Feb 5, 2025
d0c2522
display catalog version
Xiangs18 Feb 5, 2025
6b5fcde
fix failed unit test in startup_test.py
Xiangs18 Feb 5, 2025
0143000
trigger lazy load collection
Xiangs18 Feb 5, 2025
d04e2eb
clean up files
Xiangs18 Feb 6, 2025
59c653e
remove comments from Makefile
Xiangs18 Feb 7, 2025
0859dbe
validate Install Docker Compose step in ci
Xiangs18 Feb 8, 2025
dda9eac
use docker compose
Xiangs18 Feb 10, 2025
bbfc2db
map NMS to the local port 27018
Xiangs18 Feb 10, 2025
293953c
Add a clearer comment for the environment variable in test.yml
Xiangs18 Feb 11, 2025
7b4a495
move env to jobs level
Xiangs18 Feb 11, 2025
377f651
add more comments in server.py
Xiangs18 Feb 11, 2025
ae1b224
add more comments in db.py
Xiangs18 Feb 11, 2025
370c65e
remove check_db_schema related functions
Xiangs18 Feb 13, 2025
0015ac0
revert back changes
Xiangs18 Feb 13, 2025
1428c85
move self._db_schema_checked flag outside function
Xiangs18 Feb 13, 2025
d106e96
refactor _create_indexes function
Xiangs18 Feb 14, 2025
64f6fdb
encapsulate create_indexes in the DBIndexes class
Xiangs18 Feb 14, 2025
d6bd556
add comment at the top of server.py file
Xiangs18 Feb 15, 2025
96a72e6
refactor db.py file
Xiangs18 Feb 19, 2025
a4025ad
add flag for mongo client
Xiangs18 Feb 19, 2025
46731b1
update wrapper function
Xiangs18 Feb 19, 2025
9836af9
add initialize_mongo_client function manually
Xiangs18 Feb 19, 2025
1f6024a
add _close_mongo_client function
Xiangs18 Feb 19, 2025
4511078
try run tests without a lock
Xiangs18 Feb 20, 2025
8806f22
revert self.lock changes
Xiangs18 Feb 21, 2025
62bb424
Recreate the database and collection handles after reinitializing the…
Xiangs18 Feb 21, 2025
c919000
revert back tests
Xiangs18 Feb 21, 2025
dc79ddc
remove decorator related code
Xiangs18 Feb 21, 2025
0c7b906
add comment why need to manually close mongoclient
Xiangs18 Feb 21, 2025
9ae4b7b
update comments
Xiangs18 Feb 24, 2025
6c54bf0
refactor the db.py and add comments for docker-compose file
Xiangs18 Feb 25, 2025
26d8910
add a comment for _ensure_mongo_connection function
Xiangs18 Feb 25, 2025
e9db8f1
Clarify the comments in the db.py and docker-compose.yml files
Xiangs18 Feb 26, 2025
1c4ce22
remove NarrativeMethodStore section and update the test instructions …
Xiangs18 Feb 28, 2025
be5d794
expound on the explanation
Xiangs18 Feb 28, 2025
925e947
fix c&p error
Xiangs18 Feb 28, 2025
14d3590
fix typos and clarify comments
Xiangs18 Feb 28, 2025
de6d334
make sure NMS is clean after each test
Xiangs18 Mar 1, 2025
283b022
Merge pull request #140 from kbase/dev-mongo7_upgrade
Xiangs18 Mar 3, 2025
b0af933
remove uwsgi from dockerfile
Xiangs18 Mar 3, 2025
7724d77
add Pipenv and Pipenv.lock files
Xiangs18 Mar 3, 2025
8ec7c64
upload Pipfiles
Xiangs18 Mar 3, 2025
71ba35e
add missing lib pyyaml
Xiangs18 Mar 3, 2025
0c8e6c0
move semantic_version to packages
Xiangs18 Mar 3, 2025
57622f0
Merge pull request #145 from kbase/dev-pipenv
Xiangs18 Mar 4, 2025
ef31a9c
fix mongo_authMechanism name
Xiangs18 Mar 4, 2025
b9d056a
add retrywrites param
Xiangs18 Mar 5, 2025
9b1d4b9
move release notes from README.md to RELEASE_NOTES.md
Xiangs18 Mar 5, 2025
0f576bf
removed mongodb-retrywrites from test.cfg.example
Xiangs18 Mar 6, 2025
fa3fbf5
add tests for retryWrites
Xiangs18 Mar 7, 2025
3111c53
fix format
Xiangs18 Mar 7, 2025
91b5c74
add print meesage to verify retryWrites setting
Xiangs18 Mar 7, 2025
63f2287
remove print
Xiangs18 Mar 7, 2025
563a733
Merge pull request #146 from kbase/dev-add_retryWrites
Xiangs18 Mar 7, 2025
d6f3ded
fix typo in README.md
Xiangs18 Mar 7, 2025
8282bc4
Merge pull request #147 from kbase/dev-fix_typo
Xiangs18 Mar 7, 2025
8b5f518
uncomment trivy-scan
Xiangs18 Mar 11, 2025
538b42f
Merge pull request #152 from kbase/dev-pr_build
Xiangs18 Mar 11, 2025
bbac4a4
add gha to dependabot.yml
Xiangs18 Jun 12, 2025
8975a96
change schedule.interval to monthly
Xiangs18 Jun 16, 2025
85b5697
Merge pull request #166 from kbase/dev-update_dependabot
Xiangs18 Jun 16, 2025
7cb46a6
Add JITPack java client build
MrCreosote Sep 4, 2025
aad1a6d
Fix JITPack build
MrCreosote Sep 4, 2025
fd44229
Merge pull request #178 from kbase/dev-publish
MrCreosote Sep 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#
# https://help.github.com/articles/dealing-with-line-endings/
#
# Linux start script should use lf
/gradlew text eol=lf

# These are Windows script files and should use crlf
*.bat text eol=crlf

# Binary files should be left untouched
*.jar binary

52 changes: 52 additions & 0 deletions .github/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: "Code scanning - action"

on:
push:
pull_request:
schedule:
- cron: '0 19 * * 0'

jobs:
CodeQL-Build:

# CodeQL runs on ubuntu-latest and windows-latest
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2

# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
# Override language selection by uncommenting this and choosing your languages
# with:
# languages: go, javascript, csharp, python, cpp, java

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
26 changes: 26 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: 2
updates:

# Docker base image updates
- package-ecosystem: docker
directory: "/"
schedule:
interval: monthly
time: "11:00"
open-pull-requests-limit: 25

# Python (pip) dependencies
- package-ecosystem: pip
directory: "/"
schedule:
interval: monthly
time: "11:00"
open-pull-requests-limit: 25

# GitHub Actions dependencies
- package-ecosystem: "github-actions"
directory: ".github/workflows"
schedule:
interval: monthly
time: "11:00"
open-pull-requests-limit: 25
90 changes: 90 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: KBase Catalog test

on:
pull_request:
types:
- opened
- reopened
- synchronize
- ready_for_review
push:
# run workflow when merging to main or develop
branches:
- main
- master
- develop

jobs:
catalog_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- python-version: '3.9.19'
mongo-version: '3.6'
- python-version: '3.9.19'
mongo-version: '7.0.4'
services:
mongo:
image: mongo:${{matrix.mongo-version}}
ports:
- 27017:27017
options: --name mongo${{matrix.mongo-version}}

env:
KBASE_CI_TOKEN: ${{ secrets.KBASE_CI_TOKEN }}
# This env var is also used in the catalog test docker compose file for starting NMS
ADMIN_USER: ${{ secrets.KBASE_BOT_USER_CI }}

steps:
- name: Repo checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{matrix.python-version}}

- name: Install dependencies and set up test config
shell: bash

run: |

# test mongo connection
curl http://localhost:27017
returncode=$?
if [ $returncode != 0 ]; then exit $returncode; fi

# set HOMEDIR
export HOMEDIR=`pwd`

# move to parent dir to install binaries etc
cd ..

# setup kb-sdk
mkdir -p $(pwd)/bin
docker run ghcr.io/kbase/kb_sdk_patch-develop:br-0.0.4 genscript > $(pwd)/bin/kb-sdk
chmod 755 $(pwd)/bin/kb-sdk
export PATH=$(pwd)/bin:$PATH

# install catalog dependencies
cd $HOMEDIR
python -m pip install --upgrade pip
pip install pipenv
pipenv sync --system --dev

# setup test config
cp -n test/test.cfg.example test/test.cfg
sed -i "s#^nms-admin-token.*#nms-admin-token=$KBASE_CI_TOKEN#" test/test.cfg
sed -i "s#^method-spec-admin-users.*#method-spec-admin-users=$ADMIN_USER#" test/test.cfg

- name: Run tests
shell: bash
run: make test

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: true
Comment on lines +19 to +90

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium test

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 5 months ago

To fix this issue, add a permissions block at either the workflow root level (recommended for clarity and coverage) or directly to the catalog_tests job. The block should specify the minimal set of permissions required for the workflow to execute. For this workflow, only contents: read appears to be necessary, as the jobs mainly interact with repository source code and external services (Codecov, Docker, etc.), but do not interact with repository settings or write to PRs/issues. Place the following snippet under the name block (between lines 1 and 3 in the workflow) for root-level coverage, or under the catalog_tests job for per-job coverage.


Suggested changeset 1
.github/workflows/test.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -1,4 +1,6 @@
 name: KBase Catalog test
+permissions:
+  contents: read
 
 on:
   pull_request:
EOF
@@ -1,4 +1,6 @@
name: KBase Catalog test
permissions:
contents: read

on:
pull_request:
Copilot is powered by AI and may make mistakes. Always verify output.
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/bin/

# Ignore Gradle project-specific cache directory
.gradle

# Ignore Gradle build output directory
build
12 changes: 0 additions & 12 deletions .gitmodules

This file was deleted.

40 changes: 0 additions & 40 deletions .travis.yml

This file was deleted.

1 change: 1 addition & 0 deletions Catalog.html

Large diffs are not rendered by default.

4 changes: 0 additions & 4 deletions DEPENDENCIES

This file was deleted.

30 changes: 24 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,25 +1,43 @@
FROM kbase/sdkbase2:python AS build
FROM python:3.9.19 AS build

# The rsync installation is required for the Makefile
RUN apt-get update && apt-get install -y rsync
RUN mkdir -p /kb/deployment/lib/biokbase

COPY . /tmp/catalog
RUN cd /tmp/catalog && make deploy-service deploy-server-control-scripts

FROM kbase/sdkbase2:python
FROM python:3.9.19
# These ARGs values are passed in via the docker build command
ARG BUILD_DATE
ARG VCS_REF
ARG BRANCH

RUN apt-get update && apt-get install -y wget

# install dockerize
WORKDIR /opt
RUN wget -q https://github.com/kbase/dockerize/raw/master/dockerize-linux-amd64-v0.6.1.tar.gz \
&& tar xvzf dockerize-linux-amd64-v0.6.1.tar.gz \
&& rm dockerize-linux-amd64-v0.6.1.tar.gz
RUN mkdir -p /kb/deployment/bin/
RUN ln -s /opt/dockerize /kb/deployment/bin/dockerize

ENV KB_DEPLOYMENT_CONFIG "/kb/deployment/conf/deploy.cfg"

COPY --from=build /kb/deployment/lib/biokbase /kb/deployment/lib/biokbase
COPY --from=build /kb/deployment/services /kb/deployment/services
COPY --from=build /tmp/catalog/deployment/conf /kb/deployment/conf

SHELL ["/bin/bash", "-c"]
COPY requirements.txt requirements.txt
RUN source activate root && \
pip install -r requirements.txt
WORKDIR /tmp/catalog

# install pipenv
RUN pip install --upgrade pip && \
pip install pipenv

# install deps
COPY Pipfile* ./
RUN pipenv sync --system

LABEL org.label-schema.build-date=$BUILD_DATE \
org.label-schema.vcs-url="https://github.com/kbase/catalog.git" \
Expand Down
65 changes: 65 additions & 0 deletions KIDLspec.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
html, body {
height: 100%;
}
html {
display: table;
margin: auto;
}
body {
background-color: white;
color: #000;
font-family: Menlo, Monaco, Consolas, "Courier New", monospace;
font-weight: normal;
font-size: 12px;
margin: 0;
padding: 20px;
display: table-cell;
vertical-align: middle;
}
span.space {
display: inline-block;
width: 7px;
}
span.tab {
display: inline-block;
width: 30px;
}
span.keyword {
font-weight: bold;
color: #008;
}
span.name {
color: #000; !important
}
span.deprecated {
text-decoration: line-through;
}
span.annotation {
color: #303030;
}
span.primitive {
font-weight: bold;
color: #066;
}
div.body {
background-color: #ffffff;
color: #3e4349;
padding: 0 30px;
}
div.comment {
color: #A0A0A0;
}
a {
color: #004b6b;
text-decoration: none;
}
a:hover {
color: #6d4100;
text-decoration: underline;
}
:target {
background-color: #ffa;
}
div.body p, div.body dd, div.body li {
line-height: 1.4em;
}
Loading