Skip to content

Feature/update #694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ChrFr wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Feature/update #694

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
9b48e34
update to django 3
ChrFr Feb 10, 2023
10a7ea7
test setup fixes; skip graph tests
ChrFr Mar 6, 2023
39e35c7
test fixes
ChrFr Mar 9, 2023
f800c07
removed pycairo
ChrFr Mar 10, 2023
527b81b
new dockerfile; configured circle ci to test with new docker image ag…
ChrFr Mar 10, 2023
bd024fe
circleci config
ChrFr Mar 10, 2023
24b562a
circleci config
ChrFr Mar 10, 2023
e1b641e
layer proxy - fixed authentication
ChrFr Aug 28, 2024
d433eab
docker changes
ChrFr Aug 28, 2024
54f0f4c
django version
ChrFr Aug 28, 2024
168d660
django filter version
ChrFr Aug 28, 2024
282fef1
rest framework version
ChrFr Aug 28, 2024
08dfa37
fixed flow filter; Dockerfile update
ChrFr Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 24 additions & 27 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,39 @@
version: 2
version: 2.1
machine:
environment:
DJANGO_SETTINGS_MODULE: repair.settings4tests
DJANGO_SETTINGS_MODULE: repair.settings_circleci
jobs:
django testing:
docker:
- image: maxboh/docker-circleci-node-miniconda-gdal:graph_tool_stretch
- image: gertzgutscheruemenapp/repair-web
- image: circleci/postgres:13-postgis-ram
environment:
POSTGRES_USER: postgres
POSTGRES_DB: circle_test
POSTGRES_PASSWORD: postgres
steps:
- checkout
- run: mkdir test-reports
- run: source activate repair
- run:
name: Install Python deps in a venv
command: |
pip install -r requirements-dev.txt
- run:
name: Install dockerize
command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.6.1
- run:
name: Wait for Router
command: dockerize -wait tcp://localhost:8001 -timeout 1m
- run:
name: Wait for Database
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Migrate Test Database
command: |
python manage.py migrate --settings=repair.settings4tests
python manage.py migrate --settings=datentool.settings_circleci
- store_artifacts:
path: test-reports/
destination: tr1
Expand All @@ -25,13 +42,13 @@ jobs:
- run:
name: codecov
command: |
coverage run manage.py test --settings=repair.settings4tests
coverage run manage.py test --settings=repair.settings_circleci
codecov


selenium testing:
docker:
- image: maxboh/docker-circleci-node-miniconda-gdal:graph_tool_stretch
- image: gertzgutscheruemenapp/repair-web
steps:
- checkout
- run:
Expand Down Expand Up @@ -72,28 +89,9 @@ jobs:
path: repair/tests/artifacts


setup database:
docker:
- image: maxboh/docker-circleci-node-miniconda-gdal:graph_tool_stretch
steps:
- checkout
- run:
name: Install Python deps in a venv
command: |
pip install -r requirements-dev.txt
- run:
name: Migrate Test Database again
command: |
python manage.py migrate --run-syncdb --settings=repair.settings4tests
- run:
name: load fixtures to test database
command: |
python manage.py loaddata sandbox_data --settings=repair.settings4tests


jasmine tests:
docker:
- image: maxboh/docker-circleci-node-miniconda-gdal:graph_tool_stretch
- image: gertzgutscheruemenapp/repair-web
steps:
- checkout
- run: mkdir test-reports
Expand Down Expand Up @@ -129,4 +127,3 @@ workflows:
build-and-deploy:
jobs:
- django testing
- setup database
36 changes: 33 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,35 @@
FROM maxboh/docker-circleci-node-miniconda-gdal:graph_tool_stretch
FROM osgeo/gdal:ubuntu-small-latest
RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh
RUN bash nodesource_setup.sh
RUN apt-get -y update \
&& apt-get -y upgrade \
&& apt-get -y install git \
&& apt-get -y install nodejs \
&& apt-get -y install python3-pip \
&& apt-get -y install libpq-dev \
&& apt-get -y install binutils libproj-dev libgeos-dev \
&& apt-get -y install osmium-tool \
&& apt-get -y install language-pack-de wget \
&& apt-get -y install libxcursor1 libgtk-3-0 \
&& apt-get install -y libpq-dev \
&& apt-get install -y gettext imagemagick ghostscript

RUN npm install --global yarn

RUN mkdir /miniconda3
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /miniconda3/miniconda.sh
RUN bash /miniconda3/miniconda.sh -b -u -p /miniconda3
RUN /miniconda3/bin/conda create -n repair -c conda-forge graph-tool gdal python=3.10

RUN git clone https://github.com/MaxBo/REPAiR-Web.git /repairweb
WORKDIR /repairweb
RUN git pull
RUN git checkout feature/update

ENV PATH="${PATH}:/miniconda3/bin"
SHELL ["conda", "run", "-n", "repair", "/bin/bash", "-c"]

RUN python -m pip install pip==22
RUN pip install -r /repairweb/requirements.txt

RUN git clone https://github.com/MaxBo/REPAiR-Web.git $HOME/repairweb
RUN cd $HOME/repairweb

1 change: 1 addition & 0 deletions repair/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ static/bundles
!static/data/*.json
/settings_dev_local_pg.py
/settings_tests_local_pg.py
/settings_local.py
8 changes: 0 additions & 8 deletions repair/apps/asmfa/apps.py
View file
Open in desktop

This file was deleted.

3 changes: 2 additions & 1 deletion repair/apps/asmfa/graphs/graph.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
graph_tools_failed = False
try:
import graph_tool as gt
from graph_tool import stats as gt_stats
from graph_tool import draw, util
import cairo
except ModuleNotFoundError:
pass
graph_tools_failed = True

from django.db.models import Q, Sum, F
from django.db.models.functions import Coalesce
Expand Down
2 changes: 1 addition & 1 deletion repair/apps/asmfa/serializers/locations.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ class AdministrativeLocationSerializer(PatchFields,
'actor__activity__activitygroup__keyflow__casestudy__id',
'keyflow_pk':
'actor__activity__activitygroup__keyflow__id', }
actor = ActorIDField()
actor = ActorIDField(required=False)
area = serializers.PrimaryKeyRelatedField(required=False, allow_null=True,
queryset=Area.objects.all())
level = serializers.PrimaryKeyRelatedField(
Expand Down
2 changes: 1 addition & 1 deletion repair/apps/asmfa/tests/data/T3.2_Activities.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ F-4391 F-4391 Roofing activities F
F-4222 F-4222 Construction of utility projects for electricity and telecommunications F
F-4291 F-4291 Construction of water projects F
F-4299 F-4299 Construction of other civil engineering projects n.e.c. F
F-4212 F-4212 Construction of railways and underground railways� F
F-4212 F-4212 Construction of railways and underground railways F
F-4333 F-4333 Floor and wall covering F
F-4329 F-4329 Other construction installation F
F-4332 F-4332 Joinery installation F
Expand Down
29 changes: 15 additions & 14 deletions repair/apps/asmfa/tests/test_activitygroup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,21 @@ def setUpClass(cls):
keyflow_pk=cls.keyflow)
cls.url_pk = dict(pk=cls.activitygroup)

def setUp(self):
super().setUp()
self.activitygroup1 = ActivityGroupFactory(name='MyGroup',
keyflow=self.kic)
self.activitygroup2 = ActivityGroupFactory(name='AnotherGroup',
keyflow=self.kic)
self.activity1 = ActivityFactory(nace='NACE1',
activitygroup=self.activitygroup1)
self.activity2 = ActivityFactory(nace='NACE2',
activitygroup=self.activitygroup1)
self.activity3 = ActivityFactory(nace='NACE3',
activitygroup=self.activitygroup1)
self.activity4 = ActivityFactory(nace='NACE4',
activitygroup=self.activitygroup2)
@classmethod
def setUpTestData(cls):
super().setUpTestData()
cls.activitygroup1 = ActivityGroupFactory(name='MyGroup',
keyflow=cls.kic_obj)
cls.activitygroup2 = ActivityGroupFactory(name='AnotherGroup',
keyflow=cls.kic_obj)
cls.activity1 = ActivityFactory(nace='NACE1',
activitygroup=cls.activitygroup1)
cls.activity2 = ActivityFactory(nace='NACE2',
activitygroup=cls.activitygroup1)
cls.activity3 = ActivityFactory(nace='NACE3',
activitygroup=cls.activitygroup1)
cls.activity4 = ActivityFactory(nace='NACE4',
activitygroup=cls.activitygroup2)

def test_nace_codes(self):
"""
Expand Down
37 changes: 20 additions & 17 deletions repair/apps/asmfa/tests/test_actors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,24 +62,26 @@ def setUpClass(cls):
year=2017,
turnover='1000.00',
employees=2,
activity=1,
activity=cls.activity_obj.id,
BvDid='141234',
reason=cls.reason1_id)
cls.put_data = dict(name='posttestname',
year=2017,
turnover='1000.00',
employees=2,
activity=1,
activity=cls.activity_obj.id,
BvDid='141234',
reason=cls.reason2_id)
cls.patch_data = dict(name='patchtestname')

def setUp(self):
super().setUp()
self.reason1 = ReasonFactory(id=self.reason1_id, reason='Reason 1')
self.reason2 = ReasonFactory(id=self.reason2_id, reason='Reason 2')
self.obj = ActorFactory(activity__activitygroup__keyflow=self.kic,
reason=self.reason1)
@classmethod
def setUpTestData(cls):
super().setUpTestData()
cls.reason1 = ReasonFactory(id=cls.reason1_id, reason='Reason 1')
cls.reason2 = ReasonFactory(id=cls.reason2_id, reason='Reason 2')
cls.activity_obj = ActivityFactory(activitygroup__keyflow=cls.kic_obj)
cls.obj = ActorFactory(activity=cls.activity_obj,
reason=cls.reason1)

def test_reason(self):
"""Test reason for exclusion"""
Expand Down Expand Up @@ -142,12 +144,12 @@ def setUpClass(cls):
activitygroup=cls.activitygroup)
cls.patch_data = dict(name='Test Name')

def setUp(self):
super().setUp()
self.obj = ActivityFactory(
activitygroup__keyflow__casestudy=self.uic.casestudy,
activitygroup__keyflow=self.kic,
activitygroup__id=self.activitygroup)
@classmethod
def setUpTestData(cls):
super().setUpTestData()
cls.obj = ActivityFactory(
activitygroup__keyflow=cls.kic_obj,
activitygroup__id=cls.activitygroup)

def test_unique_nacecode(self):
"""Test if the nace-code number is unique"""
Expand Down Expand Up @@ -188,6 +190,7 @@ def setUpClass(cls):
cls.put_data = dict(code="P1", name='Test Code')
cls.patch_data = dict(name='P1')

def setUp(self):
super().setUp()
self.obj = ActivityGroupFactory(keyflow=self.kic)
@classmethod
def setUpTestData(cls):
super().setUpTestData()
cls.obj = ActivityGroupFactory(keyflow=cls.kic_obj)
Loading