Skip to content

Commit 2ffabed

Browse files
praveennhiremathprhirema
praveennhiremath
and
prhirema
authored
AI/ML for Food and Wine Pairing (#243)
* Added the print statement to check the execution flow. * Changes for Food and Wine Pairing Module * New Python Project for Food Wine Pairing - AI/ML Added a service for Food Wine pairing. Calling service from InventoryServiceOrderEventConsumer. Service finds the suitable wines for the Food item and send the recommendedWines to user. * Added sk-learn library and changed python to 3.7 * Given permission to sh files before commiting and deleted unnecessary files. * added sh files of 777 permissions from Paul. * install python 3.7 - Docker file updated * install python using apt * removed rm yum * Added RUN each line to know the exact line errors * removed sudo in Docker file and added apt-get update * changed python3.7 to python. * changed python version to initial one 3.6 * removed python-oci-sdk * updated with python 3.7 workaround * removed FROM oraclelinux:7-slim, added FROM ubuntu:18.04, to resolve apt issue * set python 3.7 as defaukt * fetched the latest code and updated foodwinepairing deployment yaml file * returned no wines suggested * initialized client with client builder * added kubectl apply in deploy.sh * changed hostname to localhost * reverted url. * cleared DB related stuffs in deployment yaml file * cleared non-required configs from deployment yaml file * added cx_oracle * imported WineFoodPairings file in app.py * added install pandas and numpy * added nltk * added matplotlib * added ipython * Assigned input to test food * Fixed the issues which were there in 1st working version * Added foodwinepairing-python in undeploy.sh * Added the config for calling foodwinepairing service * Added isSuggestiveSaleAIEnabled config in inventory-helidon deployment yaml * corrected indentation of isSuggestiveSaleAIEnabled value * added the same content from main branch to resolve conflicts * added foodwinepairing-python * removed foodwinepairing-python * Added foodwinepairing-python after resolving conflict * removed the sysout statements * removed foodwinepairing-python from grabdish/deploy.sh Co-authored-by: prhirema <prhirema@PRHIREMA-IN.oradev.oraclecorp.com>
1 parent fc218a0 commit 2ffabed

File tree

95 files changed

+2452968
-4
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

95 files changed

+2452968
-4
lines changed

‎grabdish/foodwinepairing-python/Dockerfile‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
FROM ubuntu:18.04
2+
3+
ARG release=19
4+
ARG update=9
5+
6+
WORKDIR /app
7+
COPY foodwinepairing/requirements.txt .
8+
9+
# Update apt packages
10+
RUN apt update
11+
RUN apt upgrade -y
12+
13+
# Install python 3.7
14+
RUN apt install software-properties-common -y
15+
RUN add-apt-repository ppa:deadsnakes/ppa
16+
RUN apt install python3.7 -y
17+
18+
# Make python 3.7 the default
19+
RUN echo "alias python=python3.7" >> ~/.bashrc
20+
RUN export PATH=${PATH}:/usr/bin/python3.7
21+
RUN /bin/bash -c "source ~/.bashrc"
22+
23+
# Add 3.7 to the available alternatives
24+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1
25+
26+
# Set python3.7 as the default python
27+
RUN update-alternatives --set python /usr/bin/python3.7
28+
29+
# Install pip
30+
RUN apt install python3-pip -y
31+
RUN python -m pip install --upgrade pip
32+
33+
#wine-pairing dependencies
34+
RUN python -m pip install -U gensim
35+
RUN python -m pip install -U scikit-learn
36+
RUN python -m pip install -U pandas
37+
RUN python -m pip install -U numpy
38+
RUN python -m pip install -U nltk
39+
RUN python -m pip install -U matplotlib
40+
RUN python -m pip install -U ipython
41+
RUN python -m pip install -r requirements.txt
42+
43+
#to locate if/where gunicorn is
44+
RUN find / -name "gunicorn"
45+
46+
ADD common .
47+
ADD foodwinepairing .
48+
RUN echo 'Adding foodwinepairing done in DockerFile!'
49+
50+
CMD ["gunicorn", "app:app", "--config=config.py"]

‎grabdish/foodwinepairing-python/build.sh‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
## Copyright (c) 2021 Oracle and/or its affiliates.
3+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
4+
5+
SCRIPT_DIR=$(dirname $0)
6+
7+
IMAGE_NAME=foodwinepairing-python
8+
IMAGE_VERSION=0.1
9+
10+
11+
if [ -z "$DOCKER_REGISTRY" ]; then
12+
echo "DOCKER_REGISTRY not set. Will get it with state_get"
13+
export DOCKER_REGISTRY=$(state_get DOCKER_REGISTRY)
14+
fi
15+
16+
if [ -z "$DOCKER_REGISTRY" ]; then
17+
echo "Error: DOCKER_REGISTRY env variable needs to be set!"
18+
exit 1
19+
fi
20+
21+
22+
export IMAGE=${DOCKER_REGISTRY}/${IMAGE_NAME}:${IMAGE_VERSION}
23+
24+
docker build -t $IMAGE .
25+
26+
docker push $IMAGE

‎grabdish/foodwinepairing-python/common/dbmgr.py‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
## Copyright (c) 2021 Oracle and/or its affiliates.
2+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
3+
# Database Manager
4+
5+
import os
6+
import sys
7+
import logging
8+
from os import environ as env
9+
import cx_Oracle
10+
import threading
11+
import time
12+
import oci
13+
import base64
14+
# Parameters
15+
db_connection_count = int(env.get("DB_CONNECTION_COUNT", "1"))
16+
db_user = env.get('DB_USER').strip()
17+
region_id = env.get('OCI_REGION').strip()
18+
vault_secret_ocid = env.get('VAULT_SECRET_OCID').strip()
19+
k8s_secret_dbpassword = env.get('DB_PASSWORD').strip()
20+
db_connect_string = env.get('DB_CONNECT_STRING')
21+
22+
readyfile = ""
23+
logger = None
24+
pool = None
25+
26+
# Set ready file name (must correspond with name in app.yaml)
27+
def setReadyFileName(fileName):
28+
global readyfile
29+
readyfile = fileName
30+
31+
# Start the Database Manager thread
32+
def start(name):
33+
global logger
34+
logger = logging.getLogger(name)
35+
t = threading.Thread(None, run, name)
36+
t.daemon = True
37+
t.start()
38+
39+
# Acquire connection from pool
40+
def acquireConn():
41+
global pool
42+
if pool:
43+
try:
44+
conn = pool.acquire()
45+
except cx_Oracle.DatabaseError as e:
46+
error, = e.args
47+
# ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
48+
# ORA-12757: instance does not currently know of requested service
49+
# ORA-12541: TNS:no listener
50+
if error.code in [12514, 12757, 12541]:
51+
reportDown(error.code)
52+
raise DatabaseDown()
53+
else:
54+
raise
55+
else:
56+
return conn
57+
else:
58+
raise DatabaseDown()
59+
60+
# Release connection back to pool
61+
def releaseConn(conn):
62+
global pool
63+
if pool and conn:
64+
try:
65+
pool.release(conn)
66+
except:
67+
pass
68+
69+
# Database Manager Thread
70+
def run():
71+
global pool
72+
while True:
73+
if pool:
74+
time.sleep(10)
75+
continue
76+
77+
logger.debug("Create Connection Pool Started")
78+
try:
79+
80+
db_password = ""
81+
82+
if vault_secret_ocid != "":
83+
reportDown(error.code)
84+
signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
85+
secrets_client = oci.secrets.SecretsClient(config={'region': region_id}, signer=signer)
86+
secret_bundle = secrets_client.get_secret_bundle(secret_id = vault_secret_ocid)
87+
logger.debug(secret_bundle)
88+
base64_bytes = secret_bundle.data.secret_bundle_content.content.encode('ascii')
89+
message_bytes = base64.b64decode(base64_bytes)
90+
db_password = message_bytes.decode('ascii')
91+
else:
92+
db_password = k8s_secret_dbpassword
93+
94+
pool = cx_Oracle.SessionPool(
95+
db_user,
96+
db_password,
97+
db_connect_string,
98+
externalauth = False if db_password else True,
99+
encoding="UTF-8",
100+
min=db_connection_count,
101+
max=db_connection_count,
102+
increment=0,
103+
threaded=True,
104+
events=False,
105+
getmode=cx_Oracle.SPOOL_ATTRVAL_TIMEDWAIT,
106+
waitTimeout=10000)
107+
108+
except cx_Oracle.DatabaseError as e:
109+
error, = e.args
110+
# ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
111+
# ORA-12757: instance does not currently know of requested service
112+
# ORA-12541: TNS:no listener
113+
if error.code in [12514, 12757, 12541]:
114+
reportDown(error.code)
115+
else:
116+
raise
117+
else:
118+
logger.debug("Create Connection Pool Ended")
119+
reportUp()
120+
finally:
121+
time.sleep(10)
122+
123+
# Report the service down (see app.yaml)
124+
def reportDown(errno):
125+
global pool
126+
if pool:
127+
pool = None
128+
os.remove(readyfile)
129+
logger.debug(f"Database Reported Down Error {errno}")
130+
131+
# Report the service up (see app.yaml)
132+
def reportUp():
133+
open(readyfile, "w+").close()
134+
logger.debug("Database Reported Up")
135+
136+
# DatabaseDown Exception
137+
class DatabaseDown(Exception):
138+
def __init__(self):
139+
pass

‎grabdish/foodwinepairing-python/deploy.sh‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
## Copyright (c) 2021 Oracle and/or its affiliates.
3+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
4+
5+
SCRIPT_DIR=$(dirname $0)
6+
7+
export DOCKER_REGISTRY="$(state_get DOCKER_REGISTRY)"
8+
#export INVENTORY_PDB_NAME="$(state_get INVENTORY_DB_NAME)"
9+
export OCI_REGION="$(state_get OCI_REGION)"
10+
export VAULT_SECRET_OCID=""
11+
12+
echo create foodwinepairing-python-deployment deployment and service...
13+
export CURRENTTIME=$( date '+%F_%H:%M:%S' )
14+
echo CURRENTTIME is $CURRENTTIME ...this will be appended to generated deployment yaml
15+
16+
cp foodwinepairing-python-deployment.yaml foodwinepairing-python-deployment-$CURRENTTIME.yaml
17+
18+
IMAGE_NAME=foodwinepairing-python
19+
IMAGE_VERSION=0.1
20+
21+
#may hit sed incompat issue with mac
22+
sed_i "s|%DOCKER_REGISTRY%|${DOCKER_REGISTRY}|g" foodwinepairing-python-deployment-$CURRENTTIME.yaml
23+
#sed_i "s|%INVENTORY_PDB_NAME%|${INVENTORY_PDB_NAME}|g" foodwinepairing-python-deployment-$CURRENTTIME.yaml
24+
sed_i "s|%OCI_REGION%|${OCI_REGION}|g" foodwinepairing-python-deployment-${CURRENTTIME}.yaml
25+
sed_i "s|%VAULT_SECRET_OCID%|${VAULT_SECRET_OCID}|g" foodwinepairing-python-deployment-${CURRENTTIME}.yaml
26+
27+
if [ -z "$1" ]; then
28+
kubectl apply -f $SCRIPT_DIR/foodwinepairing-python-deployment-$CURRENTTIME.yaml -n msdataworkshop
29+
else
30+
kubectl apply -f <(istioctl kube-inject -f $SCRIPT_DIR/foodwinepairing-python-deployment-$CURRENTTIME.yaml) -n msdataworkshop
31+
fi
32+
33+
kubectl apply -f $SCRIPT_DIR/foodwinepairing-service.yaml -n msdataworkshop

‎grabdish/foodwinepairing-python/foodwinepairing-python-deployment.yaml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
## Copyright (c) 2021 Oracle and/or its affiliates.
2+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
3+
apiVersion: apps/v1
4+
kind: Deployment
5+
metadata:
6+
name: foodwinepairing-python
7+
spec:
8+
replicas: 1
9+
selector:
10+
matchLabels:
11+
app: foodwinepairing
12+
template:
13+
metadata:
14+
labels:
15+
app: foodwinepairing
16+
version: python
17+
spec:
18+
containers:
19+
- name: foodwinepairing
20+
image: %DOCKER_REGISTRY%/foodwinepairing-python:0.1
21+
imagePullPolicy: Always
22+
ports:
23+
- containerPort: 8080
24+
env:
25+
- name: WORKERS
26+
value: "1"
27+
- name: HTTP_THREADS
28+
value: "16"
29+
- name: PORT
30+
value: "8080"
31+
- name: DEBUG_MODE
32+
value: "1"
33+
restartPolicy: Always
34+

‎grabdish/foodwinepairing-python/foodwinepairing-service.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
##
3+
## Copyright (c) 2021 Oracle and/or its affiliates.
4+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
5+
apiVersion: v1
6+
kind: Service
7+
metadata:
8+
name: foodwinepairing
9+
labels:
10+
app: foodwinepairing
11+
spec:
12+
type: NodePort
13+
ports:
14+
- port: 8080
15+
name: http
16+
selector:
17+
app: foodwinepairing

0 commit comments

Comments
 (0)