"
+ ],
+ "text/plain": [
+ " truth \\\n",
+ "0 nine \n",
+ "1 De Fabio Factory and Apple \n",
+ "2 Oyakodon Yakizakana and Carrot Halwa \n",
+ "3 where can I find the best Red ruby \n",
+ "4 Kuih Kaswi waffle and Coconut Kuih \n",
+ "5 please tell me how to cook Satay Babat \n",
+ "6 six five two three nine seven nine three \n",
+ "7 Denis D Cotta Paine Eric and Chong Tze Chien \n",
+ "8 I can't eat all of this Satay Babat \n",
+ "9 Maxi Cash Cadbury and Ajinomoto \n",
+ "\n",
+ " output \\\n",
+ "0 nine \n",
+ "1 disable Factory in apple \n",
+ "2 oil Condon and carrot halwa \n",
+ "3 where can I find the best way to be \n",
+ "4 Quaker sweet waffle in Coconut Creek \n",
+ "5 please tell me how to cook saute Bobbitt \n",
+ "6 six five to three nine seven nine three \n",
+ "7 Dennis Dakota and Chong searching \n",
+ "8 I can't eat all of this Saturday Bobbitt \n",
+ "9 Mexican Inn Camp Bowie and Angie nomoto \n",
+ "\n",
+ " truth_split \\\n",
+ "0 ['nine'] \n",
+ "1 ['de', 'fabio', 'factory', 'and', 'apple'] \n",
+ "2 ['oyakodon', 'yakizakana', 'and', 'carrot', 'halwa'] \n",
+ "3 ['where', 'can', 'i', 'find', 'the', 'best', 'red', 'ruby'] \n",
+ "4 ['kuih', 'kaswi', 'waffle', 'and', 'coconut', 'kuih'] \n",
+ "5 ['please', 'tell', 'me', 'how', 'to', 'cook', 'satay', 'babat'] \n",
+ "6 ['six', 'five', 'two', 'three', 'nine', 'seven', 'nine', 'three'] \n",
+ "7 ['denis', 'd', 'cotta', 'paine', 'eric', 'and', 'chong', 'tze', 'chien'] \n",
+ "8 ['i', \"can't\", 'eat', 'all', 'of', 'this', 'satay', 'babat'] \n",
+ "9 ['maxi', 'cash', 'cadbury', 'and', 'ajinomoto'] \n",
+ "\n",
+ " output_split \\\n",
+ "0 ['nine'] \n",
+ "1 ['disable', 'factory', 'in', 'apple'] \n",
+ "2 ['oil', 'condon', 'and', 'carrot', 'halwa'] \n",
+ "3 ['where', 'can', 'i', 'find', 'the', 'best', 'way', 'to', 'be'] \n",
+ "4 ['quaker', 'sweet', 'waffle', 'in', 'coconut', 'creek'] \n",
+ "5 ['please', 'tell', 'me', 'how', 'to', 'cook', 'saute', 'bobbitt'] \n",
+ "6 ['six', 'five', 'to', 'three', 'nine', 'seven', 'nine', 'three'] \n",
+ "7 ['dennis', 'dakota', 'and', 'chong', 'searching'] \n",
+ "8 ['i', \"can't\", 'eat', 'all', 'of', 'this', 'saturday', 'bobbitt'] \n",
+ "9 ['mexican', 'inn', 'camp', 'bowie', 'and', 'angie', 'nomoto'] \n",
+ "\n",
+ " truth_count match_count \n",
+ "0 1 1 \n",
+ "1 5 2 \n",
+ "2 5 3 \n",
+ "3 8 6 \n",
+ "4 5 2 \n",
+ "5 8 6 \n",
+ "6 6 5 \n",
+ "7 9 2 \n",
+ "8 8 6 \n",
+ "9 5 1 "
+ ]
+ },
+ "execution_count": 60,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Sample transcribed text vs truth\n",
+ "speaker = pd.read_csv(f\"output/2002.csv\")\n",
+ "speaker"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
sex
\n",
+ "
race
\n",
+ "
truth_count
\n",
+ "
match_count
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
77
\n",
+ "
60
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
84
\n",
+ "
64
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
80
\n",
+ "
53
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
F
\n",
+ "
INDIAN
\n",
+ "
80
\n",
+ "
53
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
70
\n",
+ "
45
\n",
+ "
\n",
+ "
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
\n",
+ "
\n",
+ "
293
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
74
\n",
+ "
50
\n",
+ "
\n",
+ "
\n",
+ "
294
\n",
+ "
M
\n",
+ "
MALAY
\n",
+ "
68
\n",
+ "
52
\n",
+ "
\n",
+ "
\n",
+ "
295
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
82
\n",
+ "
58
\n",
+ "
\n",
+ "
\n",
+ "
296
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
79
\n",
+ "
52
\n",
+ "
\n",
+ "
\n",
+ "
297
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
82
\n",
+ "
57
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
298 rows × 4 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " sex race truth_count match_count\n",
+ "0 F CHINESE 77 60\n",
+ "1 F CHINESE 84 64\n",
+ "2 M CHINESE 80 53\n",
+ "3 F INDIAN 80 53\n",
+ "4 M CHINESE 70 45\n",
+ ".. .. ... ... ...\n",
+ "293 M CHINESE 74 50\n",
+ "294 M MALAY 68 52\n",
+ "295 F CHINESE 82 58\n",
+ "296 F CHINESE 79 52\n",
+ "297 M CHINESE 82 57\n",
+ "\n",
+ "[298 rows x 4 columns]"
+ ]
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# output to be passed into feat tests (subgroup disparity and min/max threshold)\n",
+ "df_speaker=pd.read_csv('../data/speech_sample_output.csv')\n",
+ "df_speaker"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "race\n",
+ "CHINESE 0.671469\n",
+ "INDIAN 0.690566\n",
+ "MALAY 0.683185\n",
+ "Name: rate, dtype: float64"
+ ]
+ },
+ "execution_count": 24,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Sample accuracy stats across race\n",
+ "\n",
+ "tmp=df_speaker.copy()\n",
+ "tmp['rate']=tmp.match_count/tmp.truth_count\n",
+ "tmp.groupby('race').rate.mean()"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "veritas",
+ "language": "python",
+ "name": "veritas"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/examples/model_card_output/data/speech_to_text_example.proto b/examples/model_card_output/data/speech_to_text_example.proto
new file mode 100644
index 0000000..3528fab
Binary files /dev/null and b/examples/model_card_output/data/speech_to_text_example.proto differ
diff --git a/examples/model_card_output/model_cards/speech_to_text_example.html b/examples/model_card_output/model_cards/speech_to_text_example.html
new file mode 100644
index 0000000..0a29f35
--- /dev/null
+++ b/examples/model_card_output/model_cards/speech_to_text_example.html
@@ -0,0 +1,2097 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Model Card for Speech to Text Model
+
+
+
+
+
+
+
+
+
+ Model Card for Speech to Text Model
+
+
+
+
+
+
+
+
Model Details
+
+
Overview
+
+ Using an automated transcription tool, the government want to transcribe audio files recorded in trials for evidential purposes. The tool should be able to capture the various spoken accents and transcribe into text accurately so as not to bias against the defendant. The tool chosen here will be Google's Speech to Text Model evaluated on a set of audio files by National Speech Corpus.
+
+
Version
+
+
+
+
+
+
name: v1
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Owners
+
+
+
+
+
+
Timothy, Product Owner(s)
+
+
+
+
Swan, Model Developer(s)
+
+
+
+
Jason, Reviewer(s)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Regulatory requirements
+
+ MAS Fairness, Ethics, Accountability and Transparency (FEAT) principles
+
+
+
+
+
+
+
+
+
+
Considerations
+
+
+
+
Intended Users
+
+
+
+
+
+
+
+
+
+
Judical System and Defendant
+
+
+
+
+
+
+
+
+
+
+
+
Use Cases
+
+
+
+
+
+
+
+
+
+
Automate transcriptions and reduce human error and manpower in making court decisions
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Fairness Considerations
+
+
+
+
+
+
Group at risk: gender and race
+
+
Benefits: Reduce manpower and human error on transcription services. A more accurate model than current system will aid the judicial system in making better court decisions
+
+
Harms: Incorrectly transcribed/interpreted texts can impact on the defendant/ruling negatively.
+
+
Mitigation Strategy: Improve upon the existing model by further training and enhancements
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Datasets
+
+
+
+
+
+
+
+
National Speech Corpus Dataset
+
+
Dataset taken from https://www.imda.gov.sg/programme-listing/digital-services-lab/national-speech-corpus
+
+
+
+
Sensitive data
+
+
+
+
+
+
gender
+
+
+
+
race
+
+
+
+
+
+
+
+
Sensitive data used in model
+
+
+
+
+
+
N.A. (Protected attributes are not trained in the speech to text model)
+
+
+
+
+
+
+
+
+
+
Justification
+
+
N.A.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Quantitative Analysis
+
+
+
+
+
+
+
+
+
+
MSE - 712.513 (Test Set)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
MAE - 26.003 (Test Set)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
About 65% of the words are correctly transcribed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
MSE across sensitive groups
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Min Max Threshold Test
+
+
+
+
+
+
+
+ Description:
+
+ Test if the mae of the subgroups within sex
+is lower than the threshold of 27.
+
+
+
+
+
+
+
+
+
+
+ Threshold:
+
+ 27
+
+
+
+
+
+
+
+
+
+ Result:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
mae
+
+
+
+
passed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
sex_F
+
+
+
+
26.143
+
+
+
+
True
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
sex_M
+
+
+
+
25.891
+
+
+
+
True
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Passed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Min Max Threshold Test
+
+
+
+
+
+
+
+ Description:
+
+ Test if the mae of the subgroups within race
+is lower than the threshold of 27.
+
+
+
+
+
+
+
+
+
+
+ Threshold:
+
+ 27
+
+
+
+
+
+
+
+
+
+ Result:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
mae
+
+
+
+
passed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
race_CHINESE
+
+
+
+
27.141
+
+
+
+
False
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
race_INDIAN
+
+
+
+
25.404
+
+
+
+
True
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
race_MALAY
+
+
+
+
25.47
+
+
+
+
True
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Failed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Fairness Analysis
+
+
+
+
+
+
+
+
+
+
Equal mean squared error
+
+
+
+
+
+ Segment:
+
+ Gender and race
+
+
+
+
+
+
+
+
+
+ Description:
+
+ Disparity ratio of MSE of any 2 bins in the respective attribute should not be more than a factor of 1.1
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Subgroup Disparity Test
+
+
+
+
+
+
+
+ Description:
+
+ Test if the maximum ratio of the mean squared error of any 2
+groups within race attribute exceeds 1.1. To
+pass, this value cannot exceed the threshold.
+
+
+
+
+
+
+
+
+
+ Threshold:
+
+ 1.1
+
+
+
+
+
+
+
+
+
+ Result:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
race_mse_max_ratio
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
0
+
+
+
+
1.136
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Failed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Subgroup Disparity Test
+
+
+
+
+
+
+
+ Description:
+
+ Test if the maximum ratio of the mean squared error of any 2
+groups within sex attribute exceeds 1.1. To
+pass, this value cannot exceed the threshold.
+
+
+
+
+
+
+
+
+
+ Threshold:
+
+ 1.1
+
+
+
+
+
+
+
+
+
+ Result:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
sex_mse_max_ratio
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
0
+
+
+
+
1.023
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Passed
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/examples/sample-form-response-speech-to-text.json b/examples/sample-form-response-speech-to-text.json
new file mode 100644
index 0000000..9ef78a9
--- /dev/null
+++ b/examples/sample-form-response-speech-to-text.json
@@ -0,0 +1,301 @@
+{
+ "eventId": "4ceb5c3b-2878-4546-ad3e-9dab1147ba06",
+ "eventType": "FORM_RESPONSE",
+ "createdAt": "2021-09-26T07:14:49.410Z",
+ "data": {
+ "responseId": "100e0116-86e9-4625-8f80-dd539392104c",
+ "respondentId": "mZoZ90",
+ "formId": "mR4Nlw",
+ "formName": "Model Ops Card Builder",
+ "createdAt": "2021-09-26T07:14:49.000Z",
+ "fields": [
+ {
+ "key": "question_3yXOqx",
+ "label": "Name",
+ "type": "INPUT_TEXT",
+ "value": "Speech to Text Model"
+ },
+ {
+ "key": "question_waO1Wy",
+ "label": "Overview",
+ "type": "TEXTAREA",
+ "value": "Using an automated transcription tool, the government want to transcribe audio files recorded in trials for evidential purposes. The tool should be able to capture the various spoken accents and transcribe into text accurately so as not to bias against the defendant. The tool chosen here will be Google's Speech to Text Model evaluated on a set of audio files by National Speech Corpus."
+ },
+ {
+ "key": "question_3X5jBY",
+ "label": "Intended Users",
+ "type": "INPUT_TEXT",
+ "value": "Judical System and Defendant"
+ },
+ {
+ "key": "question_w8N9pP",
+ "label": "Intended Use Cases",
+ "type": "TEXTAREA",
+ "value": "Automate transcriptions and reduce human error and manpower in making court decisions"
+ },
+ {
+ "key": "question_m6DYq5",
+ "label": "Version",
+ "type": "INPUT_TEXT",
+ "value": "v1"
+ },
+ {
+ "key": "question_wbZMaE",
+ "label": "Product Owner(s)",
+ "type": "INPUT_TEXT",
+ "value": "Timothy"
+ },
+ {
+ "key": "question_wArYkN",
+ "label": "Model Developer(s)",
+ "type": "INPUT_TEXT",
+ "value": "Swan"
+ },
+ {
+ "key": "question_mBEYvY",
+ "label": "Reviewer(s)",
+ "type": "INPUT_TEXT",
+ "value": "Jason"
+ },
+ {
+ "key": "question_wzEBxk",
+ "label": "Please select any regulatory guidelines which the model should be in compliance with",
+ "type": "CHECKBOXES",
+ "value": [
+ "1f033a0c-1fa7-44a8-9309-0fe858283844",
+ "0a574b19-1a88-42e6-b4ba-b288b3fec5a4"
+ ],
+ "options": [
+ {
+ "id": "0a574b19-1a88-42e6-b4ba-b288b3fec5a4",
+ "text": "MAS Fairness, Ethics, Accountability and Transparency (FEAT) principles"
+ },
+ {
+ "id": "0e0e9512-6ea8-496e-9ee4-83dc1a23a5a7",
+ "text": "HKMA AI principles"
+ },
+ {
+ "id": "5fcd3e36-042c-4c71-b76a-e9ec325f6205",
+ "text": "Bank of England and FCA's guideline"
+ }
+ ]
+ },
+ {
+ "key": "question_wzEBxk_0a574b19-1a88-42e6-b4ba-b288b3fec5a4",
+ "label": "Please select any regulatory guidelines which the model should be in compliance with (MAS Fairness, Ethics, Accountability and Transparency (FEAT) principles)",
+ "type": "CHECKBOXES",
+ "value": true
+ },
+ {
+ "key": "question_wzEBxk_0e0e9512-6ea8-496e-9ee4-83dc1a23a5a7",
+ "label": "Please select any regulatory guidelines which the model should be in compliance with (HKMA AI principles)",
+ "type": "CHECKBOXES",
+ "value": false
+ },
+ {
+ "key": "question_wzEBxk_5fcd3e36-042c-4c71-b76a-e9ec325f6205",
+ "label": "Please select any regulatory guidelines which the model should be in compliance with (Bank of England and FCA's guideline)",
+ "type": "CHECKBOXES",
+ "value": false
+ },
+ {
+ "key": "question_wzEBxk_1f033a0c-1fa7-44a8-9309-0fe858283844",
+ "label": "Please select any regulatory guidelines which the model should be in compliance with (Not applicable)",
+ "type": "CHECKBOXES",
+ "value": true
+ },
+ {
+ "key": "question_n0V4jj",
+ "label": "Name of dataset",
+ "type": "INPUT_TEXT",
+ "value": "National Speech Corpus Dataset"
+ },
+ {
+ "key": "question_wM10jE",
+ "label": "Description of dataset",
+ "type": "TEXTAREA",
+ "value": "Dataset taken from https://www.imda.gov.sg/programme-listing/digital-services-lab/national-speech-corpus"
+ },
+ {
+ "key": "question_wzEVQ0",
+ "label": "Protected attributes in dataset",
+ "type": "TEXTAREA",
+ "value": "gender, race"
+ },
+ {
+ "key": "question_w5XNqE",
+ "label": "Protected attributes in production",
+ "type": "TEXTAREA",
+ "value": "N.A. (Protected attributes are not trained in the speech to text model)"
+ },
+ {
+ "key": "question_3Eqpo2",
+ "label": "Justification of use of protected attributes",
+ "type": "TEXTAREA",
+ "value": "N.A."
+ },
+ {
+ "key": "question_wdb72K",
+ "label": "Name of dataset 2",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_mJ1yEz",
+ "label": "Description of dataset 2",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_mYj2zJ",
+ "label": "Protected attributes in dataset 2",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_mDqLvZ",
+ "label": "Protected attributes in production (dataset 2)",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_3jeK8Y",
+ "label": "Justification of use of protected attributes (dataset 2)",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_3laplv",
+ "label": "Name of dataset 3",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_wgbB8M",
+ "label": "Description of dataset 3",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_mRWOBj",
+ "label": "Protected attributes in dataset 3",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_wo9JVx",
+ "label": "Protected attributes in production (dataset 3)",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_3yXO86",
+ "label": "Justification of use of protected attributes (dataset 3)",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_3je4aQ",
+ "label": "What is the key metric used in evaluating the model's performance?",
+ "type": "INPUT_TEXT",
+ "value": "Mean Squared Error"
+ },
+ {
+ "key": "question_mOQRBg",
+ "label": "2nd metric (if applicable)",
+ "type": "INPUT_TEXT",
+ "value": "Mean Absolute Error"
+ },
+ {
+ "key": "question_mVpr1y",
+ "label": "3rd metric (if applicable)",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_mKp4dz",
+ "label": "Who are the individuals and groups that are considered to be at-risk of being systematically disadvantaged by the system?",
+ "type": "TEXTAREA",
+ "value": "gender and race"
+ },
+ {
+ "key": "question_31WYq4",
+ "label": "Expected Benefits",
+ "type": "TEXTAREA",
+ "value": "Reduce manpower and human error on transcription services. A more accurate model than current system will aid the judicial system in making better court decisions"
+ },
+ {
+ "key": "question_nGeMBk",
+ "label": "Expected Harms",
+ "type": "TEXTAREA",
+ "value": "Incorrectly transcribed/interpreted texts can impact on the defendant/ruling negatively."
+ },
+ {
+ "key": "question_mJ1r5z",
+ "label": "Mitigation strategies",
+ "type": "TEXTAREA",
+ "value": "Improve upon the existing model by further training and enhancements"
+ },
+ {
+ "key": "question_nPRoBe",
+ "label": "Type of fairness analysis conducted",
+ "type": "INPUT_TEXT",
+ "value": "Equal mean squared error"
+ },
+ {
+ "key": "question_3Eqpbr",
+ "label": "Segment of analysis",
+ "type": "INPUT_TEXT",
+ "value": "Gender and race"
+ },
+ {
+ "key": "question_nrDXrR",
+ "label": "Description of fairness analysis",
+ "type": "TEXTAREA",
+ "value": "Disparity ratio of MSE of any 2 bins in the respective attribute should not be more than a factor of 1.1"
+ },
+ {
+ "key": "question_w4aZNY",
+ "label": "Type of fairness analysis conducted 2",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_3jeK94",
+ "label": "Segment of analysis 2",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_w2j2x9",
+ "label": "Description of fairness analysis 2",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_3xVGN5",
+ "label": "Segment of analysis 3",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_mZ9xRB",
+ "label": "Type of fairness analysis conducted 3",
+ "type": "INPUT_TEXT",
+ "value": null
+ },
+ {
+ "key": "question_3NqzBj",
+ "label": "Description of fairness analysis 3",
+ "type": "TEXTAREA",
+ "value": null
+ },
+ {
+ "key": "question_n0V0Q9",
+ "label": "Please enter your email",
+ "type": "INPUT_EMAIL",
+ "value": "timothy.lin@cylynx.io"
+ }
+ ]
+ }
+}
\ No newline at end of file
diff --git a/examples/speech_to_text_example.ipynb b/examples/speech_to_text_example.ipynb
new file mode 100644
index 0000000..dbcc0d5
--- /dev/null
+++ b/examples/speech_to_text_example.ipynb
@@ -0,0 +1,2605 @@
+{
+ "cells": [
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import sys\n",
+ "import json\n",
+ "from IPython import display\n",
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from sklearn.metrics import mean_squared_error, mean_absolute_error\n",
+ "\n",
+ "# parent directory to work with dev\n",
+ "sys.path.insert(0, '..')\n",
+ "import verifyml.model_card_toolkit as mctlib\n",
+ "from verifyml.model_card_toolkit import model_card_pb2, ModelCard\n",
+ "from verifyml.model_card_toolkit.utils.tally_form import tally_form_to_mc\n",
+ "from verifyml.model_tests.utils import plot_to_str\n",
+ "from verifyml.model_tests.FEAT import (\n",
+ " SubgroupDisparity,\n",
+ " MinMaxMetricThreshold,\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Read output with truth and prediction columns"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Read output\n",
+ "# truth column -> total no. of words spoken by the individual\n",
+ "# prediction column -> total no. of words correctly transcribed by google's speech to text model model\n",
+ "\n",
+ "output = pd.read_csv(\"../data/speech_sample_output.csv\")\n",
+ "output.rename(columns={'truth_count':'truth', 'match_count':'prediction'},inplace=True)\n",
+ "mse_test = round(mean_squared_error(output.truth, output.prediction),3)\n",
+ "mae_test = round(mean_absolute_error(output.truth, output.prediction), 3)\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
sex
\n",
+ "
race
\n",
+ "
truth
\n",
+ "
prediction
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
77
\n",
+ "
60
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
84
\n",
+ "
64
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
80
\n",
+ "
53
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
F
\n",
+ "
INDIAN
\n",
+ "
80
\n",
+ "
53
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
70
\n",
+ "
45
\n",
+ "
\n",
+ "
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
...
\n",
+ "
\n",
+ "
\n",
+ "
293
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
74
\n",
+ "
50
\n",
+ "
\n",
+ "
\n",
+ "
294
\n",
+ "
M
\n",
+ "
MALAY
\n",
+ "
68
\n",
+ "
52
\n",
+ "
\n",
+ "
\n",
+ "
295
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
82
\n",
+ "
58
\n",
+ "
\n",
+ "
\n",
+ "
296
\n",
+ "
F
\n",
+ "
CHINESE
\n",
+ "
79
\n",
+ "
52
\n",
+ "
\n",
+ "
\n",
+ "
297
\n",
+ "
M
\n",
+ "
CHINESE
\n",
+ "
82
\n",
+ "
57
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
298 rows × 4 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " sex race truth prediction\n",
+ "0 F CHINESE 77 60\n",
+ "1 F CHINESE 84 64\n",
+ "2 M CHINESE 80 53\n",
+ "3 F INDIAN 80 53\n",
+ "4 M CHINESE 70 45\n",
+ ".. .. ... ... ...\n",
+ "293 M CHINESE 74 50\n",
+ "294 M MALAY 68 52\n",
+ "295 F CHINESE 82 58\n",
+ "296 F CHINESE 79 52\n",
+ "297 M CHINESE 82 57\n",
+ "\n",
+ "[298 rows x 4 columns]"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "output"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Word count plot"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4VUlEQVR4nO29ebgcVbX3//nmEJKTBMjAgUtCZmNyEa4M5woYrhKiRpyIeFERfMEB9DrgGE00jyNKXvE6XLmKXLiCIpFBCPjCD0QGQZRAwhQEIhIyEJCEkDAlQEjW74+qPumc1K6u6lQPp3t9nqef7tpVtffau7prde2111oyMxzHcRwHoF+jBXAcx3GaB1cKjuM4Tg+uFBzHcZweXCk4juM4PbhScBzHcXpwpeA4juP04EqhDEnvlrRK0vOSDqpTmzdL+mg92qo1ksZJMkm7NFqWeiPpdElPSfpHA2Vo+fGX9P9JOin+fLKkPxVUb0uMXdyHV+1MHU2nFCQtl/SkpMFlZR+VdHMdmv8+8CkzG2Jmd9ehPacgyn7UV/cqv1DSN2rc9mjgC8B+ZvZPtWyrFkj6hqQLGy1HFszsaDO7oNFyNAO1+kPZdEohZhfgMw1odyzw1wa02+do4n9Uh0maWuc2xwLrzGxNvRqs5/groqH3imaQYWdJumbN+Dtq1kE+E/iipKFJOyW9XtKdkp6J31+fpVJJ/STNlbRC0hpJv5S0h6QBkp4HOoB7JT2ScK4k/TA+7xlJ90naP953vqSzJV0v6TlJf5Q0Nq+8kvaJ6/1ivH2YpD9L2iDpXklHlh17s6RvS7otbvP3kvYM1PtHSe+JPx8R/6N+W7z9Jkn3pI1PvK/0T/wjklYCN0rqkPT9eNpkGfD2Xu2eLGlZLN+jkk5IkG2kpE2ShpeVHRTX2V/Sq2L5n4nLLk7qYxnfA04P7ZR0iqS/S3pa0lWSRlaor3TeHvF4rI3HZ248Xm8CrgdGKpp2PD/h3GYe/7cCXwHeF8t/b1x+s6TvSLoN2AhMkPQhSQ/G9S2T9LGyeo6U9JikL8SyPyHpQ2X73ybpgfjc1aXveLzvGEn3SHpW0iOxTCEZev87lqSfxN+PhyRN73XNzotlWa1oiq8j3pc6dgnjNFrS5fH1XyfprCqv2cmKfrM/lPQ08A1F95/vS1qpaJbkbEmdaeMj6TvAvwFnxdftrF7y/mtc1y5lZe8pfddSMbOmegHLgTcBlwOnx2UfBW6OPw8H1gMfJHqiOD7eHpGh7g8DfwcmAEPiNn5Vtt+AVwXOnQEsBoYCAv4Z2Cfedz7wHPAGYADwY+BPWeQFbo77Nw74G3BqXD4KWAe8jUh5vzne7io77xHg1UBnvD0vIPu3gJ/En78Sn/d/y/b9uNL4xPIZ8EtgcNzmx4GHgNFxP2+Kj9klPuZZYHJ8/j7AawLy3QicUrZ9JnB2/Hk+8NV4DAYCRwTqKMk3BFgNvCkuvxD4Rvz5KOAp4OD4Ov0EuCXj9/KXwJXAbmXX6iPxviOBx1LObfbx/wZwYa+ym4GVwGvi+voT3TgnEn3/30h0oz64bAxeifvTn+h7uxEYFu9/Avi3+POwsvNeBzxD9P3uR/S9n5Iiw83AR+P9J8dtfi7e9764ruHx/gXAz+Ox2Au4A/hYvC84dgnj0wHcC/wwrqvne1jFNSvJ/Om4T53Aj4CrYjl2A34HnJFxfD7aS9aeexjwAHB02b4rgC9U/K7X6uZe7YttSmH/eDC62F4pfBC4o9c5fwFOzlD3DcAnyrYnA5tLXwTSlcJRRDeCw4B+vfadD/ymbHsIsCX+wqXKG1/YH8T9Pr7smC9TprDisuuAk8rOm1u27xPAtQHZpwP3xZ+vjcfz9nj7j8Cxlcan7As+oWz/jcDHy7bfwvY3pQ3Ae4DOCtflo8CN8WcBq4A3xNu/BM4B9q1Qx7iytj9R1r9ypXAe8L1e12kzMK5C3R3AS0Q2g1LZx8q+k0eSrhSaffy/QbJS+FaF8xYAnykbg02U3VSBNcBh8eeV8Zjt3quOnwM/DNS/gwzsqBQeB1S2/w6i39ze8TXrLNt3PHBTpbFLkONwYG1gX95rdjKwsmxbwAvAxF7tPZpxfNKUwpeBX8efhxMp6X3SrqmZNe30EWZ2P/D/gNm9do0EVvQqW0GkQSvR+9wVRBdv7wzy3AicBfw38KSkcyTtXnbIqrJjnweejtvLIu8JRP9uLysrGwscp2jqaIOkDcARRP/4SpSvdNlIdJNL4i/AqyXtDRxIdKMdrWi66XXALfFxWcZnVdnnkb22e841sxeI/rl9HHhC0tWSpgTkuww4XNFUzhuIvti3xvu+RPTDuUPSXyV9OFBHOf8D7C3pnb3Kt+tffJ3WUfm7syewKzuOTZbvHDT/+IcorxtJR0u6XdHU2waip4HyKct1ZvZK2Xb5d/I98fErFE2nHR6XjyZ6csokQwKrLb7rxawgGpexRE8PT5T9fn5O9MQAKWOXwGhgRa++lch7zXpvdwGDgMVlcl4bl5faThufNC4E3ilpCPBe4FYze6LSSU2rFGK+DpzC9j++x4kueDljiG6qleh97hiiR7knswhjZv9lZocQPc6+GphVtnt06UN8EYbH7WWR9xtE0xoXleY8ib44vzKzoWWvwWY2L4usveTeSDT19RngfjN7Gfgz8HngETN7Kj40y/iU/wCfoKzf8fHl7V5nZm8mUmQPEd2sk+TbAPye6Iv7AWB+6YduZv8ws1PMbCTRP82fqsKSOzPbDHwT+DaRQimxXf8UrXAbQeXvzlNE//56j02W71zTj3+vOhPLJQ0Afku0Qm9vMxsKXMP24xvEzO40s2OIbsoLgEviXauIpqSCp1aoepSkchnGEI3jKqInhT3Lfj+7m9lr4uNSx64Xq4AxSjYK571mvbefInrCek2ZnHuYWUmZpo1P6tiY2WqiPyTvJnp6+lXa8SWaWimY2d+Bi4HTyoqvIfrX9QFJu0h6H7Af0VNFJeYDn5M0Pr5xfxe4OPAPYDtiw82hkvoTPe69SDRFVOJtioyIuxLdjBaa2aqM8m4GjiN65P+VolUWJS0/IzaKDVRkzNs3Qz+T+CPwqfgdokfP8m3IPz6XAKdJ2lfSMMqe6iTtLeld8Y33JeB5th+v3lwE/B+if5QXldVzXFmf1xP9ENLqKfErIrvBW3u18SFJB8Y3ue8SXaflaRWZ2Za4r9+RtJuiRQSfJ7pGWWnm8X8SGKf01T27Eo3nWuAVSUcTTblURNKukk6QtEessJ8tk+U8omsyPTbajsr5RLMX0Rj0l3Qcka3vmvgf8e+B/5S0e1z3RElvjM8Ljl0CdxApkXmSBse/xdIKt6rvKQBmtpVIWf9Q0l4A8RjMiA9JG58niWwZafyS6Gn7ACKbQiahmupFbFMo2x5NdAO+uazsCKJ/Xs/E70eU7fsrcEKg7n7A14i071qiH/WwpPm4hHOnA/cR/bieAn4NDIn3nQ+cTbQK5Xmi6YDxGeW9mW1zpAOBP8T19QMOJbppPB3LezUwpvd5tm2u8k8p4zoj7t8b4+394+33ZRkfyubsy47fhcj4tg54FPhk6Riif6d/jPu8IZZ3vxT5OomM9X/tVf49on/kzxM9Rp8aOD9JvvfGZd8oK/t4XM/TRIp537h8TNzGmED9w+LxWBuPz9eIbUtUsCk0+/gTPS39iUjp3pX0/YrLPkl0I9pApHR/w7bFIDuMAdvsg7sSTYmsJ1IId7L9b+DdRL+t54iMtjNSZOgpI/rO30Y0rfsMkc3vLWXH7gH8DHgs3n838P5KYxcYozFETzjriH7//1XlNTuZXr9Tot/9d4Fl8fg8CJyWYXwOj/u8vkye7e5hRFNTzwIXVLr3ll6KT3R2AkXLEB8zs7mNlsVxHKccRUvsP2Zmf8hyfFNPHzmO4zjVo8g/xohWW2Wi6bzpHMdxnJ1HUWig/YAPWmS7yHZeLaePJH2OaE22AUuADxHNcV1MNN+2HHivma2vmRCO4zhOZmo2fSRpFNGqoW4z25/IAej9RFb+G8xsEpHjR5rV33Ecx6kjtZ4+2gXolLSZ6AnhcWAO0UoFgAuIVhN8Oa2SPffc08aNG1czIR3HcVqRxYsXP2VmXZWP3EbNlIKZrZb0fSL39k3A783s95L2ttirzsyeKK3N7Y2kU4FTAcaMGcOiRYtqJarjOE5LIinNUzuRWk4fDQOOAcYTuYIPlnRi1vPN7Bwz6zaz7q6uXIrOcRzHqZJaLkl9E1FQp7UWeTFeDryeKG7QPhCFiiYKmuU4juM0AbVUCiuJEp4MimOTTCfy1LsKOCk+5iSicMSO4zhOE1BLm8JCSZcBdxEFiLqbKATyEOASSR8hUhzH1UoGx3EcJx81XX1kZl8ninRazktETw2O4zhOk+EezY7jNJQFd6/mzOuW8viGTYwc2smsGZOZeVDWVBVO0bhScBynYSy4ezVzLl/Cps1RJO3VGzYx5/IlAK4YGoQHxHMcp2Gced3SHoVQYtPmLZx53dIGSeS4UnAcp2E8vmFTrnKn9rhScBynYYwc2pmr3Kk9rhQcx2kYs2ZMprN/x3Zlnf07mDVjcoMkctzQ7DhOwygZk331UfPgSsFxnIYy86BRrgSaCJ8+chzHcXpwpeA4juP04ErBcRzH6cGVguM4jtODKwXHcRynB1cKjuM4Tg+uFBzHcZweXCk4juM4PbhScBzHcXpwpeA4juP04ErBcRzH6cGVguM4jtODKwXHcRynB1cKjuM4Tg+uFBzHcZweKuZTkPQp4Ndmtr4O8jiO0wdYcPdqT4zTomR5Uvgn4E5Jl0h6qyTVWijHcZqXBXevZs7lS1i9YRMGrN6wiTmXL2HB3asbLZpTABWVgpnNBSYB5wEnAw9L+q6kiTWWzXGcJuTM65ayafOW7co2bd7CmdctbZBETpFksimYmQH/iF+vAMOAyyR9r4ayOY7ThDy+YVOucqdvkcWmcBpwEvAUcC4wy8w2S+oHPAx8qbYiOo7Tm0bO6Y8c2snqBAUwcmhnXdp3akuWJ4U9gWPNbIaZXWpmmwHMbCvwjppK5zjODjR6Tn/WjMl09u/YrqyzfwezZkyuS/tObQkqBUnDJQ0HfgQ8V9ouK8fMHqyTnI7jxDR6Tn/mQaM449gDGDW0EwGjhnZyxrEH+OqjFiFt+mgxYEDSaiMDJtREIsdxUmmGOf2ZB41yJdCiBJWCmY2vpyCO42TD5/SdWlLRpiDphixljuPUB5/Td2pJ8ElB0kBgMLCnpGFsm0baHRhZB9kcx0mgNG3jHsVOLUizKXwM+CyRAljMNqXwLPDftRXLcZw0fE7fqRVpNoUfAz+W9Gkz+0kdZXIcx3EaRBY/ha2ShpY2JA2T9IlKJ0maLOmestezkj4bL2m9XtLD8fuwnemA47QjC+5ezdR5NzJ+9tVMnXejxx1yCiOLUjjFzDaUNuJoqadUOsnMlprZgWZ2IHAIsBG4ApgN3GBmk4Ab4m3HcTLSaOc1p7XJohT6lUdGldQB7JqznenAI2a2AjgGuCAuvwCYmbMux2lrGu285rQ2WZTCdcAlkqZLOgqYD1ybs533x+cB7G1mTwDE73slnSDpVEmLJC1au3ZtzuYcp3VpBuc1p3XJohS+DNwI/AfwSaIpn8xB8CTtCrwLuDSPYGZ2jpl1m1l3V1dXnlMdp6UJOam585pTBBWjpJrZVknnAzeaWTXPp0cDd5nZk/H2k5L2MbMnJO0DrKmiTsdpC5Kioc6aMZlZl97L5q3Wc1z/fnLnNacQsng0vwu4h3jKSNKBkq7K0cbxbJs6AriKKBQ38fuVOepynLYhZFBetOLpHSOSeT5EpyCyTB99HXgdsAHAzO4BxmWpXNIg4M3A5WXF84A3S3o43jcvs7SO00aEDMrzF65i8xbbrnzzFnNDs1MIFaePgFfM7JlqUjOb2UZgRK+ydUSrkRzHSSFkON5illjuhmanCLIohfslfQDokDQJOA34c23Fcpz2IZRFLRQNtUNKVAzVGpobmcXNaT6yTB99GngN8BJwEfAMUUwkx3F2kjRHtFA01OMPHV1YlFR3hHN6k/qkEDuqXWVmbwK+Wh+RHKd9SHNEu232UT3H9P4X3z12eCH/7tPa96eF9iRVKZjZFkkbJe1hZs/USyjHaRcqOaKFoqEWFSXVHeGc3mSxKbwILJF0PfBCqdDMTquZVI7TJjQ6i1qj23eajyxK4er45ThOBvIYbmfNmMycy5dsN4VTbh+otRF41ozJzLrs3u2WuPbvKN4Rrl2M2a3Qzyw2hQ/GNgXHcSpQMtyWbvIlwy0QnAaCZLtB3rqqpvdCpuQVr1VTt340mFbpZ+rqIzPbAmyUtEed5HGcPk01EUxnHjSK22YfxaPz3s5ts4/aTlHUOhrqmdct3S5cBsDmrcU6wrVLVNdW6afbFBynQIo03NbDCNwqbTQDrdJPtyk4Tgp554jTDLdF1lUPeYuiXYzZrdLPis5rZnYBUUC7xfHrorjMcVqaahy7pk1JDvM+bkRn7rpCzmshI3A18uZtoxrq0UYz0Cr9zBIl9UjgYeC/gZ8Cf5P0htqK5TiNp5o54pseSk4Idfuy9VXZGs449gBGDe1EwKihnZxx7AHBf/7V2jPytFEN9WijGWiVfmaZPvpP4C2lXAqSXk305HBILQVznEZTzRxx0UHs8jipVTunXZQjXKPbaAZaoZ9ZYh/1L0+uY2Z/A/rXTiTHaQ6qyXAW2tcRiDJc9Nx9rdtwWp8sSmGRpPMkHRm//ofItuA4LcOCu1czdd6NjJ99NVPn3ZgakC5tjrgeQezS5O3fb3vl4xnZIpLGy0kmi1L4D+CvRCGzPwM8AHy8lkI5Tj0JGWiB3HPEoXnl02ceUNh8s2dky4dHgs2HLDDX2XOANBh4MXZkK3k5D4gT6NSF7u5uW7RoUb2ac9qMqfNuTFxKOGpoZ0+k0mYiJG8oz0Kz9qNe9LXrWySSFptZd55zsjwp3ACUT0p2An/I04jjNDN9zenIM7Llo69d30aTZfXRQDN7vrRhZs/HuZcdpyWoxuGsmsBnRZ1Tr4xsRdLIQHGt4lRWL7I8Kbwg6eDShqRDAFexTsuQ1+Fs7oIlueeoq5nXDp0zbUpXXYzZRdHoOf1WcSqrF1mUwmeBSyXdKulW4GLgUzWVynHqSF6Hs/kLV+V2EqvGsSx0zk0Pra25MbtIGh0orlWcyupFxekjM7tT0hRgMtFahofMbHPNJXOcOpF3jr6aufsiHeEe37Cp5hnZiqQZ5vSbcVyalSxPCpjZZjO738yWuEJwWo28DmfVOKIV6QjX1+bCW6Uf7UImpeA4zUhRDkkhp6/QHH2lufugY1lHrzYqZDhLO6eavjfKgavSnL47ljUXWVYfOU7TUXiWqwSnr+6xw+keOzxx1UyoPCTXew4ZVV2Gs4RzFq14mt8uXp2r743MCtYU2eWczASd18pXHCVhZnfVRKIE3HnN6U2RDkn1qKsax7J61NVoB65mlatVqMZ5Le1J4T/j94FAN3Av0f+pfwEWAkdUI6TjFEGzZjgr0rGsHnU12oGrWeVqZ4JKwcymAUj6DXCqmS2Jt/cHvlgf8ZxWoygnpiIdkupRV5pjWWhMinRSK9qBqx7XsZEOb+1MFkPzlJJCADCz+4EDayaR07IU6cRUpENSPeqa0JUcBGDQrv2CYzJuRPINe0LXoMKit1bTx3pcx2lTujyIXYPIohQelHRuHDb7jXHo7AdrLZjTehTpxFSkQ1I96lq2Njl+5MNrXgiOye3L1iees2ztxsKit1bTx3pcx5seWttQh7d2JkuU1IFE4bNLKThvAX5mZi/WWLYe3NDcGoyffXXighsBj857e73FqSvjZl+d63iRvjhpeQPHqx7XsZ2/K0VStKEZADN7UdLZwDXlGdgcJy/tHJgsZAcIMXJoJ/945sXEc0LOc/WiHtexnb8rjaaiUpD0LuBMYFdgvKQDgW+Z2btqLJvTYsyaMZlZl97L5q3bbnT1zAxWZGTTuQuWMH/hKraY0aHI0e30mQcAJO47/tDRXHj7yh3qnzpxOHc8uj5xTBateDrxnOMPHV2oETZvJNhZMyZv51sAxQeYq0cbzU6jDO1ZnNe+DrwOuBnAzO6RNK6GMjmtTIMyg1XjJBU659JFK7ntkad7jttitt3Nu/xzad+Jh43hxMPG7KAsuscO547lvWwH8ZiUlEzSOUU5fIX6mMVBrpY3rHq00cw00qkvi01hoZkdKuluMzsoLrvPzP6lppKV4TaF1qCRjkrVtB06J0RpWic05fPIGW+rqVyNdrZziqOoa1wTmwJwv6QPAB2SJhHlav5znkYcBxrrqFRklNIQaTaDvA5n9XJE8yxuzUkjfytZlMKnga8CLwHzgeuAb2epXNJQ4Fxgf6LFFB8GlhLlZBgHLAfea2bJa++clqLoDGehef2k8moMl6FzQlR6UqhGrjyZ1+rlbOfUnkYa2iv6KZjZRjP7KjAdmGZmX82xHPXHwLVmNgV4LZF/w2zgBjObRJT/eXZ1ojt9jbyOSmkZzuYuWMKFt6/suXGV5u7f/IObE8sH7Zr8VQ9lXUuTd+/ddk08fkLXII4/dHRwX5Jcr2zZknj8uBGduTOvVWOEDfV/zyH9g3I5taeR2eIqKgVJ/yppCXAfsETSvXFKzkrn7U7k23AegJm9bGYbgGOAC+LDLgBmVie609fI66iUluFs/sJViW08vOaFXOWhrGtp8j71fHJKkWVrN3L6zAM48bAxPU8NHRInHjYm6Lz25HMvJ5bfvmx97sxr1RggQ/1Pk8upPY3MFpfF0Hwf8EkzuzXePgL4aSVDc7x09RzgAaKnhMXAZ4DVZja07Lj1ZjYs4fxTgVMBxowZc8iKFSuy98rpU4QclUJUcuzKQzXOUGmOaCGnsrzOaxDuZz2cxNJopOOck49qDM1Zwlw8V1IIAGb2J+C5DOftAhxM5P18EPACOaaKzOwcM+s2s+6urvAjvtP3yZv5bOTQzsIcuKqZo82bka3SvtDx9chYlreuRjvOObUnqBQkHRznVLhD0s/LYh/9lNhnoQKPAY+Z2cJ4+zIiJfGkpH3iNvYB1uxUD5w+RTArWSDzWe8vaD+i+dbQ3P2kvQYnlk+dODxYV0iuUHmo7VB52r6QvMcfOjp1Xjkk79wFS5g45xrGzb6aiXOuYe6CJUnV9xBqY+rE4bn64bQOWfIplPh62eeKT5xm9g9JqyRNjsNjTCeaSnoAOAmYF79fmU9kp6+SmpUswant0bXPs7VX8VaizGOPrn0+2E6/+LjybXqVldcF5HLgOuPYZKeykrNZEt1jh3PR7St3kOvQCSNY9tRGtpR5NHf0E91jhwcduELypjnVhWRLcxJL89p2WpeKNoWdqjyyK5xLFCJjGfAhot/CJcAYYCVwnJk9HaoD3HmtVcjrKBUi7/GV6vqnPQbW3IGrHlnUQoQc55zWp1DnNUknmtmFkj6ftN/MflCpcjO7hyhrW2+mZ5bQaRnyOkqFKEohlOqqhwNXPbKohShyvJzWJ236qDTZuVs9BHFan7yOUiEa/aRQyTibx0mtyCxqIdrJOOzZ2naeoKHZzH4uqQN41sy+2ftVRxmdFiFk1Dxswg4rkgHYfUBHYvmErkFBQ+jAjnw3wMG79gs6ZIUcuNIc3kJOdSEntWqyqIXaTzNatwNFZoRrZ1KXpJrZFsBDZDuFEHLIWb4u+V/vsy8l30iXrd3Ir085fAfFMHXicF7cku8J4tmXtgQdskIOXGkObyGnulBd1WRRC7W/8eWtiY5z7WIcLjIjXDuTJfbRnyWdRRSvqMct1MzuqplUTssy86BRO9zwPnfxPbnqKP0L//Uph++wrxonsbxTUWlz+tXYR5LGpJr2H9+widNnHtA2SqA3jQwi10pkUQqvj9+/VVZmgMfPdQqh0XPk1WRFK6quavriWcmS8XEphizpOKfVQxCnsRSZlSzvObNmTObzl9xD2VJ9+gkOnzB8u3X3JUpz5El1TZ2YfM7ADiVOLU3aazCHThiROytaqO+hDGuT9hqcGH+pmvn+ojPYVROhthnxbG3FkCUd53eB78XB7JA0DPiCmc2tsWxOnSgyK1k15xw8Zo/tFALAVoPxXUMY3zUk0YEqVNe+wwYmtr3HoP68mDCvf+iEEXSPHc78O1bt4EA2vmtIYla0tKxkadnSkhRf99hkg3lFCspgtzOZ15qNds/WVhRZAuL1ZFwrK7vLzA6uqWRluPNabalX9q8ina7y1pXWRt4lqUU6nBXpCNdopzqn+ahVQLwOSQPKGukEBqQc7/Qx6pX9q0inq6KMh9U4rxXpcFakI1yjneqc1iCLoflC4AZJv2Bb9rQL0k9xqqFRc7hFZiUbObQzGDOnGoNyUXWltVHUk0K12dLyXvdmyLzW1+wNTnayZF77HnA68M/Aa4Bvx2VOgTTS8SbkvJWWZSvkQDVo136JzltzFyzJnclscEpdofZDzmu7BObcD5swLJiVbXCgPC0rWd5saaGsc2nXPdT3NKe6EKFrcvyho1MjtLqTWOuSJfPaYOD3ZvZFoqQ5AyQl/yqcqmmk403IeSsty1bIgSqU4Wz+wlW5M5mFnNfmL1wVbD/kvPZKYCZq+bpNQZlD7ReZLS2UdS7tuof6nuZUFyJ0TU6feUDQqc6dxFqbLNNHtwD/Fq86+gOwCHgfcEItBWs3Gul4k3fuHKq3DyQ5an22Cue1osalyPFNk+vxDZtyOe7Vyz4BydckrdydxFqbLIZmmdlG4FjgJ2b2bmC/2orVftQjy1aIajKJFZmxq8isZHkpcnyryZZWzXVv5HelGdp3aksmpSDpcKIng1IMgSxPGE4O0rJs1ZpKmcSC2dJ6zd/371BVGbuqzUqWlK0t1H6ovOTwFmo/b1ay0LiEsqVVc90b+V1phvYbTSjrXauQRSl8BpgDXGFmf5U0AbiptmK1H6G53Xqs6Ogem5yqsnvs8KBRcdGKp3fMv2dwXPeY3EHZuscOp6PXDb6jnzh0wgh6FW/v8JXgwDW+a0jiOYt7O6HFfG3BkmBwves/f2TiNRnfNSTYFyBxXBateDpxHIHc172R35VmaL+RtIORvaaZ14rCnddqS5ozFNCUWcnyypXG8nlvz3X8xDnXBJeq1iOLm9M4inQcrAeFZl4rq/TVwBeBceXHm1nzjYBTFdUYDps1K1k9soylGebdGay1aQcjexbbwKXA2US5lpPX6Dl9hryOVZDvH3klY2Oe9iu1UdSTQl5505zais7i5k5izUU7RGLNYlN4xcx+ZmZ3mNni0qvmkjmFk9exqhTBNK9zU972Q05yIeexQbv2Czqc7Tmkf6JcIee1tExtIXlDmeKOP3R00IHssAnDChuvVpq/7msU6TjYrGRRCr+T9AlJ+0gaXnrVXDKncPI6VpXWqed1bsrbfshJLuQ89vCaF4IOZ08+93KiXHvvkax4RuyWHFU1Td7l6zYFjekhB7Ll6zYVNl7uJNY4inQcbFayTB+dFL/PKiszYELx4ji1JK9jVYm8zk152y/aDlAPJ7FQhrNqxzhv+05jaIdrkiX20fiElyuEPkijnY5C7RSdSS1P20U7iRU5xo2+Xs6OtMM1yeSEJml/Ii/mnmdtM/tlrYRyakPRmanyZuwKZQx73+tGc9HClTskoJnYlZytrORTkJRhberE4YmRVStlKwudExqvvH2sZDsoqi6ntrRDdrcsS1K/DhxJpBSuAY4G/gS4UuhjFJmZqpqMXUCiw9nCZesSM6/ttdsA9tptwHY3/6kTh/PrUw4H4IT/+csO+8Z3DdkuHWYpsuqja58PZiubu2BJ4jkQOZb1Hi8gmHUu1McQaRns8tbl1J52yO6WJfPaEuC1wN1m9lpJewPnmtk76yEguPNaM1Kkw1kaRTmWhRg1tJN/PPNicIlpUua3apz9Qs5NRdblOL2pifMasMnMtkp6RdLuwBrcyNz2NKuTVl6j9ePxcs88ddUjU12RGewcJw9ZlMIiSUOB/wEWA88Dd9RSKGd76uHAlNZGrR3O0ghlXgvty+u8NrLCk0JRzn5pBsqi6nKcIkhVCpIEnGFmG4CzJV0L7G5m99VDOCd9zrkoxVBpXjtp33sOGbWd7QAig9u+wwYmGodLDmpJN7mBHUpMjrP7gI7gXD+QuG/3AR2J/g2h8mlTunh07fOJRusJXYMS+37wmD0S+1GqK2lfyEFv2pSu7fpRXt49dnjLGzWd5iOLTWGxmR1SJ3kSaWebQj0CcFU7rz1rxuQd/kV/4ZJ7g/+6oRifhCLrqia4XprdpEj7xG2zj/IwF85OUSubwu2S/tXM7qxSLmcnqIezTLXz2nmyqBXpoFZkXdUE10uzNRRtn8jr8OY4O0uWMBfTgL9IekTSfZKWSPLpozpRD2eZtDbytp+Wxa0oJ7Ui60rrY96MdCOHdlZ1Tp5yx6k1WZTC0cBE4CjgncA74nenYIrKzJWXtDbS9iXJm5bFLbRv6sTkJDtpGc7yZmubOnF4Yqa2Uh+TsqUdf+joxORDacEA0/rfqOvrOHnIohRON7MV5S/g9FoL1m6EImJC/sxceUnLpBXaBwTlTc2WlsCa515iSy/vtS1bjTXPvRQ8p3vs8MR2QtnaxncNSXcES8iW9uja59naq7i0HRqv02cekBgsr2Q0bsT1dZw8ZDE032VmB5dtdwBLzGy/WgtXoh0MzX0to1M1zmshI2xeqslbUKRTXchonEZfu75Oa1CooVnSHOArQKekZ0vFwMvAOVVL6STS16IvVuO8VpR5uJoMZ0U61VWj2Pra9XXal6BSMLMzgDMknWFmc6qpXNJy4DmijG2vmFl3nIvhYqL0nsuB95pZckD9NqJeGZ2KWuJYjfNaMz4pVONUV8nIneRU1xczdvly2PYkS+jsqhRCGdPM7MCyR5jZwA1mNgm4Id5ue+phcCwyk1co81koW9q0KV1M6BqUuG/3AR25yid0DQo6g03oGpQ4jqFsadOmdAXHPs3QHaIUXK+kgEpOdeNGdPYpg7JnfWtfshiai+YY4IL48wXAzAbI0HSkGXuLoshMXqHMZ6FsaTc9tJZlazcm7nvh5a2JxtkXXu5t5o1YtnZjMFvbsrUbE8dx+brkJ4GbHlobHPtfn3J4MMNaiPkLVyWW375sfZ8yKHvWt/aloqF5pyqXHgXWE63t+LmZnSNpg5kNLTtmvZnt8DdO0qnAqQBjxow5ZMWKFTWTs10YP/vqxHl9AY/mjEY6bvbVuY4XOy7wKScpGmreNtLqKrLvaaTJnDfiayOp13g5taUaQ3PFJwVJEyUNiD8fKem0OEBeFqbGK5eOBj4p6Q1ZBTOzc8ys28y6u7paJyl2I2mko1Q1jl3VOMI12kksr1zNijvVtS9Zpo9+C2yR9CrgPGA8cFGWys3s8fh9DXAF8DrgSUn7AMTva6qQu+lJclRqdPuzZkwOOnDNXbCEiXOuYdzsq5k45xrmLtgWEC9pX2i+fdJeg4NtpDl25S3Pe069nMTyygWN/64k4U517UsWpbDVzF4B3g38yMw+B+xT6SRJgyXtVvoMvAW4H7gKOCk+7CTgymoEb2YabaQLtb9oxdOJDlyXLlqZaBydu2BJ0HA6vmvIDoph6sThfHLapKCTWMixKzRHv3DZumB599jhid7GISe5ethsiNtP8s4OydXo70qIeo2X03xkcV5bCPwI+CrwTjN7VNL9ZrZ/hfMmED0dQLT09SIz+46kEcAlwBhgJXCcme0Yt7iMvua81mhHpbyOZSHSopFWG/UzD2nz86MCSzwb7QyWt/+N/q44rU2toqR+CPg48J1YIYwHLqx0kpktI0rj2bt8HTA9j5B9jUY7KuV17AqRdnyRWcmqodFjHKKoDGuN7ofTvlRUCmb2AHBa2fajwLxaCtXXabSjUl7HshCVnhTytF3qe8ghKqk8jbR2Gul0lSZXqzi1Oa1N0KZQCpEdetVTyL5Go41006Ykr9Y6bMKw3E5aRRp0Q/PncxcsSSzfe7ddE9uYtNfgYB/Hjehs6Bx9SK5Bu/ZLtM2EnABD9ThOrUkzNJdCZF8bv06IX9cAl9VetL5Lo410Nz20NrF8+bpNuZ208hqH0/oecoiav3BVYvkuHR07hMKetNdgrv/8kcE+3r5sfUOdrkJyhZz9QuWhehyn1mQxNN9mZlMrldWSvmZobjTN6ngUkitEmrxF1lUkeeUK0ehr5bQGNXFeAwZLOqKskdcDyZlMnKagWR2P8mY4S5O3yLqKpKh2Gn2tnPYli1L4MPDfkpbHYSt+Gpc5KTTSIalom0bevoSOD8l1/KGjczvVpdVVTd+Lul55g+tNnTjcncScpiJ19VGcUOeNZvZaSbsTTTc9Ux/R+i4lg2ppbrs8y1Y97AqlNopYgZO3L1mOT1pldPGdvQLJxU51tz2yzYWlZJwFemwaSX3sHjs8V9+LvF5pY5+0+uj0mQd4iGqnqchiU7jZzI6sjzjJ9DWbQis5JNXDGSt0TohqMp+l0UrXy3HKqZXz2m2SziJKjNOzVMLM7sopX9vQSg5J9XDGyjsuRSTqydJ+X7xejrOzZFEKr4/fv1VWZoD/hQrQ17KopZHXSayavofOCVF0xFF3IHOcbWTJvDYt4eUKIYW+lkUtjVBfpk3pSmx/2pSu3H0POWr19lEokRZxtBpC7bsDmdOOZMmnsIekH0haFL/+U9Ie9RCur9LXsqilEerLTQ+tTWz/pofW5u57yFFrYyAjW1rms2oIte8OZE47kmX66H+JQl6/N97+IPAL4NhaCdUKzDxoVE1XkNRzHjypL5+7+J5g+3n7ntaXkld1LXGbguNsI4tSmGhm7ynb/qake2okj5ORSvPgoeWP9Wo/RF47RJrdpCibitsUHGcbWZzXNvXyaJ4K+F+oBpNmtwglxinPplbL9kOE7CDjRiTffNOC2xVpU2l0AEPHaSay+CkcCFwA7EEUkuVp4CQzq1uk1L7mp1AvQv+UJ865JldinKLbD5E3+U+ofFT8D75I3wJ3IHNakZr4KZjZPUDJoxkze7Y68ZyiCc3dh9bxF72+vyjbQV55i/R5KFFrG5Dj9BWyrD56RNKvgQ8A+9ZeJGdnCa3jL3p9f17yBrFLC27XrEH/HKevk8WmsB/wc2AE8H1JyyRdUeEcp4GkJcZpxkB9oSB2acHt3A7gOLUhy+qjLcDm+H0r8CSwppZCOTtHaZVR79VH3WOHN22gvlAQu0rB7dwO4DjFksXQvBFYAvwA+IOZrauHYOW4obkYPPCb47QXtUqyczxwC/AJ4DeSvilpejUCOo3FnbQcx6lEltVHVwJXSpoCHA18FvgS4Ba9PkYzO2mFloTWa6moL0l1nIiKSkHSb4EDgb8DtwL/B1hYW7GcWjBrxuTtbArQHMbZUJKbRSue5reLV9fcBtLopEiO00xkMTTPA+4ysy0Vj3SamiIzshVJKLhfyVDeu/zM65bWLbhgo8fGcepNlumjO+shiFMfmtFJK69TW9E2ELe1OM42shiaHaem5HVqK9oG4o5wjrONoFKIA98haUD9xHFanSTnubxObdXaQEKOe+4I5zjbSHtS+K/4/S/1EMRpfUKRTYHExDynzzygsGRFaVFV65EUyXH6CkHnNUm3Aw8CbwMu7r3fzE6rrWjbcOe11qCRznPuuOe0I0VHSX0H8CbgKGDxzgjmONBYg64bkx0nG0GlYGZPEXkwP2hm99ZRJqdFaaTzXDM77jlOM5Fl9dE6SVdIWiPpSUm/leQhtJ3cNNKg68Zkx8lGFqXwC+AqYCQwCvhdXOY4uWikQdeNyY6TjSxRUu81s9f2KrvHzA6spWDluKHZcRwnP7WKkrpW0omSOuLXiUDdw2c7juM4tSdL7KMPA2cBPwQM+HNclglJHcAiYLWZvUPScKIlruOA5cB7zWx9PrGbH4+66ThOX6Tik4KZrTSzd5lZl5ntZWYzzWxFjjY+Q+TvUGI2cIOZTQJuiLdbijRHKcdxnGamprGP4lVKbwfOLSs+Brgg/nwBMLOWMjSCtKibjuM4zUytA+L9iCghz9aysr3N7AmA+H2vpBMlnSppkaRFa9eurbGYxeKOUo7j9FVqphQkvQNYY2ZVeUOb2Tlm1m1m3V1dXQVLV1s86qbjOH2VikpB0tyyz3kipk4F3iVpOfAb4ChJFwJPStonrm8fYE0uifsA7ijlOE5fJS109pckHQ78e1lx5oipZjbHzPY1s3HA+4EbzexEIke4k+LDTgKuzC11k+OOUo7j9FXSlqQuBY4DJki6lWgF0QhJk81sZyym84BLJH0EWBm30XI0Y4Yzx3GcSqQphfXAV4Aj49c/AzOA2bFieH3WRszsZuDm+PM6YHpV0jqO4zg1JU0pvBX4OjAR+AFwL/CCmX2oHoI5juM49SdoUzCzr5jZdCKv4wuJFEiXpD9J+l2d5HMcx3HqSJYwF9eZ2Z3AnZL+w8yOkLRnrQVzHMdx6k+WMBdfKts8OS57qlYCOY7jOI0jl/OaZ2BzHMdpbWod5sJxHMfpQ7hScBzHcXpwpeA4juP04ErBcRzH6cGVguM4jtODKwXHcRynB1cKjuM4Tg+uFBzHcZweXCk4juM4PbhScBzHcXpwpeA4juP04ErBcRzH6cGVguM4jtNDlnwKfZIFd6/mzOuW8viGTYwc2smsGZM9Z7LjOE4FWlIpLLh7NXMuX8KmzVsAWL1hE3MuXwLgisFxHCeFlpw+OvO6pT0KocSmzVs487qlDZLIcRynb9CSSuHxDZtylTuO4zgRLakURg7tzFXuOI7jRLSkUpg1YzKd/Tu2K+vs38GsGZMbJJHjOE7foCUNzSVjsq8+chzHyUdLKgWIFIMrAcdxnHy05PSR4ziOUx2uFBzHcZweXCk4juM4PbhScBzHcXpwpeA4juP0IDNrtAwVkbQWWFHl6XsCTxUoTl+jnfvvfW9f2rn/5X0fa2ZdeU7uE0phZ5C0yMy6Gy1Ho2jn/nvf27Pv0N7939m++/SR4ziO04MrBcdxHKeHdlAK5zRagAbTzv33vrcv7dz/nep7y9sUHMdxnOy0w5OC4ziOkxFXCo7jOE4PLacUJA2VdJmkhyQ9KOlwScMlXS/p4fh9WKPlLBpJkyXdU/Z6VtJn26HvAJI+J+mvku6XNF/SwHbpO4Ckz8R9/6ukz8ZlLdl/Sf8raY2k+8vKgn2VNEfS3yUtlTSjMVIXR6D/x8XXfquk7l7H5+p/yykF4MfAtWY2BXgt8CAwG7jBzCYBN8TbLYWZLTWzA83sQOAQYCNwBW3Qd0mjgNOAbjPbH+gA3k8b9B1A0v7AKcDriL7z75A0idbt//nAW3uVJfZV0n5E34XXxOf8VFIHfZvz2bH/9wPHAreUF1bT/5ZSCpJ2B94AnAdgZi+b2QbgGOCC+LALgJmNkK+OTAceMbMVtE/fdwE6Je0CDAIep336/s/A7Wa20cxeAf4IvJsW7b+Z3QI83as41NdjgN+Y2Utm9ijwdyLl2WdJ6r+ZPWhmSxMOz93/llIKwARgLfALSXdLOlfSYGBvM3sCIH7fq5FC1oH3A/Pjzy3fdzNbDXwfWAk8ATxjZr+nDfoecz/wBkkjJA0C3gaMpn36D+G+jgJWlR33WFzWLuTuf6sphV2Ag4GfmdlBwAu0ziNzJiTtCrwLuLTRstSLeP74GGA8MBIYLOnExkpVP8zsQeD/AtcD1wL3Aq80VKjmQQll7bQOP3f/W00pPAY8ZmYL4+3LiJTEk5L2AYjf1zRIvnpwNHCXmT0Zb7dD398EPGpma81sM3A58Hrao+8AmNl5Znawmb2BaGrhYdqo/4T7+hjRU1OJfYmmFtuF3P1vKaVgZv8AVkmaHBdNBx4ArgJOistOAq5sgHj14ni2TR1Be/R9JXCYpEGSRHTdH6Q9+g6ApL3i9zFEBsf5tFH/Cff1KuD9kgZIGg9MAu5ogHyNIn//zaylXsCBwCLgPmABMAwYQbQi4eH4fXij5axR3wcB64A9ysrape/fBB4iml//FTCgXfoe9/9Woj9A9wLTW/naEym8J4DNRP+EP5LWV+CrwCPAUuDoRstfo/6/O/78EvAkcF21/fcwF47jOE4PLTV95DiO4+wcrhQcx3GcHlwpOI7jOD24UnAcx3F6cKXgOI7j9OBKwWk6JJ0h6UhJMyXl8kiX1CVpYRzm5N9qINvJks4qut5aIOl8Sf/eaDmcvoUrBacZORRYCLyRaP19HqYDD5nZQWaW99w+SwtE/nSaBFcKTtMg6UxJ9wH/CvwF+CjwM0lfSzh2rKQbJN0Xv4+RdCDwPeBtcU6Jzl7nzJP0QHzO9+Oy8yWdLelWSX+T9I64fKCkX0haEj91TEuQ4e2S/iJpT0lviT/fJelSSUPiY5ZL+mZcvkTSlIR6rpH0L/Hnu0v9lfRtSR9VxJmK8iUskfS+eP+Rkm6SdBGwJD7urLiPV1MWAC+p746TSKO98/zlr/IXUVjfnwD9gdtSjvsdcFL8+cPAgvjzycBZCccPJ/LoLDlsDo3fzycKItePKATAY8BA4AvAL+JjphCF0hhYqp/Ig/RWIo/5PYni2A+Oj/8y8LX483Lg0/HnTwDnJsg2G/gksDtwJ7E3KnATMBl4D1Gwuw5g71iWfYAjiYI+jo+PP7bsuJHABuDfQ333l7+SXv6k4DQbBwH3EN2IH0g57nDgovjzr4AjKtT7LPAicK6kY4mSEJW4xMy2mtnDwLK47SPiejGzh4AVwKvj46cR3fjfbmbrgcOA/YDbJN1DFHtnbFn9l8fvi4FxCbLdSpQH5AjgamBIHAJ7nEUx8o8A5pvZFosCHf6R6GkK4A6L4uQT11E67nHgxgx9d5zt2KXRAjgOQDz1cz5RFMeniOI4Kb7JHm5mmypUkRqvxcxekfQ6IpvD+4FPAUcFzjWSQw6XWEaUu+PVRHG2BFxvZscHjn8pft9C8m/uTqA7rvd6oiePU4iUCBVkeSFB9u0L0vvuONvhTwpOU2Bm91iUSvRvRP+6bwRmWJRiNEkh/JnoBgdwAvCntPrjOf49zOwa4LNEgRNLHCepn6SJRDf7pUTTQSfE574aGBOXQ/TUcCzwS0mvAW4Hpkp6VXz8oPicrH1/mSgRynvjum4Fvsg2I/stwPskdUjqInoiSIp0eQtRRMyOOHz0tAx9d5zt8CcFp2mIb3jrzWyrpClmljZ9dBrwv5JmEWXb+1CF6ncDrpQ0kOif9+fK9i0lmpLZG/i4mb0o6afA2ZKWECWsOdnMXooic0c5sSWdQJTM6J1Etob5kgbEdc4lUnBZuZUouulGSbcSPTGVlMIVRNNl9xI9CXzJzP6RYLS+gugJYEnc9h8z9N1xtsOjpDptjaTzgf9nZpc1WhbHaQZ8+shxHMfpwZ8UHMdxnB78ScFxHMfpwZWC4ziO04MrBcdxHKcHVwqO4zhOD64UHMdxnB7+fw3bq5A6Kp1AAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
\n",
+ "\n",
+ " Model Card for Speech to Text Model\n",
+ "\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "\n",
+ "
Model Details
\n",
+ "\n",
+ "
Overview
\n",
+ "\n",
+ " Using an automated transcription tool, the government want to transcribe audio files recorded in trials for evidential purposes. The tool should be able to capture the various spoken accents and transcribe into text accurately so as not to bias against the defendant. The tool chosen here will be Google's Speech to Text Model evaluated on a set of audio files by National Speech Corpus.\n",
+ "\n",
+ "
Benefits: Reduce manpower and human error on transcription services. A more accurate model than current system will aid the judicial system in making better court decisions
\n",
+ "\n",
+ "
Harms: Incorrectly transcribed/interpreted texts can impact on the defendant/ruling negatively.
\n",
+ "\n",
+ "
Mitigation Strategy: Improve upon the existing model by further training and enhancements
\n",
+ "\n",
+ " Description: \n",
+ "\n",
+ " Test if the mae of the subgroups within sex \n",
+ "is lower than the threshold of 27.\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " Description: \n",
+ "\n",
+ " Test if the mae of the subgroups within race \n",
+ "is lower than the threshold of 27.\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " Description: \n",
+ "\n",
+ " Disparity ratio of MSE of any 2 bins in the respective attribute should not be more than a factor of 1.1 \n",
+ "\n",
+ "
\n",
+ "\n",
+ " Description: \n",
+ "\n",
+ " Test if the maximum ratio of the mean squared error of any 2\n",
+ "groups within race attribute exceeds 1.1. To\n",
+ "pass, this value cannot exceed the threshold. \n",
+ "\n",
+ "
\n",
+ "\n",
+ " Description: \n",
+ "\n",
+ " Test if the maximum ratio of the mean squared error of any 2\n",
+ "groups within sex attribute exceeds 1.1. To\n",
+ "pass, this value cannot exceed the threshold. \n",
+ "\n",
+ "