Merge pull request #715 from PyThaiNLP/dev

Update 3.1 from dev

Author: wannaphong

.github/workflows/windows-test.yml

@@ -0,0 +1,59 @@
+name: Windows Unit test and code coverage
+
+on:
+  push:
+    paths-ignore:
+      - '**.md'
+      - 'docs/**'
+  pull_request:
+    branches:
+      - dev
+    paths-ignore:
+      - '**.md'
+      - 'docs/**'
+
+jobs:
+  build:
+
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [windows-latest]
+        python-version: [3.8]
+
+    steps:
+    - uses: actions/checkout@v2
+    - uses: conda-incubator/setup-miniconda@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+        auto-activate-base: true
+        auto-update-conda: true
+    - shell: powershell
+      run: |
+        conda info
+        conda list
+    - name: Install PyTorch
+      shell: powershell
+      run: |
+        pip install torch==1.8.1
+    - name: Install dependencies
+      shell: powershell
+      run: |
+        python -m pip install --disable-pip-version-check --user --upgrade pip setuptools
+        python -m pip --version
+        python -m pip install pytest coverage coveralls
+        conda install -y -c conda-forge fairseq
+        python -m pip install https://www.dropbox.com/s/o6p2sj5z50iim1e/PyICU-2.3.1-cp38-cp38-win_amd64.whl?dl=1
+        python -m pip install -r docker_requirements.txt
+        python -m pip install .[full]
+        python -m nltk.downloader omw-1.4
+        python -m pip install spacy deepcut
+    - name: Test
+      shell: powershell
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        COVERALLS_SERVICE_NAME: github
+      run: |
+        coverage run -m unittest discover
+        coveralls

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.7-slim-buster
+FROM python:3.8-slim-buster
 
 COPY . .
 

README.md

@@ -20,12 +20,12 @@ PyThaiNLP เป็นไลบารีภาษาไพทอนสำหร
 
 **News**
 
-> Now, You can contact or ask any questions you encounter with the PyThaiNLP team. <a href="https://matrix.to/#/#thainlp:matrix.org" rel="noopener" target="_blank"><img src="https://matrix.to/img/matrix-badge.svg" alt="Chat on Matrix"></a>
+> Now, You can contact or ask any questions with the PyThaiNLP team. <a href="https://matrix.to/#/#thainlp:matrix.org" rel="noopener" target="_blank"><img src="https://matrix.to/img/matrix-badge.svg" alt="Chat on Matrix"></a>
 
 | Version | Description | Status |
 |:------:|:--:|:------:|
-| [3.0](https://github.com/PyThaiNLP/pythainlp/releases) | Stable | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/545) |
-| [`dev`](https://github.com/PyThaiNLP/pythainlp/tree/dev) | Release Candidate for 3.1  | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/643) |
+| [3.1](https://github.com/PyThaiNLP/pythainlp/releases) | Stable | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/643) |
+| [`dev`](https://github.com/PyThaiNLP/pythainlp/tree/dev) | Release Candidate for 3.2  | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/714) |
 
 
 ## Getting Started

README_TH.md

@@ -16,12 +16,12 @@ PyThaiNLP เป็นไลบารีภาษาไพทอนสำหร
 
 **ข่าวสาร**
 
->นับตั้งแต่ PyThaiNLP 3.0 พวกเราจะยุติการสนับสนุน Python 3.6 แล้ว หากคุณจำเป็นต้องใช้ PyThaiNLP บน Python 3.6 คุณสามารถใช้ PyThaiNLP 2.3.1 ได้
+> คุณสามารถพูดคุยหรือแชทกับทีม PyThaiNLP หรือผู้สนับสนุนคนอื่น ๆ ได้ที่ <a href="https://matrix.to/#/#thainlp:matrix.org" rel="noopener" target="_blank"><img src="https://matrix.to/img/matrix-badge.svg" alt="Chat on Matrix"></a>
 
 | รุ่น | คำอธิบาย | สถานะ |
 |:------:|:--:|:------:|
-| [3.0](https://github.com/PyThaiNLP/pythainlp/releases) | Stable | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/545 |
-| [`dev`](https://github.com/PyThaiNLP/pythainlp/tree/dev) | Release Candidate for 3.1  | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/643) |
+| [3.1](https://github.com/PyThaiNLP/pythainlp/releases) | Stable | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/643) |
+| [`dev`](https://github.com/PyThaiNLP/pythainlp/tree/dev) | Release Candidate for 3.2  | [Change Log](https://github.com/PyThaiNLP/pythainlp/issues/714) |
 
 ติดตามพวกเราบน [PyThaiNLP Facebook page](https://www.facebook.com/pythainlp/) เพื่อรับข่าวสารเพิ่มเติม
 

SECURITY.md

@@ -4,6 +4,7 @@
 
 | Version | Supported          |
 | ------- | ------------------ |
+| 3.1.x   | :white_check_mark: |
 | 3.0.x   | :white_check_mark: |
 | 2.3.x   | :white_check_mark: |
 | 2.2.x   | :x:                |

docker_requirements.txt

@@ -12,7 +12,7 @@ sentencepiece==0.1.91
 ssg==0.0.8
 torch==1.8.1
 fastai==1.0.61
-transformers==4.8.2
+transformers==4.22.1
 phunspell==0.1.6
 spylls==0.1.5
 symspellpy==6.7.6
@@ -31,3 +31,6 @@ thai-nner==0.3
 spacy==2.3.*
 wunsen==0.0.3
 khanaa==0.0.6
+spacy_thai==0.7.1
+esupar==1.3.8
+ufal.chu-liu-edmonds==1.0.2

docs/api/parse.rst

@@ -0,0 +1,10 @@
+.. currentmodule:: pythainlp.parse
+
+pythainlp.parse
+===============
+The :class:`pythainlp.parse` is dependency parsing for Thai.
+
+Modules
+-------
+
+.. autofunction:: dependency_parsing

docs/api/soundex.rst

@@ -11,6 +11,7 @@ Modules
 .. autofunction:: lk82
 .. autofunction:: udom83
 .. autofunction:: metasound
+.. autofunction:: prayut_and_somchaip
 
 References
 ----------
@@ -23,3 +24,5 @@ References
         Master Thesis. Chulalongkorn University, Thailand.
 
 .. [#lk82] วิชิต หล่อจีระชุณห์กุล และ เจริญ คุวินทร์พันธุ์. `โปรแกรมการสืบค้นคำไทยตามเสียงอ่าน (Thai Soundex) <http://guru.sanook.com/1520/>`_.
+
+.. [#prayut_and_somchaip] Prayut Suwanvisat, Somchai Prasitjutrakul. Thai-English Cross-Language Transliterated Word Retrieval using Soundex Technique. In 1998 [cited 2022 Sep 8]. Available from: https://www.cp.eng.chula.ac.th/~somchai/spj/papers/ThaiText/ncsec98-clir.pdf

docs/api/tokenize.rst

@@ -12,6 +12,7 @@ Modules
 .. autofunction:: sent_tokenize
 .. autofunction:: subword_tokenize
 .. autofunction:: word_tokenize
+.. autofunction:: word_detokenize
 .. autoclass:: Tokenizer
    :members:
 

docs/notes/installation.rst

@@ -31,7 +31,11 @@ where ``extras`` can be
   - ``tltk`` (to support tltk)
   - ``textaugment`` (to support text augmentation)
   - ``oskut`` (to support OSKUT)
-  - ``nlpo3`` (to support nlpo3 enging)
+  - ``nlpo3`` (to support nlpo3 engine)
+  - ``spacy_thai`` (to support spacy_thai engine)
+  - ``esupar`` (to support esupar engine)
+  - ``transformers_ud`` (to support transformers_ud engine)
+  - ``dependency_parsing`` (to support dependency parsing with all engine)
   - ``full`` (install everything)
 
 For dependency details, look at `extras` variable in `setup.py <https://github.com/PyThaiNLP/pythainlp/blob/dev/setup.py>`_.