Is the bot learning? #66
Description
So having monitored the activity of the labelbot (manually) as much as I could
I have tried to go through all the issues in the month of Jan 2019, and found out that in the following issues - LabelBot has recommended a set of labels which are either
a. Incorrect
b. Incomplete
Metric used here - Edit distance
Basically number of delete, append needed to go from predict to groud-truth
Assumption - Ground-truth is correct. ( i.e. Labels given by community are correct)
| Bot prediction | Human (groundtruth) | Edit Distance | Date | |
|---|---|---|---|---|
| apache/mxnet#14010 | Build | docker, build | 1 | 29 Jan |
| apache/mxnet#14019 | Feature | question, operator | 3 | 29 Jan |
| apache/mxnet#14009 | performance | performance, cuda, operator | 2 | 28 Jan |
| apache/mxnet#14008 | installation | R, installation | 1 | 28 Jan |
| apache/mxnet#14004 | installation | build, R | 3 | 28 Jan |
| apache/mxnet#13999 | Installation, C++, Build | build, Windows, cuda | 2 | 27 Jan |
| apache/mxnet#13991 | Installation, Build | build, question | 2 | 25 jan |
| apache/mxnet#13987 | doc | build, question | 3 | 25 jan |
| apache/mxnet#13976 | Performance | gluon | 2 | 23 jan |
| apache/mxnet#13968 | empty | Gluon, operator | 2 | 22 jan |
| apache/mxnet#13967 | empty | gluon | 1 | 22 jan |
| apache/mxnet#13958 | empty | Build, ci, cmake, windows | 4 | 22 jan |
| apache/mxnet#13956 | bug | Bug, c api | 2 | 22 Jan |
| apache/mxnet#13954 | bug | Feature-request, Julia | 3 | 21 jan |
| apache/mxnet#13953 | empty | Bug, good first issue, python | 4 | 20 jan |
| apache/mxnet#13949 | onnx | Onnx, bug | 1 | 20 jan |
| apache/mxnet#13919 | empty | Question, gluon | 2 | 17 jan |
| apache/mxnet#13879 | empty | Gluon, good first issue, call for contribution | 4 | 14 jan |
| apache/mxnet#13870 | C++, bug | C++,build | 2 | 14 jan |
| apache/mxnet#13854 | empty | Website, docs | 2 | 11 jan |
| apache/mxnet#13846 | bug | Backend bug osx test | 3 | 11 jan |
| apache/mxnet#13801 | installation | Docs, installation | 1 | 8 jan |
| apache/mxnet#13800 | test | CI | 2 | 8 jan |
| apache/mxnet#13798 | C++, test | C++, bug | 2 | 7 jan |
| apache/mxnet#13792 | bug | Bug,data-loadinng, gluon, python | 3 | 7 jan |
| apache/mxnet#13790 | build | Build, edge devices | 1 | 7 jan |
| apache/mxnet#13789 | feature | Kvstore, julia | 2 | 7 jan |
| apache/mxnet#13788 | feature | Optimizer, julia | 2 | 7 jan |
| apache/mxnet#13787 | feature | Example, julia | 2 | 7 jan |
| apache/mxnet#13785 | Build, installation | Build, cuda | 2 | 7 jan |
| apache/mxnet#13781 | feature | Arm, question | 2 | 6 jan |
| apache/mxnet#13722 | feature | ndarray | 2 | 4 jan |
| apache/mxnet#13765 | Build, installation | Build, installation, scala, clojure | 2 | 3 jan |
| apache/mxnet#13753 | empty | Optimizer, fp16, feature request, cuda | 4 | 1 jan |
| apache/mxnet#13750 | Performance, bug | Feature request | 3 | 1 jan |
I feel there is a lot of common trend here and hence was wondering if the errors are getting repeated, does this mean the bot isn't learning as expected?