Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10000 commits
Select commit Hold shift + click to select a range
fc4cabc
skeleton PT
hieuhoang Sep 24, 2013
42f9f81
Fixed some code formatting issues.
Sep 24, 2013
ae550d0
Fixed some code formatting issues.
Sep 24, 2013
515ee54
Fixed code formatting.
Sep 24, 2013
c227cde
Fixed code formatting.
Sep 24, 2013
d100792
Added Mmsapt as PT implementation type.
Sep 24, 2013
30573ec
Added missing #include.
Sep 24, 2013
d0a5921
Fixed some code formatting.
Sep 24, 2013
61dacaa
Moved duplicate struct CompareTargetPhrase to TargetPhraseCollection.…
Sep 24, 2013
794770e
Moved duplicate struct CompareTargetPhrase to TargetPhraseCollection.…
Sep 24, 2013
31ccd6c
Code reformatting. Once upon a time there was a rule of courtesy not …
Sep 24, 2013
d8686f7
Reformatting.
Sep 24, 2013
7b9e30e
Reformatting.
Sep 24, 2013
8545b4d
Added Mmsapt as a feature function.
Sep 24, 2013
e6fdcda
Reformatting. Max 80 chars per line, pretty please.
Sep 24, 2013
933e4e0
Call to EmptyHypothesisState(const InputType &input) now throws an ex…
Sep 24, 2013
bed0cb5
Fixed signed/unsigned integer comparison issue.
Sep 24, 2013
05ea314
Added module mmbitext.m4m and mmsapt as phrase table type.
Sep 24, 2013
c80c5a1
Added new contributions to headers
Sep 24, 2013
62ad7ca
Added emacs file type declaration
Sep 24, 2013
13c9a44
Phrase::Contains(const Phrase& sought) throws exception, since it's n…
Sep 24, 2013
7ce803e
Formatting.
Sep 24, 2013
db849ce
Commented out unused variables.
Sep 24, 2013
e3ba4a9
Initial check-in.
Sep 24, 2013
48c424a
Added *.d
Sep 24, 2013
ad8d0fc
cleanup
Sep 24, 2013
bfcefec
cleanup
Sep 24, 2013
96df1f2
lattices/confusion work with Hieu's binary pt, using new method
hieuhoang Sep 25, 2013
82274ff
Merge branch 'dynamic-phrase-tables'
ugermann Sep 25, 2013
deafd59
skeleton PT
hieuhoang Sep 25, 2013
1758381
skeleton PT
hieuhoang Sep 25, 2013
56c1224
put some meat onto skeleton FFs
hieuhoang Sep 25, 2013
847f7c1
skeleton PT
hieuhoang Sep 25, 2013
429a195
add type to ControlRecombination. type=2 disable recombination
hieuhoang Sep 25, 2013
e1574c2
Added dependency on boost::iostreams.
ugermann Sep 25, 2013
b736fe2
Commented out unused variables.
ugermann Sep 25, 2013
82a2639
Formatting. Fixed order in which members are initialized.
ugermann Sep 25, 2013
d02b265
Added dependency on boost::iostreams.
ugermann Sep 25, 2013
5b18448
Fixed errors introduced during a merge.
ugermann Sep 25, 2013
94d1073
Formatting. Commented out unused variables.
ugermann Sep 25, 2013
e188203
Formatting.
ugermann Sep 25, 2013
daaf71c
Changed type of m_batch_size to uint32_t to avoid compiler signed/uns…
ugermann Sep 25, 2013
4fffe80
Formatting.
ugermann Sep 25, 2013
050b089
Merge branch 'master' of github.com:moses-smt/mosesdecoder
ugermann Sep 25, 2013
d511744
Changed int to size_t to avoid compiler signed/unsigned comparison wa…
ugermann Sep 25, 2013
1e702c4
updated web interface for experiment.perl
phikoehn Sep 25, 2013
1e6a909
Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder
phikoehn Sep 25, 2013
01c0708
input feature score error when using anything other than bin pt
hieuhoang Sep 25, 2013
4285a6c
move Uli's new pt into it's own folder. Comment out bjam compile. Doe…
hieuhoang Sep 26, 2013
0eff734
move Uli's new pt into it's own folder. Comment out bjam compile. Doe…
hieuhoang Sep 26, 2013
086c658
only run beautify with specific astyle version
Sep 26, 2013
52c0e21
only run beautify with specific astyle version
Sep 26, 2013
3e5a883
only run beautify with specific astyle version
Sep 26, 2013
650f515
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Sep 26, 2013
dbaabb1
move Uli's new pt into it's own folder. Comment out bjam compile. Doe…
hieuhoang Sep 26, 2013
940591a
extract-ghkm: allow trailing whitespace in alignment file
pjwilliams Sep 26, 2013
13e1adf
Astetic Changes
Sep 26, 2013
31ce9b5
beautify
Sep 27, 2013
ba1f1cb
skelete pt
hieuhoang Sep 27, 2013
06c5621
skelete pt
hieuhoang Sep 27, 2013
9d9c43f
skelete pt
hieuhoang Sep 28, 2013
9e89047
get ready to do lattice for scfg
hieuhoang Sep 28, 2013
da95b1b
train-model.perl: add --ghkm-tree-fragments option
pjwilliams Sep 29, 2013
e497dc4
Remove NT length code missed in commit cdd9df19...
pjwilliams Sep 29, 2013
20b96fd
Oops, fix e497dc485...
pjwilliams Sep 29, 2013
2a28d1a
Merge branch 'master' into GHKMStruct
pjwilliams Sep 29, 2013
4565d63
Update InternalStructStatelessFF after merge.
pjwilliams Sep 29, 2013
841bf4b
Fix merge error.
pjwilliams Sep 29, 2013
d6aa123
score: write sparse features to third field.
pjwilliams Sep 29, 2013
6bce688
eclipse
Sep 30, 2013
5472311
defer BackwardTest
Sep 30, 2013
72981ab
defer BackwardTest
Oct 1, 2013
233062a
add DIMwid /Robin Kurtz
Oct 2, 2013
ea23629
Moses server: optionally include alignment info in n-best lists
ales-t Oct 2, 2013
bb511af
minor rename of variables
Oct 2, 2013
1b12b0c
correct creation of input paths for lattices
Oct 2, 2013
6e32bd3
correct creation of input paths for lattices
Oct 2, 2013
f2d218f
move checking of legacy phrase-table to StaticData
Oct 2, 2013
6ccc17b
speed up compiling by removing headers from StaticData.h
Oct 3, 2013
52215fd
speed up compiling by removing headers from StaticData.h
Oct 3, 2013
6759af3
beautify
Oct 3, 2013
8ccc99a
correct creation of input paths for lattices
Oct 3, 2013
73513c1
correct creation of input paths for lattices
Oct 3, 2013
a41dbeb
correct creation of input paths for lattices
Oct 3, 2013
f18e35c
correct creation of input paths for lattices
Oct 3, 2013
28e4dcb
compile error in moses server
hieuhoang Oct 3, 2013
a0b6c38
correct creation of input paths for lattices
Oct 4, 2013
01a99d5
check that there's no epsilon
Oct 4, 2013
4a166d0
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Oct 4, 2013
f4150c2
faster compilation by taking some #includes from Word.h and Phrase.h
Oct 4, 2013
f657865
update bjam source code. Copied from boost 1.54
Oct 4, 2013
7d3de78
minor error with placeholder
Oct 4, 2013
9027416
reverse 7d3de7850025c1b8da7791aa158bb4549f9ff96d
hieuhoang Oct 4, 2013
7d43165
don't create input paths longer than max phrase length
hieuhoang Oct 4, 2013
125e9a8
add debug argument
hieuhoang Oct 5, 2013
90fe351
remove unused code
ehasler Oct 5, 2013
67c294a
Add option to protect expressions - eg URLs
bhaddow Oct 7, 2013
8b9d4d1
limit size of input path for confusion networks and lattices. Prevent…
Oct 7, 2013
f9abcb5
Update nonbreaking_prefix.el
dmavroeidis Oct 7, 2013
f1e500a
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 8, 2013
a7d462c
Merge pull request #47 from dmavroeidis/master
hieuhoang Oct 8, 2013
08a88a6
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 8, 2013
69f5e4b
Merge branch 'master' into GHKMStruct
pjwilliams Oct 9, 2013
75e007d
minor fixes
phikoehn Oct 10, 2013
bedb3ba
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 10, 2013
9cbfa50
add actual config file and run command used to train and decode this …
Oct 10, 2013
e8951c9
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Oct 10, 2013
29d2c01
removed spurious $input-extension
phikoehn Oct 11, 2013
a554918
Merge remote branch 'origin/GHKMStruct'
pjwilliams Oct 11, 2013
208116c
Fix call_realloc double free
kpu Oct 12, 2013
630f33f
specification changes for multiple operation sequence models
phikoehn Oct 13, 2013
793686e
Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder
phikoehn Oct 13, 2013
8e23a3f
more enriched trace
phikoehn Oct 13, 2013
30c8c29
ooops
phikoehn Oct 13, 2013
e6257dd
one more fix
phikoehn Oct 13, 2013
ecc30bc
also report lm scores, and matched n-gram length (thanks Ken!)
phikoehn Oct 13, 2013
3b99636
correct usage
bhaddow Oct 17, 2013
11bbedb
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 19, 2013
19cc934
minor bug fix
phikoehn Oct 24, 2013
31aa9f2
Do not output phrase penalty by default
bhaddow Oct 24, 2013
24decc1
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 25, 2013
0af1df9
add NeuralLM
Oct 28, 2013
37896ed
add NeuralLM
Oct 28, 2013
518a766
add NeuralLM
Oct 28, 2013
5eb95fc
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 29, 2013
d21e99e
remove description argument from feature function constructor. Redundant
Oct 29, 2013
4cc4913
remove description argument from feature function constructor. Redundant
Oct 29, 2013
ab8402c
remove description argument from feature function constructor. Redundant
Oct 29, 2013
bd82b73
remove description argument from feature function constructor. Redundant
Oct 29, 2013
a5f391e
remove description argument from feature function constructor. Redundant
Oct 29, 2013
36a8bb5
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Oct 30, 2013
b926346
compiles with eclipse under OSX Mavericks
hieuhoang Oct 30, 2013
24f9529
compiles with clang
hieuhoang Oct 31, 2013
40eefd0
don't compile compact pt if --with-cmph not specified
hieuhoang Oct 31, 2013
d51702f
output list of available feature functions
Nov 1, 2013
4b004b2
output list of available feature functions
Nov 1, 2013
cd00ca7
eclipse
hieuhoang Nov 1, 2013
9769516
Basic support for WADE analysis
bhaddow Nov 1, 2013
88569e2
rand lm compiles
Nov 4, 2013
a57aed6
added replace-unicode-punctuation.perl
Nov 4, 2013
69ef1d4
Merge branch 'master' of git://github.com/moses-smt/mosesdecoder
Nov 4, 2013
3531bcd
basic decoder
Nov 5, 2013
7e78dc5
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Nov 5, 2013
08d8883
add DALM. Just a skeleton so far
Nov 5, 2013
9ec6f9e
delete
Nov 6, 2013
f0070e5
delete caching parameters. Now phrase-table specific
Nov 6, 2013
fe01a02
Initialize variables...
kpu Nov 7, 2013
943f6a9
extract-target-trees.py: update to match current trace format
pjwilliams Nov 10, 2013
b8f67b5
convert class TargetNgramFeature to new feature framework
hieuhoang Nov 11, 2013
ada80e6
convert class TargetNgramFeature to new feature framework
hieuhoang Nov 11, 2013
6bf4fb6
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Nov 11, 2013
d5cde1d
DALM load
hieuhoang Nov 11, 2013
2d8f9bc
bug fixes for TargetNGramFeature
Nov 11, 2013
17f1bf7
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Nov 11, 2013
8c9c81a
compiled DALM with eclipse. bjam problems still
Nov 11, 2013
42e22d4
compiled DALM with bjam
Nov 11, 2013
d2868bd
1st stab @ evaluating LM with DALM
Nov 11, 2013
ca1c62c
eclipse
hieuhoang Nov 11, 2013
f68a92e
correcting the reuse-weights.perl for a new format
karelbilek Nov 11, 2013
e4f26e2
Merge pull request #50 from runn1ng/master
hieuhoang Nov 12, 2013
df3f3d1
reuse-weights.perl --> substitute-weights.perl
Nov 12, 2013
f85d26e
delete reuse-weights.perl
Nov 12, 2013
477314c
Merge branch 'master' of github.com:hieuhoang/mosesdecoder
hieuhoang Nov 12, 2013
2981558
Merge branch 'master' of ../mosesdecoder
hieuhoang Nov 12, 2013
6bee77e
extract-ghkm: use square brackets for glue rule internal tree structure
pjwilliams Nov 12, 2013
c94c998
internal tree structure in chart decoder: use square brackets for OOV…
Nov 13, 2013
dc7d152
eclipse
hieuhoang Nov 14, 2013
5dc13db
bjam with dalm
hieuhoang Nov 14, 2013
bbb6f0b
tested DALM
hieuhoang Nov 14, 2013
c226e35
typo
hieuhoang Nov 14, 2013
0b1bb6a
mert compiles under Mac OSX Mavericks. #ifdef PreProcessFilter.cpp an…
hieuhoang Nov 14, 2013
11c0ddd
minor compile errors on clang
hieuhoang Nov 14, 2013
58a7367
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Nov 14, 2013
cd8388c
oops. little errors in DALM
hieuhoang Nov 14, 2013
dfbad35
minor compile errors on gcc now
Nov 14, 2013
862e1ad
more gcc compile errors
hieuhoang Nov 14, 2013
3d37a8f
more compile errors, with clang
hieuhoang Nov 14, 2013
ce41276
Merge pull request #1 from moses-smt/master
jnory Nov 15, 2013
17887a2
replace nth_element() with macro that execute sort() instead for gcc …
Nov 15, 2013
94bf273
not ready for prime time
Nov 15, 2013
d6d0877
add comment for future work on making fdstream more portable /Jeroen …
Nov 15, 2013
ccf9662
Merge branch 'master' of ../mosesdecoder
hieuhoang Nov 15, 2013
86315c5
Merge branch 'master' of ../mosesdecoder
hieuhoang Nov 15, 2013
952b415
delete persistent cache arguments. Still have to disable new phrase-…
hieuhoang Nov 15, 2013
d66d6be
eclipse project for mira program
hieuhoang Nov 15, 2013
79fb8a1
provide mira program with BLEU Feature. Delete assert statement.
hieuhoang Nov 15, 2013
2f00ae9
move collection of phrase-table from Staticdata to PhraseDictionary
hieuhoang Nov 15, 2013
c9d09a7
correct hashing of contextFactor for state. /Ken
hieuhoang Nov 15, 2013
3f8972d
move collection of generation-table from Staticdata to GenerationDict…
hieuhoang Nov 15, 2013
682563d
UTIL_THROW_IF2 with base Exception class
Nov 15, 2013
deb2797
replace CHECK with UTIL_THROW_IF. It's gonna be a fun week
Nov 15, 2013
1caadce
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Nov 15, 2013
4cbc47f
Merge pull request #2 from moses-smt/master
jnory Nov 17, 2013
01f10fa
modify DALM.cpp for better recombination.
Nov 17, 2013
a1584c6
compile error in mosesserver
hieuhoang Nov 17, 2013
b3ba081
Merge pull request #52 from jnory/master
hieuhoang Nov 18, 2013
1accc75
Add vocab mapping back to DALM
Nov 18, 2013
c8f1922
replace CHECK with UTIL_THROW_IFin OnDiskPt
Nov 18, 2013
d9be815
replace CHECK with UTIL_THROW_IF in mert
Nov 18, 2013
3ffc5e7
replace CHECK with UTIL_THROW_IF in moses-cmd
Nov 18, 2013
d424e69
replace CHECK with UTIL_THROW_IF in moses-chart-cmd
Nov 18, 2013
90d1154
replace CHECK with UTIL_THROW_IF in moses
Nov 18, 2013
c2c86ce
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 19, 2013
22864c2
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 19, 2013
c2ffc02
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 19, 2013
f53825c
Fix the bug in phrase-extract/extract-main.cpp: the authors forgot to…
pengli09 Nov 20, 2013
4f6f127
Merge pull request #53 from pengli09/master
hieuhoang Nov 20, 2013
a77e928
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 20, 2013
c3bd825
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 20, 2013
aca7853
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 20, 2013
65a69b7
Merge branch 'master' of github.com:moses-smt/mosesdecoder
hieuhoang Nov 20, 2013
16cf3bb
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 20, 2013
0f3f675
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 20, 2013
749bd7f
add decoder arg -no-cache for mira tuning
hieuhoang Nov 21, 2013
d1265d6
call decoder with new -no-cache arg
hieuhoang Nov 21, 2013
8381f0c
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 21, 2013
3c0eaac
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 21, 2013
819c006
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 21, 2013
43dc333
replace CHECK with UTIL_THROW_IF in Moses
hieuhoang Nov 21, 2013
0a83909
delete includes for util/check.hh
hieuhoang Nov 21, 2013
193ee8a
minor compile error on Mac OSX
hieuhoang Nov 21, 2013
5c6d88a
add new phrase table for Scope-3 parsing. Doesn't quite work...
Nov 22, 2013
c4dfc60
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Nov 22, 2013
3163230
Scope-3 parsing working again
Nov 22, 2013
a57c9d6
delete -parsing-algorithm. Scope-3 parsing depends on the phrase-tabl…
Nov 22, 2013
028edf8
add -feature-name-overwrite. Specifically to replace PhraseDictionary…
Nov 22, 2013
f13d0c8
UTIL_THROW_IF -> UTIL_THROW_IF2
Nov 22, 2013
6fcbb33
eclipse
hieuhoang Nov 24, 2013
905ab6d
stop warning about incomplete data type
hieuhoang Nov 25, 2013
cedc815
return default even if compiling with non-gcc
hieuhoang Nov 25, 2013
86a0dc3
adding CoveredReferenceFeature
ales-t Nov 29, 2013
dab6a30
make reference-from-sgm.perl more robust
phikoehn Nov 30, 2013
6824d84
bug fix with interaction of alternate weights and xml markup
phikoehn Nov 30, 2013
c527f0d
Transliteration Scripts
Dec 2, 2013
ec4fe7a
Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
Dec 2, 2013
04cbbc4
add phrase-table wrapper for Nadir's transliteration code
hieuhoang Dec 2, 2013
dd78bf0
add phrase-table wrapper for Nadir's transliteration code
hieuhoang Dec 2, 2013
fec97d2
minor, added a check in CoveredReferenceFeature
ales-t Dec 3, 2013
cd63f30
modified FF interface: Evaluate() with input has access to future-cos…
ales-t Dec 3, 2013
1d56e4c
minor verbose message
Dec 3, 2013
effd98f
Merge branch 'master' of github.com:moses-smt/mosesdecoder
ales-t Dec 3, 2013
8c0b5ff
early evaluation of CoveredReferenceFeature
ales-t Dec 3, 2013
fea32c7
debugged CoveredReferenceFeature
ales-t Dec 3, 2013
81ad92c
minor
ales-t Dec 3, 2013
f97ef5a
ran update.sh
Dec 3, 2013
52217ae
merge with sourceforge codebase eventually?
Dec 6, 2013
ce8ce5b
ran update.sh
ksoona Oct 24, 2014
30e8f84
merge
ksoona Oct 24, 2014
15cacd1
Update README.md
thomhastings Nov 3, 2014
e62f7cf
Update README.md
thomhastings Nov 3, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
77 changes: 77 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
<<<<<<< HEAD
#################
## Eclipse
#################
Expand Down Expand Up @@ -164,3 +165,79 @@ pip-log.txt

# Tortoise SVN
*.svn
=======
*.d
*.pyc
*.lo
*.o
*.so
*.lo
*.o
*.la
*.a
*.swp
*.save
*.cmd
*~
*.gch
dist*
jam-files/bjam
jam-files/engine/bootstrap
jam-files/engine/bin.*
lm/build_binary
lm/query
mert/evaluator
mert/extractor
mert/mert
mert/megam_i686.opt
mert/pro
mert/kbmira
misc/processLexicalTable
misc/processPhraseTable
misc/queryLexicalTable
mira/mira
mira/Makefile
mira/Makefile.in
misc/queryPhraseTable
moses-chart-cmd/src/moses_chart
moses-cmd/src/checkplf
moses-cmd/src/lmbrgrid
moses-cmd/src/moses
regression-testing/moses-reg-test-data-*
regression-testing/tests/mert.extractor-bin/FEATSTAT*
regression-testing/tests/mert.extractor-bin/SCORESTAT*
scripts/ems/biconcor/biconcor
scripts/release-exclude
scripts/training/cmert-0.5/mert
scripts/training/compact-rule-table/tools/compactify
scripts/training/eppex/counter
scripts/training/eppex/eppex
scripts/training/lexical-reordering/score
scripts/training/memscore/memscore
scripts/training/mbr/mbr
scripts/training/phrase-extract/consolidate
scripts/training/phrase-extract/consolidate-direct
scripts/training/phrase-extract/consolidate-reverse
scripts/training/phrase-extract/extract
scripts/training/phrase-extract/extract-ghkm/tools/extract-ghkm
scripts/training/phrase-extract/extract-lex
scripts/training/phrase-extract/extract-rules
scripts/training/phrase-extract/relax-parse
scripts/training/phrase-extract/score
scripts/training/phrase-extract/statistics
scripts/training/symal/symal
scripts/training/train-model.perl
dist
bin
previous.sh
contrib/other-builds/*.xcodeproj/project.xcworkspace/
contrib/other-builds/*.xcodeproj/xcuserdata/
*/*.xcodeproj/project.xcworkspace
*/*.xcodeproj/xcuserdata

mert/sentence-bleu
._*
.DS_Store
*.pbxuser
*.mode1v3
>>>>>>> 81ad92cc3f4653bc6ff19b63faae19e52606faec
21 changes: 21 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<<<<<<< HEAD
[submodule "TesseractTrainer"]
path = TesseractTrainer
url = git://github.com/BaltoRouberol/TesseractTrainer.git
[submodule "mosesdecoder"]
path = mosesdecoder
url = git://github.com/moses-smt/mosesdecoder.git
[submodule "tesseract-trainer"]
path = tesseract-trainer
url = git://github.com/BaltoRouberol/TesseractTrainer.git
[submodule "moses-smt"]
path = moses-smt
url = git://github.com/moses-smt/mosesdecoder.git
=======
[submodule "contrib/arrow-pipelines/python/pcl"]
path = contrib/arrow-pipelines/python/pcl
url = https://github.com/ianj-als/pcl.git
[submodule "contrib/omtc/omtc"]
path = contrib/omtc/omtc
url = https://github.com/ianj-als/omtc.git
>>>>>>> 81ad92cc3f4653bc6ff19b63faae19e52606faec
143 changes: 143 additions & 0 deletions BUILD-INSTRUCTIONS.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
PRELIMINARIES

Moses is primarily targeted at gcc on UNIX.

Moses requires gcc, Boost >= 1.36, and zlib including the headers that some
distributions package separately (i.e. -dev or -devel packages). Source is
available at http://boost.org .

There are several optional dependencies:

GIZA++ from http://code.google.com/p/giza-pp/ is used to align words in the parallel corpus during training.

Moses server requires xmlrpc-c with abyss-server. Source is available from
http://xmlrpc-c.sourceforge.net/.

The scripts support building ARPA format language models with SRILM or IRSTLM.
To apply models inside the decoder, you can use SRILM, IRSTLM, or KenLM. The
ARPA format is exchangable so that e.g. you can build a model with SRILM and
run the decoder with IRSTLM or KenLM.

If you want to use SRILM, you will need to download its source and build it.
The SRILM can be downloaded from
http://www.speech.sri.com/projects/srilm/download.html .
On x86_64, the default machine type is broken. Edit sbin/machine-type, find
this code
else if (`uname -m` == x86_64) then
set MACHINE_TYPE = i686
and change it to
else if (`uname -m` == x86_64) then
set MACHINE_TYPE = i686-m64
You may have to chmod +w sbin/machine-type first.

If you want to use IRSTLM, you will need to download its source and build it.
The IRSTLM can be downloaded from either the SourceForge website
http://sourceforge.net/projects/irstlm
or the official IRSTLM website
http://hlt.fbk.eu/en/irstlm

KenLM is included with Moses.

--------------------------------------------------------------------------

ADVICE ON INSTALLING EXTERNAL LIBRARIES

Generally, for trouble installing external libraries, you should get support
directly from the library maker:

Boost: http://www.boost.org/doc/libs/release/more/getting_started/unix-variants.html
IRSTLM: https://list.fbk.eu/sympa/subscribe/user-irstlm
SRILM: http://www.speech.sri.com/projects/srilm/#srilm-user

However, here's some general advice on installing software (for bash users):

#Determine where you want to install packages
PREFIX=$HOME/usr
#If your system has lib64 directories, lib64 should be used AND NOT lib
if [ -d /lib64 ]; then
LIBDIR=$PREFIX/lib64
else
LIBDIR=$PREFIX/lib
fi
#If you're installing to a non-standard path, tell programs where to find things:
export PATH=$PREFIX/bin${PATH:+:$PATH}
export LD_LIBRARY_PATH=$LIBDIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
export LIBRARY_PATH=$LIBDIR${LIBRARY_PATH:+:$LIBRARY_PATH}
export CPATH=$PREFIX/include${CPATH:+:$CPATH}

Add all the above code to your .bashrc or .bash_login as appropriate. Then
you're ready to install packages in non-standard paths:

#For autotools packages e.g. xmlrpc-c and zlib
./configure --prefix=$PREFIX --libdir=$LIBDIR [other options here]

#tcmalloc is a malloc implementation with threaded performance. To see how it
#improves Moses performance, read
# http://www.mail-archive.com/moses-support@mit.edu/msg07303.html
#It is part of gperftools which can be downloaded from from
# https://code.google.com/p/gperftools/downloads/list
#configure with this:
./configure --prefix=$PREFIX --libdir=$LIBDIR --enable-shared --enable-static --enable-minimal

#For bzip2:
wget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
tar xzvf bzip2-1.0.6.tar.gz
cd bzip2-1.0.6/
#Compile and install libbz2.a (static library)
make
make install PREFIX=$PREFIX
mkdir -p $LIBDIR
#Note this may be the same file; you can ignore the error
mv $PREFIX/lib/libbz2.a $LIBDIR 2>/dev/null
#Compile and install libbz2.so (dynamic library)
make clean
make -f Makefile-libbz2_so
cp libbz2.so.* $LIBDIR
ln -sf libbz2.so.1.0 $LIBDIR/libbz2.so

#For Boost:
./bootstrap.sh
./b2 --prefix=$PREFIX --libdir=$LIBDIR --layout=tagged link=static,shared threading=multi,single install || echo FAILURE

--------------------------------------------------------------------------

BUILDING

Building consists of running
./bjam [options]

Common options are:
--with-srilm=/path/to/srilm to compile the decoder with SRILM support
--with-irstlm=/path/to/irstlm to compile the decoder with IRSTLM support
-jN where N is the number of CPUs

--with-macports=/path/to/macports use MacPorts on Mac OS X.

If you leave out /path/to/macports bjam will use the /opt/local as default.
You don't have to use --with-boost with-macports as it is implicitly set.
Also note that using --with-macports automatically triggers "using darwin".

Binaries will appear in dist/bin.

You can clean up data from previous builds using
./bjam --clean

For further documentation, run
./bjam --help

--------------------------------------------------------------------------

ALTERNATIVE WAYS TO BUILD ON UNIX AND OTHER PLATFORMS

Microsoft Windows
-----------------
Moses is primarily targeted at gcc on UNIX. Windows users should
install using Cygwin. Outdated instructions can be found here:
http://ssli.ee.washington.edu/people/amittai/Moses-on-Win7.pdf .

Binaries for all external libraries needed can be downloaded from
http://www.statmt.org/moses/?n=Moses.LibrariesUsed

Only the decoder is developed and tested under Windows. There are
difficulties using the training scripts under Windows, even with
Cygwin, but it can be done.
1 change: 1 addition & 0 deletions HOWTO
Loading