💫 Port master changes over to develop (#2979)
* Create aryaprabhudesai.md (#2681)
* Update _install.jade (#2688)
Typo fix: "models" -> "model"
* Add FAC to spacy.explain (resolves #2706)
* Remove docstrings for deprecated arguments (see #2703)
* When calling getoption() in conftest.py, pass a default option (#2709)
* When calling getoption() in conftest.py, pass a default option
This is necessary to allow testing an installed spacy by running:
pytest --pyargs spacy
* Add contributor agreement
* update bengali token rules for hyphen and digits (#2731)
* Less norm computations in token similarity (#2730)
* Less norm computations in token similarity
* Contributor agreement
* Remove ')' for clarity (#2737)
Sorry, don't mean to be nitpicky, I just noticed this when going through the CLI and thought it was a quick fix. That said, if this was intention than please let me know.
* added contributor agreement for mbkupfer (#2738)
* Basic support for Telugu language (#2751)
* Lex _attrs for polish language (#2750)
* Signed spaCy contributor agreement
* Added polish version of english lex_attrs
* Introduces a bulk merge function, in order to solve issue #653 (#2696)
* Fix comment
* Introduce bulk merge to increase performance on many span merges
* Sign contributor agreement
* Implement pull request suggestions
* Describe converters more explicitly (see #2643)
* Add multi-threading note to Language.pipe (resolves #2582) [ci skip]
* Fix formatting
* Fix dependency scheme docs (closes #2705) [ci skip]
* Don't set stop word in example (closes #2657) [ci skip]
* Add words to portuguese language _num_words (#2759)
* Add words to portuguese language _num_words
* Add words to portuguese language _num_words
* Update Indonesian model (#2752)
* adding e-KTP in tokenizer exceptions list
* add exception token
* removing lines with containing space as it won't matter since we use .split() method in the end, added new tokens in exception
* add tokenizer exceptions list
* combining base_norms with norm_exceptions
* adding norm_exception
* fix double key in lemmatizer
* remove unused import on punctuation.py
* reformat stop_words to reduce number of lines, improve readibility
* updating tokenizer exception
* implement is_currency for lang/id
* adding orth_first_upper in tokenizer_exceptions
* update the norm_exception list
* remove bunch of abbreviations
* adding contributors file
* Fixed spaCy+Keras example (#2763)
* bug fixes in keras example
* created contributor agreement
* Adding French hyphenated first name (#2786)
* Fix typo (closes #2784)
* Fix typo (#2795) [ci skip]
Fixed typo on line 6 "regcognizer --> recognizer"
* Adding basic support for Sinhala language. (#2788)
* adding Sinhala language package, stop words, examples and lex_attrs.
* Adding contributor agreement
* Updating contributor agreement
* Also include lowercase norm exceptions
* Fix error (#2802)
* Fix error
ValueError: cannot resize an array that references or is referenced
by another array in this way. Use the resize function
* added spaCy Contributor Agreement
* Add charlax's contributor agreement (#2805)
* agreement of contributor, may I introduce a tiny pl languge contribution (#2799)
* Contributors agreement
* Contributors agreement
* Contributors agreement
* Add jupyter=True to displacy.render in documentation (#2806)
* Revert "Also include lowercase norm exceptions"
This reverts commit 70f4e8adf37cfcfab60be2b97d6deae949b30e9e.
* Remove deprecated encoding argument to msgpack
* Set up dependency tree pattern matching skeleton (#2732)
* Fix bug when too many entity types. Fixes #2800
* Fix Python 2 test failure
* Require older msgpack-numpy
* Restore encoding arg on msgpack-numpy
* Try to fix version pin for msgpack-numpy
* Update Portuguese Language (#2790)
* Add words to portuguese language _num_words
* Add words to portuguese language _num_words
* Portuguese - Add/remove stopwords, fix tokenizer, add currency symbols
* Extended punctuation and norm_exceptions in the Portuguese language
* Correct error in spacy universe docs concerning spacy-lookup (#2814)
* Update Keras Example for (Parikh et al, 2016) implementation (#2803)
* bug fixes in keras example
* created contributor agreement
* baseline for Parikh model
* initial version of parikh 2016 implemented
* tested asymmetric models
* fixed grevious error in normalization
* use standard SNLI test file
* begin to rework parikh example
* initial version of running example
* start to document the new version
* start to document the new version
* Update Decompositional Attention.ipynb
* fixed calls to similarity
* updated the README
* import sys package duh
* simplified indexing on mapping word to IDs
* stupid python indent error
* added code from https://github.com/tensorflow/tensorflow/issues/3388 for tf bug workaround
* Fix typo (closes #2815) [ci skip]
* Update regex version dependency
* Set version to 2.0.13.dev3
* Skip seemingly problematic test
* Remove problematic test
* Try previous version of regex
* Revert "Remove problematic test"
This reverts commit bdebbef45552d698d390aa430b527ee27830f11b.
* Unskip test
* Try older version of regex
* 💫 Update training examples and use minibatching (#2830)
<!--- Provide a general summary of your changes in the title. -->
## Description
Update the training examples in `/examples/training` to show usage of spaCy's `minibatch` and `compounding` helpers ([see here](https://spacy.io/usage/training#tips-batch-size) for details). The lack of batching in the examples has caused some confusion in the past, especially for beginners who would copy-paste the examples, update them with large training sets and experienced slow and unsatisfying results.
### Types of change
enhancements
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Visual C++ link updated (#2842) (closes #2841) [ci skip]
* New landing page
* Add contribution agreement
* Correcting lang/ru/examples.py (#2845)
* Correct some grammatical inaccuracies in lang\ru\examples.py; filled Contributor Agreement
* Correct some grammatical inaccuracies in lang\ru\examples.py
* Move contributor agreement to separate file
* Set version to 2.0.13.dev4
* Add Persian(Farsi) language support (#2797)
* Also include lowercase norm exceptions
* Remove in favour of https://github.com/explosion/spaCy/graphs/contributors
* Rule-based French Lemmatizer (#2818)
<!--- Provide a general summary of your changes in the title. -->
## Description
<!--- Use this section to describe your changes. If your changes required
testing, include information about the testing environment and the tests you
ran. If your test fixes a bug reported in an issue, don't forget to include the
issue number. If your PR is still a work in progress, that's totally fine – just
include a note to let us know. -->
Add a rule-based French Lemmatizer following the english one and the excellent PR for [greek language optimizations](https://github.com/explosion/spaCy/pull/2558) to adapt the Lemmatizer class.
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
- Lemma dictionary used can be found [here](http://infolingu.univ-mlv.fr/DonneesLinguistiques/Dictionnaires/telechargement.html), I used the XML version.
- Add several files containing exhaustive list of words for each part of speech
- Add some lemma rules
- Add POS that are not checked in the standard Lemmatizer, i.e PRON, DET, ADV and AUX
- Modify the Lemmatizer class to check in lookup table as a last resort if POS not mentionned
- Modify the lemmatize function to check in lookup table as a last resort
- Init files are updated so the model can support all the functionalities mentioned above
- Add words to tokenizer_exceptions_list.py in respect to regex used in tokenizer_exceptions.py
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [X] I have submitted the spaCy Contributor Agreement.
- [X] I ran the tests, and all new and existing tests passed.
- [X] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Set version to 2.0.13
* Fix formatting and consistency
* Update docs for new version [ci skip]
* Increment version [ci skip]
* Add info on wheels [ci skip]
* Adding "This is a sentence" example to Sinhala (#2846)
* Add wheels badge
* Update badge [ci skip]
* Update README.rst [ci skip]
* Update murmurhash pin
* Increment version to 2.0.14.dev0
* Update GPU docs for v2.0.14
* Add wheel to setup_requires
* Import prefer_gpu and require_gpu functions from Thinc
* Add tests for prefer_gpu() and require_gpu()
* Update requirements and setup.py
* Workaround bug in thinc require_gpu
* Set version to v2.0.14
* Update push-tag script
* Unhack prefer_gpu
* Require thinc 6.10.6
* Update prefer_gpu and require_gpu docs [ci skip]
* Fix specifiers for GPU
* Set version to 2.0.14.dev1
* Set version to 2.0.14
* Update Thinc version pin
* Increment version
* Fix msgpack-numpy version pin
* Increment version
* Update version to 2.0.16
* Update version [ci skip]
* Redundant ')' in the Stop words' example (#2856)
<!--- Provide a general summary of your changes in the title. -->
## Description
<!--- Use this section to describe your changes. If your changes required
testing, include information about the testing environment and the tests you
ran. If your test fixes a bug reported in an issue, don't forget to include the
issue number. If your PR is still a work in progress, that's totally fine – just
include a note to let us know. -->
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [ ] I have submitted the spaCy Contributor Agreement.
- [ ] I ran the tests, and all new and existing tests passed.
- [ ] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Documentation improvement regarding joblib and SO (#2867)
Some documentation improvements
## Description
1. Fixed the dead URL to joblib
2. Fixed Stack Overflow brand name (with space)
### Types of change
Documentation
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* raise error when setting overlapping entities as doc.ents (#2880)
* Fix out-of-bounds access in NER training
The helper method state.B(1) gets the index of the first token of the
buffer, or -1 if no such token exists. Normally this is safe because we
pass this to functions like state.safe_get(), which returns an empty
token. Here we used it directly as an array index, which is not okay!
This error may have been the cause of out-of-bounds access errors during
training. Similar errors may still be around, so much be hunted down.
Hunting this one down took a long time...I printed out values across
training runs and diffed, looking for points of divergence between
runs, when no randomness should be allowed.
* Change PyThaiNLP Url (#2876)
* Fix missing comma
* Add example showing a fix-up rule for space entities
* Set version to 2.0.17.dev0
* Update regex version
* Revert "Update regex version"
This reverts commit 62358dd867d15bc6a475942dff34effba69dd70a.
* Try setting older regex version, to align with conda
* Set version to 2.0.17
* Add spacy-js to universe [ci-skip]
* Add spacy-raspberry to universe (closes #2889)
* Add script to validate universe json [ci skip]
* Removed space in docs + added contributor indo (#2909)
* - removed unneeded space in documentation
* - added contributor info
* Allow input text of length up to max_length, inclusive (#2922)
* Include universe spec for spacy-wordnet component (#2919)
* feat: include universe spec for spacy-wordnet component
* chore: include spaCy contributor agreement
* Minor formatting changes [ci skip]
* Fix image [ci skip]
Twitter URL doesn't work on live site
* Check if the word is in one of the regular lists specific to each POS (#2886)
* 💫 Create random IDs for SVGs to prevent ID clashes (#2927)
Resolves #2924.
## Description
Fixes problem where multiple visualizations in Jupyter notebooks would have clashing arc IDs, resulting in weirdly positioned arc labels. Generating a random ID prefix so even identical parses won't receive the same IDs for consistency (even if effect of ID clash isn't noticable here.)
### Types of change
bug fix
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Fix typo [ci skip]
* fixes symbolic link on py3 and windows (#2949)
* fixes symbolic link on py3 and windows
during setup of spacy using command
python -m spacy link en_core_web_sm en
closes #2948
* Update spacy/compat.py
Co-Authored-By: cicorias <cicorias@users.noreply.github.com>
* Fix formatting
* Update universe [ci skip]
* Catalan Language Support (#2940)
* Catalan language Support
* Ddding Catalan to documentation
* Sort languages alphabetically [ci skip]
* Update tests for pytest 4.x (#2965)
<!--- Provide a general summary of your changes in the title. -->
## Description
- [x] Replace marks in params for pytest 4.0 compat ([see here](https://docs.pytest.org/en/latest/deprecations.html#marks-in-pytest-mark-parametrize))
- [x] Un-xfail passing tests (some fixes in a recent update resolved a bunch of issues, but tests were apparently never updated here)
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Fix regex pin to harmonize with conda (#2964)
* Update README.rst
* Fix bug where Vocab.prune_vector did not use 'batch_size' (#2977)
Fixes #2976
* Fix typo
* Fix typo
* Remove duplicate file
* Require thinc 7.0.0.dev2
Fixes bug in gpu_ops that would use cupy instead of numpy on CPU
* Add missing import
* Fix error IDs
* Fix tests
2018-11-29 18:30:29 +03:00
|
|
|
"""
|
|
|
|
|
Slang and abbreviations
|
|
|
|
|
|
|
|
|
|
Daftar kosakata yang sering salah dieja
|
|
|
|
|
https://id.wikipedia.org/wiki/Wikipedia:Daftar_kosakata_bahasa_Indonesia_yang_sering_salah_dieja
|
|
|
|
|
|
|
|
|
|
"""
|
2017-07-23 18:54:19 +03:00
|
|
|
# coding: utf8
|
|
|
|
|
from __future__ import unicode_literals
|
|
|
|
|
|
💫 Port master changes over to develop (#2979)
* Create aryaprabhudesai.md (#2681)
* Update _install.jade (#2688)
Typo fix: "models" -> "model"
* Add FAC to spacy.explain (resolves #2706)
* Remove docstrings for deprecated arguments (see #2703)
* When calling getoption() in conftest.py, pass a default option (#2709)
* When calling getoption() in conftest.py, pass a default option
This is necessary to allow testing an installed spacy by running:
pytest --pyargs spacy
* Add contributor agreement
* update bengali token rules for hyphen and digits (#2731)
* Less norm computations in token similarity (#2730)
* Less norm computations in token similarity
* Contributor agreement
* Remove ')' for clarity (#2737)
Sorry, don't mean to be nitpicky, I just noticed this when going through the CLI and thought it was a quick fix. That said, if this was intention than please let me know.
* added contributor agreement for mbkupfer (#2738)
* Basic support for Telugu language (#2751)
* Lex _attrs for polish language (#2750)
* Signed spaCy contributor agreement
* Added polish version of english lex_attrs
* Introduces a bulk merge function, in order to solve issue #653 (#2696)
* Fix comment
* Introduce bulk merge to increase performance on many span merges
* Sign contributor agreement
* Implement pull request suggestions
* Describe converters more explicitly (see #2643)
* Add multi-threading note to Language.pipe (resolves #2582) [ci skip]
* Fix formatting
* Fix dependency scheme docs (closes #2705) [ci skip]
* Don't set stop word in example (closes #2657) [ci skip]
* Add words to portuguese language _num_words (#2759)
* Add words to portuguese language _num_words
* Add words to portuguese language _num_words
* Update Indonesian model (#2752)
* adding e-KTP in tokenizer exceptions list
* add exception token
* removing lines with containing space as it won't matter since we use .split() method in the end, added new tokens in exception
* add tokenizer exceptions list
* combining base_norms with norm_exceptions
* adding norm_exception
* fix double key in lemmatizer
* remove unused import on punctuation.py
* reformat stop_words to reduce number of lines, improve readibility
* updating tokenizer exception
* implement is_currency for lang/id
* adding orth_first_upper in tokenizer_exceptions
* update the norm_exception list
* remove bunch of abbreviations
* adding contributors file
* Fixed spaCy+Keras example (#2763)
* bug fixes in keras example
* created contributor agreement
* Adding French hyphenated first name (#2786)
* Fix typo (closes #2784)
* Fix typo (#2795) [ci skip]
Fixed typo on line 6 "regcognizer --> recognizer"
* Adding basic support for Sinhala language. (#2788)
* adding Sinhala language package, stop words, examples and lex_attrs.
* Adding contributor agreement
* Updating contributor agreement
* Also include lowercase norm exceptions
* Fix error (#2802)
* Fix error
ValueError: cannot resize an array that references or is referenced
by another array in this way. Use the resize function
* added spaCy Contributor Agreement
* Add charlax's contributor agreement (#2805)
* agreement of contributor, may I introduce a tiny pl languge contribution (#2799)
* Contributors agreement
* Contributors agreement
* Contributors agreement
* Add jupyter=True to displacy.render in documentation (#2806)
* Revert "Also include lowercase norm exceptions"
This reverts commit 70f4e8adf37cfcfab60be2b97d6deae949b30e9e.
* Remove deprecated encoding argument to msgpack
* Set up dependency tree pattern matching skeleton (#2732)
* Fix bug when too many entity types. Fixes #2800
* Fix Python 2 test failure
* Require older msgpack-numpy
* Restore encoding arg on msgpack-numpy
* Try to fix version pin for msgpack-numpy
* Update Portuguese Language (#2790)
* Add words to portuguese language _num_words
* Add words to portuguese language _num_words
* Portuguese - Add/remove stopwords, fix tokenizer, add currency symbols
* Extended punctuation and norm_exceptions in the Portuguese language
* Correct error in spacy universe docs concerning spacy-lookup (#2814)
* Update Keras Example for (Parikh et al, 2016) implementation (#2803)
* bug fixes in keras example
* created contributor agreement
* baseline for Parikh model
* initial version of parikh 2016 implemented
* tested asymmetric models
* fixed grevious error in normalization
* use standard SNLI test file
* begin to rework parikh example
* initial version of running example
* start to document the new version
* start to document the new version
* Update Decompositional Attention.ipynb
* fixed calls to similarity
* updated the README
* import sys package duh
* simplified indexing on mapping word to IDs
* stupid python indent error
* added code from https://github.com/tensorflow/tensorflow/issues/3388 for tf bug workaround
* Fix typo (closes #2815) [ci skip]
* Update regex version dependency
* Set version to 2.0.13.dev3
* Skip seemingly problematic test
* Remove problematic test
* Try previous version of regex
* Revert "Remove problematic test"
This reverts commit bdebbef45552d698d390aa430b527ee27830f11b.
* Unskip test
* Try older version of regex
* 💫 Update training examples and use minibatching (#2830)
<!--- Provide a general summary of your changes in the title. -->
## Description
Update the training examples in `/examples/training` to show usage of spaCy's `minibatch` and `compounding` helpers ([see here](https://spacy.io/usage/training#tips-batch-size) for details). The lack of batching in the examples has caused some confusion in the past, especially for beginners who would copy-paste the examples, update them with large training sets and experienced slow and unsatisfying results.
### Types of change
enhancements
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Visual C++ link updated (#2842) (closes #2841) [ci skip]
* New landing page
* Add contribution agreement
* Correcting lang/ru/examples.py (#2845)
* Correct some grammatical inaccuracies in lang\ru\examples.py; filled Contributor Agreement
* Correct some grammatical inaccuracies in lang\ru\examples.py
* Move contributor agreement to separate file
* Set version to 2.0.13.dev4
* Add Persian(Farsi) language support (#2797)
* Also include lowercase norm exceptions
* Remove in favour of https://github.com/explosion/spaCy/graphs/contributors
* Rule-based French Lemmatizer (#2818)
<!--- Provide a general summary of your changes in the title. -->
## Description
<!--- Use this section to describe your changes. If your changes required
testing, include information about the testing environment and the tests you
ran. If your test fixes a bug reported in an issue, don't forget to include the
issue number. If your PR is still a work in progress, that's totally fine – just
include a note to let us know. -->
Add a rule-based French Lemmatizer following the english one and the excellent PR for [greek language optimizations](https://github.com/explosion/spaCy/pull/2558) to adapt the Lemmatizer class.
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
- Lemma dictionary used can be found [here](http://infolingu.univ-mlv.fr/DonneesLinguistiques/Dictionnaires/telechargement.html), I used the XML version.
- Add several files containing exhaustive list of words for each part of speech
- Add some lemma rules
- Add POS that are not checked in the standard Lemmatizer, i.e PRON, DET, ADV and AUX
- Modify the Lemmatizer class to check in lookup table as a last resort if POS not mentionned
- Modify the lemmatize function to check in lookup table as a last resort
- Init files are updated so the model can support all the functionalities mentioned above
- Add words to tokenizer_exceptions_list.py in respect to regex used in tokenizer_exceptions.py
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [X] I have submitted the spaCy Contributor Agreement.
- [X] I ran the tests, and all new and existing tests passed.
- [X] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Set version to 2.0.13
* Fix formatting and consistency
* Update docs for new version [ci skip]
* Increment version [ci skip]
* Add info on wheels [ci skip]
* Adding "This is a sentence" example to Sinhala (#2846)
* Add wheels badge
* Update badge [ci skip]
* Update README.rst [ci skip]
* Update murmurhash pin
* Increment version to 2.0.14.dev0
* Update GPU docs for v2.0.14
* Add wheel to setup_requires
* Import prefer_gpu and require_gpu functions from Thinc
* Add tests for prefer_gpu() and require_gpu()
* Update requirements and setup.py
* Workaround bug in thinc require_gpu
* Set version to v2.0.14
* Update push-tag script
* Unhack prefer_gpu
* Require thinc 6.10.6
* Update prefer_gpu and require_gpu docs [ci skip]
* Fix specifiers for GPU
* Set version to 2.0.14.dev1
* Set version to 2.0.14
* Update Thinc version pin
* Increment version
* Fix msgpack-numpy version pin
* Increment version
* Update version to 2.0.16
* Update version [ci skip]
* Redundant ')' in the Stop words' example (#2856)
<!--- Provide a general summary of your changes in the title. -->
## Description
<!--- Use this section to describe your changes. If your changes required
testing, include information about the testing environment and the tests you
ran. If your test fixes a bug reported in an issue, don't forget to include the
issue number. If your PR is still a work in progress, that's totally fine – just
include a note to let us know. -->
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [ ] I have submitted the spaCy Contributor Agreement.
- [ ] I ran the tests, and all new and existing tests passed.
- [ ] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Documentation improvement regarding joblib and SO (#2867)
Some documentation improvements
## Description
1. Fixed the dead URL to joblib
2. Fixed Stack Overflow brand name (with space)
### Types of change
Documentation
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* raise error when setting overlapping entities as doc.ents (#2880)
* Fix out-of-bounds access in NER training
The helper method state.B(1) gets the index of the first token of the
buffer, or -1 if no such token exists. Normally this is safe because we
pass this to functions like state.safe_get(), which returns an empty
token. Here we used it directly as an array index, which is not okay!
This error may have been the cause of out-of-bounds access errors during
training. Similar errors may still be around, so much be hunted down.
Hunting this one down took a long time...I printed out values across
training runs and diffed, looking for points of divergence between
runs, when no randomness should be allowed.
* Change PyThaiNLP Url (#2876)
* Fix missing comma
* Add example showing a fix-up rule for space entities
* Set version to 2.0.17.dev0
* Update regex version
* Revert "Update regex version"
This reverts commit 62358dd867d15bc6a475942dff34effba69dd70a.
* Try setting older regex version, to align with conda
* Set version to 2.0.17
* Add spacy-js to universe [ci-skip]
* Add spacy-raspberry to universe (closes #2889)
* Add script to validate universe json [ci skip]
* Removed space in docs + added contributor indo (#2909)
* - removed unneeded space in documentation
* - added contributor info
* Allow input text of length up to max_length, inclusive (#2922)
* Include universe spec for spacy-wordnet component (#2919)
* feat: include universe spec for spacy-wordnet component
* chore: include spaCy contributor agreement
* Minor formatting changes [ci skip]
* Fix image [ci skip]
Twitter URL doesn't work on live site
* Check if the word is in one of the regular lists specific to each POS (#2886)
* 💫 Create random IDs for SVGs to prevent ID clashes (#2927)
Resolves #2924.
## Description
Fixes problem where multiple visualizations in Jupyter notebooks would have clashing arc IDs, resulting in weirdly positioned arc labels. Generating a random ID prefix so even identical parses won't receive the same IDs for consistency (even if effect of ID clash isn't noticable here.)
### Types of change
bug fix
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Fix typo [ci skip]
* fixes symbolic link on py3 and windows (#2949)
* fixes symbolic link on py3 and windows
during setup of spacy using command
python -m spacy link en_core_web_sm en
closes #2948
* Update spacy/compat.py
Co-Authored-By: cicorias <cicorias@users.noreply.github.com>
* Fix formatting
* Update universe [ci skip]
* Catalan Language Support (#2940)
* Catalan language Support
* Ddding Catalan to documentation
* Sort languages alphabetically [ci skip]
* Update tests for pytest 4.x (#2965)
<!--- Provide a general summary of your changes in the title. -->
## Description
- [x] Replace marks in params for pytest 4.0 compat ([see here](https://docs.pytest.org/en/latest/deprecations.html#marks-in-pytest-mark-parametrize))
- [x] Un-xfail passing tests (some fixes in a recent update resolved a bunch of issues, but tests were apparently never updated here)
### Types of change
<!-- What type of change does your PR cover? Is it a bug fix, an enhancement
or new feature, or a change to the documentation? -->
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
* Fix regex pin to harmonize with conda (#2964)
* Update README.rst
* Fix bug where Vocab.prune_vector did not use 'batch_size' (#2977)
Fixes #2976
* Fix typo
* Fix typo
* Remove duplicate file
* Require thinc 7.0.0.dev2
Fixes bug in gpu_ops that would use cupy instead of numpy on CPU
* Add missing import
* Fix error IDs
* Fix tests
2018-11-29 18:30:29 +03:00
|
|
|
_exc = {
|
|
|
|
|
# Slang and abbreviations
|
|
|
|
|
"silahkan": "silakan",
|
|
|
|
|
"yg": "yang",
|
|
|
|
|
"kalo": "kalau",
|
|
|
|
|
"cawu": "caturwulan",
|
|
|
|
|
"ok": "oke",
|
|
|
|
|
"gak": "tidak",
|
|
|
|
|
"enggak": "tidak",
|
|
|
|
|
"nggak": "tidak",
|
|
|
|
|
"ndak": "tidak",
|
|
|
|
|
"ngga": "tidak",
|
|
|
|
|
"dgn": "dengan",
|
|
|
|
|
"tdk": "tidak",
|
|
|
|
|
"jg": "juga",
|
|
|
|
|
"klo": "kalau",
|
|
|
|
|
"denger": "dengar",
|
|
|
|
|
"pinter": "pintar",
|
|
|
|
|
"krn": "karena",
|
|
|
|
|
"nemuin": "menemukan",
|
|
|
|
|
"jgn": "jangan",
|
|
|
|
|
"udah": "sudah",
|
|
|
|
|
"sy": "saya",
|
|
|
|
|
"udh": "sudah",
|
|
|
|
|
"dapetin": "mendapatkan",
|
|
|
|
|
"ngelakuin": "melakukan",
|
|
|
|
|
"ngebuat": "membuat",
|
|
|
|
|
"membikin": "membuat",
|
|
|
|
|
"bikin": "buat",
|
|
|
|
|
|
|
|
|
|
# Daftar kosakata yang sering salah dieja
|
|
|
|
|
"malpraktik": "malapraktik",
|
|
|
|
|
"malfungsi": "malafungsi",
|
|
|
|
|
"malserap": "malaserap",
|
|
|
|
|
"maladaptasi": "malaadaptasi",
|
|
|
|
|
"malsuai": "malasuai",
|
|
|
|
|
"maldistribusi": "maladistribusi",
|
|
|
|
|
"malgizi": "malagizi",
|
|
|
|
|
"malsikap": "malasikap",
|
|
|
|
|
"memperhatikan": "memerhatikan",
|
|
|
|
|
"akte": "akta",
|
|
|
|
|
"cemilan": "camilan",
|
|
|
|
|
"esei": "esai",
|
|
|
|
|
"frase": "frasa",
|
|
|
|
|
"kafeteria": "kafetaria",
|
|
|
|
|
"ketapel": "katapel",
|
|
|
|
|
"kenderaan": "kendaraan",
|
|
|
|
|
"menejemen": "manajemen",
|
|
|
|
|
"menejer": "manajer",
|
|
|
|
|
"mesjid": "masjid",
|
|
|
|
|
"rebo": "rabu",
|
|
|
|
|
"seksama": "saksama",
|
|
|
|
|
"senggama": "sanggama",
|
|
|
|
|
"sekedar": "sekadar",
|
|
|
|
|
"seprei": "seprai",
|
|
|
|
|
"semedi": "semadi",
|
|
|
|
|
"samadi": "semadi",
|
|
|
|
|
"amandemen": "amendemen",
|
|
|
|
|
"algoritma": "algoritme",
|
|
|
|
|
"aritmatika": "aritmetika",
|
|
|
|
|
"metoda": "metode",
|
|
|
|
|
"materai": "meterai",
|
|
|
|
|
"meterei": "meterai",
|
|
|
|
|
"kalendar": "kalender",
|
|
|
|
|
"kadaluwarsa": "kedaluwarsa",
|
|
|
|
|
"katagori": "kategori",
|
|
|
|
|
"parlamen": "parlemen",
|
|
|
|
|
"sekular": "sekuler",
|
|
|
|
|
"selular": "seluler",
|
|
|
|
|
"sirkular": "sirkuler",
|
|
|
|
|
"survai": "survei",
|
|
|
|
|
"survey": "survei",
|
|
|
|
|
"aktuil": "aktual",
|
|
|
|
|
"formil": "formal",
|
|
|
|
|
"trotoir": "trotoar",
|
|
|
|
|
"komersiil": "komersial",
|
|
|
|
|
"komersil": "komersial",
|
|
|
|
|
"tradisionil": "tradisionial",
|
|
|
|
|
"orisinil": "orisinal",
|
|
|
|
|
"orijinil": "orisinal",
|
|
|
|
|
"afdol": "afdal",
|
|
|
|
|
"antri": "antre",
|
|
|
|
|
"apotik": "apotek",
|
|
|
|
|
"atlit": "atlet",
|
|
|
|
|
"atmosfir": "atmosfer",
|
|
|
|
|
"cidera": "cedera",
|
|
|
|
|
"cendikiawan": "cendekiawan",
|
|
|
|
|
"cepet": "cepat",
|
|
|
|
|
"cinderamata": "cenderamata",
|
|
|
|
|
"debet": "debit",
|
|
|
|
|
"difinisi": "definisi",
|
|
|
|
|
"dekrit": "dekret",
|
|
|
|
|
"disain": "desain",
|
|
|
|
|
"diskripsi": "deskripsi",
|
|
|
|
|
"diskotik": "diskotek",
|
|
|
|
|
"eksim": "eksem",
|
|
|
|
|
"exim": "eksem",
|
|
|
|
|
"faidah": "faedah",
|
|
|
|
|
"ekstrim": "ekstrem",
|
|
|
|
|
"ekstrimis": "ekstremis",
|
|
|
|
|
"komplit": "komplet",
|
|
|
|
|
"konkrit": "konkret",
|
|
|
|
|
"kongkrit": "konkret",
|
|
|
|
|
"kongkret": "konkret",
|
|
|
|
|
"kridit": "kredit",
|
|
|
|
|
"musium": "museum",
|
|
|
|
|
"pinalti": "penalti",
|
|
|
|
|
"piranti": "peranti",
|
|
|
|
|
"pinsil": "pensil",
|
|
|
|
|
"personil": "personel",
|
|
|
|
|
"sistim": "sistem",
|
|
|
|
|
"teoritis": "teoretis",
|
|
|
|
|
"vidio": "video",
|
|
|
|
|
"cengkeh": "cengkih",
|
|
|
|
|
"desertasi": "disertasi",
|
|
|
|
|
"hakekat": "hakikat",
|
|
|
|
|
"intelejen": "intelijen",
|
|
|
|
|
"kaedah": "kaidah",
|
|
|
|
|
"kempes": "kempis",
|
|
|
|
|
"kementrian": "kementerian",
|
|
|
|
|
"ledeng": "leding",
|
|
|
|
|
"nasehat": "nasihat",
|
|
|
|
|
"penasehat": "penasihat",
|
|
|
|
|
"praktek": "praktik",
|
|
|
|
|
"praktekum": "praktikum",
|
|
|
|
|
"resiko": "risiko",
|
|
|
|
|
"retsleting": "ritsleting",
|
|
|
|
|
"senen": "senin",
|
|
|
|
|
"amuba": "ameba",
|
|
|
|
|
"punggawa": "penggawa",
|
|
|
|
|
"surban": "serban",
|
|
|
|
|
"nomer": "nomor",
|
|
|
|
|
"sorban": "serban",
|
|
|
|
|
"bis": "bus",
|
|
|
|
|
"agribisnis": "agrobisnis",
|
|
|
|
|
"kantung": "kantong",
|
|
|
|
|
"khutbah": "khotbah",
|
|
|
|
|
"mandur": "mandor",
|
|
|
|
|
"rubuh": "roboh",
|
|
|
|
|
"pastur": "pastor",
|
|
|
|
|
"supir": "sopir",
|
|
|
|
|
"goncang": "guncang",
|
|
|
|
|
"goa": "gua",
|
|
|
|
|
"kaos": "kaus",
|
|
|
|
|
"kokoh": "kukuh",
|
|
|
|
|
"komulatif": "kumulatif",
|
|
|
|
|
"kolomnis": "kolumnis",
|
|
|
|
|
"korma": "kurma",
|
|
|
|
|
"lobang": "lubang",
|
|
|
|
|
"limo": "limusin",
|
|
|
|
|
"limosin": "limusin",
|
|
|
|
|
"mangkok": "mangkuk",
|
|
|
|
|
"saos": "saus",
|
|
|
|
|
"sop": "sup",
|
|
|
|
|
"sorga": "surga",
|
|
|
|
|
"tegor": "tegur",
|
|
|
|
|
"telor": "telur",
|
|
|
|
|
"obrak-abrik": "ubrak-abrik",
|
|
|
|
|
"ekwivalen": "ekuivalen",
|
|
|
|
|
"frekwensi": "frekuensi",
|
|
|
|
|
"konsekwensi": "konsekuensi",
|
|
|
|
|
"kwadran": "kuadran",
|
|
|
|
|
"kwadrat": "kuadrat",
|
|
|
|
|
"kwalifikasi": "kualifikasi",
|
|
|
|
|
"kwalitas": "kualitas",
|
|
|
|
|
"kwalitet": "kualitas",
|
|
|
|
|
"kwalitatif": "kualitatif",
|
|
|
|
|
"kwantitas": "kuantitas",
|
|
|
|
|
"kwantitatif": "kuantitatif",
|
|
|
|
|
"kwantum": "kuantum",
|
|
|
|
|
"kwartal": "kuartal",
|
|
|
|
|
"kwintal": "kuintal",
|
|
|
|
|
"kwitansi": "kuitansi",
|
|
|
|
|
"kwatir": "khawatir",
|
|
|
|
|
"kuatir": "khawatir",
|
|
|
|
|
"jadual": "jadwal",
|
|
|
|
|
"hirarki": "hierarki",
|
|
|
|
|
"karir": "karier",
|
|
|
|
|
"aktip": "aktif",
|
|
|
|
|
"daptar": "daftar",
|
|
|
|
|
"efektip": "efektif",
|
|
|
|
|
"epektif": "efektif",
|
|
|
|
|
"epektip": "efektif",
|
|
|
|
|
"Pebruari": "Februari",
|
|
|
|
|
"pisik": "fisik",
|
|
|
|
|
"pondasi": "fondasi",
|
|
|
|
|
"photo": "foto",
|
|
|
|
|
"photokopi": "fotokopi",
|
|
|
|
|
"hapal": "hafal",
|
|
|
|
|
"insap": "insaf",
|
|
|
|
|
"insyaf": "insaf",
|
|
|
|
|
"konperensi": "konferensi",
|
|
|
|
|
"kreatip": "kreatif",
|
|
|
|
|
"kreativ": "kreatif",
|
|
|
|
|
"maap": "maaf",
|
|
|
|
|
"napsu": "nafsu",
|
|
|
|
|
"negatip": "negatif",
|
|
|
|
|
"negativ": "negatif",
|
|
|
|
|
"objektip": "objektif",
|
|
|
|
|
"obyektip": "objektif",
|
|
|
|
|
"obyektif": "objektif",
|
|
|
|
|
"pasip": "pasif",
|
|
|
|
|
"pasiv": "pasif",
|
|
|
|
|
"positip": "positif",
|
|
|
|
|
"positiv": "positif",
|
|
|
|
|
"produktip": "produktif",
|
|
|
|
|
"produktiv": "produktif",
|
|
|
|
|
"sarap": "saraf",
|
|
|
|
|
"sertipikat": "sertifikat",
|
|
|
|
|
"subjektip": "subjektif",
|
|
|
|
|
"subyektip": "subjektif",
|
|
|
|
|
"subyektif": "subjektif",
|
|
|
|
|
"tarip": "tarif",
|
|
|
|
|
"transitip": "transitif",
|
|
|
|
|
"transitiv": "transitif",
|
|
|
|
|
"faham": "paham",
|
|
|
|
|
"fikir": "pikir",
|
|
|
|
|
"berfikir": "berpikir",
|
|
|
|
|
"telefon": "telepon",
|
|
|
|
|
"telfon": "telepon",
|
|
|
|
|
"telpon": "telepon",
|
|
|
|
|
"tilpon": "telepon",
|
|
|
|
|
"nafas": "napas",
|
|
|
|
|
"bernafas": "bernapas",
|
|
|
|
|
"pernafasan": "pernapasan",
|
|
|
|
|
"vermak": "permak",
|
|
|
|
|
"vulpen": "pulpen",
|
|
|
|
|
"aktifis": "aktivis",
|
|
|
|
|
"konfeksi": "konveksi",
|
|
|
|
|
"motifasi": "motivasi",
|
|
|
|
|
"Nopember": "November",
|
|
|
|
|
"propinsi": "provinsi",
|
|
|
|
|
"babtis": "baptis",
|
|
|
|
|
"jerembab": "jerembap",
|
|
|
|
|
"lembab": "lembap",
|
|
|
|
|
"sembab": "sembap",
|
|
|
|
|
"saptu": "sabtu",
|
|
|
|
|
"tekat": "tekad",
|
|
|
|
|
"bejad": "bejat",
|
|
|
|
|
"nekad": "nekat",
|
|
|
|
|
"otoped": "otopet",
|
|
|
|
|
"skuad": "skuat",
|
|
|
|
|
"jenius": "genius",
|
|
|
|
|
"marjin": "margin",
|
|
|
|
|
"marjinal": "marginal",
|
|
|
|
|
"obyek": "objek",
|
|
|
|
|
"subyek": "subjek",
|
|
|
|
|
"projek": "proyek",
|
|
|
|
|
"azas": "asas",
|
|
|
|
|
"ijasah": "ijazah",
|
|
|
|
|
"jenasah": "jenazah",
|
|
|
|
|
"plasa": "plaza",
|
|
|
|
|
"bathin": "batin",
|
|
|
|
|
"Katholik": "Katolik",
|
|
|
|
|
"orthografi": "ortografi",
|
|
|
|
|
"pathogen": "patogen",
|
|
|
|
|
"theologi": "teologi",
|
|
|
|
|
"ijin": "izin",
|
|
|
|
|
"rejeki": "rezeki",
|
|
|
|
|
"rejim": "rezim",
|
|
|
|
|
"jaman": "zaman",
|
|
|
|
|
"jamrud": "zamrud",
|
|
|
|
|
"jinah": "zina",
|
|
|
|
|
"perjinahan": "perzinaan",
|
|
|
|
|
"anugrah": "anugerah",
|
|
|
|
|
"cendrawasih": "cenderawasih",
|
|
|
|
|
"jendral": "jenderal",
|
|
|
|
|
"kripik": "keripik",
|
|
|
|
|
"krupuk": "kerupuk",
|
|
|
|
|
"ksatria": "kesatria",
|
|
|
|
|
"mentri": "menteri",
|
|
|
|
|
"negri": "negeri",
|
|
|
|
|
"Prancis": "Perancis",
|
|
|
|
|
"sebrang": "seberang",
|
|
|
|
|
"menyebrang": "menyeberang",
|
|
|
|
|
"Sumatra": "Sumatera",
|
|
|
|
|
"trampil": "terampil",
|
|
|
|
|
"isteri": "istri",
|
|
|
|
|
"justeru": "justru",
|
|
|
|
|
"perajurit": "prajurit",
|
|
|
|
|
"putera": "putra",
|
|
|
|
|
"puteri": "putri",
|
|
|
|
|
"samudera": "samudra",
|
|
|
|
|
"sastera": "sastra",
|
|
|
|
|
"sutera": "sutra",
|
|
|
|
|
"terompet": "trompet",
|
|
|
|
|
"iklas": "ikhlas",
|
|
|
|
|
"iktisar": "ikhtisar",
|
|
|
|
|
"kafilah": "khafilah",
|
|
|
|
|
"kawatir": "khawatir",
|
|
|
|
|
"kotbah": "khotbah",
|
|
|
|
|
"kusyuk": "khusyuk",
|
|
|
|
|
"makluk": "makhluk",
|
|
|
|
|
"mahluk": "makhluk",
|
|
|
|
|
"mahkluk": "makhluk",
|
|
|
|
|
"nahkoda": "nakhoda",
|
|
|
|
|
"nakoda": "nakhoda",
|
|
|
|
|
"tahta": "takhta",
|
|
|
|
|
"takhyul": "takhayul",
|
|
|
|
|
"tahyul": "takhayul",
|
|
|
|
|
"tahayul": "takhayul",
|
|
|
|
|
"akhli": "ahli",
|
|
|
|
|
"anarkhi": "anarki",
|
|
|
|
|
"kharisma": "karisma",
|
|
|
|
|
"kharismatik": "karismatik",
|
|
|
|
|
"mahsud": "maksud",
|
|
|
|
|
"makhsud": "maksud",
|
|
|
|
|
"rakhmat": "rahmat",
|
|
|
|
|
"tekhnik": "teknik",
|
|
|
|
|
"tehnik": "teknik",
|
|
|
|
|
"tehnologi": "teknologi",
|
|
|
|
|
"ikhwal": "ihwal",
|
|
|
|
|
"expor": "ekspor",
|
|
|
|
|
"extra": "ekstra",
|
|
|
|
|
"komplex": "komplek",
|
|
|
|
|
"sex": "seks",
|
|
|
|
|
"taxi": "taksi",
|
|
|
|
|
"extasi": "ekstasi",
|
|
|
|
|
"syaraf": "saraf",
|
|
|
|
|
"syurga": "surga",
|
|
|
|
|
"mashur": "masyhur",
|
|
|
|
|
"masyur": "masyhur",
|
|
|
|
|
"mahsyur": "masyhur",
|
|
|
|
|
"mashyur": "masyhur",
|
|
|
|
|
"muadzin": "muazin",
|
|
|
|
|
"adzan": "azan",
|
|
|
|
|
"ustadz": "ustaz",
|
|
|
|
|
"ustad": "ustaz",
|
|
|
|
|
"ustadzah": "ustaz",
|
|
|
|
|
"dzikir": "zikir",
|
|
|
|
|
"dzuhur": "zuhur",
|
|
|
|
|
"dhuhur": "zuhur",
|
|
|
|
|
"zhuhur": "zuhur",
|
|
|
|
|
"analisa": "analisis",
|
|
|
|
|
"diagnosa": "diagnosis",
|
|
|
|
|
"hipotesa": "hipotesis",
|
|
|
|
|
"sintesa": "sintesis",
|
|
|
|
|
"aktiviti": "aktivitas",
|
|
|
|
|
"aktifitas": "aktivitas",
|
|
|
|
|
"efektifitas": "efektivitas",
|
|
|
|
|
"komuniti": "komunitas",
|
|
|
|
|
"kreatifitas": "kreativitas",
|
|
|
|
|
"produktifitas": "produktivitas",
|
|
|
|
|
"realiti": "realitas",
|
|
|
|
|
"realita": "realitas",
|
|
|
|
|
"selebriti": "selebritas",
|
|
|
|
|
"spotifitas": "sportivitas",
|
|
|
|
|
"universiti": "universitas",
|
|
|
|
|
"utiliti": "utilitas",
|
|
|
|
|
"validiti": "validitas",
|
|
|
|
|
"dilokalisir": "dilokalisasi",
|
|
|
|
|
"didramatisir": "didramatisasi",
|
|
|
|
|
"dipolitisir": "dipolitisasi",
|
|
|
|
|
"dinetralisir": "dinetralisasi",
|
|
|
|
|
"dikonfrontir": "dikonfrontasi",
|
|
|
|
|
"mendominir": "mendominasi",
|
|
|
|
|
"koordinir": "koordinasi",
|
|
|
|
|
"proklamir": "proklamasi",
|
|
|
|
|
"terorganisir": "terorganisasi",
|
|
|
|
|
"terealisir": "terealisasi",
|
|
|
|
|
"robah": "ubah",
|
|
|
|
|
"dirubah": "diubah",
|
|
|
|
|
"merubah": "mengubah",
|
|
|
|
|
"terlanjur": "telanjur",
|
|
|
|
|
"terlantar": "telantar",
|
|
|
|
|
"penglepasan": "pelepasan",
|
|
|
|
|
"pelihatan": "penglihatan",
|
|
|
|
|
"pemukiman": "permukiman",
|
|
|
|
|
"pengrumahan": "perumahan",
|
|
|
|
|
"penyewaan": "persewaan",
|
|
|
|
|
"menyintai": "mencintai",
|
|
|
|
|
"menyolok": "mencolok",
|
|
|
|
|
"contek": "sontek",
|
|
|
|
|
"mencontek": "menyontek",
|
|
|
|
|
"pungkir": "mungkir",
|
|
|
|
|
"dipungkiri": "dimungkiri",
|
|
|
|
|
"kupungkiri": "kumungkiri",
|
|
|
|
|
"kaupungkiri": "kaumungkiri",
|
|
|
|
|
"nampak": "tampak",
|
|
|
|
|
"nampaknya": "tampaknya",
|
|
|
|
|
"nongkrong": "tongkrong",
|
|
|
|
|
"berternak": "beternak",
|
|
|
|
|
"berterbangan": "beterbangan",
|
|
|
|
|
"berserta": "beserta",
|
|
|
|
|
"berperkara": "beperkara",
|
|
|
|
|
"berpergian": "bepergian",
|
|
|
|
|
"berkerja": "bekerja",
|
|
|
|
|
"berberapa": "beberapa",
|
|
|
|
|
"terbersit": "tebersit",
|
|
|
|
|
"terpercaya": "tepercaya",
|
|
|
|
|
"terperdaya": "teperdaya",
|
|
|
|
|
"terpercik": "tepercik",
|
|
|
|
|
"terpergok": "tepergok",
|
|
|
|
|
"aksesoris": "aksesori",
|
|
|
|
|
"handal": "andal",
|
|
|
|
|
"hantar": "antar",
|
|
|
|
|
"panutan": "anutan",
|
|
|
|
|
"atsiri": "asiri",
|
|
|
|
|
"bhakti": "bakti",
|
|
|
|
|
"china": "cina",
|
|
|
|
|
"dharma": "darma",
|
|
|
|
|
"diktaktor": "diktator",
|
|
|
|
|
"eksport": "ekspor",
|
|
|
|
|
"hembus": "embus",
|
|
|
|
|
"hadits": "hadis",
|
|
|
|
|
"hadist": "hadits",
|
|
|
|
|
"harafiah": "harfiah",
|
|
|
|
|
"himbau": "imbau",
|
|
|
|
|
"import": "impor",
|
|
|
|
|
"inget": "ingat",
|
|
|
|
|
"hisap": "isap",
|
|
|
|
|
"interprestasi": "interpretasi",
|
|
|
|
|
"kangker": "kanker",
|
|
|
|
|
"konggres": "kongres",
|
|
|
|
|
"lansekap": "lanskap",
|
|
|
|
|
"maghrib": "magrib",
|
|
|
|
|
"emak": "mak",
|
|
|
|
|
"moderen": "modern",
|
|
|
|
|
"pasport": "paspor",
|
|
|
|
|
"perduli": "peduli",
|
|
|
|
|
"ramadhan": "ramadan",
|
|
|
|
|
"rapih": "rapi",
|
|
|
|
|
"Sansekerta": "Sanskerta",
|
|
|
|
|
"shalat": "salat",
|
|
|
|
|
"sholat": "salat",
|
|
|
|
|
"silahkan": "silakan",
|
|
|
|
|
"standard": "standar",
|
|
|
|
|
"hutang": "utang",
|
|
|
|
|
"zinah": "zina",
|
|
|
|
|
"ambulan": "ambulans",
|
|
|
|
|
"antartika": "sntarktika",
|
|
|
|
|
"arteri": "arteria",
|
|
|
|
|
"asik": "asyik",
|
|
|
|
|
"australi": "australia",
|
|
|
|
|
"denga": "dengan",
|
|
|
|
|
"depo": "depot",
|
|
|
|
|
"detil": "detail",
|
|
|
|
|
"ensiklopedi": "ensiklopedia",
|
|
|
|
|
"elit": "elite",
|
|
|
|
|
"frustasi": "frustrasi",
|
|
|
|
|
"gladi": "geladi",
|
|
|
|
|
"greget": "gereget",
|
|
|
|
|
"itali": "italia",
|
|
|
|
|
"karna": "karena",
|
|
|
|
|
"klenteng": "kelenteng",
|
|
|
|
|
"erling": "kerling",
|
|
|
|
|
"kontruksi": "konstruksi",
|
|
|
|
|
"masal": "massal",
|
|
|
|
|
"merk": "merek",
|
|
|
|
|
"respon": "respons",
|
|
|
|
|
"diresponi": "direspons",
|
|
|
|
|
"skak": "sekak",
|
|
|
|
|
"stir": "setir",
|
|
|
|
|
"singapur": "singapura",
|
|
|
|
|
"standarisasi": "standardisasi",
|
|
|
|
|
"varitas": "varietas",
|
|
|
|
|
"amphibi": "amfibi",
|
|
|
|
|
"anjlog": "anjlok",
|
|
|
|
|
"alpukat": "avokad",
|
|
|
|
|
"alpokat": "avokad",
|
|
|
|
|
"bolpen": "pulpen",
|
|
|
|
|
"cabe": "cabai",
|
|
|
|
|
"cabay": "cabai",
|
|
|
|
|
"ceret": "cerek",
|
|
|
|
|
"differensial": "diferensial",
|
|
|
|
|
"duren": "durian",
|
|
|
|
|
"faksimili": "faksimile",
|
|
|
|
|
"faksimil": "faksimile",
|
|
|
|
|
"graha": "gerha",
|
|
|
|
|
"goblog": "goblok",
|
|
|
|
|
"gombrong": "gombroh",
|
|
|
|
|
"horden": "gorden",
|
|
|
|
|
"korden": "gorden",
|
|
|
|
|
"gubug": "gubuk",
|
|
|
|
|
"imaginasi": "imajinasi",
|
|
|
|
|
"jerigen": "jeriken",
|
|
|
|
|
"jirigen": "jeriken",
|
|
|
|
|
"carut-marut": "karut-marut",
|
|
|
|
|
"kwota": "kuota",
|
|
|
|
|
"mahzab": "mazhab",
|
|
|
|
|
"mempesona": "memesona",
|
|
|
|
|
"milyar": "miliar",
|
|
|
|
|
"missi": "misi",
|
|
|
|
|
"nenas": "nanas",
|
|
|
|
|
"negoisasi": "negosiasi",
|
|
|
|
|
"automotif": "otomotif",
|
|
|
|
|
"pararel": "paralel",
|
|
|
|
|
"paska": "pasca",
|
|
|
|
|
"prosen": "persen",
|
|
|
|
|
"pete": "petai",
|
|
|
|
|
"petay": "petai",
|
|
|
|
|
"proffesor": "profesor",
|
|
|
|
|
"rame": "ramai",
|
|
|
|
|
"rapot": "rapor",
|
|
|
|
|
"rileks": "relaks",
|
|
|
|
|
"rileksasi": "relaksasi",
|
|
|
|
|
"renumerasi": "remunerasi",
|
|
|
|
|
"seketaris": "sekretaris",
|
|
|
|
|
"sekertaris": "sekretaris",
|
|
|
|
|
"sensorik": "sensoris",
|
|
|
|
|
"sentausa": "sentosa",
|
|
|
|
|
"strawberi": "stroberi",
|
|
|
|
|
"strawbery": "stroberi",
|
|
|
|
|
"taqwa": "takwa",
|
|
|
|
|
"tauco": "taoco",
|
|
|
|
|
"tauge": "taoge",
|
|
|
|
|
"toge": "taoge",
|
|
|
|
|
"tauladan": "teladan",
|
|
|
|
|
"taubat": "tobat",
|
|
|
|
|
"trilyun": "triliun",
|
|
|
|
|
"vissi": "visi",
|
|
|
|
|
"coklat": "cokelat",
|
|
|
|
|
"narkotika": "narkotik",
|
|
|
|
|
"oase": "oasis",
|
|
|
|
|
"politisi": "politikus",
|
|
|
|
|
"terong": "terung",
|
|
|
|
|
"wool": "wol",
|
|
|
|
|
"himpit": "impit",
|
|
|
|
|
"mujizat": "mukjizat",
|
|
|
|
|
"mujijat": "mukjizat",
|
|
|
|
|
"yag": "yang",
|
|
|
|
|
}
|
2017-07-23 18:56:48 +03:00
|
|
|
|
2017-07-23 18:54:19 +03:00
|
|
|
NORM_EXCEPTIONS = {}
|
2017-07-23 18:56:48 +03:00
|
|
|
|
|
|
|
|
for string, norm in _exc.items():
|
|
|
|
|
NORM_EXCEPTIONS[string] = norm
|
|
|
|
|
NORM_EXCEPTIONS[string.title()] = norm
|
