291 Commits

Author	SHA1	Message	Date
DuyguA	cba63196f9	fixed typo	2018-03-09 10:54:18 +01:00
DuyguA	7a780476af	added more abbreviations	2018-03-09 10:13:00 +01:00
DuyguA	cca87756d7	added Sti	2018-03-08 18:07:52 +01:00
DuyguA	3c994311c5	added abbrevs	2018-03-08 18:03:27 +01:00
DuyguA	56d6fb180e	added like_num to lex	2018-03-08 15:25:25 +01:00
DuyguA	26ee0590a3	added some commonly used cases	2018-03-08 12:43:58 +01:00
DuyguA	ae6473e4d5	removed some words with negation particle.	2018-03-08 12:20:32 +01:00
DuyguA	6ed59a2198	removed number words to be caried to the lexical	2018-03-08 12:19:23 +01:00
DuyguA	04784a44a6	made alphabetical order for Turkish chaaracters	2018-03-08 12:11:32 +01:00
DuyguA	af33e022a5	added example sentences for Turkish	2018-03-08 12:06:03 +01:00
Ines Montani	35634352fe	Merge pull request #2025 from dejanmarich/patch-1 ... Update stop_words.py for Croatian language	2018-02-26 18:22:32 +01:00
Matthew Honnibal	5faae803c6	Add option to not use Janome for Japanese tokenization	2018-02-26 09:39:46 +01:00
Matthew Honnibal	9b406181cd	Add Chinese.Defaults.use_jieba setting, for UD	2018-02-25 15:12:38 +01:00
Matthew Honnibal	9ccd0c643b	Add Vietnamese	2018-02-25 15:00:46 +01:00
Matthew Honnibal	6d2c1ef52c	Fix SP tag in generic tag map	2018-02-24 16:04:56 +01:00
dejanmarich	71c261d58b	Update stop_words.py ... Added more words	2018-02-23 10:31:01 +01:00
Ines Montani	14e7e0f12a	Merge pull request #2000 from jimregan/polish-tag-map ... Polish tag map	2018-02-18 19:05:58 +01:00
Matthew Honnibal	eb3040ce46	Merge pull request #1891 from fucking-signup/master ... Fix issue #1889	2018-02-18 13:47:47 +01:00
4altinok	94fb0b75e3	code for is_currency	2018-02-11 18:51:32 +01:00
Ines Montani	0954e15dda	Merge pull request #1913 from ohenrik/nb_syntax_iterator ... Norwegian Language (nb) - Added french syntax iterator with explanation	2018-02-06 04:59:07 +01:00
Ole Henrik Skogstrøm	251a7805fe	Copied French syntax iterator to simplify future changes	2018-02-05 14:45:05 +01:00
ines	f1d3deffac	Add Russian example sentences (see #1107)	2018-02-01 20:09:40 +01:00
Ole Henrik Skogstrøm	e40465487c	Added french syntax iterator with explenation	2018-01-30 15:44:29 +01:00
Matthew Honnibal	cb7110c22e	Merge pull request #1882 from ohenrik/nb_lemma_and_tag_map ... Add norwegian bokmål ('nb') lemmatizer and tag_map	2018-01-29 18:18:50 +01:00
Ali Zarezade	bb6bd3d8ae	add persian language	2018-01-27 13:27:26 +03:30
Ali Zarezade	d195675db5	add persian language	2018-01-27 13:21:38 +03:30
Kit	4b42267ba3	Fix issue #1889	2018-01-25 23:17:22 +01:00
Ole Henrik Skogstrøm	8e2c9f2475	Cleaned up nb tag_map comments	2018-01-25 11:09:28 +01:00
Ole Henrik Skogstrøm	1107e89fcf	Updated doc string on nb tag_map module	2018-01-25 11:08:28 +01:00
Ole Henrik Skogstrøm	4058a7d579	Fix æøå characters in lemmatizer	2018-01-24 14:03:14 +01:00
Ole Henrik Skogstrøm	42248f423f	Updated tag map	2018-01-24 13:50:33 +01:00
Ole Henrik Skogstrøm	74b430b49a	Correct Lemmatizer	2018-01-24 13:26:33 +01:00
Ole Henrik Skogstrøm	b9b3a40c78	Add norwegian lemmatizer and tag_map	2018-01-24 12:28:29 +01:00
Ali Zarezade	42349471bc	add ٪ as punctuation	2018-01-23 18:11:33 +03:30
Ali Zarezade	2bda582135	Add Persian character and symbols ... Add Persian characters and the following: - ٪ used instead of % - ؟ used instead of ? - ﷼ used instead of $ - ، used instead of , - ؛ used instead of ;	2018-01-23 13:20:36 +03:30
Kit	701e7cc6aa	Rename variable to keep code consistent	2018-01-08 03:38:44 +01:00
Kit	ed0db95183	Find lowercased forms of ordinal words, where possible	2018-01-08 03:28:50 +01:00
Kit	9bc524982e	Find lowercased forms of numeric words	2018-01-08 03:25:08 +01:00
Kevin Humphreys	7918fa4ef9	handle would've	2018-01-03 12:25:48 -08:00
zqhZY	f27859fa99	add ChineseDefaults class for pickling	2017-12-28 17:13:58 +08:00
Søren Lind Kristiansen	bef735aef7	Fix Danish abbreviation 'm.h.t.'	2017-12-21 09:24:31 +01:00
Ines Montani	a3dd167d7f	Merge branch 'master' into da_ud_tokenization	2017-12-20 21:05:34 +00:00
Ines Montani	97f100f69f	Merge pull request #1742 from kimfalk/master ... Two corrections in the da lan.	2017-12-20 21:02:00 +00:00
Ines Montani	d682a8803e	Merge pull request #1672 from cbilgili/master ... Adds Turkish Lemmatization	2017-12-20 21:01:00 +00:00
Benjamin Peterson	9452134cd1	remove no-break spaces from Hindi example (fixes #1750)	2017-12-20 11:35:30 -08:00
Søren Lind Kristiansen	7a2f2f6f94	Fix formatting.	2017-12-20 18:37:37 +01:00
Søren Lind Kristiansen	15d13efafd	Tune Danish tokenizer to more closely match tokenization in Universal Dependencies.	2017-12-20 17:36:52 +01:00
Kim FalkJørgensen	648dc60755	Remove the incorrect exception 'm.h.t'	2017-12-20 10:02:39 +01:00
Kim FalkJørgensen	9c9f4ef84a	Fixing a translation error in examples.py ... Adding an exception in the tokenizer_exceptions.py	2017-12-19 15:26:50 +01:00
ines	22dc744b48	Fix check for '@' in like_url (see #1715)	2017-12-16 13:48:43 +01:00