* Add emoticons to tokenization

This commit is contained in:
Matthew Honnibal 2014-11-01 15:14:46 +11:00
parent 39743323ea
commit 11e42fd070

View File

@ -100,3 +100,59 @@ U.N. U.N.
Ms. Ms. Ms. Ms.
Mr. Mr. Mr. Mr.
P. P. P. P.
:) :)
<3 <3
;) ;)
(: (:
:( :(
-_- -_-
♥ ♥
=) =)
:/ :/
:> :>
;-) ;-)
:Y :Y
:P :P
:-P :-P
:3 :3
=3 =3
xD xD
^_^ ^_^
=] =]
=D =D
<333 <333
:)) :))
:0 :0
-__- -__-
♥♥♥ ♥♥♥
♥♡♥ ♥♡♥
ಠ◡ಠ ಠ◡ಠ
⌣́_⌣̀) ⌣́_⌣̀)
☺ ☺
☹ ☹
¯\_(ツ)_/¯ ¯\_(ツ)_/¯
xDD xDD
o_o o_o
o_O o_O
\m/ \m/
V_V V_V
=[[ =[[
<33 <33
;p ;p
;D ;D
;-p ;-p
;( ;(
:p :p
:] :]
:O :O
:-\ :-\
:-/ :-/
:-) :-)
:((( :(((
:(( :((
:') :')
(^_^) (^_^)
(= (=
o.O o.O
") ")
.... ....