explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-11-14 21:57:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
db81604d54
spaCy/examples/training/textcat_example_data/jigsaw-toxic-comment.json
adrianeboyd b5d999e510 Add textcat to train CLI (#4226) 
* Add doc.cats to spacy.gold at the paragraph level

Support `doc.cats` as `"cats": [{"label": string, "value": number}]` in
the spacy JSON training format at the paragraph level.

* `spacy.gold.docs_to_json()` writes `docs.cats`

* `GoldCorpus` reads in cats in each `GoldParse`

* Update instances of gold_tuples to handle cats

Update iteration over gold_tuples / gold_parses to handle addition of
cats at the paragraph level.

* Add textcat to train CLI

* Add textcat options to train CLI
* Add textcat labels in `TextCategorizer.begin_training()`
* Add textcat evaluation to `Scorer`:
  * For binary exclusive classes with provided label: F1 for label
  * For 2+ exclusive classes: F1 macro average
  * For multilabel (not exclusive): ROC AUC macro average (currently
relying on sklearn)
* Provide user info on textcat evaluation settings, potential
incompatibilities
* Provide pipeline to Scorer in `Language.evaluate` for textcat config
* Customize train CLI output to include only metrics relevant to current
pipeline
* Add textcat evaluation to evaluate CLI

* Fix handling of unset arguments and config params

Fix handling of unset arguments and model confiug parameters in Scorer
initialization.

* Temporarily add sklearn requirement

* Remove sklearn version number

* Improve Scorer handling of models without textcats

* Fixing Scorer handling of models without textcats

* Update Scorer output for python 2.7

* Modify inf in Scorer for python 2.7

* Auto-format

Also make small adjustments to make auto-formatting with black easier and produce nicer results

* Move error message to Errors

* Update documentation

* Add cats to annotation JSON format [ci skip]

* Fix tpl flag and docs [ci skip]

* Switch to internal roc_auc_score

Switch to internal `roc_auc_score()` adapted from scikit-learn.

* Add AUCROCScore tests and improve errors/warnings

* Add tests for AUCROCScore and roc_auc_score
* Add missing error for only positive/negative values
* Remove unnecessary warnings and errors

* Make reduced roc_auc_score functions private

Because most of the checks and warnings have been stripped for the
internal functions and access is only intended through `ROCAUCScore`,
make the functions for roc_auc_score adapted from scikit-learn private.

* Check that data corresponds with multilabel flag

Check that the training instances correspond with the multilabel flag,
adding the multilabel flag if required.

* Add textcat score to early stopping check

* Add more checks to debug-data for textcat

* Add example training data for textcat

* Add more checks to textcat train CLI

* Check configuration when extending base model
* Fix typos

* Update textcat example data

* Provide licensing details and licenses for data
* Remove two labels with no positive instances from jigsaw-toxic-comment
data.


Co-authored-by: Ines Montani <ines@ines.io>
2019-09-15 22:31:31 +02:00

2987 lines
66 KiB
JSON
Raw Blame History

[
{
"id":0,
"paragraphs":[
{
"raw":"Explanation\nWhy the edits made under my username Hardcore Metallica Fan were reverted? They weren't vandalisms, just closure on some GAs after I voted at New York Dolls FAC. And please don't remove the template from the talk page since I'm retired now.89.205.38.27",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"Explanation",
"ner":"O"
},
{
"id":1,
"orth":"\n",
"ner":"O"
},
{
"id":2,
"orth":"Why",
"ner":"O"
},
{
"id":3,
"orth":"the",
"ner":"O"
},
{
"id":4,
"orth":"edits",
"ner":"O"
},
{
"id":5,
"orth":"made",
"ner":"O"
},
{
"id":6,
"orth":"under",
"ner":"O"
},
{
"id":7,
"orth":"my",
"ner":"O"
},
{
"id":8,
"orth":"username",
"ner":"O"
},
{
"id":9,
"orth":"Hardcore",
"ner":"O"
},
{
"id":10,
"orth":"Metallica",
"ner":"O"
},
{
"id":11,
"orth":"Fan",
"ner":"O"
},
{
"id":12,
"orth":"were",
"ner":"O"
},
{
"id":13,
"orth":"reverted",
"ner":"O"
},
{
"id":14,
"orth":"?",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":15,
"orth":"They",
"ner":"O"
},
{
"id":16,
"orth":"were",
"ner":"O"
},
{
"id":17,
"orth":"n't",
"ner":"O"
},
{
"id":18,
"orth":"vandalisms",
"ner":"O"
},
{
"id":19,
"orth":",",
"ner":"O"
},
{
"id":20,
"orth":"just",
"ner":"O"
},
{
"id":21,
"orth":"closure",
"ner":"O"
},
{
"id":22,
"orth":"on",
"ner":"O"
},
{
"id":23,
"orth":"some",
"ner":"O"
},
{
"id":24,
"orth":"GAs",
"ner":"O"
},
{
"id":25,
"orth":"after",
"ner":"O"
},
{
"id":26,
"orth":"I",
"ner":"O"
},
{
"id":27,
"orth":"voted",
"ner":"O"
},
{
"id":28,
"orth":"at",
"ner":"O"
},
{
"id":29,
"orth":"New",
"ner":"O"
},
{
"id":30,
"orth":"York",
"ner":"O"
},
{
"id":31,
"orth":"Dolls",
"ner":"O"
},
{
"id":32,
"orth":"FAC",
"ner":"O"
},
{
"id":33,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":34,
"orth":"And",
"ner":"O"
},
{
"id":35,
"orth":"please",
"ner":"O"
},
{
"id":36,
"orth":"do",
"ner":"O"
},
{
"id":37,
"orth":"n't",
"ner":"O"
},
{
"id":38,
"orth":"remove",
"ner":"O"
},
{
"id":39,
"orth":"the",
"ner":"O"
},
{
"id":40,
"orth":"template",
"ner":"O"
},
{
"id":41,
"orth":"from",
"ner":"O"
},
{
"id":42,
"orth":"the",
"ner":"O"
},
{
"id":43,
"orth":"talk",
"ner":"O"
},
{
"id":44,
"orth":"page",
"ner":"O"
},
{
"id":45,
"orth":"since",
"ner":"O"
},
{
"id":46,
"orth":"I",
"ner":"O"
},
{
"id":47,
"orth":"'m",
"ner":"O"
},
{
"id":48,
"orth":"retired",
"ner":"O"
},
{
"id":49,
"orth":"now.89.205.38.27",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"I'm Sorry \n\nI'm sorry I screwed around with someones talk page. It was very bad to do. I know how having the templates on their talk page helps you assert your dominance over them. I know I should bow down to the almighty administrators. But then again, I'm going to go play outside....with your mom. 76.122.79.82",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"I",
"ner":"O"
},
{
"id":1,
"orth":"'m",
"ner":"O"
},
{
"id":2,
"orth":"Sorry",
"ner":"O"
},
{
"id":3,
"orth":"\n\n",
"ner":"O"
},
{
"id":4,
"orth":"I",
"ner":"O"
},
{
"id":5,
"orth":"'m",
"ner":"O"
},
{
"id":6,
"orth":"sorry",
"ner":"O"
},
{
"id":7,
"orth":"I",
"ner":"O"
},
{
"id":8,
"orth":"screwed",
"ner":"O"
},
{
"id":9,
"orth":"around",
"ner":"O"
},
{
"id":10,
"orth":"with",
"ner":"O"
},
{
"id":11,
"orth":"someones",
"ner":"O"
},
{
"id":12,
"orth":"talk",
"ner":"O"
},
{
"id":13,
"orth":"page",
"ner":"O"
},
{
"id":14,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":15,
"orth":" ",
"ner":"O"
},
{
"id":16,
"orth":"It",
"ner":"O"
},
{
"id":17,
"orth":"was",
"ner":"O"
},
{
"id":18,
"orth":"very",
"ner":"O"
},
{
"id":19,
"orth":"bad",
"ner":"O"
},
{
"id":20,
"orth":"to",
"ner":"O"
},
{
"id":21,
"orth":"do",
"ner":"O"
},
{
"id":22,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":23,
"orth":" ",
"ner":"O"
},
{
"id":24,
"orth":"I",
"ner":"O"
},
{
"id":25,
"orth":"know",
"ner":"O"
},
{
"id":26,
"orth":"how",
"ner":"O"
},
{
"id":27,
"orth":"having",
"ner":"O"
},
{
"id":28,
"orth":"the",
"ner":"O"
},
{
"id":29,
"orth":"templates",
"ner":"O"
},
{
"id":30,
"orth":"on",
"ner":"O"
},
{
"id":31,
"orth":"their",
"ner":"O"
},
{
"id":32,
"orth":"talk",
"ner":"O"
},
{
"id":33,
"orth":"page",
"ner":"O"
},
{
"id":34,
"orth":"helps",
"ner":"O"
},
{
"id":35,
"orth":"you",
"ner":"O"
},
{
"id":36,
"orth":"assert",
"ner":"O"
},
{
"id":37,
"orth":"your",
"ner":"O"
},
{
"id":38,
"orth":"dominance",
"ner":"O"
},
{
"id":39,
"orth":"over",
"ner":"O"
},
{
"id":40,
"orth":"them",
"ner":"O"
},
{
"id":41,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":42,
"orth":" ",
"ner":"O"
},
{
"id":43,
"orth":"I",
"ner":"O"
},
{
"id":44,
"orth":"know",
"ner":"O"
},
{
"id":45,
"orth":"I",
"ner":"O"
},
{
"id":46,
"orth":"should",
"ner":"O"
},
{
"id":47,
"orth":"bow",
"ner":"O"
},
{
"id":48,
"orth":"down",
"ner":"O"
},
{
"id":49,
"orth":"to",
"ner":"O"
},
{
"id":50,
"orth":"the",
"ner":"O"
},
{
"id":51,
"orth":"almighty",
"ner":"O"
},
{
"id":52,
"orth":"administrators",
"ner":"O"
},
{
"id":53,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":54,
"orth":" ",
"ner":"O"
},
{
"id":55,
"orth":"But",
"ner":"O"
},
{
"id":56,
"orth":"then",
"ner":"O"
},
{
"id":57,
"orth":"again",
"ner":"O"
},
{
"id":58,
"orth":",",
"ner":"O"
},
{
"id":59,
"orth":"I",
"ner":"O"
},
{
"id":60,
"orth":"'m",
"ner":"O"
},
{
"id":61,
"orth":"going",
"ner":"O"
},
{
"id":62,
"orth":"to",
"ner":"O"
},
{
"id":63,
"orth":"go",
"ner":"O"
},
{
"id":64,
"orth":"play",
"ner":"O"
},
{
"id":65,
"orth":"outside",
"ner":"O"
},
{
"id":66,
"orth":"....",
"ner":"O"
},
{
"id":67,
"orth":"with",
"ner":"O"
},
{
"id":68,
"orth":"your",
"ner":"O"
},
{
"id":69,
"orth":"mom",
"ner":"O"
},
{
"id":70,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":71,
"orth":" ",
"ner":"O"
},
{
"id":72,
"orth":"76.122.79.82",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":1
}
]
},
{
"raw":"Stupid peace of shit stop deleting my stuff asshole go die and fall in a hole go to hell!",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"Stupid",
"ner":"O"
},
{
"id":1,
"orth":"peace",
"ner":"O"
},
{
"id":2,
"orth":"of",
"ner":"O"
},
{
"id":3,
"orth":"shit",
"ner":"O"
},
{
"id":4,
"orth":"stop",
"ner":"O"
},
{
"id":5,
"orth":"deleting",
"ner":"O"
},
{
"id":6,
"orth":"my",
"ner":"O"
},
{
"id":7,
"orth":"stuff",
"ner":"O"
},
{
"id":8,
"orth":"asshole",
"ner":"O"
},
{
"id":9,
"orth":"go",
"ner":"O"
},
{
"id":10,
"orth":"die",
"ner":"O"
},
{
"id":11,
"orth":"and",
"ner":"O"
},
{
"id":12,
"orth":"fall",
"ner":"O"
},
{
"id":13,
"orth":"in",
"ner":"O"
},
{
"id":14,
"orth":"a",
"ner":"O"
},
{
"id":15,
"orth":"hole",
"ner":"O"
},
{
"id":16,
"orth":"go",
"ner":"O"
},
{
"id":17,
"orth":"to",
"ner":"O"
},
{
"id":18,
"orth":"hell",
"ner":"O"
},
{
"id":19,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":1
},
{
"label":"obscene",
"value":1
},
{
"label":"severe_toxic",
"value":1
},
{
"label":"toxic",
"value":1
}
]
},
{
"raw":"D'aww! He matches this background colour I'm seemingly stuck with. Thanks. (talk) 21:51, January 11, 2016 (UTC)",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"D'aww",
"ner":"O"
},
{
"id":1,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":2,
"orth":"He",
"ner":"O"
},
{
"id":3,
"orth":"matches",
"ner":"O"
},
{
"id":4,
"orth":"this",
"ner":"O"
},
{
"id":5,
"orth":"background",
"ner":"O"
},
{
"id":6,
"orth":"colour",
"ner":"O"
},
{
"id":7,
"orth":"I",
"ner":"O"
},
{
"id":8,
"orth":"'m",
"ner":"O"
},
{
"id":9,
"orth":"seemingly",
"ner":"O"
},
{
"id":10,
"orth":"stuck",
"ner":"O"
},
{
"id":11,
"orth":"with",
"ner":"O"
},
{
"id":12,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":13,
"orth":"Thanks",
"ner":"O"
},
{
"id":14,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":15,
"orth":" ",
"ner":"O"
},
{
"id":16,
"orth":"(",
"ner":"O"
},
{
"id":17,
"orth":"talk",
"ner":"O"
},
{
"id":18,
"orth":")",
"ner":"O"
},
{
"id":19,
"orth":"21:51",
"ner":"O"
},
{
"id":20,
"orth":",",
"ner":"O"
},
{
"id":21,
"orth":"January",
"ner":"O"
},
{
"id":22,
"orth":"11",
"ner":"O"
},
{
"id":23,
"orth":",",
"ner":"O"
},
{
"id":24,
"orth":"2016",
"ner":"O"
},
{
"id":25,
"orth":"(",
"ner":"O"
},
{
"id":26,
"orth":"UTC",
"ner":"O"
},
{
"id":27,
"orth":")",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"GET FUCKED UP. GET FUCKEEED UP. GOT A DRINK THAT YOU CANT PUT DOWN???/ GET FUCK UP GET FUCKED UP. I'M FUCKED UP RIGHT NOW!",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"GET",
"ner":"O"
},
{
"id":1,
"orth":"FUCKED",
"ner":"O"
},
{
"id":2,
"orth":"UP",
"ner":"O"
},
{
"id":3,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":4,
"orth":"GET",
"ner":"O"
},
{
"id":5,
"orth":"FUCKEEED",
"ner":"O"
},
{
"id":6,
"orth":"UP",
"ner":"O"
},
{
"id":7,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":8,
"orth":" ",
"ner":"O"
},
{
"id":9,
"orth":"GOT",
"ner":"O"
},
{
"id":10,
"orth":"A",
"ner":"O"
},
{
"id":11,
"orth":"DRINK",
"ner":"O"
},
{
"id":12,
"orth":"THAT",
"ner":"O"
},
{
"id":13,
"orth":"YOU",
"ner":"O"
},
{
"id":14,
"orth":"CANT",
"ner":"O"
},
{
"id":15,
"orth":"PUT",
"ner":"O"
},
{
"id":16,
"orth":"DOWN???/",
"ner":"O"
},
{
"id":17,
"orth":"GET",
"ner":"O"
},
{
"id":18,
"orth":"FUCK",
"ner":"O"
},
{
"id":19,
"orth":"UP",
"ner":"O"
},
{
"id":20,
"orth":"GET",
"ner":"O"
},
{
"id":21,
"orth":"FUCKED",
"ner":"O"
},
{
"id":22,
"orth":"UP",
"ner":"O"
},
{
"id":23,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":24,
"orth":" ",
"ner":"O"
},
{
"id":25,
"orth":"I'M",
"ner":"O"
},
{
"id":26,
"orth":"FUCKED",
"ner":"O"
},
{
"id":27,
"orth":"UP",
"ner":"O"
},
{
"id":28,
"orth":"RIGHT",
"ner":"O"
},
{
"id":29,
"orth":"NOW",
"ner":"O"
},
{
"id":30,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":1
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":1
}
]
},
{
"raw":"Hey man, I'm really not trying to edit war. It's just that this guy is constantly removing relevant information and talking to me through edits instead of my talk page. He seems to care more about the formatting than the actual info.",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"Hey",
"ner":"O"
},
{
"id":1,
"orth":"man",
"ner":"O"
},
{
"id":2,
"orth":",",
"ner":"O"
},
{
"id":3,
"orth":"I",
"ner":"O"
},
{
"id":4,
"orth":"'m",
"ner":"O"
},
{
"id":5,
"orth":"really",
"ner":"O"
},
{
"id":6,
"orth":"not",
"ner":"O"
},
{
"id":7,
"orth":"trying",
"ner":"O"
},
{
"id":8,
"orth":"to",
"ner":"O"
},
{
"id":9,
"orth":"edit",
"ner":"O"
},
{
"id":10,
"orth":"war",
"ner":"O"
},
{
"id":11,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":12,
"orth":"It",
"ner":"O"
},
{
"id":13,
"orth":"'s",
"ner":"O"
},
{
"id":14,
"orth":"just",
"ner":"O"
},
{
"id":15,
"orth":"that",
"ner":"O"
},
{
"id":16,
"orth":"this",
"ner":"O"
},
{
"id":17,
"orth":"guy",
"ner":"O"
},
{
"id":18,
"orth":"is",
"ner":"O"
},
{
"id":19,
"orth":"constantly",
"ner":"O"
},
{
"id":20,
"orth":"removing",
"ner":"O"
},
{
"id":21,
"orth":"relevant",
"ner":"O"
},
{
"id":22,
"orth":"information",
"ner":"O"
},
{
"id":23,
"orth":"and",
"ner":"O"
},
{
"id":24,
"orth":"talking",
"ner":"O"
},
{
"id":25,
"orth":"to",
"ner":"O"
},
{
"id":26,
"orth":"me",
"ner":"O"
},
{
"id":27,
"orth":"through",
"ner":"O"
},
{
"id":28,
"orth":"edits",
"ner":"O"
},
{
"id":29,
"orth":"instead",
"ner":"O"
},
{
"id":30,
"orth":"of",
"ner":"O"
},
{
"id":31,
"orth":"my",
"ner":"O"
},
{
"id":32,
"orth":"talk",
"ner":"O"
},
{
"id":33,
"orth":"page",
"ner":"O"
},
{
"id":34,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":35,
"orth":"He",
"ner":"O"
},
{
"id":36,
"orth":"seems",
"ner":"O"
},
{
"id":37,
"orth":"to",
"ner":"O"
},
{
"id":38,
"orth":"care",
"ner":"O"
},
{
"id":39,
"orth":"more",
"ner":"O"
},
{
"id":40,
"orth":"about",
"ner":"O"
},
{
"id":41,
"orth":"the",
"ner":"O"
},
{
"id":42,
"orth":"formatting",
"ner":"O"
},
{
"id":43,
"orth":"than",
"ner":"O"
},
{
"id":44,
"orth":"the",
"ner":"O"
},
{
"id":45,
"orth":"actual",
"ner":"O"
},
{
"id":46,
"orth":"info",
"ner":"O"
},
{
"id":47,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"\"\nMore\nI can't make any real suggestions on improvement - I wondered if the section statistics should be later on, or a subsection of \"\"types of accidents\"\" -I think the references may need tidying so that they are all in the exact same format ie date format etc. I can do that later on, if no-one else does first - if you have any preferences for formatting style on references or want to do it yourself please let me know.\n\nThere appears to be a backlog on articles for review so I guess there may be a delay until a reviewer turns up. It's listed in the relevant form eg Wikipedia:Good_article_nominations#Transport \"",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"\"",
"ner":"O"
},
{
"id":1,
"orth":"\n",
"ner":"O"
},
{
"id":2,
"orth":"More",
"ner":"O"
},
{
"id":3,
"orth":"\n",
"ner":"O"
},
{
"id":4,
"orth":"I",
"ner":"O"
},
{
"id":5,
"orth":"ca",
"ner":"O"
},
{
"id":6,
"orth":"n't",
"ner":"O"
},
{
"id":7,
"orth":"make",
"ner":"O"
},
{
"id":8,
"orth":"any",
"ner":"O"
},
{
"id":9,
"orth":"real",
"ner":"O"
},
{
"id":10,
"orth":"suggestions",
"ner":"O"
},
{
"id":11,
"orth":"on",
"ner":"O"
},
{
"id":12,
"orth":"improvement",
"ner":"O"
},
{
"id":13,
"orth":"-",
"ner":"O"
},
{
"id":14,
"orth":"I",
"ner":"O"
},
{
"id":15,
"orth":"wondered",
"ner":"O"
},
{
"id":16,
"orth":"if",
"ner":"O"
},
{
"id":17,
"orth":"the",
"ner":"O"
},
{
"id":18,
"orth":"section",
"ner":"O"
},
{
"id":19,
"orth":"statistics",
"ner":"O"
},
{
"id":20,
"orth":"should",
"ner":"O"
},
{
"id":21,
"orth":"be",
"ner":"O"
},
{
"id":22,
"orth":"later",
"ner":"O"
},
{
"id":23,
"orth":"on",
"ner":"O"
},
{
"id":24,
"orth":",",
"ner":"O"
},
{
"id":25,
"orth":"or",
"ner":"O"
},
{
"id":26,
"orth":"a",
"ner":"O"
},
{
"id":27,
"orth":"subsection",
"ner":"O"
},
{
"id":28,
"orth":"of",
"ner":"O"
},
{
"id":29,
"orth":"\"",
"ner":"O"
},
{
"id":30,
"orth":"\"",
"ner":"O"
},
{
"id":31,
"orth":"types",
"ner":"O"
},
{
"id":32,
"orth":"of",
"ner":"O"
},
{
"id":33,
"orth":"accidents",
"ner":"O"
},
{
"id":34,
"orth":"\"",
"ner":"O"
},
{
"id":35,
"orth":"\"",
"ner":"O"
},
{
"id":36,
"orth":" ",
"ner":"O"
},
{
"id":37,
"orth":"-I",
"ner":"O"
},
{
"id":38,
"orth":"think",
"ner":"O"
},
{
"id":39,
"orth":"the",
"ner":"O"
},
{
"id":40,
"orth":"references",
"ner":"O"
},
{
"id":41,
"orth":"may",
"ner":"O"
},
{
"id":42,
"orth":"need",
"ner":"O"
},
{
"id":43,
"orth":"tidying",
"ner":"O"
},
{
"id":44,
"orth":"so",
"ner":"O"
},
{
"id":45,
"orth":"that",
"ner":"O"
},
{
"id":46,
"orth":"they",
"ner":"O"
},
{
"id":47,
"orth":"are",
"ner":"O"
},
{
"id":48,
"orth":"all",
"ner":"O"
},
{
"id":49,
"orth":"in",
"ner":"O"
},
{
"id":50,
"orth":"the",
"ner":"O"
},
{
"id":51,
"orth":"exact",
"ner":"O"
},
{
"id":52,
"orth":"same",
"ner":"O"
},
{
"id":53,
"orth":"format",
"ner":"O"
},
{
"id":54,
"orth":"ie",
"ner":"O"
},
{
"id":55,
"orth":"date",
"ner":"O"
},
{
"id":56,
"orth":"format",
"ner":"O"
},
{
"id":57,
"orth":"etc",
"ner":"O"
},
{
"id":58,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":59,
"orth":"I",
"ner":"O"
},
{
"id":60,
"orth":"can",
"ner":"O"
},
{
"id":61,
"orth":"do",
"ner":"O"
},
{
"id":62,
"orth":"that",
"ner":"O"
},
{
"id":63,
"orth":"later",
"ner":"O"
},
{
"id":64,
"orth":"on",
"ner":"O"
},
{
"id":65,
"orth":",",
"ner":"O"
},
{
"id":66,
"orth":"if",
"ner":"O"
},
{
"id":67,
"orth":"no",
"ner":"O"
},
{
"id":68,
"orth":"-",
"ner":"O"
},
{
"id":69,
"orth":"one",
"ner":"O"
},
{
"id":70,
"orth":"else",
"ner":"O"
},
{
"id":71,
"orth":"does",
"ner":"O"
},
{
"id":72,
"orth":"first",
"ner":"O"
},
{
"id":73,
"orth":"-",
"ner":"O"
},
{
"id":74,
"orth":"if",
"ner":"O"
},
{
"id":75,
"orth":"you",
"ner":"O"
},
{
"id":76,
"orth":"have",
"ner":"O"
},
{
"id":77,
"orth":"any",
"ner":"O"
},
{
"id":78,
"orth":"preferences",
"ner":"O"
},
{
"id":79,
"orth":"for",
"ner":"O"
},
{
"id":80,
"orth":"formatting",
"ner":"O"
},
{
"id":81,
"orth":"style",
"ner":"O"
},
{
"id":82,
"orth":"on",
"ner":"O"
},
{
"id":83,
"orth":"references",
"ner":"O"
},
{
"id":84,
"orth":"or",
"ner":"O"
},
{
"id":85,
"orth":"want",
"ner":"O"
},
{
"id":86,
"orth":"to",
"ner":"O"
},
{
"id":87,
"orth":"do",
"ner":"O"
},
{
"id":88,
"orth":"it",
"ner":"O"
},
{
"id":89,
"orth":"yourself",
"ner":"O"
},
{
"id":90,
"orth":"please",
"ner":"O"
},
{
"id":91,
"orth":"let",
"ner":"O"
},
{
"id":92,
"orth":"me",
"ner":"O"
},
{
"id":93,
"orth":"know",
"ner":"O"
},
{
"id":94,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":95,
"orth":"\n\n",
"ner":"O"
},
{
"id":96,
"orth":"There",
"ner":"O"
},
{
"id":97,
"orth":"appears",
"ner":"O"
},
{
"id":98,
"orth":"to",
"ner":"O"
},
{
"id":99,
"orth":"be",
"ner":"O"
},
{
"id":100,
"orth":"a",
"ner":"O"
},
{
"id":101,
"orth":"backlog",
"ner":"O"
},
{
"id":102,
"orth":"on",
"ner":"O"
},
{
"id":103,
"orth":"articles",
"ner":"O"
},
{
"id":104,
"orth":"for",
"ner":"O"
},
{
"id":105,
"orth":"review",
"ner":"O"
},
{
"id":106,
"orth":"so",
"ner":"O"
},
{
"id":107,
"orth":"I",
"ner":"O"
},
{
"id":108,
"orth":"guess",
"ner":"O"
},
{
"id":109,
"orth":"there",
"ner":"O"
},
{
"id":110,
"orth":"may",
"ner":"O"
},
{
"id":111,
"orth":"be",
"ner":"O"
},
{
"id":112,
"orth":"a",
"ner":"O"
},
{
"id":113,
"orth":"delay",
"ner":"O"
},
{
"id":114,
"orth":"until",
"ner":"O"
},
{
"id":115,
"orth":"a",
"ner":"O"
},
{
"id":116,
"orth":"reviewer",
"ner":"O"
},
{
"id":117,
"orth":"turns",
"ner":"O"
},
{
"id":118,
"orth":"up",
"ner":"O"
},
{
"id":119,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":120,
"orth":"It",
"ner":"O"
},
{
"id":121,
"orth":"'s",
"ner":"O"
},
{
"id":122,
"orth":"listed",
"ner":"O"
},
{
"id":123,
"orth":"in",
"ner":"O"
},
{
"id":124,
"orth":"the",
"ner":"O"
},
{
"id":125,
"orth":"relevant",
"ner":"O"
},
{
"id":126,
"orth":"form",
"ner":"O"
},
{
"id":127,
"orth":"eg",
"ner":"O"
},
{
"id":128,
"orth":"Wikipedia",
"ner":"O"
},
{
"id":129,
"orth":":",
"ner":"O"
},
{
"id":130,
"orth":"Good_article_nominations#Transport",
"ner":"O"
},
{
"id":131,
"orth":" ",
"ner":"O"
},
{
"id":132,
"orth":"\"",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"You, sir, are my hero. Any chance you remember what page that's on?",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"You",
"ner":"O"
},
{
"id":1,
"orth":",",
"ner":"O"
},
{
"id":2,
"orth":"sir",
"ner":"O"
},
{
"id":3,
"orth":",",
"ner":"O"
},
{
"id":4,
"orth":"are",
"ner":"O"
},
{
"id":5,
"orth":"my",
"ner":"O"
},
{
"id":6,
"orth":"hero",
"ner":"O"
},
{
"id":7,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":8,
"orth":"Any",
"ner":"O"
},
{
"id":9,
"orth":"chance",
"ner":"O"
},
{
"id":10,
"orth":"you",
"ner":"O"
},
{
"id":11,
"orth":"remember",
"ner":"O"
},
{
"id":12,
"orth":"what",
"ner":"O"
},
{
"id":13,
"orth":"page",
"ner":"O"
},
{
"id":14,
"orth":"that",
"ner":"O"
},
{
"id":15,
"orth":"'s",
"ner":"O"
},
{
"id":16,
"orth":"on",
"ner":"O"
},
{
"id":17,
"orth":"?",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"\"\n\nCongratulations from me as well, use the tools well. \u00a0\u00b7 talk \"",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"\"",
"ner":"O"
},
{
"id":1,
"orth":"\n\n",
"ner":"O"
},
{
"id":2,
"orth":"Congratulations",
"ner":"O"
},
{
"id":3,
"orth":"from",
"ner":"O"
},
{
"id":4,
"orth":"me",
"ner":"O"
},
{
"id":5,
"orth":"as",
"ner":"O"
},
{
"id":6,
"orth":"well",
"ner":"O"
},
{
"id":7,
"orth":",",
"ner":"O"
},
{
"id":8,
"orth":"use",
"ner":"O"
},
{
"id":9,
"orth":"the",
"ner":"O"
},
{
"id":10,
"orth":"tools",
"ner":"O"
},
{
"id":11,
"orth":"well",
"ner":"O"
},
{
"id":12,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":13,
"orth":"\u00a0",
"ner":"O"
},
{
"id":14,
"orth":"\u00b7",
"ner":"O"
},
{
"id":15,
"orth":"talk",
"ner":"O"
},
{
"id":16,
"orth":"\"",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":0
}
]
},
{
"raw":"Why can't you believe how fat Artie is? Did you see him on his recent appearence on the Tonight Show with Jay Leno? He looks absolutely AWFUL! If I had to put money on it, I'd say that Artie Lange is a can't miss candidate for the 2007 Dead pool! \n\n \nKindly keep your malicious fingers off of my above comment, . Everytime you remove it, I will repost it!!!",
"sentences":[
{
"tokens":[
{
"id":0,
"orth":"Why",
"ner":"O"
},
{
"id":1,
"orth":"ca",
"ner":"O"
},
{
"id":2,
"orth":"n't",
"ner":"O"
},
{
"id":3,
"orth":"you",
"ner":"O"
},
{
"id":4,
"orth":"believe",
"ner":"O"
},
{
"id":5,
"orth":"how",
"ner":"O"
},
{
"id":6,
"orth":"fat",
"ner":"O"
},
{
"id":7,
"orth":"Artie",
"ner":"O"
},
{
"id":8,
"orth":"is",
"ner":"O"
},
{
"id":9,
"orth":"?",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":10,
"orth":"Did",
"ner":"O"
},
{
"id":11,
"orth":"you",
"ner":"O"
},
{
"id":12,
"orth":"see",
"ner":"O"
},
{
"id":13,
"orth":"him",
"ner":"O"
},
{
"id":14,
"orth":"on",
"ner":"O"
},
{
"id":15,
"orth":"his",
"ner":"O"
},
{
"id":16,
"orth":"recent",
"ner":"O"
},
{
"id":17,
"orth":"appearence",
"ner":"O"
},
{
"id":18,
"orth":"on",
"ner":"O"
},
{
"id":19,
"orth":"the",
"ner":"O"
},
{
"id":20,
"orth":"Tonight",
"ner":"O"
},
{
"id":21,
"orth":"Show",
"ner":"O"
},
{
"id":22,
"orth":"with",
"ner":"O"
},
{
"id":23,
"orth":"Jay",
"ner":"O"
},
{
"id":24,
"orth":"Leno",
"ner":"O"
},
{
"id":25,
"orth":"?",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":26,
"orth":"He",
"ner":"O"
},
{
"id":27,
"orth":"looks",
"ner":"O"
},
{
"id":28,
"orth":"absolutely",
"ner":"O"
},
{
"id":29,
"orth":"AWFUL",
"ner":"O"
},
{
"id":30,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":31,
"orth":"If",
"ner":"O"
},
{
"id":32,
"orth":"I",
"ner":"O"
},
{
"id":33,
"orth":"had",
"ner":"O"
},
{
"id":34,
"orth":"to",
"ner":"O"
},
{
"id":35,
"orth":"put",
"ner":"O"
},
{
"id":36,
"orth":"money",
"ner":"O"
},
{
"id":37,
"orth":"on",
"ner":"O"
},
{
"id":38,
"orth":"it",
"ner":"O"
},
{
"id":39,
"orth":",",
"ner":"O"
},
{
"id":40,
"orth":"I",
"ner":"O"
},
{
"id":41,
"orth":"'d",
"ner":"O"
},
{
"id":42,
"orth":"say",
"ner":"O"
},
{
"id":43,
"orth":"that",
"ner":"O"
},
{
"id":44,
"orth":"Artie",
"ner":"O"
},
{
"id":45,
"orth":"Lange",
"ner":"O"
},
{
"id":46,
"orth":"is",
"ner":"O"
},
{
"id":47,
"orth":"a",
"ner":"O"
},
{
"id":48,
"orth":"ca",
"ner":"O"
},
{
"id":49,
"orth":"n't",
"ner":"O"
},
{
"id":50,
"orth":"miss",
"ner":"O"
},
{
"id":51,
"orth":"candidate",
"ner":"O"
},
{
"id":52,
"orth":"for",
"ner":"O"
},
{
"id":53,
"orth":"the",
"ner":"O"
},
{
"id":54,
"orth":"2007",
"ner":"O"
},
{
"id":55,
"orth":"Dead",
"ner":"O"
},
{
"id":56,
"orth":"pool",
"ner":"O"
},
{
"id":57,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":58,
"orth":" \n\n \n",
"ner":"O"
},
{
"id":59,
"orth":"Kindly",
"ner":"O"
},
{
"id":60,
"orth":"keep",
"ner":"O"
},
{
"id":61,
"orth":"your",
"ner":"O"
},
{
"id":62,
"orth":"malicious",
"ner":"O"
},
{
"id":63,
"orth":"fingers",
"ner":"O"
},
{
"id":64,
"orth":"off",
"ner":"O"
},
{
"id":65,
"orth":"of",
"ner":"O"
},
{
"id":66,
"orth":"my",
"ner":"O"
},
{
"id":67,
"orth":"above",
"ner":"O"
},
{
"id":68,
"orth":"comment",
"ner":"O"
},
{
"id":69,
"orth":",",
"ner":"O"
},
{
"id":70,
"orth":".",
"ner":"O"
}
],
"brackets":[
]
},
{
"tokens":[
{
"id":71,
"orth":"Everytime",
"ner":"O"
},
{
"id":72,
"orth":"you",
"ner":"O"
},
{
"id":73,
"orth":"remove",
"ner":"O"
},
{
"id":74,
"orth":"it",
"ner":"O"
},
{
"id":75,
"orth":",",
"ner":"O"
},
{
"id":76,
"orth":"I",
"ner":"O"
},
{
"id":77,
"orth":"will",
"ner":"O"
},
{
"id":78,
"orth":"repost",
"ner":"O"
},
{
"id":79,
"orth":"it",
"ner":"O"
},
{
"id":80,
"orth":"!",
"ner":"O"
},
{
"id":81,
"orth":"!",
"ner":"O"
},
{
"id":82,
"orth":"!",
"ner":"O"
}
],
"brackets":[
]
}
],
"cats":[
{
"label":"insult",
"value":0
},
{
"label":"obscene",
"value":0
},
{
"label":"severe_toxic",
"value":0
},
{
"label":"toxic",
"value":1
}
]
}
]
}
]
Reference in New Issue View Git Blame Copy Permalink