From def7013eec784347bdc37060f1a04f52f1d708e0 Mon Sep 17 00:00:00 2001 From: Sofie Van Landeghem Date: Fri, 8 Sep 2023 10:25:14 +0200 Subject: [PATCH] Docs for spacy-llm 0.5.0 (#12968) * Update incorrect example config. (#12893) * spacy-llm docs cleanup (#12945) * Shorten NER section * fix template references * simplify sections * set temperature to 0.0 in examples * condense model information * fix parameters for REST models * set temperature to 0.0 * spelling fix * trigger preview * fix quotes * add small note on noop.v1 * move up example noop config * set appropriate model example configs * explain config * fix Co-authored-by: Raphael Mitsch --------- Co-authored-by: Raphael Mitsch * Docs for ner.v3 and spancat.v3 spacy-llm tasks (#12949) * formatting * update usage table with NER.v3 * fix typo in links * v3 overview of parameters * add spancat.v3 * add further v3 explanations * remove TODO comment * few more small fixes * Add doc section on LLM + task factories (#12905) * Add section on LLM + task factories. * Apply suggestions from code review --------- Co-authored-by: Sofie Van Landeghem * add default config to openai models (#12961) * Docs for spacy-llm 0.5.0 (#12967) * simplify Python example * simplify Python example * Refer only to latest OpenAI model versions from usage doc * Typo fix Co-authored-by: Raphael Mitsch * clarify accuracy claim --------- Co-authored-by: Raphael Mitsch --------- Co-authored-by: Raphael Mitsch --- website/docs/api/large-language-models.mdx | 1384 +++++++----------- website/docs/usage/large-language-models.mdx | 117 +- 2 files changed, 606 insertions(+), 895 deletions(-) diff --git a/website/docs/api/large-language-models.mdx b/website/docs/api/large-language-models.mdx index e65945357..1ac9b0cef 100644 --- a/website/docs/api/large-language-models.mdx +++ b/website/docs/api/large-language-models.mdx @@ -2,7 +2,7 @@ title: Large Language Models teaser: Integrating LLMs into structured NLP pipelines menu: - - ['Config', 'config'] + - ['Config and implementation', 'config'] - ['Tasks', 'tasks'] - ['Models', 'models'] - ['Cache', 'cache'] @@ -14,45 +14,200 @@ Language Models (LLMs) into spaCy, featuring a modular system for **fast prototyping** and **prompting**, and turning unstructured responses into **robust outputs** for various NLP tasks, **no training data** required. -## Config {id="config"} +## Config and implementation {id="config"} -`spacy-llm` exposes a `llm` factory that accepts the following configuration -options: +An LLM component is implemented through the `LLMWrapper` class. It is accessible +through a generic `llm` +[component factory](https://spacy.io/usage/processing-pipelines#custom-components-factories) +as well as through task-specific component factories: -| Argument | Description | -| ---------------- | ------------------------------------------------------------------------------------------------------- | -| `task` | An LLMTask can generate prompts and parse LLM responses. See [docs](#tasks). ~~Optional[LLMTask]~~ | -| `model` | Callable querying a specific LLM API. See [docs](#models). ~~Callable[[Iterable[Any]], Iterable[Any]]~~ | -| `cache` | Cache to use for caching prompts and responses per doc (batch). See [docs](#cache). ~~Cache~~ | -| `save_io` | Whether to save prompts/responses within `Doc.user_data["llm_io"]`. ~~bool~~ | -| `validate_types` | Whether to check if signatures of configured model and task are consistent. ~~bool~~ | +- `llm_ner` +- `llm_spancat` +- `llm_rel` +- `llm_textcat` +- `llm_sentiment` +- `llm_summarization` -An `llm` component is defined by two main settings: +### LLMWrapper.\_\_init\_\_ {id="init",tag="method"} -- A [**task**](#tasks), defining the prompt to send to the LLM as well as the - functionality to parse the resulting response back into structured fields on - the [Doc](/api/doc) objects. -- A [**model**](#models) defining the model and how to connect to it. Note that - `spacy-llm` supports both access to external APIs (such as OpenAI) as well as - access to self-hosted open-source LLMs (such as using Dolly through Hugging - Face). +> #### Example +> +> ```python +> # Construction via add_pipe with default GPT3.5 model and NER task +> config = {"task": {"@llm_tasks": "spacy.NER.v3", "labels": ["PERSON", "ORGANISATION", "LOCATION"]}} +> llm = nlp.add_pipe("llm") +> +> # Construction via add_pipe with task-specific factory and default GPT3.5 model +> parser = nlp.add_pipe("llm-ner", config=config) +> +> # Construction from class +> from spacy_llm.pipeline import LLMWrapper +> llm = LLMWrapper(vocab=nlp.vocab, task=task, model=model, cache=cache, save_io=True) +> ``` -Moreover, `spacy-llm` exposes a customizable [**caching**](#cache) functionality -to avoid running the same document through an LLM service (be it local or -through a REST API) more than once. +Create a new pipeline instance. In your application, you would normally use a +shortcut for this and instantiate the component using its string name and +[`nlp.add_pipe`](/api/language#add_pipe). -Finally, you can choose to save a stringified version of LLM prompts/responses -within the `Doc.user_data["llm_io"]` attribute by setting `save_io` to `True`. -`Doc.user_data["llm_io"]` is a dictionary containing one entry for every LLM -component within the `nlp` pipeline. Each entry is itself a dictionary, with two -keys: `prompt` and `response`. +| Name | Description | +| -------------- | -------------------------------------------------------------------------------------------------- | +| `name` | String name of the component instance. `llm` by default. ~~str~~ | +| _keyword-only_ | | +| `vocab` | The shared vocabulary. ~~Vocab~~ | +| `task` | An [LLM Task](#tasks) can generate prompts and parse LLM responses. ~~LLMTask~~ | +| `model` | The [LLM Model](#models) queries a specific LLM API.. ~~Callable[[Iterable[Any]], Iterable[Any]]~~ | +| `cache` | [Cache](#cache) to use for caching prompts and responses per doc. ~~Cache~~ | +| `save_io` | Whether to save LLM I/O (prompts and responses) in the `Doc._.llm_io` custom attribute. ~~bool~~ | -A note on `validate_types`: by default, `spacy-llm` checks whether the -signatures of the `model` and `task` callables are consistent with each other -and emits a warning if they don't. `validate_types` can be set to `False` if you -want to disable this behavior. +### LLMWrapper.\_\_call\_\_ {id="call",tag="method"} -### Tasks {id="tasks"} +Apply the pipe to one document. The document is modified in place and returned. +This usually happens under the hood when the `nlp` object is called on a text +and all pipeline components are applied to the `Doc` in order. + +> #### Example +> +> ```python +> doc = nlp("Ingrid visited Paris.") +> llm_ner = nlp.add_pipe("llm_ner") +> # This usually happens under the hood +> processed = llm_ner(doc) +> ``` + +| Name | Description | +| ----------- | -------------------------------- | +| `doc` | The document to process. ~~Doc~~ | +| **RETURNS** | The processed document. ~~Doc~~ | + +### LLMWrapper.pipe {id="pipe",tag="method"} + +Apply the pipe to a stream of documents. This usually happens under the hood +when the `nlp` object is called on a text and all pipeline components are +applied to the `Doc` in order. + +> #### Example +> +> ```python +> llm_ner = nlp.add_pipe("llm_ner") +> for doc in llm_ner.pipe(docs, batch_size=50): +> pass +> ``` + +| Name | Description | +| -------------- | ------------------------------------------------------------- | +| `docs` | A stream of documents. ~~Iterable[Doc]~~ | +| _keyword-only_ | | +| `batch_size` | The number of documents to buffer. Defaults to `128`. ~~int~~ | +| **YIELDS** | The processed documents in order. ~~Doc~~ | + +### LLMWrapper.add_label {id="add_label",tag="method"} + +Add a new label to the pipe's task. Alternatively, provide the labels upon the +[task](#task) definition, or through the `[initialize]` block of the +[config](#config). + +> #### Example +> +> ```python +> llm_ner = nlp.add_pipe("llm_ner") +> llm_ner.add_label("MY_LABEL") +> ``` + +| Name | Description | +| ----------- | ----------------------------------------------------------- | +| `label` | The label to add. ~~str~~ | +| **RETURNS** | `0` if the label is already present, otherwise `1`. ~~int~~ | + +### LLMWrapper.to_disk {id="to_disk",tag="method"} + +Serialize the pipe to disk. + +> #### Example +> +> ```python +> llm_ner = nlp.add_pipe("llm_ner") +> llm_ner.to_disk("/path/to/llm_ner") +> ``` + +| Name | Description | +| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `path` | A path to a directory, which will be created if it doesn't exist. Paths may be either strings or `Path`-like objects. ~~Union[str, Path]~~ | +| _keyword-only_ | | +| `exclude` | String names of [serialization fields](#serialization-fields) to exclude. ~~Iterable[str]~~ | + +### LLMWrapper.from_disk {id="from_disk",tag="method"} + +Load the pipe from disk. Modifies the object in place and returns it. + +> #### Example +> +> ```python +> llm_ner = nlp.add_pipe("llm_ner") +> llm_ner.from_disk("/path/to/llm_ner") +> ``` + +| Name | Description | +| -------------- | ----------------------------------------------------------------------------------------------- | +| `path` | A path to a directory. Paths may be either strings or `Path`-like objects. ~~Union[str, Path]~~ | +| _keyword-only_ | | +| `exclude` | String names of [serialization fields](#serialization-fields) to exclude. ~~Iterable[str]~~ | +| **RETURNS** | The modified `LLMWrapper` object. ~~LLMWrapper~~ | + +### LLMWrapper.to_bytes {id="to_bytes",tag="method"} + +> #### Example +> +> ```python +> llm_ner = nlp.add_pipe("llm_ner") +> ner_bytes = llm_ner.to_bytes() +> ``` + +Serialize the pipe to a bytestring. + +| Name | Description | +| -------------- | ------------------------------------------------------------------------------------------- | +| _keyword-only_ | | +| `exclude` | String names of [serialization fields](#serialization-fields) to exclude. ~~Iterable[str]~~ | +| **RETURNS** | The serialized form of the `LLMWrapper` object. ~~bytes~~ | + +### LLMWrapper.from_bytes {id="from_bytes",tag="method"} + +Load the pipe from a bytestring. Modifies the object in place and returns it. + +> #### Example +> +> ```python +> ner_bytes = llm_ner.to_bytes() +> llm_ner = nlp.add_pipe("llm_ner") +> llm_ner.from_bytes(ner_bytes) +> ``` + +| Name | Description | +| -------------- | ------------------------------------------------------------------------------------------- | +| `bytes_data` | The data to load from. ~~bytes~~ | +| _keyword-only_ | | +| `exclude` | String names of [serialization fields](#serialization-fields) to exclude. ~~Iterable[str]~~ | +| **RETURNS** | The `LLMWrapper` object. ~~LLMWrapper~~ | + +### LLMWrapper.labels {id="labels",tag="property"} + +The labels currently added to the component. Empty tuple if the LLM's task does +not require labels. + +> #### Example +> +> ```python +> llm_ner.add_label("MY_LABEL") +> assert "MY_LABEL" in llm_ner.labels +> ``` + +| Name | Description | +| ----------- | ------------------------------------------------------ | +| **RETURNS** | The labels added to the component. ~~Tuple[str, ...]~~ | + +## Tasks {id="tasks"} + +### Task implementation {id="task-implementation"} A _task_ defines an NLP problem or question, that will be sent to the LLM via a prompt. Further, the task defines how to parse the LLM's responses back into @@ -86,6 +241,11 @@ objects. This depends on the return type of the [model](#models). | `responses` | The generated prompts. ~~Iterable[Any]~~ | | **RETURNS** | The annotated documents. ~~Iterable[Doc]~~ | +### Summarization {id="summarization"} + +A summarization task takes a document as input and generates a summary that is +stored in an extension attribute. + #### spacy.Summarization.v1 {id="summarization-v1"} The `spacy.Summarization.v1` task supports both zero-shot and few-shot @@ -100,12 +260,12 @@ prompting. > max_n_words = null > ``` -| Argument | Description | -| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [summarization.jinja](./spacy_llm/tasks/templates/summarization.jinja). ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `max_n_words` | Maximum number of words to be used in summary. Note that this should not expected to work exactly. Defaults to `None`. ~~Optional[int]~~ | -| `field` | Name of extension attribute to store summary in (i. e. the summary will be available in `doc._.{field}`). Defaults to `summary`. ~~str~~ | +| Argument | Description | +| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `template` | Custom prompt template to send to LLM model. Defaults to [summarization.v1.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/summarization.v1.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `max_n_words` | Maximum number of words to be used in summary. Note that this should not expected to work exactly. Defaults to `None`. ~~Optional[int]~~ | +| `field` | Name of extension attribute to store summary in (i. e. the summary will be available in `doc._.{field}`). Defaults to `summary`. ~~str~~ | The summarization task prompts the model for a concise summary of the provided text. It optionally allows to limit the response to a certain number of tokens - @@ -146,11 +306,111 @@ max_n_words = 20 path = "summarization_examples.yml" ``` +### NER {id="ner"} + +The NER task identifies non-overlapping entities in text. + +#### spacy.NER.v3 {id="ner-v3"} + +Version 3 is fundamentally different to v1 and v2, as it implements +Chain-of-Thought prompting, based on the +[PromptNER paper](https://arxiv.org/pdf/2305.15444.pdf) by Ashok and Lipton +(2023). On an internal use-case, we have found this implementation to obtain +significant better accuracy - with an increase of F-score of up to 15 percentage +points. + +> #### Example config +> +> ```ini +> [components.llm.task] +> @llm_tasks = "spacy.NER.v3" +> labels = ["PERSON", "ORGANISATION", "LOCATION"] +> ``` + +When no examples are [specified](/usage/large-language-models#few-shot-prompts), +the v3 implementation will use a dummy example in the prompt. Technically this +means that the task will always perform few-shot prompting under the hood. + +| Argument | Description | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `label_definitions` | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `template` | Custom prompt template to send to LLM model. Defaults to [ner.v3.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/ner.v3.jinja). ~~str~~ | +| `description` (NEW) | A description of what to recognize or not recognize as entities. ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | +| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | +| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | + +Note that the `single_match` parameter, used in v1 and v2, is not supported +anymore, as the CoT parsing algorithm takes care of this automatically. + +New to v3 is the fact that you can provide an explicit description of what +entities should look like. You can use this feature in addition to +`label_definitions`. + +```ini +[components.llm.task] +@llm_tasks = "spacy.NER.v3" +labels = ["DISH", "INGREDIENT", "EQUIPMENT"] +description = Entities are the names food dishes, + ingredients, and any kind of cooking equipment. + Adjectives, verbs, adverbs are not entities. + Pronouns are not entities. + +[components.llm.task.label_definitions] +DISH = "Known food dishes, e.g. Lobster Ravioli, garlic bread" +INGREDIENT = "Individual parts of a food dish, including herbs and spices." +EQUIPMENT = "Any kind of cooking equipment. e.g. oven, cooking pot, grill" +``` + +To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), +you can write down a few examples in a separate file, and provide these to be +injected into the prompt to the LLM. The default reader `spacy.FewShotReader.v1` +supports `.yml`, `.yaml`, `.json` and `.jsonl`. + +While not required, this task works best when both positive and negative +examples are provided. The format is different than the files required for v1 +and v2, as additional fields such as `is_entity` and `reason` should now be +provided. + +```json +[ + { + "text": "You can't get a great chocolate flavor with carob.", + "spans": [ + { + "text": "chocolate", + "is_entity": false, + "label": "==NONE==", + "reason": "is a flavor in this context, not an ingredient" + }, + { + "text": "carob", + "is_entity": true, + "label": "INGREDIENT", + "reason": "is an ingredient to add chocolate flavor" + } + ] + }, + ... +] +``` + +```ini +[components.llm.task.examples] +@misc = "spacy.FewShotReader.v1" +path = "${paths.examples}" +``` + +For a fully working example, see this +[usage example](https://github.com/explosion/spacy-llm/tree/main/usage_examples/ner_v3_openai). + #### spacy.NER.v2 {id="ner-v2"} -The built-in NER task supports both zero-shot and few-shot prompting. This -version also supports explicitly defining the provided labels with custom -descriptions. +This version supports explicitly defining the provided labels with custom +descriptions, and further supports zero-shot and few-shot prompting just like +v1. > #### Example config > @@ -161,84 +421,44 @@ descriptions. > examples = null > ``` -| Argument | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [ner.v2.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/ner.v2.jinja). ~~str~~ | -| `label_definitions` | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | -| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | -| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | -| `single_match` | Whether to match an entity in the LLM's response only once (the first hit) or multiple times. Defaults to `False`. ~~bool~~ | +| Argument | Description | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `label_definitions` (NEW) | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `template` (NEW) | Custom prompt template to send to LLM model. Defaults to [ner.v2.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/ner.v2.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | +| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | +| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | +| `single_match` | Whether to match an entity in the LLM's response only once (the first hit) or multiple times. Defaults to `False`. ~~bool~~ | -The NER task implementation doesn't currently ask the LLM for specific offsets, -but simply expects a list of strings that represent the enties in the document. -This means that a form of string matching is required. This can be configured by -the following parameters: - -- The `single_match` parameter is typically set to `False` to allow for multiple - matches. For instance, the response from the LLM might only mention the entity - "Paris" once, but you'd still want to mark it every time it occurs in the - document. -- The case-sensitive matching is typically set to `False` to be robust against - case variances in the LLM's output. -- The `alignment_mode` argument is used to match entities as returned by the LLM - to the tokens from the original `Doc` - specifically it's used as argument in - the call to [`doc.char_span()`](/api/doc#char_span). The `"strict"` mode will - only keep spans that strictly adhere to the given token boundaries. - `"contract"` will only keep those tokens that are fully within the given - range, e.g. reducing `"New Y"` to `"New"`. Finally, `"expand"` will expand the - span to the next token boundaries, e.g. expanding `"New Y"` out to - `"New York"`. - -To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), -you can write down a few examples in a separate file, and provide these to be -injected into the prompt to the LLM. The default reader `spacy.FewShotReader.v1` -supports `.yml`, `.yaml`, `.json` and `.jsonl`. - -```yaml -- text: Jack and Jill went up the hill. - entities: - PERSON: - - Jack - - Jill - LOCATION: - - hill -- text: Jack fell down and broke his crown. - entities: - PERSON: - - Jack -``` - -```ini -[components.llm.task] -@llm_tasks = "spacy.NER.v2" -labels = PERSON,ORGANISATION,LOCATION -[components.llm.task.examples] -@misc = "spacy.FewShotReader.v1" -path = "ner_examples.yml" -``` +The parameters `alignment_mode`, `case_sensitive_matching` and `single_match` +are identical to the [v1](#ner-v1) implementation. The format of few-shot +examples are also the same. > Label descriptions can also be used with explicit examples to give as much > info to the LLM model as possible. -You can also write definitions for each label and provide them via the -`label_definitions` argument. This lets you tell the LLM exactly what you're -looking for rather than relying on the LLM to interpret its task given just the -label name. Label descriptions are freeform so you can write whatever you want -here, but through some experiments a brief description along with some examples -and counter examples seems to work quite well. +New to v2 is the fact that you can write definitions for each label and provide +them via the `label_definitions` argument. This lets you tell the LLM exactly +what you're looking for rather than relying on the LLM to interpret its task +given just the label name. Label descriptions are freeform so you can write +whatever you want here, but a brief description along with some examples and +counter examples seems to work quite well. ```ini [components.llm.task] @llm_tasks = "spacy.NER.v2" labels = PERSON,SPORTS_TEAM + [components.llm.task.label_definitions] PERSON = "Extract any named individual in the text." SPORTS_TEAM = "Extract the names of any professional sports team. e.g. Golden State Warriors, LA Lakers, Man City, Real Madrid" ``` +For a fully working example, see this +[usage example](https://github.com/explosion/spacy-llm/tree/main/usage_examples/ner_dolly). + #### spacy.NER.v1 {id="ner-v1"} The original version of the built-in NER task supports both zero-shot and @@ -302,18 +522,48 @@ supports `.yml`, `.yaml`, `.json` and `.jsonl`. ``` ```ini -[components.llm.task] -@llm_tasks = "spacy.NER.v1" -labels = PERSON,ORGANISATION,LOCATION [components.llm.task.examples] @misc = "spacy.FewShotReader.v1" path = "ner_examples.yml" ``` +### SpanCat {id="spancat"} + +The SpanCat task identifies potentially overlapping entities in text. + +#### spacy.SpanCat.v3 {id="spancat-v3"} + +The built-in SpanCat v3 task is a simple adaptation of the NER v3 task to +support overlapping entities and store its annotations in `doc.spans`. + +> #### Example config +> +> ```ini +> [components.llm.task] +> @llm_tasks = "spacy.SpanCat.v3" +> labels = ["PERSON", "ORGANISATION", "LOCATION"] +> examples = null +> ``` + +| Argument | Description | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `label_definitions` | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `template` | Custom prompt template to send to LLM model. Defaults to [`spancat.v3.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/spancat.v3.jinja). ~~str~~ | +| `description` (NEW) | A description of what to recognize or not recognize as entities. ~~str~~ | +| `spans_key` | Key of the `Doc.spans` dict to save the spans under. Defaults to `"sc"`. ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. ~~Optional[Callable[[str], str]]~~ | +| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | +| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | + +Note that the `single_match` parameter, used in v1 and v2, is not supported +anymore, as the CoT parsing algorithm takes care of this automatically. + #### spacy.SpanCat.v2 {id="spancat-v2"} -The built-in SpanCat task is a simple adaptation of the NER task to support -overlapping entities and store its annotations in `doc.spans`. +The built-in SpanCat v2 task is a simple adaptation of the NER v2 task to +support overlapping entities and store its annotations in `doc.spans`. > #### Example config > @@ -324,20 +574,21 @@ overlapping entities and store its annotations in `doc.spans`. > examples = null > ``` -| Argument | Description | -| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [`spancat.v2.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/spancat.v2.jinja). ~~str~~ | -| `label_definitions` | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | -| `spans_key` | Key of the `Doc.spans` dict to save the spans under. Defaults to `"sc"`. ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. ~~Optional[Callable[[str], str]]~~ | -| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | -| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | -| `single_match` | Whether to match an entity in the LLM's response only once (the first hit) or multiple times. Defaults to `False`. ~~bool~~ | +| Argument | Description | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `label_definitions` (NEW) | Optional dict mapping a label to a description of that label. These descriptions are added to the prompt to help instruct the LLM on what to extract. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `template` (NEW) | Custom prompt template to send to LLM model. Defaults to [`spancat.v2.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/spancat.v2.jinja). ~~str~~ | +| `spans_key` | Key of the `Doc.spans` dict to save the spans under. Defaults to `"sc"`. ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, defaults to `spacy.LowercaseNormalizer.v1`. ~~Optional[Callable[[str], str]]~~ | +| `alignment_mode` | Alignment mode in case the LLM returns entities that do not align with token boundaries. Options are `"strict"`, `"contract"` or `"expand"`. Defaults to `"contract"`. ~~str~~ | +| `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | +| `single_match` | Whether to match an entity in the LLM's response only once (the first hit) or multiple times. Defaults to `False`. ~~bool~~ | -Except for the `spans_key` parameter, the SpanCat task reuses the configuration -from the NER task. Refer to [its documentation](#ner-v2) for more insight. +Except for the `spans_key` parameter, the SpanCat v2 task reuses the +configuration from the NER v2 task. Refer to [its documentation](#ner-v2) for +more insight. #### spacy.SpanCat.v1 {id="spancat-v1"} @@ -364,14 +615,19 @@ v1 NER task to support overlapping entities and store its annotations in | `case_sensitive_matching` | Whether to search without case sensitivity. Defaults to `False`. ~~bool~~ | | `single_match` | Whether to match an entity in the LLM's response only once (the first hit) or multiple times. Defaults to `False`. ~~bool~~ | -Except for the `spans_key` parameter, the SpanCat task reuses the configuration -from the NER task. Refer to [its documentation](#ner-v1) for more insight. +Except for the `spans_key` parameter, the SpanCat v1 task reuses the +configuration from the NER v1 task. Refer to [its documentation](#ner-v1) for +more insight. + +### TextCat {id="textcat"} + +The TextCat task labels documents with relevant categories. #### spacy.TextCat.v3 {id="textcat-v3"} -Version 3 (the most recent) of the built-in TextCat task supports both zero-shot -and few-shot prompting. It allows setting definitions of labels. Those -definitions are included in the prompt. +On top of the functionality from v2, version 3 of the built-in TextCat tasks +allows setting definitions of labels. Those definitions are included in the +prompt. > #### Example config > @@ -379,59 +635,30 @@ definitions are included in the prompt. > [components.llm.task] > @llm_tasks = "spacy.TextCat.v3" > labels = ["COMPLIMENT", "INSULT"] -> label_definitions = { -> "COMPLIMENT": "a polite expression of praise or admiration.", -> "INSULT": "a disrespectful or scornfully abusive remark or act." -> } +> +> [components.llm.task.label_definitions] +> "COMPLIMENT" = "a polite expression of praise or admiration.", +> "INSULT" = "a disrespectful or scornfully abusive remark or act." > examples = null > ``` -| Argument | Description | -| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | -| `label_definitions` | Dictionary of label definitions. Included in the prompt, if set. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [`textcat.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/textcat.jinja). ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | -| `exclusive_classes` | If set to `True`, only one label per document should be valid. If set to `False`, one document can have multiple labels. Defaults to `False`. ~~bool~~ | -| `allow_none` | When set to `True`, allows the LLM to not return any of the given label. The resulting dict in `doc.cats` will have `0.0` scores for all labels. Defaults to `True`. ~~bool~~ | -| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | +| Argument | Description | +| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `label_definitions` (NEW) | Dictionary of label definitions. Included in the prompt, if set. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `template` | Custom prompt template to send to LLM model. Defaults to [`textcat.v3.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/textcat.v3.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | +| `exclusive_classes` | If set to `True`, only one label per document should be valid. If set to `False`, one document can have multiple labels. Defaults to `False`. ~~bool~~ | +| `allow_none` | When set to `True`, allows the LLM to not return any of the given label. The resulting dict in `doc.cats` will have `0.0` scores for all labels. Defaults to `True`. ~~bool~~ | +| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | -To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), -you can write down a few examples in a separate file, and provide these to be -injected into the prompt to the LLM. The default reader `spacy.FewShotReader.v1` -supports `.yml`, `.yaml`, `.json` and `.jsonl`. - -```json -[ - { - "text": "You look great!", - "answer": "Compliment" - }, - { - "text": "You are not very clever at all.", - "answer": "Insult" - } -] -``` - -```ini -[components.llm.task] -@llm_tasks = "spacy.TextCat.v3" -labels = ["COMPLIMENT", "INSULT"] -label_definitions = { - "COMPLIMENT": "a polite expression of praise or admiration.", - "INSULT": "a disrespectful or scornfully abusive remark or act." -} -[components.llm.task.examples] -@misc = "spacy.FewShotReader.v1" -path = "textcat_examples.json" -``` +The formatting of few-shot examples is the same as those for the +[v1](#textcat-v1) implementation. #### spacy.TextCat.v2 {id="textcat-v2"} -Version 2 of the built-in TextCat task supports both zero-shot and few-shot -prompting and includes an improved prompt template. +V2 includes all v1 functionality, with an improved prompt template. > #### Example config > @@ -442,42 +669,18 @@ prompting and includes an improved prompt template. > examples = null > ``` -| Argument | Description | -| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [`textcat.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/textcat.jinja). ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. ~~Optional[Callable[[str], str]]~~ | -| `exclusive_classes` | If set to `True`, only one label per document should be valid. If set to `False`, one document can have multiple labels. Defaults to `False`. ~~bool~~ | -| `allow_none` | When set to `True`, allows the LLM to not return any of the given label. The resulting dict in `doc.cats` will have `0.0` scores for all labels. Defaults to `True`. ~~bool~~ | -| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | +| Argument | Description | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `template` (NEW) | Custom prompt template to send to LLM model. Defaults to [`textcat.v2.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/textcat.v2.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. ~~Optional[Callable[[str], str]]~~ | +| `exclusive_classes` | If set to `True`, only one label per document should be valid. If set to `False`, one document can have multiple labels. Defaults to `False`. ~~bool~~ | +| `allow_none` | When set to `True`, allows the LLM to not return any of the given label. The resulting dict in `doc.cats` will have `0.0` scores for all labels. Defaults to `True`. ~~bool~~ | +| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | -To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), -you can write down a few examples in a separate file, and provide these to be -injected into the prompt to the LLM. The default reader `spacy.FewShotReader.v1` -supports `.yml`, `.yaml`, `.json` and `.jsonl`. - -```json -[ - { - "text": "You look great!", - "answer": "Compliment" - }, - { - "text": "You are not very clever at all.", - "answer": "Insult" - } -] -``` - -```ini -[components.llm.task] -@llm_tasks = "spacy.TextCat.v2" -labels = ["COMPLIMENT", "INSULT"] -[components.llm.task.examples] -@misc = "spacy.FewShotReader.v1" -path = "textcat_examples.json" -``` +The formatting of few-shot examples is the same as those for the +[v1](#textcat-v1) implementation. #### spacy.TextCat.v1 {id="textcat-v1"} @@ -521,14 +724,15 @@ supports `.yml`, `.yaml`, `.json` and `.jsonl`. ``` ```ini -[components.llm.task] -@llm_tasks = "spacy.TextCat.v2" -labels = COMPLIMENT,INSULT [components.llm.task.examples] @misc = "spacy.FewShotReader.v1" path = "textcat_examples.json" ``` +### REL {id="rel"} + +The REL task extracts relations between named entities. + #### spacy.REL.v1 {id="rel-v1"} The built-in REL task supports both zero-shot and few-shot prompting. It relies @@ -542,14 +746,14 @@ on an upstream NER component for entities extraction. > labels = ["LivesIn", "Visits"] > ``` -| Argument | Description | -| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [`rel.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/rel.jinja). ~~str~~ | -| `label_definitions` | Dictionary providing a description for each relation label. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | -| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | +| Argument | Description | +| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `labels` | List of labels or str of comma-separated list of labels. ~~Union[List[str], str]~~ | +| `template` | Custom prompt template to send to LLM model. Defaults to [`rel.v3.jinja`](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/rel.v1.jinja). ~~str~~ | +| `label_definitions` | Dictionary providing a description for each relation label. Defaults to `None`. ~~Optional[Dict[str, str]]~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `normalizer` | Function that normalizes the labels as returned by the LLM. If `None`, falls back to `spacy.LowercaseNormalizer.v1`. Defaults to `None`. ~~Optional[Callable[[str], str]]~~ | +| `verbose` | If set to `True`, warnings will be generated when the LLM returns invalid responses. Defaults to `False`. ~~bool~~ | To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), you can write down a few examples in a separate file, and provide these to be @@ -575,10 +779,17 @@ Note: the REL task relies on pre-extracted entities to make its prediction. Hence, you'll need to add a component that populates `doc.ents` with recognized spans to your spaCy pipeline and put it _before_ the REL component. +For a fully working example, see this +[usage example](https://github.com/explosion/spacy-llm/tree/main/usage_examples/rel_openai). + +### Lemma {id="lemma"} + +The Lemma task lemmatizes the provided text and updates the `lemma_` attribute +in the doc's tokens accordingly. + #### spacy.Lemma.v1 {id="lemma-v1"} -The `Lemma.v1` task lemmatizes the provided text and updates the `lemma_` -attribute in the doc's tokens accordingly. +This task supports both zero-shot and few-shot prompting. > #### Example config > @@ -588,14 +799,14 @@ attribute in the doc's tokens accordingly. > examples = null > ``` -| Argument | Description | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [lemma.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/lemma.jinja). ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| Argument | Description | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `template` | Custom prompt template to send to LLM model. Defaults to [lemma.v1.jinja](https://github.com/explosion/spacy-llm/blob/main/spacy_llm/tasks/templates/lemma.v1.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -`Lemma.v1` prompts the LLM to lemmatize the passed text and return the -lemmatized version as a list of tokens and their corresponding lemma. E. g. the -text `I'm buying ice cream for my friends` should invoke the response +The task prompts the LLM to lemmatize the passed text and return the lemmatized +version as a list of tokens and their corresponding lemma. E. g. the text +`I'm buying ice cream for my friends` should invoke the response ``` I: I @@ -647,12 +858,16 @@ supports `.yml`, `.yaml`, `.json` and `.jsonl`. path = "lemma_examples.yml" ``` -#### spacy.Sentiment.v1 {id="sentiment-v1"} +### Sentiment {id="sentiment"} Performs sentiment analysis on provided texts. Scores between 0 and 1 are stored in `Doc._.sentiment` - the higher, the more positive. Note in cases of parsing issues (e. g. in case of unexpected LLM responses) the value might be `None`. +#### spacy.Sentiment.v1 {id="sentiment-v1"} + +This task supports both zero-shot and few-shot prompting. + > #### Example config > > ```ini @@ -661,11 +876,11 @@ issues (e. g. in case of unexpected LLM responses) the value might be `None`. > examples = null > ``` -| Argument | Description | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `template` | Custom prompt template to send to LLM model. Default templates for each task are located in the `spacy_llm/tasks/templates` directory. Defaults to [sentiment.jinja](./spacy_llm/tasks/templates/sentiment.jinja). ~~str~~ | -| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | -| `field` | Name of extension attribute to store summary in (i. e. the summary will be available in `doc._.{field}`). Defaults to `sentiment`. ~~str~~ | +| Argument | Description | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `template` | Custom prompt template to send to LLM model. Defaults to [sentiment.v1.jinja](./spacy_llm/tasks/templates/sentiment.v1.jinja). ~~str~~ | +| `examples` | Optional function that generates examples for few-shot learning. Defaults to `None`. ~~Optional[Callable[[], Iterable[Any]]]~~ | +| `field` | Name of extension attribute to store summary in (i. e. the summary will be available in `doc._.{field}`). Defaults to `sentiment`. ~~str~~ | To perform [few-shot learning](/usage/large-language-models#few-shot-prompts), you can write down a few examples in a separate file, and provide these to be @@ -691,7 +906,10 @@ supports `.yml`, `.yaml`, `.json` and `.jsonl`. path = "sentiment_examples.yml" ``` -#### spacy.NoOp.v1 {id="noop-v1"} +### NoOp {id="noop"} + +This task is only useful for testing - it tells the LLM to do nothing, and does +not set any fields on the `docs`. > #### Example config > @@ -700,10 +918,11 @@ path = "sentiment_examples.yml" > @llm_tasks = "spacy.NoOp.v1" > ``` -This task is only useful for testing - it tells the LLM to do nothing, and does -not set any fields on the `docs`. +#### spacy.NoOp.v1 {id="noop-v1"} -### Models {id="models"} +This task needs no further configuration. + +## Models {id="models"} A _model_ defines which LLM model to query, and how to query it. It can be a simple function taking a collection of prompts (consistent with the output type @@ -713,6 +932,66 @@ it's a function of type `Callable[[Iterable[Any]], Iterable[Any]]`, but specific implementations can have other signatures, like `Callable[[Iterable[str]], Iterable[str]]`. +### Models via REST API {id="models-rest"} + +These models all take the same parameters, but note that the `config` should +contain provider-specific keys and values, as it will be passed onwards to the +provider's API. + +| Argument | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `name` | Model name, i. e. any supported variant for this particular model. Default depends on the specific model (cf. below) ~~str~~ | +| `config` | Further configuration passed on to the model. Default depends on the specific model (cf. below). ~~Dict[Any, Any]~~ | +| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | +| `max_tries` | Max. number of tries for API request. Defaults to `5`. ~~int~~ | +| `max_request_time` | Max. time (in seconds) to wait for request to terminate before raising an exception. Defaults to `30.0`. ~~float~~ | +| `interval` | Time interval (in seconds) for API retries in seconds. Defaults to `1.0`. ~~float~~ | + +> #### Example config: +> +> ```ini +> [components.llm.model] +> @llm_models = "spacy.GPT-4.v1" +> name = "gpt-4" +> config = {"temperature": 0.0} +> ``` + +| Model | Provider | Supported names | Default name | Default config | +| ----------------------------- | --------- | ---------------------------------------------------------------------------------------- | ---------------------- | ------------------------------------ | +| `spacy.GPT-4.v1` | OpenAI | `["gpt-4", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]` | `"gpt-4"` | `{}` | +| `spacy.GPT-4.v2` | OpenAI | `["gpt-4", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]` | `"gpt-4"` | `{temperature=0.0}` | +| `spacy.GPT-3-5.v1` | OpenAI | `["gpt-3.5-turbo", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-0613-16k"]` | `"gpt-3.5-turbo"` | `{}` | +| `spacy.GPT-3-5.v2` | OpenAI | `["gpt-3.5-turbo", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-0613-16k"]` | `"gpt-3.5-turbo"` | `{temperature=0.0}` | +| `spacy.Davinci.v1` | OpenAI | `["davinci"]` | `"davinci"` | `{}` | +| `spacy.Davinci.v2` | OpenAI | `["davinci"]` | `"davinci"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Text-Davinci.v1` | OpenAI | `["text-davinci-003", "text-davinci-002"]` | `"text-davinci-003"` | `{}` | +| `spacy.Text-Davinci.v2` | OpenAI | `["text-davinci-003", "text-davinci-002"]` | `"text-davinci-003"` | `{temperature=0.0, max_tokens=1000}` | +| `spacy.Code-Davinci.v1` | OpenAI | `["code-davinci-002"]` | `"code-davinci-002"` | `{}` | +| `spacy.Code-Davinci.v2` | OpenAI | `["code-davinci-002"]` | `"code-davinci-002"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Curie.v1` | OpenAI | `["curie"]` | `"curie"` | `{}` | +| `spacy.Curie.v2` | OpenAI | `["curie"]` | `"curie"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Text-Curie.v1` | OpenAI | `["text-curie-001"]` | `"text-curie-001"` | `{}` | +| `spacy.Text-Curie.v2` | OpenAI | `["text-curie-001"]` | `"text-curie-001"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Babbage.v1` | OpenAI | `["babbage"]` | `"babbage"` | `{}` | +| `spacy.Babbage.v2` | OpenAI | `["babbage"]` | `"babbage"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Text-Babbage.v1` | OpenAI | `["text-babbage-001"]` | `"text-babbage-001"` | `{}` | +| `spacy.Text-Babbage.v2` | OpenAI | `["text-babbage-001"]` | `"text-babbage-001"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Ada.v1` | OpenAI | `["ada"]` | `"ada"` | `{}` | +| `spacy.Ada.v2` | OpenAI | `["ada"]` | `"ada"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Text-Ada.v1` | OpenAI | `["text-ada-001"]` | `"text-ada-001"` | `{}` | +| `spacy.Text-Ada.v2` | OpenAI | `["text-ada-001"]` | `"text-ada-001"` | `{temperature=0.0, max_tokens=500}` | +| `spacy.Command.v1` | Cohere | `["command", "command-light", "command-light-nightly", "command-nightly"]` | `"command"` | `{}` | +| `spacy.Claude-2.v1` | Anthropic | `["claude-2", "claude-2-100k"]` | `"claude-2"` | `{}` | +| `spacy.Claude-1.v1` | Anthropic | `["claude-1", "claude-1-100k"]` | `"claude-1"` | `{}` | +| `spacy.Claude-1-0.v1` | Anthropic | `["claude-1.0"]` | `"claude-1.0"` | `{}` | +| `spacy.Claude-1-2.v1` | Anthropic | `["claude-1.2"]` | `"claude-1.2"` | `{}` | +| `spacy.Claude-1-3.v1` | Anthropic | `["claude-1.3", "claude-1.3-100k"]` | `"claude-1.3"` | `{}` | +| `spacy.Claude-instant-1.v1` | Anthropic | `["claude-instant-1", "claude-instant-1-100k"]` | `"claude-instant-1"` | `{}` | +| `spacy.Claude-instant-1-1.v1` | Anthropic | `["claude-instant-1.1", "claude-instant-1.1-100k"]` | `"claude-instant-1.1"` | `{}` | + +To use these models, make sure that you've [set the relevant API](#api-keys) +keys as environment variables. + #### API Keys {id="api-keys"} Note that when using hosted services, you have to ensure that the proper API @@ -727,492 +1006,27 @@ export OPENAI_API_KEY="sk-..." export OPENAI_API_ORG="org-..." ``` -For Cohere it's +For Cohere: ```shell export CO_API_KEY="..." ``` -and for Anthropic +For Anthropic: ```shell export ANTHROPIC_API_KEY="..." ``` -#### spacy.GPT-4.v1 {id="gpt-4"} +### Models via HuggingFace {id="models-hf"} -OpenAI's `gpt-4` model family. +These models all take the same parameters: -> #### Example config: -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.GPT-4.v1" -> name = "gpt-4" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"gpt-4"`. ~~Literal["gpt-4", "gpt-4-0314", "gpt-4-32k", "gpt-4-32k-0314"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.GPT-3-5.v1 {id="gpt-3-5"} - -OpenAI's `gpt-3-5` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.GPT-3-5.v1" -> name = "gpt-3.5-turbo" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"gpt-3.5-turbo"`. ~~Literal["gpt-3.5-turbo", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-0613-16k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Text-Davinci.v1 {id="text-davinci"} - -OpenAI's `text-davinci` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Text-Davinci.v1" -> name = "text-davinci-003" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"text-davinci-003"`. ~~Literal["text-davinci-002", "text-davinci-003"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Code-Davinci.v1 {id="code-davinci"} - -OpenAI's `code-davinci` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Code-Davinci.v1" -> name = "code-davinci-002" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"code-davinci-002"`. ~~Literal["code-davinci-002"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Text-Curie.v1 {id="text-curie"} - -OpenAI's `text-curie` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Text-Curie.v1" -> name = "text-curie-001" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"text-curie-001"`. ~~Literal["text-curie-001"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Text-Babbage.v1 {id="text-babbage"} - -OpenAI's `text-babbage` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Text-Babbage.v1" -> name = "text-babbage-001" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"text-babbage-001"`. ~~Literal["text-babbage-001"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Text-Ada.v1 {id="text-ada"} - -OpenAI's `text-ada` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Text-Ada.v1" -> name = "text-ada-001" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"text-ada-001"`. ~~Literal["text-ada-001"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Davinci.v1 {id="davinci"} - -OpenAI's `davinci` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Davinci.v1" -> name = "davinci" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"davinci"`. ~~Literal["davinci"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Curie.v1 {id="curie"} - -OpenAI's `curie` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Curie.v1" -> name = "curie" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"curie"`. ~~Literal["curie"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Babbage.v1 {id="babbage"} - -OpenAI's `babbage` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Babbage.v1" -> name = "babbage" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"babbage"`. ~~Literal["babbage"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Ada.v1 {id="ada"} - -OpenAI's `ada` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Ada.v1" -> name = "ada" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"ada"`. ~~Literal["ada"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Command.v1 {id="command"} - -Cohere's `command` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Command.v1" -> name = "command" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"command"`. ~~Literal["command", "command-light", "command-light-nightly", "command-nightly"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-2.v1 {id="claude-2"} - -Anthropic's `claude-2` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-2.v1" -> name = "claude-2" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-2"`. ~~Literal["claude-2", "claude-2-100k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-1.v1 {id="claude-1"} - -Anthropic's `claude-1` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-1.v1" -> name = "claude-1" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-1"`. ~~Literal["claude-1", "claude-1-100k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-instant-1.v1 {id="claude-instant-1"} - -Anthropic's `claude-instant-1` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-instant-1.v1" -> name = "claude-instant-1" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-instant-1"`. ~~Literal["claude-instant-1", "claude-instant-1-100k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-instant-1-1.v1 {id="claude-instant-1-1"} - -Anthropic's `claude-instant-1.1` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-instant-1-1.v1" -> name = "claude-instant-1.1" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-instant-1.1"`. ~~Literal["claude-instant-1.1", "claude-instant-1.1-100k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-1-0.v1 {id="claude-1-0"} - -Anthropic's `claude-1.0` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-1-0.v1" -> name = "claude-1.0" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-1.0"`. ~~Literal["claude-1.0"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-1-2.v1 {id="claude-1-2"} - -Anthropic's `claude-1.2` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-1-2.v1 " -> name = "claude-1.2" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-1.2"`. ~~Literal["claude-1.2"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Claude-1-3.v1 {id="claude-1-3"} - -Anthropic's `claude-1.3` model family. - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Claude-1-3.v1" -> name = "claude-1.3" -> config = {"temperature": 0.3} -> ``` - -| Argument | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Model name, i. e. any supported variant for this particular model. Defaults to `"claude-1.3"`. ~~Literal["claude-1.3", "claude-1.3-100k"]~~ | -| `config` | Further configuration passed on to the model. Defaults to `{}`. ~~Dict[Any, Any]~~ | -| `strict` | If `True`, raises an error if the LLM API returns a malformed response. Otherwise, return the error responses as is. Defaults to `True`. ~~bool~~ | -| `max_tries` | Max. number of tries for API request. Defaults to `3`. ~~int~~ | -| `timeout` | Timeout for API request in seconds. Defaults to `30`. ~~int~~ | - -#### spacy.Dolly.v1 {id="dolly"} - -To use this model, ideally you have a GPU enabled and have installed -`transformers`, `torch` and CUDA in your virtual environment. This allows you to -have the setting `device=cuda:0` in your config, which ensures that the model is -loaded entirely on the GPU (and fails otherwise). - -You can do so with - -```shell -python -m pip install "spacy-llm[transformers]" "transformers[sentencepiece]" -``` - -If you don't have access to a GPU, you can install `accelerate` and -set`device_map=auto` instead, but be aware that this may result in some layers -getting distributed to the CPU or even the hard drive, which may ultimately -result in extremely slow queries. - -```shell -python -m pip install "accelerate>=0.16.0,<1.0" -``` - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Dolly.v1" -> name = "dolly-v2-3b" -> ``` - -| Argument | Description | -| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | The name of a Dolly model that is supported (e. g. "dolly-v2-3b" or "dolly-v2-12b"). ~~Literal["dolly-v2-3b", "dolly-v2-7b", "dolly-v2-12b"]~~ | -| `config_init` | Further configuration passed on to the construction of the model with `transformers.pipeline()`. Defaults to `{}`. ~~Dict[str, Any]~~ | -| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | - -Supported models (see the -[Databricks models page](https://huggingface.co/databricks) on Hugging Face for -details): - -- `"databricks/dolly-v2-3b"` -- `"databricks/dolly-v2-7b"` -- `"databricks/dolly-v2-12b"` - -Note that Hugging Face will download this model the first time you use it - you -can -[define the cached directory](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache) -by setting the environmental variable `HF_HOME`. - -#### spacy.Llama2.v1 {id="llama2"} - -To use this model, ideally you have a GPU enabled and have installed -`transformers`, `torch` and CUDA in your virtual environment. This allows you to -have the setting `device=cuda:0` in your config, which ensures that the model is -loaded entirely on the GPU (and fails otherwise). - -You can do so with - -```shell -python -m pip install "spacy-llm[transformers]" "transformers[sentencepiece]" -``` - -If you don't have access to a GPU, you can install `accelerate` and -set`device_map=auto` instead, but be aware that this may result in some layers -getting distributed to the CPU or even the hard drive, which may ultimately -result in extremely slow queries. - -```shell -python -m pip install "accelerate>=0.16.0,<1.0" -``` - -Note that the chat models variants of Llama 2 are currently not supported. This -is because they need a particular prompting setup and don't add any discernible -benefits in the use case of `spacy-llm` (i. e. no interactive chat) compared the -completion model variants. +| Argument | Description | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| `name` | Model name, i. e. any supported variant for this particular model. ~~str~~ | +| `config_init` | Further configuration passed on to the construction of the model with `transformers.pipeline()`. Defaults to `{}`. ~~Dict[str, Any]~~ | +| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | > #### Example config > @@ -1222,108 +1036,27 @@ completion model variants. > name = "llama2-7b-hf" > ``` -| Argument | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `name` | The name of a Llama 2 model variant that is supported. Defaults to `"Llama-2-7b-hf"`. ~~Literal["Llama-2-7b-hf", "Llama-2-13b-hf", "Llama-2-70b-hf"]~~ | -| `config_init` | Further configuration passed on to the construction of the model with `transformers.pipeline()`. Defaults to `{}`. ~~Dict[str, Any]~~ | -| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | +| Model | Provider | Supported names | HF directory | +| -------------------- | --------------- | ------------------------------------------------------------------------------------------------------------ | -------------------------------------- | +| `spacy.Dolly.v1` | Databricks | `["dolly-v2-3b", "dolly-v2-7b", "dolly-v2-12b"]` | https://huggingface.co/databricks | +| `spacy.Llama2.v1` | Meta AI | `["Llama-2-7b-hf", "Llama-2-13b-hf", "Llama-2-70b-hf"]` | https://huggingface.co/meta-llama | +| `spacy.Falcon.v1` | TII | `["falcon-rw-1b", "falcon-7b", "falcon-7b-instruct", "falcon-40b-instruct"]` | https://huggingface.co/tiiuae | +| `spacy.StableLM.v1` | Stability AI | `["stablelm-base-alpha-3b", "stablelm-base-alpha-7b", "stablelm-tuned-alpha-3b", "stablelm-tuned-alpha-7b"]` | https://huggingface.co/stabilityai | +| `spacy.OpenLLaMA.v1` | OpenLM Research | `["open_llama_3b", "open_llama_7b", "open_llama_7b_v2", "open_llama_13b"]` | https://huggingface.co/openlm-research | -Note that Hugging Face will download this model the first time you use it - you -can -[define the cache directory](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache) -by setting the environmental variable `HF_HOME`. +See the "HF directory" for more details on each of the models. -#### spacy.Falcon.v1 {id="falcon"} - -To use this model, ideally you have a GPU enabled and have installed -`transformers`, `torch` and CUDA in your virtual environment. This allows you to -have the setting `device=cuda:0` in your config, which ensures that the model is -loaded entirely on the GPU (and fails otherwise). - -You can do so with - -```shell -python -m pip install "spacy-llm[transformers]" "transformers[sentencepiece]" -``` - -If you don't have access to a GPU, you can install `accelerate` and -set`device_map=auto` instead, but be aware that this may result in some layers -getting distributed to the CPU or even the hard drive, which may ultimately -result in extremely slow queries. - -```shell -python -m pip install "accelerate>=0.16.0,<1.0" -``` - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.Falcon.v1" -> name = "falcon-7b" -> ``` - -| Argument | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `name` | The name of a Falcon model variant that is supported. Defaults to `"7b-instruct"`. ~~Literal["falcon-rw-1b", "falcon-7b", "falcon-7b-instruct", "falcon-40b-instruct"]~~ | -| `config_init` | Further configuration passed on to the construction of the model with `transformers.pipeline()`. Defaults to `{}`. ~~Dict[str, Any]~~ | -| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | - -Note that Hugging Face will download this model the first time you use it - you -can -[define the cache directory](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache) -by setting the environmental variable `HF_HOME`. - -#### spacy.StableLM.v1 {id="stablelm"} - -To use this model, ideally you have a GPU enabled and have installed -`transformers`, `torch` and CUDA in your virtual environment. - -You can do so with - -```shell -python -m pip install "spacy-llm[transformers]" "transformers[sentencepiece]" -``` - -If you don't have access to a GPU, you can install `accelerate` and -set`device_map=auto` instead, but be aware that this may result in some layers -getting distributed to the CPU or even the hard drive, which may ultimately -result in extremely slow queries. - -```shell -python -m pip install "accelerate>=0.16.0,<1.0" -``` - -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.StableLM.v1" -> name = "stablelm-tuned-alpha-7b" -> ``` - -| Argument | Description | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | The name of a StableLM model that is supported (e. g. "stablelm-tuned-alpha-7b"). ~~Literal["stablelm-base-alpha-3b", "stablelm-base-alpha-7b", "stablelm-tuned-alpha-3b", "stablelm-tuned-alpha-7b"]~~ | -| `config_init` | Further configuration passed on to the construction of the model with `transformers.AutoModelForCausalLM.from_pretrained()`. Defaults to `{}`. ~~Dict[str, Any]~~ | -| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | - -See the -[Stability AI StableLM GitHub repo](https://github.com/Stability-AI/StableLM/#stablelm-alpha) -for details. - -Note that Hugging Face will download this model the first time you use it - you +Note that Hugging Face will download the model the first time you use it - you can [define the cached directory](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache) by setting the environmental variable `HF_HOME`. -#### spacy.OpenLLaMA.v1 {id="openllama"} +#### Installation with HuggingFace {id="install-hf"} -To use this model, ideally you have a GPU enabled and have installed - -- `transformers[sentencepiece]` -- `torch` -- CUDA in your virtual environment. +To use models from HuggingFace, ideally you have a GPU enabled and have +installed `transformers`, `torch` and CUDA in your virtual environment. This +allows you to have the setting `device=cuda:0` in your config, which ensures +that the model is loaded entirely on the GPU (and fails otherwise). You can do so with @@ -1340,30 +1073,7 @@ result in extremely slow queries. python -m pip install "accelerate>=0.16.0,<1.0" ``` -> #### Example config -> -> ```ini -> [components.llm.model] -> @llm_models = "spacy.OpenLLaMA.v1" -> name = "open_llama_3b" -> ``` - -| Argument | Description | -| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | The name of a OpenLLaMA model that is supported. ~~Literal["open_llama_3b", "open_llama_7b", "open_llama_7b_v2", "open_llama_13b"]~~ | -| `config_init` | Further configuration passed on to the construction of the model with `transformers.AutoModelForCausalLM.from_pretrained()`. Defaults to `{}`. ~~Dict[str, Any]~~ | -| `config_run` | Further configuration used during model inference. Defaults to `{}`. ~~Dict[str, Any]~~ | - -See the -[OpenLM Research OpenLLaMA GitHub repo](https://github.com/openlm-research/open_llama) -for details. - -Note that Hugging Face will download this model the first time you use it - you -can -[define the cached directory](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache) -by setting the environmental variable `HF_HOME`. - -#### LangChain models {id="langchain-models"} +### LangChain models {id="langchain-models"} To use [LangChain](https://github.com/hwchase17/langchain) for the API retrieval part, make sure you have installed it first: @@ -1392,7 +1102,7 @@ The name of the model to be used has to be passed in via the `name` attribute. > @llm_models = "langchain.OpenAI.v1" > name = "gpt-3.5-turbo" > query = {"@llm_queries": "spacy.CallLangChain.v1"} -> config = {"temperature": 0.3} +> config = {"temperature": 0.0} > ``` | Argument | Description | @@ -1404,7 +1114,7 @@ The name of the model to be used has to be passed in via the `name` attribute. The default `query` (`spacy.CallLangChain.v1`) executes the prompts by running `model(text)` for each given textual prompt. -### Cache {id="cache"} +## Cache {id="cache"} Interacting with LLMs, either through an external API or a local instance, is costly. Since developing an NLP pipeline generally means a lot of exploration @@ -1436,9 +1146,9 @@ provide your own registered function returning your own cache implementation. If you wish to do so, ensure that your cache object adheres to the `Protocol` defined in `spacy_llm.ty.Cache`. -### Various functions {id="various-functions"} +## Various functions {id="various-functions"} -#### spacy.FewShotReader.v1 {id="fewshotreader-v1"} +### spacy.FewShotReader.v1 {id="fewshotreader-v1"} This function is registered in spaCy's `misc` registry, and reads in examples from a `.yml`, `.yaml`, `.json` or `.jsonl` file. It uses @@ -1457,7 +1167,7 @@ them depending on the file extension. | -------- | ----------------------------------------------------------------------------------------------- | | `path` | Path to an examples file with suffix `.yml`, `.yaml`, `.json` or `.jsonl`. ~~Union[str, Path]~~ | -#### spacy.FileReader.v1 {id="filereader-v1"} +### spacy.FileReader.v1 {id="filereader-v1"} This function is registered in spaCy's `misc` registry, and reads a file provided to the `path` to return a `str` representation of its contents. This @@ -1477,7 +1187,7 @@ template. | -------- | ------------------------------------------------- | | `path` | Path to the file to be read. ~~Union[str, Path]~~ | -#### Normalizer functions {id="normalizer-functions"} +### Normalizer functions {id="normalizer-functions"} These functions provide simple normalizations for string comparisons, e.g. between a list of specified labels and a label given in the raw text of the LLM diff --git a/website/docs/usage/large-language-models.mdx b/website/docs/usage/large-language-models.mdx index 4da9a8f16..86f44f5ae 100644 --- a/website/docs/usage/large-language-models.mdx +++ b/website/docs/usage/large-language-models.mdx @@ -108,7 +108,7 @@ labels = ["COMPLIMENT", "INSULT"] [components.llm.model] @llm_models = "spacy.GPT-3-5.v1" -config = {"temperature": 0.3} +config = {"temperature": 0.0} ``` Now run: @@ -142,7 +142,7 @@ pipeline = ["llm"] factory = "llm" [components.llm.task] -@llm_tasks = "spacy.NER.v2" +@llm_tasks = "spacy.NER.v3" labels = ["PERSON", "ORGANISATION", "LOCATION"] [components.llm.model] @@ -169,25 +169,17 @@ to be `"databricks/dolly-v2-12b"` for better performance. ### Example 3: Create the component directly in Python {id="example-3"} -The `llm` component behaves as any other component does, so adding it to an -existing pipeline follows the same pattern: +The `llm` component behaves as any other component does, and there are +[task-specific components](/api/large-language-models#config) defined to +help you hit the ground running with a reasonable built-in task implementation. ```python import spacy nlp = spacy.blank("en") -nlp.add_pipe( - "llm", - config={ - "task": { - "@llm_tasks": "spacy.NER.v2", - "labels": ["PERSON", "ORGANISATION", "LOCATION"] - }, - "model": { - "@llm_models": "spacy.GPT-3-5.v1", - }, - }, -) +llm_ner = nlp.add_pipe("llm_ner") +llm_ner.add_label("PERSON") +llm_ner.add_label("LOCATION") nlp.initialize() doc = nlp("Jack and Jill rode up the hill in Les Deux Alpes") print([(ent.text, ent.label_) for ent in doc.ents]) @@ -314,7 +306,7 @@ COMPLIMENT ## API {id="api"} -`spacy-llm` exposes a `llm` factory with +`spacy-llm` exposes an `llm` factory with [configurable settings](/api/large-language-models#config). An `llm` component is defined by two main settings: @@ -359,24 +351,26 @@ function. | [`task.parse_responses`](/api/large-language-models#task-parse-responses) | Takes a collection of LLM responses and the original documents, parses the responses into structured information, and sets the annotations on the documents. | Moreover, the task may define an optional [`scorer` method](/api/scorer#score). -It should accept an iterable of `Example`s as input and return a score +It should accept an iterable of `Example` objects as input and return a score dictionary. If the `scorer` method is defined, `spacy-llm` will call it to evaluate the component. -| Component | Description | -| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`spacy.Summarization.v1`](/api/large-language-models#summarization-v1) | The summarization task prompts the model for a concise summary of the provided text. | -| [`spacy.NER.v2`](/api/large-language-models#ner-v2) | The built-in NER task supports both zero-shot and few-shot prompting. This version also supports explicitly defining the provided labels with custom descriptions. | -| [`spacy.NER.v1`](/api/large-language-models#ner-v1) | The original version of the built-in NER task supports both zero-shot and few-shot prompting. | -| [`spacy.SpanCat.v2`](/api/large-language-models#spancat-v2) | The built-in SpanCat task is a simple adaptation of the NER task to support overlapping entities and store its annotations in `doc.spans`. | -| [`spacy.SpanCat.v1`](/api/large-language-models#spancat-v1) | The original version of the built-in SpanCat task is a simple adaptation of the v1 NER task to support overlapping entities and store its annotations in `doc.spans`. | -| [`spacy.TextCat.v3`](/api/large-language-models#textcat-v3) | Version 3 (the most recent) of the built-in TextCat task supports both zero-shot and few-shot prompting. It allows setting definitions of labels. | -| [`spacy.TextCat.v2`](/api/large-language-models#textcat-v2) | Version 2 of the built-in TextCat task supports both zero-shot and few-shot prompting and includes an improved prompt template. | -| [`spacy.TextCat.v1`](/api/large-language-models#textcat-v1) | Version 1 of the built-in TextCat task supports both zero-shot and few-shot prompting. | -| [`spacy.REL.v1`](/api/large-language-models#rel-v1) | The built-in REL task supports both zero-shot and few-shot prompting. It relies on an upstream NER component for entities extraction. | -| [`spacy.Lemma.v1`](/api/large-language-models#lemma-v1) | The `Lemma.v1` task lemmatizes the provided text and updates the `lemma_` attribute in the doc's tokens accordingly. | -| [`spacy.Sentiment.v1`](/api/large-language-models#sentiment-v1) | Performs sentiment analysis on provided texts. | -| [`spacy.NoOp.v1`](/api/large-language-models#noop-v1) | This task is only useful for testing - it tells the LLM to do nothing, and does not set any fields on the `docs`. | +| Component | Description | +| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | +| [`spacy.Summarization.v1`](/api/large-language-models#summarization-v1) | The summarization task prompts the model for a concise summary of the provided text. | +| [`spacy.NER.v3`](/api/large-language-models#ner-v3) | Implements Chain-of-Thought reasoning for NER extraction - obtains higher accuracy than v1 or v2. | +| [`spacy.NER.v2`](/api/large-language-models#ner-v2) | Builds on v1 and additionally supports defining the provided labels with explicit descriptions. | +| [`spacy.NER.v1`](/api/large-language-models#ner-v1) | The original version of the built-in NER task supports both zero-shot and few-shot prompting. | +| [`spacy.SpanCat.v3`](/api/large-language-models#spancat-v3) | Adaptation of the v3 NER task to support overlapping entities and store its annotations in `doc.spans`. | +| [`spacy.SpanCat.v2`](/api/large-language-models#spancat-v2) | Adaptation of the v2 NER task to support overlapping entities and store its annotations in `doc.spans`. | +| [`spacy.SpanCat.v1`](/api/large-language-models#spancat-v1) | Adaptation of the v1 NER task to support overlapping entities and store its annotations in `doc.spans`. | +| [`spacy.REL.v1`](/api/large-language-models#rel-v1) | Relation Extraction task supporting both zero-shot and few-shot prompting. | +| [`spacy.TextCat.v3`](/api/large-language-models#textcat-v3) | Version 3 builds on v2 and allows setting definitions of labels. | +| [`spacy.TextCat.v2`](/api/large-language-models#textcat-v2) | Version 2 builds on v1 and includes an improved prompt template. | +| [`spacy.TextCat.v1`](/api/large-language-models#textcat-v1) | Version 1 of the built-in TextCat task supports both zero-shot and few-shot prompting. | +| [`spacy.Lemma.v1`](/api/large-language-models#lemma-v1) | Lemmatizes the provided text and updates the `lemma_` attribute of the tokens accordingly. | +| [`spacy.Sentiment.v1`](/api/large-language-models#sentiment-v1) | Performs sentiment analysis on provided texts. | +| [`spacy.NoOp.v1`](/api/large-language-models#noop-v1) | This task is only useful for testing - it tells the LLM to do nothing, and does not set any fields on the `docs`. | #### Providing examples for few-shot prompts {id="few-shot-prompts"} @@ -469,31 +463,38 @@ provider's documentation. -| Component | Description | -| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ | -| [`spacy.GPT-4.v1`](/api/large-language-models#gpt-4) | OpenAI’s `gpt-4` model family. | -| [`spacy.GPT-3-5.v1`](/api/large-language-models#gpt-3-5) | OpenAI’s `gpt-3-5` model family. | -| [`spacy.Text-Davinci.v1`](/api/large-language-models#text-davinci) | OpenAI’s `text-davinci` model family. | -| [`spacy.Code-Davinci.v1`](/api/large-language-models#code-davinci) | OpenAI’s `code-davinci` model family. | -| [`spacy.Text-Curie.v1`](/api/large-language-models#text-curie) | OpenAI’s `text-curie` model family. | -| [`spacy.Text-Babbage.v1`](/api/large-language-models#text-babbage) | OpenAI’s `text-babbage` model family. | -| [`spacy.Text-Ada.v1`](/api/large-language-models#text-ada) | OpenAI’s `text-ada` model family. | -| [`spacy.Davinci.v1`](/api/large-language-models#davinci) | OpenAI’s `davinci` model family. | -| [`spacy.Curie.v1`](/api/large-language-models#curie) | OpenAI’s `curie` model family. | -| [`spacy.Babbage.v1`](/api/large-language-models#babbage) | OpenAI’s `babbage` model family. | -| [`spacy.Ada.v1`](/api/large-language-models#ada) | OpenAI’s `ada` model family. | -| [`spacy.Command.v1`](/api/large-language-models#command) | Cohere’s `command` model family. | -| [`spacy.Claude-1.v1`](/api/large-language-models#claude-1) | Anthropic’s `claude-1` model family. | -| [`spacy.Claude-instant-1.v1`](/api/large-language-models#claude-instant-1) | Anthropic’s `claude-instant-1` model family. | -| [`spacy.Claude-instant-1-1.v1`](/api/large-language-models#claude-instant-1-1) | Anthropic’s `claude-instant-1.1` model family. | -| [`spacy.Claude-1-0.v1`](/api/large-language-models#claude-1-0) | Anthropic’s `claude-1.0` model family. | -| [`spacy.Claude-1-2.v1`](/api/large-language-models#claude-1-2) | Anthropic’s `claude-1.2` model family. | -| [`spacy.Claude-1-3.v1`](/api/large-language-models#claude-1-3) | Anthropic’s `claude-1.3` model family. | -| [`spacy.Dolly.v1`](/api/large-language-models#dolly) | Dolly models through [Databricks](https://huggingface.co/databricks) on HuggingFace. | -| [`spacy.Falcon.v1`](/api/large-language-models#falcon) | Falcon model through HuggingFace. | -| [`spacy.StableLM.v1`](/api/large-language-models#stablelm) | StableLM model through HuggingFace. | -| [`spacy.OpenLLaMA.v1`](/api/large-language-models#openllama) | OpenLLaMA model through HuggingFace. | -| [LangChain models](/api/large-language-models#langchain-models) | LangChain models for API retrieval. | +| Model | Description | +| ----------------------------------------------------------------------- | ---------------------------------------------- | +| [`spacy.GPT-4.v2`](/api/large-language-models#models-rest) | OpenAI’s `gpt-4` model family. | +| [`spacy.GPT-3-5.v2`](/api/large-language-models#models-rest) | OpenAI’s `gpt-3-5` model family. | +| [`spacy.Text-Davinci.v2`](/api/large-language-models#models-rest) | OpenAI’s `text-davinci` model family. | +| [`spacy.Code-Davinci.v2`](/api/large-language-models#models-rest) | OpenAI’s `code-davinci` model family. | +| [`spacy.Text-Curie.v2`](/api/large-language-models#models-rest) | OpenAI’s `text-curie` model family. | +| [`spacy.Text-Babbage.v2`](/api/large-language-models#models-rest) | OpenAI’s `text-babbage` model family. | +| [`spacy.Text-Ada.v2`](/api/large-language-models#models-rest) | OpenAI’s `text-ada` model family. | +| [`spacy.Davinci.v2`](/api/large-language-models#models-rest) | OpenAI’s `davinci` model family. | +| [`spacy.Curie.v2`](/api/large-language-models#models-rest) | OpenAI’s `curie` model family. | +| [`spacy.Babbage.v2`](/api/large-language-models#models-rest) | OpenAI’s `babbage` model family. | +| [`spacy.Ada.v2`](/api/large-language-models#models-rest) | OpenAI’s `ada` model family. | +| [`spacy.Command.v1`](/api/large-language-models#models-rest) | Cohere’s `command` model family. | +| [`spacy.Claude-2.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-2` model family. | +| [`spacy.Claude-1.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-1` model family. | +| [`spacy.Claude-instant-1.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-instant-1` model family. | +| [`spacy.Claude-instant-1-1.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-instant-1.1` model family. | +| [`spacy.Claude-1-0.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-1.0` model family. | +| [`spacy.Claude-1-2.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-1.2` model family. | +| [`spacy.Claude-1-3.v1`](/api/large-language-models#models-rest) | Anthropic’s `claude-1.3` model family. | +| [`spacy.Dolly.v1`](/api/large-language-models#models-hf) | Dolly models through HuggingFace. | +| [`spacy.Falcon.v1`](/api/large-language-models#models-hf) | Falcon models through HuggingFace. | +| [`spacy.Llama2.v1`](/api/large-language-models#models-hf) | Llama2 models through HuggingFace. | +| [`spacy.StableLM.v1`](/api/large-language-models#models-hf) | StableLM models through HuggingFace. | +| [`spacy.OpenLLaMA.v1`](/api/large-language-models#models-hf) | OpenLLaMA models through HuggingFace. | +| [LangChain models](/api/large-language-models#langchain-models) | LangChain models for API retrieval. | + +Note that the chat models variants of Llama 2 are currently not supported. This +is because they need a particular prompting setup and don't add any discernible +benefits in the use case of `spacy-llm` (i. e. no interactive chat) compared to +the completion model variants. ### Cache {id="cache"} @@ -505,7 +506,7 @@ documents at each run that keeps batches of documents stored on disk. ### Various functions {id="various-functions"} -| Component | Description | +| Function | Description | | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [`spacy.FewShotReader.v1`](/api/large-language-models#fewshotreader-v1) | This function is registered in spaCy's `misc` registry, and reads in examples from a `.yml`, `.yaml`, `.json` or `.jsonl` file. It uses [`srsly`](https://github.com/explosion/srsly) to read in these files and parses them depending on the file extension. | | [`spacy.FileReader.v1`](/api/large-language-models#filereader-v1) | This function is registered in spaCy's `misc` registry, and reads a file provided to the `path` to return a `str` representation of its contents. This function is typically used to read [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) files containing the prompt template. |