Update docs [ci skip]

This commit is contained in:
Ines Montani 2020-09-13 19:29:06 +02:00
parent 416deb412f
commit 2e3d067a7b
3 changed files with 51 additions and 8 deletions

View File

@ -14,6 +14,7 @@ menu:
- ['evaluate', 'evaluate'] - ['evaluate', 'evaluate']
- ['package', 'package'] - ['package', 'package']
- ['project', 'project'] - ['project', 'project']
- ['ray', 'ray']
--- ---
spaCy's CLI provides a range of helpful commands for downloading and training spaCy's CLI provides a range of helpful commands for downloading and training
@ -1134,3 +1135,45 @@ $ python -m spacy project dvc [project_dir] [workflow] [--force] [--verbose]
| `--verbose`, `-V` |  Print more output generated by DVC. ~~bool (flag)~~ | | `--verbose`, `-V` |  Print more output generated by DVC. ~~bool (flag)~~ |
| `--help`, `-h` | Show help message and available arguments. ~~bool (flag)~~ | | `--help`, `-h` | Show help message and available arguments. ~~bool (flag)~~ |
| **CREATES** | A `dvc.yaml` file in the project directory, based on the steps defined in the given workflow. | | **CREATES** | A `dvc.yaml` file in the project directory, based on the steps defined in the given workflow. |
## ray {#ray new="3"}
The `spacy ray` CLI includes commands for parallel and distributed computing via
[Ray](https://ray.io).
<!-- TODO: add links to parallel training docs and project template -->
<Infobox variant="warning" title="Important note">
To use this command, you need the
[`spacy-ray`](https://github.com/explosion/spacy-ray) package installed.
Installing the package will automatically add the `ray` command to the spaCy
CLI.
</Infobox>
### ray train {#ray-train tag="command"}
Train a spaCy pipeline using [Ray](https://ray.io) for parallel training. The
command works just like [`spacy train`](/api/cli#train).
```cli
$ python -m spacy ray train [config_path] [--code-path] [--strategy] [--n-workers] [--address] [--gpu-id] [--verbose] [overrides]
```
> #### Example
>
> ```cli
> $ python -m spacy ray train config.cfg --n-workers 2
> ```
| Name | Description |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `config_path` | Path to [training config](/api/data-formats#config) file containing all settings and hyperparameters. ~~Path (positional)~~ |
| `--code`, `-c` | Path to Python file with additional code to be imported. Allows [registering custom functions](/usage/training#custom-functions) for new architectures. ~~Optional[Path] \(option)~~ |
| `--n-workers`, `-n` | The number of workers. Defaults to `1`. ~~int (option)~~ |
| `--address`, `-a` | Optional address of the Ray cluster. Defaults to `None`. ~~Optional[str] \(option)~~ |
| `--gpu-id`, `-g` | GPU ID or `-1` for CPU. Defaults to `-1`. ~~int (option)~~ |
| `--verbose`, `-V` | Display more information for debugging purposes. ~~bool (flag)~~ |
| `--help`, `-h` | Show help message and available arguments. ~~bool (flag)~~ |
| overrides | Config parameters to override. Should be options starting with `--` that correspond to the config section and value to override, e.g. `--paths.train ./train.spacy`. ~~Any (option/flag)~~ |

View File

@ -796,11 +796,9 @@ workflows, including
evaluation workflow that lets you compare two different models and their evaluation workflow that lets you compare two different models and their
results. results.
<Project id="integrations/prodigy"> <!-- TODO: <Project id="integrations/prodigy">
<!-- TODO: --> </Project> -->
</Project>
--- ---
@ -943,12 +941,14 @@ your results.
![Screenshot: Parameter importance using config values](../images/wandb2.jpg 'Parameter importance using config values') ![Screenshot: Parameter importance using config values](../images/wandb2.jpg 'Parameter importance using config values')
<!-- TODO:
<Project id="integrations/wandb"> <Project id="integrations/wandb">
Get started with tracking your spaCy training runs in Weights & Biases using our Get started with tracking your spaCy training runs in Weights & Biases using our
project template. It includes a simple config using the `WandbLogger`, as well project template. It includes a simple config using the `WandbLogger`, as well
as a custom logger implementation you can adjust for your specific use case. as a custom logger implementation you can adjust for your specific use case.
<!-- TODO: -->
</Project> </Project>
-->

View File

@ -14,6 +14,7 @@ import GitHubCode from './github'
import classes from '../styles/code.module.sass' import classes from '../styles/code.module.sass'
const WRAP_THRESHOLD = 30 const WRAP_THRESHOLD = 30
const CLI_GROUPS = ['init', 'debug', 'project', 'ray']
export default props => ( export default props => (
<Pre> <Pre>
@ -99,7 +100,6 @@ function replacePrompt(line, prompt, isFirst = false) {
} }
function parseArgs(raw) { function parseArgs(raw) {
const commandGroups = ['init', 'debug', 'project']
let args = raw.split(' ').filter(arg => arg) let args = raw.split(' ').filter(arg => arg)
const result = {} const result = {}
while (args.length) { while (args.length) {
@ -108,7 +108,7 @@ function parseArgs(raw) {
const isFlag = !args.length || (args[0].length > 1 && args[0].startsWith('-')) const isFlag = !args.length || (args[0].length > 1 && args[0].startsWith('-'))
result[opt] = isFlag ? true : args.shift() result[opt] = isFlag ? true : args.shift()
} else { } else {
const key = commandGroups.includes(opt) ? `${opt} ${args.shift()}` : opt const key = CLI_GROUPS.includes(opt) ? `${opt} ${args.shift()}` : opt
result[key] = null result[key] = null
} }
} }