| Get a custom spaCy pipeline, tailor-made for your NLP problem by spaCy's core developers. Streamlined, production-ready, predictable and maintainable. Start by completing our 5-minute questionnaire to tell us what you need and we'll be in touch! **[Learn more →](https://explosion.ai/spacy-tailored-pipelines)** |
+| 
| Bespoke advice for problem solving, strategy and analysis for applied NLP projects. Services include data strategy, code reviews, pipeline design and annotation coaching. Curious? Fill in our 5-minute questionnaire to tell us what you need and we'll be in touch! **[Learn more →](https://explosion.ai/spacy-tailored-analysis)** |
[spacy 101]: https://spacy.io/usage/spacy-101
[new in v3.0]: https://spacy.io/usage/v3
@@ -59,6 +60,7 @@ open-source software, released under the MIT license.
[changelog]: https://spacy.io/usage#changelog
[contribute]: https://github.com/explosion/spaCy/blob/master/CONTRIBUTING.md
+
## 💬 Where to ask questions
The spaCy project is maintained by the [spaCy team](https://explosion.ai/about).
@@ -79,7 +81,7 @@ more people can benefit from it.
## Features
-- Support for **60+ languages**
+- Support for **70+ languages**
- **Trained pipelines** for different languages and tasks
- Multi-task learning with pretrained **transformers** like BERT
- Support for pretrained **word vectors** and embeddings
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index f475b7fdd..0f7ea91f9 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -31,7 +31,7 @@ jobs:
inputs:
versionSpec: "3.7"
- script: |
- pip install flake8==3.9.2
+ pip install flake8==5.0.4
python -m flake8 spacy --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics
displayName: "flake8"
@@ -41,7 +41,7 @@ jobs:
matrix:
# We're only running one platform per Python version to speed up builds
Python36Linux:
- imageName: "ubuntu-latest"
+ imageName: "ubuntu-20.04"
python.version: "3.6"
# Python36Windows:
# imageName: "windows-latest"
@@ -50,7 +50,7 @@ jobs:
# imageName: "macos-latest"
# python.version: "3.6"
# Python37Linux:
- # imageName: "ubuntu-latest"
+ # imageName: "ubuntu-20.04"
# python.version: "3.7"
Python37Windows:
imageName: "windows-latest"
@@ -76,15 +76,24 @@ jobs:
# Python39Mac:
# imageName: "macos-latest"
# python.version: "3.9"
- Python310Linux:
- imageName: "ubuntu-latest"
- python.version: "3.10"
+ # Python310Linux:
+ # imageName: "ubuntu-latest"
+ # python.version: "3.10"
Python310Windows:
imageName: "windows-latest"
python.version: "3.10"
- Python310Mac:
- imageName: "macos-latest"
- python.version: "3.10"
+ # Python310Mac:
+ # imageName: "macos-latest"
+ # python.version: "3.10"
+ Python311Linux:
+ imageName: 'ubuntu-latest'
+ python.version: '3.11'
+ Python311Windows:
+ imageName: 'windows-latest'
+ python.version: '3.11'
+ Python311Mac:
+ imageName: 'macos-latest'
+ python.version: '3.11'
maxParallel: 4
pool:
vmImage: $(imageName)
@@ -92,20 +101,3 @@ jobs:
- template: .github/azure-steps.yml
parameters:
python_version: '$(python.version)'
- architecture: 'x64'
-
-# - job: "TestGPU"
-# dependsOn: "Validate"
-# strategy:
-# matrix:
-# Python38LinuxX64_GPU:
-# python.version: '3.8'
-# pool:
-# name: "LinuxX64_GPU"
-# steps:
-# - template: .github/azure-steps.yml
-# parameters:
-# python_version: '$(python.version)'
-# architecture: 'x64'
-# gpu: true
-# num_build_jobs: 24
diff --git a/build-constraints.txt b/build-constraints.txt
index 956973abf..c1e82f1b0 100644
--- a/build-constraints.txt
+++ b/build-constraints.txt
@@ -5,4 +5,5 @@ numpy==1.17.3; python_version=='3.8' and platform_machine!='aarch64'
numpy==1.19.2; python_version=='3.8' and platform_machine=='aarch64'
numpy==1.19.3; python_version=='3.9'
numpy==1.21.3; python_version=='3.10'
-numpy; python_version>='3.11'
+numpy==1.23.2; python_version=='3.11'
+numpy; python_version>='3.12'
diff --git a/extra/DEVELOPER_DOCS/Satellite Packages.md b/extra/DEVELOPER_DOCS/Satellite Packages.md
new file mode 100644
index 000000000..02b06a90e
--- /dev/null
+++ b/extra/DEVELOPER_DOCS/Satellite Packages.md
@@ -0,0 +1,82 @@
+# spaCy Satellite Packages
+
+This is a list of all the active repos relevant to spaCy besides the main one, with short descriptions, history, and current status. Archived repos will not be covered.
+
+## Always Included in spaCy
+
+These packages are always pulled in when you install spaCy. Most of them are direct dependencies, but some are transitive dependencies through other packages.
+
+- [spacy-legacy](https://github.com/explosion/spacy-legacy): When an architecture in spaCy changes enough to get a new version, the old version is frozen and moved to spacy-legacy. This allows us to keep the core library slim while also preserving backwards compatability.
+- [thinc](https://github.com/explosion/thinc): Thinc is the machine learning library that powers trainable components in spaCy. It wraps backends like Numpy, PyTorch, and Tensorflow to provide a functional interface for specifying architectures.
+- [catalogue](https://github.com/explosion/catalogue): Small library for adding function registries, like those used for model architectures in spaCy.
+- [confection](https://github.com/explosion/confection): This library contains the functionality for config parsing that was formerly contained directly in Thinc.
+- [spacy-loggers](https://github.com/explosion/spacy-loggers): Contains loggers beyond the default logger available in spaCy's core code base. This includes loggers integrated with third-party services, which may differ in release cadence from spaCy itself.
+- [wasabi](https://github.com/explosion/wasabi): A command line formatting library, used for terminal output in spaCy.
+- [srsly](https://github.com/explosion/srsly): A wrapper that vendors several serialization libraries for spaCy. Includes parsers for JSON, JSONL, MessagePack, (extended) Pickle, and YAML.
+- [preshed](https://github.com/explosion/preshed): A Cython library for low-level data structures like hash maps, used for memory efficient data storage.
+- [cython-blis](https://github.com/explosion/cython-blis): Fast matrix multiplication using BLIS without depending on system libraries. Required by Thinc, rather than spaCy directly.
+- [murmurhash](https://github.com/explosion/murmurhash): A wrapper library for a C++ murmurhash implementation, used for string IDs in spaCy and preshed.
+- [cymem](https://github.com/explosion/cymem): A small library for RAII-style memory management in Cython.
+
+## Optional Extensions for spaCy
+
+These are repos that can be used by spaCy but aren't part of a default installation. Many of these are wrappers to integrate various kinds of third-party libraries.
+
+- [spacy-transformers](https://github.com/explosion/spacy-transformers): A wrapper for the [HuggingFace Transformers](https://huggingface.co/docs/transformers/index) library, this handles the extensive conversion necessary to coordinate spaCy's powerful `Doc` representation, training pipeline, and the Transformer embeddings. When released, this was known as `spacy-pytorch-transformers`, but it changed to the current name when HuggingFace update the name of their library as well.
+- [spacy-huggingface-hub](https://github.com/explosion/spacy-huggingface-hub): This package has a CLI script for uploading a packaged spaCy pipeline (created with `spacy package`) to the [Hugging Face Hub](https://huggingface.co/models).
+- [spacy-alignments](https://github.com/explosion/spacy-alignments): A wrapper for the tokenizations library (mentioned below) with a modified build system to simplify cross-platform wheel creation. Used in spacy-transformers for aligning spaCy and HuggingFace tokenizations.
+- [spacy-experimental](https://github.com/explosion/spacy-experimental): Experimental components that are not quite ready for inclusion in the main spaCy library. Usually there are unresolved questions around their APIs, so the experimental library allows us to expose them to the community for feedback before fully integrating them.
+- [spacy-lookups-data](https://github.com/explosion/spacy-lookups-data): A repository of linguistic data, such as lemmas, that takes up a lot of disk space. Originally created to reduce the size of the spaCy core library. This is mainly useful if you want the data included but aren't using a pretrained pipeline; for the affected languages, the relevant data is included in pretrained pipelines directly.
+- [coreferee](https://github.com/explosion/coreferee): Coreference resolution for English, French, German and Polish, optimised for limited training data and easily extensible for further languages. Used as a spaCy pipeline component.
+- [spacy-stanza](https://github.com/explosion/spacy-stanza): This is a wrapper that allows the use of Stanford's Stanza library in spaCy.
+- [spacy-streamlit](https://github.com/explosion/spacy-streamlit): A wrapper for the Streamlit dashboard building library to help with integrating [displaCy](https://spacy.io/api/top-level/#displacy).
+- [spacymoji](https://github.com/explosion/spacymoji): A library to add extra support for emoji to spaCy, such as including character names.
+- [thinc-apple-ops](https://github.com/explosion/thinc-apple-ops): A special backend for OSX that uses Apple's native libraries for improved performance.
+- [os-signpost](https://github.com/explosion/os-signpost): A Python package that allows you to use the `OSSignposter` API in OSX for performance analysis.
+- [spacy-ray](https://github.com/explosion/spacy-ray): A wrapper to integrate spaCy with Ray, a distributed training framework. Currently a work in progress.
+
+## Prodigy
+
+[Prodigy](https://prodi.gy) is Explosion's easy to use and highly customizable tool for annotating data. Prodigy itself requires a license, but the repos below contain documentation, examples, and editor or notebook integrations.
+
+- [prodigy-recipes](https://github.com/explosion/prodigy-recipes): Sample recipes for Prodigy, along with notebooks and other examples of usage.
+- [vscode-prodigy](https://github.com/explosion/vscode-prodigy): A VS Code extension that lets you run Prodigy inside VS Code.
+- [jupyterlab-prodigy](https://github.com/explosion/jupyterlab-prodigy): An extension for JupyterLab that lets you run Prodigy inside JupyterLab.
+
+## Independent Tools or Projects
+
+These are tools that may be related to or use spaCy, but are functional independent projects in their own right as well.
+
+- [floret](https://github.com/explosion/floret): A modification of fastText to use Bloom Embeddings. Can be used to add vectors with subword features to spaCy, and also works independently in the same manner as fastText.
+- [sense2vec](https://github.com/explosion/sense2vec): A library to make embeddings of noun phrases or words coupled with their part of speech. This library uses spaCy.
+- [spacy-vectors-builder](https://github.com/explosion/spacy-vectors-builder): This is a spaCy project that builds vectors using floret and a lot of input text. It handles downloading the input data as well as the actual building of vectors.
+- [holmes-extractor](https://github.com/explosion/holmes-extractor): Information extraction from English and German texts based on predicate logic. Uses spaCy.
+- [healthsea](https://github.com/explosion/healthsea): Healthsea is a project to extract information from comments about health supplements. Structurally, it's a self-contained, large spaCy project.
+- [spacy-pkuseg](https://github.com/explosion/spacy-pkuseg): A fork of the pkuseg Chinese tokenizer. Used for Chinese support in spaCy, but also works independently.
+- [ml-datasets](https://github.com/explosion/ml-datasets): This repo includes loaders for several standard machine learning datasets, like MNIST or WikiNER, and has historically been used in spaCy example code and documentation.
+
+## Documentation and Informational Repos
+
+These repos are used to support the spaCy docs or otherwise present information about spaCy or other Explosion projects.
+
+- [projects](https://github.com/explosion/projects): The projects repo is used to show detailed examples of spaCy usage. Individual projects can be checked out using the spaCy command line tool, rather than checking out the projects repo directly.
+- [spacy-course](https://github.com/explosion/spacy-course): Home to the interactive spaCy course for learning about how to use the library and some basic NLP principles.
+- [spacy-io-binder](https://github.com/explosion/spacy-io-binder): Home to the notebooks used for interactive examples in the documentation.
+
+## Organizational / Meta
+
+These repos are used for organizing data around spaCy, but are not something an end user would need to install as part of using the library.
+
+- [spacy-models](https://github.com/explosion/spacy-models): This repo contains metadata (but not training data) for all the spaCy models. This includes information about where their training data came from, version compatability, and performance information. It also includes tests for the model packages, and the built models are hosted as releases of this repo.
+- [wheelwright](https://github.com/explosion/wheelwright): A tool for automating our PyPI builds and releases.
+- [ec2buildwheel](https://github.com/explosion/ec2buildwheel): A small project that allows you to build Python packages in the manner of cibuildwheel, but on any EC2 image. Used by wheelwright.
+
+## Other
+
+Repos that don't fit in any of the above categories.
+
+- [blis](https://github.com/explosion/blis): A fork of the official BLIS library. The main branch is not updated, but work continues in various branches. This is used for cython-blis.
+- [tokenizations](https://github.com/explosion/tokenizations): A library originally by Yohei Tamura to align strings with tolerance to some variations in features like case and diacritics, used for aligning tokens and wordpieces. Adopted and maintained by Explosion, but usually spacy-alignments is used instead.
+- [conll-2012](https://github.com/explosion/conll-2012): A repo to hold some slightly cleaned up versions of the official scripts for the CoNLL 2012 shared task involving coreference resolution. Used in the coref project.
+- [fastapi-explosion-extras](https://github.com/explosion/fastapi-explosion-extras): Some small tweaks to FastAPI used at Explosion.
+
diff --git a/licenses/3rd_party_licenses.txt b/licenses/3rd_party_licenses.txt
index d58da9c4a..851e09585 100644
--- a/licenses/3rd_party_licenses.txt
+++ b/licenses/3rd_party_licenses.txt
@@ -127,3 +127,34 @@ distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+
+
+polyleven
+---------
+
+* Files: spacy/matcher/polyleven.c
+
+MIT License
+
+Copyright (c) 2021 Fujimoto Seiji | Header 1 | Header 2 |
|---|---|
| Column 1 | Column 2 |
+
+ + +{' '} + + +
+ ++ + +{' '} + + +
+ +## Components + +### Table {id="table"} + +> #### Markdown +> +> ```markdown +> | Header 1 | Header 2 | +> | -------- | -------- | +> | Column 1 | Column 2 | +> ``` +> +> #### JSX +> +> ```markup +>| Header 1 | Header 2 |
|---|---|
| Column 1 | Column 2 |
```markdown
diff --git a/website/docs/usage/training.md b/website/docs/usage/training.mdx
similarity index 95%
rename from website/docs/usage/training.md
rename to website/docs/usage/training.mdx
index 5e064b269..6cda975cb 100644
--- a/website/docs/usage/training.md
+++ b/website/docs/usage/training.mdx
@@ -15,15 +15,17 @@ menu:
- ['Internal API', 'api']
---
-## Introduction to training {#basics hidden="true"}
-
-import Training101 from 'usage/101/\_training.md'
+## Introduction to training {id="basics",hidden="true"}
+
+
+ + Get a custom spaCy pipeline, tailor-made for your NLP problem by + spaCy's core developers. + +
+
+
+ Prodigy is an annotation tool so efficient that data + scientists can do the annotation themselves, enabling a new level of rapid + iteration. Whether you're working on entity recognition, intent + detection or image classification, Prodigy can help you{' '} + train and evaluate your models faster. +
- spaCy's new project system gives you a smooth path from prototype to + spaCy's new project system gives you a smooth path from prototype to production. It lets you keep track of all those{' '} data transformation, preprocessing and{' '} training steps, so you can make sure your project is always @@ -236,13 +243,15 @@ const Landing = ({ data }) => { button="See what's new" small > - spaCy v3.0 features all new transformer-based pipelines that - bring spaCy's accuracy right up to the current state-of-the-art - . You can use any pretrained transformer to train your own pipelines, and even - share one transformer between multiple components with{' '} - multi-task learning. Training is now fully configurable and - extensible, and you can define your own custom models using{' '} - PyTorch, TensorFlow and other frameworks. +
+ spaCy v3.0 features all new transformer-based pipelines{' '} + that bring spaCy's accuracy right up to the current{' '} + state-of-the-art. You can use any pretrained transformer to + train your own pipelines, and even share one transformer between multiple + components with multi-task learning. Training is now fully + configurable and extensible, and you can define your own custom models using{' '} + PyTorch, TensorFlow and other frameworks. +
+
+
+ In this free and interactive online course you’ll learn how + to use spaCy to build advanced natural language understanding systems, using + both rule-based and machine learning approaches. It includes{' '} + 55 exercises featuring videos, slide decks, multiple-choice + questions and interactive coding practice in the browser. +
- spaCy v3.0 introduces transformer-based pipelines that bring spaCy's
+ spaCy v3.0 introduces transformer-based pipelines that bring spaCy's
accuracy right up to the current state-of-the-art. You can
also use a CPU-optimized pipeline, which is less accurate but much cheaper
to run.
@@ -285,33 +296,8 @@ const Landing = ({ data }) => {
)
-export const Pre = props => {
- return {props.children}
+export default CodeBlock
+
+export const Pre = (props) => {
+ return {props.children}
}
export const InlineCode = ({ wrap = false, className, children, ...props }) => {
- const codeClassNames = classNames(classes.inlineCode, className, {
- [classes.wrap]: wrap || (isString(children) && children.length >= WRAP_THRESHOLD),
+ const codeClassNames = classNames(classes['inline-code'], className, {
+ [classes['wrap']]: wrap || (isString(children) && children.length >= WRAP_THRESHOLD),
})
return (
@@ -68,39 +80,76 @@ export const TypeAnnotation = ({ lang = 'python', link = true, children }) => {
const code = Array.isArray(children) ? children.join('') : children || ''
const [rawText, meta] = code.split(/(?= \(.+\)$)/)
const rawStr = rawText.replace(/\./g, TMP_DOT)
- const rawHtml = lang === 'none' || !code ? code : highlightCode(lang, rawStr)
+ const rawHtml =
+ lang === 'none' || !code ? code : Prism.highlight(rawStr, Prism.languages[lang], lang)
const html = rawHtml.replace(new RegExp(TMP_DOT, 'g'), '.').replace(/\n/g, ' ')
const result = htmlToReact(html)
const elements = Array.isArray(result) ? result : [result]
const annotClassNames = classNames(
'type-annotation',
`language-${lang}`,
- classes.inlineCode,
- classes.typeAnnotation,
+ classes['inline-code'],
+ classes['type-annotation'],
{
- [classes.wrap]: code.length >= WRAP_THRESHOLD,
+ [classes['wrap']]: code.length >= WRAP_THRESHOLD,
}
)
return (
-
+
{elements.map((el, i) => (
{linkType(el, !!link)}
))}
- {meta && {meta}}
-
+ {meta && {meta}}
+
)
}
-function replacePrompt(line, prompt, isFirst = false) {
- let result = line
- const hasPrompt = result.startsWith(`${prompt} `)
- const showPrompt = hasPrompt || isFirst
- if (hasPrompt) result = result.slice(2)
- return result && showPrompt ? `${result}` : result
+const splitLines = (children) => {
+ const listChildrenPerLine = []
+
+ if (typeof children === 'string') {
+ listChildrenPerLine.push(...children.split('\n'))
+ } else {
+ listChildrenPerLine.push([])
+ let indexLine = 0
+ if (Array.isArray(children)) {
+ children.forEach((child) => {
+ if (typeof child === 'string' && child.includes('\n')) {
+ const listString = child.split('\n')
+ listString.forEach((string, index) => {
+ listChildrenPerLine[indexLine].push(string)
+
+ if (index !== listString.length - 1) {
+ indexLine += 1
+ listChildrenPerLine[indexLine] = []
+ }
+ })
+ } else {
+ listChildrenPerLine[indexLine].push(child)
+ }
+ })
+ } else {
+ listChildrenPerLine[indexLine].push(children)
+ indexLine += 1
+ listChildrenPerLine[indexLine] = []
+ }
+ }
+
+ const listLine = listChildrenPerLine[listChildrenPerLine.length - 1]
+ if (listLine === '' || (listLine.length === 1 && listLine[0] === '')) {
+ listChildrenPerLine.pop()
+ }
+
+ return listChildrenPerLine.map((childrenPerLine, index) => (
+ <>
+ {childrenPerLine}
+ {index !== listChildrenPerLine.length - 1 && '\n'}
+
+ ))
}
function parseArgs(raw) {
- let args = raw.split(' ').filter(arg => arg)
+ let args = raw.split(' ').filter((arg) => arg)
const result = {}
while (args.length) {
let opt = args.shift()
@@ -120,208 +169,219 @@ function parseArgs(raw) {
return result
}
-function convertLine(line, i) {
- const cliRegex = /^(\$ )?python -m spacy/
- if (cliRegex.test(line)) {
- const text = line.replace(cliRegex, '')
- const args = parseArgs(text)
- const cmd = Object.keys(args).map((key, i) => {
- const value = args[key]
- return value === null || value === true || i === 0 ? key : `${key} ${value}`
- })
- return (
-
-
- python -m
- {' '}
- spacy{' '}
- {cmd.map((item, j) => {
- const isCmd = j === 0
- const url = isCmd ? `/api/cli#${item.replace(' ', '-')}` : null
- const isAbstract = isString(item) && /^\[(.+)\]$/.test(item)
- const itemClassNames = classNames(classes.cliArg, {
- [classes.cliArgHighlight]: isCmd,
- [classes.cliArgEmphasis]: isAbstract,
- })
- const text = isAbstract ? item.slice(1, -1) : item
- return (
-
- {j !== 0 && ' '}
-
-
- {text}
-
-
-
- )
- })}
-
- )
+const flattenReact = (children) => {
+ if (children === null || children === undefined || children === false) {
+ return []
}
- const htmlLine = replacePrompt(highlightCode('bash', line), '$')
- return htmlToReact(htmlLine)
+
+ if (typeof children === 'string') {
+ return [children]
+ }
+
+ if (children.props) {
+ return flattenReact(children.props.children)
+ }
+
+ return children.flatMap(flattenReact)
}
-function formatCode(html, lang, prompt) {
- if (lang === 'cli') {
- const lines = html
- .trim()
- .split('\n')
- .map(line =>
- line
- .split(' | ')
- .map((l, i) => convertLine(l, i))
- .map((l, j) => (
-
- {j !== 0 && | }
- {l}
-
- ))
- )
- return lines.map((line, i) => (
-
- {i !== 0 &&
}
- {line}
-
- ))
+const checkoutForComment = (line) => {
+ const lineParts = line.split(' # ')
+
+ if (lineParts.length !== 2) {
+ return line
}
- const result = html
- .split('\n')
- .map((line, i) => {
- let newLine = prompt ? replacePrompt(line, prompt, i === 0) : line
- if (lang === 'diff' && !line.startsWith('<')) {
- newLine = highlightCode('python', line)
- }
- return newLine
- })
- .join('\n')
- return htmlToReact(result)
+
+ return (
+ <>
+ {lineParts[0]}
+ {` `}
+
+ {`# `}
+ {lineParts[1]}
+
+
+ )
+}
+
+const handlePromot = ({ lineFlat, prompt }) => {
+ const lineWithoutPrompt = lineFlat.slice(prompt.length + 1)
+
+ const cliRegex = /^python -m spacy/
+
+ if (!cliRegex.test(lineWithoutPrompt)) {
+ return {checkoutForComment(lineWithoutPrompt)}
+ }
+
+ const text = lineWithoutPrompt.replace(cliRegex, '')
+ const args = parseArgs(text)
+ const cmd = Object.keys(args).map((key, i) => {
+ const value = args[key]
+ return value === null || value === true || i === 0 ? key : `${key} ${value}`
+ })
+ return (
+
+ python -m spacy{' '}
+ {cmd.map((item, j) => {
+ const isCmd = j === 0
+ const url = isCmd ? `/api/cli#${item.replace(' ', '-')}` : null
+ const isAbstract = isString(item) && /^\[(.+)\]$/.test(item)
+ const itemClassNames = classNames(classes['cli-arg'], {
+ [classes['cli-arg-highlight']]: isCmd,
+ [classes['cli-arg-emphasis']]: isAbstract,
+ })
+ const text = isAbstract ? item.slice(1, -1) : item
+ return (
+
+ {j !== 0 && ' '}
+
+
+ {text}
+
+
+
+ )
+ })}
+
+ )
+}
+
+const convertLine = ({ line, prompt, lang }) => {
+ const lineFlat = flattenReact(line).join('')
+ if (lineFlat.startsWith(`${prompt} `)) {
+ return handlePromot({ lineFlat, prompt })
+ }
+
+ return lang === 'none' || !lineFlat ? (
+ lineFlat
+ ) : (
+
+ )
+}
+
+const addLineHighlight = (children, highlight) => {
+ if (!highlight) {
+ return children
+ }
+ const listHighlight = rangeParser(highlight)
+
+ if (listHighlight.length === 0) {
+ return children
+ }
+
+ return children.map((child, index) => {
+ const isHighlight = listHighlight.includes(index + 1)
+ return (
+
+ {child}
+
+ )
+ })
+}
+
+export const CodeHighlighted = ({ children, highlight, lang }) => {
+ const [html, setHtml] = useState()
+
+ useEffect(
+ () =>
+ setHtml(
+ addLineHighlight(
+ splitLines(children).map((line) => convertLine({ line, prompt: '$', lang })),
+ highlight
+ )
+ ),
+ [children, highlight, lang]
+ )
+
+ return <>{html}
}
export class Code extends React.Component {
- state = { Juniper: null }
-
static defaultProps = {
lang: 'none',
executable: null,
- github: false,
}
static propTypes = {
lang: PropTypes.string,
title: PropTypes.string,
executable: PropTypes.oneOf(['true', 'false', true, false, null]),
- github: PropTypes.oneOf(['true', 'false', true, false, null]),
+ github: PropTypes.string,
prompt: PropTypes.string,
highlight: PropTypes.string,
className: PropTypes.string,
children: PropTypes.node,
}
- updateJuniper() {
- if (this.state.Juniper == null && window.Juniper !== null) {
- this.setState({ Juniper: window.Juniper })
- }
- }
-
- componentDidMount() {
- this.updateJuniper()
- }
-
- componentDidUpdate() {
- this.updateJuniper()
- }
-
render() {
- const {
- lang,
- title,
- executable,
- github,
- prompt,
- wrap,
- highlight,
- className,
- children,
- } = this.props
- const codeClassNames = classNames(classes.code, className, `language-${lang}`, {
- [classes.wrap]: !!highlight || !!wrap || lang === 'cli',
- [classes.cli]: lang === 'cli',
+ const { lang, title, executable, github, wrap, highlight, className, children } = this.props
+ const codeClassNames = classNames(classes['code'], className, `language-${lang}`, {
+ [classes['wrap']]: !!highlight || !!wrap || lang === 'cli',
+ [classes['cli']]: lang === 'cli',
})
- const ghClassNames = classNames(codeClassNames, classes.maxHeight)
- const { Juniper } = this.state
+ const ghClassNames = classNames(codeClassNames, classes['max-height'])
if (github) {
- return
+
{children}
)
}
- const codeText = Array.isArray(children) ? children.join('') : children || ''
- const highlightRange = highlight ? rangeParser.parse(highlight).filter(n => n > 0) : []
- const rawHtml = ['none', 'cli'].includes(lang)
- ? codeText
- : highlightCode(lang, codeText, highlightRange)
- const html = formatCode(rawHtml, lang, prompt)
return (
<>
- {title && {title}
}
- {html}
+ {title && {title}
}
+
+
+ {children}
+
+
)
}
}
-const JuniperWrapper = ({ Juniper, title, lang, children }) => (
- {
- const { binderUrl, binderBranch, binderVersion } = data.site.siteMetadata
- const juniperTitle = title || 'Editable Code'
- return (
-
-
- {juniperTitle}
-
- spaCy v{binderVersion} · Python 3 · via{' '}
-
- Binder
-
-
-
+const JuniperWrapper = ({ title, lang, children }) => {
+ const { binderUrl, binderVersion } = siteMetadata
+ const juniperTitle = title || 'Editable Code'
+ return (
+
+
+ {juniperTitle}
+
+ spaCy v{binderVersion} · Python 3 · via{' '}
+
+ Binder
+
+
+
-
- {children}
-
-
- )
- }}
- />
-)
-
-const query = graphql`
- query JuniperQuery {
- site {
- siteMetadata {
- binderUrl
- binderBranch
- binderVersion
- }
- }
- }
-`
+
+ {children}
+
+
+ )
+}
diff --git a/website/src/components/copy.js b/website/src/components/copy.js
index e622c0f84..4caabac98 100644
--- a/website/src/components/copy.js
+++ b/website/src/components/copy.js
@@ -1,4 +1,4 @@
-import React, { useState, useRef } from 'react'
+import React, { useState, useRef, useEffect } from 'react'
import Icon from './icon'
import classes from '../styles/copy.module.sass'
@@ -16,7 +16,11 @@ export function copyToClipboard(ref, callback) {
export default function CopyInput({ text, prefix }) {
const isClient = typeof window !== 'undefined'
- const supportsCopy = isClient && document.queryCommandSupported('copy')
+ const [supportsCopy, setSupportsCopy] = useState(false)
+
+ useEffect(() => {
+ setSupportsCopy(isClient && document.queryCommandSupported('copy'))
+ }, [isClient])
const textareaRef = useRef()
const [copySuccess, setCopySuccess] = useState(false)
const onClick = () => copyToClipboard(textareaRef, setCopySuccess)
diff --git a/website/src/components/dropdown.js b/website/src/components/dropdown.js
index ae5c42415..bccf4ccd0 100644
--- a/website/src/components/dropdown.js
+++ b/website/src/components/dropdown.js
@@ -1,17 +1,18 @@
import React from 'react'
import PropTypes from 'prop-types'
import classNames from 'classnames'
-import { navigate } from 'gatsby'
+import { useRouter } from 'next/router'
import classes from '../styles/dropdown.module.sass'
export default function Dropdown({ defaultValue, className, onChange, children }) {
+ const router = useRouter()
const defaultOnChange = ({ target }) => {
const isExternal = /((http(s?)):\/\/|mailto:)/gi.test(target.value)
if (isExternal) {
window.location.href = target.value
} else {
- navigate(target.value)
+ router.push(target.value)
}
}
return (
diff --git a/website/src/components/embed.js b/website/src/components/embed.js
index 9f959bc99..53f4e9184 100644
--- a/website/src/components/embed.js
+++ b/website/src/components/embed.js
@@ -1,11 +1,12 @@
import React, { Fragment } from 'react'
import PropTypes from 'prop-types'
import classNames from 'classnames'
+import ImageNext from 'next/image'
import Link from './link'
import Button from './button'
import { InlineCode } from './code'
-import { markdownToReact } from './util'
+import { MarkdownToReact } from './util'
import classes from '../styles/embed.module.sass'
@@ -57,18 +58,12 @@ SoundCloud.propTypes = {
color: PropTypes.string,
}
-function formatHTML(html) {
- const encoded = encodeURIComponent(html)
- return `${encoded}`
-}
-
-const Iframe = ({ title, src, html, width = 800, height = 300 }) => {
- const source = html ? `data:text/html,${formatHTML(html)}` : src
+const Iframe = ({ title, src, width = 800, height = 300 }) => {
return (