CLI Docs¶
Warning
The CLI is experimental and may change between versions
dask-yarn¶
Deploy Dask on Apache YARN
usage: dask-yarn [--help] [--version] command ...
-
--help
,
-h
¶
Show this help message then exit
-
--version
¶
Show version then exit
dask-yarn kill¶
Kill a Dask application
usage: dask-yarn kill [--help] APP_ID
-
app_id
¶
The application id
-
--help
,
-h
¶
Show this help message then exit
dask-yarn services¶
Manage Dask services
usage: dask-yarn services [--help] command ...
-
--help
,
-h
¶
Show this help message then exit
dask-yarn services client¶
Start a Dask client process
usage: dask-yarn services client [--help] script [args...]
-
script
¶
Path to a Python script to run.
-
args
¶
Any additional arguments to forward to script
-
--help
,
-h
¶
Show this help message then exit
dask-yarn services scheduler¶
Start a Dask scheduler process
usage: dask-yarn services scheduler [--help]
-
--help
,
-h
¶
Show this help message then exit
dask-yarn services worker¶
Start a Dask worker process
usage: dask-yarn services worker [--nthreads NTHREADS]
[--memory_limit MEMORY_LIMIT] [--help]
-
--nthreads
<nthreads>
¶ Number of threads. Defaults to number of vcores in container
-
--memory_limit
<memory_limit>
¶ Maximum memory available to the worker. This can be an integer (in bytes), a string (like ‘5 GiB’ or ‘500 MiB’), or 0 (no memory management). Defaults to the container memory limit.
-
--help
,
-h
¶
Show this help message then exit
dask-yarn status¶
Check the status of a submitted Dask application
usage: dask-yarn status [--help] APP_ID
-
app_id
¶
The application id
-
--help
,
-h
¶
Show this help message then exit
dask-yarn submit¶
Submit a Dask application to a YARN cluster
usage: dask-yarn submit [--name NAME] [--queue QUEUE] [--user USER]
[--tags TAGS] [--environment ENVIRONMENT]
[--deploy-mode DEPLOY_MODE]
[--worker-count WORKER_COUNT]
[--worker-vcores WORKER_VCORES]
[--worker-memory WORKER_MEMORY]
[--worker-restarts WORKER_RESTARTS]
[--worker-env WORKER_ENV]
[--client-vcores CLIENT_VCORES]
[--client-memory CLIENT_MEMORY]
[--client-env CLIENT_ENV]
[--scheduler-vcores SCHEDULER_VCORES]
[--scheduler-memory SCHEDULER_MEMORY]
[--temporary-security-credentials] [--help]
script [args...]
-
script
¶
Path to a python script to run on the client
-
args
¶
Any additional arguments to forward to script
-
--name
<name>
¶ The application name
-
--queue
<queue>
¶ The queue to deploy to
-
--user
<user>
¶ The user to submit the application on behalf of. Default is the current user - submitting as a different user requires proxy-user permissions.
A comma-separated list of strings to use as tags for this application.
-
--environment
<environment>
¶ Path to the Python environment to use. See the docs for more information
-
--deploy-mode
<deploy_mode>
¶ Either ‘remote’ (default) or ‘local’. If ‘remote’, the scheduler and client will be deployed in a YARN container. If ‘local’, they will be run locally.
-
--worker-count
<worker_count>
¶ The number of workers to initially start.
-
--worker-vcores
<worker_vcores>
¶ The number of virtual cores to allocate per worker.
-
--worker-memory
<worker_memory>
¶ The amount of memory to allocate per worker. Accepts a unit suffix (e.g. ‘2 GiB’ or ‘4096 MiB’). Will be rounded up to the nearest MiB.
-
--worker-restarts
<worker_restarts>
¶ The maximum number of worker restarts to allow before failing the application. Default is unlimited.
-
--worker-env
<worker_env>
¶ Environment variables to set on the workers. Pass a key-value pair like
--worker-env key=val
. May be used more than once.
-
--client-vcores
<client_vcores>
¶ The number of virtual cores to allocate for the client.
-
--client-memory
<client_memory>
¶ The amount of memory to allocate for the client. Accepts a unit suffix (e.g. ‘2 GiB’ or ‘4096 MiB’). Will be rounded up to the nearest MiB.
-
--client-env
<client_env>
¶ Environment variables to set on the client. Pass a key-value pair like
--client-env key=val
. May be used more than once.
-
--scheduler-vcores
<scheduler_vcores>
¶ The number of virtual cores to allocate for the scheduler.
-
--scheduler-memory
<scheduler_memory>
¶ The amount of memory to allocate for the scheduler. Accepts a unit suffix (e.g. ‘2 GiB’ or ‘4096 MiB’). Will be rounded up to the nearest MiB.
-
--temporary-security-credentials
¶
Instead of using a consistent set of TLS credentials for all clusters, create a fresh set just for this application.
-
--help
,
-h
¶
Show this help message then exit