Managing Endpoints

Starting an Endpoint

To initiate a single Endpoint, execute the command endpointsctl start endpoint. You can utilize the environment variable ENDPOINTS_ENDPOINT_ID to uniquely identify each Endpoint.

endpointsctl start pod $ENDPOINT_POD_ID

To commence multiple Endpoints in bulk, utilize the command endpointsctl create endpoints.

For instance, to start up to 10 Endpoints named bulk-task utilizing 3070 and 3080 GPUs and execute a bash command, input the following command.

endpointsctl create endpoints \
  --name bulk-task \
  --gpuType "NVIDIA GeForce RTX 3070,NVIDIA GeForce RTX 3080" \
  --imageName "endpointsai/your-image-name" \
  --containerDiskSize 10 \
  --volumeSize 0 \
  --mem 1 \
  --args "bash -c 'your-bash-command'"