API Reference

This section provides detailed documentation for all Chiltepin modules. Click on any function, class, or method to view its documentation and source code.

Configuration Module

chiltepin.configure.create_executor(name: str, config: Dict[str, Any], client: Client | None = None) → ParslExecutor[source]

Create an Executor specified by the given resource configuration

Parameters:

name (str) – The name of the resource
config (Dict[str, Any]) – YAML configuration block that contains the resource’s configuration
client (Client | None) – A Globus Compute client to use when instantiating Globus Compute resources. The default is None. If None, one will be instantiated automatically for any Globus Compute resources in the configuration. Only applies to Globus Compute resources

Return type:

ParslExecutor

chiltepin.configure.create_globus_compute_executor(name: str, config: Dict[str, Any], client: Client | None = None) → GlobusComputeExecutor[source]

Construct a GlobusComputeExecutor from the input configuration

Parameters:

name (str) – A label that will be assigned to the returned GlobusComputeExecutor for naming purposes
config (Dict[str, Any]) –
YAML configuration block that contains the following configuration options:

Option key Default value “endpoint id”: No default, this option is required “mpi” False “max mpi apps”: 1 “mpi_launcher”: “srun” for Slurm, otherwise “mpiexec” “provider”: “localhost” “cores per node”: 1 “nodes per block”: 1 “init blocks”: 0 “min blocks”: 0 “max blocks”: 1 “exclusive”: True “partition”: “” “queue”: “” “account”: “” “walltime”: “00:10:00” “environment”: []
client (Client | None) – The Globus Compute client to use for instantiating the GlobusComputeExecutor. If not specified, Globus Compute will instantiate and use a default client.

Return type:

GlobusComputeExecutor

chiltepin.configure.create_htex_executor(name: str, config: Dict[str, Any]) → HighThroughputExecutor[source]

Construct a HighThroughputExecutor from the input configuration

Parameters:

name (str) – A label that will be assigned to the returned HighThroughputExecutor for naming purposes
config (Dict[str, Any]) –
YAML configuration block that contains the following configuration options:

Option key Default value “provider”: “localhost” “cores per node”: 1 “nodes per block”: 1 “init blocks”: 0 “min blocks”: 0 “max blocks”: 1 “exclusive”: True “partition”: None “queue”: None “account”: None “walltime”: “00:10:00” “environment”: []

Return type:

HighThroughputExecutor

chiltepin.configure.create_mpi_executor(name: str, config: Dict[str, Any]) → MPIExecutor[source]

Construct a MPIExecutor from the input configuration

Parameters:

name (str) – A label that will be assigned to the returned MPIExecutor for naming purposes
config (Dict[str, Any]) –
YAML configuration block that contains the following configuration options:

Option key Default value “max mpi apps”: 1 “mpi_launcher”: “srun” for Slurm, otherwise “mpiexec” “provider”: “localhost” “cores per node”: 1 “nodes per block”: 1 “init blocks”: 0 “min blocks”: 0 “max blocks”: 1 “exclusive”: True “partition”: None “queue”: None “account”: None “walltime”: “00:10:00” “environment”: []

Return type:

MPIExecutor

chiltepin.configure.create_provider(config: Dict[str, Any]) → ExecutionProvider[source]

Create the appropriate ExecutionProvider from the given configuration

Parameters:

config (Dict[str, Any]) –

YAML configuration block that contains the following configuration options. Not all options are valid for all providers.

Options for all providers:

Option key Default value “mpi” False “provider”: “localhost” “init blocks”: 0 “min blocks”: 0 “max blocks”: 1 “environment”: []

Options for Slurm provider: “cores per node”: 1 “nodes per block”: 1 “exclusive”: True “partition”: None “queue”: None “account”: None “walltime”: “00:10:00”

Options for PBSPro provider: “cores per node”: 1 “nodes per block”: 1 “queue”: None “account”: None “walltime”: “00:10:00”

Return type:

ExecutionProvider

chiltepin.configure.load(config: Dict[str, Any], include: List[str] | None = None, client: Client | None = None, run_dir: str | None = None) → Config[source]

Return a Parsl Config initialized by a list of Executors created from the input configuration dictionary.

The Config object returned by this function is used in parsl.load(config)

Parameters:

config (Dict[str, Any]) – YAML configuration block that contains the configuration for a list of resources
include (List[str] | None) – A list of the labels of the resource configurations to load. The default is None. If None, all resource configurations are loaded. Otherwise the configurations for resources whose labels are in the list will be loaded.
client (Client | None) – A Globus Compute client to use when instantiating Globus Compute resources. The default is None. If None, one will be instantiated automatically for any Globus Compute resources in the configuration.
run_dir (str | None) – The directory to use for runtime files. The default is None, which means Parsl’s default runinfo directory location will be used.

Return type:

Config

chiltepin.configure.parse_file(filename: str) → Dict[str, Any][source]

Parse a YAML resource comfiguration file and return its contents as a dict

Parameters:: filename (str) – Name of configuration file to parse
Return type:: Dict[str, Any]

Endpoint Management Module

chiltepin.endpoint.configure(name: str, config_dir: str | None = None, timeout: float | None = None) → bool[source]

Configure a Globus Compute Endpoint

This is a thin wrapper around the globus-compute-endpoint configure command. Additional configuration steps, usually done manually by the user after configuration, are taken to hide complexity from the user.

Parameters:

name (str) – Name of the endpoint to configure
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is to be stored. If None (the default), then $HOME/.globus_compute is used
timeout (float | None) – Number of seconds to wait for the command to complete before timing out Default is None, meaning the command will never time out.

chiltepin.endpoint.delete(name: str, config_dir: str | None = None, timeout: float | None = None)[source]

Delete the specified Globus Compute Endpoint

This is a thin wrapper around the globus-compute-endpoint delete command

Parameters:

name (str) – Name of the endpoint to delete
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used
timeout (float | None) – Number of seconds to wait for the command to complete before timing out Default is None, meaning the command will never time out.

chiltepin.endpoint.exists(name: str, config_dir: str | None = None) → bool[source]

Return True if the endpoint exists, otherwise False

Parameters:

name (str) – Name of the endpoint to check
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used

Return type:

bool

chiltepin.endpoint.get_chiltepin_apps() -> (<class 'globus_sdk.globus_app.app.GlobusApp'>, <class 'globus_sdk.globus_app.app.GlobusApp'>)[source]

Log in to the Chiltepin app

This instantiates GlobusApp objects for use in creating Globus Compute and Globus Transfer clients. If the environment contains settings that specify client ids and/or client secrets, those will be used to create the Globus Apps. Otherwise, the default Chiltepin thick client will be used. If a secret is present in the environment, ClientApp objects will be created. Otherwise, UserApp objects will be created. This is used by the login() and logout() functions where login and logout flows are initiated after the apps are retreived. A tuple is returned where the first item is the compute app and the second item is the transfer app.

Return type:: (GlobusApp, GlobusApp)

chiltepin.endpoint.is_running(name: str, config_dir: str | None = None) → bool[source]

Return True if the endpoint is running, otherwise False

Parameters:

name (str) – Name of the endpoint to check
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used

Return type:

bool

chiltepin.endpoint.login() → Dict[str, Client | TransferClient][source]

Log in to the Chiltepin app

This initiates the Globus login flow to log the user in to the Globus compute and transfer services. The login will use the registered Chiltepin thick client by default, or the client id and/or secret specified in the environment. This returns a Globus Compute client and a Globus Transfer client in a dictionary. Those clients can then be used for accessing those services.

Return type:: Dict[str, Client | TransferClient]

chiltepin.endpoint.login_required() → bool[source]

Check whether a chiltepin login is required to use the requested Globus scopes needed by the Chiltepin transfer and computer Apps.

Return type:: bool

chiltepin.endpoint.logout()[source]

Log out of the Chiltepin app

This logs the user out of the Globus compute and transfer services and revokes all credentials associated with them.

chiltepin.endpoint.show(config_dir: str | None = None) → Dict[str, Dict[str, str | None]][source]

Return a dictionary of configured Globus Compute Endpoints

This returns endpoint information in a dict with keys corresponding to the endpoint names.

Parameters:: config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used
Return type:: Dict[str, Dict[str, Optional[str]]]

chiltepin.endpoint.start(name: str, config_dir: str | None = None, timeout: float | None = None)[source]

Start the specified Globus Compute Endpoint

This is a thin wrapper around the globus-compute-endpoint start command

Parameters:

name (str) – Name of the endpoint to start
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used
timeout (float | None) – Number of seconds to wait for the command to complete before timing out Default is None, meaning the command will never time out.

chiltepin.endpoint.stop(name: str, config_dir: str | None = None, timeout: float | None = None)[source]

Stop the specified Globus Compute Endpoint

This is a thin wrapper around the globus-compute-endpoint stop command

Parameters:

name (str) – Name of the endpoint to stop
config_dir (str | None) – Path to endpoint configuration directory where endpoint information is stored. If None (the default), then $HOME/.globus_compute is used
timeout (float | None) – Number of seconds to wait for the command to complete before timing out Default is None, meaning the command will never time out.

Tasks Module

Task decorators for Chiltepin workflows.

This module provides decorators for defining workflow tasks that can be executed on configured resources. Tasks are the fundamental units of work in Chiltepin workflows.

Available Decorators

python_task(): Execute Python functions as workflow tasks
bash_task(): Execute shell commands as workflow tasks
join_task(): Coordinate multiple tasks without blocking workflow execution

For comprehensive usage examples and best practices, see the Tasks documentation.

Examples

Define a simple Python task:

from chiltepin.tasks import python_task

@python_task
def add_numbers(a, b):
    return a + b

# Execute on a specific resource
result = add_numbers(5, 3, executor="compute").result()

Define a bash task:

from chiltepin.tasks import bash_task

@bash_task
def list_files(directory):
    return f"ls -la {directory}"

# Returns exit code (0 = success)
exit_code = list_files("/tmp", executor="compute").result()

class chiltepin.tasks.MethodWrapper(func, wrapper_func)[source]

Bases: object

Wrapper that preserves method behavior for decorated functions.

This descriptor ensures that when a decorated function is accessed as a method, it properly creates a bound method with the instance.

chiltepin.tasks.bash_task(function: Callable) → Callable[source]

Decorator function for making Chiltepin bash tasks.

The decorator transforms the function into a Parsl bash_app but adds an executor argument such that the executor for the function can be chosen dynamically at runtime.

Parameters:: function (Callable) – The function to be decorated to yield a Bash workflow task. This function can be a stand-alone function or a class method. If it is a class method, it can make use of self to access object state. The function must return a string that contains a series of bash commands to be executed.
Return type:: Callable

chiltepin.tasks.join_task(function: Callable) → Callable[source]

Decorator function for making Chiltepin join tasks.

The decorator transforms the function into a Parsl join_app. A parsl @join_app decorator accomplishes the same thing. This decorator is added to provide API consistency so that users can use @join_task rather than @join_app along with @python_task and @bash_task.

Parameters:: function (Callable) – The function to be decorated to yield a join workflow task. This function can be a stand-alone function or a class method. If it is a class method, it can make use of self to access object state. The function is expected to call multiple python or bash tasks and return a Future that encapsulates the result of those tasks.
Return type:: Callable

chiltepin.tasks.python_task(function: Callable) → Callable[source]

Decorator function for making Chiltepin python tasks.

The decorator transforms the function into a Parsl python_app but adds an executor argument such that the executor for the function can be chosen dynamically at runtime.

Parameters:: function (Callable) – The function to be decorated to yield a Python workflow task. This function can be a stand-alone function or a class method. If it is a class method, it can make use of self to access object state.
Return type:: Callable

Data Module

Data transfer and management for Chiltepin workflows.

This module provides specialized tasks for transferring and deleting data between Globus data transfer endpoints. These tasks integrate seamlessly with Parsl workflows.

Available Tasks

transfer_task(): Transfer files/directories between Globus data endpoints
delete_task(): Delete files/directories from Globus data endpoints

Available Functions

transfer(): Synchronous data transfer using Globus
delete(): Synchronous data deletion using Globus

For comprehensive usage examples and best practices, see the Data Transfer and Management documentation.

Examples

Stage, process, and cleanup data in a workflow:

from chiltepin.data import transfer_task, delete_task
from chiltepin.tasks import python_task

@python_task
def process_data(input_path):
    # Process the data
    return result

# Stage data to compute resource
stage = transfer_task(
    src_ep="my-laptop",
    dst_ep="hpc-scratch",
    src_path="/data/input.dat",
    dst_path="/scratch/input.dat",
    executor="local"
)

# Process the staged data
result = process_data("/scratch/input.dat", executor="compute", dependencies=stage)

# Clean up
cleanup = delete_task(
    src_ep="hpc-scratch",
    src_path="/scratch/input.dat",
    executor="local",
    dependencies=result
)

chiltepin.data.delete(src_ep: str, src_path: str, timeout: int = 3600, polling_interval: int = 30, client: TransferClient | None = None, recursive: bool = False)[source]

Delete data synchronously with Globus.

This deletes data from a Globus endpoint. This function will not return until the deletion has completed or failed.

Parameters:

src_ep (str) – Name of the source endpoint for the data to be deleted. Can be a display name or a UUID string.
src_path (str) – Path to the file or directory on the source endpoint that is to be deleted.
timeout (int) – Number of seconds to wait for the deletion to complete.
polling_interval (int) – Number of seconds to wait between checking the status of the deletion
client (TransferClient | None) – Transfer client to use for submitting the deletion. If None, one will be retreived via the login process. If a login has already been performed, no login flow prompts will be issued. NOTE: Yes, deletion is done using a TransferClient.
recursive (bool) – Whether or not a recursive deletion should be performed

chiltepin.data.delete_task(src_ep: str, src_path: str, timeout: int = 3600, polling_interval: int = 30, client: TransferClient | None = None, recursive: bool = False)[source]

Delete data asynchronously in a Parsl task

This wraps synchronous Globus data deletion into a Parsl python_app task. Calling this function will immediately return a future. The result of the future will be True if the deletion completed successfully, or False if it did not.

Parameters:

src_ep (str) – Name of the source endpoint for the data to be deleted. Can be a display name or a UUID string.
src_path (str) – Path to the file or directory on the source endpoint that is to be deleted.
timeout (int) – Number of seconds to wait for the deletion to complete.
polling_interval (int) – Number of seconds to wait between checking the status of the deletion
client (TransferClient | None) – Transfer client to use for submitting the deletion. If None, one will be retreived via the login process. If a login has already been performed, no login flow prompts will be issued. NOTE: Yes, deletion is done using a TransferClient.
recursive (bool) – Whether or not a recursive deletion should be performed

chiltepin.data.transfer(src_ep: str, dst_ep: str, src_path: str, dst_path: str, timeout: int = 3600, polling_interval: int = 30, client: TransferClient | None = None, recursive: bool = False)[source]

Transfer data synchronously with Globus

This performs a Globus transfer of data from one Globus transfer endpoint to another. This function will not return until the transfer completes or fails.

Parameters:

src_ep (str) – Name of the source endpoint for the transfer. Can be a display name or a UUID string.
dst_ep (str) – Name of the destination endpoint for the transfer. Can be a display name or a UUID string.
src_path (str) – Path to the file or directory on the source endpoint that is to be transferred.
dst_path (str) – Path to the file or directory on the destination endpoint where the data is to be transferred.
timeout (int) – Number of seconds to wait for the transfer to complete.
polling_interval (int) – Number of seconds to wait between checking the status of the transfer
client (TransferClient | None) – Transfer client to use for submitting the transfers. If None, one will be retreived via the login process. If a login has already been performed, no login flow prompts will be issued.
recursive (bool) – Whether or not a recursive transfer should be performed

chiltepin.data.transfer_task(src_ep: str, dst_ep: str, src_path: str, dst_path: str, timeout: int = 3600, polling_interval: int = 30, client: TransferClient | None = None, recursive: bool = False)[source]

Transfer data asynchronously in a Parsl task

This wraps synchronous Globus data transfer into a Parsl python_app task. Calling this function will immediately return a future. The result of the future will be True if the transfer completed successfully, or False if it did not.

Parameters:

src_ep (str) – Name of the source endpoint for the transfer. Can be a display name or a UUID string.
dst_ep (str) – Name of the destination endpoint for the transfer. Can be a display name or a UUID string.
src_path (str) – Path to the file or directory on the source endpoint that is to be transferred.
dst_path (str) – Path to the file or directory on the destination endpoint where the data is to be transferred.
timeout (int) – Number of seconds to wait for the transfer to complete.
polling_interval (int) – Number of seconds to wait between checking the status of the transfer
client (TransferClient | None) – Transfer client to use for submitting the transfers. If None, one will be retreived via the login process. If a login has already been performed, no login flow prompts will be issued.
recursive (bool) – Whether or not a recursive transfer should be performed

API Reference

Configuration Module

Endpoint Management Module

Tasks Module

Available Decorators

Data Module

Available Tasks

Available Functions

Command-Line Interface