Transferring parameters between functions

Show a generic wrapper class, for using a function that works for a single 'file' to adapt it to work on directories of files, using automatically the information on options from the original file

Transferring paramters from one function to another

HelpOutputCode

> python examples/adv/param_transfer.py --help
Usage: examples/adv/param_transfer.py [OPTIONS]

╭─ Eager Callbacks ────────────────────────────────────────────────────────────────────────────────╮
│   --help     Display the help message                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
│   --input-dir     Path                                                                           │
│   --param1        str                                                                            │
│   --param2        float                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

> python examples/adv/param_transfer.py --input-dir foo --param1 bar --param2 0.5
input_dir: foo
kwargs: {'param1': 'bar', 'param2': 0.5}

"""
Example of parameter transfer
"""
from pathlib import Path
from typing import Any

from thermite import Config, Event, ObjSignature, process_function_to_obj_signature, run


def process_file(input_file: Path, param1: str, param2: float):
    ...


def process_dir(input_dir: Path, **kwargs):
    print(f"input_dir: {input_dir}")
    print(f"kwargs: {kwargs}")
    # for input_file in input_dir.glob("*"):
    #    process_file(input_file, **kwargs)


def transfer_params(sig: ObjSignature, _: Any):
    proc_file_sig = process_function_to_obj_signature(process_file)

    del sig.params["kwargs"]
    del proc_file_sig.params["input_file"]
    sig.params.update(proc_file_sig.params)

    return sig


if __name__ == "__main__":
    config = Config()
    config.event_cb_deco(Event.SIG_EXTRACT, process_dir)(transfer_params)
    run(process_dir, config=config)

Using JSON configuration files

We can also use a YAML or JSON file to dynamically set the defaults of the CLI. This can be useful for complex data science projects in order not to have to hardcode defaults.

Setting defaults using a config file

HelpConfig fileCode

> python examples/adv/config_file.py --defaults-file examples/adv/config_file.yml#version1 --help
Initialize self.  See help(type(self)) for accurate signature.

Usage: examples/adv/config_file.py [OPTIONS] SUBCOMMAND

╭─ Eager Callbacks ────────────────────────────────────────────────────────────────────────────────╮
│   --defaults-file     Read defaults from file                                                    │
│   --help              Display the help message                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
│   --global-param1     Path     a_path                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────╮
│ cmd1  First subcommand                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

version1:
  opts:
    - ["--global-param1", "a_path"]
  cmds: 
    example1:
      opts:
        - ["--param1", "foo"]
version2:
  cmds: 
    example1:
      opts:
        - ["--param1", "bar"]

"""
CLI with subcommands.
"""
from pathlib import Path
from typing import List

from thermite import run
from thermite.config import Config
from thermite.plugins.default_defs import defaults_cli_callback


class Subcommands:
    def __init__(self, global_param1: Path):
        pass

    def cmd1(self, param1: str):
        """First subcommand"""
        pass


if __name__ == "__main__":
    config = Config(cli_callbacks=[defaults_cli_callback])
    run(Subcommands, config=config)

While not available yet, in a very similar way plugins can be created that use environment variables this way as well.