Optional DVC arguments

There are more DVC Options https://dvc.org/doc/command-reference/run#options that can be important to some workflows. In the following we show how to use them with DVC.

External

For dvc run --external the following can be used:

from zntrack import Node, dvc, zn, config, ZnTrackProject
from pathlib import Path
import tempfile

config.nb_name = "OptionalDVC.ipynb"

project = ZnTrackProject()
project.create_dvc_repository()

2022-01-13 21:16:34,655 (INFO): Setting up GIT/DVC repository.

temp_file = tempfile.NamedTemporaryFile()

class WriteExternal(Node):
    file: Path = dvc.outs()
    data = zn.params()

    def __init__(self, data: str = "", file: str = "", **kwargs):
        super().__init__(**kwargs)
        self.data = data
        self.file = Path(file)

    def run(self):
        self.file.write_text(self.data)

    def read_text(self):
        print(self.file.read_text())

2022-01-13 21:16:35,929 (CRITICAL): DeprecationWarning for __init__: This Option was moved to zntrack.zn.params (Deprecated since v0.3)

WriteExternal(data="HelloWorld", file=temp_file.name).write_graph(
    external=True, no_exec=False
)

2022-01-13 21:16:35,944 (WARNING): Jupyter support is an experimental feature! Please save your notebook before running this command!
Submit issues to https://github.com/zincware/ZnTrack.
2022-01-13 21:16:35,945 (WARNING): Converting OptionalDVC.ipynb to file WriteExternal.py
2022-01-13 21:16:38,256 (WARNING): --- Writing new DVC file! ---
2022-01-13 21:16:38,257 (WARNING): You will not be able to see the stdout/stderr of the process in real time!
2022-01-13 21:16:40,544 (INFO): Running stage 'WriteExternal':
> python -c "from src.WriteExternal import WriteExternal; WriteExternal.load(name='WriteExternal').run_and_save()"
2022-01-13 21:16:40,095 (CRITICAL): DeprecationWarning for __init__: This Option was moved to zntrack.zn.params (Deprecated since v0.3)
Creating 'dvc.yaml'
Adding stage 'WriteExternal' in 'dvc.yaml'
Generating lock file 'dvc.lock'
Updating lock file 'dvc.lock'

To track the changes with git, run:

        git add dvc.yaml dvc.lock

WriteExternal.load().read_text()

HelloWorld

temp_file.close()

no-commit

For dvc run --no-commit the interface is similar and looks like:

class HelloWorld(Node):
    def run(self):
        pass


HelloWorld().write_graph(no_commit=True)

2022-01-13 21:16:40,581 (WARNING): Jupyter support is an experimental feature! Please save your notebook before running this command!
Submit issues to https://github.com/zincware/ZnTrack.
2022-01-13 21:16:40,582 (WARNING): Converting OptionalDVC.ipynb to file HelloWorld.py
2022-01-13 21:16:42,974 (WARNING): --- Writing new DVC file! ---
2022-01-13 21:16:44,379 (INFO): Adding stage 'HelloWorld' in 'dvc.yaml'

To track the changes with git, run:

        git add dvc.yaml

For dvc run --no-exec you can pass no_exec=True. This is the default value, because Experiments are usually queued and then run collectively if the full graph was build.