Dump command file export
★ 0.0 (0 ratings)30 min0 engineers solved this
Your assignment
This is a command-line tool for inspecting the results of past runs of data pipelines. Each run has steps, and each step has tasks; every task stores its named artifacts in a datastore. The dump subcommand takes a pathspec of the form run_id/step_name or run_id/step_name/task_id and reads the matching task datastores through TaskDataStoreSet. For each task it either prints a formatted listing of the artifacts to the terminal or, when a file option is given, converts the artifacts to a dictionary keyed by pathspec and pickles them to that file. Display options such as the private flag, the include list and the value size option are passed to the datastore's formatting and dictionary conversion methods. The command is a click command that receives shared state through the click context object, which provides the flow datastore and the quiet flag.
What the team is seeing
Users who export results with the dump command to a file find that large values in the saved file are replaced by short placeholder text instead of their full contents. The same values appear in full when the output is printed to the terminal.
Acceptance checks 1
- Check 1
Discussion is locked — log in and complete this problem to join the discussion and rate your experience.
