Downloading results without data to save bandwidth

My mission consists of a python script, a requirements file and a large data file (a few hundred MB jsonl file). The result of the computation is a small .jsonl file (kb), which is all I’m interested in.

Is it possible to download the result but leave the data on the mission (on the cargo bay?). This would avoid uploading 500MB, only to download 500MB at the end, when I really only need to download a few kb.

I tried navigating into the Results folder, but it contains a zip which can’t be inspected, only downloaded as far as I can see.

Is there some configuration for what gets included in “Results”?

Hey @alvaro,

Thanks for the question. In the next week we will be releasing a feature that will allow you to be able to optionally set which files or filetypes you want returned to you upon completion of your job.

As for your current situation, we currently only collect data that is generated by your script in the /home/galileo directory. So if you are using your own Dockerfile and not one generated for you by Galileo, then you’ll need to be sure to set your working directory (WORKDIR) to be /home/galileo.

Also, until our new mini-feature is released, you could add some additional logic to your python script to delete all unnecessary data from your runtime folder after your calculations are complete, that way they would not be returned to you in your results folder.

Todd

Thank you @todd, looking forward to the update.
I’ll go with os.remove("datafile") for now.