I am writing a Stata .ado program that performs the same subroutine many times. As such, I would like to turn this subroutine into its own sub-program for efficiency and readability. However, this subroutine involves working with and rewriting multiple intermediate datasets that I now save as tempfiles. From my understanding tempfiles cannot be accessed outside of the program they are created in, so any tempfiles I create in the subroutine would not be accessible in the wider program (and vice-versa).

I know I could just save what I save now as tempfiles as regular datasets (.dta), but since I intend for this command to be used by others I’m not sure if it’s best to save additional (potentially large) files to an arbitrary location. What is the best way to handle this problem? Is there a way to save a dataset to the tempfile directory but not have it delete the file and local path automatically at the conclusion of the program? Is it worth trying to rewrite this subroutine as a program at all? I would like to avoid using Frames so the code can run with slightly earlier versions of stata.

Here is a toy example of the issue at play. I would like to access the intermediate datasets (my_temp1, my_temp2) in the large .ado file the program is called in. The following, as expected, gives an invalid file specification error.

Code:
capture program drop my_prog
program my_prog
    sysuse auto
    tempfile my_temp1
    save `my_temp1' // save intermediate tempfiles
    
    sysuse citytemp
    tempfile my_temp2
    save `my_temp2' // save second intermediate file
    end

clear

my_prog

use `my_temp1'
Thanks!