I should specify, I am new to stata, so if i misuse terminology, please forgive me.
I have several big .dta files in a folder. I cant have all of them in memory together so I'm going to loop over them. What I hope to do is avoid unnecessary i/o.
Is there any way to save (in memory only) the output from each file in a frame, append those frames and then write/save this final frame only? I an unable to append a frame that is only in memory.
Currently I have to save the output of each file, so later when I load these files again, I would have to merge them.
Code:
clear all
local files : dir "path/to/the/folder" files "*.dta"
cd "path/to/the/folder"
display `"`files'"'
foreach file of local {
use "`file'", clear
collapase (sum) variable1, by (variable2)
save `"new_`file'.dta"', replace
}Code:
clear all
local files : dir "path/to/the/folder" files "*.dta"
cd "path/to/the/folder"
display `"`files'"'
frame create `final_frame' //everything will be appended to this
foreach file of local {
use "`file'", clear
collapase (sum) variable1, by (variable2)
frame rename default, to_append
frame use final_frame
frame final_frame : append using `to_append'
}
save "final_frame.dta", replace
0 Response to loop over files in a folder to do an operation and save the combined result as dataset
Post a Comment