What happens to files being written to when a program is aborted from the terminal
I am doing some computer modeling of a three body system, and my most recent run looks like it is going to take 4 times longer than I had anticipated (many, many hours), which I cannot afford at the moment. I would like to terminate the program while still retaining the data that has been collected.
The program writes to a .csv file as it runs, but does not close the file until the program ends.
If I just end the program, will the data that has been written to the file so far be accessible or is there any way to make that the case?
I am running the program from debugging in Microsoft Visual Studio.
I am not hugely familiar with c++. I normally use python, but converted the code over to c++ so we could run the program faster (to get better resolution).
The program writes to a .csv file as it runs, but does not close the file until the program ends.
If I just end the program, will the data that has been written to the file so far be accessible or is there any way to make that the case?
I am running the program from debugging in Microsoft Visual Studio.
I am not hugely familiar with c++. I normally use python, but converted the code over to c++ so we could run the program faster (to get better resolution).
Last edited on
it depends somewhat.
on windows, and most unix, the file is usually intact up to the last few lines for text files.
binary files may or may not be readable; you may have to write a fixer or use a hex editor to adjust something (like if it has # of records at the start but only 1/10 of them were written, or removal a partial final record).
You may also have to edit a text file's 'how many lines follow' entry but this should be trivial.
so odds are if you break the program you will get 'most' of your data.
other things to think about: some programs buffer huge amounts and dump to file later in chunks. I do that for efficiency on large programs. So you could lose MANY megabytes of data if you buffer heavily; Ive been known to buffer at a 1GB boundary on modern hardware... You should know how your program is doing such things. C++ auto buffers so most programs just write lines and let it be handled by the auto buffering, in which case you lose only a relatively small amount.
if it takes hours to run, some things to think about:
try a release build. I know you said running in debugger, but is that needed? Debug builds are often much, much slower than highly optimal release builds.
you may also want to figure out where it is slow and look for a tweak. common issues are misuse of heap memory, eg calling the same function in a tight loop where the function creates memory, does stuff, releases memory -- over and over. Its pretty cool how much just a single static keyword can do for a slow program in such cases. Remember that STL things create and destroy memory like mad.
on windows, and most unix, the file is usually intact up to the last few lines for text files.
binary files may or may not be readable; you may have to write a fixer or use a hex editor to adjust something (like if it has # of records at the start but only 1/10 of them were written, or removal a partial final record).
You may also have to edit a text file's 'how many lines follow' entry but this should be trivial.
so odds are if you break the program you will get 'most' of your data.
other things to think about: some programs buffer huge amounts and dump to file later in chunks. I do that for efficiency on large programs. So you could lose MANY megabytes of data if you buffer heavily; Ive been known to buffer at a 1GB boundary on modern hardware... You should know how your program is doing such things. C++ auto buffers so most programs just write lines and let it be handled by the auto buffering, in which case you lose only a relatively small amount.
if it takes hours to run, some things to think about:
try a release build. I know you said running in debugger, but is that needed? Debug builds are often much, much slower than highly optimal release builds.
you may also want to figure out where it is slow and look for a tweak. common issues are misuse of heap memory, eg calling the same function in a tight loop where the function creates memory, does stuff, releases memory -- over and over. Its pretty cool how much just a single static keyword can do for a slow program in such cases. Remember that STL things create and destroy memory like mad.
Last edited on
Topic archived. No new replies allowed.