Arc Forumnew | comments | leaders | submitlogin
1 point by venizio 5687 days ago | link | parent

hello,

i was missing the flush upon close. so for instance in this scenario:

1. write

2. close (flush happens, buffers sit on kernel space but not disk)

3. rename

4. a soft-update commit happens so the rename is already out on disk.

5. process crashes

the big question is then: will the os then eventually sync the data flushed in step 2? even after a process crash? i would think yes, but not sure.

an os crash at this point will probably just zero the file out. http://www.freebsd.org/doc/en/books/handbook/configtuning-di... (11.12.2.1)

i get your point regarding performance. arc does it right. it basically leverages file system magic to sync asynchronously at the (small) risk of zeroing out files on an os crash or power failure.

thanks.



1 point by CatDancer 5687 days ago | link

Yes, once the operating system has the data it will write it out to disk, regardless of what happens to the process.

-----

1 point by venizio 5687 days ago | link

out of curiosity, do you know what is the backup policy? thanks @CatDancer.

-----

1 point by CatDancer 5687 days ago | link

the backup policy? of what?

-----

[deleted]
1 point by CatDancer 5687 days ago | link

Is your question, "how do I back up my data files written by my Arc program?"

In Unix, rename is an atomic operation in the kernel. Renaming a file does not immediately delete the old contents of the file, instead, the old contents are deleted when no process no longer has the file open.

Say you have some file "data" which contains "foo". You have some process B such as tar or rsync which is reading your files to back them up. Meanwhile Arc is writing "bar" to "data".

B opens "data" for reading.

Arc creates "data.tmp" and writes "bar" to it. Then Arc renames "data.tmp" to "data".

A process which now opened "data" would read "bar" from it. However B is still reading the previous version of the file, and so reads "foo".

Thus the backup process will always read either the complete previous version of the file, or the complete next version of the file.

(Whether or not the data has already been physically written out to disk makes no difference to what the processes see).

-----