O_DIRECT flag for the file
There is a famous Linus quote that says O_DIRECT is useless, even hell.
If one has a super need for speed, is there really no fair use of this flag in Linux for regular file scenarios?
EDIT: What about “write-only” access to log files?
Solution
You can use fdatasync and posix_fadvise( advice=POSIX_FADV_DONTNEED
fdatasync simply writes your data to the disc, but does not take it out of the cache.
If you write to a log file at a very high speed (and almost never read it), the pages of the log file can often push more useful pages out of the buffer cache. This is not desirable. But with O_DIRECT they won’t.
But to achieve a similar effect, you can use posix_fadvise to require the kernel to discard all cached pages within a given range for a given file (or all cached pages if you prefer).
For example, if you are writing a high-write database and your transaction log fills up at 10 MB/s, you might want to clear cached pages every 100M or so to allow memory to be used more usefully by something else.