The .NET Universe revealed! - techniques for the .NET Developer

Have you never wondered how you could stream into a file while reading from it in a separate process or application.  I know I have.  And first I'll tell you why.

I'm making a CMS (Content Management System) of my own where I want my resources (files, images, ...) to be stored in the database.  Imagine this database and the server serving the website having a not so superfast connection, what happens if a user requests a file, and only a few milliseconds later another one requests the same file.  The first request is not completed yet, so the file could not have been cached to disk yet.  What would the second request do?  Get another connection to the SQL server and start another stream from the database slowing down the first request even more.  What if the second request could just start reading from the bytes that where already on the server, and while the new bytes are coming in just streaming those to the client as they come in.

In reality the connection between your SQL and webserver isn't really slow, but the files can be big enough to pose the same issue (Wanting to read a file that has already started streaming to the same server).  In reality you would not let the first request handle the download itself, but just let it start a thread that would start the download and then start reading from the file in the request thread.  And the same counts for all consecutive requests to the same file.

I've been trying out several ways to achieve this goal and I found that a normal FileStream could do the job (partly).
The FileStream can read a file that is being written to if you specify the FileShare parameter of both the streams to FileShare.ReadWrite and FileAccess.ReadWrite
Why ReadWrite? I have to admit that I don't know the answer to that, but the fact is that the Reading stream also needs Write access in order to do the reading and writing simultaneously.

There is one drawback, if the file is not written to disk before the reading stream catches up, the stream just thinks the file is complete and finishes.  I first tried working out a Copy method that involved retrying, but that wasn't really usefull for most applications where you want to pass your stream to other third party code that doesn't know about your retrying.  Hence I made my own GrowingFileStream class that has its own system of trying to read from the file untill a timeout is exceeded.  It is made very easy to make differences between a filestream for Reading or for Writing.  A boolean configures the default FileStream this way that the stream will ony do exactly that what you require from it.  Because it could give strange behaviour when you start writing to a stream that was meant for reading a file that is incomplete.  The code throws an exception if you try to do the opposite.  You may descide yourself if you want this functionality, but I figured it would come out handy.

Below you see the most important part of the code, the reading code.  The writing code is not changed from the default FileStream's Write method.

public override int Read(byte[] array, int offset, int count) {
            CheckForReading();
            int bytesRead = base.Read(array, offset, count);
            DateTime begin = DateTime.Now;
            while(_realSize > 0 && this.Length != _realSize && bytesRead == 0) {
                bytesRead = base.Read(array, offset, count);
                TimeSpan tp = DateTime.Now - begin;
                if(tp.TotalMilliseconds >= this.ReadTimeout) {
                    if(_throwsException) {
                        throw new TimeoutException(ERROR_TIMEOUT_EXCEEDED);
                    }
                    break;
                }
            }
            return bytesRead;
        }

As you can see the Read method will try to re-Read if no bytes where found, and it will retry this untill the ReadTimeout exceeds.  The behaviour then can be different, If you specified the ThrowsExceptionOnTimeout as false it just stops reading, but if you leave it to True it will throw a TimeoutException by default.  The other most important thing to do right is, like I said before, use the ReadWrite enum parameters correctly.  But if you download my class then this is done correctly because the GrowingFileStream class doesn't let you specify these parameters that are required to be of a specific value for this purpose.

Feel free to download and use my class found below, and if you like it (or don't) let me know your thoughts in the comments.  Please leave the link to my blog intact.  Just in case you want to find back this information after a few months or if you pass it to someone else.  It's the only thing I ask