Intent


The initial publishing of these files is in the form of a web project and a windows process. The key components of the
solution are an HttpModule, a peice of code for staging the web role LocalResource, and a console application with code
to upload the files from a local directory into storage. The eventual goal is to morph the HttpModule into a project item
for VS and to convert the storage upload code into VS extension; with any luck it will eventually get there.

About the Solution and Projects


This code can be downloaded and run to test and the individual pieces may be pulled out and added to your own project. The
LocalCacheModule.cs file contains two primary pieces of code: 1) HttpModule to resolve and serve content requests and
2) LocalStorageCacheManager class that is used by the global.asax file to fetch the Azure Storage content and store it in
the configured local resource. The LoadStorage project contains the code to move the files from the local machine to Azure
Storage. There is no direct dependency between LoadStorage and the web role runtime pieces (LocalStorageCacheManager
and LocalStorageCacheModule). The indirect link is that the runtime pieces can only fetch and serve content that was previously
staged by either the LoadStorage process or another process.

Configuration Settings and Runtime Dependencies


LoadStorage

LoadStorage is used to stage content from a local (web) folder to Azure Storage. It requires 4 application settings
to be configured to work properly. Those settings can be found in App.config and are:

1) CloudStorageConnectionString -- string used to connect to Azure Storage; it should be the same as the same named setting in
settings of the web role
2) LocalStorageConnectionString -- used to connect to local storage and is usually assigned the value "UseDevelopmentStorage=true";
like the first setting this should match in name and value the setting in the settings of the
web role
3) ResourcesRoot -- this is the path that points to the root of the files to be uploaded from your web app to Azure
Storage; you will usually want the last folder to be the root of the resources folder as
specified for a given URI. For example, the value might be set to
"C:\inetpub\wwwroot\TestWeb01\Resources" where the final folder "resources" is the root of
files such as images, css, js, etc., that you might want to push to Storage and cache locally
in the web role. Thus, a request for image01.jpg would be
"http://localhost/webapp/resources/image01.jpg" from the point of resources the paths match.
4) TargetContainer -- This should be set to be the name of the target container. The setting "AzureStorageCacheContainer"
in the web role settings should be the same. To be consistent I would use the a name the same as
the root folder, in this case "resources", but it could be anything as long as it matches in both
configuration settings

Once these configuration settings are assigned proper values you should be able to set the LoadStorage project as the startup project
and run it to upload all of the files from your local web resources folder to the configured container name.

LocalStorageCacheManager

Initially the code contained here was placed directly into the global.asax, but for future use and addition of methods the code
was moved into the class LocalStorageCacheManager which is currently a class defined in the LocalStorageCacheModule.cs; I expect to
refactor it to another file eventually. The staging of the local cache content currently happens when the app starts up and is not
threaded or refreshed. Hopefully, I will eventually get reworking this to happen on a background thread at start-up and provide
mechanisms for refresh and expiry.

This part of the code is responsible for reaching out to the Storage container that it is configured to look in and download all of the files.
It relies on settings in the web role much the same as the HttpModule. Those settings are as follows:

1) LocalStorageConnectionString -- This is the connection string for the dev storage. It is always set to "UseDevelopmentStorage=true"

2) LocalFileCache -- This is the name of the Local Storage that you will need to create. It will be used as the endpoint to
save the files locally for the web role once the app is started. If you have a Local Storage defined
as "FileCache" then you would also assign this setting (LocalFileCache) the value of "FileCache"

3) AzureStorageCacheContainer -- This is the container that serves as the source for the files that are going to be pulled to the local
cache. It should match the value of the TargetContainer setting for the LoadStorage app.

4) CloudStorageConnectionString -- This is the connection string for the actual Azure Storage account. It should match the value for the
same LoadStorage setting.
e.g., value="DefaultEndpointsProtocol=http;AccountName=yourAccountName;AccountKey=yourAccountKey"

For this to work as coded the container will have to be Public. If LoadStorage created the container it will be Public. If it was created
by another means you'll need to ensure that it is Public. Files that are downloaded from storage and cached locally are URL encoded as to
make the filename a valid file system filename. As such for a file flower.jpg that is in the relative path "/resources/images/" would have
a qualified name that included its relative path from the root folder. The "/resources" part of the path should match the container name.
Thus, the container name/root follder "/resources" is stripped from the path, the remainder path including the filename is URL encoded, and
finally it is saved to the local cache. For a file that is found by the URI "www.myweb.com/resources/images/flower.jpg" the first part of
the path should match the container name and is stripped and the rest is encoded thus leaving us with a local file named
"images%2fflower.jpg".

LocalStorageCacheModule

The job of this piece is to inspect requests coming in and determine if the requested resource is one that is contained in the Local Cache.
This should be set to the parent folder that contains the resources. It uses the same settings as described above for the
LocalStorageCacheManager with addition of one other setting:

MappedPath -- This is the value that defines what requests to which the HttpModule attempts to respond. It does a
simple search of the existence of the string the the request URI. As such, if the web URL is
www.myweb.com and the URI is www.myweb.com/resources/scripts/main.js and the value of MappedPath is
"/resources" the HttpModule will find "/resources" in the URI and look in its local cache for a file
name "scripts%2fmain.js" />

For the HttpModule to work it will need to be added to the web.config. That config entry looks like this:

<add name="LocalCache" type="AzureFileCacheWeb.LocalStorageCacheModule, AzureFileCacheWeb"></add>

So, if you extract the module code and use it be sure to add the entry into your web.config.

Other Notes

I had hoped to be further along before publishing, but hopefully this will still be helpful to others. There is some complexity in setup
as the settings need to be set properly and certain ones match (as noted above). Currently, there is no tooling to help with setup,
inclusion in your project, or consistent assigment of settings for each of the pieces.

The next step is to refactor the CacheManager and HttpModule pieces into separate files and also pull those two into a separate project that
is outside of the web project. I will update this document when that work is done.

Last edited May 5, 2010 at 6:24 PM by jofultz, version 1

Comments

No comments yet.