Modifying the HTTP Response Using Filters

By Scott Mitchell

Introduction

When a browser requests an ASP.NET page from a web server, the ASP.NET engine takes that request through a number of steps that, together, generate the resulting markup, which is returned to the requesting browser for display. The stages in this process are referred to as the HTTP Pipeline and perform tasks like authentication, authorization, and having the requested page render its content. During one of the later stages in the HTTP Pipeline the rendered markup is handed off to a response filter which, if supplied, has an opportunity to inspect and modify the markup before it is returned to the requesting browser.

With a little bit of code you can create your own response filters and associate them with a particular page, a particular type of page (such as ASP.NET resources that generate HTML), or for all ASP.NET resources. And if you are using IIS 7's integrated mode you can have your filter work with the output of any content type. This article provides an overview of response filters and shows two such filters: a naive filter that strips out whitespace to reduce the size of the markup sent over the wire, and a filter that adds a copyright message to the bottom of all web pages. You can download these two filters, along with a simple demo application, at the end of this article, with examples in both C# and Visual Basic.

Read on to learn more!

- continued -

A Quick Primer on Response Filters

Response filters were introduced in ASP.NET version 1.1 and provide developers a means to programmatically inspect and modify the content of a requested resource after said has been generated but before it has been sent back to the client. When an ASP.NET page is requested it is rendered and its output is written to an HttpWriter instance. The HttpWriter object takes the content written to it and, periodically, writes the content out to a stream, which is the data stream returned to the client. For performance purposes, the HttpWriter object buffers the output it sends to the stream in chunks (rather than sending each character, one at a time, to the stream).

A stream is an object that extends the Stream class in the System.IO namespace. In a nutshell, a stream functions as a buffer where data can be written to and read from. There are a number of built in streams in the .NET Framework that use various backing stores for their buffer. For instance, the MemoryStream class is a stream that holds its buffer in memory; a FileStream, on the other hand, implements its buffer as a file on disk.

Streams have an input and an output, an place from which to add data and a place from which to extract data. Data is added to a stream via a call to its Write method and extracted via a call to its Read method. The following diagram shows the stream used by the HttpWriter object - the HttpWriter object writes data from the rendered page into the stream, which is then read and sent down to the client.

The HttpWriter object writes the output to a stream, whose data is read and sent to the requesting client.

One nice feature of streams is that they can be chained. It is possible to hook up two (or more) streams so that the output of one feeds into the input of another. Response filters provide developers a way to chain custom stream with the stream used by the HttpWriter object, as the following diagram shows.

You can chain in a custom stream (or streams) between the HttpWriter object and the output stream.

Using this technique, it's possible to construct a custom stream (or streams) that get a whack at the output before it is sent along to the final stream and returned to the requesting client. These custom streams are referred to as response filters. Response filters are helpful tools for messaging a page's rendered output. They can be used to compress unnecessary whitespace, add boilerplate markup to all web pages (such as copyright notices or JavaScript for web-based tools like Google Analytics), rewrites the page's markup to produce XHTML-compliant pages, or apply compression or encryption to the outgoing data stream. The remainder of this article shows how to create and use a response filter, showcasing two very simple response filters: one that removes whitespace and another that adds a copyright message to the bottom of every page. The complete code for these response filters - in both Visual Basic and C# code - is available for download at the end of this article.

Building a Simple Whitespace Stripping Response Filter

The download available at the end of this article includes two response filter examples. The first one I'd like to explore is the WhitespaceFilter response filter.

The Following Whitespace Stripping Filter Should NOT Be Used in Production

The following filter was created to illustrate how response filters work. It should not be used in a production environment because it overaggressively strips out whitespace. For example, the code removes all carriage returns from the page's rendered output. This can cause problems if you have JavaScript within the page. Consider the following script:

<script type="text/javascript"> // Show a messagebox alert('Hello, world!'); </script>

The above script will get compressed to the following:

<script type="text/javascript">// Show a messagebox alert('Hello, world!');</script>

As you can see, the comment and alert function now appear on the same line; as a result, the alert function is treated as if it's commented out. Feel free to fix the code below to avoid this issue, but be aware that there are a lot of little edge cases like this that make effectively and maximally stripping out whitespace a not so simple endeavor.

As noted earlier in this article, response filters are streams. As a result, when building a response filter you must create a class that extends the Stream class. The Stream class is an abstract class that defines the base functionality for all streams. If you have your response filter inherit from this class directly you will have to implement a number of methods yourself. A better choice is to extend the MemoryStream class, which alreay defines these boilerplate methods. Here is the shell of our response stream, WhitespaceFilterVB. (Download the code from the bottom of this article to see the C# version of this code.)

Imports Microsoft.VisualBasic Imports System.IO Imports System.Text Imports System.Text.RegularExpressions Public Class WhitespaceFilterVB Inherits MemoryStream ... End Class

As illustrated by the diagram above, a chained stream is one that receives input and then outputs it into another stream. Because a response filter serves as a chained stream we need to tell the response filter what stream to output its data to. This is handled by creating a private member variable of type Stream (outputStream) and then creating a constructor that accepts a Stream as input.

Public Class WhitespaceFilterVB Inherits MemoryStream Private outputStream As Stream = Nothing Public Sub New(ByVal output As Stream) outputStream = output End Sub ... End Class

The last step is to override the base class's Write method. The Write method is passed a buffer of bytes ready to be sent to the output stream. From this method we can inspect the buffer and perform any modifications to it before sending it along to the output stream. Keep in mind that this Write method may be called multiple times for a single page. Recall that the HttpWriter object chunks together data and then writes it to its output stream. Each time the HttpWriter sends out a chunk of data your response filter's Write method is invoked.

In the case of a filter that strips whitespace we need the overridden Write method to first convert the passed-in byte buffer into a string, strip out any unecessary whitespace, and then send that stripped output to the output stream. I accomplished this with a couple of regular expressions that remove carriage returns, tabs, and multiple, consecutive spaces.

Public Class WhitespaceFilterVB Inherits MemoryStream ... Private tabsRe As New Regex("\t", RegexOptions.Compiled Or RegexOptions.Multiline) Private carriageReturnRe As New Regex(">\r\n<", RegexOptions.Compiled Or RegexOptions.Multiline) Private carriageReturnSafeRe As New Regex("\r\n", RegexOptions.Compiled Or RegexOptions.Multiline) Private multipleSpaces As New Regex(" ", RegexOptions.Compiled Or RegexOptions.Multiline) Private spaceBetweenTags As New Regex(">\s<", RegexOptions.Compiled Or RegexOptions.Multiline) Public Overrides Sub Write(ByVal buffer As Byte(), ByVal offset As Integer, ByVal count As Integer) ' Convert the content in buffer to a string Dim contentInBuffer As String = UTF8Encoding.UTF8.GetString(buffer) ' Strip out all whitespace... kill all tabs, replace carriage returns with a space, and compress multiple spaces contentInBuffer = tabsRe.Replace(contentInBuffer, String.Empty) contentInBuffer = carriageReturnRe.Replace(contentInBuffer, "><") contentInBuffer = carriageReturnSafeRe.Replace(contentInBuffer, " ") While (multipleSpaces.IsMatch(contentInBuffer)) contentInBuffer = multipleSpaces.Replace(contentInBuffer, " ") End While contentInBuffer = spaceBetweenTags.Replace(contentInBuffer, "><") outputStream.Write(UTF8Encoding.UTF8.GetBytes(contentInBuffer), offset, UTF8Encoding.UTF8.GetByteCount(contentInBuffer)) End Sub End Class

Applying a Response Filter

The Response object has a Filter property that is used to chain together response filters. There are three ways you can set this property:

From a particular web page. From within a web page you can set the Response.Filter property. This affects the specific page that has this code. Add the following code to a page's Page_Load event handler to apply a response filter:

Response.Filter = new ResponseFilterClassName(Response.Filter)

In the case of the WhitespaceFilterVB response filter, the code would look like:

Response.Filter = new WhitespaceFilterVB(Response.Filter)

From an appropriate event handler in Global.asax. Response filters are applied after the HttpApplication object's PostReleaseRequestState event. Therefore, you can add a response filter during this event or earlier. For instance, the following event handler in Global.asax adds the WhitespaceFilterVB filter to all pages with a ContentType of "text/html":

Sub Application_PostReleaseRequestState(ByVal sender As Object, ByVal e As EventArgs) If Response.ContentType = "text/html" Then Response.Filter = New WhitespaceFilterVB(Response.Filter) End If End Sub

From an HTTP Module. HTTP Modules are managed classes that can respond to application events (such as PostReleaseRequestState). An HTTP Module could be used to assign the Response.Filter property.

Download the demo available at the end of this article to see the WhitespaceFilterVB response filter in action. The demo's homepage (Default.aspx) contains two TextBox controls - the first one shows the raw file contents of the WhitespaceFilterDemo.aspx page, which includes carriage returns, tabs, and miscellaneous spaces and clocks in at 2,603 bytes. The second TextBox control on the page shows the content of the WhitespaceFilterDemo.aspx page when viewed through a browser. The extraneous whitespace has been stripped, reducing the page's rendered content down to 2,282 bytes. The screen shot below shows the Default.aspx page in action.

The top TextBox shows the page's contents on disk, whereas the second TextBox shows the rendered content that's sent to the browser. Note that the second TextBox's content has been stripped of extraneous whitespace.

Chainging Together Multiple Response Filters

As we just saw, it's possible to chain a custom response filter between the HttpWriter object and the output stream, but there's nothing that limits you to chaining in just one stream. This article's download includes a second response filter named AddCopyrightFilter that injects a copyright message at the bottom of the pages that the filter has been applied to. Specifically, it adds the following content immediately before the closing body tag (</body>):

<div class="Copyright"> Copyright Scott Mitchell CurrentYear </div>

This concept could be exteneded to injecting other types of boilerplate markup to pages, such as the JavaScript used by web tools like Google Analytics or website hit counters.

The AddCopyrightFilter response filter extends the MemoryStream class and overrides its Write method, but works a little differently than the WhitespaceFilter. Instead of processing each chunk of data that comes into the Write method, AddCopyrightFilter buffers the incoming data into a StringBuilder. Once the buffer contains the content "</html>", it injects the copyright notice and sends the entire buffered text to the next stream in the chain. (This behavior presupposes that the pages it works with end with the text "</html>". If this response filter is used on a page without an ending "</html>" no data will be sent down to the client!)

The AddCopyrightFilter and WhitespaceFilter response filters can be chained together. For example, you can set the Response.Filter property like so:

Response.Filter = new AddCopyrightFilter(new WhitespaceFilter(Response.Filter))

The demo available for download does not use the exact code above, but instead implements the chained response filters by hooking up AddCopyrightFilter in Global.asax and WhitespaceFilter in the WhitespaceFilterDemo.aspx page's code-behind class.

Conclusion

Response filters are a mechanism by which developers can programmatically inspect and modify the outgoing data stream after a web page's content has been rendered but before the data has been returned to the client. Response filters can be used to compress data or modify the outgoing content, as in adding a copyright notice or other boilerplate text. This article showed two response filter examples: one that (naively) stripped whitespace from the rendered HTML and another that added a copyright message to the bottom of all pages the filter applies to.

Happy Programming!

By Scott Mitchell

Attachments:

Download the Response Fitler Code and Demo Application