With version 1.8.0, REserve offers the capture helper that enables the copy of the response content while processing a request. In this article, an example will be presented in two different flavors to illustrate the new possibilities introduced by this tool.
Caching the content of a site
First, let’s build a local web server with REserve that redirects received requests to an existing remote site thanks to the
url handler. The source file is listed below.
A simple web server built with REserve
Preview of the Face the force today website as seen through the local web server
Now, we improve the web server by introducing a custom mapping that captures some of the responses to save them locally. This new code is inserted before the existing mapping as listed below.
The web server improved with a capturing mapping
Using the request url, the resource extension is extracted and if it matches a list of known types (.ico, .js, .css …), a file path is computed, the corresponding folder is created (if missing) and a write stream is initiated.
Then, the capture helper is called to copy the content of the response to the write stream. This call returns a promise that is resolved when the stream is finished.
Note that this mapping will not answer the request, meaning that the processing will continue to the next mapping (which redirects to the remote site).
After restarting the server and reloading the page in the browser, the cache folder contains all the copied resources. For instance, the next screenshot shows the content of the image folder.
Content of the cache\images folder
If we focus on the request made to grab the file
styles.css, we observe that it was sent with the
gzip content-encoding. This can be verified in the network tab of the debugger as illustrated below.
styles.css request detail
But as you can see in the following notepad screenshot, the cached file is readable as plain text.
cached styles.css content
Another benefit of the capture helper is that it automatically decompresses streams.
The local server is capable of capturing the resources and saving them locally. One last addition to the mappings will reduce the load to the remote site by serving the resources locally if they exist.
This is done by adding a
file mapping as listed below. When the file exists, the mapping will handle the request. If the file does not exist, the
ignore-if-not-found option will tell the handler ignore the request and give a chance to the following mappings to process it.
The web server modified to serve locally cached files
If we look again on the request made to grab the file
styles.css, we notice that it was sent with no encoding (and very few response headers).
cached styles.css request detail
Caching the content of any site
The previous web server works fine but it requires adjustments every time we want to capture another remote site.
On the other hand, it does not need a lot of modifications to be transformed into a simple web proxy server. Like our initial server, web proxies are designed to forward HTTP requests. The request from the client is the same as a regular HTTP request except the full URL is passed.
In the new code below, caching works the same but the server name is used as a root folder under the
Note that the
custom handler function receives capturing group values of the matching regular expression as additional parameters (respectively
The previous code modified to build a web proxy server
Once the server is started, we use the operating system to setup the HTTP proxy. As shown in the next screenshot, Chrome offers a shortcut to access these settings.
On Ubuntu, you need to click the Network Proxy button.
Ubuntu network settings
And then point to the running server, like in the next screenshot.
Ubuntu network settings
Once everything is configured, you can browse any HTTP site and the resources are saved automatically.
Proxy cache folder
The sample given here is far from being perfect and it can be improved to better handle :
- other file types
- optional URL parameters
- cache control headers
- the growing size of the
But the purpose here is to demonstrate how REserve can be useful by providing essential tools and features to quickly build an advanced web site.