I'm rebuilding my blog so I still see requests from old URLs. Some of these I want to redirect to new URLs like the /xml/rss to /atom.xml. Zola doesn't support redirects so I need the Compute service to handle them. In hosting setup I wrote that I use Object Storage for content and a Config store for configuration, but both of these have some drawbacks for handling redirects.

The site content is stored in Object Storage so I'd need the service to perform some special handling to detect if an object fetched from the backend was a page (200 OK) or redirect. Another option is to encode redirects with a special path. Both of these would require special handling in the hit-path which I don't want to do.

A Config store for redirects is another option, but keys are limited to 256 bytes. I shouldn't have any URLs longer than that, but I don't want to limit myself.

I chose to use a KV store for redirects. Keys may be up to 1024 bytes and may have metadata in case I ever want to use a different redirect code than 308 Permanant Redirect. While there are billing considerations for using a KV store, we can mitigate those by caching lookups.

The KV Store allows URL characters as keys and values so I can store the path I want to redirect from as a key and the path I want to redirect to as a value. If a path is not found in the Object Storage bucket then I can look it up the KV Store and return a redirect if found, otherwise proceed to return 404 Not Found.

Setup

I created a blog_segment7_net_redirect KV store and attached it to my service. I then loaded it with some redirects:

module blog {
  const REDIRECTS = [
    [key value];
    [/xml/rss /atom.xml]
    [/articles.rss /atom.xml]
  ]

  def do-redirects [
    domain: string
  ] {
    # Map `blog_segment7_net_redirect` to the Store ID
    let store_id = get-kv-store-id $domain "redirect"

    let redirects = $REDIRECTS
    # Convert each redirect into a JSON entry
    | each {
      {
        key: $in.key
        value: ($in.value | encode base64)
      }
      | to json --raw --serialize
    }
    # Join with a newline
    | str join "\n"
    | fastly kv-store-entry create --store-id $store_id --stdin --add --quiet --json
    | from json

    { redirects: $redirects }
  }

  # Update redirects for the given environment
  export def update-redirects [
    name: string
  ] {
    let environment = select-environment $name

    let domain = $environment.domain
    let service_id = $environment.service_id

    check-FASTLY_API_TOKEN $service_id

    do-redirects $domain
  }
}

Service redirect lookup

Now to look for a redirect in the KV store if we couldn't find the path in the Object Storage bucket:

#[fastly::main]
fn main(mut req: Request) -> Result<Response, Error> {
    // …
    let original_path = req.get_path().to_owned();

    // Send the request to the backend
    let mut beresp = bereq.send(config::BACKEND_NAME)?;

    // If backend response is 404, try for index.html
    if is_not_found(&beresp) && !original_path.ends_with("index.html") {
        // Lookup /path/index.html if /path was not found in object storage…
    }

    // If backend response is still 404
    if is_not_found(&beresp) {
        // Try to redirect
        if let Ok(Some(kv_store)) = KVStore::open("blog_segment7_net_redirect")
            && let Ok(mut location) = kv_store.lookup(&original_path)
            && let Ok(location) =
                HeaderValue::from_str(&location.take_body().into_string())
        {
            return Ok(Response::redirect(location));
        }

        // Lookup up 404 page in bucket and return it
        // …
    }
}

After publishing the service I can test the redirect by running the nushell command http get --full --redirect-mode manual https://blog.segment7.net/xml/rss

Caching KV store lookups

I use the Core Cache API to store the KV store lookups. A cache key may be up to 4KB, much longer than a KV Store key, so I construct one from the original path without any hashing. Since I don't set any stale time for the cached lookup I only have to worry about the found and must-insert cases for a lookup transaction.

Lookups are cached if the KV store can be found and has an entry. Otherwise the transaction is dropped and a 404 is returned.

For debugging the Cache-Status header is set. The response will have whether or not the request was cached and the remaining TTL. We can clear cached redirects by purging the redirect surrogate key, so a long TTL of about a month is OK.

const SURROGATE_REDIRECT: [&str; 2] = ["blog.segment7.net", "redirect"];

const MONTH: Duration = Duration::from_secs(86400 * 35);

#[fastly::main]
fn main(mut req: Request) -> Result<Response, Error> {
    // …

    // If backend response is still 404
    if is_not_found(&beresp) {
        let key = CacheKey::from_owner(format!("redirect-{original_path}"));
        let tx = Transaction::lookup(key).execute()?;

        // Found cached redirect
        if let Some(found) = tx.found()
            && let Ok(body) = found.to_stream()
            && let Ok(location) = HeaderValue::from_str(&body.into_string())
        {
            let res = Response::redirect(location).with_header(
                "Cache-Status",
                format!(
                    "{fastly_hostname}; hit; ttl={}",
                    found.remaining_ttl().as_secs()
                ),
            );

            return Ok(res);

        // If we can get a redirect from the KV store, cache it and return it
        } else if let Ok(Some(kv_store)) = KVStore::open("blog_segment7_net_redirect")
            && let Ok(mut location) = kv_store.lookup(&original_path)
        {
            let location = location.take_body().into_string();

            if let Ok(location_header) = HeaderValue::from_str(&location) {
                if let Ok(mut stream) = tx
                    .insert(MONTH)
                    .surrogate_keys(SURROGATE_REDIRECT)
                    .known_length(location.len() as u64)
                    .execute()
                {
                    stream.write_all(location.as_bytes())?;
                    stream.finish()?;
                }

                let res = Response::redirect(location_header).with_header(
                    "Cache-Status",
                    format!("{fastly_hostname}; fwd=miss; stored=true",),
                );

                return Ok(res);
            }
        }

        // Lookup and return the 404 page
        // …
    }

    // …
}