feat: add static file server backend type

README.md

@@ -30,10 +30,11 @@ Trickster is a fully-featured Reverse Proxy Cache for HTTP applications like sta
 * Built-in Prometheus [metrics](./docs/metrics.md) and customizable [Health Check](./docs/health.md) Endpoints for end-to-end monitoring
 * [Negative Caching](./docs/negative-caching.md) to prevent domino effect outages
 * High-performance [Collapsed Forwarding](./docs/collapsed-forwarding.md)
-* Best-in-class [Byte Range Request caching and acceleration](./docs/range_request.md).
-* [Distributed Tracing](./docs/tracing.md) via OpenTelemetry, supporting OTLP protocol.
+* Best-in-class [Byte Range Request caching and acceleration](./docs/range_request.md)
+* [Distributed Tracing](./docs/tracing.md) via OpenTelemetry, supporting OTLP protocol
 * Per-backend [Access and Error Logs](./docs/access-logs.md) with Apache-style customizable formats
 * Rules engine for custom request routing and rewriting
+* Built-in [Static File Server](./docs/static.md) for hosting websites and other local content
 
 ## Time Series Database Accelerator
 

deploy/kube/configmap.yaml

@@ -401,6 +401,60 @@ data:
     # Configuration options for mapping Origin(s)
     backends:
 
+      # # example static file server backend, which serves a local directory rather
+      # # than proxying to an origin. paths, req_rewriter_name, origin_url and other
+      # # proxying options are not valid for this provider.
+      # website:
+      #   provider: static
+      #   hosts: [ www.example.com ]
+      #   # authenticator_name optionally requires users to authenticate before any file is served
+      #   # authenticator_name: example_auth_1
+      #   # the static block, and its root, are required when the provider is static
+      #   static:
+      #     # root is the directory holding the content to serve. it must exist
+      #     root: /var/www/html
+      #     # default_file is served when a directory is requested. default is index.html
+      #     default_file: index.html
+      #     # cache_control is the Cache-Control header sent with every file. by default none is
+      #     # sent, and clients judge a file's freshness from its Last-Modified time
+      #     cache_control: no-cache
+      #     # cache_control_by_extension overrides cache_control for matching files
+      #     cache_control_by_extension:
+      #       .js: public, max-age=31536000, immutable
+      #       .css: public, max-age=31536000, immutable
+      #     # response_headers are attached to every response
+      #     response_headers:
+      #       X-Content-Type-Options: nosniff
+      #     # mime_types adds to and overrides the built-in Content-Types by file extension
+      #     mime_types:
+      #       .md: text/markdown; charset=utf-8
+      #     # not_found_file is a file within the root served in place of a plain 404 response.
+      #     # not_found_status is the status it is served with: 404 (the default) for an error
+      #     # page, or 200 for a single-page application whose routes exist only in the browser
+      #     not_found_file: errors/404.html
+      #     not_found_status: 404
+      #     # directory_listing lists a directory that has no default_file, rather than
+      #     # answering it with a 404. default is false
+      #     directory_listing: false
+      #     # cache configures the fileserver cache, which holds small files in memory and drops
+      #     # them when they change on disk. it is separate from, and unrelated to, the caches
+      #     # configured in the caches section, and a static backend does not use a cache_name
+      #     cache:
+      #       # disabled serves every request from disk. default is false
+      #       disabled: false
+      #       # max_file_size_bytes is the largest file held in memory. default is 1048576 (1 MiB)
+      #       max_file_size_bytes: 1048576
+      #       # max_size_bytes is the most memory the cache will use, counting each held file
+      #       # as its size plus 1 KiB of bookkeeping. default is 134217728 (128 MiB)
+      #       max_size_bytes: 134217728
+      #       # max_files is the most objects held (a file, and each compressed rendition of it, is
+      #       # one), which also bounds the directories watched. the least recently used make room
+      #       # for new ones once this or max_size_bytes is reached. default is 10000
+      #       max_files: 10000
+      #       # revalidation_interval is how often held files are compared to disk, as a
+      #       # backstop to filesystem change events. default is 10s
+      #       revalidation_interval: 10s
+
       # # example mysql backend, exposed by a listener with protocol mysql (see the
       # # listeners section). MySQL client connections to the listener are proxied
       # # to the origin. Exactly one mysql backend may map to a mysql listener.
@@ -745,11 +799,15 @@ data:
     #     # The default is false.
     #     multipart_ranges_disabled: false
 
-    #     # compressable_types defines the Content Types that will be compressed when stored in the Trickster cache
-    #     # reasonable defaults are set, so use this with care. To disable compression, set compressable_types: []
+    #     # compressible_types defines the Content Types that will be compressed when stored in the Trickster cache
+    #     # or sent to a client. reasonable defaults are set, so use this with care. To disable compression, set
+    #     # compressible_types: []
+    #     # The encoding used is the one the client's Accept-Encoding header weights highest (q) of those
+    #     # Trickster supports (zstd, br, gzip, deflate). Among equal weights, or when the client gives none,
+    #     # Trickster prefers them in that order. A wildcard (*) stands for the encodings the client didn't
+    #     # name. An encoding the client refuses (q=0), or weights below identity, is never used.
     #     # Default list is provided here:
-    #     compressable_types:
-    #     - text/javascript, text/css, text/plain, text/xml, text/json, application/json, application/javascript, application/xml ]
+    #     compressible_types: [ text/javascript, text/css, text/plain, text/xml, text/json, application/json, application/javascript, application/xml ]
 
     #     # timeout defines how long Trickster will wait before aborting an upstream http request. Default: 60s
     #     timeout: 60s

docs/developer/environment/README.md

@@ -32,6 +32,16 @@ You can stop the developer environment by running `make developer-stop`. To
 delete the developer environment, run `make developer-delete` which will destroy
 all data including named volumes.
 
+## Static File Server
+
+The `static1` backend in `trickster-config/trickster.yaml` serves the files under
+[static-site](./static-site/) with Trickster's [Static File Server](../../static.md)
+backend provider, at <http://127.0.0.1:8480/static1/>. It needs no container. Files
+added or changed under `static-site` are picked up while Trickster runs, with no restart.
+
+The site's `root` is relative to the root of the repo, which is where `make serve-dev`
+runs Trickster from.
+
 ## Graphite
 
 The environment runs a Graphite origin (`graphiteapp/graphite-statsd`: carbon-cache

docs/developer/environment/static-site/index.html

@@ -0,0 +1,12 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>Trickster Static Backend</title>
+</head>
+<body>
+  <h1>Hello, World!</h1>
+  <p>Served by the <code>static1</code> backend of your local Trickster developer instance.</p>
+</body>
+</html>

docs/developer/environment/trickster-config/trickster.yaml

@@ -263,6 +263,12 @@ backends:
       - path: /
         match_type: prefix
         handler: proxycache
+  # static1 serves the files under static-site at http://127.0.0.1:8480/static1/
+  static1:
+    provider: static
+    static:
+      # relative to the root of the repo, where make serve-dev runs Trickster
+      root: docs/developer/environment/static-site
   mecone-rp:
     provider: reverseproxy
     origin_url: 'http://127.0.0.1:8497'

docs/metrics.md

@@ -282,6 +282,37 @@ The following metrics are available only for Caches Types whose object lifecycle
     * `cache_name` - the name of the configured cache$
     * `provider` - the type of the configured cache
 
+The following metrics are available for [Static File Server](./static.md) Backends. Requests they serve are also counted, like those of any other Backend, by the `trickster_frontend_requests_*` metrics with a `provider` of `static`. Their Fileserver cache is separate from the caches above, and is not reported by the `trickster_cache_*` metrics.
+
+* `trickster_fileserver_responses_total` (Counter) - The total number of files served, by how the Fileserver cache figured in the response. Responses that send no file (such as a `404` with no not-found file, a redirect or a directory listing) are not counted.
+  * labels:
+    * `backend_name` - the name of the configured backend
+    * `cache_status` - `hit` (served as it was held), `phit` (the file was held, and was encoded for the response and the rendition then held), `kmiss` (read from disk for the response, and then held) or `disk` (sent from disk without being held, as for a large file, a byte range, a `HEAD` or a `304`)
+    * `encoding` - the encoding of the rendition the file server sent: `identity`, `zstd`, `br`, `gzip` or `deflate`. A response counted as `identity` may still be encoded on its way out, as a large compressible file is.
+
+* `trickster_fileserver_cache_events_total` (Counter) - The total number of objects removed from the Fileserver cache.
+  * labels:
+    * `backend_name` - the name of the configured backend
+    * `event` - `eviction` (the least recently used, removed to make room) or `invalidation` (removed because the file changed on disk, or the cache was stopped)
+
+A backend's series are published only once it is in service, so a configuration that is rejected publishes nothing. They are deleted when a reload removes or renames the backend; across a reload that keeps its name, the counters carry on rather than start over. The four gauges that follow are published only while the backend has a Fileserver cache, and are removed when it is disabled.
+
+* `trickster_fileserver_cache_usage_objects` (Gauge) - The current count of objects in the Fileserver cache, including files being read into it. Each held rendition of a file is an object.
+  * labels:
+    * `backend_name` - the name of the configured backend
+
+* `trickster_fileserver_cache_usage_bytes` (Gauge) - The current accounted size of the Fileserver cache in bytes, which includes each object's bookkeeping allowance.
+  * labels:
+    * `backend_name` - the name of the configured backend
+
+* `trickster_fileserver_cache_max_usage_objects` (Gauge) - The configured `max_files` of the Fileserver cache.
+  * labels:
+    * `backend_name` - the name of the configured backend
+
+* `trickster_fileserver_cache_max_usage_bytes` (Gauge) - The configured `max_size_bytes` of the Fileserver cache.
+  * labels:
+    * `backend_name` - the name of the configured backend
+
 The following metrics are available when the Kubernetes Gateway/Ingress controller is enabled (the top-level `kubernetes` section; see [kubernetes-gateway.md](./kubernetes-gateway.md)):
 
 * `trickster_kgw_reconciles_total` (Counter) - Count of controller reconcile passes, by result