Upgrade guide
0.26.0
Routes port matching
Pomerium’s route matching behavior has changed with regards to port numbers in incoming requests. Previously, when matching an incoming request against the defined routes, Pomerium would require that the request’s Host
(or :authority
) header match the route’s from
URL including any port number. This can cause problems in deployments with a NAT with port mapping in front of Pomerium.
As of v0.26, if a Pomerium route’s from
URL does not include an explicit port number, the matching behavior is more lenient: the route will match an incoming request with any port number. For example, take a route with the from
URL https://app.example.com
. Incoming requests with a host header of app.example.com
, app.example.com:443
, and app.example.com:1234
would all match this route.
However, if you specify a port number explicitly in the from
URL, then incoming requests must include the same port number in the host header in order to match that route.
You can temporarily revert this change in behavior by setting the runtime flag match_any_incoming_port
to false.
Host header rewrite behavior
Pomerium will now consistently rewrite the host header of an incoming request to match the host and port specified in the route to
URL. (Previously Pomerium would never include a port number even if specified, and Pomerium would not rewrite the host header for any to
URLs with a host of localhost
or an IP address.) The new behavior is intended to be more consistent and predictable.
Please set the Preserve Host Header option for any routes where Pomerium should not rewrite the host header.
Improved session refresh reliability
We’ve updated the way Pomerium refreshes OAuth access tokens in order to improve reliability. Previously, Pomerium could fall behind on access token refresh, leading to users being prompted to sign in again before their Pomerium session should have expired. This may result in a higher rate of requests to your configured identity provider.
If you suspect this is causing any problems for your deployment, you can temporarily revert to the previous implementation by setting the runtime flag legacy_identity_manager
to true.
Deprecations
Support for the deprecated client_ca
config file key (and CLIENT_CA
environment variable) is now removed. Please update any remaining usage to downstream_mtls.ca
or the DOWNSTREAM_MTLS_CA
environment variable. See https://www.pomerium.com/docs/reference/downstream-mtls-settings#ca for more information about this option.
0.25.0
Breaking
Base64-encoded Certificates
Previously, the certificates
key supported base64-encoded certificates as a value (this option was not documented). We've removed support for base64-encoded certificates for this setting and now require that you only use the certificate file location. To avoid incompatibilities between versions, please update these values accordingly.
Note: The certificates
key is distinct from the certificate
key. The certificate
key setting still supports base64-encoded certificates; the certificates
list does not.
Remove Debug Option
We've removed support for the Debug setting, which changed the format of logs from JSON to a pretty-print format. If you prefer to review logs in a pretty-print format, you can use a command-line processing tool like jq
.
New
Authentication Flows
In v0.21, we modified the Core authentication flow to support the Hosted Authenticate service. However, this flow posed some limitations for self-hosted deployments (see #4819 for more details). In v0.25, we’ve updated the Core authentication flow so that it’s more versatile: Deployments configured to use the Hosted Authenticate service will use the newer authentication flow introduced in v0.21, while self-hosted deployments will use the older flow.
0.24.0
Breaking
Set Authorization Header
The deprecated set_authorization_header
configuration setting has been removed. You can use the Set Request Headers setting to pass IdP tokens to upstream services in any header.
Base64-encoded Certificates
Previously, the certificates
key supported base64-encoded certificates as a value (this option was not documented). We've removed support for base64-encoded certificates for this setting and now require that you only use the relative file location.
Note: The certificates
key is distinct from the certificate
key. The certificate
key setting still supports base64-encoded certificates; the certificates
list does not.
See Certificates for more information.
Redis Storage Backend
PostgreSQL has been the recommended databroker storage backend since v0.18. Support for Redis has now been removed.
See Persistence for more information.
Performance
v0.24.0 includes several performance enhancements for Pomerium Core. See the sections below for more information.
Policy evaluator reuse
The authorize service maintains one policy evaluator object for each route, which is responsible for all authorization decisions for that route.
Previously, the authorize service would recreate all policy evaluator objects in response to a configuration change.
Now, it will avoid recreating policy evaluator objects for any policies that have not changed, provided that the overall settings are compatible between the old and new configuration. (Some settings affect all routes, e.g. the downstream_mtls options, and changes to these options will still require all policy evaluators to be recreated.)
Parallelization
Core will now build route configuration objects and policy evaluator objects in parallel, each using up to half the number of available CPU cores.
0.23.0
New
Logging Configurations
The new Access Log Fields and Authorize Log Fields settings allow you to customize the fields logged in the access and authorize logs. You can now opt to log ID tokens or specific ID token claims, custom request headers, and the request query params. See the linked reference pages for details.
How the new Logging Configurations work
1. Configure logs
Specify which fields you want to log (omitting the setting will display all the default fields):
# Access logs from Proxy service
access_log_fields:
- authority
- path
# Authorize logs from Authorize service
authorize_log_fields:
- request-id
- method
- path
2. Access a route
For example, Pomerium’s Verify service:
routes:
- from: https://verify.localhost.pomerium.io
to: http://verify:8000
3. Find logs
After you access a route, filter your logs by searching for “http-request”
and “authorize check”
:
// Search for “message”: “http-request”
{
"level": "info",
"service": "envoy",
"authority": "verify.pomerium.com",
"path": "/img/json.svg",
"time": "2023-08-04T12:12:35-04:00",
"message": "http-request"
}
// Search for “message”: “authorize check”
{
"level": "info",
"service": "authorize",
"request-id": "c9afae5a-ec5a-4242-864f-df4189f20e99",
"method": "GET",
"path": "/index.css",
"allow": true,
"allow-why-true": ["domain-ok"],
"deny": false,
"deny-why-false": [],
"time": "2023-08-07T10:26:33-04:00",
"message": "authorize check"
}
New Downstream mTLS Settings
Downstream mTLS refers to the requirement that end users connecting to Pomerium-managed routes must present a trusted client certificate. The options for configuring downstream mTLS have been expanded and moved to a new settings group, under a new downstream_mtls
configuration file key.
The existing Certificate Authority setting has moved from client_ca
to downstream_mtls.ca
. The client_ca
configuration file key will continue to function as an alias for the new setting (but will be removed in a future release).
Support for certificate revocation via CRLs is newly introduced. Please see the CRL reference for more information and some important limitations.
A new Enforcement Mode option has been added, to control the behavior when a client does not present a trusted certificate. The default setting preserves the behavior of previous Pomerium releases: client certificates are required only for user-configured Pomerium routes, and Pomerium will serve an HTML error page for requests without a trusted certificate. The new reject_connection
setting allows for stricter client certificate enforcement: in this mode any attempt to make a TLS connection without a trusted client certificate will be rejected. Naturally, this means that client certificates will be required not only for user-configured Pomerium routes, but also for internal Pomerium routes. This mode allows you to use mTLS as an isolated security layer, entirely separate from Pomerium policy enforcement. Please review the reference page carefully before enabling this mode.
To give further control over which specific client certificates are allowed, Pomerium now also offers a Match Subject Alt Names setting. This allows you to trust only those client certificates containing a Subject Alternative Name (SAN) of a specific type, matching a particular regular expression.
When the new Max Verify Depth option is set, Pomerium will consider client-supplied intermediate CA certificates when verifying a client certificate. The default setting preserves the behavior of previous Pomerium releases: all client certificates must be issued directly by a certificate authority included in the CA setting (no client-supplied intermediate CA certificates are allowed). This default behavior may change in a future release.
Certificate Matcher PPL Criteria (beta)
The new Certificate Matcher can be used with the new PPL criterion client_certificate
to build policies that grant or deny access based on the client certificate’s fingerprint or Subject Public Key Info (SPKI) hash. Both of these options allow you to create an allowlist or denylist of specific certificates.
See the Certificate Matcher policy page for more information and examples.
Set Request Headers options
You can configure the Set Request Headers setting to send the client certificate fingerprint (downstream mTLS must be enabled) to the upstream application or service. The fingerprint can be built into your authorization policy with the new Certificate Matcher (beta) PPL criteria to grant or deny users based on the fingerprint’s value.
See the Set Request Headers settings page for more information and examples.
Deprecated
Set Authorization Header
The Set Authorization Header option is deprecated in favor of the new variable substitution support in the Set Request Headers option. This new support allows you to pass IdP tokens to upstream services in any header, not just the Authorization
header.
The Set Authorization Header option will be removed in a future release.