Skpr provides development teams with a robust system for configuring their applications.
This blog post covers the considerations that went into the design and development of that system.
What Is Configuration?
Configuration can be a loaded term for infrastructure, we define configuration as the values used to modify the behaviour of an application.
These behaviours are typical:
- Connecting to services eg. database, search, external api etc
- Flags to enable/disable features
- Provide context to an application for where it is running. eg. dev/stg/prod
Existing Patterns
Before building our configuration system we evaluated the cloud hosting ecosystem as a whole to find existing patterns.
In this section, we cover two common patterns we've observed.
1) Include Files
The hosting platform provides a file that is language/application type sensitive that can be included in the application's codebase eg. Acquia provides a settings.php which can be included by a Drupal application.
This include file only configures the bare minimum to avoid conflicts with the application. This includes:
- Database connection
- Filesystem locations
If you have deployed an application on Acquia or Pantheon than you have most likely had to use this pattern before.
# Inheriting the platform's configuration.
require '/var/www/my-application/settings.php';
# Setting my own custom configuration.
$settings['googleanalytics_account'] = 'UA-XXXXXXXX-X';
The example above demonstrates how this pattern can lead to a simple configuration solution. The downside is that you need to manage any custom configuration yourself.
The typical workaround we have seen is for teams to hardcode these configurations in the application and commit them to the code base.
This example from Pantheon's documentation demonstrates how a development team might set a custom configuration.
if (PANTHEON_ENVIRONMENT == 'dev') {
// Google Analytics.
$conf['googleanalytics_account'] = 'UA-XXXXXXXX-X';
}
else if (PANTHEON_ENVIRONMENT == 'test') {
// Google Analytics.
$conf['googleanalytics_account'] = 'UA-XXXXXXXX-Y';
}
else if (PANTHEON_ENVIRONMENT == 'live') {
// Google Analytics.
$conf['googleanalytics_account'] = 'UA-XXXXXXXX-Z';
}
This leads to security and development workflow issues as:
- Credentials are now committed to the codebase and accessible on developer workstations, code hosting services, CI/CD pipelines, etc.
- Configuration changes need to be deployed eg. an API Token update
In summary, this approach works great until custom configuration is required. Development teams then need to make compromises on security and workflow to have a custom configuration set.
2) Environment Variables
Environment variables are a popular approach for configuring applications and are supported by a wide range of hosting providers. They are also very easy to integrate into an application's codebase.
For example:
$conf['googleanalytics_account'] = getenv('GOOGLEANALYTICS_ACCOUNT');
Acquia, Lagoon and Pantheon all have support for environment variables. However, they all have varying workflows to support them:
- Acquia provides a web interface
- Lagoon and Pantheon provide this functionality via lower level tooling eg. command-line interface, API interface and codebase "commit" options
While environment variables are convenient, they do come with the downside that the application must be restarted or redeployed for an update to take effect. While this should not lead to downtime, it could under certain conditions.
A potential condition would be setting the configuration 10 times requiring 10 restarts or redeployments on the platform. While this shouldn't be a problem with rolling updates and robust traffic routing configurations, we were still concerned that this could happen.
One method to combat the restart/redeploy downside is to set the configuration (via the web, command line or API interface) and wait for the next deployment for the configuration update to take effect. While this will solve the issue, it creates a new one - predictability.
We found that coupling configuration updates to deployments led to confusion eg.
- Is the configuration already applied?
- Are we still waiting for a deployment?
- Are there other configuration values which have been set that are waiting for a deployment?
With all of the above in mind, we ultimately chose not to use environment variables for our configuration system.
Developer Experience Matters
While all the cloud providers mentioned in this blog post provide a solution, we felt they lacked a great developer experience.
Developers deserve a simple set of tools to interact with, and certainty when setting their configuration.
Skpr's Solution
Taking the above patterns into consideration, we have built a system with a focus on:
- Avoiding application restarts/redeployments to apply configuration
- Improved Developer Experience
Developer Experience
Skpr provides a simple command-line interface for developers to set, list and delete the configuration.
Setting Configuration
Configuration can be created or updated using the set command:
# Setting a non-secret config.
$ skpr config set prod api.username nickschuch
# Setting a secret.
$ skpr config set prod --secret api.password xxxyyyzzz
Listing Configuration
Developers can view which configuration is set for each environment, this example demonstrates which config is set for production.
$ skpr config list prod
───────────────────────── ───────────────────
KEY (26) VALUE
───────────────────────── ───────────────────
api.username nickschuch
api.password [secret]
Delete Configuration
Configuration can also be deleted using the delete command:
$ skpr config delete prod api.username
$ skpr config delete prod api.password
How Do We Avoid Restarts?
Skpr provides applications with a JSON file containing all the configuration provided by both the platform and the developer.
We chose a JSON file because:
- It's language agnostic
- Can be automatically updated without restarts/redeployments
Below is an example of the JSON file which can be loaded by an application.
$ cat /etc/skpr/data/config.json | jq
{
"api.username": "nickschuch",
"api.password": "xxxyyyzzz"
}
The file is typically much larger given it contains, by default, connection details for the following services:
- MySQL
- Solr/Elasticsearch
- ClamAV
- SMTP
The Skpr platform team maintain a set of packages (PHP, Node, Ruby and Go) to smooth out the integration between the Skpr configuration system and the application.
Integrating It Into Your Site
Below is an example using the PHP package with a Drupal application.
use Skpr\SkprConfig;
$skpr = SkprConfig::create()->load();
# Simple example.
$conf['api_username'] = $skpr->get('api.username');
$conf['api_password'] = $skpr->get('api.password');
# Advanced example (with fallback).
$databases['default']['default'] = [
'driver' => 'mysql',
'database' => $skpr->get('mysql.default.database') ?: 'local',
'username' => $skpr->get('mysql.default.username') ?: 'local',
'password' => $skpr->get('mysql.default.password') ?: 'local',
'host' => $skpr->get('mysql.default.hostname') ?: '127.0.0.1',
'port' => $skpr->get('mysql.default.port') ?: '3306',
];
Extending This Solution
This solution isn't just for setting API keys etc. It can also be used to provide additional context about the environment to which the application is deployed.
Hostnames
Skpr also exposes a list of hostnames as a JSON file.
$ cat /etc/skpr/data/hostnames.json | jq
[
"dev.example.cluster.skpr.dev"
]
This allows for applications like Drupal 8+ to automatically configure its trusted_host_patterns setting based on the environment.
foreach ($skpr->hostNames() as $hostname) {
$settings['trusted_host_patterns'][] = '^' . preg_quote($hostname) . '$';
}
IP Ranges
Environments also have access to a list of internal IP ranges which can be useful when determining external addresses eg. the client IP address.
The following example demonstrates how easy it is to integrate with Drupal.
$settings['reverse_proxy_addresses'] = $skpr->ipRanges();
Environment Variables
The PHP library also transforms Skpr configuration into environment variables for each request using putenv.
This allows for a frictionless experience when migrating applications with environment variables configuration to the Skpr platform.
# Using the library.
$conf['api_username'] = $skpr->get('api.username');
$conf['api_password'] = $skpr->get('api.password');
# Using environment variables.
$conf['api_username'] = getenv('API_USERNAME');
$conf['api_password'] = getenv('API_PASSWORD');
Resources
The following resources were used during the writing of this blog post.