1 of 64

Vostok.Configuration

Home

Vostok.Configuration in a nutshell

Vostok.Configuration is a set of libraries offering configuration tools to .NET applications. It handles fetching configuration data from files or APIs, parsing and converting it to user-defined model classes.

Guiding design principles

Configuration data sources should be composable regardless of storage formats.
- It should be easy to combine settings from JSON files, environment variables and custom APIs. This is achieved with concept and universal abstraction.
"Hot" configuration updates should be easy to leverage.

Features

and methods to obtain settings;
Support for and settings;
Support for custom settings ;

Guarantees

See / settings scenarios, and sections for details.

Good starting points

Quickstart

Here's the simplest way to experience Vostok.Configuration for the first time:

Install main, abstractions and sources.json modules:

Install-Package Vostok.Configuration
Install-Package Vostok.Configuration.Abstractions
Install-Package Vostok.Configuration.Sources.Json

Define a settings model class:

class MySettings 
{
    public string Key1 { get; }
    public string Key2 { get; }
}

Create a JSON file with settings:

Obtain a model instance with a from a file and print it:

Great job! Here are some possible next steps:

Learn the most basic concepts: , , , ;
Explore available and ;
Go over the .

Concepts and basics

Settings nodes

Settings nodes are the intermediate representation of configuration data and one of the core concepts of the library. Their main purpose is to abstract away various configuration formats (JSON, XML, etc.) so that can be implemented once without regard to the nature of being used.

Key points

A settings node is a tree with string keys and string values in its leaf nodes.

Value nodes

Value nodes are key-value pairs with optional keys. They cannot have child nodes and thus are always the leaves of settings node trees. Standalone values are rare: most of the time value nodes can be found inside objects or arrays.

Value nodes are typically mapped to primitive types during binding.

Properties

Property

Equality

Two value nodes are considered equal if their values match exactly and their names match up to differences in case.

Representation

Array nodes

are containers used to represent sequences, such as JSON arrays. Each array node contains an ordered list of child nodes. There is no limit to nesting: arrays can contain other arrays and objects. Elements of an array are not required to have names.

Array nodes are typically mapped to ordered collections during .

Properties

Object nodes

are containers used to represent objects with named fields/properties. Each object node contains a map of child nodes with their names as keys. There is no limit to nesting: objects can contain other objects and arrays. Elements of an object are required to have non-null names.

Object nodes are typically mapped to arbitrary classes and structs during .

Properties

Settings nodes merging

Merge is the operation of reducing two to one: left + right --> result.

Its primary use is to .

API

Each node type implements a Merge method that accept another node (right in our terms) and an instance of merge options with following properties:

Settings nodes scoping

Scoping is the operation of navigating a tree by accessing child nodes of via names in a case-insensitive manner. A sequence of names resembling a path in the object structure, such as ["property1", "property2"] is called a scope.

Scoping does not work on and nodes (always results in null).

Scoping is used to map object fields/properties to nodes of settings tree during . It also allows to — create a source that returns a subtree of settings returned by the original source.

Binding nodes to models

Binding is the process of initializing a model (an instance of almost arbitrary type) with data from a obtained from a :

node {A: 1, B: 2} --> new CustomModel { A = 1, B = 2}

The resulting model is queried by the application code with a to access settings.

Binders

Binding is implemented by a set of , each of which knows how to convert a

Configuration sources

Configuration sources fetch data from storage (local files or remote APIs) and convert it to settings nodes, abstracting away actual data formats such as JSON or YAML.

They are not meant to be consumed directly and should be used in conjunction with a configuration provider (see assign sources to types and obtain settings from provider scenarios).

Sources are also responsible for data change detection. They expose a reactive interface with subscription support:

IObservable<(ISettingsNode settings, Exception error)> Observe();

On each update, triggered either periodically or by an internal event, the source emits a pair: (settings, null) on success or (null, error) on failure. It's not required to deduplicate settings or errors at this level, although it's never wrong to do so.

Sources must never block indefinitely while waiting for data and should rather publish null settings after a short initial timeout.

Sources must be thread-safe and should be designed to support multiple concurrent observers. It is also expected that every new observer would immediately receive a notification with current state upon subscription.

Here are some of the often used source implementations:

It's also possible to .

Configuration provider

Overview

is responsible for the and offers methods to obtain final settings models. It's also responsible for and . Providers are used directly by the application code to either or .

Caching and performance

cache settings for each (type, source) pair where sources are compared by reference. Caching ensures a solid performance level: only the first Get call is somewhat expensive while all the subsequent ones are extremely cheap. The cache is automatically updated when the underlying issues new data.

Due to caching, instances should be reused as much as possible. Ideally there should be just one singleton instance in the application.

Error handling

This page describes how deal with errors arising from and .

It can be summarized in two simple rules:

If an error occurs and no correct settings instance has been observed for the requested type thus far, the error is propagated to the calling code, resulting in exceptions from Get method. A subsequent settings update with correct data automatically "heals" future Get calls.

Sources.Json

Description

This library contains configuration implementations: and . It also exposes a configuration for integration with other sources, such as .

Command line source

Location: main sources module.

CommandLineSource converts CLI arguments into settings nodes.

It supports 7 syntax options for key-value parameters:

--key=value
--key value
-key=value
-key value
/key=value
/key value
key=value

Keys with dots (such as a.b.c) are treated as hierarchical and get split into segments:

Multiple occurrences of the same key are merged into arrays.

This source is static and never issues data updates.

Default keys and values

Standalone keys may optionally be supplied with a default value.

Standalone values may optionally be grouped under default key.

Environment variables source

Location: main sources module.

EnvironmentVariablesSource converts process environment variables into settings nodes.

var source = new EnvironmentVariablesSource();

Keys with dots, colons and double underscores are treated as hierarchical and get split into segments:

throttling.capacity=50 --> { "throttling": { "capacity": "50" } }
throttling:capacity=50 --> { "throttling": { "capacity": "50" } }
throttling__capacity=50 --> { "throttling": { "capacity": "50" } }

This source is static and never issues data updates.

Configuration sources

Obtain settings from provider

Requires: main module.

Settings can be obtained on-demand with configuration provider's Get method.

With prior assignment of sources:

var settings = provider.Get<MySettings>();

With sources passed on per-call basis:

Get method behavior

Always returns the most recent version of settings (updates may happen in background):
- Call Get on every settings access for "hot" configuration;
- Call Get once and cache the result for "cold" configuration;
First call for a type may block or due to source latency, data unavailability or incorrect data format. This behavior persists until a data update remedies the error;
Once warmed up, subsequent calls never block or throw errors and are extremely cheap due to . Future errors are not propagated to the calling code, but . Get calls return the last seen correct settings object;

Observe settings via provider

Requires: main module.

One can subscribe to settings updates for a type with configuration provider's Observe method.

With prior assignment of sources:

With sources passed on per-call basis:

Temporary subscriptions should be disposed of as they might hold on to resources in sources:

Observe method behavior

OnError and OnCompleted notifications are never produced. Only successful settings updates are propagated to the observers. This means that there will be no notifications if initial attempt to provide settings and no further data updates are published by the .
- The only way to notice errors when using Observe is to .

Best practices

It's recommended to obtain initial settings instance with Get method before subscribing to updates with Observe. This practice ensures correct error propagation, warms up the and eliminates situations where the settings are not ready yet on access attempt.

Print settings

Requires: main module.

Settings models can be printed in JSON or YAML formats to generate configuration file examples.

var printSettings = new PrintSettings 
{
    Format = PrintFormat.Json
};

var defaultSettings = new MySettings();

var printed = ConfigurationPrinter.Print(defaultSettings);

Console.Out.WriteLine(printed);

Log errors

Requires: logging module.

See error handling for cases where this logging is useful.

See logging documentation for the log interface required here.

var providerSettings = new ConfigurationProviderSettings();

providerSettings = providerSettings.WithErrorLogging(log);

var provider = new ConfigurationProvider(providerSettings);

Secret settings should be marked with an attribute to avoid being logged.

Make settings required

Requires: abstractions module.

By default all settings are optional; that is, absence of relevant data in the source results in default values during binding and does not produce errors.

Some parts of configuration may be crucial to the application to the point that it's pointless to start without them being initialized. These fields and properties should be annotated with [Required] attribute:

class MySettings 
{
    [Required]
    public string DbConnectionString { get; }
    
    [Required]
    public MySecrets Secrets { get; }
}

class MySecrets 
{
    public string ApiKey { get; }
    public string Login { get; }
    public string Password { get; }
}

Changing defaults

Default behavior can be inverted in the scope of a type with [RequiredByDefault] attribute. Individual fields and properties can then be made optional with [Optional] attribute:

Transform sources

Requires: sources module.

This operation allows to modify source contents by applying a custom delegate. Most often this involves rewriting values stored in value nodes:

class MyTransformer : ValueNodeTransformer
{
    public MyTransformer()
        : base(node => new ValueNode(node.Name, node.Value?.Replace(..., ...)))
    {
    }
}

var transformedSource = source.Transform(new MyTransformer());

Configuration sources Settings nodes Use value substitutions

Vostok.Configuration

Home