Envprobe

Envprobe (envprobe) is a shell hook tool that helps you manage your environment variables on a per-shell basis easily. It allows for doing so without requiring typing in clunky export sequences, or manually source-ing who-knows-where hand-written script files.

Audience

Envprobe is provided for power users of Unix-derivative systems, especially for software developers. If your day-to-day life on your computer does not involve juggling shell configuration, you will not find much use of this tool.

Comparison to existing tools

Envprobe was conceived to be the tool between some already existing ones that allow managing your environment.

Scripts can be evaluated in a running shell with source. This is a cumbersome operation, as the scripts need to be maintained, especially when one is still writing the initial version.
Shell Modules (module load) allow loading pre-installed versions of software and apply their environment configuration to the local shell. These modulefiles are to be written usually by package maintainers and are distributed with the installed tools and allow the user to dynamically load or unload the “availability” of a tool.
direnv loads and applies the export directives (specified in the .envrc files) in the context of the current directory, and its parents.

Envprobe provides the environment variable modification experience inside the current shell, with no need of manually writing configuration files. You can change your environments on the fly and if the setup works, Envprobe can save the configuration for you, and load it later.

Documentation

Install

Envprobe requires at least Python 3.6 to be installed on the system. Apart from Python and one of the supported POSIX-compatible shells, there are no additional dependencies.

Obtaining Envprobe

You can download Envprobe’s official releases from PyPI. pip will automatically install Envprobe to an appropriate location for your local user.

pip install envprobe

From GitHub

You can download Envprobe from the official repository, either using Git or a tar download. Extract the downloaded archive to any location comfortable. In the documentation, we will use ~/envprobe as the location where Envprobe is installed to.

Downloading via Git

git clone http://github.com/whisperity/envprobe.git ~/envprobe \
    --origin upstream --single-branch --branch master --depth 1

Downloading the current version from GitHub as a release

mkdir ~/envprobe
wget http://github.com/whisperity/envprobe/tarball/master -O envprobe.tar.gz
tar xzf envprobe.tar.gz --strip-components=1 -C ~/envprobe/

Setting up the shell hook

Envprobe can work its “magic” and apply the changes to the running shell’s environment through a hook which is executed every time a prompt is generated. This hook must be registered for the executed shell before using Envprobe. The easiest way to have the hook registered is by adding the invocation of Envprobe’s hook generator to the configuration file of the shell you are using.

Warning

Envprobe’s hook execution at the start of the shell involves considering the environment as-is at that moment to be the initial state used by the saved snapshots feature. Because scripts loaded after Envprobe can change the state of environment and result in otherwise unintended, automated changes picked up by Envprobe as if the user made them, it is well-advised to load Envprobe last.

Bash

Put the following code as-is (including quotes, etc.) at the end of ~/.bashrc:

# If obtained from PyPI:
eval "$(envprobe config hook bash $$)";

# If obtained from GitHub:
eval "$(~/envprobe/envprobe config hook bash $$)";

Zsh

Stock Zsh

Put the following code as-is (including quotes, etc.) at the end of ~/.zshrc:

# If obtained from PyPI:
eval "$(envprobe config hook zsh $$)";

# If obtained from GitHub:
eval "$(~/envprobe/envprobe config hook zsh $$)";

Zsh with Oh-My-Zsh

If you are using Oh-My-Zsh to manage your Zsh, create a new file ~/.oh-my-zsh/custom/zzzzzz_envprobe.zsh with the following contents as-is (including quotes, etc.):

# If obtained from PyPI:
eval "$(envprobe config hook zsh $$)";

# If obtained from GitHub:
eval "$(~/envprobe/envprobe config hook zsh $$)";

Testing the hook

If Envprobe is successfully installed, the hook code itself will register the shell functions envprobe, and envprobe-config, and their shorthand aliases ep, and epc, for the main mode and config mode, respectively.

After adding the hook script to your configuration, start a new shell, and type in ep. If something similar to the following is visible on the screen (instead of a “bash: Command not found” or a “python: No module named” error), Envprobe is working as intended:

$ ep
usage: envprobe [-h] ...

Officially supported configuration

Below are the configuration combinations that the continuous integration testing is done for. However, due to Envprobe being a straightforward tool, other distributions are expected to work fine.

Operating system	Required dependencies	Shells supported
Ubuntu 18.04 LTS (Bionic Beaver)	Python ≥ 3.6	Bash (≥ 4.4), Zsh (≥ 5.4)
Ubuntu 20.04 LTS (Focal Fossa)	Python ≥ 3.8	Bash (≥ 5.0), Zsh (≥ 5.8)

Quick overview

This page contains excerpts of Envprobe’s usage that detail typical everyday tasks which are helped by the tool.

Table of Contents

Managing environment variables in the current shell
Saved snapshots
Type-safe access

Managing environment variables in the current shell

(Read the full documentation for this section.)

$ ep get USER
USER=root

$ ep USER
USER=root

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

$ echo $SOME_VARIABLE
# No result, variable is not defined.
$ ep SOME_VARIABLE=MyValue
$ echo $SOME_VARIABLE
MyValue

$ ep ^SOME_VARIABLE
$ echo $SOME_VARIABLE
# No result.



$ fancy
fancy: command not found!

$ ep +PATH /opt/fancy/bin
$ fancy
Fancy tool works!

$ ep PATH
PATH=/opt/fancy/bin:/usr/local/bin:/usr/bin:/sbin:/bin

$ pwd
/root

$ ep -PATH /opt/fancy/bin
$ ep PATH+ .
$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin:/root

$ ep +PATH ..
PATH=/:/usr/local/bin:/usr/bin:/sbin:/bin:/root

Saved snapshots

(Read the full documentation for this section.)

$ ep %
# (No output, initially the environment hasn't been changed yet.)

$ ep PATH
PATH=/usr/local/bin:/usr/bin/:/sbin:/bin

$ ep SOME_VARIABLE=foo
$ ep +PATH /tmp
$ ep -PATH /sbin
$ ep PATH+ /home/user/bin

$ ep %
(+) Added:       SOME_VARIABLE
        defined value: foo

(!) Changed:     PATH
        added:         /tmp
        added:         /home/user/bin
        removed:       /sbin

$ ep } mypath PATH
For variable 'PATH' the element '/tmp' was added.
For variable 'PATH' the element '/home/user/bin' was added.
For variable 'PATH' the element '/sbin' was removed.

$ ep } other_vars -p
New variable 'SOME_VARIABLE' with value 'foo'.
Save this change? (y/N) _



$ ep list
mypath
other_vars

$ ep delete mypath
$ ep list
other_var



$ ep load custompaths
For variable 'PATH' the element '/srv/custom/bin' will be added.

$ ep PATH
PATH=/srv/custom/bin:/tmp:/home/user/bin

$ ep { foobar -n
New variable 'FOO' will be created with value 'bar'.

$ ep FOO
FOO is not defined

$ ep { foobar -p
New variable 'FOO' will be created with value 'bar'.
Load and apply this change? (y/N) _

Type-safe access

(Read the full documentation for this section.)

Prohibit passing non-numbers to number-expecting variables.

$ echo $SSH_AGENT_PID
12345

$ export SSH_AGENT_PID="invalid-value"
# The above example works, even though a "_PID" variable should only
# contain numbers.

$ ep SSH_AGENT_PID=98765
$ ep SSH_AGENT_PID="foo"
[ERROR] Failed to execute: could not convert string to number.

$ ep SSH_AGENT_PID
SSH_AGENT_PID=98765

Convenient handling of array-like environment variables (e.g. PATH).

$ ep add USER foo
[ERROR] Failed to execute: 'add' can not be called on non-arrays.

$ ep add PATH /foo

$ ep PATH
PATH=/foo:/bin:/sbin

Environment variable management (`main` mode)

Envprobe’s main entry point is called the main mode, which is responsible for managing environment variables. This is the mode that is interfaced with most of the time.

If Envprobe is properly installed, the shell will have the envprobe command, and its shorthand alias, ep, defined. Calling ep automatically brings up the argument parser for main mode.

The main mode is also unique in offering shortcuts to further lessen the amount of typing needed to interface with the tool.

Attention

The list of available subcommands for a mode is generated in a context-sensitive fashion. This means that in case the current context does not allow the execution of an action, Envprobe will usually fail with an invalid choice error.

Managing environment variables

This page details all the user-facing commands in the main mode which deal with reading and writing environment variables. The changes to the variables are applied to the environment of the shell the command was executed in.

All commands are offered through shortcuts to ease access to the core functions.

Note

These commands are only available if Envprobe has been hooked in the current shell.

Reading (`get`, `?`)

get(VARIABLE, info=False)

Read and print the value of the environment variable VARIABLE to the standard output in the format VARIABLE=value.

Parameters

VARIABLE – The name of the environment variable to query.
info (True if -i or info is given) – Whether to print additional information.

Possible invocations

ep get [-i|--info] VARIABLE
ep ?VARIABLE
ep VARIABLE 1

Examples

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

$ ep get -i PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

PATH=
      /usr/local/bin
      /usr/bin
      /sbin
      /bin

Type: 'path'

Priting additional information:

If -i/--info is given, additional information about the variable will be printed after the initial print of the value. This additional information includes:

The individual elements of the variable in order if it is an array variable (Array), after the variable name repeated, one per line.
The type class of the variable within Envprobe.
Additional stored information such as a description of the variable, based on the stored configuration or the downloaded community knowledge-base.

1: The shorthand format ep VARIABLE for ep get VARIABLE is available only if the given variable name ("VARIABLE") is not “shadowed” by a subcommand name that is valid at the time the command is executed. E.g. if get is an environment variable defined in the shell, saying ep get will not be resolved as if the user said ep get get, but instead, it will simply call ep get without a variable name, resulting in an error. As most environment variables are named in SCREAMING_SNAKE_CASE, this should not pose an issue except in the rarest of situations.

Writing (`set`, `!`, `=`)

set(VARIABLE, VALUE)

Set the value of VARIABLE to the specified VALUE.

Parameters

VARIABLE – The name of the environment variable to set.
VALUE – The new value to set to.

Possible invocations

ep set VARIABLE VALUE
ep !VARIABLE VALUE
ep VARIABLE=VALUE

Examples

$ echo $SOME_VARIABLE
# No result, the variable is not set.

$ ep set SOME_VARIABLE MyValue

$ echo $SOME_VARIABLE
MyValue

$ which ls
/bin/ls

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

$ ep PATH="/tmp"

$ which ls
# No result.

Undefining (`undefine`, `^`)

undefine(VARIABLE)

Undefine the VARIABLE.

In some cases, there can be subtle differences between a variable that is defined (but usually empty string), and variables that are not defined at all. However, in many cases, the two are equivalent.

Parameters

VARIABLE – The name of the environment variable to undefine.

Possible invocations

ep undefine VARIABLE
ep ^VARIABLE

Examples

$ echo $USER
root

$ ep undefine USER

$ echo $SOME_VARIABLE
# No result, the variable is not set.

$ echo $HOME/bin
/home/user/bin

$ ep ^HOME

$ echo $HOME/bin
/bin

Adding to arrays (`add`, `+`)

Traditionally, extending a variable such as PATH with your current working directory required executing a lengthy sequence: export PATH="$(pwd):${PATH}".

add(VARIABLE, VALUE..., position=0)

Add the given VALUE (or values, can be multiple) to the VARIABLE array. The values will be located starting at the given position index, while all subsequent elements will be shifted to the right (to higher indices).

Parameters

VARIABLE – The name of the environment variable to add to.
VALUE – The value(s) to add.
position (int) – The position where the added value(s) will be put to. A positive position counts from the beginning of the array, while a negative position counts from the end. 0 is the first, and -1 is the last element’s position.

Possible invocations

ep add [--position] VARIABLE VALUE
ep +VARIABLE VALUE (for position = 0)
ep VARIABLE+ VALUE (for position = -1)

Examples

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin
$ fancy
fancy: command not found!

$ ep add --position 0 PATH /opt/fancy/bin
$ fancy
Fancy tool works!

$ ep PATH
PATH=/opt/fancy/bin:/usr/local/bin:/usr/bin:/sbin:/bin

Using --position to control where the values will be added to. Note the ^1 markers indicating what the individual variables’ positions are understood as.

$ ep SOME_ARRAY
SOME_ARRAY=Foo:Bar:Baz
#          ^0  ^1  ^2
#          -3^ -2^ -1^

$ ep add --position 1 SOME_ARRAY BLAH
$ ep SOME_ARRAY
SOME_ARRAY=Foo:BLAH:Bar:Baz
#          ^0  ^1   ^2  ^3
#          -4^ -3^  -2^ -1^

$ ep add --position -2 SOME_ARRAY FIZZ
$ ep SOME_ARRAY
SOME_ARRAY=Foo:BLAH:FIZZ:Bar:Baz

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

$ ep PATH+ /

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin:/

Note

The add command only works with environment variables that are Array. In case Envprobe did not correctly resolve the type of the variable, you can configure it yourself.

Removing from arrays (`remove`, `-`)

remove(VARIABLE, VALUE...)

Remove all occurrences of VALUE (or values, can be multiple) from the VARIABLE array.

Parameters

VARIABLE – The name of the environment variable to remove from.
VALUE – The value(s) to remove.

Possible invocations

ep remove VARIABLE VALUE
ep -VARIABLE VALUE

Examples

$ ep PATH
PATH=/opt/fancy/bin:/usr/local/bin:/usr/bin:/sbin:/bin
$ fancy
Fancy tool works!

$ ep remove PATH /opt/fancy/bin
$ fancy
fancy: command not found!

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin:/bin

All occurrences are removed. The following array has /bin in it 7 times.

$ ep PATH
PATH=/bin:/bin:/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/bin:/bin:/bin

$ ep -PATH /bin

$ ep PATH
PATH=/usr/local/bin:/usr/bin:/sbin

Note

The remove command only works with environment variables that are Array. In case Envprobe did not correctly resolve the type of the variable, you can configure it yourself.

Saved snapshots

This page details the user-facing commands which deal with interactively accessing the saved snapshots for environment variables. Saved snapshots allow interactively storing some values for environment variables with a name, and later loading these changes in another shell, resulting in the environment set up the same way.

A recurring theme in this section is the mention of current values versus the saved state:

Current values are the values for all environment variables when an Envprobe command started executing. This state is the same as if env was called in the shell.

The saved state is the knowledge about environment variables’ values when Envprobe last saved or loaded a snapshot.

Initially, when Envprobe is loaded into a shell, the current values at the time of loading becomes the first saved state itself.

Difference of current environment (`diff`, `%`)

diff(VARIABLE..., format="normal")

Note

This command is only available if Envprobe has been hooked in the current shell.

Show the difference between the current values of environment variables and the last known saved state. (A saved state is updated when save or load is called. The initial saved state is generated when Envprobe is loaded.)

Parameters

VARIABLE – The names of environment variables to show the diff for. If empty, all tracked variables which changed values are shown.
format (choice) –
The output format to generate.
- -n/--normal: Generate a human-readable output. This is the default option.
- -u/--unified: Generate a more machine-readable output akin to unified diffs.

Possible invocations

ep diff [VARIABLE]
ep % [VARIABLE]

Examples

$ ep PATH
PATH=/foo:/bar
$ ep FOO
FOO is not defined
$ ep NUM
NUM=8

$ ep +PATH /mnt
$ ep -PATH /bar
$ ep FOO=Bar
$ ep ^NUM

$ ep %
(+) Added:       FOO
        defined value: Bar

(-) Removed:     NUM
        value was:     8

(!) Changed:     PATH
        added:         /mnt
        removed:       /bar

Unified diff output format for the above code example, as if ep % -u (or ep diff --unified) was called.

--- /dev/null
+++ FOO
@@ -0,0 +1,1 @@
+Bar

--- NUM
+++ /dev/null
@@ -1,1 +0,0 @@
-8

--- PATH
+++ PATH
@@ -1,2 +1,2 @@
 /foo
-/bar
+/mnt

Save the values/changes to a snapshot (`save`, `}`)

save(SNAPSHOT, VARIABLE..., patch=False)

Note

This command is only available if Envprobe has been hooked in the current shell.

Create or update a named snapshot which will contain the values of environment variables.

Parameters

SNAPSHOT – The name of the snapshot to create.
VARIABLE – The names of the environment variables which values should be saved. If empty, all tracked variables which changed values will be saved.
patch (bool) – If -p/--patch is specified, the user is asked about individual change interactively.

Possible invocations

ep save [--patch] SNAPSHOT [VARIABLE]
ep } SNAPSHOT [-p] [VARIABLE]

Examples

$ ep +PATH /root
$ ep save rootpath PATH
For variable 'PATH' the element '/root' was added.

$ ep FOO=Bar
$ ep } foobar -p
New variable 'FOO' with value 'bar'.
Save this change? (y/N) _

Load a snapshot (`load`, `{`)

load(SNAPSHOT, VARIABLE..., dry_run=False, patch=False)

Note

This command is only available if Envprobe has been hooked in the current shell.

Load values from a saved snapshot and apply the changes to the current shell.

Parameters

SNAPSHOT – The name of the snapshot to load from.
VARIABLE – The names of the environment variables which values should be updated. If empty, all tracked variables in the snapshot will be loaded.
patch (bool) – If -p/--patch is specified, the user is asked about individual change interactively.
dry_run (bool) – If -n/--dry-run is specified, only the would-be loaded changes are printed to the standard output, but no actual change is made to the variables.

Possible invocations

ep load [--dry-run] [--patch] SNAPSHOT [VARIABLE]
ep { SNAPSHOT [-n] [-p] [VARIABLE]

Examples

$ ep PATH
PATH=/bin

$ ep load rootpath PATH
For variable 'PATH' the element '/root' will be added.

$ ep PATH
PATH=/root:/bin

$ ep FOO
FOO is not defined

$ ep { foobar -n
New variable 'FOO' will be created with value 'bar'.

$ ep FOO
FOO is not defined

$ ep { foobar -p
New variable 'FOO' will be created with value 'bar'.
Load and apply this change? (y/N) _

List snapshots (`list`)

list()

List the names of the snapshots available for the current user.

Possible invocations

ep list

Examples

$ ep list
# No result, no snapshots saved yet.

$ ep FOO=bar
$ ep save FOObar
$ ep list
FOObar

Delete a snapshot (`delete`)

delete(SNAPSHOT)

Delete SNAPSHOT.

Parameters

SNAPSHOT – The name of the snapshot that is to be deleted.

Possible invocations

ep delete SNAPSHOT

Examples

$ ep list
FOObar
other_variables

$ ep delete FOObar
$ ep list
other_variables

Variable tracking

Saving certain environment variables (such as PWD, SHLVL, etc.) to a snapshot might not be beneficial. The tracking configuration for variables can be used to toggle whether a particular variable (in the current shell, or globally for your user account) is useful to be saved, or not. If a variable is tracked, changes to it are allowed to be saved and loaded from snapshots. Otherwise, a variable is called ignored. An ignored variable can still be queried and modified through Envprobe for the current shell.

The tracking behaviour for any given variable is resolved in the following order:

If the local configuration (for the current shell session) contains an explicit decision for the variable, that decision is used.

If the global configuration (for your user account) contains an explicit decision for the variable, that decision is used.

The local configuration’s default setting is used.

The global configuration’s default setting is used.

If there are no explicit nor default settings in either configuration files, the variables are tracked, by default.

The tracking of a variable can be changed by the track configuration command.

Settings and configuration (`config` mode)

Envprobe’s settings entry point is called the config mode, which is responsible for allowing you to tweak how Envprobe behaves.

If Envprobe is properly installed, the shell will have the envprobe-config command, and its shorthand alias, epc, defined. Calling epc automatically brings up the argument parser for config mode.

Attention

The list of available subcommands for a mode is generated in a context-sensitive fashion. This means that in case the current context does not allow the execution of an action, Envprobe will usually fail with an invalid choice error.

Generating the hook (`hook`)

hook(SHELL, PID)

Print the shell script that is used to hook and set up Envprobe to the current shell.

Parameters

SHELL – The “type” identifier of the current shell. This usually corresponds to the shell’s name, such as bash or zsh.
PID (int) – The process ID (“pid”) of the current shell. This is commonly specified by letting the shell expand $$ and passing the result.

Possible invocations

envprobe config hook SHELL PID 1

Examples

$ ep
ep: command not found!

$ envprobe config hook bash $$
# If Envprobe isn't registered already...
if [[ ! "$PROMPT_COMMAND" =~ "__envprobe" ]]; then
    # ... multiple lines of Shell script follow ...
fi

$ ep
ep: command not found!

$ eval "$(envprobe config hook bash $$);"
$ ep
usage: envprobe [-h] ...

Hint

The output of this command is not directly useful for any particular purpose other than to execute the resulting script (commonly by calling eval on it) in the context of a running shell. See the install guide on how to set up that all running shells are with Envprobe installed.

1: While the shorthand epc is used for commands in the envprobe config mode (see config mode), the hook command is special, as it is meant to be available before the shell has been hooked. Thus, the full command name (envprobe config ...) must be spelled, as the shorthand is not available.

Obtaining the control commands (`consume`)

consume(detach=False)

Emit the pending changes to environment variables in the form of a shell script. Calling this function will consume the pending changes and clear the list.

The output shell script needs to be evaluated (by calling eval) in the current shell’s context.

Warning

This function is not meant to be called by the user directly! Envprobe will automatically evaluate the pending changes and make them applied to the shell’s state every time a new command prompt is generated.

Parameters: detach (bool) – If -d/--detach is specified, Envprobe will emit code that is meant to unhook it from the shell and clean up temporary files after itself.

Danger

Specifying --detach is a destructive operation, passing a point of no return! Envprobe will irrevocably delete its temporary files related to the shell the command is executed in. This will make Envprobe in that shell unusable.

Detach is automatically performed when the shell exits, normally by the user typing in and executing exit or closing the terminal.

Possible invocations

epc consume [-d]

Examples

$ ep +PATH Foo && ep FOO=Bar && epc consume
export PATH=/Foo:...
export FOO=Bar

Hint

The output of this command is not directly useful for any particular purpose other than to execute the resulting script (commonly by calling eval on it) in the context of a running shell.

Setting additional information for variables (`set`)

set(VARIABLE, type=None, description=None)

Set or change the additional information stored for a variable.

Parameters

VARIABLE – The name of the environment variable to alter, e.g. PATH.
type (choice selection) – Sets the type of the variable to the specified value. To delete the setting, set it explicitly to empty string (--type ""). This change affects the behaviour of the variable moving forward.
description (str) – Sets the user-friendly description of the variable to the given argment. To delete the setting, set it explicitly to empty string (--description ""). This is a cosmetic or informational argument, and changes to it does not affect behaviour.

Possible invocations

envprobe config set VARIABLE [options...]
epc set VARIABLE [options...]

Examples

Setting a behaviour-affecting configuration option --type which changes how Envprobe handles a variable.

$ ep get -i PATH
PATH=/bin:/sbin
PATH:
        /bin
        /sbin
Type: 'path'

$ epc set PATH --type string
Set type for 'PATH'.

$ ep get -i PATH
PATH=/bin:/sbin
Type: 'string'
Source: local

Setting a “cosmetic” configuration option --description which is queried by other commands.

$ ep get -i USER
USER=envprobe-user
Type: 'string'

$ epc set USER --description "The user's name."
Set description for 'USER'.

$ ep get -i USER
USER=envprobe-user
Type: 'string'
Description:
        "The user's name."
Source: local

Note

When both the community description knowledge-base and the user’s local settings contain someting for a VARIABLE, the local settings take priority.

Changing the tracking status of variables (`track`)

track(VARIABLE, default=False, mode=query, scope=<...>)

Gets or sets the tracking setting for a VARIABLE (or the default behaviour) in the local or global scope.

The variable’s tracking setting and the tracking behaviour affects whether the variable’s changes are loaded from or saved to saved snapshots.

Parameters

VARIABLE – The name of the environment which the tracking status is accessed for, e.g. PATH or EDITOR.
default – If -d/--default is given instead of a VARIABLE, the default tracing behaviour will be queried or set.
mode (choice) –
Either one of the following modes. If neither is specified, --query is assumed.
- -t/--track: set the variable to be tracked explicitly, or the default behaviour to track all variables that do not have an explicit setting.
- -i/--ignore/--no-track: set the variable to be ignored explicitly, or the default behaviour to ignore all variables that do not have an explicit setting.
- -r/--reset: remove the explicit setting for the variable.
- -q/--query: retrieve the tracking status for the variable (or the default setting), and print it to the standard output
scope (choice) –
The scope of the configuration to affect. There are two scopes available:
- -l/--local: The setting only applies to the current shell session Envprobe is running in.
- -g/--global: The setting applies to the current user’s local configuration, and thus to all shells. If Envprobe is not available in the current shell, only accessing the global configuration is possible through track.
Note

The --local option is only available if Envprobe has been installed and hooked in the current shell. If so, scope is local, unless otherwise specified.

If Envprobe is not available in the current shell, the --local option is not available, only --global is. In this case, accessing the user-wide global configuration is the only option.

Possible invocations

epc track VARIABLE [mode] [scope]
epc track --default [mode] [scope]

Examples

$ epc track SOMETHING
SOMETHING: tracked

$ epc track --local --default --query
local default: not configured

$ epc track -l -d --ignore
$ epc track SOMETHING
SOMETHING: ignored

$ epc track --global --track ALWAYS_TRACK
$ epc track -l -t SOMETHING
$ epc track SOMETHING
SOMETHING: tracked
    local explicit TRACK

$ epc track OTHER_THING
OTHER_THING: ignored

$ epc track ALWAYS_TRACK
ALWAYS_TRACK: tracked
    global explicit TRACK

$ epc track -l --no-track ALWAYS_TRACK
$ epc track ALWAYS_TRACK
ALWAYS_TRACK: ignored
    local explicit IGNORE
    global explicit TRACK

$ epc track -l --reset SOMETHING
$ epc track SOMETHING
SOMETHING: ignored

Interface with the community descriptions database (`descriptions`)

The descriptions commands allows interfacing the community descriptions knowledge-base with its local copy. This command is implemented with further subcommands, which you need to choose from.

Update the community descriptions database (`update`)

descriptions update()

Check if the description project’s repository has a newer version available. If so, download, extract and install the data contained therein. Subsequent calls to Envprobe will behave according to the new information.

Possible invocations

envprobe config descriptions update
epc descriptions update

Important

This command requires Internet access, and will connect to GitHub to download the new information.

Note

The local settings of the user for a variable will take priority, and are stored in a location separate from where the community descriptions are stored.

Type-safe access

Conventionally, environment variables are stored as strings (character sequences, text) in the system. A running program that uses the environment variable receives it as such a textual data, and in case the variable represents a number and the program wants to use it as a number, the program must perform the conversion. This causes a problematic situation, as users are allowed to set any environment variable to any value, e.g. setting SSH_AGENT_PID (referring a service program’s identifier, which is a number) to any unrelated, non-numeric, bad value.

Envprobe offers type-safe modification of environment variables. This is achieved through a set of built-in heuristics, e.g. _PATH variables should be considered pathlike (:-separated arrays of directories), or _PID variables should be considered numeric. (You can read more details about the available types.) In case you want to assign an invalid value considering the variable’s type, an error is given instead.

$ echo $SSH_AGENT_PID
12345

$ export SSH_AGENT_PID="invalid-value"
# The above example works, even though a "_PID" variable should only
# contain numbers.

$ ep SSH_AGENT_PID=98765
$ ep SSH_AGENT_PID="foo"
[ERROR] Failed to execute: could not convert string to number.

$ ep SSH_AGENT_PID
SSH_AGENT_PID=98765

Certain operations, such as adding and removing elements are only allowed on Array types.

$ ep add USER foo
[ERROR] Failed to execute: 'add' can not be called on non-arrays.

$ ep add PATH /foo

$ ep PATH
PATH=/foo:/bin:/sbin

Apart from the built-in heuristics, you can set the type of a variable yourself with the envprobe config set command.

$ ep add USER foo
[ERROR] Failed to execute: 'add' can not be called on non-arrays.

$ epc set USER --type colon_separated   # Explicitly set "arr:ay".
Set type for 'USER'.

$ ep USER
USER=envprobe-user

$ ep add USER foo
$ ep USER
USER=foo:envprobe-user

$ epc set USER --type ""                # Reset to default heuristic.
$ ep USER
USER="foo:envprobe-user"

$ ep add USER foo
[ERROR] Failed to execute: 'add' can not be called on non-arrays.

Information about environment variables that are useful for others can be obtained from the community knowledge-base project.

Variable Descriptions Knowledge Base

The Envprobe Variable Descriptions Knowledge Base is a sister project to Envprobe which aims to aggregate the settings, such as the variable’s type, about environment variables that are commonly used across the world.

The canonical repository where the knowledge-base lives is available on GitHub at whisperity/Envprobe-Descriptions.

Using this additional information, Envprobe can automatically configure its own behaviour towards a variable from an additional source of information. If the user’s local settings do not contain any setting, and the built-in heuristics do not say anything about a variable, this source will be used.

Note

The knowledge-base is NOT installed automatically when Envprobe is downloaded to your machine. The envprobe config descriptions update command must be executed manually to fetch the initial data, and subsequent updates.

Installing or updating

Execute the envprobe config descriptions update command as shown below. The process is automatic and self-contained, and will update the local copy of the knowledge-base.

$ envprobe config descriptions update
Checking for latest version of the Envprobe Variable Descriptions Knowledge Base project.
Extracting 'default'...
        extracted 7 variables.
Extracting 'python'...
        extracted 10 variables.
Cleaning up old information...
        cleaned up 2 records.

$ envprobe config descriptions update
Checking for latest version of the Envprobe Variable Descriptions Knowledge Base project.
Nothing to update - the latest data is already available.

Contribute to the project

If you would like to contribute to the knowledge-base project, please submit an issue or a pull request. Advanced details about the project’s layout is available in the Implementation reference section.

Implementation reference

Envprobe is implemented as a single Python package, named envprobe.

Warning

While we strive to make the code as modular and easily extensible as possible, at the core, Envprobe is a single program that is offered whole. The core functions that people use Envprobe for are not fully available if the Python package is used as a library, as some of the business logic is part of the command implementations; and as such, the below documentation is geared and intended more for people who would like to contribute to the tool, rather than people who would import it as a library.

Shell interface

The envprobe.shell package implements the logic that allows Envprobe to interface with system shells.

class CapabilityError: Indicates that the requested operation cannot be done in the current Shell.

get_current_shell(environment_dict)

Create a Shell based on the configured environment variables.

Returns

The Shell is instantiated with the environment settings.

Return type

Shell

Raises

KeyError – Raised if the environment_dict did not contain the necessary variables configured.
ModuleNotFoundError – Raised if the shell to be used was identified, but the implementation was unable to load.

class Shell(pid, configuration_dir, control_filename)

Base class for keeping configuration related to a running shell.

Parameters

pid (int) – The process ID (pid) of the shell.
configuration_dir (str) – The directory where the control and persistent data of the shell process will be stored.
control_filename (str) – The name of the file which is used by the hook to execute commands in the shell’s context.

property configuration_directory

The configuration_dir the shell was constructed with.

This is the directory where the data of the shell hook is written to on storage.

Note

In almost all cases, this is a temporary directory!

property control_file

The full path to the control file the shell was constructed with.

The contents of this file are directives written in the target shell’s syntax, and is used to apply the changes commandeered by Envprobe in the shell’s context itself.

abstract get_shell_hook(envprobe_callback_location)

Create the shell code that drives Envprobe’s control evaluation.

This code should be evaluated inside the context of an existing shell process to faciliate execution of the control_file’s contents.

abstract get_shell_unhook()

Create the shell code that detaches/unhooks Envprobe.

This code should be evaluated inside the context of an existing shell process as an inverse operation of get_shell_hook().

abstract property is_envprobe_capable: Whether the current shell is capable of loading and meaningfully running Envprobe.

abstract property manages_environment_variables: Whether the current shell is capable of managing the environment variables through Envprobe.

set_environment_variable(env_var)

Write the code setting env_var’s value to the control_file.

Raises: CapabilityError – If the shell is not capable of managing environment variables.

Note

The implementation for the actual code writing should be provided in the overriden method _set_environment_variable() instead.

property shell_pid: The pid the shell was constructed with.

property shell_type: The identifier of the shell’s type as registered in Envprobe.

property state_file: The full path of the file persisted in storage that is used to store the “saved” state and knowledge about the shell.

unset_environment_variable(env_var)

Write the code that undefines env_var to the control_file.

Raises: CapabilityError – If the shell is not capable of managing environment variables.

Note

The implementation for the actual code writing should be provided in the overriden method _unset_environment_variable() instead.

Dynamic loading

The Shell library is designed to be dynamically loading the individual shell implementations. This is accomplished by the following functions, used extensively in clients of the library.

get_class(kind)

Retrieves the implementation class for the given name from the dynamic registry.

Parameters: kind (str) – The name of the shell, as registered.
Returns: The class implementation.
Return type: class
Raises: KeyError – Raised if the given kind is not registered.

get_kind(clazz)

Retrieves the name for the given implementation class from the dynamic registry.

Parameters: clazz (class) – The implementation class object.
Returns: The registered name for the implementation.
Return type: str
Raises: KeyError – Raised if the given clazz is not registered.

get_known_kinds()

Get the list of dynamically registered and loaded Shell implementations.

Returns: The names.
Return type: list(str)

load(kind)

Attempt to load a Shell implementation.

The function loads the module kind from envprobe.shell an expects it to register the Shell named kind.

Parameters

kind (str) – The name of the shell to load. The loading will be done from the similarly named Python module under envprobe.shell.

Returns

clazz – If the loading succeeded or the given kind is already loaded, the class implementation is returned.

Return type

class

Raises

ModuleNotFoundError – Raised if the module to load is not found.
NotImplementedError – Raised if the module successfully loaded, but there were no shell with the name kind registered by it.

load_all()

Loads all Shell implementations to the interpreter found under envprobe.shell in the install.

This method does not throw if a module does not actually register anything.

Registering implementations

register_type(kind, clazz)

Register the Shell implementation to the dynamic lookup mechanism.

Parameters

kind (str) – The name of the shell implementation.
clazz (class) – The Python class that implements the functions.

Raises

TypeError – clazz must be a type that is a subclass of Shell, otherwise an exception is raised.

Example

The implementing modules of Shell subclasses should be named similarly as the shell they are implementing, and after defining the class, in the global code of the module itself, a call to register_type should be present:

Registering a hypothetical implementation of the PytShell in envprobe.shell.pyt

from envprobe.shell.core import Shell, register_type

class Pyt(Shell):
    # ... the implementation of the class ...

register_type('pyt', Pyt)

Environment variable types

The envprobe.vartypes package implements the logic that maps different environment variables allows Envprobe to interface with system shells.

class EnvVar(name, raw_value)

Base class for an environment variable type’s implementation.

Note

The implementations of this class are responsible for handling the type of an environment variable. The classes are instantiated with a given value, and are detached from actual environment afterwards, i.e. a change in the environment will NOT be reflected by what is available from the instances.

Parameters

name (str) – The name of the environment variable.
raw_value (str) – The stringified raw value of the environment variable, as present in the operating system.

apply_diff(diff)

Applies the given difference to the value of the instance.

Parameters: diff (list(char, str)) – A difference action list, as returned by EnvVar.diff().
Raises: ValueError – If the diff is invalid or non-applicable.

Note

In most cases, the - (remove) side of the diff is ignored when applying.

classmethod diff(old, new)

Generate an iterable difference “actions” between two variables.

Parameters

old (EnvVar or None) – The “left” or “baseline” side of the diff.
new (EnvVar or None) – The “right” or “new” side of the diff.

Returns

The differences between the two variables’ values.

The first element of the tuple, (char), is either +, - or = for new/added, old/removed, and unchanged, respectively. For each entry in the list, the second element (str) is the affected value itself. The description of the difference is always of type str.

If either the old or the new did not contain a value, the respective side will be missing from the list.

If the two variables are equal, an empty list is returned.

Return type

list(char, str)

Raises

TypeError – Only variables of the same type, and only subclasses of EnvVar may be differentiated.

Note

The implementation for the actual calculating of the difference should be provided in the overridden method _diff() in the subclasses.

property extended_attributes

Returns the object managing the extended attributes (user-facing knowledge) about the variable.

Returns: configuration
Return type: EnvVarExtendedInformation

classmethod merge_diff(diff_a, diff_b)

Creates a merged diff from two diffs that simulates the transitive application of the first and the second diff in order.

Parameters

diff_a (list(char, str)) – The difference actions to apply first, as returned by EnvVar.diff().
diff_b (list(char, str)) – The difference actions to apply second, in the same format.

Returns

The merged difference actions’ list. The format is kept.

Return type

list(char, str)

Raises

ValueError – If the diff is invalid or non-applicable.

property name: The name of the variable.

abstract raw(): Convert the value of the variable to a raw shell representation.

classmethod type_description(): The description of the environment variable type.

abstract property value

Retrieves the value of the environment variable.

Returns: The value. The (Python) type of the returned value is specific to the subclass.
Return type: unknown

The extended information for an environment variable is stored in objects of the following class.

class EnvVarExtendedInformation

Implements a storage for the extended user-facing knowledge about a variable.

apply(configuration)

Applies the persisted configuration on the current object.

Parameters: configuration (dict) – The configuration mapping too apply. See envprobe.settings.variable_information.VariableInformation.__getitem__() for the format generated.

property description: The human description about the usage of the variable.

property source: The annotated source repository where the variable’s information was obtained from.

property type: The type class identifier (kind) for the current variable.

Implemented types

The list of known variable types in Envprobe are documented in the pages accessible from the links below.

`string.String`	The standard type of environment variables.
`numeric.Numeric`	This type may only hold a numeric (`int` or `float`) value.
`array.Array`	An environment variable where elements are separated by a `separator`.
`colon_separated.ColonSeparatedArray`	A helper class that binds the array's `separator` to `:`.
`semi_separated.SemicolonSeparatedArray`	A helper class that binds the array's `separator` to `;`.
`path.Path`	A POSIX-compatible `PATH` environment variable.

Dynamic loading

The library is designed to be dynamically loading the individual environment variable type implementations. This is accomplished by the following functions, used extensively in clients of the library.

get_class(kind)

Retrieves the implementation class for the given name from the dynamic registry.

Parameters: kind (str) – The name of the environment variable type, as registered.
Returns: The class implementation.
Return type: class
Raises: KeyError – Raised if the given kind is not registered.

get_kind(clazz)

Retrieves the name for the given implementation class from the dynamic registry.

Parameters: clazz (class) – The implementation class object.
Returns: The registered name for the implementation.
Return type: str
Raises: KeyError – Raised if the given clazz is not registered.

get_known_kinds()

Get the list of dynamically registered and loaded EnvVar implementations.

Returns: The names.
Return type: list(str)

load(kind)

Attempt to load an EnvVar implementation.

The function loads the module kind from envprobe.vartypes and expects it to register the EnvVar named kind.

Parameters

kind (str) – The name of the environment variable type to load. The loading will be done from the similarly named Python module under envprobe.vartypes.

Returns

clazz – If the loading succeeded or the given kind is already loaded, the class implementation is returned.

Return type

class

Raises

ModuleNotFoundError – Raised if the module to load is not found.
NotImplementedError – Raised if the module successfully loaded, but there were no implementation with the name kind registered by it.

load_all()

Loads all EnvVar implementations to the interpreter found under envprobe.vartypes in the install.

This method does not throw if a module does not actually register anything.

Registering implementations

register_type(kind, clazz)

Register the EnvVar implementation to the dynamic lookup mechanism.

Parameters

kind (str) – The name of the environment variable type implementation.
clazz (class) – The Python class that implements the functions.

Raises

TypeError – clazz must be a type that is a subclass of EnvVar, otherwise an exception is raised.

Example

The implementing modules of EnvVar subclasses should be named similarly as the shell they are implementing, and after defining the class, in the global code of the module itself, a call to register_type() should be present:

Registering a hypothetical implementation of the Circle in envprobe.vartypes.circle

from envprobe.vartypes.envvar import EnvVar, register_type

class Circle(EnvVar):
    # ... the implementation of the class ...

register_type('circle', Circle)

Environment management

The envprobe.environment module implements the logic that allow Envprobe to keep track of the current environment it is running in. At the core of the library is the Environment class, which is instantiated together with the current shell.Shell.

create_environment_variable(name, env, pipeline=None)

Create a vartypes.EnvVar instance for a variable, using a specific pipeline in an environment.

Parameters

name (str) – The name of the environment variable to create.
env (dict) – The raw mapping of environment variables to their values, as in os.environ.
pipeline (HeuristicStack, optional) – The heuristics pipeline to use to decide on the variable’s type. If not specified, default_heuristic is used.

Returns

The instantiated environment variable. The variable is instantiated even if the variable is not found in env.

Return type

vartypes.EnvVar

Raises

KeyError – If the variable was not resolved to a valid type by the pipeline, either because no heuristic mapped to anything, or a heuristic mapped to False.

class Environment(shell, env, variable_type_heuristics)

Owns and manages the understanding of environment variables’ state attached to a shell.

Environment manages two “states”, the current_environment and the stamped_environment. The current environment is usually instantiated from the environment of the shell Envprobe is running in, while the stamped environment is loaded from storage and lives between executions of an Envprobe operation.

Create an environment manager.

Parameters

shell (shell.Shell) – The shell to attach the manager to. The shell.Shell.configuration_directory of the shell will be respected as data storage.
env (dict, optional) – The raw mapping of environment variables to their values, as in os.environ.
variable_type_heuristic (HeuristicStack, optional) – The variable-to-vartypes.EnvVar type mapping heuristics. If not specified, default_heuristic is used.

__getitem__(variable_name)

Retrieve a vartypes.EnvVar environment variable from the current_environment’s values.

Returns

env_var (.vartypes.EnvVar) – The typed environment variable object. This object is always constructed, if there is no value associated with it then to a default empty state.
is_defined (bool) – True if the variable was actually defined in the environment.

apply_change(variable, remove=False)

Applies the changes in variable to the stamped_environment, i.e. modifying the pristine state.

Parameters

variable (vartypes.EnvVar) – The environment variable which value has been modified. (If the variable does not exist in the environment, it will be added with its current value.)
remove (bool) – If True, the variable will be removed from the environment, otherwise, the new value is saved.

Warning

The application of the change only happens to the “knowledge” of the environment manager instance. This method does not attempt to guarantee in any way that the change of the value is respected by the underlying shell.

Difference of environments

The environment.diff() function creates and returns a VariableDifference for each changed variable.

class VariableDifference(kind, var_name, old_value=None, new_value=None, difference_actions=None)

The difference of a single variable between two states.

Parameters

kind (VariableDifferenceKind) – The “direction” of the difference’s result.
var_name (str) – The name of the variable that was differentiated.
old_value (unknown) – The “old” value of the variable.
new_value (unknown) – The “new” value of the variable.
difference_actions (list(char, str)) –
The individual changes in the variable’s value, as returned by vartypes.EnvVar.diff().

The list contains the changes with a mode symbol (-, = or + for removed, unchanged/same, added, respectively), and the value that was affected.

Raises

TypeError – kind must be one of the constants in VariableDifferenceKind.

property is_new: Whether the current instance represents the definition of a new variable.

property is_simple_change

Whether the current instance represents a simple change.

A simple change is either the creation or removal of a value, or a single-step modification of a value from one value to another.

property is_unset: Whether the current instance represents the removal/unsetting of an existing variable.

class VariableDifferenceKind(value)

Named enumeration to indicate the “direction” of the VariableDifference.

ADDED: The variable is not defined in “old” but defined in “new”.

CHANGED: The variable is defined on both sides (“old” and “new”) of the difference, but to different values.

REMOVED: The variable is defined in “old” but not defined in “new”.

Type heuristics

To ensure that environment variables (which are in almost all cases handled simply as strings until they are parsed by a program) can be managed in a type-safe manner, heuristics that map a raw environment variable to a proper type can be passed to create_environment_variable() and Environment.

class EnvVarTypeHeuristic

A heuristic maps a variable’s name and it’s potential value in the environment to an envprobe.vartypes type.

__call__(name, env=None)

Resolve the variable of name (in the env environment) to an envprobe.vartype type identifier (as registered by vartypes.register_type()).

The default implementation resolves everything to be 'string', corresponding to vartypes.string.String.

Parameters

name (str) – The name of the environment variable.
env (dict, optional) – The raw mapping of environment variables to their values, as in os.environ.

Returns

vartype_name (str) – The name of an envprobe.vartypes implementation, if the heuristic resolved successfully.
None – None is returned if the heuristic could not resolve a type to be used.
False (bool) – False is returned if the heuristic resolved the variable not to be managed by Envprobe.

class HeuristicStack

A stack of EnvVarTypeHeuristic objects which resolve a variable in a set order.

Example

# Build the pipeline.
pipeline = HeuristicStack()
pipeline += HeuristicThatAlwaysReturnsString()
pipeline += HeuristicThatMapsTESTVarToFalse()

# (HeuristicStack executes the added heuristics in reverse order.)

# Use the pipeline to resolve.
pipeline("TEST", env)
>>> False

pipeline("OTHER_VAR", env)
>>> 'string'

__call__(name, env=None)

Resolve the variable of name (in the env environment) to an envprobe.vartype type identifier, using the heuristics in the order of the stack.

The resolution is run until a heuristic returns non-None (either False or an identifier).

Parameters

name (str) – The name of the environment variable.
env (dict, optional) – The raw mapping of environment variables to their values, as in os.environ.

Returns

vartype_name (str) – The name of an envprobe.vartypes implementation, if a heuristic resolved successfully.
None – None is returned if no heuristic could resolve a type to use.
False (bool) – False is returned if a heuristic resolved the variable not to be managed by Envprobe.

default_heuristic

Provides the default EnvVarTypeHeuristic added to an instantiated HeuristicStack.

The default heuristic maps every variable to a pure vartypes.string.String.

Available heuristics

The following heuristics are implemented in the vartype_heuristics module. All these classes are subclasses of EnvVarTypeHeuristic.

`ConfigurationResolvedHeuristic`	Implements a heuristic that reads from a `settings.variable_information.VariableInformation` configuration manager, and resolves the type of the variable based on these settings contained therein.
`EnvprobeEnvVarHeuristic`	Disable access to internal variables that begin with `ENVPROBE_`.
`HiddenEnvVarHeuristic`	Disable access to every variable that begins with `_`, similar to how files named as such are considered "hidden".
`NumericalNameEnvVarHeuristic`	Regard commonly numeric-only variables as `vartypes.numeric.Numeric`.
`NumericalValueEnvVarHeuristic`	Regard environment variables that currently have a numeric value as `vartypes.numeric.Numeric`.
`PathNameEnvVarHeuristic`	Regard `PATH` and variables that end with `_PATH` as `vartypes.path.Path`.

assemble_standard_type_heuristics_pipeline(varcfg_user_loader, varcfg_description_loader)

Creates the standard environment.HeuristicStack pipeline that decides the type for an environment variable.

This pipeline uses the configuration of the user and the community first, and then the heuristics pre-implemented in Envprobe to deduce a vartype for an environment variable.

Parameters

varcfg_user_loader (str -> object) – The function used for the ConfigurationResolvedHeuristic internally. This function is called with the name of a variable and should return an object similar to settings.variable_information.VariableInformation in which the type can be looked up.
varcfg_description_loader (str -> object) – The function used for the ConfigurationResolvedHeuristic internally. This function is called with the name of a variable and should return an object similar to settings.variable_information.VariableInformation in which the type can be looked up.

Handling user configuration

The envprobe.settings package implements classes that deal with handling configuration that is specific to the user, or a particular shell.

Configuration files are put under the modifying persistent configuration directory of the user, as given by the following function.

get_configuration_directory()

Returns the location where Envprobe’s user-specific configuration files are stored.

This method uses $XDG_CONFIG_HOME as per FreeDesktop specifications, if exists. Otherwise, the .config directory under the user’s HOME will be used.

Snapshot configuration

The Snapshot class implements the wrapping handler over the configuration files stored for the user containing the environment variables’ values involved in a saved snapshot.

class Snapshot(configuration=None)

Represents a persisted configuration file of the user which stores the state of some environment variables.

Initialise a snapshot manager.

This instantiation is cheap. Accessing the underlying data is only done when a query or a setter function is called.

Parameters: configuration (context-capable dict, optional) –

UNDEFINE

UNDEFINE is a special tag instance that is returned by Snapshot if the stored action for a variable is to undefine it.

This value should always be identity-compared with the is keyword.

__delitem__(variable_name)

Marks the given variable to be undefined when the snapshot is loaded.

Parameters: variable_name (str) – The name of the variable to mark for undefinition.

__getitem__(variable_name)

Retrieve the stored actions for the given variable.

Returns

diff_actions (list(char, str)) – The representation of the diff actions to be taken for the variable to have the value in the saved snapshot.

See envprobe.vartypes.EnvVar.diff() for the format.
UNDEFINE – Returned if the variable was marked to be undefined.

__setitem__(variable_name, difference)

Sets the stored action of the given variable to the new value.

Parameters

variable_name (str) – The name of the variable to set.
difference (list(char, str)) –
The difference actions to save into the snapshot file.

See envprobe.vartypes.EnvVar.diff() for the format.

Warning

__setitem__() overwrites the information that is stored in the snapshot. In most cases, envprobe.vartypes.EnvVar.merge_diff() should be used to first create a diff that appends to the current one, and save that result.

keys(): Returns the variable names that are affected by the snapshot.

Storing additional information about variables

The additional information associated with envprobe.vartypes.EnvVar instances are persisted through this configuration handler.

class VariableInformation(configuration=None)

Represents a persisted configuration file of the user which stores additional information about environment variables.

Initialise a variable configuration manager.

This instantiation is cheap. Accessing the underlying data is only done when a query or a setter function is called.

Parameters: configuration (context-capable dict, optional) –

__delitem__(variable_name)

Removes the configuration associated with the given variable.

Parameters: variable_name (str) – The name of the variable to remove.

__getitem__(variable_name)

Retrieve the configuration for the given variable.

Returns: config – The configuration mapping associated with the variable. This can be used to instantiate an envprobe.vartypes.envvar.EnvVarExtendedInformation.
Return type: dict

keys(): Returns the variable names that have a configuration.

set(variable_name, configuration, source)

Sets the stored configuration of the given variable to a new value.

Parameters

variable_name (str) – The name of the variable to set.
config (envprobe.vartypes.envvar.EnvVarExtendedInformation) – The configuration associated with the variable. This is automatically mapped to the persisted representation.
source (str) – The identifier of the configuration source repository to annotate the saved configuration with.

Tracking variables

Environment variable tracking is an integral feature of Envprobe’s persistence layer. A variable that is not tracked will not be handled when saved snapshots are accessed.

class VariableTracking(global_configuration=None, local_configuration=None)

Allows access to the user’s configuration files about which variables are tracked.

In most cases, the tracking is configured on two levels, a local (usually a shell’s configuration) level, and a global (user’s) level. The class’s behaviour reflects the two configurations in an overlay, with the local settings trumping the global ones.

Initialises the tracking manager.

This instantiation is cheap. Access to the underlying data is only done when a query or setter function is called.

Parameters

global_configuration (context-capable dict, optional) –
local_configuration (context-capable dict, optional) –

property default_tracking: Returns True if the current default tracking behaviour is ON.

property global_tracking: Returns True if the default behaviour for tracking is ON in the global configuration.

ignore_global(variable_name): Marks the variable to be ignored in the global scope.

ignore_local(variable_name): Marks the variable to be ignored in the local scope.

is_explicitly_configured_global(variable_name): Returns whether variable_name has an explicit track/ignore record in the global scope.

is_explicitly_configured_local(variable_name): Returns whether variable_name has an explicit track/ignore record in the local scope.

is_tracked(*variables)

Resolves whether the variable_name are tracked according to the loaded configuration.

The resolution order for the tracking status of a variable is the following:

If the variable is explicitly tracked/ignored locally, the result.

If the variable is explicitly tracked/ignored globally, the result.

If there is a default tracking set locally, that value.

The default global tracking setting.

True, i.e. the variable is deemed tracked.

Parameters: *variables (list(str) or str) – The list of variable names to check, either a single string, a variable length argument pass, or a list.
Returns: If only one element was given to variables, a single result is returned, which is True if the variable should be considered tracked, and False otherwise. If multiple names were passed, a list is returned, with each result being the result for the passed varaible, in order.
Return type: list(bool) or bool

is_tracked_global(variable_name): Returns whether the variable is set to be tracked in the global configuration file.

is_tracked_local(variable_name): Returns whether the variable is set to be tracked in the local configuration file.

property local_tracking: Returns True if the default behaviour for tracking is ON in the local configuration.

track_global(variable_name): Marks the variable to be tracked in the global scope.

track_local(variable_name): Marks the variable to be tracked in the local scope.

unset_global(variable_name): Removes the variable from the track or ignore lists in the global scope, making the default behaviour apply.

unset_local(variable_name): Removes the variable from the track or ignore lists in the local scope, making the default behaviour apply.

Abstract configuration files

The ConfigurationFile class allows accessing a key-value mapping (dict) of configuration objects that is saved to the user’s file system to a JSON file.

class ConfigurationFile(file_path, default_content=None, read_only=False, file_mode=384, directory_mode=448)

A glorified dict that is backed into a JSON file and locked on access.

This class embeds file locking logic to ensure atomic access to the backing file is given.

Initialise a configuration file.

Parameters

file_path (str) – The path of the backing file of the configuration storage.
default_content (dict, optional) – The contents of the configuration if the backing file does not exist.
read_only (bool, optional) – Whether the configuration is opened read-only. Creating the configuration read-only will prevent any changes from being applied from Python code, and will also prevent the backing file from writing.
file_mode (int, optional) – If the backing file does not exist, it will be created with the mode flags given. Defaults to owner read, write, no execute permission, or permissions for group and world.
directory_mode (int, optional) – If the directory where the backing file should be put does not exist, it will be created with the mode flags given. Defaults to owner read, write, list, no permission for group and world.

__contains__(key): Returns if the data in memory contains key.

__delitem__(key): Deletes the key.

__enter__()

Acquires the backing file, read its contents, and return a context where the ConfigurationFile can be used.

Returns: The self instance.
Return type: ConfigurationFile

Note

This method should be used instead of load() and save() if a changing access to the contents is expected, as the context keeps the lock on the file active throughout.

__exit__(exc_type, exc_value, traceback): Saves the changes to the current file (if not read-only) and releases the lock.

__getitem__(key): Retrieves the element for key.

__iter__(): Returns an iterator over the data loaded to memory.

__len__(): Returns the number of keys in memory.

__setitem__(key, value): Sets the element for key to value.

property changed: Returns whether the data in memory changed since the last save or load.

get(key, default=None): Returns the data in memory for key, or if no key found, default.

load(): Load the contents of the backing file into memory.

Warning

After the file’s contents are loaded into memory, the lock is released, meaning that subsequent changes in the file might be overwritten if save() is called with the current contents!

If you intend to change the configuration and save it, use ConfigurationFile as a context manager instead.

save()

Save the contents of memory to the backing file.

Raises: io.UnsupportedOperation – Raised if the file was opened read_only, but save is called.

Warning

The file’s contents are overwritten by this method after acquiring the lock, destroying potential changes that might have been written before. The lock is released afterwards immediately.

If you intend to change the configuration in one go, use ConfigurationFile as a context manager instead.

Community descriptions

Envprobe implements the programs responsible for interfacing with the Variable Descriptions Knowledge Base in the community_descriptions module.

Local storage

The data downloaded from the knowledge-base is stored in a dedicated location (see envprobe.settings.get_data_directory()), separate from the user’s own configuration. The MetaConfiguration class allows accessing the metadata of the storage.

get_storage_configuration(read_only=True): Returns the configuration manager for the metainformation about the local storage.

class MetaConfiguration(configuration=None)

Represents a persisted configuration file of the local description storage.

Initialises the metadata configuration manager.

This instantiation is cheap. Accessing the underlying data is only done when a query or a setter function is called.

Parameters: configuration (context-capable dict, optional) –

delete_source(source): Delete the record associated with the given source.

get_comment_for(source): Returns the textual comment associated with the given information source.

set_comment_for(source, comment): Save the textual comment associated with the given information source.

property version: Returns the commit identifier of the data in the storage.

The individual configuration for variables can be accessed through a envprobe.settings.variable_information.VariableInformation instance, in the exact same fashion as the user’s local configuration is accessible.

get_variable_information_manager(variable_name, read_only=True)

Creates the extended information attribute manager for environment variables based on the requested variable’s name, using the locally installed community descriptions data as source.

Parameters

varable_name (str) – The name of the variable to configure.
read_only (bool) – If True, the associated file will be opened read-only and not saved at exit.

Returns

The configuration handler engine. Access to the underlying file is handled automatically through this instance.

Return type

envprobe.settings.variable_information.VariableInformation

Downloader

The following program elements are used from the higher-level logic to handle downloading and unpacking the contents of the knowledge-base.

fetch_latest_version_information()

Obtain the version of the latest downloadable knowledge base.

Note

Executing this function requires Internet connection, and will make a connection to GitHub’s API.

Raises: URLError – Executing the request may fail due to configuration issues, bad connection, or GitHub API’s throttling of the user. Throttling is implemented for unauthenticated requests against the IP address of the machine. Read more at http://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting.

download_latest_data(location)

Downloads the latest information to the given location.

Parameters: location (str) – The path the downloader can use to extract the data to. This directory must exist already, and will not be cleared up by this function. This is only used temporarily.
Returns: The handlers of the extracted knowledge source documents.
Return type: list(DescriptionSource)

Note

Executing this function requires Internet connection, and will make a connection to GitHub’s API.

Raises: URLError – Executing the request may fail due to configuration issues, bad connection, or GitHub API’s throttling of the user. Throttling is implemented for unauthenticated requests against the IP address of the machine. Read more at http://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting.

class DescriptionSource(source_file_path)

The handler class that is used to parse and understand the files of the downloaded knowledge-base.

Parameters: source_file_path (str) – The path to the CSV database.

Note

Instantiating the object is a cheap operation. To load the data, call the parse() method.

__getitem__(variable)

Get the information for the given variable.

Returns: The configuration mapping of the variable as parsed from the source file. The format of the dictionary is appropriate for use with envprobe.vartypes.envvar.EnvVarExtendedInformation.
Return type: dict

__iter__(): Iterate over the variables in the source.

__len__(): The number of variables in the source.

property comment: A textual comment about the contents of the source.

property name: The identifier of the source.

parse(): Execute loading of the data from the backing file.

Layout of the knowledge-base repository

The repository is hosted on GitHub and managed as a separate Git repository.

`format.ver`

The format of the repository is versioned internally and the current version of the data files in the commit is specified in the format.ver file. Versions are expressed in the X.Y (major and minor version) format. Envprobe will only accept the download if the format is supported.

MIN_SUPPORTED_FORMAT = '1.0': The minimum version of description releases that is supported by the current implementation.

MAX_SUPPORTED_FORMAT = '1.0': The maximum version of description releases that is supported by the current implementation.

Data files (`something.csv`)

The data itself is stored in comma-separated values (CSV) files. (Read more about CSV files in the csv module documentation.) CSV files allow easy editing and diffing of the files, and also easy handling during download.

The CSV file beings with a header row that defines the title of each column. Each data file identifies a conceptual “group” of variables by usage, scope, or associated context. However, the downloader merges the information from all data files together when creating the local copy.

Variable,TypeKind,Description
__META__,COMMENT,"Example datafile."
MY_VARIABLE,string,"A variable."
NAMES,comma_separated,"A list of names."

Miscellaneous functions

The envprobe.library module contains some miscellaneous functions that are used to tie the creation of the global objects accessing the environment and the user’s state in a meaningful way.

get_shell_and_env_always(env_dict=None, vartype_pipeline=None)

Return an environment.Environment and shell.Shell, no matter what.

Parameters

env_dict (dict, optional) – The raw mapping of environment variables to their values, as in os.environ.
vartype_pipeline (envprobe.environment.HeuristicStack, optional) – The type resolution heuristics pipeline to use. If not specified, the environment.default_heuristic will be used.

Returns

shell (.shell.Shell) – The shell which use can be deduced from the current environment (by calling shell.get_current_shell()). If none such can be deduced, a shell.FakeShell is returned, which is a valid shell.Shell subclass that tells the client code it is not capable of anything.
env (.environment.Environment) – The environment associated with the current context and the instantiated Shell.

get_snapshot(snapshot_name, read_only=True)

Creates the snapshot instance for the snapshot of the given name.

Parameters

snapshot_name (str) – The name of the snapshot to load or create.
read_only (bool) – If True, the file will be opened read-only and not saved at exit.

Returns

The snapshot manager object. Access to the underlying file is handled automatically through this instance.

Return type

settings.snapshot.Snapshot

get_variable_information_manager(variable_name, read_only=True)

Creates the extended information attribute manager for environment variables based on the requested variable’s name.

Parameters

varable_name (str) – The name of the variable to configure.
read_only (bool) – If True, the associated file will be opened read-only and not saved at exit.

Returns

The configuration handler engine. Access to the underlying file is handled automatically through this instance.

Return type

settings.variable_information.VariableInformation

get_variable_tracking(shell=None)

Creates a read-only tracking manager for the standard global and local configuration files.

Parameters: shell (shell.Shell, optional) – The shell which is used in the current environment, used to retrieve the local configuration directory.
Returns: The tracking handler engine. The configuration files are opened read-only.
Return type: settings.variable_tracking.VariableTracking

Envprobe

Audience

Comparison to existing tools

Documentation

Install

Obtaining Envprobe

From GitHub

Setting up the shell hook

Bash

Zsh

Stock Zsh

Zsh with Oh-My-Zsh

Testing the hook

Officially supported configuration

Quick overview

Environment variable management (main mode)

Managing environment variables

Reading (get, ?)

Writing (set, !, =)

Undefining (undefine, ^)

Adding to arrays (add, +)

Removing from arrays (remove, -)

Saved snapshots

Difference of current environment (diff, %)

Save the values/changes to a snapshot (save, })

Load a snapshot (load, {)

List snapshots (list)

Delete a snapshot (delete)

Variable tracking

Settings and configuration (config mode)

Generating the hook (hook)

Obtaining the control commands (consume)

Setting additional information for variables (set)

Changing the tracking status of variables (track)

Interface with the community descriptions database (descriptions)

Update the community descriptions database (update)

Type-safe access

Variable Descriptions Knowledge Base

Installing or updating

Contribute to the project

Implementation reference

Shell interface

Dynamic loading

Registering implementations

Environment variable types

Implemented types

Dynamic loading

Registering implementations

Environment management

Difference of environments

Type heuristics

Available heuristics

Handling user configuration

Snapshot configuration

Storing additional information about variables

Tracking variables

Abstract configuration files

Community descriptions

Local storage

Downloader

Layout of the knowledge-base repository

format.ver

Data files (something.csv)

Miscellaneous functions

Indices and tables

Environment variable management (`main` mode)

Reading (`get`, `?`)

Writing (`set`, `!`, `=`)

Undefining (`undefine`, `^`)

Adding to arrays (`add`, `+`)

Removing from arrays (`remove`, `-`)

Difference of current environment (`diff`, `%`)

Save the values/changes to a snapshot (`save`, `}`)

Load a snapshot (`load`, `{`)

List snapshots (`list`)

Delete a snapshot (`delete`)

Settings and configuration (`config` mode)

Generating the hook (`hook`)

Obtaining the control commands (`consume`)

Setting additional information for variables (`set`)

Changing the tracking status of variables (`track`)

Interface with the community descriptions database (`descriptions`)

Update the community descriptions database (`update`)

`format.ver`

Data files (`something.csv`)