Helpers

Config

es_client.helpers.config.cli_opts(value: str, settings: dict = None, onoff: dict = None, override: dict = None) → Tuple[Tuple[str], dict]

In order to make building a Click interface more cleanly, this function returns all Click option settings indicated by value, both forming the lone argument (e.g. --option), and all key word arguments as a dict.

The single arg is rendered as f'--{value}'. Likewise, value is the key to extract all keyword args from the supplied dictionary. The facilities to override default values and show hidden values is added here. For default value overriding, the NOPE constant is used as None and False are valid default values

es_client.helpers.config.cloud_id_override(args: dict, params: dict, client_args: ClientArgs) → dict

If hosts are in the config file, but cloud_id is specified at the command-line, we need to remove the hosts parameter as cloud_id and hosts are mutually exclusive

This function returns an updated dictionary args to be used for the final configuration as well as updates the client_args object.

Parameters:

• args (dict) – The argument dictionary derived params.

• params (dict from params) – Click context params, e.g. ctx.params

• client_args (ClientArgs) – The working list of client args

Return type:

dict

Returns:

Updated version of args

es_client.helpers.config.context_settings()

Return Click context settings dictionary

es_client.helpers.config.get_arg_objects(config: dict) → Tuple[ClientArgs, OtherArgs]

Return initial tuple of ClientArgs, OtherArgs

They will be either empty/default, or with values from config

Parameters:

config (dict) – Config dict derived from a YAML configuration file.

Return type:

tuple

Returns:

ClientArgs and OtherArgs objects in a tuple

es_client.helpers.config.get_args(params: dict, config: dict) → Tuple[ClientArgs, OtherArgs]

Return ClientArgs, OtherArgs tuple from params overriding whatever may be in config

Parameters:

• params (dict from params`) – Click context params, e.g. ctx.params

• default_filepath (str) – File path to a default config file location.

Return type:

tuple

Returns:

ClientArgs and OtherArgs objects in a tuple

es_client.helpers.config.get_client(configdict: dict = None, configfile: str = None, autoconnect: bool = False, version_min: tuple = (8, 0, 0), version_max: tuple = (8, 99, 99)) → Elasticsearch

Get an Elasticsearch Client using es_client.builder.Builder

Build a client out of settings from configfile or configdict If neither configfile nor configdict is provided, empty defaults will be used. If both are provided, configdict will be used, and configfile ignored.

Parameters:

• configdict (dict) – A configuration dictionary

• configfile (str) – A configuration file

• autoconnect (bool) – Connect to client automatically

• verion_min – Minimum acceptable version of Elasticsearch (major, minor, patch)

• verion_max – Maximum acceptable version of Elasticsearch (major, minor, patch)

Returns:

A client connection object

Return type:

Elasticsearch

es_client.helpers.config.get_config(params: dict, default_config: str = None) → dict

If params[‘config’] is a valid path, return the validated dictionary from the YAML

If nothing has been provided to params[‘config’], but default_config is populated, use that.

Parameters:

• params (dict from params`) – Click context params, e.g. ctx.params

• default_config (str) – Path to a configuration file

Returns:

Configuration dictionary

Return type:

dict

es_client.helpers.config.get_hosts(params: dict) → list

Return hostlist suitable for client object. Validate url schema for each entry.

Parameters:

params (dict from params`) – Click context params, e.g. ctx.params

Returns:

List of hosts

Return type:

list

es_client.helpers.config.get_width()

Determine terminal width

es_client.helpers.config.hosts_override(args: dict, params: dict, client_args: ClientArgs) → dict

If hosts are provided at the command-line, but cloud_id was in the config file, we need to remove the cloud_id parameter from the config file-based dictionary before merging. This function returns an updated dictionary args to be used for the final configuration as well as updates the client_args object.

Parameters:

• args (dict) – The argument dictionary derived params.

• params (dict from params) – Click context params, e.g. ctx.params

• client_args (ClientArgs) – The working list of client args

Return type:

dict

Returns:

Updated version of args

es_client.helpers.config.override_client_args(params: dict, client_args: ClientArgs) → ClientArgs

Override client_args settings with any values found in params

Parameters:

• params (dict from params`) – Click context params, e.g. ctx.params

• client_args (ClientArgs) – The working list of client args

Returns:

The updated working list ClientArgs object

Return type:

ClientArgs

es_client.helpers.config.override_other_args(params: dict, other_args: OtherArgs) → OtherArgs

Override other_args settings with any values found in params

Parameters:

• params (dict from params`) – Click context params, e.g. ctx.params

• other_args (OtherArgs) – The working list of other args

Returns:

The updated working list OtherArgs object

Return type:

OtherArgs

es_client.helpers.config.override_settings(settings: dict, override: dict) → dict

Override keys in settings with values matching in override

Parameters:

• settings (dict) – The source data

• override (dict) – The data which will override settings

Return type:

dict

Returns:

An dictionary based on settings updated with values from override

Logging

es_client.helpers.logging.check_logging_config(config)

Ensure that the top-level key logging is in config before passing it to SchemaCheck for value validation.

Parameters:

config (dict) – Logging configuration data

Returns:

SchemaCheck validated logging configuration.

es_client.helpers.logging.configure_logging(config: dict, params: dict) → None

Configure logging based on a combination of config and params

Values in params will override anything set in config

Parameters:

• config (dict) – Config dict derived from a YAML configuration file.

• params (dict from params`) – Click context params, e.g. ctx.params. A dict of configuration options.

Return type:

None

es_client.helpers.logging.de_dot(dot_string: str, msg: str) → dict

Turn message and dotted string into a nested dictionary. Used by JSONFormatter

Parameters:

• dot_string (str) – The dotted string

• msg (str) – The message

Return type:

dict

Returns:

Nested dictionary of keys with the final value being the message

es_client.helpers.logging.deepmerge(source: dict, destination: dict) → dict

Recursively merge deeply nested dictionary structures, source into destination

Parameters:

• source (dict) – Source dictionary

• destination (dict) – Destination dictionary

Returns:

destination

Return type:

dict

es_client.helpers.logging.is_docker() → bool

Check if we’re running in a docker container

es_client.helpers.logging.override_logging(config: dict, loglevel: str, logfile: str, logformat: str) → dict

Get logging config and override from command-line options

Parameters:

• config (dict) – The configuration from file

• loglevel (str) – The log level

• logfile (str) – The log file to write

• logformat (str) – Which log format to use

Returns:

Log configuration ready for validation

Return type:

dict

es_client.helpers.logging.set_logging(log_opts: dict) → None

Configure global logging options

Parameters:

log_opts (dict) – Logging configuration data

Return type:

None

class es_client.helpers.logging.Whitelist(*whitelist)

How to whitelist logs

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

filter(record)

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

class es_client.helpers.logging.Blacklist(*whitelist)

Blacklist monkey-patch of Whitelist

Initialize a filter.

Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.

filter(record)

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

class es_client.helpers.logging.LogInfo(cfg)

Logging Class

Class Setup

Parameters:

cfg – The logging configuration

Type:

cfg: dict

format_string

Attribute. The logging format string to use.

numeric_log_level

Attribute. The numeric equivalent of cfg['loglevel']

class es_client.helpers.logging.JSONFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)

JSON message formatting

Initialize the formatter with specified format strings.

Initialize the formatter either with the specified format string, or a default as described above. Allow for specialized date formatting with the optional datefmt argument. If datefmt is omitted, you get an ISO8601-like (or RFC 3339-like) format.

Use a style parameter of ‘%’, ‘{’ or ‘$’ to specify that you want to use one of %-formatting, str.format() ({}) formatting or string.Template formatting in your format string.

Changed in version 3.2: Added the style parameter.

format(record)

Parameters:

record – The incoming log message

Return type:

json.dumps()

SchemaCheck

es_client.helpers.schemacheck.password_filter(data: dict) → dict

Recursively look through all nested structures of data for the key 'password' and redact the value.

Parameters:

data (dict) – Configuration data

Returns:

A deepcopy of data with the value obscured by REDACTED if the key is 'password'.

Return type:

dict

class es_client.helpers.schemacheck.SchemaCheck(config, schema, test_what, location)

Validate config with the provided Schema. test_what and location are for reporting the results, in case of failure. If validation is successful, the method returns config as valid through the result() method.

Parameters:

• config (dict) – A configuration dictionary.

• schema (Schema) – A voluptuous schema definition

• test_what (str) – which configuration block is being validated

• location (str) – An string to report which configuration sub-block is being tested.

parse_error()

Report the error, and try to report the bad key or value as well.

result()

Return the result of the Schema test, if successful. Otherwise, raise a FailedValidation

Utils

es_client.helpers.utils.check_config(config)

Ensure that the top-level key elasticsearch and its sub-keys, other_settings and client as contained in config before passing it (or empty defaults) to SchemaCheck for value validation.

Parameters:

config (dict) – The configuration

es_client.helpers.utils.ensure_list(data)

Return a list, even if data is a single value

Parameters:

data – A list or scalar variable to act upon

Return type:

list

es_client.helpers.utils.file_exists(file: str) → bool

Verify the file exists

Parameters:

file (str) – The file to test

Returns:

Whether the file exists

Return type:

bool

es_client.helpers.utils.get_version(client) → tuple

Get the Elasticsearch version of the connected node

Parameters:

client (Elasticsearch) – An Elasticsearch client object

Returns:

The Elasticsearch version as a 3-part tuple, (major, minor, patch)

Return type:

tuple

es_client.helpers.utils.get_yaml(path: str) → dict

Read the file identified by path and import its YAML contents.

Parameters:

path (str) – The path to a YAML configuration file.

Returns:

The contents of path translated from YAML to dict

Return type:

dict

es_client.helpers.utils.option_wrapper()

Return the click decorator passthrough function

es_client.helpers.utils.parse_apikey_token(token)

Split a base64 encoded API Key Token into id and api_key

Parameters:

token (str) – The base64 encoded API Key

Returns:

A tuple of (id, api_key)

Return type:

tuple

es_client.helpers.utils.passthrough(func)

Wrapper to make it easy to store click configuration elsewhere

es_client.helpers.utils.prune_nones(mydict)

Remove keys from mydict whose values are None

Parameters:

mydict – The dictionary to act on

Return type:

dict

es_client.helpers.utils.read_file(myfile)

Read a file and return the resulting data.

Parameters:

myfile – A file to read.

Return type:

str

es_client.helpers.utils.verify_ssl_paths(args)

Verify that the various certificate/key paths are readable. The read_file() function will raise a ConfigurationError if a file fails to be read.

Parameters:

args (dict) – The client block of the config dictionary.

es_client.helpers.utils.verify_url_schema(url)

Ensure that a valid URL schema (HTTP[S]://URL:PORT) is used

Parameters:

url (str) – The url to verify

Returns:

Verified URL

Return type:

str