• API
• FAQ
• Tutorials (Java)
• Download
• Source
• Support

CloudI API Documentation

version 2.0.7
last updated on February 24^th 2024

CloudI API - Making a Service

• 1.0 - Introduction
• 1.1 - (initialization)
• 1.2 - (termination)
• 1.3 - subscribe
• 1.4 - subscribe_count
• 1.5 - unsubscribe
• 1.6 - get_pid (internal services only)
• 1.7 - get_pids (internal services only)
• 1.8 - send_sync
• 1.9 - send_async
• 1.10 - send_async_active (internal services only)
• 1.11 - mcast_async
• 1.12 - mcast_async_active (internal services only)
• 1.13 - recv_async
• 1.14 - recv_asyncs (internal services only)
• 1.15 - return
• 1.16 - forward
• 1.17 - poll (external services only)
• 1.18 - shutdown

CloudI Service API - Controlling CloudI

• 2.0 - Introduction
• 2.1 - acl_add
• 2.2 - acl_remove
• 2.3 - acl
• 2.4 - service_subscriptions
• 2.5 - services_add
• 2.6 - services_remove
• 2.7 - services_restart
• 2.8 - services_suspend
• 2.9 - services_resume
• 2.10 - services_search
• 2.11 - services_status
• 2.12 - services_update
• 2.13 - services
• 2.14 - nodes_set
• 2.15 - nodes_get
• 2.16 - nodes_add
• 2.17 - nodes_remove
• 2.18 - nodes_alive
• 2.19 - nodes_dead
• 2.20 - nodes_status
• 2.21 - nodes_status_reset (new in 2.0.8)
• 2.22 - nodes
• 2.23 - logging_set
• 2.24 - logging_file_set
• 2.25 - logging_level_set
• 2.26 - logging_stdout_set
• 2.27 - logging_syslog_set
• 2.28 - logging_formatters_set
• 2.29 - logging_redirect_set
• 2.30 - logging_status
• 2.31 - logging_status_reset
• 2.32 - logging
• 2.33 - code_path_add
• 2.34 - code_path_remove
• 2.35 - code_path
• 2.36 - code_status

CloudI API - Making a Service

1.0 - Introduction

The CloudI API provides a simple messaging API which allows CloudI services to send requests. The CloudI API supports both publish/subscribe and request/reply communication in an intuitive way. It is not necessary to understand the Erlang programming language (or Elixir), to use the CloudI API since a full CloudI API implementation is provided for every supported programming language (ATS, C/C++, Elixir, Erlang, Go, Haskell, Java, JavaScript, OCaml, Perl, PHP, Python, Ruby, and Rust, currently).

The CloudI API messaging is different from other messaging APIs and provides simpler integration for a few reasons:

• CloudI API usage requires CloudI usage as an application server to enforce fault-tolerance and scalability constraints on source code execution
• The CloudI service that receives a request determines whether a reply occurs (returning no response data is the same as not providing a reply)
• All required callbacks are minimal (only a single request callback is necessary for a CloudI service to handle requests) to keep CloudI services simpler, so they are less error-prone than other solutions
• Requests are not persisted to database storage to avoid persisting errors since errors are often transient and only relate to a specific context
• All CloudI API programming language integration makes CloudI services first-class processes within the Erlang VM to provide consistent functionality and fault tolerance
• Every CloudI API request contains a priority
• Every CloudI API request contains a unique v1 UUID for identifying the request and its response
• Every CloudI API request contains a timeout which is updated based on the queuing and processing delays the request encounters

The subscribe function subscribes to a service name pattern string which may contain "*" and "?" wildcard characters, to accept any matching service requests. Either "*" or "?" in a service name pattern will match one or more characters with "?" never matching the character that follows it ("?" is unable to be the last character, i.e., "/?/" matches "/a/" but never "/a/b/" while "/*/" will match either). The send_sync function and the send_async function provide point-to-point communication based on the service name provided. When multiple services subscribe with the same service name pattern the destination is picked based on the sending service's "destination refresh method", which can be any of the following:

The "lazy" prefix and the "immediate" prefix on the destination refresh method determines whether stale data is used within the service's data or if a single lookup process is used to get the most current destination result, respectively ("lazy" is for when long-lived services are the destination but consumes more service memory, and "immediate" is for when short-lived services are the destination but creates contention for the lookup process).

When separate service processes subscribe with the same service name pattern, each subscription is used based on random selection (if more than one service processes are available based on the destination refresh method), when a service request is sent to the service name. If the same service subscribes with the same service name pattern more than once within a single external service thread, each subscription is used in round-robin order (first subscription is called first, so order is preserved), when the service thread receives a request for the specific service name pattern.

The mcast_async function provides publish functionality by sending a request asynchronously to all services that have subscribed to the same service name pattern. To receive an asynchronous request recv_async is used with the "TransId" (i.e., Transaction Id, a v1 UUID) or a null UUID to receive the oldest service request.

The return function is used to respond to a service request and terminate the current request handler (i.e., the service request is finished, at that point). A service can return a null response if the sending service should not receive a response, which can be used for typical response-less publish functionality. The forward function provides a new destination for the same service request, delaying the request's completion, but still terminating the current request handler.

1.1 - (initialization)

The service configuration will control the CloudI API initialization, which is done automatically, but does influence the source code. The service configuration defines the number of Operating System (OS) processes to create and the number of threads for an external service. For an internal service, the configuration defines the number of Erlang processes to create. A number specified as an integer in the configuration is the exact number of processes or threads. However, if the number is specified as a floating point number, it is used as a CPU count (i.e., Erlang scheduler count) multipler where >1.0 implies floor and <1.0 implies round. The external service APIs provide the thread_count function so that the total number of threads can be used for thread creation, with each thread holding an instance of the CloudI API (to avoid lock contention):

int cloudi_initialize_thread_count(unsigned int * const thread_count);
int cloudi_initialize(cloudi_instance_t * api,
                      unsigned int const thread_index,
                      void * state);

unsigned int CloudI::API::thread_count();
CloudI::API(unsigned int const thread_index,
            bool const terminate_return_value = true);

  def cloudi_service_init(_args, _prefix, _timeout, dispatcher) do
    # ...
    {:ok, :undefined}
  end

cloudi_service_init(_Args, _Prefix, _Timeout, Dispatcher) ->
    % ...
    {ok, #state{}}.

cloudi.ThreadCount() (uint32, error)
cloudi.API(threadIndex uint32, state interface{}) (*Instance, error)

int org.cloudi.API.thread_count();
org.cloudi.API(final int thread_index);

CloudI.API.thread_count();
CloudI.API(thread_index, callback);

CloudI::API->thread_count();
CloudI::API->new($thread_index);

\CloudI\API::thread_count();
\CloudI\API($thread_index);

cloudi.API.thread_count()
cloudi.API(thread_index)

CloudI::API.thread_count()
CloudI::API.new(thread_index)

An internal service uses the configured initialization timeout to limit the execution time spent within the cloudi_service_init/4 cloudi_service behaviour callback function. An external service uses the configured initialization timeout to limit the execution time spent between creating an instance of the CloudI API and calling the CloudI API poll function (for the first time). During the service initialization the CloudI API functions that may not be called are: send_sync, recv_async, return and forward (i.e., send_sync and recv_async block to receive a service request response but service initialization must be asynchronous and within the initialization timeout period, return and forward are only valid when completing the handling of a service request).

If the initialization timeout is exceeded a service failure has occurred and a restart will occur (if possible) based on the configured MaxR (maximum restarts) and MaxT (maximum time period in seconds) service configuration values. If the service was configured in the CloudI configuration file (i.e., /usr/local/etc/cloudi/cloudi.conf, used when CloudI is first started) and one of the configured services fails initialization MaxR times, CloudI will shutdown to prevent erroneous operation. To avoid the fail-fast handling of the CloudI configuration file, the CloudI Service API services_add function may be used to provide service configuration (the return value will provide information about service initialization failures exceeding MaxR).

The service configuration also allows Access Control Lists (ACLs) to define explicit service name patterns for allowing or denying service destinations when the service sends a service request. The ACLs along with the destination refresh method determine how service requests are sent while other service options can tweak default settings.

External (non-Erlang) services are provided both the command line and the environmental variables specified within the service configuration. External sevice configuration uses the full path to the executable while internal services use the module name (and the OTP application name) within the code search paths. All environmental variables set in the shell executing the Erlang VM can be used within the executable path, arguments and environment set in the configuration of an external service, using standard shell syntax (e.g., "${USER}" or "$USER", where "\\$" is a literal "$" character).

Please see the CloudI Service API (services_add) for more details about service configuration.

Specific Language Integration Notes:

The Elixir/Erlang CloudI API functions shown below accept the most function parameters in cloudi_service but functions with less parameters do exist and they utilize default values for timeouts and request priority. Both the Timeout parameter and the Priority parameter accept the 'undefined' atom to assign the default configured value. Please see the cloudi_service module to see all the available functions and the behavior interface functions that are implemented within an Erlang service. The cloudi_service module is used within CloudI services, however, it is also possible to use CloudI services from external Erlang processes with a subset of the CloudI API functions in the cloudi module.

Both the C and the C++ CloudI API rely on the same underlying code, with the C++ API object as a wrapper around the C API pointer, so there should be no large performance difference. STL is avoided, to avoid the libstdc++ memory pool and internal memory pools are used. The C++ CloudI API functions below use the STRING type to represent either char const * const (or) std::string const &, since both are supported with overloaded functions.

The Java CloudI API doesn't have any C or C++ integration. It only uses reflection to utilize the low-level file descriptor object and store object function pointers.

The python CloudI API is provided as both the "cloudi" module and the "cloudi_c" module. The "cloudi_c" module uses the C++ CloudI API for more efficiency, while the "cloudi" module only uses Python source code.

1.2 - (termination)

An internal service uses the termination timeout to limit the execution time spent within the cloudi_service_terminate/3 cloudi_service behaviour callback function. An external service uses the termination timeout to limit the execution time spent between returning from the CloudI API poll function and the service OS process exit. The termination timout is slightly less than MaxT (maximum time period in seconds) divided by MaxR (maximum restarts) to ensure service failures are finite (MaxR and MaxT are both service configuration values). During termination no CloudI API functions may be called. The termination execution time is used to cleanup the service's state (e.g., close connections or files).

1.3 - subscribe

typedef void (*cloudi_callback_t)(int const request_type,
                                  char const * const name,
                                  char const * const pattern,
                                  void const * const request_info,
                                  uint32_t const request_info_size,
                                  void const * const request,
                                  uint32_t const request_size,
                                  uint32_t timeout,
                                  int8_t priority,
                                  char const * const trans_id,
                                  char const * const source,
                                  uint32_t const source_size,
                                  void * state,
                                  cloudi_instance_t * api);
int cloudi_subscribe(cloudi_instance_t * api,
                     char const * const pattern,
                     cloudi_callback_t f);

template <typename T>
int CloudI::API::subscribe(STRING pattern,
                           T & object,
                           void (T::*f) (CloudI::API const &,
                                         int const,
                                         STRING,
                                         STRING,
                                         void const * const,
                                         uint32_t const,
                                         void const * const,
                                         uint32_t const,
                                         uint32_t,
                                         int8_t,
                                         char const * const,
                                         char const * const,
                                         uint32_t const)) const;
int CloudI::API::subscribe(STRING pattern,
                           void (*f) (API const &,
                                      int const,
                                      STRING,
                                      STRING,
                                      void const * const,
                                      uint32_t const,
                                      void const * const,
                                      uint32_t const,
                                      uint32_t,
                                      int8_t,
                                      char const * const,
                                      char const * const,
                                      uint32_t const)) const

:cloudi_service.subscribe(dispatcher, pattern)

cloudi_service:subscribe(Dispatcher :: pid(), Pattern :: string()) ->
    ok.

func (api *cloudi.Instance) Subscribe(pattern string,
                                      function cloudi.Callback) error

void org.cloudi.API.subscribe(final String pattern,
                              final Object instance,
                              final String methodName);
// with Java ≥ 8 a method reference can be used
void org.cloudi.API.subscribe(final String pattern,
                              final FunctionInterface9 callback);

var callback = function () {};
CloudI.API.subscribe(pattern, object, object_function, callback);

CloudI::API->subscribe($pattern, $function);
CloudI::API->subscribe($pattern, $object, $method);

\CloudI\API::subscribe($pattern, $object, $method);

cloudi.API.subscribe(pattern, function)

CloudI::API.subscribe(pattern, function)

Subscribes with a service name pattern which provides a destination for other services to send to. The subscribing service will receive a service request, if a different service sends a service request with a service name that matches the service name pattern. The service name pattern is a string that may contain "*" and "?" wildcard characters, to accept any matching service requests. Either "*" or "?" in a service name pattern will match one or more characters with "?" never matching the character that follows it ("?" is unable to be the last character, i.e., "/?/" matches "/a/" but never "/a/b/" while "/*/" will match either). The service names and service name patterns are expected to be in a filepath format (e.g., "/root/directory/file.extension") by some provided CloudI services, though nothing enforces this convention. Good design dictates that service names operate within a given scope. Both the service names and the service name patterns should represent an appropriate scope, which the service manages (i.e., the same concept as a Uniform Resource Identifier (URI)).

When a service subscribes to a service name pattern, the supplied pattern string is appended to the service name prefix from the service's configuration, to provide the full service name pattern. The prefix provided within the service's configuration declares the scope of all service operations, as they are seen from other running services. Multiple subscribe function calls can increase the probability of receiving a service request when other services are subscribed with the same service name pattern.

1.4 - subscribe_count

int cloudi_subscribe_count(cloudi_instance_t * api,
                           char const * const pattern);
// cloudi_get_subscribe_count(p) to get the result

int CloudI::API::subscribe_count(STRING pattern) const;
// CloudI::API::get_subscribe_count() to get the result

:cloudi_service.subscribe_count(dispatcher, pattern)

cloudi_service:subscribe_count(Dispatcher :: pid(),
                               Pattern :: string()) ->
    non_neg_integer().

func (api *cloudi.Instance) SubscribeCount(pattern string)
                                          (uint32, error)

int org.cloudi.API.subscribe_count(final String pattern);
// return value is result

var callback = function (count) {};
CloudI.API.subscribe_count(pattern, callback);

CloudI::API->subscribe_count($pattern);

\CloudI\API::subscribe_count($pattern);

cloudi.API.subscribe_count(pattern)
# return value is result

CloudI::API.subscribe_count(pattern)
# return value is result

Provide a count of how many times a subscription has occurred for a specific service name pattern with the current service process. Often the result is either 0 or 1, but it is possible to subscribe any number of times to change the probability of the service process getting a service request, when many service processes are subscribing with the same service name. This function will always check the authoritative service process registry, so the result is always correct at that point in time (i.e., a lazy destination refresh method will not affect the subscribe_count result).

subscribe_count would be most common during testing and first learning about CloudI, but the function may also be used if the subscribe function usage is complex and not tracked separately (i.e., tracked within the service process' internal state).

1.5 - unsubscribe

int cloudi_unsubscribe(cloudi_instance_t * api,
                       char const * const pattern);

int CloudI::API::unsubscribe(STRING pattern) const;

:cloudi_service.unsubscribe(dispatcher, pattern)

cloudi_service:unsubscribe(Dispatcher :: pid(), Pattern :: string()) ->
    ok.

func (api *cloudi.Instance) Unsubscribe(pattern string) error

void org.cloudi.API.unsubscribe(final String pattern);

var callback = function () {};
CloudI.API.unsubscribe(pattern, callback);

CloudI::API->unsubscribe($pattern);

\CloudI\API::unsubscribe($pattern);

cloudi.API.unsubscribe(pattern)

CloudI::API.unsubscribe(pattern)

Unsubscribe will remove the service's subscription for the specific service name pattern. If a service has subscribed with the same service name pattern multiple times, the unsubscribe will only remove one subscription instance. The subscription instance which is removed is whatever subscription would have been called next, for a matching service request.

1.6 - get_pid (internal services only)

:cloudi_service.get_pid(dispatcher, name, timeout)

cloudi_service:get_pid(Dispatcher :: pid(),
                       Name :: string(),
                       Timeout :: non_neg_integer() | 'undefined' |
                                  'limit_min' | 'limit_max') ->
    {'ok', PatternPid :: {string(), pid()}} |
    {'error', Reason :: atom()}.

Internal (Elixir/Erlang-only) services can request an Erlang process based on the service name provided, before calling either the send_sync function or the send_async function. The get_pid function should rarely be necessary, but it can allow other logic to be used for determining which service should receive a request (e.g., based on apparent processing power, like within the hexpi test). The Erlang PatternPid tuple returned could become invalid if the service destination terminated, so the Erlang process monitoring becomes the burden of the get_pid function user. Due to the intimate nature of this function, it only exists within the Elixir/Erlang CloudI API (to implement it in other languages would cause service destination inconsistencies due to the function delay and the potential storage before the destination is used).

The get_pid function provides a way to split the service name lookup latency from the service request latency so that two separate timeout values can be used, instead of a single timeout.

1.7 - get_pids (internal services only)

:cloudi_service.get_pids(dispatcher, name, timeout)

cloudi_service:get_pids(Dispatcher :: pid(),
                        Name :: string(),
                        Timeout :: non_neg_integer() | 'undefined' |
                                   'limit_min' | 'limit_max') ->
    {'ok', PatternPids :: nonempty_list({string(), pid()})} |
    {'error', Reason :: atom()}.

Internal (Elixir/Erlang-only) services can request a list of Erlang processes based on the service name provided, before calling either the send_sync function or the send_async function. If all Erlang processes returned need to be used with send_async, it is easier to use the mcast_async function. The get_pids function should rarely be necessary, but it can allow other logic to be used for determining which service should receive a request (e.g., based on apparent processing power, like within the hexpi test). The Erlang PatternPids tuple list returned could contain invalid Erlang processes if the service destination terminated, so the Erlang process monitoring becomes the burden of the get_pids function user. Due to the intimate nature of this function, it only exists within the Elixir/Erlang CloudI API (to implement it in other languages would cause service destination inconsistencies due to the function delay and the potential storage before the destination is used).

The get_pids function provides a way to split the service name lookup latency from the service request latency so that two separate timeout values can be used, instead of a single timeout (e.g., with mcast_async).

1.8 - send_sync

int cloudi_send_sync_(cloudi_instance_t * api,
                      char const * const name,
                      void const * const request_info,
                      uint32_t const request_info_size,
                      void const * const request,
                      uint32_t const request_size,
                      uint32_t timeout,
                      int8_t const priority);

int CloudI::API::send_sync(STRING name,
                           void const * const request_info,
                           uint32_t const request_info_size,
                           void const * const request,
                           uint32_t const request_size,
                           uint32_t timeout,
                           int8_t const priority) const;

:cloudi_service.send_sync(dispatcher, name,
                          request_info, request, timeout, priority)
:cloudi_service.send_sync(dispatcher, name,
                          request_info, request, timeout, priority,
                          pattern_pid)

cloudi_service:send_sync(Dispatcher :: pid(),
                         Name :: string(),
                         RequestInfo :: any(),
                         Request :: any(),
                         Timeout :: non_neg_integer() | 'undefined' |
                                    'limit_min' | 'limit_max',
                         Priority :: integer() | 'undefined') ->
    {'ok', ResponseInfo :: any(), Response :: any()} |
    {'ok', Response :: any()} |
    {'error', Reason :: atom()}.
cloudi_service:send_sync(Dispatcher :: pid(),
                         Name :: string(),
                         RequestInfo :: any(),
                         Request :: any(),
                         Timeout :: non_neg_integer() | 'undefined' |
                                    'limit_min' | 'limit_max',
                         Priority :: integer() | 'undefined',
                         PatternPid :: {string(), pid()}) ->
    {'ok', ResponseInfo :: any(), Response :: any()} |
    {'ok', Response :: any()} |
    {'error', Reason :: atom()}.

func (api *cloudi.Instance) SendSync(name string,
                                     requestInfo, request []byte,
                                     timeoutPriority ...interface{})
                                    ([]byte, []byte, []byte, error)

Response org.cloudi.API.send_sync(String name, byte[] request_info,
                                  byte[] request, Integer timeout,
                                  Byte priority);

CloudI.API.send_sync(name, request, callback,
                     timeout, request_info, priority);

CloudI::API->send_sync($name, $request,
                       $timeout, $request_info, $priority);

\CloudI\API::send_sync($name, $request,
                       $timeout = null, $request_info = null,
                       $priority = null);

cloudi.API.send_sync(name, request,
                     timeout=None, request_info=None, priority=None)

CloudI::API.send_sync(name, request,
                      timeout=nil, request_info=nil, priority=nil)

Send a synchronous request to a service name with a specific timeout and a specific priority. If a timeout is not provided, the default synchronous timeout from the service configuration is used. If a priority is not provided, the default priority from the service configuration options is used (normally the default priority is 0).

cloudi_get_response(p)
cloudi_get_response_size(p)
cloudi_get_response_info(p)
cloudi_get_response_info_size(p)
cloudi_get_trans_id_count(p)
cloudi_get_trans_id(p, i)

char const * CloudI::API::get_response() const;
uint32_t CloudI::API::get_response_size() const;
char const * CloudI::API::get_response_info() const;
uint32_t CloudI::API::get_response_info_size() const;
uint32_t CloudI::API::get_trans_id_count() const;
char const * CloudI::API::get_trans_id(unsigned int const i = 0) const;

{:ok, response_info, response}
{:ok, response}
{:error, reason}

{'ok', ResponseInfo :: any(), Response :: any()}
{'ok', Response :: any()}
{'error', Reason :: atom()}

(responseInfo, response, transId, err)

org.cloudi.API.Response

callback(response_info, response, trans_id);

($response_info, $response, $trans_id)

array($response_info, $response, $trans_id)

(response_info, response, trans_id)

[response_info, response, trans_id]

The send_sync response data is provided in ways typical to each programming language, as shown above. The non-Erlang send_sync functions provide the TransId of the request because the calling service may need to use the v1 UUID to manipulate and/or store the response.

1.9 - send_async

int cloudi_send_async_(cloudi_instance_t * api,
                       char const * const name,
                       void const * const request_info,
                       uint32_t const request_info_size,
                       void const * const request,
                       uint32_t const request_size,
                       uint32_t timeout,
                       int8_t const priority);

int CloudI::API::send_async(STRING name,
                            void const * const request_info,
                            uint32_t const request_info_size,
                            void const * const request,
                            uint32_t const request_size,
                            uint32_t timeout,
                            int8_t const priority) const;

:cloudi_service.send_async(dispatcher, name,
                           request_info, request, timeout, priority)
:cloudi_service.send_async(dispatcher, name,
                           request_info, request, timeout, priority,
                           pattern_pid)

cloudi_service:send_async(Dispatcher :: pid(),
                          Name :: string(),
                          RequestInfo :: any(),
                          Request :: any(),
                          Timeout :: non_neg_integer() | 'undefined' |
                                    'limit_min' | 'limit_max',
                          Priority :: integer() | 'undefined') ->
    {'ok', TransId :: <<_:128>>} |
    {'error', Reason :: atom()}.
cloudi_service:send_async(Dispatcher :: pid(),
                          Name :: string(),
                          RequestInfo :: any(),
                          Request :: any(),
                          Timeout :: non_neg_integer() | 'undefined' |
                                    'limit_min' | 'limit_max',
                          Priority :: integer() | 'undefined',
                          PatternPid :: {string(), pid()}) ->
    {'ok', TransId :: <<_:128>>} |
    {'error', Reason :: atom()}.

func (api *cloudi.Instance) SendAsync(name string,
                                      requestInfo, request []byte,
                                      timeoutPriority ...interface{})
                                     ([]byte, error)

TransId org.cloudi.API.send_async(String name, byte[] request_info,
                                  byte[] request, Integer timeout,
                                  Byte priority);

CloudI.API.send_async(name, request, callback,
                      timeout, request_info, priority);

CloudI::API->send_async($name, $request,
                        $timeout, $request_info, $priority);

\CloudI\API::send_async($name, $request,
                        $timeout = null, $request_info = null,
                        $priority = null);

cloudi.API.send_async(name, request,
                      timeout=None, request_info=None, priority=None)

CloudI::API.send_async(name, request,
                       timeout=nil, request_info=nil, priority=nil)

Send an asynchronous request to a service name with a specific timeout and a specific priority. If a timeout is not provided, the default asynchronous timeout from the service configuration is used. If a priority is not provided, the default priority from the service configuration options is used (normally the default priority is 0).

An asynchronous send will block until a live service matches the service name destination or the timeout expires (when the service configuration option request_name_lookup is set to 'sync', the default, if an asynchronous lookup is required set request_name_lookup to 'async'). Once the asynchronous request is sent the TransId which identifies the request is returned.

cloudi_get_trans_id_count(p)
cloudi_get_trans_id(p, i)

uint32_t CloudI::API::get_trans_id_count() const;
char const * CloudI::API::get_trans_id(unsigned int const i = 0) const;

{:ok, trans_id}
{:error, reason}

{'ok', TransId :: <<_:128>>}
{'error', Reason :: atom()}

(transId, err)

org.cloudi.API.TransId

callback(trans_id);

$trans_id

$trans_id

trans_id

trans_id

The send_async result is provided in ways typical to each programming language, as shown above. A TransId is a v1 UUID.

1.10 - send_async_active (internal services only)

:cloudi_service.send_async_active(dispatcher, name,
                                  request_info, request,
                                  timeout, priority)
:cloudi_service.send_async_active(dispatcher, name,
                                  request_info, request,
                                  timeout, priority,
                                  pattern_pid)

cloudi_service:send_async_active(Dispatcher :: pid(),
                                 Name :: string(),
                                 RequestInfo :: any(),
                                 Request :: any(),
                                 Timeout :: non_neg_integer() |
                                            'undefined' |
                                            'limit_min' | 'limit_max',
                                 Priority :: integer() | 'undefined') ->
    {'ok', TransId :: <<_:128>>} |
    {'error', atom()}.
cloudi_service:send_async_active(Dispatcher :: pid(),
                                 Name :: string(),
                                 RequestInfo :: any(),
                                 Request :: any(),
                                 Timeout :: non_neg_integer() |
                                            'undefined' |
                                            'limit_min' | 'limit_max',
                                 Priority :: integer() | 'undefined',
                                 PatternPid :: {string(), pid()}) ->
    {'ok', TransId :: <<_:128>>} |
    {'error', atom()}.

The send_async_active function provides the same functionality as the send_async function within an Erlang process, but the response is automatically sent to the Erlang process, after completion. Using send_async_active is the preferred way to send an asynchronous service request in Erlang because it utilizes Erlang's concurrency without requiring a blocking operation (a passive send, using Erlang vernacular, since it would otherwise require a call of the function recv_async to receive the request). The send_async_active function is not implemented in other languages because of their lack of native event handling.

{:return_async_active, name, pattern,
 response_info, response, timeout, trans_id}
{:timeout_async_active, trans_id}

{'return_async_active', Name :: string(), Pattern :: string(),
 ResponseInfo :: any(), Response :: any(),
 Timeout :: non_neg_integer(), TransId :: <<_:128>>}
{'timeout_async_active', TransId :: <<_:128>>}

The send_async_active message is sent to the Erlang process as an Erlang message, so it arrives in the cloudi_service_handle_info function of the Erlang service module (i.e., the module that implements the cloudi_service behavior). The message formats are also provided as records that are accessible with:

-include_lib("cloudi_core/include/cloudi_service.hrl").

1.11 - mcast_async

int cloudi_mcast_async_(cloudi_instance_t * api,
                        char const * const name,
                        void const * const request_info,
                        uint32_t const request_info_size,
                        void const * const request,
                        uint32_t const request_size,
                        uint32_t timeout,
                        int8_t const priority);

int CloudI::API::mcast_async(STRING name,
                             void const * const request_info,
                             uint32_t const request_info_size,
                             void const * const request,
                             uint32_t const request_size,
                             uint32_t timeout,
                             int8_t const priority) const;

:cloudi_service.mcast_async(dispatcher, name,
                            request_info, request, timeout, priority)

cloudi_service:mcast_async(Dispatcher :: pid(),
                           Name :: string(),
                           RequestInfo :: any(),
                           Request :: any(),
                           Timeout :: non_neg_integer() | 'undefined' |
                                      'limit_min' | 'limit_max',
                           Priority :: integer() | 'undefined') ->
    {'ok', TransIdList :: list(<<_:128>>)} |
    {'error', Reason :: atom()}.

func (api *cloudi.Instance) McastAsync(name string,
                                       requestInfo, request []byte,
                                       timeoutPriority ...interface{})
                                      ([][]byte, error)

List<TransId> org.cloudi.API.mcast_async(String name,
                                         byte[] request_info,
                                         byte[] request,
                                         Integer timeout,
                                         Byte priority);

CloudI.API.mcast_async(name, request, callback,
                       timeout, request_info, priority);

CloudI::API->mcast_async($name, $request,
                         $timeout, $request_info, $priority);

\CloudI\API::mcast_async($name, $request,
                         $timeout = null, $request_info = null,
                         $priority = null);

cloudi.API.mcast_async(name, request,
                       timeout=None, request_info=None, priority=None)

CloudI::API.mcast_async(name, request,
                        timeout=nil, request_info=nil, priority=nil)

Multicast asynchronously, which is the same as publish, except that it is possible to respond to the service request. The function mcast_async will send the service request asynchronously to all services that have subscribed to a service name pattern that matches the service name destination. The mcast_async function will block until at least a single request has been sent or the timeout has expired (when the service configuration option request_name_lookup is set to 'sync', the default, if an asynchronous lookup is required set request_name_lookup to 'async'). The result of the function call is a list of TransIds (one TransId per service request). If a publish request is required, the destination service should have a null response, so that the service request response is ignored.

cloudi_get_trans_id_count(p)
cloudi_get_trans_id(p, i)

uint32_t CloudI::API::get_trans_id_count() const;
char const * CloudI::API::get_trans_id(unsigned int const i = 0) const;

{:ok, trans_id_list}
{:error, reason}

{'ok', TransIdList :: list(<<_:128>>)}
{'error', Reason :: atom()}

(transIds, err)

List<org.cloudi.API.TransId>

callback(trans_ids);

@trans_ids

$trans_ids

[trans_id]

[trans_id]

The mcast_async result is provided in ways typical to each programming language, as shown above. A TransId is a v1 UUID.

1.12 - mcast_async_active (internal services only)

:cloudi_service.mcast_async_active(dispatcher, name,
                                   request_info, request,
                                   timeout, priority)

cloudi_service:mcast_async_active(Dispatcher :: pid(),
                                  Name :: string(),
                                  RequestInfo :: any(),
                                  Request :: any(),
                                  Timeout :: non_neg_integer() |
                                             'undefined' |
                                             'limit_min' | 'limit_max',
                                  Priority :: integer() |
                                              'undefined') ->
    {'ok', TransIdList :: list(<<_:128>>)} |
    {'error', Reason :: atom()}.

The mcast_async_active function provides the same functionality as the mcast_async function within an Erlang process, but the response is automatically sent to the Erlang process, after completion. Using mcast_async_active is the preferred way to publish an asynchronous service request in Erlang because it utilizes Erlang's concurrency without requiring a blocking operation (a passive send, using Erlang vernacular, since it would otherwise require a call of the function recv_async to receive the request). The mcast_async_active function is not implemented in other languages because of their lack of native event handling.

{:return_async_active, name, pattern,
 response_info, response, timeout, trans_id}
{:timeout_async_active, trans_id}

{'return_async_active', Name :: string(), Pattern :: string(),
 ResponseInfo :: any(), Response :: any(),
 Timeout :: non_neg_integer(), TransId :: <<_:128>>}
{'timeout_async_active', TransId :: <<_:128>>}

The mcast_async_active message is sent to the Erlang process as an Erlang message, so it arrives in the cloudi_service_handle_info function of the Erlang service module (i.e., the module that implements the cloudi_service behavior). The message formats are also provided as records that are accessible with:

-include_lib("cloudi_core/include/cloudi_service.hrl").

1.13 - recv_async

int cloudi_recv_async(cloudi_instance_t * api,
                      uint32_t timeout,
                      char const * const trans_id,
                      int consume);

int CloudI::API::recv_async(uint32_t timeout,
                            STRING trans_id,
                            bool consume) const;

:cloudi_service.recv_async(dispatcher, timeout, trans_id, consume)

cloudi_service:recv_async(Dispatcher :: pid(),
                          Timeout :: non_neg_integer() | 'undefined' |
                                     'limit_min' | 'limit_max',
                          TransId :: <<_:128>>,
                          Consume :: boolean()) ->
    {'ok', ResponseInfo :: any(), Response :: any(),
           TransId :: <<_:128>>} |
    {'error', Reason :: atom()}.

func (api *cloudi.Instance) RecvAsync(extra ...interface{})
                                     ([]byte, []byte, []byte, error)

Response org.cloudi.API.recv_async(Integer timeout, byte[] transId,
                                   boolean consume);

CloudI.API.recv_async(callback,
                      timeout, trans_id, consume);

CloudI::API->recv_async($timeout, $trans_id, $consume);

\CloudI\API::recv_async($timeout = null, $trans_id = null,
                        $consume = true);

cloudi.API.recv_async(timeout=None, trans_id=None, consume=True)

CloudI::API.recv_async(timeout=nil, trans_id=nil, consume=true)

Receive an asynchronous service request's response. If a TransId is not provided, a null UUID is used to request the oldest response that has not timed out. By default, the recv_async function will consume the service request so it is not accessible with the same function call in the future. The TransId of the service request is always returned for any external use or tracking of the request or response.

cloudi_get_response(p)
cloudi_get_response_size(p)
cloudi_get_response_info(p)
cloudi_get_response_info_size(p)
cloudi_get_trans_id_count(p)
cloudi_get_trans_id(p, i)

char const * CloudI::API::get_response() const;
uint32_t CloudI::API::get_response_size() const;
char const * CloudI::API::get_response_info() const;
uint32_t CloudI::API::get_response_info_size() const;
uint32_t CloudI::API::get_trans_id_count() const;
char const * CloudI::API::get_trans_id(unsigned int const i = 0) const;

{:ok, response_info, response, trans_id}
{:error, reason}

{'ok', ResponseInfo :: any(), Response :: any(),
       TransId :: <<_:128>>}
{'error', Reason :: atom()}

(responseInfo, response, transId, err)

org.cloudi.API.Response

callback(response_info, response, trans_id);

($response_info, $response, $trans_id)

array($response_info, $response, $trans_id)

(response_info, response, trans_id)

[response_info, response, trans_id]

1.14 - recv_asyncs (internal services only)

:cloudi_service.recv_asyncs(dispatcher,
                            timeout, trans_id_list, consume)

cloudi_service:recv_asyncs(Dispatcher :: pid(),
                           Timeout :: non_neg_integer() | 'undefined' |
                                      'limit_min' | 'limit_max',
                           TransIdList :: list(<<_:128>>),
                           Consume :: boolean()) ->
    {'ok', list({ResponseInfo :: any(), Response :: any(),
                 TransId :: <<_:128>>})} |
    {'error', Reason :: atom()}.

Internal (Elixir/Erlang-only) services can block to receive multiple asynchronous service request responses. By default, the recv_asyncs function will consume the service request so it is not accessible with the same function call in the future. The TransId of the service request is always returned for any external use or tracking of the request or response. The recv_asyncs function is not implemented in other languages to avoid unbounded memory consumption and caching/heap allocation impossibilities.

1.15 - return

int cloudi_return(cloudi_instance_t * api,
                  int const request_type,
                  char const * const name,
                  char const * const pattern,
                  void const * const response_info,
                  uint32_t const response_info_size,
                  void const * const response,
                  uint32_t const response_size,
                  uint32_t timeout,
                  char const * const trans_id,
                  char const * const source,
                  uint32_t const source_size);

int CloudI::API::return_(int const request_type,
                         STRING name,
                         STRING pattern,
                         void const * const response_info,
                         uint32_t const response_info_size,
                         void const * const response,
                         uint32_t const response_size,
                         uint32_t timeout,
                         char const * const trans_id,
                         char const * const source,
                         uint32_t const source_size) const;

:cloudi_service.return(dispatcher, request_type, name, pattern,
                       response_info, response,
                       timeout, trans_id, source)

cloudi_service:return(Dispatcher :: pid(),
                      RequestType :: 'send_async' | 'send_sync',
                      Name :: string(),
                      Pattern :: string(),
                      ResponseInfo :: any(),
                      Response :: any(),
                      Timeout :: non_neg_integer(),
                      TransId :: <<_:128>>,
                      Source :: pid()) ->
    none().

func (api *cloudi.Instance) Return(requestType int,
                                   name, pattern string,
                                   responseInfo, response []byte,
                                   timeout uint32,
                                   transId [16]byte,
                                   source cloudi.Source)

void org.cloudi.API.return_(Integer request_type,
                            String name, String pattern,
                            byte[] response_info, byte[] response,
                            Integer timeout, byte[] transId,
                            OtpErlangPid source);

CloudI.API.return_(request_type, name, pattern,
                   response_info, response, timeout, trans_id, source);

CloudI::API->return_($request_type, $name, $pattern,
                     $response_info, $response,
                     $timeout, $trans_id, $source);

\CloudI\API::return_($request_type, $name, $pattern,
                     $response_info, $response,
                     $timeout, $trans_id, $source);

cloudi.API.return_(request_type, name, pattern, response_info, response,
                   timeout, trans_id, source)

CloudI::API.return_(request_type, name, pattern, response_info, response,
                    timeout, trans_id, source)

Return a response to a service request. The return function will throw a caught exception so that the request handler execution is aborted after returning the service request response. The simplest and preferred way to return a response within an Erlang service is to utilize the cloudi_service_handle_request functon return values used by the cloudi_service behavior. You can also utilize the request handler return value for the response in the programming languages Java, Python, and Ruby. However, within the external services it is more explicit (i.e., easier to understand the source code) when the source code uses the return functions.

If the service is configured with the request_timeout_adjustment option set to true (the default is false), the request handler execution time will automatically decrement the request timeout, after the request has been handled. If the service is configured with the response_timeout_adjustment option set to true (the default is false), the response timeout is automatically decremented based on the sender-side's timing (more accurate).

1.16 - forward

int cloudi_forward(cloudi_instance_t * api,
                   int const request_type,
                   char const * const name,
                   void const * const request_info,
                   uint32_t const request_info_size,
                   void const * const request,
                   uint32_t const request_size,
                   uint32_t timeout,
                   int8_t const priority,
                   char const * const trans_id,
                   char const * const source,
                   uint32_t const source_size);

int CloudI::API::forward_(int const request_type,
                          STRING name,
                          void const * const request_info,
                          uint32_t const request_info_size,
                          void const * const request,
                          uint32_t const request_size,
                          uint32_t timeout,
                          int8_t const priority,
                          char const * const trans_id,
                          char const * const source,
                          uint32_t const source_size) const;

:cloudi_service.forward(dispatcher, request_type, name,
                        request_info, request,
                        timeout, priority, trans_id, source)

cloudi_service:forward(Dispatcher :: pid(),
                       RequestType :: 'send_async' | 'send_sync',
                       Name :: string(),
                       RequestInfo :: any(),
                       Request :: any(),
                       Timeout :: non_neg_integer(),
                       Priority :: integer(),
                       TransId :: <<_:128>>,
                       Source :: pid()) ->
    none().

func (api *cloudi.Instance) Forward(requestType int,
                                    name string,
                                    requestInfo, request []byte,
                                    timeout uint32,
                                    priority int8,
                                    transId [16]byte,
                                    source cloudi.Source)

Response org.cloudi.API.forward_(Integer request_type, String name,
                                 byte[] request_info, byte[] request,
                                 Integer timeout, Byte priority,
                                 byte[] transId, OtpErlangPid source);

CloudI.API.forward_(request_type, name, request_info, request,
                    timeout, priority, trans_id, source);

CloudI::API->forward_($request_type, $name, $request_info, $request,
                      $timeout, $priority, $trans_id, $source);

\CloudI\API::forward_($request_type, $name, $request_info, $request,
                      $timeout, $priority, $trans_id, $source);

cloudi.API.forward_(request_type, name, request_info, request,
                    timeout, priority, trans_id, source)

CloudI::API.forward_(request_type, name, request_info, request,
                     timeout, priority, trans_id, source)

Forward the service request to a different destination, possibly with different parameters (e.g., a completely different request). The forward function will throw a caught exception so that the request handler execution is aborted after forwarding the service request. The simplest and preferred way to forward a request within an Erlang service is to utilize the cloudi_service_handle_request functon return values used by the cloudi_service behavior. All external services must use a forward function when forwarding a request.

If the service is configured with the request_timeout_adjustment option set to true (the default is false), the request handler execution time will automatically decrement the request timeout, after the request has been handled. If the service is configured with the response_timeout_adjustment option set to true (the default is false), the response timeout is automatically decremented based on the sender-side's timing (more accurate).

1.17 - poll (external services only)

int cloudi_poll(cloudi_instance_t * api,
                int timeout);

int CloudI::API::poll(int timeout = -1);

func (api *cloudi.Instance) Poll(timeout int32) (bool, error)

boolean org.cloudi.API.poll(int timeout);

var callback = function (timeout_) {};
CloudI.API.poll(callback, timeout);

CloudI::API->poll($timeout);

\CloudI\API::poll($timeout = -1);

cloudi.API.poll(timeout=-1)

CloudI::API.poll(timeout=nil)

External services use the poll function to accept service requests while blocking execution until either the timeout value expires or the service terminates. The execution time before the first poll function call is service initialization. The timeout value is specified in milliseconds. A timeout value of 0 can be used to not block on the poll function call. If the timeout value is not provided a value of -1 is used to make the poll function call block until service termination (if the programming language allows default function argument or function overloading). A boolean true value is used as the poll function return value if a timeout occurred (a non zero return value if the return value is an integer). A boolean false value is used as the poll function return value if service termination is in-progress.

1.18 - shutdown

int cloudi_shutdown(cloudi_instance_t * api,
                    char const * const reason);

int CloudI::API::shutdown() const;
int CloudI::API::shutdown(STRING reason) const;

% cloudi_service behaviour return values:
{:stop, :shutdown}
{:stop, {:shutdown, 'reason'}}
{:stop, :shutdown, state}
{:stop, {:shutdown, 'reason'}, state}

% cloudi_service behaviour return values:
{stop, shutdown}
{stop, {shutdown, "reason"}}
{stop, shutdown, State}
{stop, {shutdown, "reason"}, State}

func (api *cloudi.Instance) Shutdown(extra ...interface{}) error

void org.cloudi.API.shutdown();
void org.cloudi.API.shutdown(String reason);

CloudI.API.shutdown(callback, reason);

CloudI::API->shutdown($reason);

\CloudI\API::shutdown($reason = null);

cloudi.API.shutdown(reason=None)

CloudI::API.shutdown(reason=nil)

The shutdown functionality provides a way of successfully stopping the service instance when no error has occurred. All service processes are stopped and the service is removed without causing any service process restarts. An optional reason string may be provided to log the reason for the service shutdown.

CloudI Service API - Controlling CloudI

2.0 - Introduction

When CloudI is first started, the configuration file at /usr/local/etc/cloudi/cloudi.conf is used to determine what Access Control Lists (ACLs) should be used for services, what services should be started, what nodes should be connected, and what logging should occur. All the configuration functionality for CloudI can be done dynamically, after startup, with the CloudI Service API. A typical way to use the Service API is with either erlang terms or JSON-RPC over HTTP (using cloudi_service_api_requests and cloudi_service_http_cowboy). The CloudI Service API can also be accessed directly within the Erlang VM by using the cloudi_service_api module.

The data returned in both examples is Erlang terms within a string. All of the examples below use the Erlang protocol.

2.1 - acl_add

curl -X POST -d '[{sensitive, ["/accouting/*", "/finance/*"]}]' http://localhost:6464/cloudi/api/rpc/acl_add.erl

Add more ACL entries to be later used when starting services. An ACL entry is an Erlang atom() -> list(atom() | string()) relationship which provides a logical grouping of service name patterns (e.g., {api, ["/cloudi/api/*"]}).

2.2 - acl_remove

curl -X POST -d '[sensitive]' http://localhost:6464/cloudi/api/rpc/acl_remove.erl

Remove ACL entries that are no longer needed. Running services will retain their configuration, so this impacts services that are started in the future.

2.3 - acl

curl http://localhost:6464/cloudi/api/rpc/acl.erl

List all the current ACL entries as lists of service name patterns.

2.4 - service_subscriptions

curl -X POST -d '"6a675470-7a1f-11e2-d40e-a5dd00000058"' http://localhost:6464/cloudi/api/rpc/service_subscriptions.erl

List the subscriptions a service instance has initiated.

2.5 - services_add

curl -X POST -d '[{internal, "/tests/http_req/", cloudi_service_request_rate, [{service_name, "/tests/http_req/ruby.xml/get"}, {request_rate, dynamic}], lazy_closest, 5000, 5000, 5000, undefined, undefined, 1, 5, 300, [{duo_mode, true}]}]' http://localhost:6464/cloudi/api/rpc/services_add.erl

Start services and return their Service UUIDs. Provide service configuration using the same syntax found in the configuration file (i.e., /usr/local/etc/cloudi/cloudi.conf). Internal services will need to be located in a code path that the running Erlang VM is aware of (see code_path_add). The syntax of the configuration entries is shown below:

% proplist format with cloudi_service_api types
[{type, internal | external},               % inferred from module or file_path
 {prefix, cloudi:service_name_pattern()},   % default is "/"
 {module, atom() | file:filename()},        % internal service only
 {file_path, file:filename()},              % external service only
 {args, list()},                            % default is []
 {env, list({string(), string()})},         % default is []
 {dest_refresh, dest_refresh()},            % default is immediate_closest
 {protocol, default | local | tcp | udp},   % default is local
 {buffer_size, default | pos_integer()},    % default is 65536 bytes
 {timeout_init, 101..4294967195},           % default is 5000 milliseconds
 {timeout_async, 499..4294967295},          % default is 5000 milliseconds
 {timeout_sync, 499..4294967295},           % default is 5000 milliseconds
 {dest_list_deny, dest_list()},             % default is undefined
 {dest_list_allow, dest_list()},            % default is undefined
 {count_process, pos_integer() | float()},  % default is 1
 {count_thread, pos_integer() | float()},   % default is 1
 {max_r, non_neg_integer()},                % default is 5
 {max_t, seconds()},                        % default is 300 seconds
 {options, service_options_internal() |     % default is []
           service_options_external()}]

% internal service tuple format
{internal,
 (ServiceNamePrefix),
 (ErlangModuleName),
 (ModuleInitializationList),
 (DestinationRefreshMethod),
 (InitializationTimeout in milliseconds),
 (DefaultAsynchronousTimeout in milliseconds),
 (DefaultSynchronousTimeout in milliseconds),
 (DestinationDenyACL),
 (DestinationAllowACL),
 (ProcessCount),
 (MaxR),
 (MaxT in seconds),
 (ServiceOptionsPropList)}

% external service tuple format
{external,
 (ServiceNamePrefix),
 (ExecutableFilePath),
 (ExecutableCommandLineArguments),
 (ExecutableEnvironmentalVariables),
 (DestinationRefreshMethod),
 (Protocol, use 'default'),
 (ProtocolBufferSize, use 'default'),
 (InitializationTimeout in milliseconds),
 (DefaultAsynchronousTimeout in milliseconds),
 (DefaultSynchronousTimeout in milliseconds),
 (DestinationDenyACL),
 (DestinationAllowACL),
 (ProcessCount),
 (ThreadCount),
 (MaxR),
 (MaxT in seconds),
 (ServiceOptionsPropList)}

The ACL lists contain either atoms that reference the current ACL configuration or pattern strings. The ProcessCount and ThreadCount can be specified as integers for an exact count or as a floating point number to provide a CPU multiplier (X < 1.0 is round, X > 1.0 is floor). MaxR is the maximum restarts allowed within MaxT seconds (same parameters used by Erlang supervisors). The ServiceOptionsPropList provides the configurable defaults:

Timeout configuration values in milliseconds may be provided as 'limit_min' or 'limit_max' to use the extreme values (be careful, since 'limit_max' is the equivalent of 49.7 days and no one wants to wait 49.7 days to discover a failure). This is possible with the configuration values timeout_init, timeout_async, timeout_sync and the service configuration options values request_timeout_immediate_max, response_timeout_immediate_max, timeout_terminate, restart_delay, monkey_latency .

rate_request_max:

count_process_dynamic:

restart_delay:

monkey_latency:

monkey_chaos:

bind:

If a string value is provided all logical processors are assigned explicitly based on the string contents. The string may contain comma delimited non-negative integers with hyphens for integer ranges. All internal service processes and external service threads need exactly one logical processor specified. For example, the string value "1,13,3,15,5,17,7,19,9,21" could be used with an internal service that has 10 processes or an external service that has 10 total threads. To understand what logical processors are available, check the result of the Erlang function erlang:system_info(cpu_topology).

The bind service configuration option support depends on programming language support of bind functionality. That means bind use currently is supported by the ATS, C/C++, Erlang and Python/C CloudI APIs. Using the bind service configuration option should be preferred instead of using the cgroup service configuration option with "cpuset.cpus" because bind has cross-platform support with less CloudI execution user privileges required.

limit:

The value for any option can be an integer or 'infinity' to set the current limit or a list to set both the current and the maximum (setting the maximum requires that the user executing CloudI has permission to set the maximum limit) (e.g., {limit, [{stack, [{current, 8388608}, {maximum, infinity}]}]}).

(see your OS manpage for setrlimit to check availability)

owner:

Set the owner of an external service's OS processes.

cgroup:

Set the cgroup membership of an external service's OS processes.

syscall_lock:

Set the syscall names that are permitted in any OS processes an external service creates. All other syscalls will cause the OS process to exit with an uncatchable signal.

The syscalls used by a service executable can change due to:

• using dynamic linking
• using static linking
• an OS version change
• a kernel version change
• a libc version change

Examples of syscall_lock names used for only C/C++ CloudI API use (http_req test) with comments is below:

% Example 1: OpenBSD 6.8
{syscall_lock,
 [{type, pledge},
  {names,
   [
    % C/C++ CloudI API use
    "stdio",
    % dynamically linked library loading
    "rpath"
   ]}]}
% Example 2: Ubuntu 20.04 Linux (5.4.0 kernel, glibc 2.31)
{syscall_lock,
 [{type, function},
  {names,
   [
    % C/C++ CloudI API use
    "execve","clock_gettime","poll","read","write","close","exit",
    "brk","mmap","munmap", % (malloc/free)
    % Linux-specific
    "exit_group",
    % dynamically linked library loading
    "arch_prctl","mprotect","access","openat","stat","fstat","pread64"
    % static linking requires
    %"arch_prctl","mprotect","uname","readlink"
   ]}]}

Adding the syscall_lock configuration requires knowing all the syscalls a service may call. To log syscall information from a running service, it is possible to run the service with ktrace or strace:

OpenBSD: /usr/bin/ktrace -t c -f /tmp/ktrace.out SERVICE
Linux:   /usr/bin/strace -o /tmp/strace.log SERVICE

On Linux, the CloudI service configuration would have the file_path as "/usr/bin/strace" and the args as "-o /tmp/strace.log SERVICE" with SERVICE as the path to the external service executable.

aspects_init_after:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}". Will not be called if initialization does not occur successfully (aspects_terminate_before will still be called with the State as 'undefined').

fun((Args :: list(),
     Prefix :: cloudi_service:service_name_pattern(),
     Timeout :: cloudi_service_api:timeout_milliseconds(),
     State :: any(),
     Dispatcher :: cloudi_service:dispatcher()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

% the first function call has State =:= undefined
fun((CommandLine :: list(string()),
     Prefix :: cloudi:service_name_pattern(),
     Timeout :: cloudi_service_api:timeout_milliseconds(),
     State :: any()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_request_before:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}".

fun((RequestType :: cloudi_service:request_type(),
     Name :: cloudi_service:service_name(),
     Pattern :: cloudi_service:service_name_pattern(),
     RequestInfo :: cloudi_service:request_info(),
     Request :: cloudi_service:request(),
     Timeout :: cloudi_service:timeout_value_milliseconds(),
     Priority :: cloudi_service:priority(),
     TransId :: cloudi_service:trans_id(),
     Source :: cloudi_service:source(),
     State :: any(),
     Dispatcher :: cloudi_service:dispatcher()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

fun((RequestType :: cloudi_service:request_type(),
     Name :: cloudi_service:service_name(),
     Pattern :: cloudi_service:service_name_pattern(),
     RequestInfo :: cloudi_service:request_info(),
     Request :: cloudi_service:request(),
     Timeout :: cloudi_service:timeout_value_milliseconds(),
     Priority :: cloudi_service:priority(),
     TransId :: cloudi_service:trans_id(),
     Source :: cloudi_service:source(),
     State :: any()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_request_after:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}".

fun((RequestType :: cloudi_service:request_type(),
     Name :: cloudi_service:service_name(),
     Pattern :: cloudi_service:service_name_pattern(),
     RequestInfo :: cloudi_service:request_info(),
     Request :: cloudi_service:request(),
     Timeout :: cloudi_service:timeout_value_milliseconds(),
     Priority :: cloudi_service:priority(),
     TransId :: cloudi_service:trans_id(),
     Source :: cloudi_service:source(),
     Result :: cloudi_service:request_result(),
     State :: any(),
     Dispatcher :: cloudi_service:dispatcher()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

fun((RequestType :: cloudi_service:request_type(),
     Name :: cloudi_service:service_name(),
     Pattern :: cloudi_service:service_name_pattern(),
     RequestInfo :: cloudi_service:request_info(),
     Request :: cloudi_service:request(),
     Timeout :: cloudi_service:timeout_value_milliseconds(),
     Priority :: cloudi_service:priority(),
     TransId :: cloudi_service:trans_id(),
     Source :: cloudi_service:source(),
     Result :: cloudi_service:request_result(),
     State :: any()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_info_before:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}".

fun((Request :: any(),
     State :: any(),
     Dispatcher :: cloudi_service:dispatcher()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_info_after:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}".

fun((Request :: any(),
     State :: any(),
     Dispatcher :: cloudi_service:dispatcher()) ->
    {ok, NewState :: any()} |
    {stop, Reason :: any(), NewState :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_terminate_before:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}". Will still be called if initialization does not occur successfully (with State as 'undefined').

fun((Reason :: any(),
     Timeout :: cloudi_service_api:timeout_milliseconds(),
     State :: any()) ->
    {ok, State :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_suspend and aspects_resume:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}". Each function will be called immediately before the service becomes suspended or resumed.

fun((State :: any()) ->
    {ok, State :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

hibernate:

Please see the configuration file /usr/local/etc/cloudi/cloudi.conf for more specific examples.

2.6 - services_remove

curl -X POST -d '["6e81f0a6-7a1f-11e2-d40e-a5dd00000058", "6e81f0ec-7a1f-11e2-d40e-a5dd00000058"]' http://localhost:6464/cloudi/api/rpc/services_remove.erl

Provide the Service UUIDs for the services that should be stopped. The Service UUID is shown in the output of services. When the service is stopped, its running instance is removed from CloudI, but does not impact any other running instances (even if they are the same service module or binary).

When an internal service is removed and it is the last instance of the service module, the service module is purged to avoid later module conflicts. All instances of the internal service module should be configured in the same way (either a single module, an application, or a release with an application), so that the last instance is removed completely. If an application was used that is named the same as the service module, the application and its dependencies are removed (applications are stopped, modules are purged, and applications are unloaded) if the dependencies are not utilized by other applications. The same occurs if a release was used to start an application that contains the service module (the single top-level application of the release is used to determine dependencies, where the single top-level application within the release is the application that includes the service module).

2.7 - services_restart

curl -X POST -d '["6a675470-7a1f-11e2-d40e-a5dd00000058"]' http://localhost:6464/cloudi/api/rpc/services_restart.erl

Restart the services with the UUIDs provided. The service UUID is shown in the output of services. When the service is restarted, the old instance is stopped and a new instance is started. During the restart delay, it is possible to lose queued service requests and received asynchronous responses. Keeping the state separate between the service instances is important to prevent failures within the new instance.

2.8 - services_suspend

curl -X POST -d '["6a675470-7a1f-11e2-d40e-a5dd00000058"]' http://localhost:6464/cloudi/api/rpc/services_suspend.erl

Suspend the services with the UUIDs provided. Suspended services will not process more service requests (internal services will also not process info messages) and will only keep the data queued for processing in the future (allowing service requests to expire based on their timeout values). Suspended services can be updated while they are suspended. If a service is already suspended, services_suspend will have no effect.

2.9 - services_resume

curl -X POST -d '["6a675470-7a1f-11e2-d40e-a5dd00000058"]' http://localhost:6464/cloudi/api/rpc/services_resume.erl

Resume the services with the UUIDs provided. Resumed services will continue processing service requests (internal services will also process info messages). If a service is not suspended, services_resume will have no effect.

2.10 - services_search

curl -X POST -d '"/tests/http/text/post"' http://localhost:6464/cloudi/api/rpc/services_search.erl

curl -X POST -d '{cloudi_service_test_msg_size, "/tests/msg_size/erlang"}' http://localhost:6464/cloudi/api/rpc/services_search.erl

List the service configuration parameters with each service's UUID that are receiving service requests for a given service name. To search within a custom scope, provide both the scope and the service name within a tuple.

2.11 - services_status

curl -X POST -d '[]' http://localhost:6464/cloudi/api/rpc/services_status.erl

curl -X POST -d '["65762d3262a511e882359563d34a433e"]' http://localhost:6464/cloudi/api/rpc/services_status.erl

For each service UUID, provide the current uptime, downtime, interrupt and availability estimates at a single point in time. Each service has a {UUID, Status} pair with the Status described below:

[{type, internal | external},
 {prefix, service_name_pattern()},
 {module, atom()},                             % internal service only
 {file_path, file:filename()},                 % external service only
 {count_process, pos_integer()},               % count_process_dynamic may vary
 {count_thread, pos_integer()},                % external service only
 {pids_os, list(pos_integer())},               % external service only
 {pids_erlang, list(pid())},
 {size_erlang, pos_integer()},
 {suspended, boolean()},
 {uptime_total, nonempty_string()},
 {uptime_running, nonempty_string()},
 {uptime_processing, nonempty_string()},
 {uptime_restarts, nonempty_string()},
 {downtime_day_restarting, nonempty_string()},
 {downtime_week_restarting, nonempty_string()},
 {downtime_month_restarting, nonempty_string()},
 {downtime_year_restarting, nonempty_string()},
 {outages_day_restarting, nonempty_string()},
 {outages_week_restarting, nonempty_string()},
 {outages_month_restarting, nonempty_string()},
 {outages_year_restarting, nonempty_string()},
 {interrupt_day_updating, nonempty_string()},
 {interrupt_week_updating, nonempty_string()},
 {interrupt_month_updating, nonempty_string()},
 {interrupt_year_updating, nonempty_string()},
 {interrupt_day_suspended, nonempty_string()},
 {interrupt_week_suspended, nonempty_string()},
 {interrupt_month_suspended, nonempty_string()},
 {interrupt_year_suspended, nonempty_string()},
 {availability_day_total, nonempty_string()},
 {availability_day_running, nonempty_string()},
 {availability_day_updated, nonempty_string()},
 {availability_day_processing, nonempty_string()},
 {availability_week_total, nonempty_string()},
 {availability_week_running, nonempty_string()},
 {availability_week_updated, nonempty_string()},
 {availability_week_processing, nonempty_string()},
 {availability_month_total, nonempty_string()},
 {availability_month_running, nonempty_string()},
 {availability_month_updated, nonempty_string()},
 {availability_month_processing, nonempty_string()},
 {availability_year_total, nonempty_string()},
 {availability_year_running, nonempty_string()},
 {availability_year_updated, nonempty_string()},
 {availability_year_processing, nonempty_string()}]

The amount of time elapsed is provided in the string format "0 days 0 hours 0 seconds 0 nanoseconds" (with the largest 0 durations omitted and a singular unit if 1) for the uptime, downtime and interrupt values in the status. The total time shows how long the service has been running, including all restarts (in uptime_total). The running time shows how long the last restart of the service has been running (in uptime_running). The processing time shows how long the service has been processing service requests in the currently running service processes. The total number of restarts during the lifetime of the service is provided in uptime_restarts.

The service downtime spent restarting service processes is provided for the past day, week, month and year. The downtime elapsed due to restarting is measured as the time period from immediately before termination starts until the end of the initialization within the new service instance. The downtime values are used to determine the total availability values (e.g., availability_day_total, availability_week_total, availability_month_total and availability_year_total).

The service outage due to restarting is shown visually as a count in the time period for each "outages" string value. The outages_day_restarting value shows a single string character for each 30 minute segment after 00:00 UTC. Each restart causes an integer count to be updated in the string character associated with the restart time period. If a restart time period crosses a string character boundary all associated string characters will have an integer count updated. If the count is above 9 for a single character it is represented as a X character. The | (pipe) character is a cursor that shows the position of the current time in the string. Any characters after the cursor are representing the previous time period. The outages_week_restarting value shows a single string character for each 4 hour segment after 00:00 UTC on Monday of the current week. The outages_month_restarting value shows a single string character for each day of the current month. The outages_year_restarting value shows a single string character for each 1/3rd of a month during the current year.

The service interrupt spent updating service processes is provided for the past day, week, month and year. The interrupt elapsed will always overestimate the amount of time spent updating, by including extra coordination delay and should be considered the total time spent updating all service processes. Only the time spent updating the most recent service processes gets tracked as the interrupt (i.e., a fraction of the running time period which is after any restarts occurred). The interrupt values are used to determine the updated availability values (e.g., availability_day_updated, availability_week_updated, availability_month_updated and availability_year_updated).

The service interrupt spent with service processes suspended is provided for the past day, week, month and year. The processing time is the running time minus the updating interruptions and the suspended interruptions.

The amount of time the service has been running since any restarts occurred is used to determine the running availability values (e.g., availability_day_running, availability_week_running, availability_month_running and availability_year_running). All the availability values are percentages to describe the fraction of estimated uptime during each time period. All status data is determined using the same point in time, so the values may be compared among separate services.

The currently executing Erlang processes for each service are provided in pids_erlang. These Erlang processes that represent the service are the source pids provided when handling a CloudI service request. The current external service OS pids are provided in pids_os. Both lists are ordered by process index and may be smaller than expected (based on count_process and count_thread) if the service is not initialized.

The size_erlang value provides the size of the service within the Erlang VM as the size of all the pids_erlang Erlang processes in bytes. The size_erlang value represents the memory used for the most critical parts of a CloudI service in the Erlang VM related to receiving CloudI service requests and responses (i.e., the source pids). For internal services the size_erlang value will not include the size of the info_pid or the request_pid if the duo_mode service configuration option is false (the default value). If an internal service has the duo_mode service configuration option set to true, the size_erlang value will not include the size of the dispatcher_pid or the request_pid. That means the size_erlang value always represents the long-lived CloudI service memory in the Erlang VM.

Example output is available at
https://cloudi.org/config.html#services_status.

2.12 - services_update

curl -X POST -d '[{"", [{module, cloudi_service_test_messaging}, {modules_load, [cloudi_service_test_messaging]}, {sync, false}]}]' http://localhost:6464/cloudi/api/rpc/services_update.erl

Update services while they are running without any interruption in their operation. Each update that should occur is specified as a {UUID, Options} pair with the Options described below:

% internal service update options
[{type, internal},
 {module, atom()},                            % only required field
 {module_state, module_state_internal()},     % update service state
 {sync, boolean()},                           % defaults to true
 {modules_load, list(atom())},                % defaults to []
 {modules_unload, list(atom())},              % defaults to []
 {code_paths_add, list(string())},            % defaults to []
 {code_paths_remove, list(string())},         % defaults to []
 {dest_refresh, dest_refresh()},              % update service configuration
 {timeout_init, timeout_milliseconds()},      % update service configuration
 {timeout_async, timeout_milliseconds()},     % update service configuration
 {timeout_sync, timeout_milliseconds()},      % update service configuration
 {dest_list_deny, dest_list()},               % update service configuration
 {dest_list_allow, dest_list()},              % update service configuration
 {options, service_update_plan_options_internal()}] % update service configuration options

Internal service updates require a module field provide the service module name. If the module is used in more than one service instance, provide the service UUID as "" to ensure the update applies to all service instances that use the module. If the module needs to be reloaded, it must be added to the modules_load list and the new module will be loaded using Erlang hot-code loading during the update. If the service state needs to be modified during the update, a function can be provided in the module_state field.

The module_state value for an internal service update can be provided as an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}" with the types below:

fun((OldModuleVersion :: list(any()),
     NewModuleVersion :: list(any()),
     OldState :: any()) ->
    {ok, NewState :: any()} |
    {error, Reason :: any()}).

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above. Providing the module version before and after the module update can help guide any upgrade or downgrade logic to return the appropriate state data. If the module_state function returns an error (or any other problem occurs during the update), the update does not change any service state, though the module and code_path modifications take effect after the update has been processed (i.e., modules_load, modules_unload, code_paths_add, code_paths_remove, all occur even when an update fails).

External service updates require either the type be set to external or that the file_path, args, or env field be set (to cause the OS process of the external service to restart during the update).

% external service update options
[{type, external},
 {file_path, string()},                       % restarts OS process
 {args, string()},                            % restarts OS process
 {env, list({string(), string()})},           % restarts OS process
 {sync, boolean()},                           % defaults to true
 {modules_load, list(atom())},                % defaults to []
 {modules_unload, list(atom())},              % defaults to []
 {code_paths_add, list(string())},            % defaults to []
 {code_paths_remove, list(string())},         % defaults to []
 {dest_refresh, dest_refresh()},              % update service configuration
 {timeout_init, timeout_milliseconds()},      % update service configuration
 {timeout_async, timeout_milliseconds()},     % update service configuration
 {timeout_sync, timeout_milliseconds()},      % update service configuration
 {dest_list_deny, dest_list()},               % update service configuration
 {dest_list_allow, dest_list()},              % update service configuration
 {options, service_update_plan_options_external()}] % update service configuration options

The sync field is used to determine whether the update occurs while no service processes are processing a service request. If global state is being used while handling a service request, it is safest to have sync set to true, which is the default. If a service sends synchronous service requests to itself, the update can timeout when sync is set to true but can succeed with sync set to false.

Do not set the sync field to false unless the service update requires it (due to the service having a cyclic dependency on itself, as described above). If an internal service update has the sync field set to false and the service's module is present in the modules_load list, the service's request process may use the new module prematurely if the request process is spawned slower than the module load (or if it calls any exported functions within the module) and that would occur before the module_state function gets called.

A subset of the service configuration options can be updated:

% internal service configuration options that are updatable
[{priority_default, priority()} |
 {queue_limit, undefined | non_neg_integer()} |
 {queue_size, undefined | pos_integer()} |
 {rate_request_max,
  list({period, period_seconds()} |
       {value, number()}) | number() | undefined} |
 {dest_refresh_start, dest_refresh_delay_milliseconds()} |
 {dest_refresh_delay, dest_refresh_delay_milliseconds()} |
 {request_name_lookup, sync | async} |
 {request_timeout_adjustment, boolean()} |
 {request_timeout_immediate_max,
  request_timeout_immediate_max_milliseconds()} |
 {response_timeout_adjustment, boolean()} |
 {response_timeout_immediate_max,
  response_timeout_immediate_max_milliseconds()} |
 {monkey_latency,
  list({time_uniform_min, latency_time_milliseconds()} |
       {time_uniform_max, latency_time_milliseconds()} |
       {time_gaussian_mean, latency_time_milliseconds()} |
       {time_gaussian_stddev, float()} |
       {time_absolute, latency_time_milliseconds()}) | system | false} |
 {monkey_chaos,
  list({probability_request, float()} |
       {probability_day, float()}) | system | false} |
 {dispatcher_pid_options,
  list({priority, low | normal | high} |
       {fullsweep_after, non_neg_integer()} |
       {min_heap_size, non_neg_integer()} |
       {min_bin_vheap_size, non_neg_integer()} |
       {max_heap_size, non_neg_integer() |
                       #{size => non_neg_integer(),
                         kill => boolean(),
                         error_logger => boolean()}} |
       {sensitive, boolean()} |
       {message_queue_data, off_heap | on_heap})} |
 {aspects_init_after, list(aspect_init_after_internal())} |
 {aspects_request_before, list(aspect_request_before_internal())} |
 {aspects_request_after, list(aspect_request_after_internal())} |
 {aspects_info_before, list(aspect_info_before_internal())} |
 {aspects_info_after, list(aspect_info_after_internal())} |
 {aspects_terminate_before, list(aspect_terminate_before_internal())} |
 {aspects_suspend, list(aspect_suspend())} |
 {aspects_resume, list(aspect_resume())} |
 {init_pid_options,
  list({priority, low | normal | high} |
       {fullsweep_after, non_neg_integer()} |
       {min_heap_size, non_neg_integer()} |
       {min_bin_vheap_size, non_neg_integer()} |
       {max_heap_size, non_neg_integer() |
                       #{size => non_neg_integer(),
                         kill => boolean(),
                         error_logger => boolean()}} |
       {sensitive, boolean()} |
       {message_queue_data, off_heap | on_heap})} |
 {request_pid_uses, infinity | pos_integer()} |
 {request_pid_options,
  list({priority, low | normal | high} |
       {fullsweep_after, non_neg_integer()} |
       {min_heap_size, non_neg_integer()} |
       {min_bin_vheap_size, non_neg_integer()} |
       {max_heap_size, non_neg_integer() |
                       #{size => non_neg_integer(),
                         kill => boolean(),
                         error_logger => boolean()}} |
       {sensitive, boolean()} |
       {message_queue_data, off_heap | on_heap})} |
 {info_pid_uses, infinity | pos_integer()} |
 {info_pid_options,
  list({priority, low | normal | high} |
       {fullsweep_after, non_neg_integer()} |
       {min_heap_size, non_neg_integer()} |
       {min_bin_vheap_size, non_neg_integer()} |
       {max_heap_size, non_neg_integer() |
                       #{size => non_neg_integer(),
                         kill => boolean(),
                         error_logger => boolean()}} |
       {sensitive, boolean()} |
       {message_queue_data, off_heap | on_heap})} |
 {hibernate,
  list({period, period_seconds()} |
       {rate_request_min, number()}) | boolean()} |
 {reload, boolean()}]

% external service configuration options that are updatable
[{priority_default, ?PRIORITY_HIGH..?PRIORITY_LOW} |
 {queue_limit, undefined | non_neg_integer()} |
 {queue_size, undefined | pos_integer()} |
 {rate_request_max,
  list({period, period_seconds()} |
       {value, number()}) | number() | undefined} |
 {dest_refresh_start, dest_refresh_delay_milliseconds()} |
 {dest_refresh_delay, dest_refresh_delay_milliseconds()} |
 {request_name_lookup, sync | async} |
 {request_timeout_adjustment, boolean()} |
 {request_timeout_immediate_max,
  request_timeout_immediate_max_milliseconds()} |
 {response_timeout_adjustment, boolean()} |
 {response_timeout_immediate_max,
  response_timeout_immediate_max_milliseconds()} |
 {monkey_latency,
  list({time_uniform_min, latency_time_milliseconds()} |
       {time_uniform_max, latency_time_milliseconds()} |
       {time_gaussian_mean, latency_time_milliseconds()} |
       {time_gaussian_stddev, float()} |
       {time_absolute, latency_time_milliseconds()}) | system | false} |
 {monkey_chaos,
  list({probability_request, float()} |
       {probability_day, float()}) | system | false} |
 {dispatcher_pid_options,
  list({priority, low | normal | high} |
       {fullsweep_after, non_neg_integer()} |
       {min_heap_size, non_neg_integer()} |
       {min_bin_vheap_size, non_neg_integer()} |
       {max_heap_size, non_neg_integer() |
                       #{size => non_neg_integer(),
                         kill => boolean(),
                         error_logger => boolean()}} |
       {sensitive, boolean()} |
       {message_queue_data, off_heap | on_heap})} |
 {aspects_init_after, list(aspect_init_after_external())} |
 {aspects_request_before, list(aspect_request_before_external())} |
 {aspects_request_after, list(aspect_request_after_external())} |
 {aspects_terminate_before, list(aspect_terminate_before_external())} |
 {aspects_suspend, list(aspect_suspend())} |
 {aspects_resume, list(aspect_resume())}]

For more information about a specific service configuration option refer to the service configuration documentation.

2.13 - services

curl http://localhost:6464/cloudi/api/rpc/services.erl

List the service configuration parameters with each service's UUID. Any default service configuration options are omitted to keep the output concise. The service tuple format is provided in the output after any defaults have been assigned, as described in services_add.

% service tuple format list
[{UUID,
  {internal,
   Prefix :: service_name_pattern(),
   Module :: atom(),
   Args :: list(),
   DestRefresh :: dest_refresh(),
   TimeoutInit :: timeout_initialize_milliseconds(),
   TimeoutAsync :: timeout_send_async_milliseconds(),
   TimeoutSync :: timeout_send_sync_milliseconds(),
   DestListDeny :: dest_list(),
   DestListAllow :: dest_list(),
   CountProcess :: pos_integer(),
   MaxR :: non_neg_integer(),
   MaxT :: seconds(),
   Options :: service_options_internal()} |
  {external,
   Prefix :: service_name_pattern(),
   FilePath :: file:filename(),
   Args :: string(),
   Env :: list({string(), string()}),
   DestRefresh :: dest_refresh(),
   Protocol :: local | tcp | udp,
   BufferSize :: pos_integer(),
   TimeoutInit :: timeout_initialize_milliseconds(),
   TimeoutAsync :: timeout_send_async_milliseconds(),
   TimeoutSync :: timeout_send_sync_milliseconds(),
   DestListDeny :: dest_list(),
   DestListAllow :: dest_list(),
   CountProcess :: pos_integer(),
   CountThread :: pos_integer(),
   MaxR :: non_neg_integer(),
   MaxT :: seconds(),
   Options :: service_options_external()}}]

2.14 - nodes_set

curl -X POST -d "[{reconnect_delay, 300}]" http://localhost:6464/cloudi/api/rpc/nodes_set.erl

Set the node configuration to specify how CloudI node connections are handled (using distributed Erlang).

discovery:

multicast:

LAN multicast auto-discovery requires that ntpd is running to keep the time of all LAN nodes as close as possible. However, all CloudI nodes should have ntpd running to keep all transaction ids similar (even if LAN multicast auto-discovery is not used).

ec2:

EC2 auto-discovery configuration requires setting 'access_key_id' and 'secret_access_key' with either 'groups' and/or 'tags'. Both 'groups' and 'tags' allow boolean expressions with nesting (specified as a list), e.g.:

[TAG | GROUP | OPERATOR]                 % implicitly an OR relationship
{'OR', [TAG | GROUP | OPERATOR]}         % OR boolean OPERATOR
{'AND', [TAG | GROUP | OPERATOR]}        % AND boolean OPERATOR

"security-group-name"                    % GROUP

"key1" | ["key3", "key4"]                % TAG (key names)

{"key2", "value2"} |                     % TAG (key/value combinations)
{["key5", "key6"], "value5"} |           %
{"key5", ["value5", "value6"]} |         %
{["key5", "key6"], ["value5", "value6"]} %

% groups example #1 (implicit OR relationship):
["security-group-a", "security-group-b"]

% groups example #2 (explicit AND relationship):
[{'AND', ["security-group-a", "security-group-b"]}]

% groups example #3 (explicit OR relationship, same functionally as #1):
[{'OR', ["security-group-a", "security-group-b"]}]

% tags example
[{'AND', [{"deployment", "development"}, {"cluster", "project42"}]}]

The EC2 auto-discovery requires that the security group(s) used by the instances expose distributed Erlang ports (with TCP rules):

• port 4369 (epmd) 10.0.0.0/8
• ports 4374-4474 (inet_dist_listen) 10.0.0.0/8

The EC2 auto-discovery needs distributed Erlang longnames usage so -name should be used instead of -sname within the vm.args configuration file. All EC2 nodes that will be discovered need to be using longnames to avoid distributed Erlang error log messages. EC2 auto-discovery requires that all Erlang nodes that want to be discovered use the same -name and the same -setcookie.

2.15 - nodes_get

curl http://localhost:6464/cloudi/api/rpc/nodes_get.erl

List the current nodes configuration in the same format provided to nodes_set with default settings ignored to keep the output concise.

2.16 - nodes_add

curl -X POST -d "['cloud001@cluster1']" http://localhost:6464/cloudi/api/rpc/nodes_add.erl

Explicitly add a CloudI node name, so that services between all other CloudI nodes and the added nodes can send each other service requests. A nodes_add call is similar to nodes_set with [{set, all}, {nodes, Nodes}] where Nodes is the final list of all nodes (all connected nodes are modified).

2.17 - nodes_remove

curl -X POST -d "['cloud001@cluster1']" http://localhost:6464/cloudi/api/rpc/nodes_remove.erl

Explicitly remove a CloudI node name. The CloudI node must have been added explicitly to be removed explicitly (not added by an auto-discovery method). A nodes_remove call is similar to nodes_set with [{set, all}, {nodes, Nodes}] where Nodes is the final list of all nodes (all connected nodes are modified).

2.18 - nodes_alive

curl http://localhost:6464/cloudi/api/rpc/nodes_alive.erl

List all the CloudI nodes known to be connected.

2.19 - nodes_dead

curl http://localhost:6464/cloudi/api/rpc/nodes_dead.erl

List all the CloudI nodes that are disconnected but expected to reconnect.

2.20 - nodes_status

curl -X POST -d '[]' http://localhost:6464/cloudi/api/rpc/nodes_status.erl

curl -X POST -d "['cloudi@hostname']" http://localhost:6464/cloudi/api/rpc/nodes_status.erl

Provide information about all the CloudI nodes (including the local node). Each node has a {Node, Status} pair with the Status described below:

[{services_running, nonempty_string()},
 {services_restarted, nonempty_string()},
 {services_failed, nonempty_string()},
 {uptime, nonempty_string()},
 {uptime_cost_total, nonempty_string()},
 {uptime_cost_day, nonempty_string()},
 {uptime_cost_week, nonempty_string()},
 {uptime_cost_month, nonempty_string()},
 {uptime_cost_year, nonempty_string()},
 {tracked, nonempty_string()},
 {tracked_cost_total, nonempty_string()},
 {tracked_cost_day, nonempty_string()},
 {tracked_cost_week, nonempty_string()},
 {tracked_cost_month, nonempty_string()},
 {tracked_cost_year, nonempty_string()},
 {tracked_disconnects, nonempty_string()},
 {disconnected, boolean()},
 {downtime_day_disconnected, nonempty_string()},
 {downtime_week_disconnected, nonempty_string()},
 {downtime_month_disconnected, nonempty_string()},
 {downtime_year_disconnected, nonempty_string()},
 {outages_day_disconnected, nonempty_string()},
 {outages_week_disconnected, nonempty_string()},
 {outages_month_disconnected, nonempty_string()},
 {outages_year_disconnected, nonempty_string()},
 {availability_day, nonempty_string()},
 {availability_week, nonempty_string()},
 {availability_month, nonempty_string()},
 {availability_year, nonempty_string()}]

The local node will provide uptime results while remote nodes will provide tracked results and information related to being disconnected. If cost was provided with nodes_set the cumulative cost of the node will also be present. All the status information is provided from information stored on the local node.

The connection outage due to a disconnected remote node is shown visually as a count in the time period for each "outages" string value. The outages_day_disconnected value shows a single string character for each 30 minute segment after 00:00 UTC. Each disconnect causes an integer count to be updated in the string character associated with the disconnect time period. If a disconnect time period crosses a string character boundary all associated string characters will have an integer count updated. If the count is above 9 for a single character it is represented as a X character. The | (pipe) character is a cursor that shows the position of the current time in the string. Any characters after the cursor are representing the previous time period. The outages_week_disconnected value shows a single string character for each 4 hour segment after 00:00 UTC on Monday of the current week. The outages_month_disconnected value shows a single string character for each day of the current month. The outages_year_disconnected value shows a single string character for each 1/3rd of a month during the current year.

Example output is available at
https://cloudi.org/config.html#nodes_status.

2.21 - nodes_status_reset (new in 2.0.8)

curl http://localhost:6464/cloudi/api/rpc/nodes_status_reset.erl

Remove all nodes status data for nodes that are currently dead. If a currently disconnected node (i.e., "dead") reconnects in the future, the remote node's status on the local node will make it appear to be a new remote node. If nodes that are currently dead were added manually (i.e., with 'nodes' configuration instead of 'discovery' configuration; without the use of Distributed Erlang node auto-discovery functionality), no reconnect (initiated from the local node) with the dead nodes will occur in the future after the nodes status reset occurs (unless the nodes are added manually again).

2.22 - nodes

curl http://localhost:6464/cloudi/api/rpc/nodes.erl

List both the connected and disconnected CloudI nodes.

2.23 - logging_set

curl -X POST -d '[{file, undefined}, {syslog, []}]' http://localhost:6464/cloudi/api/rpc/logging_set.erl

Set the logging configuration for a CloudI node.

% logging configuration options
[{file, undefined | string()},                % defaults to "cloudi.log"
 {file_sync,
  logging_file_sync_milliseconds()},          % defaults to 0 milliseconds
 {level, off | fatal | error |
         warn | info | debug | trace},        % defaults to trace
 {redirect, undefined | node()},              % defaults to undefined
 {syslog, undefined | list()},                % defaults to undefined
 {stdout, boolean()},                         % defaults to false
 {queue_mode_async, pos_integer()},           % defaults to 750
 {queue_mode_sync, pos_integer()},            % defaults to 1000
 {queue_mode_overload, pos_integer()},        % defaults to 10000
 {formatters, undefined | list()},            % defaults to undefined
 {log_time_offset,
  off | fatal | error |
  warn | info | debug | trace},               % defaults to off
 {aspects_log_before,
  list(aspects_log_before())},                % defaults to []
 {aspects_log_after,
  list(aspects_log_after())}]                 % defaults to []

file_sync:

Ensure all log file data has been written to the file by flushing any operating system buffers that contain pending write data, with the interval provided in milliseconds.

queue_mode_async:

The logger process will use asynchronous mode while its message queue length is less than the configured value.

queue_mode_sync:

The logger process will use synchronous mode when its message queue length is greater than the configured value. The gap between queue_mode_async and queue_mode_sync values avoids latency associated with changing the queue_mode. The sync queue_mode will cause all services to log more slowly while the logger process writes its pending log requests (messages) as quickly as possible.

queue_mode_overload:

The logger process will use overload mode when its message queue length is greater than the configured value. The value prevents CloudI from terminating due to extreme memory consumption.

log_time_offset:

Log when the Erlang VM adjusts its internal view of the system time with the size of the change in nanoseconds using the provided loglevel.

aspects_log_before:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}". Each function will be called before the log data is stored to disk.

fun((Level :: fatal | error | warn | info | debug | trace,
     Timestamp :: erlang:timestamp(),
     Node :: node(),
     Pid :: pid(),
     Module :: module(),
     Line :: pos_integer(),
     Function :: atom() | undefined,
     Arity :: arity() | undefined,
     MetaData :: list({atom(), any()}),
     LogMessage :: iodata()) ->
    ok.

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

aspects_log_after:

Provide a list of functions with the type specification shown below, with each function as either an anonymous Erlang function or an Erlang tuple "{module(), FunctionName :: atom()}". Each function will be called after the log data is stored to disk.

fun((Level :: fatal | error | warn | info | debug | trace,
     Timestamp :: erlang:timestamp(),
     Node :: node(),
     Pid :: pid(),
     Module :: module(),
     Line :: pos_integer(),
     Function :: atom() | undefined,
     Arity :: arity() | undefined,
     MetaData :: list({atom(), any()}),
     LogMessage :: iodata()) ->
    ok.

It is also possible to use the Erlang tuple "{{module(), FunctionName :: atom()}}" to specify an arity 0 function that returns an anonymous function as described above.

2.24 - logging_file_set

curl -X POST -d '"different_filename.log"' http://localhost:6464/cloudi/api/rpc/logging_file_set.erl

Set the file path for logging output. If set to 'undefined', logging output will only be sent to syslog and formatters with an output module.

2.25 - logging_level_set

curl -X POST -d 'warn' http://localhost:6464/cloudi/api/rpc/logging_level_set.erl

Modify the loglevel. The loglevel is changed with an Erlang module update internally so any logging statements that are turned off create no latency. If set to 'undefined' or 'off', logging output will only be sent to syslog and formatters with an output module. The available log levels values are: off, fatal, error, warn, info, debug, trace.

2.26 - logging_stdout_set

curl -X POST -d 'true' http://localhost:6464/cloudi/api/rpc/logging_stdout_set.erl

Send all logging output to stdout.

2.27 - logging_syslog_set

curl -X POST -d '[{identity, "CloudI"}, {facility, local0}, {level, trace}]' http://localhost:6464/cloudi/api/rpc/logging_syslog_set.erl

Send all logging output to syslog.

2.28 - logging_formatters_set

curl -X POST -d '[{any, [{output, lager_file_backend}, {output_args, [{file, "lager.log"}]}, {formatter, lager_default_formatter}, {level, trace}]}]' http://localhost:6464/cloudi/api/rpc/logging_formatters_set.erl

Provide integration with lager-compatible formatters and lager-compatible backends. Each formatter entry specifies a list of modules for the source of the logging output to match against paired with formatter options (e.g., {[module1, module2], Options}). A separate formatter entry with an 'any' atom instead of a list of modules is used if the source of the logging output is not provided (e.g., {any, Options}). Use 'STDOUT' and 'STDERR' as a module name entry to control the stdout and stderr output coming from external services (logging output for both streams provides the OS pid next to the stream name, instead of a module line number). If only a formatter is specified (i.e., without an output module, a lager-compatible backend), the formatter transforms the logging output to be logged to the CloudI log file and/or syslog. If an output module is provided (that implements the gen_event Erlang/OTP behaviour), it will consume the logging output separately from the CloudI log file and syslog.

2.29 - logging_redirect_set

curl -X POST -d 'cloudi@host' http://localhost:6464/cloudi/api/rpc/logging_redirect_set.erl

Redirect all local log output to a remote CloudI node. Use 'undefined' as the node name to log locally.

2.30 - logging_status

curl http://localhost:6464/cloudi/api/rpc/logging_status.erl

Provide the current logging status. If any errors occur when writing to the log file, the error information is provided in the logging status output.

[{queue_mode, async | sync | overload},
 {queue_mode_sync_last_start, nonempty_string()},
 {queue_mode_sync_last_start_event, nonempty_string()},
 {queue_mode_sync_last_end, nonempty_string()},
 {queue_mode_sync_last_end_event, nonempty_string()},
 {queue_mode_sync_last_total, nonempty_string()},
 {queue_mode_overload_last_start, nonempty_string()},
 {queue_mode_overload_last_start_event, nonempty_string()},
 {queue_mode_overload_last_end, nonempty_string()},
 {queue_mode_overload_last_end_event, nonempty_string()},
 {queue_mode_overload_last_total, nonempty_string()},
 {time_offset_last_change, nonempty_string()},
 {time_offset_last_event, nonempty_string()},
 {file_messages_fatal, nonempty_string()},
 {file_messages_error, nonempty_string()},
 {file_messages_warn, nonempty_string()},
 {file_messages_info, nonempty_string()},
 {file_messages_debug, nonempty_string()},
 {file_messages_trace, nonempty_string()},
 {file_sync_fail_count, nonempty_string()},
 {file_sync_fail_types, nonempty_list(atom())},
 {file_write_fail_count, nonempty_string()},
 {file_write_fail_types, nonempty_list(atom())},
 {file_read_fail_count, nonempty_string()},
 {file_read_fail_types, nonempty_list(atom())}]

The queue_mode is async while logging is occurring asynchronously (the default). If the logger receives a large number of logging requests, it will change to sync (synchronous) mode which forces Erlang service processes to wait until their logging request has finished. The logger may still receive too many logging requests if it is the destination of a redirect and that could cause the logger to use overload mode. In overload mode, the logger will discard logging requests as quickly as possible (i.e., without any write to the log occurring) to reduce the extreme memory consumption until it is possible to go back to sync mode (each mode has a limit set by logging_set).

When the overload mode transitions to sync mode an error is logged with the duration of the overload mode. The most recent durations of both sync mode and overload mode are provided in the logging status output (if either modes were used during CloudI's runtime). The event timestamps are the same timestamps present in the log output while the start/end timestamps may change based on changes to the OS time. The logging status output will also provide the last OS time change in seconds (if an OS time change occurred during CloudI's runtime).

2.31 - logging_status_reset

curl http://localhost:6464/cloudi/api/rpc/logging_status_reset.erl

Reset the logging status.

2.32 - logging

curl http://localhost:6464/cloudi/api/rpc/logging.erl

List the current logging configuration in the same format provided to the configuration file with default settings ignored to keep the output concise.

2.33 - code_path_add

curl -X POST -d '"/home/user/code/services"' http://localhost:6464/cloudi/api/rpc/code_path_add.erl

Add a directory to the CloudI Erlang VM code server's search paths. The path is always appended to the list of search paths (you should not need to rely on search path order because of unique naming).

2.34 - code_path_remove

curl -X POST -d '"/home/user/code/services"' http://localhost:6464/cloudi/api/rpc/code_path_remove.erl

Remove a directory from the CloudI Erlang VM code server's search paths. This doesn't impact any running services, only services that will be started in the future.

2.35 - code_path

curl http://localhost:6464/cloudi/api/rpc/code_path.erl

List all the CloudI Erlang VM code server search paths (in the same order the directories are searched).

2.36 - code_status

curl http://localhost:6464/cloudi/api/rpc/code_status.erl

Provide information about the execution environment:

[{build_machine, nonempty_string()},
 {build_kernel_version, nonempty_string()},
 {build_operating_system, nonempty_string()},
 {build_erlang_otp_release, nonempty_string()},
 {build_cloudi_time, nonempty_string()},
 {build_cloudi_version, nonempty_string()},
 {build_cloudi_cxx_compiler_version, nonempty_string()},
 {build_cloudi_cxx_dependencies_versions, nonempty_string()},
 {build_erlang_erts_c_compiler_version, nonempty_string()},
 {install_erlang_erts_time, nonempty_string()},        % ISO8601 timestamp
 {install_erlang_kernel_time, nonempty_string()},      % ISO8601 timestamp
 {install_erlang_stdlib_time, nonempty_string()},      % ISO8601 timestamp
 {install_erlang_sasl_time, nonempty_string()},        % ISO8601 timestamp
 {install_erlang_compiler_time, nonempty_string()},    % ISO8601 timestamp
 {install_cloudi_time, nonempty_string()},             % ISO8601 timestamp
 {runtime_erlang_erts_version, nonempty_string()},
 {runtime_erlang_kernel_version, nonempty_string()},
 {runtime_erlang_stdlib_version, nonempty_string()},
 {runtime_erlang_sasl_version, nonempty_string()},
 {runtime_erlang_compiler_version, nonempty_string()},
 {runtime_erlang_compilation, nonempty_string()},      % "aot" | "jit"
 {runtime_cloudi_version, nonempty_string()},
 {runtime_machine_processors, pos_integer()},          % logical processors
 {runtime_start, nonempty_string()},                   % ISO8601 timestamp
 {runtime_clock, nonempty_string()},                   % ISO8601 timestamp
 {runtime_clock_offset, nonempty_string()},            % offset to HW clock
 {runtime_total, nonempty_string()},
 {runtime_cloudi_start, nonempty_string()},            % ISO8601 timestamp
 {runtime_cloudi_total, nonempty_string()},
 {runtime_cloudi_changes,                              % service file changes
  list([{type, internal | external},
        {file_age, nonempty_string()},
        {file_path, nonempty_string()},
        {file_loaded, boolean()},               % internal only (new in 2.0.8)
        {file_version, nonempty_list(byte())},  % internal only (new in 2.0.8)
        {service_ids, nonempty_list(service_id())}])}]

The runtime_cloudi_changes list shows CloudI service files that have changed on the filesystem after CloudI started. The files may have changed due to services_update use.

Example output is available at
https://cloudi.org/config.html#code_status.