CloudI User Tutorials

Service creation requires splitting functionality based on capabilities so that functionality and its associated development risk is isolated. However, the latency associated with the functionality should also be isolated within the programming language best suited to the task. Part of the decision to isolate service latency is minimizing the amount of data that must be exchanged in service requests.

A common reason for creating a service is to isolate the risk associated with an external component that is sufficiently complex and is likely to require a variable amount of latency which needs to be managed as a unique entity to provide dependable processing (e.g., a large source code library, a database, a messaging bus, a filesystem, a third-party API, etc.). It is also common to create separate services due to source code libraries existing in a separate programming language.

The simplest approach to service development, when creating a new service, is to develop the business logic that is necessary using as many services as required based on design requirements, basic testing, and benchmarking (if required), without isolating the use of external components. Then, as risks are identified during development, functionality can be moved into separate services to manage source code that needs fault tolerance guarantees. With this approach, the business logic will always have fault tolerance guarantees to keep all source code changes (which may include changing the external components used) isolated from other development during the lifetime of the business logic source code service(s).

The main data throughput to the business logic should utilize CloudI service requests for reliable timeouts and service redundancy. Often the main data throughput is HTTP protocol usage coming from any of the provided HTTP servers (both cloudi_service_http_cowboy and cloudi_service_http_elli are Erlang CloudI services for HTTP servers). The incoming HTTP requests are automatically load-balanced among the available services, based on the incoming URL path matching a service name pattern for a pool of service processes.

CloudI services provide process pooling automatically when their configuration has a count_process or count_thread (of an external service) greater than 1. A CloudI service’s process pooling can be adjusted dynamically based on the incoming service request rate by using the count_process_dynamic service configuration option. When a service request is sent it will automatically select a service execution process (which represents an external service thread within an OS process or an internal service Erlang process within the Erlang VM) randomly from those available, due to subscribing with the same service name pattern.

Relying on this process pooling keeps services dependable and helps to reduce their potential complexity, since a service developer only needs to focus on developing serial source code. The main exception to the pursuit of serial source code is due to the use of global state, which may require locking for consistency. Ideally, the use of global state can be avoided in the service source code because global state usage naturally increases latency unless low-level atomic operations are used to avoid locking.

The decision of what programming language to use is generally based on the knowledge of the developer. The libraries that already exist and are known to be dependable will often determine which programming language to use.

If the system (the combination of services during development) will be sending service requests from many separate services or will be receiving service requests within many separate services, usage of separate programming languages for each separate service can increase the computational requirements for the system, or at least will be a service configuration task to determine based on the computing resources available. External CloudI service instances (any CloudI service written in a programming language that doesn’t execute on the Erlang VM) can create more than one OS process, but must create at least one OS process, so that service memory is isolated and the service processing is fault-tolerant. If the service request messaging is able to keep many separate external CloudI services busy the processing will be at the mercy of the OS kernel scheduler, which may make it easy to exhaust the computational resources available.

Controlling the potential service request latency with the selection of the programming languages can avoid prematurely exhausting computational resources. A good approach is using as few programming languages that can reasonably be used for the business logic design. Based on CloudI loadtesting (ordered based on average latency during the loadtest), service requests are handled with low latency in Erlang/Elixir, C/C++, OCaml, Java, Python/C, Haskell, Go (less than 6 milliseconds) and higher latency in Python, Ruby, Perl, PHP, Javascript/node.js (greater than 2000 milliseconds, when under high load) due to inefficiencies within the programming language runtimes. By considering the latency requirements of the system early during development it will be easier to scale the deployment and avoid wasted development effort.

If a programming language that runs on the Erlang VM is used, it is possible to develop with finer-grained fault tolerance in an internal CloudI service due to the usage of Erlang processes (similar to user-level threads with isolated memory) by CloudI. External CloudI services require that the memory used for execution of the service is isolated within an OS process and it is possible to hide an extreme amount of risk to reliability within a single external service instance, making this approach coarser-grained fault tolerance.

CloudI’s service request and response data (i.e., request_info, request, response_info, response) is protocol agnostic, so any data format can be used for CloudI service communication. External services can only receive data that is sent as a binary type in the programming language used for the service request send. However, internal services can receive data of any type, though it is common to use binary types to allow the processing of service requests sent from external services.

If you require a "universal protocol" (i.e., a protocol that can encode types from any programming language and handle them transparently in other programming languages), you may be disappointed to find out that no complete solution exists. If you approach the concept of a "universal protocol" as only handling the most minimal and common types available in programming languages, msgpack is a popular choice. The data format should be based on the requirements and it may require a human readable format, with JSON as a popular choice.

There is nothing that prevents the usage of multiple data formats by a single CloudI service and typically a file extension suffix is used in the service name pattern to distinguish between different data formats.