How to better manage resources in OpenStack and Kubernetes Clouds with Sardina FishOS
An optimized way to Resource Management system in OpenStack and Kubernetes environment
The advanced resource management system built into FishOS provides smart, efficient management of resources within OpenStack and Kubernetes. This is achieved with a combination of 2 decision engines — FishOS Placement Engine and Rebalancing Engine that work in conjunction to ensure highly efficient data center operations.
FishOS Placement Engine and Rebalancing Engine are two decision engines that work in conjunction to ensure highly efficient data center operations.
In order to achieve high resource utilization efficiency, there are 2 situations that need to be handled:
- Given a particular set of new VMs to be initiated, where should the VMs be placed, considering live resource utilization across the entire facility?
- Given a set of VMs running within the facility, how can the VMs be packed onto the optimal number of servers while catering to the resource requirements of the VMs, without performance impact?
The first is referred to as Placement, while the second is called Rebalancing.
While Placement and Rebalancing are related, they have to be treated as different computational tasks, and thus different decisions.
As it will be evident below, the smart resource management system in FishOS relies on the availability of live migration. For Kubernetes, this scheme hinges upon Kubernetes running within VMs.
At a high level, the system analyzes:
- all hypervisors in the system by determining their capacity and
- all running VMs by determining load they create
Information about capacity vs load is obtained from metrics collected from hypervisors and VMs. These metrics reflect such resources as memory, disk, network and CPU.
Using this approach we would minimize the number of hypervisors used to host VMs without putting an additional constraint on how those VMs work.
It allows using a smaller number of servers to host the same amount of VMs, to achieve lower OpEx and CapEx, lowering license and electricity costs.
FishOS Placement Engine
The purpose of the Placement Engine purpose is to determine where to should VM(s) be started (placed) when requested by a Service Consumer.
Where do new VMs go?
The answer to the question of where should new VMs be placed is derived as follows:
If all hypervisors in the system are free, VMs first go to the hypervisor with the smallest capacity to host them. This way, we would not need to use larger servers to just to host a set of small VMs.
If there are hypervisors in the system that are in use (ie: with some VMs on them), new VMs will preferentially be placed there (provided there is a capacity on those hypervisors to host VMs). This is so as to minimize the number of hypervisors that would be required to be turned on to host the VMs.
The Placement Engine makes estimations based on the flavor selected for VM which helps to check if particular VM will fit on a particular hypervisor.
FishOS Rebalancing Engine
The purpose of the Rebalancing Engine purpose is to determine how can an already running infrastructure be rebalanced to be more efficient.
Does the infrastructure need to be optimized? Prior to proceeding to carry out any rebalancing tasks, the first question that arises is whether the infrastructure needs to be optimized.
For each VM there is a series of questions to be answered, including:
- Does it need to be moved to another hypervisor at all?
- Can it be moved? Are there any operations such as excessive disk writes processes or network activity that may prevent live migration from completion?
How to optimize the infrastructure?
Once it has been determined that the running infrastructure is sub-optimal, the next question is how the infrastructure can be optimized. Associated with that, the Rebalancing Engine also handles questions such as:
- What is the optimal mix of VMs movements?
- If a VM can and needs to be moved, where do we move it? (See Placement Engine section.)
- What is the impact following the rebalancing operation?
Once the Rebalancing Plan has been calculated, the Rebalancing Engine issues the requests to perform those migrations.
Placement and Rebalancing
FishOS Placement Engine and Rebalancing Engine use live utilization metrics from the entire facility, to solve the live resource optimization problem. The approach will not only use the reservation values which are specified by the flavor to determine if particular VM will fit on a particular hypervisor, but also use important actual utilization data, obtained by collecting metrics from VMs and hypervisors.
The Placement Engine ensures that the right servers capable of hosting the VM(s) are used while minimizing the number of servers required to host the totality of VM workloads. Placement Engine treats flavors as defined in OpenStack to be upper bound limits on resources that can be assigned to VMs, indicating the maximum amount to be consumed by each VM, controlling the right resources.
FishOS ensures that if a host that is already used is, based on actual resource utilization, still capable of hosting additional VMs, preference would be given to run new additional VMs on that host. FishOS allows an Operator to easily control the number of hypervisors that will be used at any point in time, improving OpenStack resource utilization rates.
The Rebalancing Engine ensures that an already running infrastructure is operated in an efficient manner, enabling Operators to optimize the use of facility resources considering the live resource utilization of workloads.
FishOS Rebalancing Engine by default runs automatically in the system while allowing an Operator to configure rebalancing frequency. Once a Rebalancing Plan has been calculated, the Rebalancing Engine issues the requests to perform those migrations, without requiring intervention from an Operator, and is able to also provide such features as:
- automatic migration of VMs from the disabled hypervisors
- rebalance load by live-migrating non-anchor VMs
- logging of the results to understand the state of the infrastructure
- CLI interface with dry-run option to help Operator understand the impact
- disabled hypervisors are force live-migrated
To enable Placement and Rebalancing Engines to make decisions, each of the Placement Engine and Rebalancing Engine needs data. The data required is collected by a set of state data collectors implemented as plugins of collectd.
Sardina chose to implement the state data collectors as plugins of collectd because collectd has a number of advantages, ranging from it being commonly seen in Linux environments, open source, as well as being easily extensible and has a low cost of integration, thus allowing Linux sysadmins to easily extend and expand upon it.
The state data collected are dispatched by a collectd plugin and eventually ends up in a horizontally-scalable time-series database designed to handle a large number of small writes and small number of large reads.
The data in the time-series database are then consumed by Placement and Rebalancing Engines.
Enabled by Default
Both Placement Engine and Rebalancing Engine are enabled by default in FishOS systems, with Rebalancing Engine set up to run periodically.
As Sardina wants to support OpenStack and its Open Source model, we are now offering free no-charge access to FishOS Deployer for a limited period of time.