What operators need to run GPU as a service on infrastructure they control, and where an open-source-based platform like FishOS fits.
Key takeaways
- For a provider, GPU as a service isn’t only the hardware. It is the operating layer that turns GPU capacity into a product customers can launch, use, and pay for.
- That operating layer needs four things: standard GPU products, tenant isolation, billing, and a platform that can be upgraded over time.
- FishOS, an open-source-based cloud platform, provides this layer. Gigabrain, a GPU-as-a-Service provider in South Africa, is already using it as its cloud-platform technology provider.
For a GPU cloud provider, buying hardware is the easier part. The harder part is turning them into GPU as a service that multiple customers can launch, use, and pay for. Therefore, the biggest strategic problem is building infrastructure that can be run as a dependable, multi-tenant service.
Why GPUs Alone Do Not Make GPU as a Service
A rack of GPUs is not a cloud. It only creates capabilities. However, before your first customer logs in, the provider has to answer practical questions:
- How do users get GPU capacity in a standard form, rather than by requesting individual servers?
- How is each customer’s environment isolated?
- How is usage measured and billed?
- How to maintain and support the platform continuously and without rebuilding?
Without answers to these, the provider has to either build cloud plumbing from scratch – or end up with a pack of expensive hardware that sits idle most of the time.
What a GPU as a Service Platform Has to Do
Based on common practices of GPU cloud operators, here are the most useful things a workable platform should have:
- Standard GPU products that customers can launch on their own.
- Virtual machines and Kubernetes, since many AI workloads run in containers.
- Multi-tenant isolation, so consumers and projects can be separated by design.
- Billing and usage tracking that includes GPU capacity and provides clear and transparent data.
- Repeatable environments, so each customer does not rebuild drivers and configuration by hand.
- Monitoring and access controls for the operator.
- A life-cycle that keeps the platform upgradeable for the life of the service.
Notably, these requirements apply to any provider, be it a commercial AI cloud service, or a research facility running projects on their GPU infrastructure. The only thing that’s different can be the workload patterns and billing setup.
Running GPU as a Service on FishOS
FishOS is an open-source-based platform built to manage cloud and AI infrastructure. For GPU cloud providers, it offers all the necessary tools and features that make the job easier, allowing operators to shift focus from the platform maintenance and rebuilding to serving their customers.
Here are four examples of jobs that Sardina Systems simplifies for GPU as a Service providers:
- GPU capacity as a standard product. The operator defines GPU and vGPU flavors, which are fixed hardware configurations, alongside CPU flavors. Customers launch GPU-enabled instances from these flavors rather than requesting servers.
- Multi-tenancy by design. Customers work in isolated domains and projects. GPU instances follow the same separation as the rest of the cloud.
- Easy-to-manage billing. FishOS Billing charges by flavor and time: the hardware configuration a customer rents, for as long as the instance exists. GPU and vGPU flavors are billed the same way as any other flavor. The billing component also includes automated invoicing and payment management.
- Upgrades without rebuilding. FishOS follows the OpenStack official release cycle with two releases a year. The upgrade preparation is handled by Sardina Systems’ engineers, so you can get an upgraded system with no action or downtime planning.
- Kubernetes – as an option for teams running AI workloads in containers. FishOS offers Kubernetes as a service through OpenStack Magnum, so customers can create their own Kubernetes clusters inside their projects.
Moreover, FishOS is built to run different types of workloads. If your portfolio includes standard cloud or backup services, FishOS is fit to run them alongside your GPU infrastructure. See the full component stack of the FishOS platform.
GPU as a Service on Infrastructure You Control
The current geopolitical and regulatory developments demonstrate that the question of data residency and digital sovereignty will continue to matter.
FishOS allows AI cloud operators to build an infrastructure that meets these regulations:
- Operators can deploy FishOS on any hardware in any region.
- Operators get full control over their infrastructure operations. Sardina Systems doesn’t access your users’ data or interfere with your operations.
- Sardina Systems is not subject to US regulations.
- The platform is built on an open-source foundation, making you protected from permanently accessing your storage data or OpenStack environment.
Who Runs GPU as a Service on FishOS
GPU-as-a-service providers in different markets already run their GPU clouds on FishOS.
Sardina Systems is the cloud-platform technology provider for the GPU-accelerated cloud of Gigabrain, a GPU-as-a-service provider in South Africa. Gigabrain describes its offer as a sovereign AI cloud, built and owned in Africa, with infrastructure in Johannesburg. FishOS provides GPU-enabled instances, multi-tenancy, and isolated customer domains, with monitoring and access controls for the operator.
Datacomm Diangraha, an IT service provider in Indonesia with more than 30 years in the market, also uses FishOS to provide GPU and other cloud services to its customers. Growing demand for GPU and container workloads was one of the reasons Datacomm added a fourth data center on FishOS. For Datacomm, GPU capacity is one part of a broader service on the same platform, which also covers built-in billing, Ceph storage, and VMware migrations for its customers.
Last year, Datacomm Diangraha more than doubled its capacity for GPU and container workloads, and runs four data centers from one FishOS control plane. Read the Datacomm case study.
Who This Is For
The FishOS infrastructure management platform fits teams that want to offer GPU capacity while keeping control of their environment:
- Cloud providers and data center operators adding GPU as a service to an existing cloud, or launching a GPU or AI cloud.
- AI and HPC infrastructure providers that sell VM-based GPU capacity to many customers.
- Research and public-sector infrastructure teams offering GPU capacity to their own users under clear identity and data-governance rules.
In each case, the GPUs deliver capability, and the operating layer decides whether it becomes a dependable service.
FAQ
1. What is GPU as a service?
GPU as a service (GPUaaS) is a cloud model in which GPU capacity is offered on demand instead of being bought as hardware. For the customer, it means using GPU instances as needed. For the provider, it means running a platform that packages GPU capacity into standard products, separates customers, bills for usage, and stays upgradeable.
2. Where is the GPU as a service market heading?
According to Gartner, worldwide spending on AI-optimized infrastructure as a service will grow 96% in 2026, to $42.3 billion. For the first time, spending on inference (about $23.3 billion) is expected to exceed spending on model training ($19 billion). Gartner links the shift to the rise of agentic AI. For providers, this points to demand for capacity that serves many users continuously, rather than a few large training jobs.
3. Can GPU as a service run on OpenStack?
Yes, GPU cloud can be provided on OpenStack. It supports GPU instances through GPU passthrough and virtual GPUs (vGPU), offered to customers as flavors. On top of that, a provider still needs the operating layer: tenancy, billing, and upgrades. All the are provided by FishOS on top of an OpenStack environment.
