During nearly two years of close collaboration with one of Germany’s leading research centers, the University of Tübingen, Sardina Systems has provided essential support to enhance the efficiency and productivity of the university’s machine learning cloud infrastructure.
In this case study, we’ll delve into how FishOS, Sardina Systems’ fully automated, scalable, and cost-effective cloud management platform, helps a small team of experts handle their daily tasks so that data scientists can focus their efforts on conducting groundbreaking research in such fields as medicine, agriculture, autonomous driving, and other areas, and delivering transformative solutions.
Meeting the ML Cloud
The ML Cloud at the University of Tübingen, comprising the AI Centre and the ML Cluster of Excellence, drives impactful projects in medical imaging, including cancer and brain study, as well as computer vision and human movement analysis, climate simulation and prediction initiatives, and even training a Boston Dynamics’ robot for agriculture tasks. With the involvement of 48 research groups, these projects are pushing boundaries and making significant contributions across various domains.
Currently, limited personnel resources are involved in maintaining a cluster consisting of about 200 cores, responsible for all data operations. Let’s explore the challenges the specialists faced before implementing FishOS, and how the platform assisted them overcome these difficulties.
Embracing the Challenge
The university IT team encounters distinctive challenges inherent to public organizations. Operating within stringent government funding constraints, they must ensure the seamless operation of a scientific computing system across multiple units while also addressing the diverse computing needs of various research groups. Stability is crucial for researchers seeking precise results, especially in fields such as cancer and brain research where computing power is indispensable for machine learning applications.
Navigating the Obstacles
In response to these challenges, the university team chose to leverage on-premises infrastructure to maximize budget efficiency and fulfill capacity requirements. They developed a hardware architecture tailored to meet the diverse needs of small research groups dispersed throughout Germany. This architecture combines virtualization and bare metal in a balanced manner, complemented by a distributed Ceph storage facility for archival purposes.
Approximately 15-20% of system nodes are allocated for load balancers, with the remaining nodes dedicated to computing power. Hypervisors are equipped with CPU-only, GPU V100, and GPU A100 configurations, with approximately 70% of nodes currently utilized through bare metal. GPU resources, located both on hypervisors and bare metal, are prioritized to enhance system responsiveness. Additionally, the system accommodates more conventional tasks by providing classic CPUs for functions such as dataset management.
FishOS: Supporting Stable Infrastructure and Streamlining Operations
- Unified Interface. With FishOS, managing the ML Cloud’s infrastructure, spread across several areas, through a unified interface has become remarkably streamlined.
- Flexibility. Perhaps one of the most notable advantages of the ML Cloud is FishOS’ versatility – it can run on both bare metal nodes and GPUs. This flexibility greatly boosts performance and seamlessly facilitates the achievement of research goals.
- Adaptability. Kristina Kapanova, Head of the ML Cloud, praises FishOS for its adaptability.
“It offers a wide range of service options tailored to specific needs: whether it’s managing smaller or larger workloads, publishing datasets, or conducting benchmarks. For me, it’s also a positive outcome, as we can easily manage changes through FishOS.”, says Ms Kapanova.
- Stability. Another crucial advantage of the platform that perfectly aligns with the university’s requirements is the overall system’s stability.
- Time-Saving. Moreover, the considerable time saved by FishOS for other tasks and purposes is of utmost importance, as all necessary modifications can be executed with a few commands. For instance, in just 3-4 simple steps, experts at the ML Cloud can swiftly set up a fully running system for users. The pivotal aspect here lies in the effectiveness and speed with which this task is accomplished, meeting the ever-growing demands of users who are always seeking more.
“I was able to deliver a service to users within a week that would have likely taken us a month. The rapid provision of user functionality and the resulting ease in my workload are invaluable,” highlights Ms Kapanova.
When addressing user needs, the ML Cloud’s ticketing system serves as the organization’s primary internal communication tool, playing a vital role in promptly resolving any emerging issues. Its smooth functioning is credited to FishOS as well.
- Network Management. In addition to stability, network management is a priority. The platform seamlessly oversees the entire system by assigning and managing IP addresses, relieving experts of another heavy responsibility and allowing them to focus on higher-level tasks.
- No Disruptions. Kristina Kapanova observes no disruptions in the platform’s usage. Although the ML Cloud specialists may encounter hardware issues occasionally, once resolved, the cloud infrastructure is back up and running within a couple of minutes. “It was the first time I experienced such a seamless return to service,” Ms Kapanova notes.
- Support. An undeniable advantage of FishOS is its incredibly fast and reliable support provided to the university. Right from the start of the ML Cloud’s adoption of the platform, Sardina Systems’ technical team offered detailed guidance. This enabled specialists, even those unfamiliar with routine operations, to quickly master them.
When several virtual machines experienced a sudden slowdown during the L2L Challenge, one of the most significant machine learning competitions for scientists, the issue of insufficient RAM was swiftly identified and resolved within a few days. The effective and well-coordinated efforts of Sardina Systems experts were instrumental in preventing similar incidents in the future. “Considering its excellent support, functionality, and time-saving benefits, as well as the continuous availability of all systems, FishOS appears to offer excellent value for money,” Ms Kapanova adds.
What’s Next: Tübingen’s Course for Innovation and Growth
In terms of upcoming plans, the AI Centre within the ML Cloud is set to double in size over the next few years. Consequently, the ML Cloud is expanding its infrastructure to meet the growing computing demands. The university aims to broaden its capabilities by incorporating non-GPU processors for specific AI and ML workloads across different areas. This initiative aims to develop customized solutions to reduce energy consumption while maintaining performance.
“I’m delighted that Sardina Systems has tested FishOS for compatibility with Gaudi2, and initial benchmarks look promising. Additionally, I plan to use FPGAs with the platform to offer users more options. The opportunity for bare-metal deployments with these nodes is particularly gratifying,” comments Ms Kapanova.
Furthermore, the ML Cloud is increasing its utilization of Ceph storage, and Sardina Systems will provide support for this, given that FishOS is built on Ceph technology.
The Future Step for Machine Learning Cloud Success
FishOS stands as a game-changer for academic institutions. Simplifying workflows and freeing researchers from technical complexities, it empowers universities like Tübingen to innovate seamlessly while optimizing operational activities.
