The relentless migration of high-performance computing and artificial intelligence workloads to the cloud has exposed a critical gap in the storage offerings of major providers, leaving a substantial segment of the enterprise market to navigate complex, self-managed deployments. For over two decades, the HPC landscape has been defined by a duopoly of powerful parallel file systems: the open-source Lustre and IBM’s proprietary General Parallel File System (GPFS), now rebranded as IBM Storage Scale. While cloud giants have successfully rolled out managed services for Lustre to meet customer demand, the vast and loyal user base of GPFS has been conspicuously underserved. This disparity has forced organizations reliant on GPFS to undertake the burdensome task of manually building and maintaining their storage infrastructure in the cloud, a challenge that a new managed service from Sycomp is designed to eliminate by offering a truly cloud-native experience on leading public cloud platforms.
The Persistent Demand for Parallel File Systems
The long-standing market dominance of Lustre and GPFS is rooted in their unparalleled ability to handle the extreme demands of scientific research, AI model training, and other data-intensive applications. These are not ordinary file systems; they are engineered to manage immense storage pools, often scaling to tens or even hundreds of petabytes, while delivering the staggering I/O bandwidth required to feed thousands of compute nodes simultaneously, with throughput reaching terabytes per second. For years, these two systems have been the de facto standards, with Lustre claiming approximately half the capacity share on pre-exascale and exascale supercomputers and GPFS commanding a significant portion of the remaining market. As organizations seek the flexibility and scalability of the cloud, they are not looking to abandon the robust and familiar storage architectures that have powered their on-premises operations. The demand is not just for high performance, but for the specific, proven capabilities that their existing applications and workflows are built upon.
This established user preference created a significant hurdle for enterprises moving to the cloud. While major providers like Amazon Web Services, Microsoft Azure, Google Cloud, and Oracle Cloud Infrastructure recognized the need and launched managed services for Lustre, the proprietary nature of GPFS presented a different challenge. Consequently, GPFS users were confined to a “bring-your-own-license” (BYOL) model. This approach placed the full operational burden on the customer, who was responsible for deploying the software on virtual machine instances, configuring complex networking, managing updates, and troubleshooting performance issues. This manual process is not only resource-intensive and requires specialized expertise but also runs counter to the primary value proposition of the cloud: simplicity, agility, and reduced operational overhead. The absence of a managed option created a clear market need for a solution that could deliver the power of GPFS with the convenience of a cloud-native service.
A Cloud Native Approach to Managed GPFS
Addressing this market need directly, global services provider Sycomp, in collaboration with IBM, has introduced the Sycomp Intelligent Data Storage Platform. This new offering is a fully managed GPFS service, initially available for deployment on Google Cloud and Microsoft Azure, with plans to extend support to AWS and OCI as customer demand dictates. The platform is architected from the ground up to align with modern cloud operational practices, distinguishing itself through its implementation as “infrastructure as code.” Instead of relying on a graphical user interface for setup, Sycomp has packaged the entire IBM Storage Scale software stack into modules for Terraform, the widely adopted provisioning tool from HashiCorp. This enables customers to integrate the deployment of their high-performance file system directly into their existing DevOps and CI/CD pipelines, automating the provisioning of storage alongside their compute and networking resources for consistent, repeatable, and scalable deployments.
The technical foundation of the platform’s versatility lies in a powerful GPFS feature known as the Active File Manager (AFM). Sycomp leverages AFM to deliver two crucial capabilities that differentiate the service. First, it enables sophisticated data mobility, allowing the platform to connect seamlessly with existing on-premises NFS file systems and any object storage system that adheres to the standard AWS S3 protocol. A key advantage of this implementation is its commitment to open data formats. Unlike some storage systems that write data to object stores in a proprietary format, the AFM in Sycomp’s service reads standard objects and, more importantly, writes them back in an open, accessible format. This prevents vendor lock-in and ensures that data remains interoperable with other native cloud tools. Second, AFM provides the mechanism for creating a global namespace, which can present a single, unified view of files and directories that spans multiple cloud data centers, different cloud regions, and even on-premises installations for a truly global file system.
Delivering a Seamless and Optimized Cloud Experience
Sycomp’s contribution goes well beyond simply packaging GPFS for cloud deployment; the company has invested significant engineering effort to enhance the platform for a truly cloud-native environment. Recognizing that on-premises features do not always translate directly to the cloud, Sycomp’s engineers developed custom code to make critical functions “cloud-aware.” For example, they wrote new software to enable the NFS IP failover feature in GPFS to interact correctly with the specific networking infrastructure of platforms like Google Cloud, ensuring high availability in a dynamic cloud environment. Furthermore, the platform integrates directly with native cloud services for simplified monitoring. On Google Cloud, it is configured to generate Google Alerts, allowing operations teams to incorporate storage monitoring into their existing dashboards and workflows without needing to adopt new tools or processes. This deep integration streamlines management and reduces complexity for cloud administrators.
These enhancements culminated in a platform that fundamentally simplified the user experience and maximized performance. Sycomp focused on automating complex GPFS operations that would traditionally require deep administrative expertise. By targeting a specific and optimized scope of cloud hardware, the service streamlines tasks that once required a sequence of ten or more commands into a single, user-friendly command. This automation covers critical functions such as monitoring the progress of data migrations initiated via AFM and performing batch operations to add or remove client nodes from the file system. The Sycomp Intelligent Data Storage Platform was made generally available on the Google Cloud Marketplace, built on enterprise-grade Linux distributions including Rocky Linux and Red Hat Enterprise Linux. It was optimized for Google’s most I/O-intensive virtual machines, such as the C3 and Z3 instance families, which leverage high-performance local flash storage and proprietary I/O offload engines to ensure the underlying infrastructure could match the powerful capabilities of the GPFS file system.
