In a significant development for the realm of cloud infrastructure, San Francisco-based startup Together AI has made headlines by successfully raising $305 million in its Series B funding round. This impressive round of financing brings the company’s valuation to a substantial $3.3 billion. The funding was co-led by General Catalyst and Prosperity7, with notable contributions from Nvidia Corp., Salesforce Ventures, and John Chambers, the former CEO of Cisco Systems Inc. This injection of capital is expected to drive innovations in Together AI’s cloud platform optimized for artificial intelligence models, signifying a pivotal moment in the integration of AI and cloud services.
Technological Advancements in AI Models
Optimized Cloud for AI Models
Together AI’s innovative approach centers around creating a public cloud infrastructure specifically optimized for running AI models. Their flagship platform allows developers to configure server clusters equipped with high-performance Nvidia Blackwell B200 GPUs. These GPUs are pivotal in the realm of AI due to their exceptional processing power, which significantly enhances the efficiency and speed of running complex AI models. Leveraging these GPUs, Together AI’s technological offerings promise unprecedented advancements in AI model execution, marking a notable shift in cloud infrastructure capabilities.
The technological backbone of Together AI’s optimized cloud lies in its proprietary software system known as the Inference Engine. The Inference Engine claims to double the inference performance of major public cloud providers, a substantial improvement in computational efficiency. This is primarily achieved through FlashAttention-3, an algorithm designed to optimize the attention mechanism in large language models (LLMs). The algorithm works by reorganizing computational order and reducing data movement between the GPU’s logic circuits and HBM memory. Such optimization not only enhances the speed but also maximizes the resource utilization in the AI model execution process.
FlashAttention-3 and Speculative Decoding
FlashAttention-3 also introduces speculative decoding, a technique that revolutionizes the way AI models generate outputs. This innovative approach allows the generation of multiple tokens simultaneously, significantly speeding up the process of model inference. By incorporating speculative decoding, FlashAttention-3 reduces latency and enhances real-time processing capabilities, which are crucial for applications dependent on rapid data processing and decision-making. This advancement positions Together AI at the forefront of offering highly efficient and quick AI solutions within their cloud infrastructure.
The benefits of FlashAttention-3 are further amplified through its integration with other cutting-edge tools provided by Together AI. Their Training Stack, a suite of training tools, also utilizes FlashAttention-3 to enhance processing speed. The Training Stack is designed to facilitate the efficient training of AI models by optimizing the order of operations and minimizing redundant data transfers. This comprehensive approach ensures that developers can maximize the performance of their models without being hindered by traditional computational bottlenecks, providing a significant advantage in model training and deployment.
Expanding AI Capabilities
Open-source AI Models and Datasets
One of the standout features of Together AI’s platform is its extensive library of over 200 open-source neural networks. These neural networks provide a robust foundation for developers seeking pre-built large language models (LLMs) that can be customized to meet specific project requirements. Together AI’s fine-tuning tool streamlines this process, allowing developers to initiate projects with a single command. This simplifies the workflow, ensuring that developers can quickly adapt and refine models to suit their unique needs, thus accelerating the overall development cycle.
In addition to its extensive neural network library, Together AI has also developed an impressive open-source dataset containing over 30 trillion tokens. This dataset is designed to support the training of customer AI models, providing a vast resource of data to enhance model accuracy and performance. By offering such comprehensive datasets, Together AI enables developers to train their models on diverse and extensive data, leading to more robust and reliable AI solutions. This approach not only democratizes access to high-quality training data but also fosters innovation by empowering developers to experiment and iterate on their models.
Developer Adoption and Performance Milestones
The impact of Together AI’s technological advancements is evident in its growing user base, with over 450,000 developers currently utilizing the platform. This includes engineers from renowned organizations such as Salesforce Inc., DuckDuckGo Inc., and the Mozilla Foundation. The widespread adoption of Together AI’s platform underscores its efficacy and the confidence that leading technology companies place in its solutions. These developers benefit from the platform’s high-performance infrastructure and cutting-edge AI tools, enabling them to create and deploy sophisticated AI models with ease.
Reaching a significant milestone, Together AI has achieved $100 million in annualized recurring revenue. This financial success is a testament to the value and potential of their platform in the competitive AI and cloud infrastructure landscape. The substantial funding acquired in the Series B round will be pivotal in further enhancing the platform’s capabilities. With new capital, Together AI is poised to expand its infrastructure, develop advanced AI tools, and support a broader range of applications and industries, thus consolidating its position as a leader in AI-focused cloud solutions.
Future Directions and Innovations
Scaling Infrastructure and Power
One of the critical aspects of Together AI’s vision for the future involves scaling their infrastructure to meet the growing demand for high-performance AI solutions. The recent acquisition of 20 gigawatts of power generation capacity is a strategic move to support new AI clusters. This substantial power generation capacity will ensure that Together AI can maintain and expand its high-performance infrastructure, essential for running large-scale AI models. The company’s commitment to scaling its infrastructure demonstrates its forward-thinking approach and readiness to accommodate the increasing computational demands of AI advancements.
An upcoming AI cluster is set to feature 36,000 Nvidia GB200 NVL72 chips, each equipped with two central processing units and four Blackwell B200 graphics cards. This cluster represents a significant augmentation of Together AI’s computational power, enabling more complex and resource-intensive AI models to be executed efficiently. By continuously scaling its infrastructure and incorporating the latest hardware, Together AI aims to provide unmatched computational capabilities, supporting the next generation of AI-driven applications and research. This strategic expansion will position Together AI as a frontrunner in delivering scalable and efficient AI infrastructure solutions.
Commitment to AI Efficiency and Scalability
In a significant milestone for cloud infrastructure, San Francisco-based startup Together AI has captured attention by securing $305 million in its Series B funding round. This impressive funding surge boosts the company’s valuation to a notable $3.3 billion. The successful funding round was led by General Catalyst and Prosperity7 Ventures, with notable investments from Nvidia Corp., Salesforce Ventures, and John Chambers, the former CEO of Cisco Systems Inc. This substantial investment aims to foster advancements in Together AI’s cloud platform, which is specially designed for artificial intelligence models. This marks a crucial step in the evolving convergence of AI and cloud computing services. The added financial support is anticipated to accelerate the development and deployment of innovative solutions within the AI and cloud sectors, positioning Together AI as a prominent player in the industry.
