Chip and artificial intelligence (AI) model developer SambaNova Systems has launched an AI cloud offering.
Dubbed SambaNova Cloud, it provides cloud-based AI inference services using the SN40L AI chip.
The service can be accessed by developers via an API and create their own generative AI applications using the open-source Llama 3.1 405B and Llama 3.1 70B models. SambaNova can run the models at 100 tokens per second and 461 tokens per second respectively.
“SambaNova Cloud is the fastest API service for developers. We deliver world record speed and in full 16-bit precision — all enabled by the world’s fastest AI chip,” said Rodrigo Liang, CEO of SambaNova Systems. “SambaNova Cloud is bringing the most accurate open source models to the vast developer community at speeds they have never experienced before.”
“Competitors are not offering the 405B model to developers today because of their inefficient chips. Providers running on Nvidia GPUs are reducing the precision of this model, hurting its accuracy, and running it at almost unusably slow speeds,” continued Liang. “Only SambaNova is running 405B — the best open-source model created - at full precision and at well over 100 tokens per second.”
According to SambaNova, its SN40L chips reduce the cost and complexity of running Llama 3.1 405B compared to Nvidia H100s, describing the model as the largest frontier open-weight model released to date.
The company claims it is the fastest AI inference platform, citing independent benchmarking by Artifical Analysis.
SambaNova cloud is available from today (September 10) and comes in Free, Developer, and Enterprise Tiers.
It launched the SN40L chip in September 2023 and is capable of running models with up to 5 trillion parameters. Manufactured by TSMC, SN40L supports 256k+ sequence length possible on a single SN40L node.
SambaNova Systems was founded in 2017 and is headquartered in Palo Alto, California. Investors include SoftBank Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others.
