It is enormously expensive to run AI models. A single Nvidia AI chip can cost over $30,000. It costs OpenAI $700,000 a day to operate ChatGPT, according to one estimate. With costs like that, only the biggest companies have the money to develop and run AI. SqueezeBits, a two-year-old startup in South Korea, says it can lower that cost and democratize the hot technology. To help it do that, SqueezeBits recently got support from one of the country’s top internet companies.
In January, SqueezeBits raised 2.5 billion won (almost $2 million) in pre-Series A funding. The Seoul-based startup declined to disclose valuation, but a person familiar with the matter says the round values SqueezeBits at about $15 million. The round’s investors include Kakao Ventures, the venture capital arm of billionaire Kim Beom-su’s Kakao, one of South Korea’s top two internet companies.
“We look at not just how companies will use AI, but also how AI will be democratized and spread to create more AI applications,” says Justin Shin, senior associate at Kakao Ventures, in a video interview. Kakao Ventures is also an early backer of South Korea’s Rebellions, which is developing cheaper AI chips (about half the price of Nvidia’s H100 chips) and recently raised $124 million at a $650 million valuation, becoming the best-funded AI chip startup in the country.
Other investors in the round include Samsung Electronics’ Samsung Next; Posco Venture Capital, the venture capital arm of South Korean steel giant Posco; and Postech Holdings, an accelerator backed by Pohang University of Science and Technology (Postech), one of South Korea’s top technology universities.
SqueezeBits previously raised 1 billion won in seed funding in 2022 from D2 Startup Factory—backed by South Korean billionaire Lee Hae-jin’s Naver, the other South Korean internet giant besides Kakao—and Postech Holdings. In total the startup has raised 3.5 billion won in venture funding.
“For a lot of these AI applications, they’re looking to minimize costs and maximize performance with AI models,” says Shin. “Cost is definitely the biggest issue. You need a lot of money to build scalable AI models and then translate that to a working product. That takes a lot of money.”
SqueezeBits says it is able to lower the cost by making it more efficient. “AI models are very overparameterized,” Hyungjun Kim, cofounder and CEO of SqueezeBits, says in a separate video interview. “Many companies are not actually fully optimizing the model and using it.” The number of parameters is a key metric of an AI model’s size and usually, but not always, correlates to performance. For example, OpenAI’s GPT-3 language model has 175 billion parameters and its GPT-4, which powers the latest version of ChatGPT, reportedly has 1.7 trillion parameters.
“There exists some useless parameters and data in the models. We are basically getting rid of those kinds of useless or less important data in the model or in the computation process, so that we can reduce the computation cost and memory usage,” explains Kim, 29, who holds a doctorate in electrical engineering and computer science from Postech. “That leads to cheaper and faster AI inference.”
Kim says SqueezeBits can help make a model three to five times faster and reduce its memory usage by four times. Last month, the startup launched a software-as-a-service toolkit to help companies optimize open-source AI models, or even their own large language models, for cloud services.
Of course, SqueezeBits is not the only company optimizing AI models. Other companies include OmniML, founded in San Jose in 2021, and Seattle-based Xnor.ai, which in 2017 spun off from the Allen Institute for AI, a nonprofit dedicated to AI research started by late Microsoft cofounder Paul Allen.
OmniML, whose investors include GGV Capital, Qualcomm Ventures and IMO Ventures (its portfolio includes billionaire Peter Thiel’s data-mining company Palantir Technologies and Google-backed AI drug discovery startup Xtalpi), was bought by Nvidia in February, according to tech news site the Information. Xnor.ai, backed by the likes of Madrona Venture Group (Seattle’s largest VC firm and an early investor in Amazon) and Nokia-funded NGP Capital, was acquired by Apple in 2020 for about $200 million, reported tech news site GeekWire.
Other independent startups include Deci in Israel—backed by Insight Partners, one of the world’s largest technology investors; and Square Peg Capital, an Australian VC firm whose portfolio includes Airwallex and Canva—and Neural Magic in Massachusetts, whose investors include Andreessen Horowitz and NEA, an early investor in Robinhood, cloud-based networking and cybersecurity services provider Cloudflare and data analytics company Databricks.
SqueezeBits’ domestic competitors include Nota, which raised a $14.7 million Series B in 2021. Investors in that round included Stonebridge Ventures (an early backer of Seoul- and Seattle-based chip design startup MangoBoost) and Company K Partners (portfolio companies include autonomous drone startup Nearthlab, which made last year’s Forbes Asia 100 to Watch).
Kim is not fazed by the competition. The development of hardware is always going to be slower than software development, creating demand for AI optimization providers like SqueezeBits. “There’s a bunch of new stuff coming out regarding AI, but hardware can’t keep up with that kind of speed,” says Kim. “So there’s this big gap between the super fast progress of AI algorithms and model size and the hardware that doesn’t support it.”
“And that’s where we come in,” he says.
