H2O AI releases Danube, a super-tiny LLM for cellular functions

Right this moment, H2O AI, the corporate working to democratize AI with a variety of open-source and proprietary instruments, introduced the discharge of Danube, a brand new super-tiny giant language mannequin (LLM) for cellular units.

Named after the second-largest river in Europe, the open-source mannequin comes with 1.8 billion parameters and is alleged to match or outperform equally sized fashions throughout a variety of pure language duties. This places it in the identical class as sturdy choices from Microsoft, Stability AI and Eleuther AI.

The timing of the announcement makes good sense. Enterprises constructing client units are racing to discover the potential of offline generative AI, the place fashions run domestically on the product, giving customers fast help throughout capabilities and eliminating the necessity to take data out to the cloud.

“We’re excited to launch H2O-Danube-1.8B as a transportable LLM on small units like your smartphone… The proliferation of smaller, lower-cost {hardware} and extra environment friendly coaching now permits modestly-sized fashions to be accessible to a wider viewers… We consider H2O-Danube-1.8B will probably be a recreation changer for cellular offline functions,” Sri Ambati, CEO and co-founder of H2O, mentioned in an announcement.

What to anticipate from Danube-1.8B LLM?

Whereas Danube has simply been introduced, H2O claims it may be fine-tuned to deal with a variety of pure language functions on small units, together with widespread sense reasoning, studying comprehension, summarization and translation. 

To coach the mini mannequin, the corporate collected a trillion tokens from numerous net sources and utilized strategies refined from Llama 2 and Mistral fashions to reinforce its era capabilities.

“We adjusted the Llama 2 structure for a complete of round 1.8B parameters. We (then) used the unique Llama 2 tokenizer with a vocabulary measurement of 32,000 and educated our mannequin as much as a context size of 16,384. We included the sliding window consideration from Mistral with a measurement of 4,096,” the corporate famous whereas describing the mannequin structure on Hugging Face.

When examined on benchmarks, the mannequin was discovered to be acting on par or higher than most fashions within the 1-2B-parameter class. 

For instance, within the Hellaswag check geared toward evaluating widespread sense pure language inference, it carried out with an accuracy of 69.58%, sitting simply behind Stability AI’s Steady LM 2 1.6 billion parameter mannequin pre-trained on 2 trillion tokens. Equally, within the Arc benchmark for superior query answering, it ranks third behind Microsoft Phi 1.5 (1.3-billion parameter mannequin) and Steady LM 2 with an accuracy of 39.42%.

H2O has launched Danube-1.8B underneath an Apache 2.0 license for business use. Any staff seeking to implement the mannequin for a cellular use case can obtain it from Hugging Face and carry out application-specific fine-tuning. 

To make this course of simpler, the corporate additionally plans to launch extra tooling quickly. It has additionally launched a chat-tuned model of the mannequin (H2O-Danube-1.8B-Chat), which might be carried out for conversational functions.

In the long term, the provision of Danube and comparable small-sized fashions is anticipated to drive a surge in offline generative AI functions throughout telephones and laptops, serving to with duties like electronic mail summarization, typing and picture modifying. Actually, Samsung has already moved on this route with the launch of its S24 line of smartphones.