Google launches PaLM 2, its next-gen large language model

At its I/O developer conference, Google today announced the launch of PaLM 2, its newest large language model (LLM). PaLM 2 will power Google’s updated Bard chat tool, the company’s competitor to OpenAI’s ChatGPT, and function as the foundation model for most of the new AI features the company is announcing today. PaLM 2 is now available to developers through Google’s PaLM API, Firebase and on Colab.

Google, similar to OpenAI, didn’t provide a lot of the technical details about how it trained this next-gen model, including parameter counts (PaLM 1 is a 540-billion parameter model, for what it’s worth). The only technical details Google provided here are that PaLM 2 was built on top of Google’s latest JAX and TPU v4 infrastructure.

Image Credits: Google

“What we found in our work is that it’s not really the sort of size of model — that the larger is not always better,” DeepMind VP Zoubin Ghahramani said in a press briefing ahead of today’s announcement. “That’s why we’ve provided a family of models of different sizes. We think that actually parameter count is not really a useful way of thinking about the capabilities of models and capabilities are really to be judged by people using the models and finding out whether they’re useful in the tests that they try to achieve with these models.”

Instead, the company decided to focus on its capabilities. Google says the new model is better at common sense reasoning, mathematics and logic. Indeed, as Ghahramani noted, the company trained the model on a large amount of math and science texts, as well as mathematical expressions. It’s no secret that large language models — with their focus on language — have struggled with handling math questions without resorting to third-party plugins. Google, however, argues that PaLM 2 can easily solve math puzzles, reason through problems and even provide diagrams.

PaLM 2 also now features improved support for writing and debugging code. The model was trained on 20 programming languages, including popular ones like JavaScript and Python, but also the likes of Prolog, Verilog and Fortran. PaLM 2 forms the basis of Codey, Google’s specialized model for coding and debugging, which it is also launching today as part of its code completion and generation service, among other things.

Google today also highlighted that PaLM 2 was trained on a corpus that features over 100 languages, making it, in Google’s words, “excel at multilingual tasks,” including more nuanced phrasing than previous models.

Google talks about PaLM as a family of models, which include the likes of Codey but also Med-PaLM 2, the company’s model focused on medical knowledge. There is also Sec-PaLM, a version that focuses on security use cases and a smaller PaLM 2 model that can run on smartphones, which could potentially open up PaLM to more privacy-centric use cases, though Google wouldn’t commit to any timeline for this. Google says this model can process 20 tokens per second, which isn’t extremely fast, but may just be acceptable for some use cases (Google wouldn’t say which phone it tested this on, though).

It’s no secret that Google has taken a very deliberate approach to launching these AI features — something the company acknowledges. But at the same time, the standard line from Google’s representatives about this is that the company wants to build these tools responsibly and with safety in mind. That, of course, is also what the company says about PaLM. Without being able to test it before today’s announcement, we obviously don’t know how well it performs and how it handles edge cases.