What can and can't language models do? Lessons learned from BIGBench

So what exactly can and can’t language models do? What's the least impressive thing GPT-4 won't be able to do? What will GPT-4 be incapable of? BIGBench is kind of a way to figure this out. BigBench, aka “The Beyond the Imitation Game” Benchmark, is an attempt to explore the capabilities of large language models over a wide variety of tasks. All the tasks are enumerated here. I looked through every BIGBench task and took the ones that compared both GPT3 and PaLM against humans. * Spreadsheet

Large language models encode clinical knowledge

linkpost] The final AI benchmark: BIG-bench — LessWrong

Sebastian Raschka, PhD on LinkedIn: In the new Language Models

Benchmark of LLMs (Part 1): Glue & SuperGLUE, Adversarial NLI, Big

PaLM 2 And 19 Other AI Tools For Large Language Models

Language Models Perform Reasoning via Chain of Thought – Google

TECHTALK. AI scientists are studying the “emergent” abilities of

BIG-Bench: The New Benchmark for Language Models

Large Language Model: Most Up-to-Date Encyclopedia, News & Reviews

Pathways Language Model (PaLM): Scaling to 540 Billion Parameters