Why Build an LLM Benchmark 📊

📊 Dive Deep into the World of LLM Benchmarks! 📊

Objective: By the end of this session, you should have a good understanding of how to select and maintain your own LLM benchmark.

Agenda:
🔬 Demo!
🔍Discover what ARC, HellSwag, and MMLU are exactly
🧫 Learn how to select the right benchmark
🧪 Methods to test LLMs tailored to your unique use case
🧱 Q&A

Speaker: J. Yarkoni ex-Google AI/ML Specialist (Shujin.ai)
Jonathan comes from a background of leading R&D teams. Previously he co-founded NAM, an advertising startup, and AA-TLV meetup, which at its peak had 3,500 members. Over the last six years, he spearheaded AI/ML initiatives at Google Cloud Israel. More recently, he established Shujin.AI, a consultancy specializing in ML projects with an emphasis on Generative AI.

https://big-data-demystified.ninja/2024/01/17/why-you-should-build-an-llm-benchmark/