Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
LLMs have enormous potential, but also challenge existing workflows in industry that require modularity, transparency and data privacy. In this talk, Ines shows some practical solutions for using the latest models in real-world applications and distilling their knowledge into smaller and faster components that you can run and maintain in-house.
-
Ines Montani Explosion A practical guide to human-in-the-loop distillation
-
SOFTWARE IN Industry
-
modular SOFTWARE IN Industry
-
modular transparent SOFTWARE IN Industry
-
modular transparent explainable SOFTWARE IN Industry
-
modular transparent explainable data-private SOFTWARE IN Industry
-
modular transparent explainable data-private reliable SOFTWARE IN Industry
-
modular transparent explainable data-private reliable a ordable SOFTWARE IN Industry
-
Exceeds expectations kinda meh, really Just got the SpacePhone Nebula
and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
Exceeds expectations kinda meh, really find mentions of products Just
got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
Exceeds expectations kinda meh, really find mentions of products link
mentions to catalog SpacePhone Nebula Released: June 2024 P3204-W2130 Just got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
Exceeds expectations kinda meh, really extract sentiment for di erent
attributes Battery Camera Performance Design camera battery design battery camera find mentions of products link mentions to catalog SpacePhone Nebula Released: June 2024 P3204-W2130 Just got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
add results to database Exceeds expectations kinda meh, really extract
sentiment for di erent attributes Battery Camera Performance Design camera battery design battery camera find mentions of products link mentions to catalog SpacePhone Nebula Released: June 2024 P3204-W2130 Just got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
add results to database Exceeds expectations kinda meh, really extract
sentiment for di erent attributes Battery Camera Performance Design camera battery design battery camera find mentions of products link mentions to catalog SpacePhone Nebula Released: June 2024 P3204-W2130 Just got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
add results to database Exceeds expectations kinda meh, really extract
sentiment for di erent attributes Battery Camera Performance Design camera battery design battery camera find mentions of products link mentions to catalog SpacePhone Nebula Released: June 2024 P3204-W2130 Just got the SpacePhone Nebula and I’m honestly blown away! The camera quality is amazing. And the battery life is incredible, easily lasting me a full day on a single charge. the nebula surely looks nice and all but for that price tag i expected more tbh… never had to carry a powerbank with my old iphone 13 but now i need it all the time 🙃 and night mode doesn’t really work. my pics are way too dark!
-
large generative model
-
in-context learning Falcon MIXTRAL GPT-4 large generative model
-
CLOSE THE GAP BETWEEN prototype AND production
-
processing pipeline prototype
-
human IN THE LOOP
-
continuous evaluation baseline human IN THE LOOP
-
continuous evaluation baseline prompting human IN THE LOOP
-
continuous evaluation baseline prompting human IN THE LOOP
-
continuous evaluation baseline prompting transfer learning human IN THE LOOP
-
CASE STUDY #1 400mb model size 2k+ words/second 8hr data
dev time spacy.fyi/pydata-nyc • PyData NYC 2023 workshop: extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • beat few-shot LLM baseline of 0.74 with task-specific model • 20× inference time speedup
-
THINK OF IT AS A refactoring PROCESS
-
MAKE PROBLEM easier
-
🛠 application MAKE PROBLEM easier less operational complexity means less
can go wrong development complexity beginner 🤓 intermediate 🥸 advanced 😎 🎓 research • build a commons of knowledge • make direct comparisons using standard evaluations • standardize what isn’t novel • learn from commons of knowledge • align evaluation to project goals • do whatever works
-
CASE STUDY #3 1 year of support tickets 6× speedup
explosion.ai/blog/gitlab-support-insights • GitLab: extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions
-
CASE STUDY #3 1 year of support tickets 6× speedup
explosion.ai/blog/gitlab-support-insights • GitLab: extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic
-
CASE STUDY #3 1 year of support tickets 6× speedup
explosion.ai/blog/gitlab-support-insights • GitLab: extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic
-
REALITY IS NOT AN end-to-end PREDICTION PROBLEM explosion.ai/blog/human-in-the-loop-distillation
-
REALITY IS NOT AN end-to-end PREDICTION PROBLEM Iteration and the
right tooling can get you past the prototype plateau. Human-in-the- loop distillation is a refactoring process. Less operational complexity means less can go wrong. Expect surprises from the data, and plan for change. explosion.ai/blog/human-in-the-loop-distillation
-
REALITY IS NOT AN end-to-end PREDICTION PROBLEM Iteration and the
right tooling can get you past the prototype plateau. Human-in-the- loop distillation is a refactoring process. Less operational complexity means less can go wrong. Expect surprises from the data, and plan for change. There’s no need to compromise on development best practices or privacy. explosion.ai/blog/human-in-the-loop-distillation
Explosion AI Blog
https://speakerdeck.com/inesmontani/taking-llms-out-of-the-black-box-a-practical-guide-to-human-in-the-loop-distillationSign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
modelapplicationcomponent
Netflix AI Team Just Open-Sourced VOID: an AI Model That Erases Objects From Videos — Physics and All
Video editing has always had a dirty secret: removing an object from footage is easy; making the scene look like it was never there is brutally hard. Take out a person holding a guitar, and you re left with a floating instrument that defies gravity. Hollywood VFX teams spend weeks fixing exactly this kind of problem. [ ] The post Netflix AI Team Just Open-Sourced VOID: an AI Model That Erases Objects From Videos — Physics and All appeared first on MarkTechPost .
Sharing Two Open-Source Projects for Local AI & Secure LLM Access 🚀
Hey everyone! I’m finally jumping into the dev.to community. To kick things off, I wanted to share two tools I’ve been developing at the University of Jaén that tackle two common headaches in the AI space: running out of VRAM, and keeping your API chats truly private. 🦥 Quansloth: TurboQuant Local AI Server The Problem: Standard LLM inference hits a "Memory Wall" with long documents. As context grows, your GPU runs out of memory (OOM) and crashes. The Solution: Quansloth is a fully private, air-gapped AI server that brings elite KV cache compression to consumer hardware. By bridging a Gradio Python frontend with a highly optimized llama.cpp CUDA backend, it prevents GPU crashes and lets you run massive contexts on a budget. Key Features: 75% VRAM Savings: Based on Google's TurboQuant (ICL
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Models
Netflix AI Team Just Open-Sourced VOID: an AI Model That Erases Objects From Videos — Physics and All
Video editing has always had a dirty secret: removing an object from footage is easy; making the scene look like it was never there is brutally hard. Take out a person holding a guitar, and you re left with a floating instrument that defies gravity. Hollywood VFX teams spend weeks fixing exactly this kind of problem. [ ] The post Netflix AI Team Just Open-Sourced VOID: an AI Model That Erases Objects From Videos — Physics and All appeared first on MarkTechPost .
Sharing Two Open-Source Projects for Local AI & Secure LLM Access 🚀
Hey everyone! I’m finally jumping into the dev.to community. To kick things off, I wanted to share two tools I’ve been developing at the University of Jaén that tackle two common headaches in the AI space: running out of VRAM, and keeping your API chats truly private. 🦥 Quansloth: TurboQuant Local AI Server The Problem: Standard LLM inference hits a "Memory Wall" with long documents. As context grows, your GPU runs out of memory (OOM) and crashes. The Solution: Quansloth is a fully private, air-gapped AI server that brings elite KV cache compression to consumer hardware. By bridging a Gradio Python frontend with a highly optimized llama.cpp CUDA backend, it prevents GPU crashes and lets you run massive contexts on a budget. Key Features: 75% VRAM Savings: Based on Google's TurboQuant (ICL
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!