Applied NLP with LLMs: Beyond Black-Box Monoliths

Resources

A practical guide to human-in-the-loop distillation

https://explosion.ai/blog/human-in-the-loop-distillation

This blog post presents practical solutions for using the latest state-of-the-art models in real-world applications and distilling their knowledge into smaller and faster components that you can run and maintain in-house.

Applied NLP Thinking: How to Translate Problems into Solutions

https://explosion.ai/blog/applied-nlp-thinking

This blog post discusses some of the biggest challenges for applied NLP and translating business problems into machine learning solutions, including the distinction between utility and accuracy.

How S&P Global is making markets more transparent with NLP, spaCy and Prodigy

https://explosion.ai/blog/sp-global-commodities

A case study on S&P Global’s efficient information extraction pipelines for real-time commodities trading insights in a high-security environment using human-in-the-loop distillation.

How GitLab uses spaCy to analyze support tickets and empower their community

https://explosion.ai/blog/gitlab-support-insights

A case study on GitLab’s large-scale NLP pipelines for extracting actionable insights from support tickets and usage questions.

Using LLMs for human-in-the-loop distillation in Prodigy

https://prodi.gy/docs/large-language-models

Prodigy comes with preconfigured workflows for using LLMs to speed up and automate annotation and create datasets for distilling large generative models into more accurate, smaller, faster and fully private task-specific components.

Transcript

• Ines Montani Explosion LLM

• Falcon MIXTRAL GPT-4 LLM

• Falcon MIXTRAL GPT-4 good contextual results LLM

• Pro t ot y pe & Productio n CLOSE THE

GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs 📈 start with evaluation 🔮 assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking How to avoid the prototype plateau?

• Pro t ot y pe & Productio n CLOSE THE

GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs 📈 start with evaluation 🔮 assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking 🛠 work on data iteratively How to avoid the prototype plateau?

• Pro t ot y pe & Productio n CLOSE THE

GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs 📈 start with evaluation 🔮 assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking 💬 consider structure and ambiguity of natural language 🛠 work on data iteratively How to avoid the prototype plateau?

• in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation LLM

• Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr

400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation model size words/second data dev time spacy.fyi/pydata-nyc

• Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr

400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup model size words/second data dev time spacy.fyi/pydata-nyc

• Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr

400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup • beat few-shot LLM baseline of 0.74 with task-specific model model size words/second data dev time spacy.fyi/pydata-nyc

• Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr

400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup • beat few-shot LLM baseline of 0.74 with task-specific model model size words/second data dev time spacy.fyi/pydata-nyc

• Case Stud y : S&P Global 99% 6mb 16k+ 99%

6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment model size words/second F-score explosion.ai/blog/sp-global-commodities

• Case Stud y : S&P Global 99% 6mb 16k+ 99%

6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation model size words/second F-score explosion.ai/blog/sp-global-commodities

• Case Stud y : S&P Global 99% 6mb 16k+ 99%

6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop model size words/second F-score explosion.ai/blog/sp-global-commodities

• Case Stud y : S&P Global 99% 6mb 16k+ 99%

6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production model size words/second F-score explosion.ai/blog/sp-global-commodities

• Case Stud y : S&P Global 99% 6mb 16k+ 99%

6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production model size words/second F-score explosion.ai/blog/sp-global-commodities

• break down larger problems

• break down larger problems make problem easier

• break down larger problems make problem easier reassess dependencies

• Case Stud y : GitLab 1 year 6× 1 year

6× • extract actionable insights from support tickets and usage questions • high-security environment speedup of support tickets explosion.ai/blog/gitlab-support-insights

• Case Stud y : GitLab 1 year 6× 1 year

6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions speedup of support tickets explosion.ai/blog/gitlab-support-insights

• Case Stud y : GitLab 1 year 6× 1 year

6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic speedup of support tickets explosion.ai/blog/gitlab-support-insights

• Case Stud y : GitLab 1 year 6× 1 year

6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic speedup of support tickets explosion.ai/blog/gitlab-support-insights

• Reason and refactor. The key to success lies in your

data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Iterate. The right tooling and mindset gets you past the “prototype plateau”.