In this episode, Vishnu and Alex reflect on Wisecube’s 8-year journey and over 15 years of experience in AI and NLP. They discuss pioneering search engines using TF-IDF to build knowledge graphs (Orpheus), addressing LLM reliability with Pythia, exploring key milestones in AI development, and the evolution of NLP. Topics include the Eliza effect, real-world healthcare and research applications, CAC, drug discovery, and Wisecube's recent acquisition by John Snow Labs. They explore the future of NLP and AI in healthcare.

Alex Thomas's book, "𝗡𝗮𝘁𝘂𝗿𝗮𝗹 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗦𝗽𝗮𝗿𝗸 𝗡𝗟𝗣: 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝘁𝗼 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗧𝗲𝘅𝘁 𝗮𝘁 𝗦𝗰𝗮𝗹𝗲": https://www.amazon.com/Natural-Language-Processing-Spark-NLP/dp/1492047767

Timestamps

00:00 Introduction and Personal Notes

01:13 Wisecube is Now Part of John Snow Labs!

02:15 History and Evolution of NLP

03:27 Early Search Engine Projects

07:55 CAC (Computer-Aided Coding) Healthcare Project

18:05 Drug Discovery Research

28:12 Knowledge Graphs and Orpheus/Pythia Projects

35:51 Future Outlook and Conclusion

Available on:

• ⁠⁠⁠YouTube⁠⁠⁠: https://youtube.com/@WisecubeAI/podcasts

• ⁠⁠⁠Apple Podcast⁠⁠⁠: https://apple.co/4kPMxZf

• ⁠⁠⁠Spotify⁠⁠⁠: https://open.spotify.com/show/1nG58pwg2Dv6oAhCTzab55

• ⁠⁠⁠Amazon Music⁠⁠⁠: https://bit.ly/4izpdO2

Follow us:

- John Snow Labs: https://www.johnsnowlabs.com/?utm_source=acquisition&utm_medium=link&utm_campaign=wisecube

- LinkedIn: https://www.linkedin.com/company/wisecube/

#AI #NLP #LLM #MachineLearning #KnowledgeGraphs #ArtificialIntelligence #DataScience #HealthcareAI #StartupJourney #AIResearch #DrugDiscovery #NaturalLanguageProcessing

In this episode of Gradient Descent, we explore two competing approaches to fine-tuning LLMs: Reinforcement Learning with Human Feedback (RLHF) and Direct Preference Optimization (DPO). Dive into the mechanics of RLHF, its computational challenges, and how DPO simplifies the process by eliminating the need for a separate reward model. We also discuss supervised fine-tuning, emerging methods like Identity Preference Optimization (IPO) and Kahneman-Tversky Optimization (KTO), and their real-world applications in models like Llama 3 and Mistral. Learn practical LLM optimization strategies, including task modularization to boost performance without extensive fine-tuning.

Timestamps:

Intro - 0:00

Overview of LLM Fine-Tuning - 00:48

Deep Dive into RLHF - 02:46

Supervised Fine-Tuning vs. RLHF - 10:38

DPO and Other RLHF Alternatives - 14:43

Real-World Applications in Frontier Models - 22:23

Practical Tips for LLM Optimization - 25:18

Closing Thoughts - 36:05

References:

[1] Training language models to follow instructions with human feedback https://arxiv.org/abs/2203.02155

[2] Direct Preference Optimization: Your Language Model is Secretly a Reward Model https://arxiv.org/abs/2305.18290

[3] Hugging Face Blog on DPO: Simplifying Alignment: From RLHF to Direct Preference Optimization (DPO) https://huggingface.co/blog/ariG23498/rlhf-to-dpo

[4] Comparative Analysis: RLHF and DPO Compared https://crowdworks.blog/en/rlhf-and-dpo-compared/

[5] YouTube Explanation: How to fine-tune LLMs directly without reinforcement learning https://www.youtube.com/watch?v=k2pD3k1485A

Listen on:

• Apple Podcasts:

https://podcasts.apple.com/us/podcast/gradient-descent-podcast-about-ai-and-data/id1801323847

• Spotify:

https://open.spotify.com/show/1nG58pwg2Dv6oAhCTzab55

• Amazon Music:

https://music.amazon.com/podcasts/79f6ed45-ef49-4919-bebc-e746e0afe94c/gradient-descent---podcast-about-ai-and-data

• YouTube: https://youtube.com/@WisecubeAI/podcasts

Our solutions:

- https://askpythia.ai/ - LLM Hallucination Detection Tool

- https://www.wisecube.ai - Wisecube AI platform for large-scale biomedical knowledge analysis

Follow us:

- Pythia Website: https://askpythia.ai/

- Wisecube Website: https://www.wisecube.ai

- LinkedIn: https://www.linkedin.com/company/wisecube/

- Facebook: https://www.facebook.com/wisecubeai

- Twitter: https://x.com/wisecubeai

- Reddit: https://www.reddit.com/r/pythia/

- GitHub: https://github.com/wisecubeai

#FineTuning #LLM #RLHF #AI #MachineLearning #AIDevelopment

Explore the evolution of prompt engineering in this episode of Gradient Descent. Manual prompt tuning — slow, brittle, and hard to scale — is giving way to DSPy, a framework that turns LLM prompting into a structured, programmable, and optimizable process.

Learn how DSPy’s modular approach — with Signatures, Modules, and Optimizers — enables LLMs to tackle complex tasks like multi-hop reasoning and math problem solving, achieving accuracy comparable to much larger models. We also dive into real-world examples, optimization strategies, and why the future of prompting looks a lot more like programming.

Listen to our podcast on these platforms:

• YouTube: https://youtube.com/@WisecubeAI/podcasts

• Apple Podcasts: https://apple.co/4kPMxZf

• Spotify: https://open.spotify.com/show/1nG58pwg2Dv6oAhCTzab55

• Amazon Music: https://bit.ly/4izpdO2

Mentioned Materials:

• DSPy Paper - https://arxiv.org/abs/2310.03714

• DSPy official site - https://dspy.ai/

• DSPy GitHub - https://github.com/stanfordnlp/dspy

• LLM abstractions guide - https://www.twosigma.com/articles/a-guide-to-large-language-model-abstractions/

Our solutions:

- https://askpythia.ai/ - LLM Hallucination Detection Tool

- https://www.wisecube.ai - Wisecube AI platform for large-scale biomedical knowledge analysis

Follow us:

- Pythia Website: https://askpythia.ai/

- Wisecube Website: https://www.wisecube.ai

- LinkedIn: https://www.linkedin.com/company/wisecube/

- Facebook: https://www.facebook.com/wisecubeai

- Twitter: https://x.com/wisecubeai

- Reddit: https://www.reddit.com/r/pythia/

- GitHub: https://github.com/wisecubeai

#AI #PromptEngineering #DSPy #MachineLearning #LLM #ArtificialIntelligence #AIdevelopment