Brought to you by:

Eppo—Run reliable, impactful experiments

Stripe—Helping companies of all sizes grow revenue

Vanta—Automate compliance. Simplify security

Sander Schulhoff is the OG prompt engineer. He created the very first prompt engineering guide on the internet (two months before ChatGPT’s release) and recently wrote the most comprehensive study of prompt engineering ever conducted (co-authored with OpenAI, Microsoft, Google, Princeton, and Stanford), analyzing over 1,500 academic papers and covering more than 200 prompting techniques. He also partners with OpenAI to run what was the first and is the largest AI red teaming competition, HackAPrompt, which helps discover the most state-of-the-art prompt injection techniques (i.e. ways to get LLMS to do things it shouldn’t). Sander teaches AI red teaming on Maven, advises AI companies on security, and has educated millions of people on the most state-of-the-art prompt engineering techniques.

In this episode, you’ll learn:

1. The 5 most effective prompt engineering techniques

2. Why “role prompting” and threatening the AI no longer works—and what to do instead

3. The two types of prompt engineering: conversational and product/system prompts

4. A primer on prompt injection and AI red teaming—including real jailbreak tactics that are still fooling top models

5. Why AI agents and robots will be the next major security threat

6. How to get started in AI red teaming and prompt engineering

7. Practical defense to put in place for your AI products

Some takeaways:

1. Prompt engineering is very much alive—and more important than ever. If anything, it’s become more critical as companies rely on LLMs to drive user-facing features and core functionality. Sander explains how prompt quality can make or break AI performance—especially when scaled across products.

2. There are two distinct types of prompt engineering: “conversational” and “product-focused.” Most people think of prompting as chatting with ChatGPT, but Sander explains that real leverage comes from crafting high-performing prompts inside products. These prompts are used at scale, run millions of times, and must be hardened and optimized like production code.

3. “Few-shot prompting” can improve accuracy from 0% to 90%. One of the most powerful techniques is to show the model examples of exactly what you want—called few-shot prompting. Sander shares how this single technique took a medical-coding use case from complete failure to near-perfect output, simply by adding a few example-label pairs.

4. Role prompting (e.g. “You are a math professor. . .”) is largely ineffective, counter to what most people think. Sander breaks down the research showing that while role prompts may help with tone or writing style, they have little to no effect on improving correctness.

5. Advanced techniques like decomposition and self-criticism unlock better performance. Sander outlines how asking a model to first break a problem into sub-problems (decomposition) or critique its own answer (self-criticism) can lead to smarter, more accurate outputs. These are especially valuable in agent-like settings where multi-step reasoning is required.

6. Context (“additional information”) is underrated—and massively impactful. Simply giving the model more relevant background can drastically improve performance. Sander shares examples where including extra data (like bios, research papers, or past interactions) made or broke a prompt, especially when included in the right format and order.

7. Prompt injection is real, dangerous, and unsolvable in the traditional sense. We explore how attackers can “jailbreak” LLMs—tricking them into outputting harmful, restricted, or unintended responses. These attacks often bypass traditional defenses like “do not do X” guardrails. And according to Sander (and even Sam Altman), there’s no silver bullet.

8. Sander runs the world’s largest AI red teaming competition, HackAPrompt. With over 600,000 prompts collected and ongoing collaborations with OpenAI and Anthropic, Sander’s platform is at the center of real-world LLM stress testing. It’s a unique blend of crowd-sourced security and game mechanics—and it’s shaping how labs think about AI safety.

9. Agent-based AI systems are far more vulnerable to attacks than chatbots. Today’s concerns about prompt injection are just the beginning. As AI agents start booking flights, sending emails, and even walking around in humanoid form, the risks multiply. Sander shares why agent security is the next frontier—and why most teams aren’t ready.

10. The “grandma” trick, typos, and obfuscation still break state-of-the-art models. Even the most advanced LLMs can be fooled with surprisingly simple hacks. Sander walks through jailbreak techniques that still work, including emotional manipulation (e.g. “Tell me like my grandma used to”), encoded inputs, and creative phrasing.

11. Most companies are using broken defenses. Sander breaks down why “prompt separation” or adding phrases like “ignore malicious inputs” doesn’t work. Guardrails are easily bypassed, and current classifiers often lack the intelligence to catch encoded attacks. The future of security must be model-level, not bolted on.

12. Despite the risks, the upside of AI is massive and worth pursuing. While Sander takes security seriously, he’s not a doomer. He believes AI will save lives (especially in health care), unlock productivity, and solve real problems—if we build responsibly. Stopping progress isn’t the answer; smarter, safer development is.

Where to find Sander Schulhoff:

• X: https://x.com/sanderschulhoff

• LinkedIn: https://www.linkedin.com/in/sander-schulhoff/

• Website: https://sanderschulhoff.com/

• AI Red Teaming and AI Security Masterclass on Maven: https://bit.ly/44lLSbC

• Free Lightning Lesson “How to Secure Your AI System” on 6/24: https://bit.ly/4ld9vZL

In this episode, we cover:

(00:00) Introduction to Sander Schulhoff

(04:56) The importance of prompt engineering

(09:01) Two modes for thinking about prompt engineering

(12:02) Few-shot prompting

(17:30) Prompting techniques to avoid

(24:52) Decomposition

(28:26) Self-criticism and context

(40:29) Ensembling

(45:59) Thought generation

(48:23) Conversational vs. product-focused prompt engineering

(51:56) Introduction to prompt injection and red teaming

(53:37) AI red teaming competitions

(55:23) The growing importance of AI security

(01:03:39) Techniques to bypass AI safeguards

(01:06:17) Challenges in AI security and future outlook

(01:09:31) Common defenses to prompt injection that don't actually work

(01:13:18) Defenses that do work

(01:16:33) Misalignment and AI's potential risks

(01:19:29) Are LLMs behaving maliciously?

(01:26:05) Final thoughts and lightning round

Referenced:

• Reid Hoffman’s tweet about using AI agents: https://x.com/reidhoffman/status/1930416063616884822

• AI Engineer World’s Fair: https://www.ai.engineer/

• What Is Artificial Social Intelligence?: https://learnprompting.org/blog/asi

• Devin: https://devin.ai/

• Cursor: https://www.cursor.com/

• Inside Devin: The world’s first autonomous AI engineer that’s set to write 50% of its company’s code by end of year | Scott Wu (CEO and co-founder of Cognition): https://www.lennysnewsletter.com/p/inside-devin-scott-wu

• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell

• Granola: https://www.granola.ai/

• Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika

• Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder & CEO of StackBlitz): https://www.lennysnewsletter.com/p/inside-bolt-eric-simons

• Behind the product: Replit | Amjad Masad (co-founder and CEO): https://www.lennysnewsletter.com/p/behind-the-product-replit-amjad-masad

• Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders | Guillermo Rauch (founder and CEO of Vercel, creators of v0 and Next.js): https://www.lennysnewsletter.com/p/everyones-an-engineer-now-guillermo-rauch

• Technique #3: Examples in Prompts: From Zero-Shot to Few-Shot: https://learnprompting.org/docs/basics/few_shot?srsltid=AfmBOor2owyGXtzJZ8n0fJVCctM7UPZgZmH-mBuxRW4t9-kkaMd3LJVv

• The Prompt Report: Insights from the Most Comprehensive Study of Prompting Ever Done: https://learnprompting.org/blog/the_prompt_report?srsltid=AfmBOoo7CRNNCtavzhyLbCMxc0LDmkSUakJ4P8XBaITbE6GXL1i2SvA0

• State-of-the-Art Prompting for AI Agents | Y Combinator: https://www.youtube.com/watch?v=DL82mGde6wo

• Use XML tags to structure your prompts: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags

• Role Prompting: https://learnprompting.org/docs/basics/roles?srsltid=AfmBOor2jcxJQvWBZyFa030Qt0fIIov3hSiWvI9VFyjO-Qp478EPJIU7

• Is Role Prompting Effective?: https://learnprompting.org/blog/role_prompting?srsltid=AfmBOooiiyLD-0CsCYZ4m3SDhYOmtTyaTzeDo0FvK_i1x1gLM8MJS-Sn

• Introduction to Decomposition Prompting Techniques: https://learnprompting.org/docs/advanced/decomposition/introduction?srsltid=AfmBOoojJmTQgBlmSlGYQ8kl-JPpVUlLKkL4YcFGS5u54JyeumUwlcBI

• LLM Self-Evaluation: https://learnprompting.org/docs/reliability/lm_self_eval

• Philip Resnik on X: https://x.com/psresnik

• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next

• Introduction to Ensembling Prompting: https://learnprompting.org/docs/advanced/ensembling/introduction?srsltid=AfmBOooGSyqsrjnEbXSYoKpG0ZlpT278NHQA6Fd8gMvNTJlWu7-qEYzh

• Random forest: https://en.wikipedia.org/wiki/Random_forest

• Chain-of-Thought Prompting: https://learnprompting.org/docs/intermediate/chain_of_thought?srsltid=AfmBOoqwE7SXlluy2sx_QY_VOKduyBplWtIWKEJaD6FkJW3TqeKPSJfx

• Prompt Injecting: https://learnprompting.org/docs/prompt_hacking/injection?srsltid=AfmBOoqGgqbfXStrD6vlw5jy8HhEaESgGo2e57jyWL8lkZKktt_P6Zvn

• Announcing HackAPrompt 2.0: The World’s Largest AI Red-Teaming Hackathon: https://learnprompting.org/blog/announce-hackaprompt-2?srsltid=AfmBOopXKsHxy4aUtsvPCUtEu7x74NCAEnlTIdNzo7nfMDVwZ9ilTlkp

• Infant with rare, incurable disease is first to successfully receive personalized gene therapy treatment: https://www.nih.gov/news-events/news-releases/infant-rare-incurable-disease-first-successfully-receive-personalized-gene-therapy-treatment

• Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf | Varun Mohan (co-founder and CEO): https://www.lennysnewsletter.com/p/the-untold-story-of-windsurf-varun-mohan

• Copilot: https://copilot.microsoft.com/chats/rcxhzvKgZvz8ajUrKdBtX

• GitHub Copilot: https://github.com/features/copilot

• Defensive Measures: https://learnprompting.org/docs/prompt_hacking/defensive_measures/introduction

• Sam Altman on X: https://x.com/sama

• Three Laws of Robotics: https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

• Anthropic’s new AI model turns to blackmail when engineers try to take it offline: https://techcrunch.com/2025/05/22/anthropics-new-ai-model-turns-to-blackmail-when-engineers-try-to-take-it-offline/

• Palisade Research: https://palisaderesearch.org/

• When AI Thinks It Will Lose, It Sometimes Cheats, Study Finds: https://time.com/7259395/ai-chess-cheating-palisade-research/

• A.I. Chatbots Defeated Doctors at Diagnosing Illness: https://www.nytimes.com/2024/11/17/health/chatgpt-ai-doctors-diagnosis.html

• 1883 on Paramount+: https://www.paramountplus.com/shows/1883/

• Black Mirror on Netflix: https://www.netflix.com/title/70264888

• Daylight Computer: https://daylightcomputer.com/

• Theodore Roosevelt’s quote: https://www.goodreads.com/quotes/622252-i-wish-to-preach-not-the-doctrine-of-ignoble-ease

• HackAPrompt 2.0: https://www.hackaprompt.com/

Recommended books:

• Ender’s Game: https://www.amazon.com/Enders-Ender-Quintet-Orson-Scott/dp/0812550706

• The River of Doubt: Theodore Roosevelt’s Darkest Journey: https://www.amazon.com/River-Doubt-Theodore-Roosevelts-Darkest/dp/0767913736

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.