Part 2

The Harness Around the Model

Learn how context, prompts, tools, retrieval, output formats, and safety checks turn a model into a working system. Treat the harness as product architecture, not as a bag of prompt tricks.

61 notesOpen reading · no login

Part 261notes

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What will the “The Harness Around the Model” AI learning path help you do?

Learn how context, prompts, tools, retrieval, output formats, and safety checks turn a model into a working system. Treat the harness as product architecture, not as a bag of prompt tricks. The path contains 61 free notes, each centered on one question you can understand and test.

DECISION RULE

Core themes include Context Engineering, Prompt Engineering, Prompt Security, Agent Engineering.

TRY NEXT

Begin with “Context Window: AI's Working Memory,” then choose the next note by the task in front of you.

WATCH FOR

Do not optimize for finishing the list. Explaining one trade-off with your own example matters more than opening more titles.

What this route helps you practice

Open the first note

Each chapter follows a class of real decisions. Follow the sequence, or enter at the problem you are solving today.

61notes

01Context Window: AI's Working MemoryWindow composition visualization; drag to simulate overflow; compare capacities of mainstream modelsInteractive3 min↗02Context Overflow: Three Handling StrategiesTruncation / summary compression / selective retention — visual comparison of each strategy's trade-offsInteractive3 min↗03Why LLMs Chose MarkdownPlain-text model + formatting needs = MD as the best choice; step-by-step reasoning why HTML / Word / LaTeX don't workAnimation3 min↗04Markdown Syntax & Rendering PipelineCommon syntax cheat sheet + live editor + marked.js / markdown-it rendering solutionsInteractive3 min↗05You Say It, AI Becomes ItFive roles switch in real time; output format control; System Prompt core principlesInteractive3 min↗06Advanced Prompt TechniquesFew-Shot / CoT / constraints / task decomposition — good vs bad interactive comparisonInteractive3 min↗07Output Format Trade-offsPlain text / JSON / Markdown / YAML / XML — scenario fit comparison and trade-offsInteractive3 min↗08Streaming Output & Format PairingJSON needs full text to parse / MD streams char-by-char / XML renders on tag capture — live demoInteractive3 min↗09Prompt Injection: Why Attacks WorkSQL injection analogy → message list essence → lack of parameterization → overview of 5 attack typesSecurity3 min↗10Prompt Injection: 12 Attack CasesPrivilege escalation / role-play / Few-Shot / structural injection / metaphor disguise — vulnerable vs defended versionsSecurity3 min↗11Prompt Defense: Three-Layer InterceptionInput-layer regex → prompt-layer constraints → output-layer leak detection → secondary review; simulate the full attack chainHands-on5 min↗12AI Safety Red Lines: Four BoundariesWhat must not be done, consequences, and the four types of safety boundaries every PM must upholdSecurity3 min↗13Risk Classification & AccountabilityAI output risk classification model, role-based responsibility assignment and governance frameworkSecurity3 min↗14Agent: AI That Gets Things DoneFour capabilities: Plan / Tool / Memory / Act — click to see real-world casesInteractive4 min↗15The Secret of Tool CallingModel outputs JSON → framework parses & executes → result injected back; four-step flow visualizationInteractive3 min↗165 Messages Behind One ConversationDeconstructing the real Function Calling chain: the user sees 1 reply, but 5 API messages happen behind the scenesDeep Dive3 min↗17The Art of Tool DescriptionsSame functionality, but good vs bad descriptions differ by 3× in success rate — a contrast experimentDeep Dive3 min↗18Multi-Tool Orchestration: Parallel vs SerialisConcurrencySafe determines whether tools can run in parallel; scheduling strategy visualizationDeep Dive3 min↗19MCP Protocol: The USB Port for ToolsComparing stdio / SSE / Streamable HTTP transport methods; data-flow animationDeep Dive3 min↗20ReAct in Practice: Weather Query Full ChainThought / Action / Observation step-by-step demo; complete 7-step Agent chainAnimation3 min↗21Short-Term Memory = Context WindowMessage list visualization showing each message's Token usage and role labelDeep Dive3 min↗22Context Compression: Four Defense Lines60% trim → 75% micro-compress → 85% fold → 95% emergency; drag the slider to watch the processDeep Dive3 min↗23Long-Term Memory: Vector RetrievalEmbedding → vector database → semantic search; design decisions for topK and minScoreDeep Dive3 min↗24From Embeddings to MilvusSemantic similarity, ANN, and the responsibility boundary of a vector databaseDeep Dive3 min↗25The Milvus Mental ModelCollection, Schema, Entity, Index, Search, Query, and LoadDeep Dive3 min↗26Hands-on MilvusConnect, create, batch insert, index, search, query, and deleteHands-on3 min↗27From Retrieval to RAGChunking, filtering, hybrid search, RRF, reranking, and evaluationDeep Dive3 min↗28ReAct Loop: Think → Act → ObserveA task to remove console.log goes through 14 iterations, including self-correctionDeep Dive3 min↗295 Patterns of Agent DeadlockParameter format errors / hallucinated tools / infinite recursion / insufficient info / API exceptionsDeep Dive3 min↗30Permissions & Security5 permission modes + LLM risk levels + Human-in-the-loop designDeep Dive3 min↗31Skill: Helping Agents Avoid DetoursSkill = workflow instructions + tool-calling guidance; a "laundry on the balcony" analogy for good loops vs bad loopsInteractive3 min↗32The Essence of SkillGood loop vs bad loop upgraded; how Skill changes an Agent's execution pathDeep Dive3 min↗33Scaffolding: From Prototype to ProductSimulate an Agent booking flights & hotels; full comparison with and without scaffolding; 5 key capabilities explainedInteractive3 min↗345 Engineering GuardrailsIteration cap, output truncation, timeout control, interrupt recovery, context emergencyDeep Dive3 min↗35Multi-Agent CollaborationSub-Agent dispatch, worker threads, parallel vs serial execution strategiesDeep Dive3 min↗36ObservabilityEvent stream visualization, Token tracking, OpenTelemetry integrationDeep Dive3 min↗37Agent Engineering OverviewFrom four capabilities to production deployment — the complete Agent knowledge map on one pageRecap3 min↗38Why Multi-Turn Conversations Get More ExpensiveToken cost accumulation visualization; drag the turn count to see exponential cost growthInteractive3 min↗39KV Cache: Trading Space for Time (and Money)Analogy-based explanation + savings calculator; drag the turn count to see savings ratioInteractive3 min↗40Explicit Caching: A Practical Comparisoncache_control syntax, cache-hit detection, pricing discounts — real savings demonstrationInteractive3 min↗41Dynamic Timestamps: The Most Expensive System PromptBad design vs good design; three timestamp handling approaches side by sideAnti-pattern3 min↗42Comprehensive Cost Optimization: A System-Level Approach5-layer optimization strategy, cost breakdown visualization, system designs that save 70-90%System Design3 min↗43Image Tokens: Pixels Cost Money TooImage billing formula, scaling mechanics, resolution traps, task-based tier strategyMultimodal3 min↗44Match Resolution to TaskHigh / medium / low resolution tiers; Token consumption comparison and selection advice by scenarioMultimodal3 min↗45Syntax-Layer Optimization: Prompts Written for MachinesYAML vs JSON, CSV vs arrays, compressed JSON output — save 10-30% on formatting TokensPrompt Craft3 min↗46Semantic-Layer Optimization: Don't Treat Context as a DumpsterDynamic Few-Shot, LLMLingua-2 compression, key info at start/end — boost information densityRAG3 min↗47Output Layer + Advanced KV CacheNegative constraints, diff-based polishing, stop sequences; KV Cache tool traps and sliding-window issuesArchitecture3 min↗48Model Selection: Capability vs CostMainstream model capability/cost matrix, selection decision tree, model-matching strategies by scenarioSelection3 min↗49Simplicity First: Stick to First PrinciplesThe essence of AI Harness / build vs skip trade-offs / what will be obsoleted / the ultimate questionFinale3 min↗50Human-AI Knowledge Boundary: A Four-Quadrant StrategyWhat to delegate to AI, what to do yourself — a quick four-quadrant task allocation frameworkTips3 min↗51Good Questions vs Bad QuestionsContext determines output quality; same question, good vs bad side-by-side demoTips3 min↗52Can You Trust What AI Says? Spotting HallucinationsThree quick verification methods; spotting when AI confidently gets it wrongTips3 min↗53The Art of Iteration: Knowing When to StopA rough-to-fine iteration rhythm, and when to stop prompting AITips3 min↗54Scenario Cheat Sheet: When to Trust AIHigh confidence / needs verification / use with caution / don't use — a four-tier AI applicability guideTips3 min↗55Recap (Part A) · Prompt Engineering + AgentContext overflow strategies / Prompt six elements / tool calling truth / Skill + scaffoldingRecap3 min↗56Recap (Part B) · Cost Optimization + PM PerspectiveFive-layer cost system / KV Cache principles / image Tokens / full course capability checklistRecap4 min↗57Course Summary · A PM's First AI LessonFrom LLM fundamentals to engineering deployment — the full course on one pageRecap4 min↗58Summary (Part A) · Fundamentals + HarnessLLM cognitive framework / hallucination mitigation / Prompt and Agent essentialsRecap3 min↗59Summary (Part B) · Practice + CostSecurity defense / cost optimization / the right way to use AI / next learning pathRecap4 min↗60Make Five Runs in a Row All UsableSame-input five-run demo; three tasks: write the first prompt, classify the instability, add examples and a ban list then rerun and compareHands-on6 min↗61AI Harness · 30 Tough QuestionsEach with intent, framework, and bonus points: context overflow / Prompt engineering / injection defense / tool calling / cost accounting / KV Cache / output formatsQuiz18 min↗