Large Language Models

• Course cost: $1,400
• Dates: Aug 21-22 
• Location: 2127 Innerbelt Business Center Drive, St. Louis, MO 63114
  • The program will be held in person, with a livestream option if needed.
• Prerequisites: Basic python, familiarity with machine learning concepts.

At the end of this course, students should be able to:

1. Describe the evolution of language models and the significance of LLMs in modern NLP.
2. Understand the architecture and functionality of transformer-based models.
3. Apply prompt engineering techniques for various tasks such as summarization, Q&A, and creative generation.
4. Use libraries and tools like Hugging Face Transformers, LangChain etc. to develop LLM-based solutions.
5. Build a simple Retrieval-Augmented Generation (RAG) pipeline for domain-specific applications.
6. Design and implement a small-scale LLM-powered project demonstrating practical skills learned in the course.

• Introduction to Language Models
  • Evolution of language modeling: N-grams → RNNs → Transformers → LLMs
  • Why LLMs matter: zero-shot, few-shot, and chain-of-thought reasoning
  • Overview of popular models: Open-source (LLaMA, Mistral) vs proprietary (GPT, Claude)
• Tokens and Embeddings
  • What are tokens? Tokenization strategies: Byte Pair Encoding (BPE), WordPiece, SentencePiece
  • From static to contextual embeddings: Word2Vec → BERT → GPT
  • Hands-on: Visualize tokenization and embedding spaces using dimensionality reduction (e.g., t-SNE or UMAP)
• Transformer Model
  • Key components: self-attention, multi-head attention, positional encodings
  • Transformer variants: encoder-only (BERT), decoder-only (GPT), encoder-decoder (T5)
  • Anatomy of a layer: attention heads, feedforward layers, residual connections, layer normalization
  • Hands-on: Explore a simplified transformer architecture
• Prompt Engineering
  • Prompting strategies: zero-shot, few-shot, chain-of-thought, and system-level prompts
  • Instruction tuning vs prompt engineering
  • Hands-on: Design prompt templates for summarization, Q&A, and reasoning
  • Build a prompt-based mini app (e.g., travel planner, story generator) using OpenAI API or open-source LLMs via Hugging Face
• Retrieval-Augmented Generation (RAG)
  • What is RAG? Concept, motivation, and key applications
  • Core components: document chunking, text embeddings, vector stores, retrievers, re-rankers
  • Popular frameworks: LangChain, LlamaIndex, FAISS, Chroma etc.
  • Hands-on: Build a mini RAG system to answer questions from PDF files or Wikipedia articles
• Final Project & Wrap-Up