• Currently on my mind: these are some areas I’m probably actively updating
  • Foundations
    • Preconditioning, optimization algorithms
    • Variational methods, probabilistic methods
    • Reparameterization, Gumbel-softmax, STE
  • ML
    • Sparse routing approaches (MoE)
    • Early-fusion multimodal
    • EBMs
• Old notes repo
• AI
  • Foundations
    • Deep learning foundations
      • Diffusion foundations
      • Flow-based, normalizing flow foundations
    • Reinforcement learning foundations
      • Policy gradient derivation
    • Machine learning foundations
      • Classic ML foundations
      • Optimizing sparsity and non-differentiable computations
    • Optimization
      • Preconditioning
    • Monte Carlo inference, probabilistic generative modeling foundations
      • Importance sampling
      • Variational inference
  • AI research
    • Transformer and architecture research
    • Large language models (LLMs)
      • More LLMs
    • Vision and vision-language models
      • OCR systems
    • Image generation
    • Image processing
    • Video generation
    • Audio generation
    • Speech generation
    • Speech processing
    • General deep learning research
    • Flow-based generative models
    • Optimizer research
      • Shampoo optimizer
    • Reasoning
    • Deep/formal reasoning
    • Reinforcement learning research
      • RLHF research
      • Monte Carlo tree search
    • Misc cool generative research
  • Building models (applied ML advice)
  • ML systems research and engineering
    • LLM scaling
    • Modeling compute, FLOPs, transformers
    • LLM scaling algorithms (systems)
    • LLM scaling cases / data
    • Inference performance optimization
    • Accelerator hardware
      • GPU computing
      • TPU computing
    • Megatron-DeepSpeed
    • gpt-neox
      • gpt-neox MoE design doc
    • Megablocks
    • Applied LLMs
  • More specific codebases
    • llama2.c
    • Sweep codebase
  • AI research questions
  • Small models / scaling down LLMs
  • Datasets
  • LLM evals
  • AI organizations
  • Non-LLM deep learning and software engineering
  • LLM applications
    • LLMs for coding
    • Codex
    • Sourcegraph Cody
    • axolotl
    • LLM code snippets
    • Dev scratch journal
    • Applied Stable Diffusion (classic)
• Computer science and engineering
  • Algorithms and data structures
  • Misc data or statistical algorithms
  • Compression algorithms and media codecs
  • Systems
    • Database systems
    • Dataflow systems
    • Computer architecture
    • Operating systems
  • Programming languages
    • Compilers
    • JIT
    • Memory management, garbage collection
  • Information retrieval
  • Sandboxing
    • Server sandboxing: https://yz.mit.edu/posts/server-sandboxing/
    • Browser sandboxing
  • Distributed systems
  • Concepts
  • ‣
• Math
• Physics
• Development
• Software
• Design
• Misc
• Meta

Monte Carlo tree search