Importance sampling: MC sampling may miss rare but important samples. Overrepresent these then correct the overweighting.

• Also need to adjust how variance is calculated.
• IS reduces the sample error/variance.
• Importance = p(x)*g(x)

• Background: MC sampling

• Now the trick: multiply by q/q=1 to change the dist of the expectation

• Hard, very noise estimates due to rare events:

• Easier/more precise with new dist and adjusted func:

• When?

• References
  • https://www.youtube.com/watch?v=C3p2wI4RAi8
  • https://www.youtube.com/watch?v=7A2jXWmnUFw

Importance sampling vs accept-reject (rejection) sampling

• Both let you avoid sampling the actual distribution you care about (maybe because you can’t)
• Rejection sampling discards samples (potentially many), but importance sampling doesn’t—it just reweighs
• However, IS can have high variance in high dimensions