Glossary: Statistical Inference and A/B Testing

Type I error

Also written: false positive

Rejecting a null that is actually true, i.e. calling a do-nothing change significant. Alpha is the rate you accept.