Applying data science to predict fraud becomes an industry standard. But machine learning in fraud has its own specifics compared to other domains: highly imbalanced data, different cost of false positives and false negatives, interaction of ML models with the whole fraud system, etc. In my talk, I will share what to do with such caveats: how to choose the correct metric for model training based on business problems, how to improve models with undersampling methods, what to do in case of real-time fraud detection systems. Also, I will talk about an important business part: how to measure impact of ML models and why sometimes you need to let fraudsters in.