Bias Variance Tradeoff

Before we get into bias and variance, it's important to understand the different types of error in machine learning.

• Reducible error 

These errors can be reduced to improve the model's accuracy. This error can be classified as follows:

• Bias
• Variance

• Irreducible error 

The errors that cannot be reduced and will always be present in models

 

Bias and variance explained

 

Bias is defined as the difference between actual and predicted values; a model with a high bias is oversimplified and less complex and doesn't perform better on test or training data.

 

Variance is defined as the amount of variation in prediction if different training data is used; ideally, there should not be much variation when the model is changed. High variance implies that the model was not able to map input and output variables.