A loss function is a function L:(z,y)∈R×Y⟼L(z,y)∈R that takes as inputs the predicted value z corresponding to the real data value y and outputs how different they are.

The common loss functions are summed up in the table below:

Least squared error	Logistic loss	Hinge loss	Cross-entropy
1/2(y−z)2	log(1+exp(−yz))	max(0,1−yz)	−[ylog(z)+(1−y)log(1−z)]
Linear regression	Logistic regression	SVM	Neural Network