Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 5645084

Browse files
tianyizheng02github-actions
and
github-actions
authored
Consolidate loss functions into a single file (TheAlgorithms#10737)
* Consolidate loss functions into single file * updating DIRECTORY.md * Fix typo --------- Co-authored-by: github-actions <${GITHUB_ACTOR}@users.noreply.github.com>
1 parent 52a987e commit 5645084

File tree

8 files changed

+253
-373
lines changed

8 files changed

+253
-373
lines changed

‎DIRECTORY.md‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -549,13 +549,7 @@
549549
* Local Weighted Learning
550550
* [Local Weighted Learning](machine_learning/local_weighted_learning/local_weighted_learning.py)
551551
* [Logistic Regression](machine_learning/logistic_regression.py)
552-
* Loss Functions
553-
* [Binary Cross Entropy](machine_learning/loss_functions/binary_cross_entropy.py)
554-
* [Categorical Cross Entropy](machine_learning/loss_functions/categorical_cross_entropy.py)
555-
* [Hinge Loss](machine_learning/loss_functions/hinge_loss.py)
556-
* [Huber Loss](machine_learning/loss_functions/huber_loss.py)
557-
* [Mean Squared Error](machine_learning/loss_functions/mean_squared_error.py)
558-
* [Mean Squared Logarithmic Error](machine_learning/loss_functions/mean_squared_logarithmic_error.py)
552+
* [Loss Functions](machine_learning/loss_functions.py)
559553
* [Mfcc](machine_learning/mfcc.py)
560554
* [Multilayer Perceptron Classifier](machine_learning/multilayer_perceptron_classifier.py)
561555
* [Polynomial Regression](machine_learning/polynomial_regression.py)

‎machine_learning/loss_functions.py‎

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
import numpy as np
2+
3+
4+
def binary_cross_entropy(
5+
y_true: np.ndarray, y_pred: np.ndarray, epsilon: float = 1e-15
6+
) -> float:
7+
"""
8+
Calculate the mean binary cross-entropy (BCE) loss between true labels and predicted
9+
probabilities.
10+
11+
BCE loss quantifies dissimilarity between true labels (0 or 1) and predicted
12+
probabilities. It's widely used in binary classification tasks.
13+
14+
BCE = -Σ(y_true * ln(y_pred) + (1 - y_true) * ln(1 - y_pred))
15+
16+
Reference: https://en.wikipedia.org/wiki/Cross_entropy
17+
18+
Parameters:
19+
- y_true: True binary labels (0 or 1)
20+
- y_pred: Predicted probabilities for class 1
21+
- epsilon: Small constant to avoid numerical instability
22+
23+
>>> true_labels = np.array([0, 1, 1, 0, 1])
24+
>>> predicted_probs = np.array([0.2, 0.7, 0.9, 0.3, 0.8])
25+
>>> binary_cross_entropy(true_labels, predicted_probs)
26+
0.2529995012327421
27+
>>> true_labels = np.array([0, 1, 1, 0, 1])
28+
>>> predicted_probs = np.array([0.3, 0.8, 0.9, 0.2])
29+
>>> binary_cross_entropy(true_labels, predicted_probs)
30+
Traceback (most recent call last):
31+
...
32+
ValueError: Input arrays must have the same length.
33+
"""
34+
if len(y_true) != len(y_pred):
35+
raise ValueError("Input arrays must have the same length.")
36+
37+
y_pred = np.clip(y_pred, epsilon, 1 - epsilon) # Clip predictions to avoid log(0)
38+
bce_loss = -(y_true * np.log(y_pred) + (1 - y_true) * np.log(1 - y_pred))
39+
return np.mean(bce_loss)
40+
41+
42+
def categorical_cross_entropy(
43+
y_true: np.ndarray, y_pred: np.ndarray, epsilon: float = 1e-15
44+
) -> float:
45+
"""
46+
Calculate categorical cross-entropy (CCE) loss between true class labels and
47+
predicted class probabilities.
48+
49+
CCE = -Σ(y_true * ln(y_pred))
50+
51+
Reference: https://en.wikipedia.org/wiki/Cross_entropy
52+
53+
Parameters:
54+
- y_true: True class labels (one-hot encoded)
55+
- y_pred: Predicted class probabilities
56+
- epsilon: Small constant to avoid numerical instability
57+
58+
>>> true_labels = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])
59+
>>> pred_probs = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1], [0.0, 0.1, 0.9]])
60+
>>> categorical_cross_entropy(true_labels, pred_probs)
61+
0.567395975254385
62+
>>> true_labels = np.array([[1, 0], [0, 1]])
63+
>>> pred_probs = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1]])
64+
>>> categorical_cross_entropy(true_labels, pred_probs)
65+
Traceback (most recent call last):
66+
...
67+
ValueError: Input arrays must have the same shape.
68+
>>> true_labels = np.array([[2, 0, 1], [1, 0, 0]])
69+
>>> pred_probs = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1]])
70+
>>> categorical_cross_entropy(true_labels, pred_probs)
71+
Traceback (most recent call last):
72+
...
73+
ValueError: y_true must be one-hot encoded.
74+
>>> true_labels = np.array([[1, 0, 1], [1, 0, 0]])
75+
>>> pred_probs = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1]])
76+
>>> categorical_cross_entropy(true_labels, pred_probs)
77+
Traceback (most recent call last):
78+
...
79+
ValueError: y_true must be one-hot encoded.
80+
>>> true_labels = np.array([[1, 0, 0], [0, 1, 0]])
81+
>>> pred_probs = np.array([[0.9, 0.1, 0.1], [0.2, 0.7, 0.1]])
82+
>>> categorical_cross_entropy(true_labels, pred_probs)
83+
Traceback (most recent call last):
84+
...
85+
ValueError: Predicted probabilities must sum to approximately 1.
86+
"""
87+
if y_true.shape != y_pred.shape:
88+
raise ValueError("Input arrays must have the same shape.")
89+
90+
if np.any((y_true != 0) & (y_true != 1)) or np.any(y_true.sum(axis=1) != 1):
91+
raise ValueError("y_true must be one-hot encoded.")
92+
93+
if not np.all(np.isclose(np.sum(y_pred, axis=1), 1, rtol=epsilon, atol=epsilon)):
94+
raise ValueError("Predicted probabilities must sum to approximately 1.")
95+
96+
y_pred = np.clip(y_pred, epsilon, 1) # Clip predictions to avoid log(0)
97+
return -np.sum(y_true * np.log(y_pred))
98+
99+
100+
def hinge_loss(y_true: np.ndarray, y_pred: np.ndarray) -> float:
101+
"""
102+
Calculate the mean hinge loss for between true labels and predicted probabilities
103+
for training support vector machines (SVMs).
104+
105+
Hinge loss = max(0, 1 - true * pred)
106+
107+
Reference: https://en.wikipedia.org/wiki/Hinge_loss
108+
109+
Args:
110+
- y_true: actual values (ground truth) encoded as -1 or 1
111+
- y_pred: predicted values
112+
113+
>>> true_labels = np.array([-1, 1, 1, -1, 1])
114+
>>> pred = np.array([-4, -0.3, 0.7, 5, 10])
115+
>>> hinge_loss(true_labels, pred)
116+
1.52
117+
>>> true_labels = np.array([-1, 1, 1, -1, 1, 1])
118+
>>> pred = np.array([-4, -0.3, 0.7, 5, 10])
119+
>>> hinge_loss(true_labels, pred)
120+
Traceback (most recent call last):
121+
...
122+
ValueError: Length of predicted and actual array must be same.
123+
>>> true_labels = np.array([-1, 1, 10, -1, 1])
124+
>>> pred = np.array([-4, -0.3, 0.7, 5, 10])
125+
>>> hinge_loss(true_labels, pred)
126+
Traceback (most recent call last):
127+
...
128+
ValueError: y_true can have values -1 or 1 only.
129+
"""
130+
if len(y_true) != len(y_pred):
131+
raise ValueError("Length of predicted and actual array must be same.")
132+
133+
if np.any((y_true != -1) & (y_true != 1)):
134+
raise ValueError("y_true can have values -1 or 1 only.")
135+
136+
hinge_losses = np.maximum(0, 1.0 - (y_true * y_pred))
137+
return np.mean(hinge_losses)
138+
139+
140+
def huber_loss(y_true: np.ndarray, y_pred: np.ndarray, delta: float) -> float:
141+
"""
142+
Calculate the mean Huber loss between the given ground truth and predicted values.
143+
144+
The Huber loss describes the penalty incurred by an estimation procedure, and it
145+
serves as a measure of accuracy for regression models.
146+
147+
Huber loss =
148+
0.5 * (y_true - y_pred)^2 if |y_true - y_pred| <= delta
149+
delta * |y_true - y_pred| - 0.5 * delta^2 otherwise
150+
151+
Reference: https://en.wikipedia.org/wiki/Huber_loss
152+
153+
Parameters:
154+
- y_true: The true values (ground truth)
155+
- y_pred: The predicted values
156+
157+
>>> true_values = np.array([0.9, 10.0, 2.0, 1.0, 5.2])
158+
>>> predicted_values = np.array([0.8, 2.1, 2.9, 4.2, 5.2])
159+
>>> np.isclose(huber_loss(true_values, predicted_values, 1.0), 2.102)
160+
True
161+
>>> true_labels = np.array([11.0, 21.0, 3.32, 4.0, 5.0])
162+
>>> predicted_probs = np.array([8.3, 20.8, 2.9, 11.2, 5.0])
163+
>>> np.isclose(huber_loss(true_labels, predicted_probs, 1.0), 1.80164)
164+
True
165+
>>> true_labels = np.array([11.0, 21.0, 3.32, 4.0])
166+
>>> predicted_probs = np.array([8.3, 20.8, 2.9, 11.2, 5.0])
167+
>>> huber_loss(true_labels, predicted_probs, 1.0)
168+
Traceback (most recent call last):
169+
...
170+
ValueError: Input arrays must have the same length.
171+
"""
172+
if len(y_true) != len(y_pred):
173+
raise ValueError("Input arrays must have the same length.")
174+
175+
huber_mse = 0.5 * (y_true - y_pred) ** 2
176+
huber_mae = delta * (np.abs(y_true - y_pred) - 0.5 * delta)
177+
return np.where(np.abs(y_true - y_pred) <= delta, huber_mse, huber_mae).mean()
178+
179+
180+
def mean_squared_error(y_true: np.ndarray, y_pred: np.ndarray) -> float:
181+
"""
182+
Calculate the mean squared error (MSE) between ground truth and predicted values.
183+
184+
MSE measures the squared difference between true values and predicted values, and it
185+
serves as a measure of accuracy for regression models.
186+
187+
MSE = (1/n) * Σ(y_true - y_pred)^2
188+
189+
Reference: https://en.wikipedia.org/wiki/Mean_squared_error
190+
191+
Parameters:
192+
- y_true: The true values (ground truth)
193+
- y_pred: The predicted values
194+
195+
>>> true_values = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
196+
>>> predicted_values = np.array([0.8, 2.1, 2.9, 4.2, 5.2])
197+
>>> np.isclose(mean_squared_error(true_values, predicted_values), 0.028)
198+
True
199+
>>> true_labels = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
200+
>>> predicted_probs = np.array([0.3, 0.8, 0.9, 0.2])
201+
>>> mean_squared_error(true_labels, predicted_probs)
202+
Traceback (most recent call last):
203+
...
204+
ValueError: Input arrays must have the same length.
205+
"""
206+
if len(y_true) != len(y_pred):
207+
raise ValueError("Input arrays must have the same length.")
208+
209+
squared_errors = (y_true - y_pred) ** 2
210+
return np.mean(squared_errors)
211+
212+
213+
def mean_squared_logarithmic_error(y_true: np.ndarray, y_pred: np.ndarray) -> float:
214+
"""
215+
Calculate the mean squared logarithmic error (MSLE) between ground truth and
216+
predicted values.
217+
218+
MSLE measures the squared logarithmic difference between true values and predicted
219+
values for regression models. It's particularly useful for dealing with skewed or
220+
large-value data, and it's often used when the relative differences between
221+
predicted and true values are more important than absolute differences.
222+
223+
MSLE = (1/n) * Σ(log(1 + y_true) - log(1 + y_pred))^2
224+
225+
Reference: https://insideaiml.com/blog/MeanSquared-Logarithmic-Error-Loss-1035
226+
227+
Parameters:
228+
- y_true: The true values (ground truth)
229+
- y_pred: The predicted values
230+
231+
>>> true_values = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
232+
>>> predicted_values = np.array([0.8, 2.1, 2.9, 4.2, 5.2])
233+
>>> mean_squared_logarithmic_error(true_values, predicted_values)
234+
0.0030860877925181344
235+
>>> true_labels = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
236+
>>> predicted_probs = np.array([0.3, 0.8, 0.9, 0.2])
237+
>>> mean_squared_logarithmic_error(true_labels, predicted_probs)
238+
Traceback (most recent call last):
239+
...
240+
ValueError: Input arrays must have the same length.
241+
"""
242+
if len(y_true) != len(y_pred):
243+
raise ValueError("Input arrays must have the same length.")
244+
245+
squared_logarithmic_errors = (np.log1p(y_true) - np.log1p(y_pred)) ** 2
246+
return np.mean(squared_logarithmic_errors)
247+
248+
249+
if __name__ == "__main__":
250+
import doctest
251+
252+
doctest.testmod()

‎machine_learning/loss_functions/binary_cross_entropy.py‎

Lines changed: 0 additions & 59 deletions
This file was deleted.

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /