Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 98f4bdd

Browse files
Add LSE & Softmax functions to Maths category
New source code files added for each function. Meta data for each function was added to maths.ini
1 parent 47be72f commit 98f4bdd

File tree

3 files changed

+55
-0
lines changed

3 files changed

+55
-0
lines changed

‎collection/663.dat

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
function LSE(const A: array of Double): Double;
2+
var
3+
MaxElem: Double;
4+
Elem: Double;
5+
Sum: Double;
6+
begin
7+
if System.Length(A) = 0 then
8+
raise SysUtils.EArgumentException.Create('Empty array');
9+
// Using the centering "trick": see https://rpubs.com/FJRubio/LSE
10+
MaxElem := MaxOfArray(A);
11+
Sum := 0.0;
12+
for Elem in A do
13+
Sum := Sum + System.Exp(Elem - MaxElem);
14+
Result := System.Ln(Sum) + MaxElem;
15+
end;

‎collection/664.dat

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function SoftMax(const A: array of Double): Types.TDoubleDynArray;
2+
var
3+
LSEOfA: Double;
4+
Idx: Integer;
5+
begin
6+
LSEOfA := LSE(A); // raise EArgumentException if A is empty
7+
System.SetLength(Result, System.Length(A));
8+
for Idx := 0 to Pred(System.Length(A)) do
9+
Result[Idx] := System.Exp(A[Idx] - LSEOfA);
10+
end;

‎collection/maths.ini

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1841,3 +1841,33 @@ AdvancedTest.URL="https://github.com/delphidabbler/code-snippets/tree/master/tes
18411841
Snip=662.dat
18421842
DelphiXE=Y
18431843
Delphi12A=Y
1844+
1845+
[LSE]
1846+
DisplayName=LSE
1847+
DescEx="<p>Returns the logarithm of the sum of the exponentials of the given array of floating pointing point numbers.</p><p>An <var>EArgumentException</var> exception is raised if the array is empty.</p>"
1848+
Extra="<p>The mathematical definition of <em>LSE</em> is <em>LSE(x1,...,xn) = log(exp(x1) + ... + exp(xn))</em>. The version of the algorithm used here uses an <a href="https://gregorygundersen.com/blog/2020/02/09/log-sum-exp/">algebraic trick</a> to minimise the risk of overflow.</p><p>For more information see <a href="https://nhigham.com/2021/01/05/what-is-the-log-sum-exp-function/">What Is the Log-Sum-Exp Function?</a> by Nick Higham.</p>"
1849+
Kind=routine
1850+
Units=SysUtils
1851+
Depends=MaxOfArray_Double
1852+
SeeAlso=Softmax
1853+
TestInfo=advanced
1854+
AdvancedTest.Level=unit-tests
1855+
AdvancedTest.URL="https://github.com/delphidabbler/code-snippets/tree/master/tests/Cat-Maths"
1856+
Snip=663.dat
1857+
DelphiXE=Y
1858+
Delphi12A=Y
1859+
1860+
[Softmax]
1861+
DisplayName=Softmax
1862+
DescEx="<p>Applies the <a href="https://en.wikipedia.org/wiki/Softmax_function"><em>softmax</em></a> function to each element of floating point array <var>A</var> and normalizes them into a probability distribution proportional to the exponentials of the elements of <var>A</var>. The normalised values are returned as an array of the same size as <var>A</var>.</p><p>An <var>EArgumentException</var> exception is raised if <var>A</var> is empty.</p>"
1863+
Extra="<p>The <em>softmax</em> function is often used in statistics and in machine learning. It is closely related to the <em>log-sum-exp</em> (aka <em>LSE</em>) function since, for vector <strong>x</strong>=(x1,...,xn), <em>softmax(<strong>x</strong>)=exp[<strong>x</strong>&#8722;LSE(<strong>x</strong>)]</em>.</p>"
1864+
Kind=routine
1865+
Units=SysUtils
1866+
Depends=LSE
1867+
SeeAlso=LSE,MaxOfArray_Double
1868+
TestInfo=advanced
1869+
AdvancedTest.Level=unit-tests
1870+
AdvancedTest.URL="https://github.com/delphidabbler/code-snippets/tree/master/tests/Cat-Maths"
1871+
Snip=664.dat
1872+
DelphiXE=Y
1873+
Delphi12A=Y

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /