Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 09bccec

Browse files
jalveszperazz
andauthored
feat: activation intrinsics for neural networks (#860)
* start working on activations module * softmax for ranks from 1 to 4 * move activations to specialfunctions, add specs * fix float constant definition * update src CMakeLists * add tests for activations * add tests for sigmoid and gelu * missing module procedure * missing interface and change of kind definition for elemental module functions * add SiLU activation * add any rank support for softmax and logsoftmax * add selu activation * Add SELU documentation * add tests * examples * fix relu example * fix tests * improve specs * examples bugfix * replace ifs with merge * add leaky relu activation * Update src/stdlib_specialfunctions.fypp Co-authored-by: Federico Perini <federico.perini@gmail.com> * lowercase procedure names * single shape macro * refactor tanh, add docs, tests on all real precisions * use stdlib_sum * remove unused dim variable --------- Co-authored-by: Federico Perini <federico.perini@gmail.com>
1 parent 64cbb85 commit 09bccec

19 files changed

+2229
-35
lines changed

‎doc/specs/stdlib_specialfunctions_activations.md‎

Lines changed: 846 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
ADD_EXAMPLE(elu)
2+
ADD_EXAMPLE(gaussian)
3+
ADD_EXAMPLE(gelu)
4+
ADD_EXAMPLE(leaky_relu)
5+
ADD_EXAMPLE(relu)
6+
ADD_EXAMPLE(selu)
7+
ADD_EXAMPLE(silu)
8+
ADD_EXAMPLE(softmax)
9+
ADD_EXAMPLE(logsoftmax)
10+
ADD_EXAMPLE(softplus)
11+
ADD_EXAMPLE(step)
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_elu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: elu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = elu( x , 1.0 )
11+
print *, y
12+
end program example_elu
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_gaussian
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: gaussian
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = gaussian( x )
11+
print *, y
12+
end program example_gaussian
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_gelu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: gelu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = gelu( x )
11+
print *, y
12+
end program example_gelu
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_gelu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: leaky_relu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = leaky_relu( x , 0.1_sp )
11+
print *, y
12+
end program example_gelu
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_logsoftmax
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: logsoftmax
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = logsoftmax( x )
11+
print *, y
12+
end program example_logsoftmax
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_relu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: relu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = relu( x )
11+
print *, y
12+
end program example_relu
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_selu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: selu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = selu( x )
11+
print *, y
12+
end program example_selu
13+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
program example_silu
2+
use stdlib_kinds, only: sp
3+
use stdlib_math, only: linspace
4+
use stdlib_specialfunctions, only: silu
5+
implicit none
6+
integer, parameter :: n = 10
7+
real(sp) :: x(n), y(n)
8+
9+
x = linspace(-2._sp, 2._sp, n)
10+
y = silu( x )
11+
print *, y
12+
end program example_silu
13+

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /