ImageProcessingBenchmarks

This project is an attempt to benchmark the speed of image processing libraries available in Elixir.

My goal is to find the fastest image processing library to use it in Etaweb to optimize and thumbnail uploaded images by the user.

I quickly focused on the Vix and Image libraries which uses Vips. They are by far the best options in terms of supported features and performances at the moment.

Since these libraries are only interfaces in Elixir to use Vips which is an external image processing software, the memory usage that could be measured with Benchee wouldn't be representative of the real memory usage of Vips. Therefore, I didn't include memory usage results. In my experience, by watching live ressources usage while the benchmark was running, Vips seems to use between 20MB and 60MB depending on the resolution/size of the input image to be thumbnailed.

Installation and usage

• Install dependencies with mix deps.get
• Add any sample image in the ./images/ folder
• run iex -S mix
• run ImageProcessingBenchmarks.run_benchmark()

Results

These results are the one I get on my computer.

HTML Results

Click here to see the results in HTML

Raw results

##### With input cows-goat.JPG | 4592x3448 | 8.48MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 11.79 84.83 ms ±15.26% 87.28 ms 125.80 ms
Image.open |> Image.thumbnail (dynamic shrink) 11.53 86.71 ms ±13.33% 90.61 ms 105.29 ms
Image.thumbnail(path) 1.21 823.65 ms ±3.08% 825.66 ms 859.99 ms
Image.open |> Image.thumbnail (no shrink) 0.71 1407.12 ms ±7.67% 1350.92 ms 1627.90 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 11.79
Image.open |> Image.thumbnail (dynamic shrink) 11.53 - 1.02x slower +1.88 ms
Image.thumbnail(path) 1.21 - 9.71x slower +738.82 ms
Image.open |> Image.thumbnail (no shrink) 0.71 - 16.59x slower +1322.29 ms
##### With input earth.jpg | 8000x8000 | 7.25MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 13.29 75.25 ms ±9.50% 76.15 ms 88.61 ms
Image.open |> Image.thumbnail (dynamic shrink) 13.29 75.25 ms ±10.07% 76.06 ms 89.70 ms
Image.thumbnail(path) 1.12 891.26 ms ±2.68% 893.87 ms 933.23 ms
Image.open |> Image.thumbnail (no shrink) 0.128 7828.43 ms ±11.59% 7828.43 ms 8470.04 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 13.29
Image.open |> Image.thumbnail (dynamic shrink) 13.29 - 1.00x slower +0.00838 ms
Image.thumbnail(path) 1.12 - 11.84x slower +816.02 ms
Image.open |> Image.thumbnail (no shrink) 0.128 - 104.04x slower +7753.19 ms
##### With input forest.jpg | 4048x3032 | 2.66MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 13.36 74.83 ms ±14.21% 77.13 ms 93.83 ms
Image.open |> Image.thumbnail (dynamic shrink) 12.86 77.75 ms ±11.51% 79.60 ms 92.55 ms
Image.thumbnail(path) 2.34 426.67 ms ±2.50% 425.68 ms 442.05 ms
Image.open |> Image.thumbnail (no shrink) 0.84 1191.81 ms ±5.80% 1188.38 ms 1323.72 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 13.36
Image.open |> Image.thumbnail (dynamic shrink) 12.86 - 1.04x slower +2.92 ms
Image.thumbnail(path) 2.34 - 5.70x slower +351.84 ms
Image.open |> Image.thumbnail (no shrink) 0.84 - 15.93x slower +1116.98 ms
##### With input forest.webp | 4048x3032 | 1.29MB #####
Name ips average deviation median 99th %
Image.thumbnail(path) 14.86 67.28 ms ±11.50% 68.28 ms 79.61 ms
Image.open |> Image.thumbnail (shrink CHEAT) 13.78 72.54 ms ±12.56% 73.85 ms 89.38 ms
Image.open |> Image.thumbnail (dynamic shrink) 13.52 73.96 ms ±13.49% 76.15 ms 90.41 ms
Image.open |> Image.thumbnail (no shrink) 0.59 1688.11 ms ±10.35% 1694.46 ms 1966.22 ms
Comparison:
Image.thumbnail(path) 14.86
Image.open |> Image.thumbnail (shrink CHEAT) 13.78 - 1.08x slower +5.27 ms
Image.open |> Image.thumbnail (dynamic shrink) 13.52 - 1.10x slower +6.69 ms
Image.open |> Image.thumbnail (no shrink) 0.59 - 25.09x slower +1620.84 ms
##### With input jupiter.tif | 3298x1856 | 14.92MB #####
Name ips average deviation median 99th %
Image.thumbnail(path) 1.62 619.04 ms ±6.47% 625.87 ms 683.03 ms
Image.open |> Image.thumbnail (no shrink) 1.47 681.47 ms ±4.35% 687.75 ms 735.26 ms
Image.open |> Image.thumbnail (dynamic shrink) 1.45 689.44 ms ±4.31% 689.94 ms 750.65 ms
Image.open |> Image.thumbnail (shrink CHEAT) 1.43 697.46 ms ±4.92% 691.08 ms 793.89 ms
Comparison:
Image.thumbnail(path) 1.62
Image.open |> Image.thumbnail (no shrink) 1.47 - 1.10x slower +62.43 ms
Image.open |> Image.thumbnail (dynamic shrink) 1.45 - 1.11x slower +70.39 ms
Image.open |> Image.thumbnail (shrink CHEAT) 1.43 - 1.13x slower +78.42 ms
##### With input landscape.jpg | 1200x900 | 0.14MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 12.43 80.45 ms ±14.34% 82.15 ms 101.18 ms
Image.open |> Image.thumbnail (no shrink) 12.18 82.09 ms ±15.35% 85.72 ms 103.04 ms
Image.open |> Image.thumbnail (dynamic shrink) 12.08 82.81 ms ±14.67% 84.93 ms 103.23 ms
Image.thumbnail(path) 5.83 171.56 ms ±4.57% 172.04 ms 187.14 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 12.43
Image.open |> Image.thumbnail (no shrink) 12.18 - 1.02x slower +1.64 ms
Image.open |> Image.thumbnail (dynamic shrink) 12.08 - 1.03x slower +2.36 ms
Image.thumbnail(path) 5.83 - 2.13x slower +91.11 ms
##### With input le-faune-carlos-schwabe-1923.jpeg | 1200x930 | 0.76MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 13.08 76.42 ms ±15.36% 80.24 ms 97.24 ms
Image.open |> Image.thumbnail (no shrink) 12.73 78.53 ms ±13.15% 81.22 ms 95.74 ms
Image.open |> Image.thumbnail (dynamic shrink) 12.39 80.69 ms ±13.25% 83.42 ms 97.34 ms
Image.thumbnail(path) 4.60 217.43 ms ±3.04% 217.26 ms 234.09 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 13.08
Image.open |> Image.thumbnail (no shrink) 12.73 - 1.03x slower +2.11 ms
Image.open |> Image.thumbnail (dynamic shrink) 12.39 - 1.06x slower +4.26 ms
Image.thumbnail(path) 4.60 - 2.85x slower +141.01 ms
##### With input milky-way.avif | 3000x1500 | 0.64MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 1.99 502.77 ms ±4.62% 501.31 ms 539.03 ms
Image.open |> Image.thumbnail (dynamic shrink) 1.91 523.62 ms ±3.70% 526.81 ms 554.68 ms
Image.open |> Image.thumbnail (no shrink) 1.90 525.20 ms ±4.60% 528.29 ms 572.30 ms
Image.thumbnail(path) 0.70 1434.69 ms ±1.39% 1441.27 ms 1457.69 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 1.99
Image.open |> Image.thumbnail (dynamic shrink) 1.91 - 1.04x slower +20.85 ms
Image.open |> Image.thumbnail (no shrink) 1.90 - 1.04x slower +22.43 ms
Image.thumbnail(path) 0.70 - 2.85x slower +931.92 ms
##### With input milky-way.jpg | 9725x4862 | 3.84MB #####
Name ips average deviation median 99th %
Image.open |> Image.thumbnail (shrink CHEAT) 12.13 82.45 ms ±13.25% 85.16 ms 97.36 ms
Image.open |> Image.thumbnail (dynamic shrink) 11.79 84.82 ms ±12.66% 87.92 ms 102.79 ms
Image.thumbnail(path) 1.50 667.00 ms ±2.64% 666.50 ms 692.90 ms
Image.open |> Image.thumbnail (no shrink) 0.22 4475.63 ms ±14.35% 4573.64 ms 5063.24 ms
Comparison:
Image.open |> Image.thumbnail (shrink CHEAT) 12.13
Image.open |> Image.thumbnail (dynamic shrink) 11.79 - 1.03x slower +2.37 ms
Image.thumbnail(path) 1.50 - 8.09x slower +584.55 ms
Image.open |> Image.thumbnail (no shrink) 0.22 - 54.28x slower +4393.18 ms

License

GNU AFFERO GENERAL PUBLIC LICENSE version 3