improve: more smart conditions of string decoding functions #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

gfx merged 3 commits into master from select_best_function_in_str_decoding

May 29, 2019

Merged

improve: more smart conditions of string decoding functions #39

gfx merged 3 commits into master from select_best_function_in_str_decoding

May 29, 2019

Conversation

gfx

Copy link

Member

@gfx gfx commented May 29, 2019 •

edited

Loading

This is a kind of tragedy that TextDecoder is not always the best way. I wish someday when these complex conditions are completely obsolete, but for now, this is the best.

benchmark results:

$ npx ts-node benchmark/decode-string.ts
## string "A" x 10 (byteLength=10)
utf8DecodeJs x 6,820,213 ops/sec ±5.21% (82 runs sampled)
utf8DecodeWasm x 1,030,221 ops/sec ±3.86% (81 runs sampled)
TextDecoder x 788,986 ops/sec ±2.04% (79 runs sampled)
## string "A" x 100 (byteLength=100)
utf8DecodeJs x 825,191 ops/sec ±1.89% (86 runs sampled)
utf8DecodeWasm x 507,129 ops/sec ±2.98% (81 runs sampled)
TextDecoder x 495,937 ops/sec ±10.66% (66 runs sampled)
## string "A" x 200 (byteLength=200)
utf8DecodeJs x 406,791 ops/sec ±3.24% (80 runs sampled)
utf8DecodeWasm x 339,505 ops/sec ±3.38% (83 runs sampled)
TextDecoder x 485,291 ops/sec ±2.81% (76 runs sampled)
## string "A" x 1000 (byteLength=1000)
utf8DecodeJs x 65,969 ops/sec ±4.95% (69 runs sampled)
utf8DecodeWasm x 83,295 ops/sec ±3.22% (76 runs sampled)
TextDecoder x 260,622 ops/sec ±4.55% (83 runs sampled)
## string "A" x 10000 (byteLength=10000)
utf8DecodeJs x 8,696 ops/sec ±2.89% (81 runs sampled)
utf8DecodeWasm x 9,953 ops/sec ±3.42% (82 runs sampled)
TextDecoder x 45,540 ops/sec ±4.64% (82 runs sampled)
## string "A" x 100000 (byteLength=100000)
utf8DecodeJs x 408 ops/sec ±4.20% (83 runs sampled)
utf8DecodeWasm x 84.42 ops/sec ±3.50% (72 runs sampled)
TextDecoder x 3,443 ops/sec ±4.34% (80 runs sampled)
## string "あ" x 10 (byteLength=30)
utf8DecodeJs x 4,456,073 ops/sec ±4.75% (75 runs sampled)
utf8DecodeWasm x 1,030,237 ops/sec ±3.24% (86 runs sampled)
TextDecoder x 626,129 ops/sec ±5.00% (70 runs sampled)
## string "あ" x 100 (byteLength=300)
utf8DecodeJs x 512,368 ops/sec ±9.75% (75 runs sampled)
utf8DecodeWasm x 423,180 ops/sec ±5.45% (80 runs sampled)
TextDecoder x 390,576 ops/sec ±2.21% (81 runs sampled)
## string "あ" x 200 (byteLength=600)
utf8DecodeJs x 339,130 ops/sec ±0.90% (93 runs sampled)
utf8DecodeWasm x 347,386 ops/sec ±2.08% (90 runs sampled)
TextDecoder x 250,218 ops/sec ±4.65% (77 runs sampled)
## string "あ" x 1000 (byteLength=3000)
utf8DecodeJs x 54,151 ops/sec ±7.31% (75 runs sampled)
utf8DecodeWasm x 71,328 ops/sec ±9.33% (75 runs sampled)
TextDecoder x 62,094 ops/sec ±7.82% (68 runs sampled)
## string "あ" x 10000 (byteLength=30000)
utf8DecodeJs x 6,474 ops/sec ±2.55% (87 runs sampled)
utf8DecodeWasm x 9,338 ops/sec ±1.50% (90 runs sampled)
TextDecoder x 8,195 ops/sec ±2.76% (89 runs sampled)
## string "あ" x 100000 (byteLength=300000)
utf8DecodeJs x 372 ops/sec ±3.82% (80 runs sampled)
utf8DecodeWasm x 85.82 ops/sec ±2.21% (73 runs sampled)
TextDecoder x 895 ops/sec ±1.22% (89 runs sampled)
## string "🌏" x 20 (byteLength=40)
utf8DecodeJs x 2,951,171 ops/sec ±1.22% (91 runs sampled)
utf8DecodeWasm x 732,553 ops/sec ±7.60% (72 runs sampled)
TextDecoder x 511,127 ops/sec ±9.85% (59 runs sampled)
## string "🌏" x 200 (byteLength=400)
utf8DecodeJs x 140,319 ops/sec ±16.40% (39 runs sampled)
utf8DecodeWasm x 255,222 ops/sec ±9.40% (74 runs sampled)
TextDecoder x 320,214 ops/sec ±4.21% (77 runs sampled)
## string "🌏" x 400 (byteLength=800)
utf8DecodeJs x 169,404 ops/sec ±6.21% (83 runs sampled)
utf8DecodeWasm x 176,243 ops/sec ±5.15% (79 runs sampled)
TextDecoder x 177,882 ops/sec ±10.15% (67 runs sampled)
## string "🌏" x 2000 (byteLength=4000)
utf8DecodeJs x 26,572 ops/sec ±6.99% (71 runs sampled)
utf8DecodeWasm x 43,111 ops/sec ±3.76% (80 runs sampled)
TextDecoder x 52,740 ops/sec ±5.80% (75 runs sampled)
## string "🌏" x 20000 (byteLength=40000)
utf8DecodeJs x 3,234 ops/sec ±4.96% (75 runs sampled)
utf8DecodeWasm x 4,254 ops/sec ±4.55% (80 runs sampled)
TextDecoder x 5,502 ops/sec ±3.99% (78 runs sampled)
## string "🌏" x 200000 (byteLength=400000)
utf8DecodeJs x 176 ops/sec ±6.27% (68 runs sampled)
utf8DecodeWasm x 33.61 ops/sec ±6.59% (44 runs sampled)
TextDecoder x 546 ops/sec ±4.38% (79 runs sampled)

@gfx


 improve: more smart selection of string decoding functions

5d85f73

@gfx

Copy link

Member Author

gfx commented May 29, 2019

cc: @sergeyzenchenko

gfx

gfx commented

May 29, 2019

View reviewed changes

benchmark/decode-string.ts

@@ -1,50 +1,45 @@

/* eslint-disable no-console */

import { utf8Encode, utf8Count, utf8Decode } from "../src/utils/utf8";

import { utf8Encode, utf8Count, utf8DecodeJs, utf8DecodeTD } from "../src/utils/utf8";

Copy link

Member Author

@gfx gfx May 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To benchmark them separately, I have split utf8decode functions.

@codecov-io

Copy link

codecov-io commented May 29, 2019 •

edited

Loading

Codecov Report

Merging #39 into master will decrease coverage by 0.21%.
The diff coverage is 83.33%.

Impacted file tree graph

@@ Coverage Diff @@
## master #39 +/- ##
==========================================
- Coverage 95.65% 95.43% -0.22% 
==========================================
 Files 14 14 
 Lines 759 767 +8 
 Branches 160 163 +3 
==========================================
+ Hits 726 732 +6 
- Misses 14 15 +1 
- Partials 19 20 +1

Impacted Files	Coverage Δ
src/wasmFunctions.ts	`93.87% <100%> (+3.4%)`	⬆️
src/Decoder.ts	`98.55% <66.66%> (-0.72%)`	⬇️
src/utils/utf8.ts	`88.23% <80.95%> (-1.42%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f338b2...46f1ec9. Read the comment docs.

gfx added 2 commits

May 29, 2019 10:33

@gfx


 more tests for TextDecoder

a788f19

@gfx


 tewaks for test coverage

46f1ec9

@codecov-io

Copy link

codecov-io commented May 29, 2019 •

edited

Loading

Codecov Report

Merging #39 into master will decrease coverage by 0.21%.
The diff coverage is 83.33%.

Impacted file tree graph

@@ Coverage Diff @@
## master #39 +/- ##
==========================================
- Coverage 95.65% 95.43% -0.22% 
==========================================
 Files 14 14 
 Lines 759 767 +8 
 Branches 160 163 +3 
==========================================
+ Hits 726 732 +6 
- Misses 14 15 +1 
- Partials 19 20 +1

Impacted Files	Coverage Δ
src/wasmFunctions.ts	`93.87% <100%> (+3.4%)`	⬆️
src/Decoder.ts	`98.55% <66.66%> (-0.72%)`	⬇️
src/utils/utf8.ts	`88.23% <80.95%> (-1.42%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f338b2...46f1ec9. Read the comment docs.

@gfx gfx changed the title ~~(削除) improve: more smart selection of string decoding functions (削除ここまで)~~ (追記) improve: more smart conditions of string decoding functions (追記ここまで)

May 29, 2019

@gfx gfx merged commit c2f8f6b into master

May 29, 2019

@gfx gfx deleted the select_best_function_in_str_decoding branch

May 29, 2019 02:11

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

improve: more smart conditions of string decoding functions #39

improve: more smart conditions of string decoding functions #39

Uh oh!

Conversation

@gfx gfx commented May 29, 2019 •

edited

Loading

Uh oh!

Uh oh!

gfx commented May 29, 2019

Uh oh!

@gfx gfx May 29, 2019

Choose a reason for hiding this comment

Uh oh!

codecov-io commented May 29, 2019 •

edited

Loading

Uh oh!

Codecov Report

Uh oh!

codecov-io commented May 29, 2019 •

edited

Loading

Uh oh!

Codecov Report

Uh oh!

Uh oh!

improve: more smart conditions of string decoding functions #39

improve: more smart conditions of string decoding functions #39

Uh oh!

Conversation

@gfx gfx commented May 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfx commented May 29, 2019

Uh oh!

@gfx gfx May 29, 2019

Choose a reason for hiding this comment

Uh oh!

codecov-io commented May 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codecov-io commented May 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

@gfx gfx commented May 29, 2019 •

edited

Loading

codecov-io commented May 29, 2019 •

edited

Loading

codecov-io commented May 29, 2019 •

edited

Loading