Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

How are lines of code counted? #840

Answered by spenserblack
jtmr05 asked this question in Q&A
Discussion options

I've noticed that it seems a bit irregular in how it handles line counting. I've tried replicating the behavior using wc and removing blank lines but I can't figure it out! So, what is it?
You must be logged in to vote

We use tokei for not only detecting languages, but getting lines of code for each language. Tokei identifies not only blank lines, but also comments. So the count you're seeing should be specifically code, excluding blank lines and comments.

https://docs.rs/tokei/12.1.2/tokei/struct.Language.html#structfield.code

for (language_name, language) in languages.iter() {
let mut code = language.code;
let has_children = !language.children.is_empty();
if has_children {
for reports in language.children.values() {
for stats in reports.iter().map(|r| r.stats.summarise()) {
code += stats.code;

Replies: 3 comments 5 replies

Comment options

We use tokei for not only detecting languages, but getting lines of code for each language. Tokei identifies not only blank lines, but also comments. So the count you're seeing should be specifically code, excluding blank lines and comments.

https://docs.rs/tokei/12.1.2/tokei/struct.Language.html#structfield.code

for (language_name, language) in languages.iter() {
let mut code = language.code;
let has_children = !language.children.is_empty();
if has_children {
for reports in language.children.values() {
for stats in reports.iter().map(|r| r.stats.summarise()) {
code += stats.code;
}
}
}
if code == 0 {
continue;
}
language_distribution.insert(Language::from(*language_name), code as f64);
}

Also, moving this to the Q&A category 🙂
You must be logged in to vote
1 reply
Comment options

Ah, obviously... Thanks.
Answer selected by jtmr05
Comment options

I notice there is often a difference between running tokei and onefetch when it comes to lines of code (onefetch 2.19.1, tokei 12.1.2).

Is there a reason that could be the case?

For example, on the onefetch repo itself:

✦ ❯ onefetch
 R RR RR Luke Hsiao ~ git version 2.41.0
 R RRRRRRRR R R -------------------------------
 R RR R RRRRRRRRRRRRR R RR Project: onefetch (6 branches, 44 tags)
rR RRR R RRRRRRRRRRRRRRRRR R RRR R Description: Command-line Git information tool
RRR RR RRRRRRRRRRRRRRRRRRRRRRR RRRRR HEAD: 17c1f463454f (main, origin/main)
 RRRRR RRRRRRRRRRRRRRRRRRRRRRRR RRRR Version: 2.18.1
 RRR RRRRRRRRRRRRRRRRRRRRRRRRRRRR RR Created: 4 years ago
 R RRRRRRRRRR= RR = RRRRRRRRRRR Languages:
 RRRRRRRRRRRR= RR = RRRRRRRRRR ●くろまる Rust (97.5 %) ●くろまる Shell (2.1 %)
 RRRRRRRRRRR RR RRRRRRRRRR ●くろまる Makefile (0.4 %)
 RR==RRRRRRRRRRRRRRRRRRRRRR===RR Dependencies: 28 (Cargo)
 RR = ==RRRRRRR RRRRRR== = RR Authors: 54% Ossama Hjaji 1193
 RR = =========== = RR 19% dependabot[bot] 429
 RR R 4% Spenser Black 86
 R R Last change: 4 days ago
 R Contributors: 132
 URL: ssh://git@github.com/o2sh/onefetch.git
 Commits: 2224
 Churn (222): Cargo.lock 69
 Cargo.toml 56
 .../vercel/package.json 25
 Lines of code: 4954
 Size: 2.43 MiB (142 files)
 License: MIT
✦ ❯ tokei
===============================================================================
 Language Files Lines Code Comments Blanks
===============================================================================
 JSON 1 25 25 0 0
 Makefile 1 27 21 0 6
 Markdown 18 1913 0 1353 560
 Shell 5 135 105 6 24
 SVG 3 264 264 0 0
 Tera 1 164 157 0 7
 TOML 5 146 128 2 16
 YAML 2 2926 2922 0 4
-------------------------------------------------------------------------------
 Rust 42 5393 4736 43 614
 |- Markdown 5 106 33 59 14
 (Total) 5499 4769 102 628
===============================================================================
 Total 78 10993 8358 1404 1231
===============================================================================

Note the 4954 vs 8358.
You must be logged in to vote
2 replies
Comment options

Makefile: 21
Shell: 105
Rust: 4769
Markdown embedded in Rust: 59
21ドル + 105 +たす 4769 +たす 59 =わ 4954$

We ignore prose (Markdown) and data (JSON, YAML, TOML) by default. See the -T option in onefetch --help. Some other languages we don't support yet, and would welcome a contribution 😉

Tokei counts Markdown text as "comments." We have an override to count Markdown's "comments" as code, instead, since what tokei counts as comments is in fact significant syntax. This doesn't effect Markdown files, which are prose (again, ignored by default), but it does affect the Markdown doc comments inside Rust files.
Comment options

Makes sense, thanks!
Comment options

is this counting .gitignore-d files? I saw it seems counting my tailwind generated codes.
You must be logged in to vote
2 replies
Comment options

I believe tokei respects common ignore files, so onefetch should as well. If you could confirm ingored files are counted, preferably with a minimal reproduction, we could investigate that.
Comment options

okay my fault (I forgot to remove the bootstrap css from project which microsoft included by default in their blazor template XD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet

AltStyle によって変換されたページ (->オリジナル) /