Allow locally installed libraries in sketch profiles. #2930

The PR has no duplicates (please search among the Pull Requests
before creating one)
The PR follows
our contributing guidelines
Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)
UPGRADING.md has been updated with a migration guide (for breaking changes)
configuration.schema.json updated if new parameters are added.

What kind of change does this PR introduce?

This change allows us to specify libraries in the sketch profile using a path to a local directory by prefixing the path with the string dir: .
It also provides this information in compile --dump-profile

What is the current behavior?

Only libraries in the Arduino Libraries Index could be specified in a sketch profile. A typical example may be the following:

profiles:
 uno:
 fqbn: arduino:avr:uno
 platforms:
 - platform: arduino:avr (1.8.6)
 libraries:
 - Adafruit SSD1306 (2.5.13)
 - Adafruit GFX Library (1.11.11)
 - Adafruit BusIO (1.17.0)

What is the new behavior?

A library installed in the filesystem could be specified as well:

profiles:
 uno:
 fqbn: arduino:avr:uno
 platforms:
 - platform: arduino:avr (1.8.6)
 libraries:
 - dir: libraries/MyLib
 - dir: /path/to/library/MyLibOutsideTheSketch 
 - Adafruit SSD1306 (2.5.13)
 - Adafruit GFX Library (1.11.11)
 - Adafruit BusIO (1.17.0)

In this case the two libraries:

libraries/MyLib
/path/to/library/MyLibOutsideTheSketch
will be included during the compilation. In particular, a relative path refers to a path relative to the sketch path, in this case libraries/MyLib is to be intended as SKETCHPATH/libaries/MyLib.

Does this PR introduce a breaking change, and is titled accordingly?

Other information

@cmaglie cmaglie self-assigned this

Jun 10, 2025

@cmaglie cmaglie added type: enhancement topic: build-process labels

Jun 10, 2025

@cmaglie cmaglie force-pushed the allow_paths_in_profile_libs branch from 634d074 to cdd7f4e Compare

June 10, 2025 14:50

@cmaglie cmaglie requested a review from a team

June 10, 2025 15:11

@codecov Codecov

Copy link

codecov bot commented Jun 10, 2025 •

edited

Loading

Codecov Report

❌ Patch coverage is 65.95745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.96%. Comparing base (cf62bee) to head (1a933cb).
⚠️ Report is 29 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/arduino/sketch/profiles.go	45.00%	6 Missing and 5 partials ⚠️
commands/instances.go	68.75%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@
## master #2930 +/- ##
==========================================
+ Coverage 67.91% 67.96% +0.05% 
==========================================
 Files 238 238 
 Lines 22489 22524 +35 
==========================================
+ Hits 15273 15309 +36 
+ Misses 6017 6013 -4 
- Partials 1199 1202 +3

Flag	Coverage Δ
unit	`67.96% <65.95%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

alessio-perugini

alessio-perugini suggested changes

Jun 11, 2025

View reviewed changes

Copy link

Contributor

@alessio-perugini alessio-perugini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation looks good, I'd just add a test to assert the compilation given a sketch.yaml with also relative lib.

internal/integrationtest/sketch/profiles_test.go Show resolved Hide resolved

@cmaglie cmaglie linked an issue

Jun 11, 2025

that may be closed by this pull request

Support for custom libraries when using sketch.yaml #2133

Open

6 tasks

@cmaglie cmaglie mentioned this pull request

Jun 11, 2025

Support for custom libraries when using sketch.yaml #2133

Open

6 tasks

@cmaglie cmaglie force-pushed the allow_paths_in_profile_libs branch from 1d46837 to 0bd806e Compare

June 11, 2025 13:55

alessio-perugini

alessio-perugini approved these changes

Jun 11, 2025

View reviewed changes

@egnor

Copy link

egnor commented Jun 13, 2025

This is amazing! I will finally be able to get rid of my various hacky uses of src/ directory symlinks!

@alessio-perugini

Copy link

Contributor

alessio-perugini commented Jun 13, 2025 •

edited

Loading

@egnor 😄 If you want to give it a try, before we merge this, you can find the test binaries here: https://github.com/arduino/arduino-cli/actions/runs/15586877233
Or you can git-checkout this PR, and run go build .

Feedbacks are always welcomed 🤓

@per1234 per1234 mentioned this pull request

Jun 7, 2025

Support manually installed platforms in build profiles #2927

Closed

3 tasks

cmaglie added 6 commits

July 3, 2025 18:14

@cmaglie


 Allow directories in profile libraries

b35a0d2

@cmaglie


 Updated docs

9dcc9b7

@cmaglie


 Dump custom libraries in profiles with --dump-profile command

65d1029

@cmaglie


 Added integration tests

a0a53ed

@cmaglie


 Fix integration test in Windows

9483f8d

@cmaglie


 Improved integration tests

@cmaglie cmaglie force-pushed the allow_paths_in_profile_libs branch from 0bd806e to 1589119 Compare

July 3, 2025 16:14

@cmaglie


 Fixed flaky intergration test.

1a933cb

@cmaglie cmaglie removed a link to an issue

Jul 3, 2025

Support for custom libraries when using sketch.yaml #2133

Open

6 tasks

@cmaglie cmaglie merged commit 3dca438 into arduino:master

Jul 3, 2025

102 checks passed

@cmaglie cmaglie deleted the allow_paths_in_profile_libs branch

July 3, 2025 16:34

@per1234 per1234 mentioned this pull request

Aug 18, 2025

Panic if sketch has build profile using local library dependency arduino/arduino-lint#905

Open

3 tasks

@mike-lloyd03

Copy link

mike-lloyd03 commented Aug 27, 2025

I cannot get this to work. I'm trying to share some code between two sketches. I have this sketch file:

default_fqbn: esp32:esp32:esp32s3:CDCOnBoot=cdc
profiles:
 main:
 port: /dev/ttyACM1
 platforms:
 - platform: esp32:esp32 (3.2.0)
 libraries:
 - Adafruit BusIO (1.17.2)
 - Adafruit GFX Library (1.12.1)
 - Adafruit ST7735 and ST7789 Library (1.11.0)
 - ESP32Encoder (0.11.7)
 - NimBLE-Arduino (2.3.2)
 - OneButton (2.6.1)
 - dir: ../common

With this file structure:

project_dir/
 common/
 fs.h
 sketch1/
 sketch1.ino
 sketch.yaml
 sketch2/
 sketch2.ino
 sketch.yaml

Inside sketch1.ino:

#include "fs.h"

Compiler throws an error:

$ cd sketch 1
$ arduino-cli compile
project_dir/sketch1/sketch1.ino:1:10: fatal error: fs.h: No such file or directory
 10 | #include "fs.h"
 | ^~~~~~
compilation terminated.

When I run with the --json flag, the common library is absent from the builder_result.used_libraries array. Looks like it's not even being picked up.

@egnor

Copy link

egnor commented Aug 28, 2025 •

edited

Loading

It should probably be generating an error, but I suspect your problem is that the library directory can't just include files, it has to be a proper library in the newer format with a library.properties file and code in the src/ subdirectory. It does work for me fwiw.

@mike-lloyd03

Copy link

mike-lloyd03 commented Aug 28, 2025

I actually tried exactly that. Code in the src directory with a library.properties file in the library root. Same issue. No error and library not included in compilation.

I also passed a non-existent directory in the libraries list and got no error. So it seems like the cli is parsing the file correctly but not doing anything with the directory being included.

@egnor

Copy link

egnor commented Aug 28, 2025 •

edited

Loading

I actually tried exactly that. Code in the src directory with a library.properties file in the library root. Same issue. No error and library not included in compilation.

I also passed a non-existent directory in the libraries list and got no error. So it seems like the cli is parsing the file correctly but not doing anything with the directory being included.

Ok, file a bug and we can debug there instead of commenting on the PR? Ideally include a zip file (or github pointer) for a directory layout that reproduces the issue?

@per1234 per1234 added the topic: code label

Aug 28, 2025

@per1234 per1234 mentioned this pull request

Sep 10, 2025

Incorrect relative local library dependency path resolution when sketch file path used as argument #3005

Open

3 tasks

Labels

topic: build-process topic: code type: enhancement

5 participants

@cmaglie @egnor @alessio-perugini @mike-lloyd03 @per1234

Uh oh!

Allow locally installed libraries in sketch profiles. #2930

Allow locally installed libraries in sketch profiles. #2930

Uh oh!

Conversation

@cmaglie cmaglie commented Jun 10, 2025

Please check if the PR fulfills these requirements

What kind of change does this PR introduce?

What is the current behavior?

What is the new behavior?

Does this PR introduce a breaking change, and is titled accordingly?

Other information

Uh oh!

codecov bot commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

@alessio-perugini alessio-perugini left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

egnor commented Jun 13, 2025

Uh oh!

alessio-perugini commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mike-lloyd03 commented Aug 27, 2025

Uh oh!

egnor commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mike-lloyd03 commented Aug 28, 2025

Uh oh!

egnor commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jun 10, 2025 •

edited

Loading

alessio-perugini commented Jun 13, 2025 •

edited

Loading

egnor commented Aug 28, 2025 •

edited

Loading

egnor commented Aug 28, 2025 •

edited

Loading