Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Proposal: Python API #410

JulianOrteil started this conversation in Ideas
Mar 20, 2022 · 2 comments · 5 replies
Discussion options

@mvp I'm a user of uhubctl for a couple of projects I work on that require USB device integration with this project managing device states. I have to commend you and the other contributors for the work done here. Phenomenal.

Anyways, all of these projects use Python as the core language as it can cover really every aspect I need. That being said, using uhubctl can sometimes be a challenge for a couple of reasons:

  1. Non-native support from Python. To use uhubctl, I have to open a dedicated subprocess to use your CLI and then use some pretty advanced regex to parse the output. As a side-effect, this is a security hole as well.
  2. Lacking JSON output option #124 results in pretty harsh parsing requirements on us users.

These issues are pretty obvious, especially 1, so I'd like to propose a Python wrapper library for uhubctl. I want to fully disclose I have little experience with C and creating Python bindings for C (most of my C-like experience is Arduino), but considering the projects that support Python through C bindings (OpenCV, NumPy, TensorFlow, etc.), doing this is well within the realm of possibility.

I'll have to experiment with this to find the optimal approach, but I thought I'd get your opinion on it.

Hope to hear back,
Jules

You must be logged in to vote

Replies: 2 comments 5 replies

Comment options

There are few ways to address this:

  1. Parse uhubctl output in python wrapper. I don't think it's necessarily difficult or security sensitive. I believe it is possible to do it reliably.
  2. Implement -j (--json) option to support json output, then have python wrapper to call uhubctl appropriately. Parsing will be slightly easier, but will use json module.
  3. Reimplement uhubctl completely in python. Will require pyusb - not a simple requirement because pyusb is not installed by default. Also, some constructs used by uhubctl are not trivial (and maybe impossible) to implement in python.

I believe first option is much easier overall. That said, I would not mind implementing -j, which will need changing command line parsing and output quite extensively (the only reason it was not done just yet).

You must be logged in to vote
4 replies
Comment options

Apologies, I did not put that security comment in the correct spot. Opening and running subprocesses through Python are a security threat, not the parsing of output.

I agree that 3 is likely not possible--I wasn't even considering the use of pyusb because I wanted to avoid having to change much of your current source code. Like how pre-commit has a mypy-mirror project that wraps around mypy to make it compatible with pre-commit.

I'd also like to see how feasible it'd be to call uhubctl natively. This would require some changes to your source code, but nothing notably difficult (likely returning output instead of printing it as you are now).

Comment options

After some experimentation on my end, I have determined that running uhubctl natively from python is possible, but impossible with the current design of the program. I was able to successfully compile uhubctl with Python wrappers and got it to run via main() from Python a few times; however, I occasionally ran into a few issues like the notoriously helpful "segmentation fault (core dumped)".

If we decide to follow this approach, I may need some help with some C-related details to do it right. Regardless, this approach is the most intensive as it would likely require exposing custom functions in uhubctl to make it compatible as an API, not just a standalone program. What these would look like or their requirements are unknowns but can be determined as development progresses.

Overall, I still favor this approach (albeit to a lesser degree lol) as it seems to be the most "holistic". I dislike using subprocesses, especially in a situation like this, as I view them to be "hacky". However, I have no issue using them if the alternative is a truckload of work.

Jules

Comment options

mvp Mar 22, 2022
Maintainer

I understand that you don't like to spawn external process.
That said, using python wrapper seems like really bad idea in this particular case, because it would want to pull full libusb library in.

In terms of portability, spawning external process is safest: you can write python script which will not pull any module dependencies (except for import subprocess, which is standard anyway), and will only require that uhubctl executable is present on the PATH - thats all you need. And if uhubctl is not available in PATH, you can complain back to the user asking them to install missing tool. Any other approach is much more fragile and demanding.

Note that all projects in this list: https://github.com/mvp/uhubctl#notable-projects-using-uhubctl are using uhubctl as external process, and it seems to work for everyone just fine.

Also, even if we implement -j, that will make parsing output quite a bit easier, but it will not remove a need for external process.

Comment options

Sure, why don't we move forward with running this through subprocesses then. Are you leaning towards implementing -j then, or do you want me to move forward with creating a custom parser?

Comment options

Implementing -j will take time and considerable effort. In meantime, I recommend making custom parser, I can help you to write python code for such parser. I guess it should be python class which supports all functionality that uhubctl offers - various filters (-p, -l, -L, -n, -s), options (-f, -e, -N, -r, -R, -w, -e, -d) and actions (-a).

You must be logged in to vote
1 reply
Comment options

That would be most beneficial, yes. You know the output of this project more than anyone.

Because I don't see this library benefitting from maintaining any object states, I believe structuring this similar to numpy will likely be the best approach (mostly global functions implementing standalone code).

So, for example:

import uhubctl
# Examples for commonly-implemented functions
hub_count = uhubctl.get_hub_count()
compatible_hubs = uhubctl.get_compatible_hubs()
# Example of the main implementation function
uhubctl.exec(action=..., location=..., level=..., vendor=..., ports=...)
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Ideas
Labels
None yet
2 participants

AltStyle によって変換されたページ (->オリジナル) /