Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Improve build performance on Travis #266

ioquatix started this conversation in Ideas
Discussion options

I've implemented https://travis-ci.org/ioquatix/vips-thumbnail which includes travis tests.

The build time on travis is pretty horrendous. Is there something we can do to improve this? Precompiled binaries? Also, is it possible to install without sudo? That also improves performance a bit.

You must be logged in to vote

Replies: 18 comments

Comment options

I think you need to cache the vips build, so just build the vips lib once, then reuse that binary for all your rvm tests.

https://docs.travis-ci.com/user/migrating-from-legacy/#How-Do-I-Cache-Dependencies-and-Directories%3F

I'll see if I can get the main ruby-vips repo to do this.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 22, 2017
Collaborator Author

Have you done this for ruby-vips?

You must be logged in to vote
0 replies
Comment options

No, but it should be simple.

(edit: stupid kb, posted too soon)

You must be logged in to vote
0 replies
Comment options

It seems to be working, it's reusing the vips binary now:

https://travis-ci.org/jcupitt/ruby-vips/jobs/213787127

https://github.com/jcupitt/ruby-vips/tree/use-travis-containers

What do you think?

You must be logged in to vote
0 replies
Comment options

I've merged to master, and got php-vips doing this as well.

You must be logged in to vote
0 replies
Comment options

Also, I like your thumbnailer. You're right, it should get quicker with vips_thumbnail(), when 8.5 is done.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 22, 2017
Collaborator Author

I like what you did, and I think I improved it a bit. I put all vips specific environment in install file.

https://github.com/ioquatix/vips-thumbnail/blob/master/install-vips.sh

Invoked using

before_script:
 - source install-vips.sh
You must be logged in to vote
0 replies
Comment options

I'm not sure that will work, will it? The gem will need GI_TYPELIB_PATH to run, for example, and it'll need PKG_CONFIG_PATH to build, so they'll need to be in the global env table.

My idea was to share install-vips.sh with php-vips, so I wanted to have all the config in the .travis.yml file. If you have the config there it's easy to add things like test against various vips versions too.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 23, 2017
Collaborator Author

It works.

That's why it use source. It works in the current "shell instance" as it were rather than executing a separate shell.

As much as possible should be in install-vips.sh, otherwise it's cumbersome to manage .travis.yml IMHO.

You must be logged in to vote
0 replies
Comment options

Ah OK, I hadn't noticed the source, duh.

I like having all the config stuff at the top so it's easy to see. And global env vars should be defined in env: global: (imo), it's easy to miss them if they are set elsewhere.

I moved some other stuff into install-vips.sh, does that look a bit cleaner?

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 23, 2017
Collaborator Author

The only thing I'd be inclined to put in .travis.yml would be things which I'd want to vary on the build matrix, so perhaps:

- VIPS_VERSION_MAJOR=8
- VIPS_VERSION_MINOR=4
- VIPS_VERSION_MICRO=5

Everything else should go in install-vips.sh. Keeping .travis.yml as light as possible makes the most sense to me. Think about if you moved to a different CI platform - you'd want to reuse as much as possible. But, in any case, it's up to you.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 23, 2017
Collaborator Author

I thin w.r.t. versions, it would be even better to just write VIPS_VERSION=8.4.5 as that approach is more common IMHO.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 25, 2017
Collaborator Author

I realised something: every single thing that depends on vips, needs to use this script. It's really heavy for travis, and it's really heavy for things which depend on vips.

Specifically, each project needs to:

  • Declare all the appropriate packages/addons (may change which would have a significant maintenance burden).
  • Include the install-vips.sh script and maintain it.
  • The build process is so long that each project needs to have a cache.

What we should do is try to get a decent version of vips added to travis whitelist.

@BanzaiMan just wondering if you are able to provide some advice here. I'll try to kick off the whitelist process. @jcupitt is there a debian/ubuntu package for libvips? Is it up to date?

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 25, 2017
Collaborator Author

Is this the current package for trusty ubuntu? http://packages.ubuntu.com/trusty/libvips-dev - It seems a bit old.

We need to follow the procedure here: https://github.com/travis-ci/apt-package-whitelist

I take it the package we want is libvips-dev?

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 25, 2017
Collaborator Author

There is a more recent package here: https://launchpad.net/ubuntu/+source/vips/8.4.5-1 perhaps we can use a PPA to get the latest versions of the packages available on Travis?

You must be logged in to vote
0 replies
Comment options

Hi, sure, a PPA would be great.

There have been quite a few attempts to get vips whitelisted for travis, I guess you've seen. For example:

travis-ci/apt-source-safelist#68

And many others.

You must be logged in to vote
0 replies
Comment options

ioquatix
Mar 25, 2017
Collaborator Author

That's odd, when I searched for it I didn't get any results. It's almost like there are too many open issues.

There is a PR here: travis-ci/apt-source-safelist#137 but it's a bit out of date now.

You must be logged in to vote
0 replies
Comment options

You could also pupload travis compatible linux binaries to an S3 bucket and use that for builds. The build could either be a tarball unpacked to a local directory with all the dependencies or a deb-package that still requires the dependencies like libpng to be installed in the system.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Ideas
Labels
None yet
Converted from issue

This discussion was converted from issue #100 on January 31, 2021 13:10.

AltStyle によって変換されたページ (->オリジナル) /