Caching Dependencies and Directories

These features are also still experimental, please contact us with any questions, issues and feedback.

Cache content can be accessed by pull requests

Do note that cache content will be available to any build on the repository, including Pull Requests. Do exercise caution not to put any sensitive information in the cache, lest a malicious attacker potentially exposes it.

Caching directories (Bundler, dependencies)

With caches, Travis CI can persist directories between builds. This is especially useful for dependencies that need to be downloaded and/or compiled from source.

Build phases

Travis CI attempts to upload cache after script, but before either after_success or after_failure is run. Note that the failure to upload the cache does not mark the job a failure.

Bundler

On Ruby and Objective-C projects, installing dependencies via Bundler can make up a large portion of the build duration. Caching the bundle between builds drastically reduces the time a build takes to run.

The logic for fetching and storing the cache is described below.

Enabling Bundler caching

Bundler caching is automatically enabled for Ruby projects that include a Gemfile.lock.

(Bundler caching is not yet enabled automatically)

You can explicitly enable Bundler caching in your .travis.yml:

language: ruby
cache: bundler

Whenever you update your bundle, Travis CI will also update the cache.

Determining the bundle path

Travis CI tries its best at determining the path bundler uses for storing dependencies.

If you have custom Bundler arguments, and these include the –path option, Travis CI will use that path. If –path is missing but –deployment is present, it will use vendor/bundle.

Otherwise it will automatically add the –path option. In this case it will either use the value of the environment variable BUNDLE_PATH or, if it is missing, vendor/bundle.

CocoaPods

On Objective-C projects, installing dependencies via CocoaPods can take up a good portion of your build. Caching the compiled Pods between builds helps reduce this time.

Enabling CocoaPods caching

You can enable CocoaPods caching for your repository by adding this to your .travis.yml:

language: objective-c
cache: cocoapods

If you want to enable both Bundler caching and CocoaPods caching, you can list them both:

language: objective-c
cache:
  - bundler
  - cocoapods

Note that CocoaPods caching won’t have any effect if you are already vendoring the Pods directory in your Git repository.

Determining the Podfile path

By default, Travis CI will assume that your Podfile is in the root of the repository. If this is not the case, you can specify where the Podfile is like this:

language: objective-c
podfile: path/to/Podfile

pip cache

For caching pip files, use:

language: python

cache: pip

caches $HOME/.cache/pip.

ccache cache

For caching ccache files, use:

language: c # or other C/C++ variants

cache: ccache

caches $HOME/.ccache, and adds /usr/lib/ccache to the front of $PATH.

R package cache

For caching R packages, use:

language: R

cache: packages

This caches $HOME/R/Library, and sets R_LIB_USER=$HOME/R/Library environment variable.

Rust Cargo cache

For caching Cargo packages, use:

language: rust

cache: cargo

This caches $HOME/.cargo and $TRAVIS_BUILD_DIR/target.

Arbitrary directories

You can cache arbitrary directories, such as Gradle, Maven, Composer and npm cache directories, between builds by listing them in your .travis.yml:

cache:
  directories:
  - .autoconf
  - $HOME/.m2

As you can see, it is also possible to use environment variables in the directories.

The logic for fetching and storing the cache is described below.

Things not to cache

The cache’s purpose is to make installing language-specific dependencies easy and fast, so everything related to tools like Bundler, pip, Composer, npm, Gradle, Maven, is what should go into the cache.

Large files that are quick to install but slow to download do not benefit from caching, as they take as long to download from the cache as from the original source:

  • Android SDKs
  • Debian packages
  • JDK packages
  • Compiled binaries

Fetching and storing caches

  • Travis CI fetches the cache for every build, including feature branches and pull requests.
  • There is one cache per branch and language version/ compiler version/ JDK version/ Gemfile location/ etc.
  • Pull requests use the cache of the target of the pull request.
  • If a branch does not have its own cache yet, it uses the master branch cache (unless it is a pull request, see above).
  • Only modifications made to the cached directories from normal pushes are stored.

before_cache phase

When using caches, it may be useful to run command just prior to uploading the new cache archive. For example, the dependency management utility may write log files into the directory you are watching, and you would do well to ignore these.

For this purpose, you can use before_cache phase.

cache:
  directories:
    - $HOME/.cache/pip
⋮
before_cache:
  - rm -f $HOME/.cache/pip/log/debug.log

Failures in this stage does not mark the job a failure.

Clearing Caches

Sometimes you spoil your cache by storing bad data in one of the cached directories.

Caches can also become invalid if language runtimes change and the cache contains native extensions. (This often manifests as segmentation faults.)

You can access caches in one of the two ways. Each method also gives you a means of deleting caches.

  1. On the web https://travis-ci.com/OWNER/REPOSITORY/caches for private repositories or https://travis-ci.org/OWNER/REPOSITORY/caches for public repositories, which is accessible from the Settings menu

    Image of cache UI

  2. With command line client:

travis cache --delete

Running travis cache --delete inside the project directory.

There is also a corresponding API for clearing the cache.

Configuration

Enabling multiple caching features

When you want to enable multiple caching features, you can list them as an array:

cache:
- bundler
- pip

This does not work when caching arbitrary directories. If you want to combine that with other caching modes, you will have to use a hash map:

cache:
  bundler: true
  directories:
  - vendor/something
  - .autoconf

Explicitly disabling caching

You can explicitly disable all caching by setting the cache option to false in your .travis.yml:

cache: false

It is also possible to disable a single caching mode:

cache:
  bundler: false
  pip: true

Setting the timeout

Caching has a timeout set to 5 minutes by default. The timeout is there in order to guard against any issues that may result in a stuck build. Such issues may be caused by a network issue between worker servers and S3 or even by a cache being to big to pack it and upload it in timely fashion. There are, however, situations when you might want to set a bigger timeout, especially if you need to cache large amount. In order to change the timeout you can use the timeout property with a desired time in seconds:

cache:
  timeout: 1000

Caches and build matrices

When you have multiple jobs in a build matrix, some characteristics of each job are used to identify the cache each of the jobs should use.

These factors are:

  1. OS name (currently, linux or osx)
  2. OS distribution (for Linux, precise or trusty)
  3. OS X image name (e.g., xcode7.2)
  4. Names and values of visible environment variables set in .travis.yml or Settings panel
  5. Language runtime version (for the language specified in the language key) if applicable
  6. For Bundler-aware jobs, the name of the Gemfile used

If these characteristics are shared by more than one job in a build matrix, they will share the same URL on the network. This could corrupt the cache, or the cache may contain files that are not usable in all jobs using it. In this case, we advise you to add a defining public environment variable name; e.g.,

CACHE_NAME=JOB1

to .travis.yml.

How does the caching work?

The caching tars up all the directories listed in the configuration and uploads them to S3, using a secure and protected URL, ensuring security and privacy of the uploaded archives.

Note that this makes our cache not network-local, it’s still bound to network bandwidth and DNS resolutions for S3. That impacts what you can and should store in the cache. If you store archives larger than a few hundred megabytes in the cache, it’s unlikely that you’ll see a big speed improvement.

Before the build, we check if a cached archive exists. If it does, we pull it down and unpack it to the specified locations.

After the build we check for changes in the directory, create a new archive and upload the updated archive back to S3.

The upload is currently part of the build cycle, but we’re looking into improving that to happen outside of the build, giving faster build feedback.