Private Dependencies Bitbucket

Some of the features described here are currently only available for private repositories on travis-ci.com.

When testing a private repository, you might need to pull in other private repositories as dependencies via git submodules, a custom script, or a dependency management tool like Bundler or Composer.

Git submodules must be cloned early on in the build process, and so must use either the Deploy Key or User Key method.

If the dependency is also on Bitbucket, there are several different ways of fetching the repository from within a Travis CI VM. Each one has advantages and disadvantages, so read each method carefully and pick the one that applies best to your situation.

You can use a dedicated CI user account for all but the deploy key approach. This allows you to limit access to a well-defined list of repositories, and make sure that access is read-only.

Deploy Key #

Bitbucket allows you to set up SSH keys for a repository. These deploy keys have some great advantages:

• They are not bound to a user account, so they will not get invalidated by removing users from a repository.
• They do not give access to other, unrelated repositories.
• The same key can be used for dependencies that are not stored on Bitbucket.

However, using deploy keys is complicated by the fact that Bitbucket does not allow you to reuse keys. So, a single private key cannot access multiple Bitbucket repositories.

You could include a different private key for every dependency in the repository, possibly encrypting them. Maintaining complex dependency graphs this way can be complex and hard to maintain. For that reason, we recommend using a repo key instead.

Repo Key #

Travis CI will add a new access key for your repository. It will allow us to read the travis.yml file content.

Password #

Assumptions:

• The repository you are running the builds for is called “myorg/main” and depends on “myorg/lib1” and “myorg/lib2”.
• You know the credentials for a user account that has at least read access to all three repositories.

To pull in dependencies with a password, you will have to use the user name and password in the Git HTTPS URL: https://ci-user:mypassword123@github.com/myorg/lib1.git.

Alternatively, you can also write the credentials to the ~/.netrc file:

machine bitbucket.com
  login ci-user
  password mypassword123

You can also encrypt the password and then write it to the netrc in a before_install step in your .travis.yml:

$ travis env set CI_USER_PASSWORD mypassword123 --private -r myorg/main

before_install:
- echo -e "machine bitbucket.com\n  login ci-user\n  password $CI_USER_PASSWORD" > ~/.netrc

It is also possible to inject the credentials into URLs, for instance, in a Gemfile, it would look like this:

source 'https://rubygems.org'
gemspec

if ENV['CI']
  # use HTTPS with password on Travis CI
  git_source :bitbucket do |repo_name|
    repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
    "https://ci-user:#{ENV.fetch("CI_USER_PASSWORD")}@bitbucket.com/#{repo_name}.git"
  end
end

gem 'lib1', bitbucket: "myorg/lib1"
gem 'lib2', bitbucket: "myorg/lib2"

In case of private git submodules, be aware that the git submodule update --init recursive command runs before the ~/.netrc credentials are updated. If you are writing credentials to ~/.netrc, disable the automatic loading of submodules, update the credentials and add an explicit step to update the submodules:

git:
  submodules: false
before_install:
  - echo -e "machine bitbucket.com\n  login ci-user\n  password $CI_USER_PASSWORD" >~/.netrc
  - git submodule update --init --recursive

API Token #

Assumptions:

• The repository you are running the builds for is called “myorg/main” and depends on “myorg/lib1” and “myorg/lib2”.
• You know the credentials for a user account that has at least read access to all three repositories.

This approach works just like the password approach outlined above, except instead of the username/password pair, you use a Bitbucket API token.

Under the Bitbucket account settings for the user you want to use, navigate to

and then generate a “Personal access tokens”.

Make sure the token has the “repo” scope.

Your ~/.netrc should look like this:

machine bitbucket.com
  login the-generated-token

You can also use it in URLs directly: https://the-generated-token@bitbucket.com/myorg/lib1.git.

Use the encrypt command to add the token to your .travis.yml.

$ travis env set CI_USER_TOKEN the-generated-token --private -r myorg/main

You can then have Travis CI write to the ~/.netrc on every build.

before_install:
- echo -e "machine bitbucket.com\n  login $CI_USER_TOKEN" > ~/.netrc

It is also possible to inject the token into URLs, for instance, in a Gemfile, it would look like this:

source 'https://rubygems.org'
gemspec

if ENV['CI']
  # use HTTPS with token on Travis CI
  git_source :bitbucket do |repo_name|
    repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
    "https://#{ENV.fetch("CI_USER_TOKEN")}@bitbucket.com/#{repo_name}.git"
  end
end

gem 'lib1', bitbucket: "myorg/lib1"
gem 'lib2', bitbucket: "myorg/lib2"

In case of private git submodules, be aware that the git submodule update --init --recursive command runs before the ~/.netrc credentials are updated. If you are writing credentials to ~/.netrc, disable the automatic loading of submodules, update the credentials and add an explicit step to update the submodules:

git:
  submodules: false
before_install:
  - echo -e "\n\nmachine bitbucket.com\n  $CI_TOKEN\n" >~/.netrc
  - git submodule update --init --recursive

Dedicated User Account #

As mentioned a few times, it might make sense to create a dedicated CI user for the following reasons:

• The CI user will only have access to the repositories you want it to have access to.
• You can limit the access to read access.
• Less risk when it comes to leaking keys or credentials.
• The CI user will not leave the organization for non-technical reasons and accidentally break all your builds.

In order to do so, you need to register on Bitbucket as if you would be signing up for a normal user. Registering users cannot be automated, since that would violate the Bitbucket Terms of Service.