Testing Your Project on Multiple Operating Systems
If your code is used on multiple operating systems it probably should be tested on multiple operating systems. Travis CI can test on Linux and macOS.
To enable testing on multiple operating systems add the os
key to your .travis.yml
:
os:
- linux
- osx
The value of the $TRAVIS_OS_NAME
variable is set to linux
or osx
according to the operating system a particular build is running on, so you can use it to conditionalize your build scripts.
If you are already using a build matrix to test multiple versions, the os
key also multiplies the matrix.
Operating System differences #
When you test your code on multiple operating systems, be aware of differences that can affect your tests:
-
Not all tools may be available on macOS.
We are still working on building up the toolchain on the macOS Environment. Missing software may be available via Homebrew.
-
Language availability.
Not all languages are available on all operating systems, and different versions maybe installed on different systems. Before you embark on the multi-os testing journey, be sure to check this GitHub issue detailing what languages are available.
-
The file system behavior is different.
The HFS+ file system on our macOS workers is case-insensitive (which is the default for macOS), and the files in a directory are returned sorted. On Linux, the file system is case-sensitive, and returns directory entries in the order they appear in the directory internally.
Your tests may implicitly rely on these behaviors, and could fail because of them.
-
They are different operating systems, after all.
Commands may have the same name on the Mac and Linux, but they may have different flags, or the same flag may mean different things. In some cases, commands that do the same thing could have different names. These need to be investigated case by case.
Allowing Failures on Jobs Running on One Operating System #
To ignore the results of jobs on one operating system, add the following
to your .travis.yml
:
jobs:
allow_failures:
- os: osx
Example Multi OS Build Matrix #
Here’s an example .travis.yml
file using if/then directives to customize the build lifecycle to use Graphviz in both Linux and macOS.
language: c
os:
- linux
- osx
compiler:
- gcc
- clang
addons:
apt:
packages:
- graphviz
before_install:
- if [ "$TRAVIS_OS_NAME" = "osx" ]; then brew update ; fi
- if [ "$TRAVIS_OS_NAME" = "osx" ]; then brew install graphviz; fi
script:
- cd src
- make all
There are many options available and using the matrix.include
key is essential to include any specific entries. For example, this matrix would route builds to the Trusty build environment and to a macOS image using Xcode 7.2:
jobs:
include:
-
os: linux
dist: trusty
-
os: osx
osx_image: xcode7.2
Python example (unsupported languages) #
For example, this .travis.yml
uses the matrix.include
key to include four specific entries in the build matrix. It also takes advantage of language: generic
to test Python on macOS. Custom requirements are installed in ./.travis/install.sh
below.
language: python
jobs:
include:
- os: linux
python: 3.2
env: TOXENV=py32
- os: linux
python: 3.3
env: TOXENV=py33
- os: osx
language: generic
env: TOXENV=py32
- os: osx
language: generic
env: TOXENV=py33
install:
- ./.travis/install.sh
script: make test
This custom install script (pseudo code only) uses the $TRAVIS_OS_NAME
and $TOXENV
variables to install (Python) prerequisites specific to macOS, Linux and each specific python version.
#!/bin/bash
if [ $TRAVIS_OS_NAME = 'osx' ]; then
# Install some custom requirements on macOS
# e.g. brew install pyenv-virtualenv
case "${TOXENV}" in
py32)
# Install some custom Python 3.2 requirements on macOS
;;
py33)
# Install some custom Python 3.3 requirements on macOS
;;
esac
else
# Install some custom requirements on Linux
fi
Travis CI then tests the four expanded builds using make test
automatically.