Kiba Pro is a commercial extension for Kiba, a popular Ruby data processing & ETL toolkit.
Its full test suite is a bit heavy, running multiple versions of MySQL (from 5.5 to 8.0) and Postgres (9.3 to 12), across a matrix of Ruby versions. Due to this heaviness, it is only used before issuing releases, or when finalizing a PR, as a QA tool.
I wanted to have a more lightweight setup that I could run on CircleCI:
- At every PR.
- Also once a week, to be notified of potential troubles with dependencies.
I thought such a lightweight setup could end up testing against:
- Only the latest version of MySQL and Postgres (rather than each supported version)
- The very latest version of each rubygem dependency (such as
sequel,concurrent-ruby,pg,mysql2…) - Still, all the currently supported versions of Ruby itself (2.5 to 3.0)
I recently came across @schneems post titled “Migrating a Ruby Library from TravisCI to CircleCI”, and decided to take the plunge. Here is the rundown!
Declaring the build “parameter”
CircleCI provides a way to declare parameters in a build.
I’m using it as a way to vary the Ruby version:
version: 2.1
jobs:
build:
parameters:
ruby_version:
type: string
Adding the Docker images
From there I can add:
- A Ruby image (
cimg/ruby:<< parameters.ruby_version >>) - A single Postgres image (latest).
- A single MySQL image (latest).
jobs:
build:
parameters: # SNIP
# https://circleci.com/docs/2.0/docker-image-tags.json
docker:
# https://circleci.com/developer/images/image/cimg/ruby
- image: cimg/ruby:<< parameters.ruby_version >>
# https://circleci.com/docs/2.0/circleci-images/#postgresql
- image: circleci/postgres:latest
environment:
POSTGRES_USER: some-user
POSTGRES_PASSWORD: some-pass
POSTGRES_DB: some-db
# https://circleci.com/docs/2.0/circleci-images/#mysql
- image: circleci/mysql:latest
command: [--default-authentication-plugin=mysql_native_password]
environment:
MYSQL_DATABASE: some-db
MYSQL_ROOT_PASSWORD: some-pass
Adding the build steps
Then the build can be declared, checking out the code and running the tests against each database (sequentially for now).
It could be parallelized later too if needed, but no need at this point since the tests are decently fast:
version: 2.1
jobs:
build:
parameters: # SNIP
docker: # SNIP
steps:
- checkout
- run: ruby --version
- run: bundle install
- run:
name: Run tests (Postgres)
command: bundle exec rake
environment:
DATABASE_URL: postgres://some-user:some-pass@localhost/some-db
- run:
name: Run tests (MySQL)
command: bundle exec rake
environment:
DATABASE_URL: mysql2://root:some-pass@127.0.0.1/some-db
Creating the “commit” workflow
Once the build is ready, we’ll need to add triggers. The first one commit, will occur on each commit.
I’m extracting a reusable reference (using YAML “anchors”) for the list of Ruby versions, since I want to also test against the same versions in the later weekly workflow:
version: 2.1
jobs: # SNIP
references:
matrix_build: &matrix_build
build:
matrix:
# https://circleci.com/docs/2.0/reusing-config/
parameters:
ruby_version: ["2.5", "2.6", "2.7", "3.0"]
workflows:
version: 2
commit:
jobs:
- <<: *matrix_build
Creating the “weekly” workflow
Similarly, it is nice to have a weekly build to get a heads-up in case of troubles.
CircleCI provides a nice cron syntax for these use-cases:
version: 2.1
jobs: # SNIP
references: # SNIP
workflows:
version: 2
commit: # SNIP
weekly:
triggers:
- schedule:
cron: "0 6 * * 1"
filters:
branches:
only:
- master
jobs:
- <<: *matrix_build
Validating the CircleCI configuration
CircleCI provides a local CLI which you can use to lint or verify the resulting “expanded” build (after iterating over the matrix of versions):
$ circleci config validate config.yml
Config file at config.yml is valid.
$ circleci config process config.yml
The resulting structure is (truncated):
version: 2
jobs:
build-2.5:
docker:
- image: cimg/ruby:2.5
# ...
build-2.6:
docker:
- image: cimg/ruby:2.6
# ...
build-2.7:
docker:
- image: cimg/ruby:2.7
# ...
build-3.0:
docker:
- image: cimg/ruby:3.0
# ...
workflows:
version: 2
commit:
jobs:
- build-2.5
- build-2.6
- build-2.7
- build-3.0
nightly:
triggers:
- schedule:
cron: 0 6 * * 1
filters:
branches:
only:
- master
jobs:
- build-2.5
- build-2.6
- build-2.7
- build-3.0
Locking the PR to ensure the checks
Finally, make sure to go to your repository settings to require the commit workflow success before any merge on the main branch:
That’s it. Hope it was useful!
Thank you for sharing this article!