Kiba Pro is a commercial extension for Kiba, a popular Ruby data processing & ETL toolkit.
Its full test suite is a bit heavy, running multiple versions of MySQL (from 5.5 to 8.0) and Postgres (9.3 to 12), across a matrix of Ruby versions. Due to this heaviness, it is only used before issuing releases, or when finalizing a PR, as a QA tool.
I wanted to have a more lightweight setup that I could run on CircleCI:
- At every PR.
- Also once a week, to be notified of potential troubles with dependencies.
I thought such a lightweight setup could end up testing against:
- Only the latest version of MySQL and Postgres (rather than each supported version)
- The very latest version of each rubygem dependency (such as
- Still, all the currently supported versions of Ruby itself (2.5 to 3.0)
I recently came across @schneems post titled “Migrating a Ruby Library from TravisCI to CircleCI”, and decided to take the plunge. Here is the rundown!
Declaring the build “parameter”
CircleCI provides a way to declare parameters in a build.
I’m using it as a way to vary the Ruby version:
version: 2.1 jobs: build: parameters: ruby_version: type: string
Adding the Docker images
From there I can add:
- A Ruby image (
cimg/ruby:<< parameters.ruby_version >>)
- A single Postgres image (latest).
- A single MySQL image (latest).
jobs: build: parameters: # SNIP # https://circleci.com/docs/2.0/docker-image-tags.json docker: # https://circleci.com/developer/images/image/cimg/ruby - image: cimg/ruby:<< parameters.ruby_version >> # https://circleci.com/docs/2.0/circleci-images/#postgresql - image: circleci/postgres:latest environment: POSTGRES_USER: some-user POSTGRES_PASSWORD: some-pass POSTGRES_DB: some-db # https://circleci.com/docs/2.0/circleci-images/#mysql - image: circleci/mysql:latest command: [--default-authentication-plugin=mysql_native_password] environment: MYSQL_DATABASE: some-db MYSQL_ROOT_PASSWORD: some-pass
Adding the build steps
Then the build can be declared, checking out the code and running the tests against each database (sequentially for now).
It could be parallelized later too if needed, but no need at this point since the tests are decently fast:
version: 2.1 jobs: build: parameters: # SNIP docker: # SNIP steps: - checkout - run: ruby --version - run: bundle install - run: name: Run tests (Postgres) command: bundle exec rake environment: DATABASE_URL: postgres://some-user:some-pass@localhost/some-db - run: name: Run tests (MySQL) command: bundle exec rake environment: DATABASE_URL: mysql2://root:email@example.com/some-db
Creating the “commit” workflow
Once the build is ready, we’ll need to add triggers. The first one
commit, will occur on each commit.
I’m extracting a reusable reference (using YAML “anchors”) for the list of Ruby versions, since I want to also test against the same versions in the later
version: 2.1 jobs: # SNIP references: matrix_build: &matrix_build build: matrix: # https://circleci.com/docs/2.0/reusing-config/ parameters: ruby_version: ["2.5", "2.6", "2.7", "3.0"] workflows: version: 2 commit: jobs: - <<: *matrix_build
Creating the “weekly” workflow
Similarly, it is nice to have a weekly build to get a heads-up in case of troubles.
CircleCI provides a nice cron syntax for these use-cases:
version: 2.1 jobs: # SNIP references: # SNIP workflows: version: 2 commit: # SNIP weekly: triggers: - schedule: cron: "0 6 * * 1" filters: branches: only: - master jobs: - <<: *matrix_build
Validating the CircleCI configuration
CircleCI provides a local CLI which you can use to lint or verify the resulting “expanded” build (after iterating over the matrix of versions):
$ circleci config validate config.yml Config file at config.yml is valid. $ circleci config process config.yml
The resulting structure is (truncated):
version: 2 jobs: build-2.5: docker: - image: cimg/ruby:2.5 # ... build-2.6: docker: - image: cimg/ruby:2.6 # ... build-2.7: docker: - image: cimg/ruby:2.7 # ... build-3.0: docker: - image: cimg/ruby:3.0 # ... workflows: version: 2 commit: jobs: - build-2.5 - build-2.6 - build-2.7 - build-3.0 nightly: triggers: - schedule: cron: 0 6 * * 1 filters: branches: only: - master jobs: - build-2.5 - build-2.6 - build-2.7 - build-3.0
Locking the PR to ensure the checks
Finally, make sure to go to your repository settings to require the
commit workflow success before any merge on the main branch:
That’s it. Hope it was useful!
Thank you for sharing this article!