Alex's Slip-box

These are my org-mode notes in sort of Zettelkasten style

docker-compose Rails PG Redis Sidekiq Travis-CI

# Instructions

This is a generator for Rails project setup with docker, Postgres, Redis, Sidekiq; with container orchestration using docker-compose. The goal of this is to do everything in Docker including the initial project setup in order to eliminate the need to install anything locally.

  1. Set the :HEADER-ARGS: property for each file with the :tangle path. This is where the generated file will be written. See Using Header Arguments.
  2. Edit the config Association List below. Select the images and app name you want to use.
  3. call org-babel-tangle to generate the files in your chosen directory.
  4. Run the setup script as an entry point to a temp container based on the selected image. The command is generated in a comment in setup.sh
  5. Start the app with docker-compose up

NOTE: this will run the config elisp block below several times. Set the org-confirm-babel-evaluate variable to nil to skip prompting.

(setq config
      '(("ruby-image" . "ruby:3.0")
        ("pg-image" . "postgres:12")
        ("redis-image" . "redis:6")
        ("app-name" . "hotwire")))
(cdr (assoc key config))

# Dockerfile

FROM <<config("ruby-image")>>

RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg -o /root/yarn-pubkey.gpg && apt-key add /root/yarn-pubkey.gpg
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y build-essential locales nodejs yarn

RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

RUN mkdir -p /app
WORKDIR /app

COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5

COPY . ./

EXPOSE 3000
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

# docker-compose

This orchestrates the following containers:

  • web
  • database
  • redis
  • sidekiq

Environment variables are provided via a .env file in the project root. This file is automatically generated by the setup script below.

version: "3.6"

services:
  web:
    tty: true
    stdin_open: true
    build: .
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    volumes:
      - ".:/app"
    ports:
      - "3000:3000"
    links:
      - "database"
      - "redis"
    env_file:
      - ".env"
  database:
    image: <<config("pg-image")>>
    volumes:
      - 'postgres:/var/lib/postgresql/data'
    ports:
      - "5432"
    env_file:
      - ".env"
  redis:
    image: <<config("redis-image")>>
    volumes:
      - "redis:/data"
    ports:
      - "6379"
  sidekiq:
    depends_on:
      - "database"
      - "redis"
    build: .
    volumes:
      - ".:/app"
      - "/app/tmp" # don't mount tmp dir
    command: "bundle exec sidekiq"
    env_file:
      - ".env"

volumes:
  redis:
  postgres:

NOTE: It can be helpful to expose the redis and database ports to the host machine so you can connect to them with tools from the host machine. The example below only exposes port 3000 of the web service to the host.

# Setup the project

At this point we don’t have a Gemfile, Gemfile.lock or any of the Rails’ framework files, etc. But we can still do all the project setup within docker by pulling down the image, mounting the project directory as a volume and running a few commands.

docker run --rm -it -v "$PWD":/app -w /app ruby:3.0 sh setup.sh

# setup.sh

# Usage:
#   docker run --rm -it -v "$PWD":/app -w /app <<config("ruby-image")>> sh setup.sh

# Install dependencies
gem install rails

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg -o /root/yarn-pubkey.gpg && apt-key add /root/yarn-pubkey.gpg

echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list

apt-get update
apt-get install nodejs yarn -y

# Setup rails app
rails new <<config("app-name")>> --database=postgresql

cd <<config("app-name")>>
mv * .*  ../
cd ..
rmdir <<config("app-name")>>

bin/rails webpacker:install

bundle add sidekiq

# Configure database
DBCONFIG=$(cat <<EOF
default: &default
  adapter: postgresql
  encoding: unicode
  host: <%= ENV['POSTGRES_HOST'] %>
  user: <%= ENV['POSTGRES_USER'] %>
  password: <%= ENV['POSTGRES_PASSWORD'] %>
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
development:
  <<: *default
  database: <<config("app-name")>>_development
test:
  <<: *default
  database: <<config("app-name")>>_test
production:
  <<: *default
  database: <<config("app-name")>>_production
  username: <<config("app-name")>>
  password: <%= ENV['SUPER_SECRET_DATABASE_PASSWORD'] %>
EOF
)
echo "$DBCONFIG" > config/database.yml

# Add env vars for development
touch .env
echo "POSTGRES_DB=<<config("app-name")>>_development" >> .env
echo "POSTGRES_USER=postgres" >> .env
echo "POSTGRES_PASSWORD=postgres" >> .env
echo "POSTGRES_HOST=database" >> .env
echo "REDIS_URL=redis://redis:6379/1" >> .env

# Run it

To start it up run:

docker-compose up

Shut it down with:

docker-compose down

See https://github.com/apmiller108/astronomania-api README for example of more commands like debugged with pry and running tests.

# Travis-CI

This is the travis.yml. Most of this is boiler plate from Travis’ docs. The env stuff I had to figure out becuase I am using a .env file to pass config vars into containers with docker-compose. First I had to add the env vars to Travis’ project settings. Then echo them into a .env file for docker-compose to read from.

NOTE: the COMPOSE_VERSION is NOT the docker-compose file version, but the docker-compose release version: Releases · docker/compose · GitHub

NOTE: this file doesn’t get generated. Use it if you want.

language: bash

sudo: required

services:
  - docker

env:
  COMPOSE_VERSION: 1.26.2

before_install:
 - sudo rm /usr/local/bin/docker-compose
 - curl -L https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
 - chmod +x docker-compose
 - sudo mv docker-compose /usr/local/bin
 - docker --version
 - docker-compose --version

script:
  - touch .env
  - echo "POSTGRES_DB=${POSTGRES_DB}" >> .env
  - echo "POSTGRES_USER=${POSTGRES_USER}" >> .env
  - echo "POSTGRES_PASSWORD=${POSTGRES_PASSWORD}" >> .env
  - echo "POSTGRES_HOST=${POSTGRES_HOST}" >> .env
  - echo "NASA_API_KEY=${NASA_API_KEY}" >> .env
  - docker-compose up --detach --build
  - docker ps -a
  - docker-compose exec web bin/rails db:schema:load RAILS_ENV=test
  - docker-compose exec web bundle exec rspec

after_script:
  - docker-compose down
  - rm .env

notifications:
  email: false

# Resources