Migrate from webdevops/vagrant-docker-vm to codekitchen/dinghy

The missing manual for macOS users who are using Vagrant, Parallels Desktop Pro/Business and the webdevops VM for their local docker-compose projects

Foreword

I assume you’ve worked with webdevops/vagrant-docker-vm in the past in conjunction with Parallels Desktop and docker-compose projects like webdevops/TYPO3-docker-boilerplate. Since the project webdevops/vagrant-docker-vm hasn’t been actively maintained for a couple of years (and there are no active contributors anymore), you should consider using the available drop-in-replacement solution codekitchen/dinghy.

Dinghy allows you to run your existing docker-compose projects without changing anything inside your project files. Dinghy relies on boot2docker/boot2docker and doesn’t use Vagrant anymore. It is based on Docker Machine and the official Docker Machine Parallels driver/plugin. Dinghy uses only Docker Machine to provision the VM. There is also no need for running dnsmasq right in macOS anymore.

Whatto expect from the switch? You can access your docker-compose based projects like before via http://my-lovely-customer.vm/ or http://my-lovely-customer.docker/ or whatever TlD you use in your local development.

Why should someone switch?

As already mentioned, there are no contributors working on the project actively anymore. As long as your existing environment works, you don’t have to switch. But in view of the fact that the provisioning needs some adaptation and the used ubuntu version is outdated, I decided to not invest time in contributing to the webdevops/vagrant-docker-vm project. So the next time you destroy the existing VM (vagrant destroy ), it may happen that the provisioning and DNS resolution inside the VM is no longer working.

A) Requirements

1. OS X 10.10+
2. Parallels Desktop Business or Pro license
3. Homebrew
4. Docker and Docker Machine (not Docker Desktop for Mac):brew install docker docker-machine or a package such as Docker Toolbox
5. The VM provider for Docker Machine provided by Parallels: brew install docker-machine-parallels
6. Reading the instructions and FAQ of codekitchen/dinghy.

B) Backup of databases and application user data in existing projects

You have to backup/extract data of your existing running projects, since we’re not migrating existing docker containers or docker volumes within webdevops/vagrant-docker-vm to the new docker host. In case of TYPO3 projects this will mostly be the database and maybe Apache Solr data.

Since you don’t have to delete the “old” VM you can start it also later unless you delete it. Only the DNS resolving might be broken due to the usage of the /etc/resolver/ configurations by Dinghy.

C) Install and configure codekitchen/dinghy

C.1. General Installation

I’ve already mentioned to read the readme and FAQ of codekitchen/dinghy before following this step by step manual. Let’s install Dinghy:

brew tap codekitchen/dinghy
brew install dinghy

C.2. Add configuration file

Create the dinghy preferences file and fill in the following configuration:mkdir ~/.dinghy && nano ~/.dinghy/preferences.yml

Depending on how you worked with webdevops/vagrant-docker-vm before, you can activate or disable the reverse proxy. If you worked with a .vm domain, you most likely have worked without the http proxy. If you used the .docker domain, you most likely have worked with the dinghy http proxy right in your webdevops/vagrant-docker-vm VM. In other words: if you exposed the ports in your project’s docker-compose.yml files, you didn’t use the HTTP proxy in your previous VM. I’ve added both configuration examples therefore. Decide yourself which one you have/want to use.

The following settings are the ones used on my MacBook Pro Early 2015 3,1GHz Intel Core i7, 16GB DDR3.

• The size of the vm disk is set to 160GB (this was usually the default in webdevops/vagrant-docker-vm)
• The memory is limited to 4GB (¼ of my available memory)
• Half of my CPU cores are used

If you don’t want to automatically provision your files within /etc/resolver with Dinghy you have to set the option :dns_disabled: true in your configuration file. You can safely ignore this hint in 99.9% of the cases.

C.2.1. No HTTP proxy and .vm domain

---
:preferences:
  :proxy_disabled: true
  :fsevents_disabled: false
  :create:
    provider: parallels
    memory: 4096
    cpus: 2
    disk: 160000
  :machine_name: dinghy
  :dinghy_domain: vm
  :unfs_disabled: false
  :dns_disabled: false

C.2.2. No HTTP proxy and .docker domain

Same as C.2.1 except that you have to set :dinghy_domain: docker in the configuration shown above.

---
:preferences:
  :proxy_disabled: true
  :fsevents_disabled: false
  :create:
    provider: parallels
    memory: 4096
    cpus: 2
    disk: 160000
  :machine_name: dinghy
  :dinghy_domain: docker
  :unfs_disabled: false
  :dns_disabled: false

C.2.3. Active HTTP proxy and .docker domain

---
:preferences:
  :proxy_disabled: false
  :fsevents_disabled: false
  :create:
    provider: parallels
    memory: 4096
    cpus: 2
    disk: 20000
  :machine_name: dinghy
  :dinghy_domain: docker
  :unfs_disabled: false
  :dns_disabled: false

C.2.4. Save the file as soon as you’ve finished writing the preferences.yml file

Once you have finished editing ~/.dinghy/preferences.yml in your editor, save and close it. If you used nano (as shown above), press CTRL+O, Return, CTRL+X.

C.3. Add necessary boot2docker ISO URL

(THIS IS ALREADY FIXED IN UPSTREAM! This section is only here for further reference)

There are some upstream incompatibilities as of writing this article. Therefore we strictly have to set the version of boot2docker/boot2docker to 18.06.1-ce.

Important information: Since it looks like the sunset of the entire Docker Machine project has been reached it might be that there will be no fix and therefore no newer boot2docker version available.

Depending on your shell you have to add the environment variable PARALLELS_BOOT2DOCKER_URL to your environment:

# when using bash:
echo 'export PARALLELS_BOOT2DOCKER_URL=https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso' >> ~/.bashrc
# when using zshell:
echo 'export PARALLELS_BOOT2DOCKER_URL=https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso' >> ~/.zshrc

C.4. Configure the NFS mount directory

In this example I assume you have all your docker-compose projects within /Volumes/CS on your macOS host. Dinghy automatically uses two environment variables for the NFS share between your macOS host filesystem and the boot2docker VM. Let’s add them also to your shell environment:

# when using bash:
echo 'export DINGHY_HOST_MOUNT_DIR=/Volumes/CS' >> ~/.bashrc
echo 'export DINGHY_GUEST_MOUNT_DIR=/Volumes/CS' >> ~/.bashrc
# when using zshell:
echo 'export DINGHY_HOST_MOUNT_DIR=/Volumes/CS' >> ~/.zshrc
echo 'export DINGHY_GUEST_MOUNT_DIR=/Volumes/CS' >> ~/.zshrc

If your project files are in your macOS user folder (e.g. ~/Users/projects/..) you can safely skip this step as your macOS user folder is mounted per default.

D) Create and start your Dinghy VM

Make sure you have stopped your running webdevops/vagrant-docker-vm VM first.

If not done yet, please start a new shell or reload the changes in your environment! In case you’re using zshell you can run exec zsh. By running export you can confirm that the newly added environment variables PARALLELS_BOOT2DOCKER_URL, DINGHY_HOST_MOUNT_DIR, … are set. Otherwise Dinghy or Docker Machine uses an incompatible boot2docker ISO version!!!

D.1. Let’s create the VM if you followed all the above steps

Create and run the VM with dinghy create . This will output something like this if all is working as expected:

Creating the dinghy VM...
Running pre-create checks...
Creating machine...
(dinghy) Downloading /Users/myuser/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso...
(dinghy) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
(dinghy) Creating SSH key...
(dinghy) Creating Parallels Desktop VM...
(dinghy) Starting Parallels Desktop VM...
(dinghy) Waiting for VM to come online...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env dinghy
Starting NFS daemon, this will require sudo
Waiting for NFS daemon...
Mounting NFS /Volumes/CS
Starting the FsEvents daemon
Starting DNS
setting up DNS resolution, this will require sudo
Password:
5ab6b62010897562341234123123232e77289e20622342342c23ee312ee
   VM: running
  NFS: running
 FSEV: running
  DNS: running
PROXY: stopped
To connect the Docker client to the Docker daemon, please set these environment variables.
You can set them by running:
    eval $(dinghy env)
It's recommended to add this to your shell config such as ~/.bash_profile
export DOCKER_HOST=tcp://localIp.of.the.vm:2376
    export DOCKER_CERT_PATH=/Users/myuser/.docker/machine/machines/dinghy
    export DOCKER_TLS_VERIFY=1
    export DOCKER_MACHINE_NAME=dinghy

D.2. Follow the instructions described in the output

You have to run the command eval $(dinghy env) so your docker and docker-compose commands can communicate with the docker host inside the just created VM.

Now check by executing docker ps . Depending on your dinghy preferences.yml file one running docker container codekitchen/dinghy-http-proxy is running:

CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS              PORTS                                         NAMES
5aab6b620108        codekitchen/dinghy-http-proxy:2.5   "/app/docker-entrypo…"   9 minutes ago       Up 9 minutes        80/tcp, 19322/tcp, 0.0.0.0:19322->19322/udp   dinghy_http_proxy

Your Docker host is running! Be happy!

D.2.1. Automatically set necessary docker environment variables

The necessary docker environment variables are gone if you reboot your Mac or opening a new shell. Like it’s described you have to add the command eval $(dinghy env) to your shell config:

# when using bash:
echo 'eval $(dinghy env)' >> ~/.bashrc
# when using zshell:
echo 'eval $(dinghy env)' >> ~/.zshrc

D.2.2. Add NFS specific sudo commands to sudoers

You have to use sudo whenever the NFS server gets started, stopped or configured by the dinghy command. You can allow running the commands dinghy up or dinghy halt without the need of your password to reduce the amount of constantly entering your macOS user password.

1. Add Cmnd_Alias DINGHY_NFS = /usr/local/bin/dinghy nfs * to the Cmnd_Alias section of your sudoers file (HowTo edit sudoers file under macOS).
2. Now assign the newly added Cmnd_Alias DINGHY_NFS to your macOS user or every system administrator like the following line: %admin ALL=(root) NOPASSWD: DINGHY_NFS. (Just add the new Cmnd_Alias to the end of the list if there are already other command aliases defined for the admin group: %admin ALL=(root) NOPASSWD: VAGRANT_EXPORTS_ADD, VAGRANT_NFSD, VAGRANT_EXPORTS_REMOVE, VAGRANT_HOSTSUPDATER_ADD, VAGRANT_HOSTSUPDATER_REMOVE, VAGRANT_HOSTMANAGER_UPDATE, ADMIN_TOOLS_HTOP, WIFI_DISCONNECT, DINGHY_NFS)

D.3. Dinghy CLI usage

D.3.1. On overview of dinghy commands:

  dinghy create          # create the docker-machine VM
  dinghy destroy         # stop and delete all traces of the VM
  dinghy halt            # stop the VM and services
  dinghy help [COMMAND]  # Describe available commands or one specific command
  dinghy ip              # get the VM's IP address
  dinghy restart         # restart the VM and services
  dinghy shellinit       # returns env variables to set, should be run like $(dinghy shellinit)
  dinghy ssh [args...]   # ssh to the VM
  dinghy status          # get VM and services status
  dinghy up              # start the Docker VM and services
  dinghy version         # display dinghy version

D.3.2. What commands are you using from day to day?

You can run dinghy commands from everywhere in your shell environment.

vagrant up is now dinghy up (which you do probably when starting the day)

vagrant halt is now dinghy halt (which you do probably when shutting down your Mac)

vagrant ssh or ssh vm is now dinghy ssh

vagrant status is now dinghy status (with more informations about DNS, …)

E) Optional: cleanup and delete webdevops/vagrant-docker-vm related configurations, apps and the VM

Based on your previous usage it may differ what features and binaries you’ve installed on your macOS machine.

E.1. Destroy the webdevops/vagrant-docker-vm virtual machine and user data

1. Change your current directory to the project (e.g. cd /Volumes/CS/devtools/vagrant-docker-vm where the existing Vagrantfile is located)
2. Run vagrant destroy to delete the VM
3. Since databases, … are saved within a sparse bundle you mostly have to run git submodule foreach --recursive git clean -d -f -f -x to clear the “external” disk of the deleted VM. (You can skip this step of you completely delete the folder (step 4)
4. Completely delete the folder: cd .. && rm -rf /Volumes/CS/devtools/vagrant-docker-vm
5. Remove the vagrant box vagrant box remove bento/ubuntu-16.04 --all. Alternatively use the following command to delete each local vagrant base box (not only the latest one used by webdevops/vagrant-docker-vm):

# delete each locally existing vagrant base image (useful if you don't use vagrant anymore on your development machine)
vagrant box list | cut -f 1 -d ' ' | xargs -L 1 vagrant box remove -f

E.2. Remove dnsmasq service

1. Run brew uninstall dnsmasq to remove the local DNS server if you don’t use it for any other purpose than the removed webdevops/vagrant-docker-vm.
2. Delete the configuration file rm -f/usr/local/etc/dnsmasq.conf.

E.3. Remove not used SSH host vm in you ~/.ssh/config file

You can safely delete the following host configuration.

Host vm vagrant 192.168.56.2
     Hostname 192.168.56.2
     User     vagrant
     ForwardAgent  yes
     Compression   no
     StrictHostKeyChecking no
     UserKnownHostsFile=/dev/null

E.4. What else can be removed/uninstalled?

E.4.1. Vagrant related tools:

• Vagrant https://www.vagrantup.com/docs/installation/uninstallation.html
• VagrantManager
• vagrant-completion brew uninstall vagrant-completion

E.4.2. Vagrant related command alias configuration in your visudo configuration can be deleted:

The command alias configurations

 Cmnd_Alias VAGRANT_EXPORTS_ADD = /usr/bin/tee -a /etc/exports
 Cmnd_Alias VAGRANT_NFSD = /sbin/nfsd restart
 Cmnd_Alias VAGRANT_EXPORTS_REMOVE = /usr/bin/sed -E -e /*/ d -ibak /etc/exports
 Cmnd_Alias VAGRANT_HOSTSUPDATER_ADD = /bin/sh -c echo "*" >> /etc/hosts
 Cmnd_Alias VAGRANT_HOSTSUPDATER_REMOVE = /usr/bin/sed -i -e /*/ d /etc/hosts
 Cmnd_Alias VAGRANT_HOSTMANAGER_UPDATE = /bin/cp /Users/jglatz/.vagrant.d/tmp/hosts.local /etc/hosts

and the related user specifications:

# example before cleanup
%admin ALL=(root) NOPASSWD: VAGRANT_EXPORTS_ADD, VAGRANT_NFSD, VAGRANT_EXPORTS_REMOVE, VAGRANT_HOSTSUPDATER_ADD, VAGRANT_HOSTSUPDATER_REMOVE, VAGRANT_HOSTMANAGER_UPDATE, ADMIN_TOOLS_HTOP, WIFI_DISCONNECT, DINGHY_NFS
#after cleanup
%admin ALL=(root) NOPASSWD: ADMIN_TOOLS_HTOP, WIFI_DISCONNECT, DINGHY_NFS

Just delete each entry beginning with VAGRANT_.

Please be careful when working withvisudo. If you are not sure, do not change anything. It’s not that bad if you leave the entries.

Final outlook: switch to DDEV, Docksal or Lando

Since the Docker Machine project is officially only in maintenance mode and boot2docker is also not maintained anymore you have to stick to Docker Desktop for Mac or Docker Desktop for Windows or Native Docker under Linux.

There are also some nice looking projects like, DDEV, Docksal and Lando which aim to simplify working with Docker and adding an ecosystem around your web application like TYPO3, Drupal, Neos, … The use of such tools minimizes your own workload enormously once you have looked through. You can concentrate on your web projects.

My main reason why I’m not sticking to any of the three tools mentioned primary is the performance only when writing this article. Using a commercial Parallels license with a docker host VM is still faster compared to Docker Desktop for Mac when writing this article.

Another valid reason is the usage of own docker images. If you are using or plan to use own images in development and production it’s probably not easy to use them without hassle e.g. in DDEV.

Let’s see what the future brings. I think Docker Desktop for Mac will be the solution in the end (from my today’s point of view).

I’d like to say thank you to my employer supseven.at for giving me the time to write this article.

I hope not only my colleagues can/must benefit from it. ツ

Related Snippets