This is a long post about the development environment I’m currently using.
The Basics
There are lots of technology available for developers that can make development much easier and many are free. I’ve been working on a birding website, using the Perl web framework Mojolicious. My development system is a Macbook Pro. It comes with an old version of Perl, so I used PerlBrew to manage different Perl versions. The editor I use is Atom.
Database Support
Most websites are connected to a database, and my birding website is no different. I’m currently using MySQL. MySQL has support for spatial types, which I hope to take advantage of in the website. The standard way for using a database in Perl is to use DBI and the corresponding DBD driver for the database that is being connected to. Many of these DBD drivers are linked to a C library implementation. However, a few are written that do not link to a C library. A native pure Perl DBD driver is nice because it doesn’t require the database software to be installed. I first experienced a native driver when writing some Go code that connected to MySQL.
Mojolicious has some modules that simplifies querying databases. Being a long time DBI user, I didn’t really see the need for such modules. However, the Mojolicious module Mojo::mysql has an interesting method named migrations. It is way to automatically update/downgrade a database when schema changes are made. It seems like most of the Mojolicious database drivers try to support migrations. Apparently Postgres has the best support for migrations since the DML statements can be rolled back. A great resource on the theory behind migrations can be found here.
The Mojo::mysql driver doesn’t have direct support for DBD::mysqlPP (the pure Perl driver for MySQL). But it is simple enough to get around that limitation by setting the dsn parameter of the new method to dbi:mysqlPP:<remaining parts>
. However, I quickly learned that the DBD::mysqlPP driver was missing some functionality and it had a bug. The bug is exhibited itself when DBI’s bind_param method is used. The missing functionality is transaction support. I’ve have a fork of DBD::mysql for those interested in a fix. With those fixes, my Mojolicious app seems to functional, but I have since decided not to pursue the native driver approach.
Docker
For development, the MySQL database is run within a Docker container. The default Docker container for MySQL is very capable and can be used as is. Here’s was my initial startup script:
mysql_opts="-e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=migratus -e MYSQL_USER=migratus -e MYSQL_PASSWORD=migratus"
docker run -p 3306:3306 --name migratus-db ${mysql_opts} -d mysql:8 --default-authentication-plugin=mysql_native_password
The DBD::mysqlPP driver does not support the newer authentication method, so the --default-authentication-plugin=mysql_native_password
is needed.
Carton
With PerlBrew on OS X and MySQL within a Docker container, development works fairly well, but a better way of dealing with Perl application development is to use Carton. With Carton, a file named cpanfile is created that lists the application’s depencies, but only at the highest level. Issuing the command carton install
pulls over the listed modules and their dependencies. This also creates a file cpanfile.snapshot
which contains any additional modules the dependencies needed. With these two files, another developer can recreate the same environment. Carton installs all the modules in directory named local
alongside the application. I’ve long preferred this method of building applications with Perl versus installing modules into the system Perl. My experience with Carton is new, so expect more posts regarding this tool in the future.
Carton and Docker finds a bug
Since the birding website is going to run on a Linux based system, it makes sense that Mojolicious framework is developed on Linux as well. Perl is suppose to be platform neutral, but I ran into an interesting problem when moving the code into a Docker container. I had the following code:
use Base Mojo::mysql;
This worked fine on OS X. However, it broke when moving to Linux. I got the following error:
Can't locate Base.pm in @INC
Base is actually base, but OS X uses a case-insensitive filesystem, so Perl was able to find the module.
Containerizing Perl
To move the Mojolicious app to Docker, I use the following Dockerfile that initializes a base container with a recent Perl and some additional packages:
FROM perl:5.30
RUN cpanm Carton \
&& mkdir -p /usr/src/app
RUN apt-get update
RUN apt install -y default-libmysqlclient-dev default-mysql-client default-mysql-client-core
EXPOSE 3000
WORKDIR /usr/src/app
With this Dockerfile, I created an image:
docker build -t macfisherman/migratus .
Create a Docker Network
To simplify container to container communication, Docker’s networking feature is used:
docker network create migratus
This creates a named network that containers can connect to. All containers that use the same network can connect directly to each other. Since the birding website is a website that uses a database backend, having these as two separate containers make sense. With the network in place, the development system can be brought up. The script brings up the database connected to the migratus
network:
#!/bin/bash
mysql_opts="-e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=migratus -e MYSQL_USER=migratus -e MYSQL_PASSWORD=migratus"
docker run -P --network migratus --name migratus-db ${mysql_opts} -d mysql:8 --default-authentication-plugin=mysql_native_password
Initializing the application dependencies
This new Perl container is then started and Carton is used to install the dependencies. Docker’s virtual mount point is used to keep the source code on OS X so Atom can be used.
docker run -it --rm --name migratus --network migratus -p 3000:3000 -v "$PWD":/usr/src/app -w /usr/src/app macfisherman/migratus bash
This command causes a Bash shell to be running within the container. The following command is run in the container:
carton install
This creates a local
directory. Because /usr/src/app
is also mapped to the development directory in OS X it means the modules loaded into local
will survive a container removal. The local
directory should be exclude from any source code management system, but the files cpanfile
and cpanfile.snapshot
can and should be version controlled.
Running the application
With all the applications dependencies in place, it can be run. But first some Docker related checks. The database container should be pingable within the Perl Docker container. Since in this development environment it was named migratus-db, The following command should return positive results:
ping migratus-db
Although Carton provides an exec command that runs a Perl application with the correct module paths, the environment variable PERL5LIB can also be used. Since the application requires additional module paths, this is the route I chose. To run the app, the following is used:
PERL5LIB=$PWD/local/lib/perl5:$PWD/lib local/bin/morbo migratus
Summary
I can now do development on my birding site, keeping the application and its dependencies isolated from the OS X environment. The development website is reachable at http://localhost:3000. Code is locally accessible from OS X, and Perl related modules all all contained in a central spot. This post contained a lot of information, feel free to ask for any clarifications.