Has anyone come up with a way to speed up the "bundle install" step in a Docker build? The smallest change will cause this step to completely rerun, which takes a long time for a Ruby application with lots of gem dependencies.
One approach might be to base the final Docker image on another Docker image, which has a snapshot of all Rubygem dependencies at a certain point. In the depending image, the 'bundle install' will then do an incremental update and the Docker build will go a lot faster.
But I was wondering how other people are solving this?
If you add the Gemfile and Gemfile.lock to the image before you add all the application files you can cache that step IIRC, then only if that step changes will bundle install run, so long as it's placed after that step.
I usually structure the Dockerfile to make the installation step cacheable, like so:
# This only re-runs the slow installation when requirements.txt changes
ADD requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# This re-runs every time you change any file, but is very fast
ADD . /app/
Put the things that are going to change as the last step; use very few steps (eg dont copy files one by one) and let Docker cache the results.
We use WhaleWare as a templating tool: https://github.com/l3nz/whaleware and it works great (or well enough)
I do "bundle install --standalone" in a separate Docker image whenever I update dependencies, and don't install/run bundler at all in the deployment image.
A typical large Ruby app ends up dragging in tons of build dependencies that does not need to be in the final image to be able to do things like compile extensions etc.
You can achieve that by letting the caching take care of it for you, there's no need for two images there.
I think OP wants to not have to reinstall every dependency again when they update just one. For that I think you'd have to use multiple Gemfiles, one for slow things that rarely change and the other for anything else new.
I have a few docker containers I use as a testing environment. I ended up mounting gems from the host. They where mounted from a /tmp dir so docker would not pollute the gems on the host.
It's worth noting that this is also the approach that e.g. the standard nginx image takes, so if you run nginx in front of your Rails app, you can bring up nginx with a mount of the same volume.
I still prefer to deploy my Ruby apps as part of any image, though. Of course you can combine the two: Build an image whose sole purpose is to package your app and export it as a volume for use by pre-made Rails/Nginx containers.
One approach might be to base the final Docker image on another Docker image, which has a snapshot of all Rubygem dependencies at a certain point. In the depending image, the 'bundle install' will then do an incremental update and the Docker build will go a lot faster.
But I was wondering how other people are solving this?