Speeding up JRuby/Rails on Vagrant and VirtualBox

Posted on 29 October 2014 by Joseph Turner

After much head banging (and not the good kind) over the slow startup time of Rails, I finally decided to bite the bullet and figure out what was going on. I have been plagued by multi-minute startups in my development environment for a while, and I just couldn't take it anymore. Because we develop Interlock in a virtual machine set up by Vagrant atop VirtualBox, and because the slow-to-start parts of our software use Rails atop JRuby atop the venerable Java, there were quite a few moving pieces to isolate and try to speed up.

My first thought was that it was OpenJDK that was slowing things down, so I installed the Oracle JDK. If this yielded any speedups, they weren't large enough to notice qualitatively.

Next, I thought JRuby itself was the culprit. I found a handy guide the project owners maintain, and tried a number of the suggestions. I found that using the JRuby flags --dev and -J-Djruby.launch.inproc=true did yield some significant speedups, probably 10-15%. This wasn't going to alleviate the bruise on my forehead though, so I had to dig deeper.

As an aside, you can always pass in these flags to JRuby using the JRUBY_OPTS environment variable. For example, if you wanted to start the Rails console with the above flags, you could use

JRUBY_OPTS="--dev -J-Djruby.launch.inproc=true" rails c

After reading one of the last-ditch suggestions in the article above, I decided that the problem might be in the Ruby code itself. I added the --profile flag to a rails c call to get some information about where the Ruby functions were spending their time. Conspicuously, almost all of the time was being spent in IO functions like Dir.[]. This suggested that the problem was perhaps in VirtualBox itself.

I started Googling for things like slow virtualbox io, but that didn't bear much fruit. There were lots of suggestions like 'use SATA and not IDE' which my setup was already doing. So I tried doing some disk IO inside the VM to see what the throughput rates were, and got some surprising results.

vagrant@precise64:/vagrant$ time dd if=/dev/zero of=/tmp/file bs=1M
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.03046 s, 1.0 GB/s 

That's not quite as fast as my SSD-havin' Mac host, but it's pretty quick. What's going on here? Oh right, my project lives in /vagrant which is not really a part of the VM filesystem but is a mounted shared directory! It must be shared directory performance that is the dog. A quick search yielded this Vagrant documentation page. TL;DR:

In some cases the default shared folder implementations (such as VirtualBox shared folders) have high performance penalties.

I also found this page with some older Vagrant documentation that suggests that the problem is with folders with lots of files:

It’s a long known issue that VirtualBox shared folder performance degrades quickly as the number of files in the shared folder increases. As a project reaches 1000+ files, doing simple things like running unit tests or even just running an app server can be many orders of magnitude slower than on a native filesystem (e.g. from 5 seconds to over 5 minutes).

A ha! How many files do we have in our project?

vagrant@precise64:/vagrant$ find | wc -l
44086

Looks like we found our culprit. To fix, I enabled private networking and NFS for the shared folder, as suggested in the documentation linked above. YMMV, but for me, on a Mac host running an Ubuntu guest, it involved adding these lines to my Vagrantfile:

config.vm.network :private_network, ip: "10.11.12.13"
config.vm.synced_folder ".", "/vagrant", type: "nfs"

Afterwards, I got a DHCP error when trying to reload my VM. I found a bug report that suggested that VirtualBox had a default DHCP instance that was colliding, and suggested this a workaround. Run this on the host machine:

VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0

With that working, Vagrant will get to a point where it needs to edit /etc/exports, your list of NFS exports on the host. You'll need to provide your password to edit this file.

Afterwards, I was getting a timeout error from the VM while trying to mount the NFS drive. I tried a few more things, but sometimes the old magic is best: I rebooted the host and the VM booted like a charm.

I tested my Rails startup time, and got between a 5X and 10X speedup. My forehead feels better already.

Replacing the Rails Asset Pipeline with Grunt, Bower and Browserify

Posted on 29 October 2014 by Chase Courington

At Mobile System 7 we're always exploring ways to improve our process. Early this summer we had some disucssion about how to better decouple the UI, 99% Javascript, from the Rails application, which primarily powers our REST api. The opportunity to address this decoupling came when we added a story to address some UI build process issues.

Our UI was using combination of Rails 3.x Asset Pipeline and Grunt to build the UI in development and production environments. We needed streamline this process and agreed that the UI shouldn't rely on Rails to build assets.

The first step in replacing the asset pipeline is identifying what we're relying on and what can we use as replacement. The objective is to decouple the UI from the Rails toolchain, maintaining benefits of the asset pipeline while improving configuration ability.

Note: Since we're already using Grunt and have Node/NPM installed and setup I won't be going over getting that going but you can get some help here. NPM will be initialized in your /app/assets/ directory


De-activate the Asset Pipeline

The asset pipeline allows us to specify in our /app/assets/ directories what javascripts, stylesheets and images to compile/minify. We're going to replace most of this with Grunt tasks. Deactivating the asset pipeline is done with a simple boolean in the application configuration file.

/config/application.rb

config.assets.enabled = false

We should also update our development environment config variable that expands compiled assets for debugging since Rails isn't handling any of the uglification/minification.

/config/environments/development.rb

config.assets.debug = false

Update the Gemfile

With the asset pipeline enabled Rails automatically adds Gems to our Gemfile for certain dependencies. We have dependencies like Bootstrap and jQuery that we need for our UI but we want more control over these assets and with the asset pipeline de-activated it's doesn't make sense to have the Gemfile manage these assets.

We have an :assets group in our Gemfile that we no longer need. The same gems also belong to a :spec group, which is for our Javascript unit tests, but we'll be setting those up independent of the Rails app as well. We can remove this entire block in our Gemfile as well as remove the Jasmine gem from our :development group (your Gemfile may differ).

/Gemfile

...
group :assets, :spec do
    gem 'jquery-rails', '2.2.1'
    gem 'less-rails', '2.2.2'
    gem 'therubyrhino', '2.0.2'
    gem 'twitter-bootstrap-rails', '2.2.3'
end

group :development do
    gem 'jasmine', '1.3.2'
end
...

We use Jasmine for Javascript unit tests. Since these don't rely on any part of the Rails app and we can run them independently with Grunt, we'll be setting that up as well.

We'll use Bower to replace those Gems in the :assets group. Bower gives us a little more control over the configuration of where we get the assets, where we install the assets, what to call the assets, etc. We'll setup Bower in just a little bit.

Once you've removed those gems/groups from your Gemfile, don't forget to run $ bundle install to get a fresh install of your Gemfile and restart your webserver $ rails s. Your app should look pretty different now without those assets.


Architecture of the UI

We'll be using Node/NPM, Grunt, Bower and a litte Browserify. Before we setup any Grunt tasks or load any Bower assets let's get our UI architecture in the Rails app setup.

We'll keep our development UI code in /app/assets/ just as before, we'll add a couple of new things to make our UI a "independent Node-ish" application. The /images/, /javascripts/ and /stylesheets/ should look familiar.

We'll be storing vendor assets from Bower in /vendor/assets/, where Rails previously stored assets managed by the Gemfile. With Bower we can configure how this directory is structured a little better than we can relying on the Gemfile.

In our Rails app, /public/ will contain the files that are served up to our application and it also includes some of the default Rails files like 404.html. This will be where all our compiled, concatenated, uglified/minified files will end up.

/app/assets/

|-- /app/assets/
|  |-- images/
|  |-- javascripts/
|  |-- stylesheets/
|  |-- tests/
|  |-- grunt_tasks/
|  |-- node_modules/
|  |-- Gruntfile.js
|  |-- package.json

/vendor/assets/

|-- /vendor/assets/
|   |-- fonts/
|   |-- images/
|   |-- javascripts/
|   |-- stylesheets/
|   |-- less/
|   |-- tests/

/public/

|-- /public/
|   |-- fonts/
|   |-- images/
|   |-- javascripts/
|   |-- stylesheets/
...
  • /app/assets/images/

    We want to optimize the contents and copy optimized images to /public/images/ to be served up for the application. We'll be using:

  • /app/assets/javascripts/

    The bulk of our UI, we organize with sub-directories for our Backbone.js app and Handlebars templates. We need to lint, pre-compile, concatenate and copy the contents /public/javascripts/ to be served to the application. Here we're using:

  • /app/assets/stylesheets/

    We use LESS as our CSS pre-processor, you could use SASS or any other option if you'd like. We chose LESS for a slight performance advantage over SASS because it uses Node instead of Ruby for compilation. In the effort to decouple from Rails, we want the pre-processor that doesn't require another dependency outside of our Node toolchain.

    We want to compile our LESS files to CSS, concatenate them with any other vendor stylesheets we use and copy the resulting stylesheet to our /public/stylesheets/ to be served up to the application. We'll be using:

  • /app/assets/tests/

    We're moving our Javascript unit tests away from being managed by the Rails toolchain so we can have them standalone and run more easily in development. We can write our tests and easily target our /app/assets/javascripts/ for testing. Our tests use:


Setting up Node Modules

We need to initialize or update our existing package.json file to tell NPM what packages to install. In our case we'll be installing all our packages as devDependencies and configuring some of the scripts to automate global installs for grunt-cli and bower.

If you don't have a package.json then $ cd /app/assets/, remember this is our "node app", and $ npm init (You need Node/NPM installed).

If you have a package.json then make sure it's in /app/assets/ and open it in your favorite text editor and add:

/app/assets/package.json

...
"scripts": {
    "pre-install": "npm install -g grunt-cli bower"
},
...

Running $ npm install will now install grunt-cli and bower globally before installing any of your Node packages. NPM has it's opinion on install scripts, but we've found this to work for us and we're not publishing this as a package, tread lightly.

You'll now have $ grunt and $ bower commands available to you for running tasks and installing packages.

Now that we have those in place we need to install our devDependencies, mostly all Grunt tasks. Here's what our devDependencies attribute in package.json should look something like.

/app/assets/package.json

...
"devDependencies": {
    "grunt": "^0.4.5",
    "grunt-bower-task": "^0.4.0",
    "grunt-browserify": "^3.0.1",
    "grunt-concurrent": "^1.0.0",
    "grunt-contrib-clean": "^0.6.0",
    "grunt-contrib-concat": "^0.5.0",
    "grunt-contrib-connect": "^0.8.0",
    "grunt-contrib-copy": "^0.6.0",
    "grunt-contrib-cssmin": "^0.10.0",
    "grunt-contrib-imagemin": "^0.8.1",
    "grunt-contrib-jasmine": "^0.8.0",
    "grunt-contrib-jshint": "^0.10.0",
    "grunt-contrib-less": "^0.11.4",
    "grunt-contrib-uglify": "^0.6.0",
    "grunt-contrib-watch": "^0.6.1",
    "grunt-modernizr": "^0.6.0",
    "grunt-newer": "^0.7.0",
    "grunt-open": "^0.2.3",
    "handlebars": "^2.0.0",
    "hbsfy": "^2.2.0",
    "time-grunt": "^1.0.0"
}
...

Install Bower packages

Run $ bower init from your /app/assets/ and it'll take you through creating a Bower.json file, that is a configuration file very similar to the package.json to tell Bower what packages to install. We can creat/edit a .bowerrc file to tell Bower where we want our bower_components installed, in our case we want to install to /vendor/assets/.

We use grunt-bower-task to add a Grunt task that will install our packages and give us more control over what gets installed and where. We can configure the Bower.json with exportsOverride to install specific packages js, css, img, font where we want...in corresponding /vendor/assets/ and then we use grunt-contrib-copy to move files from /vendor/ to /public/ and - grunt-contrib-concat to concatenate javascripts and stylesheets together and move from /vendor/ to /public/.

/app/assets/bower.json

...
"dependencies": {
    "backbone": "1.0.0",
    "bootstrap": "2.3.2",
    "moment": "2.5.0",
    "d3": "~3.4.9",
    "leaflet": "~0.7.3",
    "underscore": "~1.6.0",
    "mustache": "~0.8.2",
    "jquery-ujs": "~1.0.0",
    "fontawesome": "3.2.1",
    "jquery": "2.0.3",
    "bootstrap-datepicker": "~1.2.0",
    "jquery-Mustache": "~0.2.7",
    "leaflet.markercluster": "~0.4.0",
    "modernizr": "~2.8.3"
},
"devDependencies": {
    "jasmine-jquery": "2.0.5",
    "sinon": "http://sinonjs.org/releases/sinon-1.10.3.js"
},
"exportsOverride": {
    "bootstrap": {
      "javascripts": "js/*.js",
      "stylesheets": "css/*.css",
      "less": "less/*.less",
      "images": "img/*.png"
    },
    "backbone": {
      "javascripts": "backbone.js"
    },
    "d3": {
      "javascripts": "d3.js"
    },
    "fontawesome": {
      "stylesheets": "css/*.css",
      "less": "less/*.less",
      "fonts": "font/*"
    },
    "leaflet.markercluster": {
      "stylesheets": "dist/*.css",
      "javascripts": "dist/*.js",
      "fonts": "font/*"
    },...
...

Setting up Grunt tasks

We have our bower.json setup, our package.json setup and finally we need to setup some Grunt tasks to work build our UI.

To keep our Gruntfile manageable we break up our Grunt tasks in the /app/assets/grunt_tasks/ and load them in commonjs style to our /app/assets/Gruntfile.js.

We need a task that can prep our directories/files. The prep task will be the baseline task for:

  • cleaning compiled code, /vendor/assets/, and /public/
  • installing Bower assets to /vendor/assets/
  • hinting our javascripts in /app/assets/javascripts/
  • pre-compiling our handlebars templates
  • compiling our less to css
  • concatenating our javascripts, vendor assets and stylesheets
  • moving assets to /public/ to be served
  • optimizing our /public/images/
  • running our javascript unit tests

/app/assets/Gruntfile.js

module.exports = function (grunt) {

    // output task timing
    require('time-grunt')(grunt);

    // Project config
    grunt.initConfig({

        // read grunt tasks from npm
        pkg: grunt.file.readJSON('package.json'),

        // configure paths for grunt plugins
        paths: {
            assets: '../../vendor/assets',
            tests: 'tests',

            src_js: 'javascripts', 
            src_css: 'stylesheets',
            src_img: 'images',
            src_font: 'fonts',
            src_json: 'json',
            src_tmp: 'tmp',

            dist_js: '../../public/javascripts',
            dist_css: '../../public/stylesheets',
            dist_img: '../../public/images',
            dist_font: '../../public/fonts',
            dist_json: '../../public/json'
        }

    });

    // load grunt plugins from directory
    grunt.loadTasks('grunt_tasks');

    grunt.registerTask('prep',
        'Prepare project assets',
        ['clean:nuke', 'bower', 'jshint', 'browserify', 'less', 'concat', 'copy', 'newer:imagemin', 'jasmine:ci']
    );

    grunt.registerTask('dev',
        'Prepare project assets',
        ['prep', 'watch']
    );

    grunt.registerTask('prod',
        'Prepare project assets',
        ['prep', 'cssmin', 'uglify', 'clean:prod']
    );

    grunt.registerTask('default', ['prep', 'dev', 'prod']);

};

Once we configure those Grunt tasks We can run $ grunt prep and watch things build. From here we have grunt dev that adds a watch task in the mix to run while changes are made to files we're watching.

/app/assets/grunt_tasks/contrib-watch.js

module.exports = function (grunt) {

    grunt.config.set('watch', {

        scripts: {
            files: [
                '!.grunt',
                '<%= paths.src_js %>/**/*.js',
                '<%= paths.src_css %>/less/**',
                '<%= paths.tests %>/**/*.js'
            ],
            tasks: ['newer:jshint', 'newer:browserify', 'newer:less', 'newer:concat', 'newer:copy', 'newer:imagemin', 'jasmine:ci'],
            options: {
                interrupt: true
            }
        }

    });

    grunt.loadNpmTasks('grunt-contrib-watch');

};

We also have a grunt prod task that will run our grunt prep task and then also run grunt-contrib-cssmin and grunt-contrib-uglify on our stylesheets and javascripts to minify and uglify. Then it will clean out the non-minified/uglified files from /public/** so that only those minified files are served up.

Let's take a look at bower-task.js to install our Bower packages. Below is what we've got.

/app/assets/grunt_tasks/bower-task.js

module.exports = function(grunt) {

    grunt.config.set('bower', {

        install: {
            options: {
                targetDir: '<%= paths.assets %>',
                install: true,
                cleanTargetDir: true,  // clean the targetDir
                cleanBowerDir: true,  // clean the bower_components dir
                cleanup: true,  // set cleanBowerDir & cleanTargetDir
                copy: true,  // copy bower_components packages to targetDir
                layout: 'byType',  // format tree by component => js/css/less/img
                verbose: false,
                bowerOptions: {
                    forceLatest: false,  // Force latest version on conflict
                    production: false  // Do not install project devDependencies
                }
            }
        }

    });

    grunt.loadNpmTasks('grunt-bower-task');

};

The other heavy lifter of our Grunt tasks is the grunt-contrib-concat task. Concat takes all our vendor files, javascripts and stylesheets, and allows us to concatenate them together in a load order so that we can serve up a single file, reducing requests and making uglification/minification easier. It also handles a lot of the moving of files from our /app/assets/** to our /public/** with the destination concatenated files.

/app/assets/grunt_tasks/contrib-concat.js

module.exports = function (grunt) {

    grunt.config.set('concat', {

        options: {},

        testhelpers: {
            src: [
                '<%= paths.assets %>/test/jasmine-jquery/jasmine-jquery.js',
                '<%= paths.assets %>/test/sinon/index.js',
            ],
            dest: '<%= paths.tests %>/sinon-jasmine-jquery.js'
        },


        app_js: {
            src: [
                '<%= paths.assets %>/js/modernizr/modernizr.custom.js',
                '<%= paths.assets %>/js/jquery/jquery.js',
                '<%= paths.assets %>/jquery-ujs/rails.js',
                '<%= paths.assets %>/js/bootstrap/*.js',
                '<%= paths.assets %>/bootstrap-datepicker/bootstrap-datepicker.js',
                '<%= paths.assets %>/js/d3/d3.js',
                '<%= paths.assets %>/js/underscore/underscore.js',
                '<%= paths.assets %>/js/backbone/backbone.js',
                '<%= paths.assets %>/js/mustache/mustache.js',
                '<%= paths.assets %>/js/jquery-Mustache/jquery.mustache.js',
                '<%= paths.assets %>/js/moment/moment.js',
                '<%= paths.assets %>/leaflet/leaflet.js',
                '<%= paths.assets %>/js/leaflet.markercluster/leaflet.markercluster.js',
                '<%= paths.src_js %>/vendor/bootstrap-overrides.js'
            ],
            dest: '<%= paths.dist_js %>/app.js'
        },


        styles: {
            src: [
                '<%= paths.src_css %>/compiled/app.css',
                '<%= paths.assets %>/bootstrap-datepicker/datepicker.css',
                '<%= paths.assets %>/leaflet/leaflet.css',
                '<%= paths.assets %>/css/leaflet.markercluster/MarkerCluster.css'
            ],
            dest: '<%= paths.dist_css%>/app.css'
        },
...

Serving up the UI

In our Rails app we still use .erb stylesheet_link_tag and javascript_include_tag since they target the /public/ directory and automatically add a hash to aid in cache busting. Here's an example of how we're including styles and javascripts.

/app/views/layouts/application.html.erb

<% if Rails.env =~ /production/ %>
    <%= stylesheet_link_tag "/stylesheets/app.min.css", media: "all" %>
    <%= javascript_include_tag "/javascripts/app.min.js" %>
<% else %>
    <%= stylesheet_link_tag "/stylesheets/app.css", media: "all" %>
    <%= javascript_include_tag "/javascripts/app.js" %>
<% end %>

Conclusion

While we're able to make some major strides in decoupling our UI from the rest of the Rails app there is still some work to be done. It would be good to get grunt-filerev and grunt-usemin included so we don't have to use the Rails asset hashing for cache busting and can also remove those ugly .erb tags. We'd like to introduce Browserify more into the process to reduce the size/complexity of our contrib-concat.js file and grunt-contrib-concat task by using commonjs style includes into our javascripts. All in all, decoupling the UI has worked well and helped to improve our UI development process by giving us more control over the configuration.

For questions or comments feel free to reach out @chasecourington. Cheers!

Welcome!

Posted on 28 October 2014 by Matt Dew

Whether you stumbled upon this blog accidentally, were led here by someone who was playing a practical joke, or deliberately came here in pursuit of knowledge or a solution to a nagging technical problem, allow me to welcome you and let you know you've come to the right place!

If you're not familiar with Mobile System 7, that's ok. While you're encouraged (by me) to look into how our small team is changing Enterprise Security, you don't need to care about Enterprise Software or Security to find this blog interesting or useful - and that's the point. At Mobile System 7 we've benefitted greatly from the open-source community and the depth of technical resources available on the Internet, and we'd like to do our part by sharing the wisdom that our developers have acquired over the years - either while building our Interlock product, or while serving in many diverse roles at previous companies.

Mobile System 7's development team is comprised of entrepreneurs, startup veterans, and former consultants; vegans, vegetarians, and omnivores; Republicans, Democrats, and the politically-indifferent; runners, rock climbers, martial artists, and cyclists; Rubyists, Pythonauts, Java & JavaScript nerds (most of us can even explain the difference), Haskellites, C/C++ slingers, ActionScripters, Linux geeks, and reluctant C# writers; and, without exception, unabashed Star Wars purists.

Another thing we all have in common is that we're opinionated, and, usually to someone's chagrin, willing to share our opinions. This is often made evident during debates in our internal IRC channel, but when we get down to business our opinions and passion for sharing them help us find better solutions to the many complicated problems we're solving. Point is - we have a lot to share, we want to share it, this blog is where we're going to do it, and we think you just might benefit from it.

So what is there to look forward to? The team will be posting haphazardly in the near future, likely as we're inspired, but hopefully more regularly as time marches on. You can expect to see posts along the lines of:

  • Interesting code snippets
  • Libraries we've found useful
  • How we use build tools/toolchains
  • Series of posts explaining how we're approaching certain concepts
  • Details as to how we've integrated with various platforms
  • Development processes that have worked (or have not worked) for us
  • Maybe the occasional recipe or food/beer recommendation

For now, though, we have a couple great posts for you to peruse. Joseph Turner has some tips about improving JRuby/Rails startup time while running on VirtualBox, and Chase Courington has shared the details of his experience migrating from the Rails asset pipeline to Grunt/Bower/Browserify. Check them out, and add us to your feed reader so we can drop in on you from time to time.

We're excited to share our knowledge with you! You may not be excited yet, but I promise you will be....oh, you will be.


Copyright © 2014 Mobile System 7 - www.mobilesystem7.com