Fast Autocomplete Search Terms - Rails
13 Jul 14
Learn how to achieve a lightening-fast global autocompletion to a rails app and improve performance using soulmate, soulmate.js and redis
In many cases you find that you need to introduce a global search in your rails application i.e. search multiple models at a go using one form. A good example is the autocomplete search at
SeatGeek. See how nicely they autocomplete performers, events and venues in a single form, interesting isn't it? In this tutorial we will try and achieve something similar to seatgeek by building a global autocomplete search across multiple models in our rails application. To do this we will use Soulmate Gem which is actually developed by the guys at seatgeek, soulmate.js which is a jQuery front-end for the soulmate auto-suggestion gem and Redis that the soulmate gem uses as a backend.
One huge plus I think soulmate has is being able to completely separate the auto-suggestion engine from the main app. This allows it to get overwhelmed with a huge number of requests without significantly affecting the user’s experience. Take a sneak peek of the Demo App we will be creating.
I'm guessing you already have Redis installed in your system by now but if not, installing it is actually very easy
- On a mac, with homebrew just run: brew install redis
- On a Unix/Linux base systems, for debian based systems e.g. ubuntu just run sudo apt-get install redis and on redhat based systems e.g. Fedora/Opensuse run sudo yum install redis
You can then verify if redis was successfully installed by starting redis server. On you terminal run:
You should have something similar to
Now that we have redis up and ready, lets start up by creating a rails 4 application to try out soulmate on.
$ rails new fast-autocomplete
Now lets add soulmate gem to our gemfile. I'm also adding the
faker gem to help us populate our database with test records instead of creating them and then run bundle install to install our Gems.
Creating models and adding test data
Now that we have our Gems installed, lets quicky scaffold two models. We will have a
noun and a verb models to test this. We will easily populate this tables using the Faker gem shortly. On your terminal
$ rails g scaffold noun name:string
$ rails g scaffold verb name:string
rake db:migrate to create the respective tables in our database. We can now easily perfom CRUD operations for both nouns and verbs at http://0.0.0.0:3000/nouns and http://0.0.0.0:3000/verbs. We can automate the process of adding test data in our tables by using the Faker Gem and writing this code in our seeds.rb file.
This will create 500 dummy nouns and an additional 500 dummy verbs in our database. We can now populate our database by running
$ rake db:seed
Adding the after_save and before_destroy callbacks
We will then add an
after_save and a before_destroy callback to our models (Noun and Verb). The after_save will be responsible for adding data to Redis through the Soulmate::Loader module provided by the soulmate gem and the before_destroy will remove data from redis when a record is deleted as we don’t want to have orphaned or unused records in our Redis database and also to save on Memory.
You will notice an optional
data key. This is an optional container for metadata you'd like to return when an item is matched. For our case we add a link to the respective noun / verb url which will be useful later in that the search results will be clickable.
Loading data into Redis
Now that we have our model callbacks in place we can load all the data we ‘seeded’ earlier on into Redis by invoking a resave of each record which will trigger the
load_into_soulmate callback in our model. Load rails console and execute the following
Make sure you have redis server running before you run this
$ rails console
2.1.0 :001 > Noun.find_each(&:save)
2.1.0 :001 > Verb.find_each(&:save)
This will re-save all records triggering the
load_into_soulmate callback. Lets take a sneak peek at the data inside Redis and how it got saved by soulmate. On a separate terminal type in redis-cli to access the redis’ console.
We then query redis to fetch records in the soulmate-data hash of our first noun and verb respectively. You should get something similar to this
127.0.0.1:6379> hget soulmate-data:nouns 1
127.0.0.1:6379> hget soulmate-data:verbs 1
The Redis key names use the pattern “
soulmate-data:nouns” and "soulmate-data:verbs" because I passed “nouns” and "verbs" as arguments when calling the Soulmate::Loader class.
Loading soulmate into our rails app
Soulmate can be loaded as a separate ‘web-server’ as its already a full sinatra app but for our case we want to have it inside rails. This can simply be done by adding it to our routes.rb file
With that in place we can query soulmate at the
/autocomplete url. Lets visit http://0.0.0.0:3000/autocomplete and see whats there. This should give you soulmate’s status
Now lets try querying some some data. We’ll start by querying some nouns. I will try querying the term
pro and see what I got. Visiting http://0.0.0.0:3000/autocomplete/search?types=nouns&limit=6&term=pro give me
For the verbs, I will try searching any verbs starting with the characters
co. Visiting http://0.0.0.0:3000/autocomplete/search?types=verbs&limit=6&term=co gives me
Waaaat!! That looks really Good. Don’t you agree? The interesting part and most powerful part of soulmate is that we can search all occurrences of a certain term across our models. To do this we need to append them to the
types array in our url. This tells soulmate which results to include.
NOTE: THE TYPES YOU PASS IN THE URL MUST MATCH THE ONES YOU PASSED WHEN SAVING INTO SOULMATE (INSIDE OUR MODELS)
For our case we only have NOUNS and VERBS . To search both of them we make a call like this in the url
http://0.0.0.0:3000/autocomplete/search?types=verbs&types=nouns&limit=6&term=ha . Note how we pushed the NOUNS and the VERBS inside the types array. My results will now include occurrences of “ha” in both nouns and verbs
Now that looks really great!! all we are now left with, is building the UI to utilize this awesome search functionality.
Piecing it all together using jquery and soulmate.js
Soulmate.js like we said earlier is an jQuery front-end for the soulmate auto-suggestion gem and together, they provide lightning-fast plug-n-play auto-complete / auto-suggestion. With soulmate.js we will provide our application with an auto-complete look similar to that at
Lets start by grabbing
//= require jquery
//= require jquery_ujs
//= require jquery.soulmate
//= require turbolinks
Next we grab the
demo.css stylesheet located in soulmate.js-master/demo and add it to the stylesheets folder in the vendor directory of our rails app. In this file we only need lines 30-115 you can do away with the rest as they were specific to the demo of its original author. We could also lets rename it to soulmate.css to keep the name semantic
*= require_tree .
*= require soulmate
Adding a search form
Lets create a
home controller and make it the default root path of our app. We will then put a search from in our index view of our home controller.
$ rails g controller home index
Make the root path point to our home's controller index action. In
routes.rb append the following
Now lets add a search form in the index view of our home controller
Some quick simple styling
With this we should now have our 'fancy' home page with our search form
Our most awaited step comes next. Lets now power our search form with soulmate and soulmate.js goodness. Rename
url option, we pass our soulmate search url we were using earlier. In the types option we pass an array of what we want to search from soulmate. If you have other models, add them here with the key you used to save into redis.
And finally lets test if this actually works . I'll start typing the work “
pa” and see what I get.
And that worked!!! Hurray! we now have a powerful lightening fast autocomplete. Play around with different search terms and see how fast and accurate the autocomplete is. Thats it for this tutorial and hope this gave you insight on how to come up with a fast autocomplete search in your rails applications.
Have any problems? Feel free to clone this application on
github. For other questions and/or comments use the comments section below. Thanks for reading.
Update: This post was featured in Ruby Weekly. Thanks!