Fast Autocomplete Search Terms - Rails

Fast Autocomplete Search Terms - Rails

___

Learn how to achieve a lightening-fast global autocompletion to a rails app and improve performance using soulmate, soulmate.js and redis

Introduction

In many cases you find that you need to introduce a global search in your rails application i.e. search multiple models at a go using one form. A good example is the autocomplete search at SeatGeek. See how nicely they autocomplete performers, events and venues in a single form, interesting isn't it? In this tutorial we will try and achieve something similar to seatgeek by building a global autocomplete search across multiple models in our rails application. To do this we will use Soulmate Gem which is actually developed by the guys at seatgeek, soulmate.js which is a jQuery front-end for the soulmate auto-suggestion gem and Redis that the soulmate gem uses as a backend.

One huge plus I think soulmate has is being able to completely separate the auto-suggestion engine from the main app. This allows it to get overwhelmed with a huge number of requests without significantly affecting the user’s experience. Take a sneak peek of the Demo App we will be creating.

Installing Redis

I'm guessing you already have Redis installed in your system by now but if not, installing it is actually very easy

  • On a mac, with homebrew just run: brew install redis
  • On a Unix/Linux base systems, for debian based systems e.g. ubuntu just run sudo apt-get install redis and on redhat based systems e.g. Fedora/Opensuse run sudo yum install redis

You can then verify if redis was successfully installed by starting redis server. On you terminal run:

$ redis-server

You should have something similar to

Getting started

Now that we have redis up and ready, lets start up by creating a rails 4 application to try out soulmate on.

$ rails new fast-autocomplete

Now lets add soulmate gem to our gemfile. I'm also adding the faker gem to help us populate our database with test records instead of creating them and then run bundle install to install our Gems.

Loading Gist

Creating models and adding test data

Now that we have our Gems installed, lets quicky scaffold two models. We will have a noun and a verb models to test this. We will easily populate this tables using the Faker gem shortly. On your terminal

$ rails g scaffold noun name:string
$ rails g scaffold verb name:string

Next run rake db:migrate to create the respective tables in our database. We can now easily perfom CRUD operations for both nouns and verbs at http://0.0.0.0:3000/nouns and http://0.0.0.0:3000/verbs. We can automate the process of adding test data in our tables by using the Faker Gem and writing this code in our seeds.rb file.

seeds.rb

Loading Gist

This will create 500 dummy nouns and an additional 500 dummy verbs in our database. We can now populate our database by running

$ rake db:seed

Adding the after_save and before_destroy callbacks

We will then add an after_save and a before_destroy callback to our models (Noun and Verb). The after_save will be responsible for adding data to Redis through the Soulmate::Loader module provided by the soulmate gem and the before_destroy will remove data from redis when a record is deleted as we don’t want to have orphaned or unused records in our Redis database and also to save on Memory.

noun.rb

Loading Gist

verb.rb

Loading Gist

You will notice an optional data key. This is an optional container for metadata you'd like to return when an item is matched. For our case we add a link to the respective noun / verb url which will be useful later in that the search results will be clickable.

Loading data into Redis

Now that we have our model callbacks in place we can load all the data we ‘seeded’ earlier on into Redis by invoking a resave of each record which will trigger the load_into_soulmate callback in our model. Load rails console and execute the following

Make sure you have redis server running before you run this
$ rails console
2.1.0 :001 >  Noun.find_each(&:save)
2.1.0 :001 > Verb.find_each(&:save)

This will re-save all records triggering the load_into_soulmate callback. Lets take a sneak peek at the data inside Redis and how it got saved by soulmate. On a separate terminal type in redis-cli to access the redis’ console.

$ redis-cli

We then query redis to fetch records in the soulmate-data hash of our first noun and verb respectively. You should get something similar to this

127.0.0.1:6379> hget soulmate-data:nouns 1
"{\"term\":\"alarm\",\"id\":1,\"data\":{\"link\":\"/nouns/1\"}}
127.0.0.1:6379> hget soulmate-data:verbs 1
"{\"term\":\"calculate\",\"id\":1,\"data\":{\"link\":\"/verbs/1\"}}"

The Redis key names use the pattern “ soulmate-data:nouns” and "soulmate-data:verbs" because I passed “nouns” and "verbs" as arguments when calling the Soulmate::Loader class.

Loading soulmate into our rails app

Soulmate can be loaded as a separate ‘web-server’ as its already a full sinatra app but for our case we want to have it inside rails. This can simply be done by adding it to our routes.rb file

routes.rb

Loading Gist

With that in place we can query soulmate at the /autocomplete url. Lets visit http://0.0.0.0:3000/autocomplete and see whats there. This should give you soulmate’s status

Now lets try querying some some data. We’ll start by querying some nouns. I will try querying the term pro and see what I got. Visiting http://0.0.0.0:3000/autocomplete/search?types[]=nouns&limit=6&term=pro give me

For the verbs, I will try searching any verbs starting with the characters co. Visiting http://0.0.0.0:3000/autocomplete/search?types[]=verbs&limit=6&term=co gives me

Waaaat!! That looks really Good. Don’t you agree? The interesting part and most powerful part of soulmate is that we can search all occurrences of a certain term across our models. To do this we need to append them to the types array in our url. This tells soulmate which results to include.

NOTE: THE TYPES YOU PASS IN THE URL MUST MATCH THE ONES YOU PASSED WHEN SAVING INTO SOULMATE (INSIDE OUR MODELS)

For our case we only have NOUNS and VERBS . To search both of them we make a call like this in the url http://0.0.0.0:3000/autocomplete/search?types[]=verbs&types[]=nouns&limit=6&term=ha . Note how we pushed the NOUNS and the VERBS inside the types array. My results will now include occurrences of “ha” in both nouns and verbs

Now that looks really great!! all we are now left with, is building the UI to utilize this awesome search functionality.

Piecing it all together using jquery and soulmate.js

Soulmate.js like we said earlier is an jQuery front-end for the soulmate auto-suggestion gem and together, they provide lightning-fast plug-n-play auto-complete / auto-suggestion. With soulmate.js we will provide our application with an auto-complete look similar to that at seatgeek.com.

Lets start by grabbing soulmate.js from github. Once downloaded grab jquery.soulmate.js located in soulmate.js-master/src/compiled folder and add it to the javascripts vendor directory of our rails app

//= require jquery
//= require jquery_ujs
//= require jquery.soulmate
//= require turbolinks
//= require_tree

Next we grab the demo.css stylesheet located in soulmate.js-master/demo and add it to the stylesheets folder in the vendor directory of our rails app. In this file we only need lines 30-115 you can do away with the rest as they were specific to the demo of its original author. We could also lets rename it to soulmate.css to keep the name semantic

*= require_tree .
*= require soulmate
*= require_self
*/

soulmate.css

Loading Gist

Adding a search form

Lets create a home controller and make it the default root path of our app. We will then put a search from in our index view of our home controller.

$ rails g controller home index

Make the root path point to our home's controller index action. In routes.rb append the following

root 'home#index'

Now lets add a search form in the index view of our home controller

index.html.erb

Loading Gist

Some quick simple styling

home.css.scss

Loading Gist

With this we should now have our 'fancy' home page with our search form

Our most awaited step comes next. Lets now power our search form with soulmate and soulmate.js goodness. Rename home.js.coffee in app/javascripts to home.js as we will be writing in pure javascript . In this file will call soulmate.js with our form search field. Remember we gave it an id of search.

home.js

Loading Gist

In the url option, we pass our soulmate search url we were using earlier. In the types option we pass an array of what we want to search from soulmate. If you have other models, add them here with the key you used to save into redis.

And finally lets test if this actually works . I'll start typing the work “ pa” and see what I get.

global autocomplete search rails

And that worked!!! Hurray! we now have a powerful lightening fast autocomplete. Play around with different search terms and see how fast and accurate the autocomplete is. Thats it for this tutorial and hope this gave you insight on how to come up with a fast autocomplete search in your rails applications.

Have any problems? Feel free to clone this application on github. For other questions and/or comments use the comments section below. Thanks for reading.

Update: This post was featured in Ruby Weekly. Thanks!


7 Comments

___

A39644cef2c1177d9a442676473be792026b1c19

Stanislas Boyet

05 Sep 14

Thanks for this step by step. Really useful. Seeing the results and the speed of the autocomplete, it will definitely come in handy !

983c42ef2855ef8a861f3c7182e0101cabbe47ad

Vignesh Nagarajan

16 Feb 15

Hi Joseph. In this how can we increase the score according to the user click on the search term.

F1f8b38759e30a9af5d46541f20aba18a62e4a78

penabianca

08 Aug 15

Hi Joseph, I followed the steps everything works except the last one. I cannot search from the search form. In you example your form resides in the index file , so your POST to "/" goes directly to home.js and run the script. In my case I have the search form of the header of my page, which is common across the entire application. I believe my POST to "/" cannot find soulmate. That's exactly the part I do not understand, doe the file naming matters, could you please describe how rails find the javascript from the moment I enter a term in the search form ? Thanks.

233b8d5f2d52fbf59043418ecca6e391c10255c2

esolamfelix

18 Dec 15

Hi Joseph, How do we setup Redis on Heroku? I have it working on dev but not in production. Please help Thanks

814006d5ae5e4fbb2d8ab97aa2a6a4578e1542c8

Jon Kern

31 May 16

Brilliantly described step-bystep instructions along with just the right amount of explanations and references! In ~1-2 hours I have some running on develop for two of my mongomapper-based models. Now off to apply more of my specific needs. Thank you!

2e149507fced40725ce778fa5030f2a552ec04d2

Nejurgis

27 Feb 17

Thank you for this great tutorial! Maybe someone knows how would you add json file to the seed.rb?

75f544f3674d0a372a617385a37405e69ec5828c

kinyo

17 Sep 17

having a probelm with the rack-contrib gem installation there seems to be a problem with it and ails 5 is there a work around without it

Latest Tutorials

___

Private Inbox System in Rails with Mailboxer New

Introduction It's been quite a while since my last tutorial and since then I've recieved alot of requests by email to implement a private messaging system ...

Ajax Sortable Lists Rails 4

With me, is a simple to-do list application where users can create dummy to-do lists and displays them in card-like form just like in Trello. We want to e...

Managing ENV variables in Rails

Often when developing Rails applications, you will find a need to setup a couple of environment variables to store secure information such as passwords, a...

Gmail Like Chat Application in Ruby on Rails

Introduction We are all fond of the Gmail and Facebook inline chat modules. About a week ago, I came across a tutorial on how to replicate hangouts chat...

Fast Autocomplete Search Terms - Rails

Introduction In many cases you find that you need to introduce a global search in your rails application i.e. search multiple models at a go using one form...

Load more scroll top