Cassandra Migrations

Cassandra schema management for a multi-environment development.

A gem to manage Cassandra database schema for Rails. This gem offers migrations and environment specific databases out-of-the-box for Rails users.

This enables you to use Cassandra in an organized way, combined with your ActiveRecord relational database.

Requirements

• Cassandra 1.2 or higher with the native_transport_protocol turned on (Instructions to install cassandra locally)
• Ruby 1.9
• Rails > 3.2

Installation

gem install cassandra_migrations

Quick start

Configure Cassandra

The native transport protocol (sometimes called binary protocol, or CQL protocol) is not on by default in Cassandra 1.2, to enable it edit the CASSANDRA_DIR/conf/cassandra.yaml file on all nodes in your cluster and set start_native_transport to true. You need to restart the nodes for this to have effect.

Prepare Project

In your rails root directory:

prepare_for_cassandra .

Configuring cassandra access

Open your newly-created config/cassandra.yml and configure the database name for each of the environments, just like you would do for your regular database. The other options defaults should be enough for now.

development:
  host: '127.0.0.1'
  port: 9042
  keyspace: 'my_keyspace_name'
  replication:
    class: 'SimpleStrategy'
    replication_factor: 1

Create your database

There are a collection of rake tasks to help you manage the cassandra database (rake cassandra:create, rake cassandra:migrate, rake cassandra:drop, etc.). For now this one does the trick:

rake cassandra:setup

Create a test table

rails generate cassandra_migration create_posts

In your migration file, make it create a table and drop it on its way back:

class CreatePosts < CassandraMigrations::Migration
  def up
    create_table :posts do |p|
      p.integer :id, :primary_key => true
      p.timestamp :created_at
      p.string :title
      p.text :text
    end
  end
  
  def self.down
    drop_table :posts
  end
end

And now run:

rake cassandra:migrate 

To create a table with compound primary key just specify the primary keys on table creation, i.e.:

class CreatePosts < CassandraMigrations::Migration
  def up
    create_table :posts, :primary_keys => [:id, :created_at] do |p|
      p.integer :id
      p.timestamp :created_at
      p.string :title
      p.text :text
    end
  end
  
  def self.down
    drop_table :posts
  end
end

To create a table with a compound partition key specify the partition keys on table creation, i.e.:

class CreatePosts < CassandraMigrations::Migration
  def up
    create_table :posts, :partition_keys => [:id, :created_month], :primary_keys => [:created_at] do |p|
      p.integer :id
      p.string :creation_month
      p.timestamp :created_at
      p.string :title
      p.text :text
    end
  end
  
  def self.down
    drop_table :posts
  end
end

To create a table with a secondary index you add it similar to regular rails indexes, i.e.:

class CreatePosts < CassandraMigrations::Migration
  def up
    create_table :posts, :primary_keys => [:id, :created_at] do |p|
      p.integer :id
      p.timestamp :created_at
      p.string :title
      p.text :text
    end
    
    create_index :posts, :title, :name => 'by_title'
  end
  
  def self.down
 	drop_index 'by_title'
  
    drop_table :posts
  end
end

There are some other helpers like add_column too.. take a look inside!

Migrations for Cassandra Collections

Support for C* collections is provided via the list, set and map column types.

class CollectionsListMigration < CassandraMigrations::Migration
  def up
    create_table :collection_lists do |t|
      t.uuid :id, :primary_key => true
      t.list :my_list, :type => :string
      t.set :my_set, :type => :float
      t.map :my_map, :key_type => :uuid, :value_type => :float
    end
  end
end

Querying cassandra

There are two ways to use the cassandra interface provided by this gem

1. Acessing through query helpers

# selects all posts
CassandraMigrations::Cassandra.select(:posts)

# more complex select query 
CassandraMigrations::Cassandra.select(:posts, 
  :projection => 'title, created_at',
  :selection => 'id > 1234',
  :order_by => 'created_at DESC',
  :limit => 10
)

# selects single row by uuid
CassandraMigrations::Cassandra.select(:posts,
  :projection => 'title, created_at',
  :selection => 'id = 6bc939c2-838e-11e3-9706-4f2824f98172',
  :allow_filtering => true  # needed for potentially expensive queries
)

# adding a new post
CassandraMigrations::Cassandra.write!(:posts, {
  :id => 9999,
  :created_at => Time.current,
  :title => 'My new post',
  :text => 'lorem ipsum dolor sit amet.'
})

# adding a new post with TTL
CassandraMigrations::Cassandra.write!(:posts, 
  {
    :id => 9999,
    :created_at => Time.current,
    :title => 'My new post',
    :text => 'lorem ipsum dolor sit amet.'
  },
  :ttl => 3600
)

# updating a post
CassandraMigrations::Cassandra.update!(:posts, 'id = 9999', 
  :title => 'Updated title'
)

# updating a post with TTL
CassandraMigrations::Cassandra.update!(:posts, 'id = 9999', 
  { :title => 'Updated title' },
  :ttl => 3600
)

# deleting a post
CassandraMigrations::Cassandra.delete!(:posts, 'id = 1234')

# deleting a post title
CassandraMigrations::Cassandra.delete!(:posts, 'id = 1234'
  :projection => 'title'
)

# deleting all posts
CassandraMigrations::Cassandra.truncate!(:posts)

4. Manipulating Collections

Given a migration that generates a set type column as shown next:

class CreatePeople < CassandraMigrations::Migration
  def up
    create_table :people, :primary_keys => :id do |t|
      t.uuid :id
      t.string :ssn
      ...
      t.set :emails, :type => :string
    end
  end

  ...
end

You can add new emails to the existing collection:

CassandraMigrations::Cassandra.update!(:people, "ssn = '867530900'",
                                       {emails: ['[email protected]', '[email protected]']},
                                       {operations: {emails: :+}})

You can remove emails from the collection:

CassandraMigrations::Cassandra.update!(:people, "ssn = '867530900'",
                                       {emails: ['[email protected]']},
                                       {operations: {emails: :-}})

Or, completely replace the existing values in the collection:

CassandraMigrations::Cassandra.update!(:people, "ssn = '867530900'",
                                       {emails: ['[email protected]', '[email protected]']})

The same operations (addition :+ and subtraction :-) are supported by all collection types.

Read more about C* collections at http://cassandra.apache.org/doc/cql3/CQL.html#collections

3. Using raw CQL3

CassandraMigrations::Cassandra.execute('SELECT * FROM posts')

Reading query results

Select queries will return an enumerable object over which you can iterate. All other query types return nil.

CassandraMigrations::Cassandra.select(:posts).each |post_attributes|
  puts post_attributes
end

# => {'id' => 9999, 'created_at' => 2013-05-20 18:43:23 -0300, 'title' => 'My new post', 'text' => 'lorem ipsum dolor sit amet.'}

If your want some info about the table metadata just call it on a query result:

CassandraMigrations::Cassandra.select(:posts).metadata

# => {'id' => :integer, 'created_at' => :timestamp, 'title' => :varchar, 'text' => :varchar}

Using uuid data type

Please refer to the wiki: Using uuid data type

Deploy integration with Capistrano

This gem comes with built-in compatibility with Passenger and its smart spawning functionality, so if you're using Passenger all you have to do is deploy and be happy!

To add cassandra database creation and migrations steps to your Capistrano recipe, just add the following line to you deploy.rb:
require 'cassandra_migrations/capistrano'

Acknowledgements

This gem is built upon the cql-rb gem, and I thank Theo for doing an awesome job working on this gem for us.