Ruby Class Annotations Demystified

If you’re working with a framework (like Rails, Django, Sinatra, or Flask), learning to use it is just scratching the surface. Be rigorous. Go deeper. Learn how it works.

Let’s take a close look at real-world uses of metaprogramming.

Metaprogramming is the writing of computer programs with the ability to treat programs as their data. It means that a program could be designed to read, generate, analyse and/or transform other programs, and even modify itself while running.

We’ll deconstruct how Rails’ ActiveRecord creates methods on the fly in has_secure_token and enum class annotations. By the end of this post, you’ll better understand the underlying mechanisms used by Rails and your favorite gems to create their class annotations.

Metaprogramming is one of my all-time favorite topics, so let’s get started!

Source code for Rails’ has_secure_token annotation reproduced below:

module ActiveRecord
  module SecureToken
    extend ActiveSupport::Concern

    module ClassMethods
      # Example using has_secure_token
      #
      #   # Schema: User(token:string, auth_token:string)
      #   class User < ActiveRecord::Base
      #     has_secure_token
      #     has_secure_token :auth_token
      #   end
      #
      #   user = User.new
      #   user.save
      #   user.token # => "4kUgL2pdQMSCQtjE"
      #   user.auth_token # => "77TMHrHJFvFDwodq8w7Ev2m7"
      #   user.regenerate_token # => true
      #   user.regenerate_auth_token # => true
      #
      # SecureRandom::base58 is used to generate the 24-character unique token, so collisions are highly unlikely.
      #
      # Note that it's still possible to generate a race condition in the database in the same way that
      # <tt>validates_uniqueness_of</tt> can. You're encouraged to add a unique index in the database to deal
      # with this even more unlikely scenario.
      def has_secure_token(attribute = :token)
        # Load securerandom only when has_secure_token is used.
        require 'active_support/core_ext/securerandom'
        define_method("regenerate_#{attribute}") { update! attribute => self.class.generate_unique_secure_token }
        before_create { self.send("#{attribute}=", self.class.generate_unique_secure_token) unless self.send("#{attribute}?")}
      end

      def generate_unique_secure_token
        SecureRandom.base58(24)
      end
    end
  end
end

Note that lib modules such as SecureToken are automatically loaded into Active Record. Below is an excerpt of active_record.rb showing how modules are imported:

module ActiveRecord
  extend ActiveSupport::Autoload
  autoload :SecureToken
end

Also, extend ActiveSupport::Concern does what’s called the include-and-extend trick.

The first thing you’ll notice is module ClassMethods. This module allows class annotations such as has_secure_token to be imported.

Next, let’s take a closer look at has_secure_token:

  def has_secure_token(attribute = :token)
    # Load securerandom only when has_secure_token is used.
    require 'active_support/core_ext/securerandom'
    define_method("regenerate_#{attribute}") { update! attribute => self.class.generate_unique_secure_token }
    before_create { self.send("#{attribute}=", self.class.generate_unique_secure_token) unless self.send("#{attribute}?")}
  end

Whenever the has_secure_token annotation is included like so:

class User < ActiveRecord::Base
  has_secure_token :auth_token
end

The has_secure_token method is executed within the scope of the class. It does two things:

  • Using the native Module#define_method method (ha), we dynamically define a new regenerate_* method under our class based on the attribute parameter passed into the has_secure_token annotation. It takes a method name string or symbol and a Proc, Method, or an UnboundMethod object.
  • We also define a new before_create filter for the current class.

As you can see, the class annotation generates new methods on the fly and modifies the class which it annotates as the code is being interpreted. Cool, huh?

Now, let’s see another example. Here’s an example usage of Rails’ enum class annotation:

class Conversation < ActiveRecord::Base
  # Note that you can have multiple enums in a single model
  enum status: [ :active, :archived ]
end
# conversation.update! status: 0
conversation.active!
conversation.active? # => true
conversation.status  # => "active"
# conversation.update! status: 1
conversation.archived!
conversation.archived? # => true
conversation.status    # => "archived"
# conversation.update! status: 1
conversation.status = "archived"
# conversation.update! status: nil
conversation.status = nil
conversation.status.nil? # => true
conversation.status      # => nil

What!? I never defined those methods in my class, how did Rails know how to respond to .active! and .archived? when I never explicitly defined them?

The answer: Metaprogramming.

Let’s take a closer look. Source code for Rails’ enum annotation reproduced below:

def enum(definitions)
  klass = self
  definitions.each do |name, values|
    # statuses = { }
    enum_values = ActiveSupport::HashWithIndifferentAccess.new
    name        = name.to_sym

    detect_enum_conflict!(name, name.to_s.pluralize, true)
    klass.singleton_class.send(:define_method, name.to_s.pluralize) { enum_values }

    detect_enum_conflict!(name, name)
    detect_enum_conflict!(name, "#{name}=")

    attribute name, EnumType.new(name, enum_values)

    _enum_methods_module.module_eval do
      pairs = values.respond_to?(:each_pair) ? values.each_pair : values.each_with_index
      pairs.each do |value, i|
        enum_values[value] = i

        # def active?() status == 0 end
        klass.send(:detect_enum_conflict!, name, "#{value}?")
        define_method("#{value}?") { self[name] == value.to_s }

        # def active!() update! status: :active end
        klass.send(:detect_enum_conflict!, name, "#{value}!")
        define_method("#{value}!") { update! name => value }

        # scope :active, -> { where status: 0 }
        klass.send(:detect_enum_conflict!, name, value, true)
        klass.scope value, -> { klass.where name => value }
      end
    end
    defined_enums[name.to_s] = enum_values
  end
end

The code seems complex and the result feels magical, but in reality it’s quite simple. The enum method does three things of note:

  • For each enum annotation (definition), checks that the enum name doesn’t conflict with existing getter and setter method names.

Each enum annotation takes a name string and values symbol array of enum types.

  • For each enum, defines a new attribute under the current class with name name pointing to its enum value
  • For each enum AND for each of that enum’s value value, defines two methods: #{value}? and #{value}!. It also defines a new scope scope value, -> { where #{name} => #{value} }.

Cool, huh? Metaprogramming is how most Ruby magic happens under the hood! Refer back to the enum source and make sure you can understand how those methods are generated.

There’s a lot more to metaprogramming than what I’ve mentioned, such as method_missing, class_eval, instance_eval, and hook methods (inherited, included, and son on.) Each have found its use in many Ruby frameworks and libraries.

If you’re interested to learn more about metaprogramming in Ruby, be sure to read the links below!

Additional reading: