Scaling Rails with SyslogLogger

Posted by Jason, Sat Aug 11 02:58:00 UTC 2007

Scaling Rails with SyslogLogger



If you work with Ruby on Rails, I'm sure you have also heard repeated concerns from clients asking "but can Ruby on Rails scale?" This has been a very popular topic of conversation with Rails getting more mainstream respect. Back in May, I attended a conference where some of the brightest Rails folks in the bay area where speaking on that very issue. Sitting in the first row, with my pen and paper, I was ready to hear experiences that companies such as Sun, Joyent and Twitter have had in dealing with Rails and scalability. You really should have been there, as I am far to busy to type everything said here. In a nutshell, know your system. How busy is your database? Where are your bottlenecks? When considering the general architecture of you application, are there things in your relational database that could be on the filesystem, or maybe loaded from YAML files on boot and kept in memory? Have you spent excess time in over engineering the finer points of your application, yet neglected other aspects which are far more worthy of attention (because you don't really know your system). There are various tools available to help you and your application get a little more intimate, but do you use them?

Last Tuesday, I found myself eating taco's with the people from Joyent. It was nice to stop in and meet some of them in person, as they do all of the hosting for the Rails company that I work for. If you work professionally with Rails, I'm sure you have at least heard their name. While eating taco's and getting drunk, I did what any other Rails developer would do, I tried to squeeze as much information about scaling Rails as I could without making things feel like work. I don't after all want them to bill me. Joyent handles the co-location for Twitter, which is quite possibly the largest Rails application in the wild. Rumor has it that Twitter exceeds over 200 mongrels, which is.. A lot. So, even though none of the folks at Joyent would speak anything of Twitters setup, we did get into a very interesting conversation about how expensive the Logger facility is in a Rails application. Let me elaborate. The problem is, that when you do something in your code like:


logger.debug "Print the contents of this object #{@object}"


The logger will output the contents of @object to your logs. Because it is logger.debug, it will only output its result if your in a development environment. This is a cause for some misunderstanding, which leads to a huge performance impact that most people are not aware of. A problem with the default Logger facility is that your environment is the last thing to be evaluated. So, for everyone of these babies your system churns, it may have a routine where it reads the logger.debug statement, evaluates it for a result, formats the log, and then checks if it should output it to a log. If for instance, you're application is running in a production environment, at the very end of all of that evaluation, it will realize that this logger statement is intended for a development environment and not output the log. So, even though you're application does not have the expense of writing to the log file, it does incur the expense of evaluating the statement. This makes logging a very, very costly operation if you are looking at traffic similar to Twitter. Enter the need for a better logging system, SyslogLogger. Since we are talking scaling, it makes sense to redirect production logging to its own centralized logging server, via syslog. This is exactly what SyslogLogger allows us to do. Best of all, its easy. The first thing that you have to do is grab the Rails Analyzer tool kit from rubyforge. Its available as a gem as well, so fetching it is as easy as:


$ gem install rails_analyzer_tools


Then, add the following line to your applications config/environment.rb:


require 'analyzer_tools/syslog_logger'


And also add the following to config/enviroments/production.rb:


config.logger = SyslogLogger.new


By default, the above line will start logging with the program name "rails". If you want something customer modify it to something like:


config.logger = SyslogLogger.new(program_name = 'mephisto')


Now restart your application and setup Syslog. By default, a valid Solaris line in /etc/syslog.conf would be:


user.debug                                      /var/log/rails


The actual lines in syslog look like this:


Mar 31 10:55:24 nevele rails[223883]: [ID 712911 user.info] Processing
MephistoController#dispatch (for 127.0.0.1 at 2007-03-31 10:55:24) [GET]
Mar 31 10:55:24 nevele rails[223883]: [ID 712911 user.info] Parameters:
{"action"=>"dispatch", "controller"=>"mephisto", "path"=>["stylesheets",
"images", "background.gif"]}
Mar 31 10:55:24 nevele rails[223883]: [ID 712911 user.debug] Site Load
(0.001754)   SELECT * FROM sites WHERE (sites.`host` = 'localhost') LIMIT 1
Mar 31 10:55:24 nevele rails[223883]: [ID 712911 user.debug] Site Load
(0.001819)   SELECT * FROM sites ORDER BY id LIMIT 1


If you are running a rather high availability Rails application, you will be utterly amazed with how much resource this will save, and with your logging facility being on a separate server, it is that much easier to scale. While your at it, since you have the rails analyzer tools you should take a look at the rest of whats available. Bench, Crawler, RailsStat and IOTail are other utilities well worthy of mention. But, better saved for a future post.

I'd also like to take this opportunity to thank the folks at Joyent for buying me tacos and alcohol, and schooling me on rails at their Tuesday night get-together.. Jill loves Ruby, therefor.... :)

0 comments | Filed Under: | Tags:

Keep your rails logs manageable

Posted by Jason, Fri Aug 10 01:55:00 UTC 2007

Override RAILS_DEFAULT_LOGGER to keep your logs short



It seems like the one of the greatest resources a developer has are his logs. As such, I have found that it really helps to keep development.rb somewhat organized and easy to find anything in while working. Ok, maybe organized isn’t the best word, as there is really only so much you can do with a log.. Maybe, “short” is better. It only took a year and a half until I got tired of allowing my rails development log to grow to a size that would take me a month to page through looking for, well, that’s the point.. A lot of the time I’m not even sure what it is exactly that I’m looking for. Getting to the point, you can override RAILS_DEFAULT_LOGGER and have your rails application parse your log on a set interval, I prefer daily. Doing so is easy, just drop a line like this in your config/environment.rb file:


RAILS_DEFAULT_LOGGER = Logger.new("#{RAILS_ROOT}/log/#{RAILS_ENV}.log", shift_age = 'daily')


As you most likely can see, this overrides the default logger constant, and sets it to cycle its logs daily. If this isn't cool enough, feel free to check out Rails::Configuration for more options. If you were feeling really ambitious you could always add a rake task to nuke any logs over a month old, assuming your working on a project that takes that long :)

0 comments | Filed Under: | Tags:

attachment_fu and restful_authentication

Posted by Jason, Sat Jun 09 04:04:00 UTC 2007

Howto override methods in attachment_fu to get the file system storage design that you want...



The days of file_column are long over. Its hard to stay current with everything going on in the world of rails. I am constantly keeping an eye on my IRC scroll trying to keep as up to date on what is the up to date way of doing things. Today when logging into Trac I noticed that I had a ticket which required a user to upload images. A few things crossed my mind, the first of which was file_column, a Ruby on Rails plugin that I have used many times in the past for doing just that. A few things to consider. I have no idea what kind of image processor will be used on the server. I need a way to resize images, filter content types, and since this application is expected to get just short of a million hits a day, Amazon s3 integration is a needed scaling option. I soon realized that file_column was going to need some personalization if this was going to work. Sitting back for a minute and thinking about how was the best method to approach this, I did what many Rails developers do, I checked out what Rick Olsen's been up to. Wow, look at that.. Acts_as_attachment. Very nice, I had never heard of that one before. Amazingly enough, before I had even heard about this plugin (its been a while since I had to deal with uploading images, I'll admit), he had rewrote it. Sometimes it truly amazes me how fast that guy moves. Welcome to my world attachment_fu.

I found a really nice post which explains a basic overview of how to get things set up. The purpose of my post is not to document how to get attachment_fu working, as the above post and Rick Olsen's documentation (source code) do a very good job of that. Instead, I intend just to point out a few things that I noticed that may not be so obvious if you are just getting intimate with attachment_fu. I wanted to have attachment_fu and restful_authentication share the same model and form, so that a user can sign up, upload a picture, and submit all in one clean action. The example configuration in the post above assumes that you will be using a separate model, with a belongs_to association. Photo belongs_to Person, or something of the like. When I followed along, without much thought I included the parent_id field in my database migration not realizing the importance of that field. As it turns out, attachment_fu is smart enough to see if this field exists, and if it does it sets the directory to that name to store the images under. Note the following code in filesystem_backend.rb:


def attachment_path_id
  ((respond_to?(:parent_id) && parent_id) || id).to_i
end


So, if you are trying to get restful_authentication and attachment_fu to play nicely together, do not include parent_id as this example suggests. Reading on.. In the event that your application is small enough where you really don't need the level of separation that attachment_fu gives, you can override the full_filename method in the model. Rick made a reference to this, but didn't really get into much detail. He's probably busy writing plugins to assist me in keeping my boss happy, while allowing me a social life. Anyhow, to give an example of how one would do this to get a really simple file system storage, drop something like the following into your model (a separate model, or your restful_authentication model, either is fine):


class User < ActiveRecord::Base
   
  has_attachment :storage => :file_system, :path_prefix => 'public/pictures'
  validates_as_attachment
  validates_uniqueness_of :filename
 
  def full_filename(thumbnail = nil)
    file_system_path = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix].to_s
    File.join(RAILS_ROOT, file_system_path, thumbnail_name_for(thumbnail))
  end
  
end


This will override the full_filename method found in lib/attachment_fu/technoweenie/attachment_fu/backends/file_system_backend.rb. You can override any other methods in that file, or any plugin for that matter the same way. While at it, a requirement was that if the user did not wish to upload an image, they could choose from a selection of stock images present below the upload form. This is nice, but proved to be a little tricky as if you selected a radio button for one of the stock images, all of the validation for attachment_fu failed. The requirements stated that since the company paying for this application may wish to change the default pictures from time to time, and if they did, they didn't want it to change the image that visitors had previously selected, it was necessary to have the attachment_fu upload these stock images from their home in public, to their place on the filesystem. I handled the logic for figuring out if the user was uploading a picture or using one of the supplied default images in the update method of the user_controller, provided by restful authentication, which looks something like:


  if params[:default_image] # controlled by the radio button
    @user.uploaded_data = default_image_filehandle(params[:default_image])  #radio button image
  else
    @user.uploaded_data = params[:user][:uploaded_data]  # use the upload form
  end


You may have noticed the default_image_filehandle method above, that is my slightly hackish way to inject some necessary information into the user object before its sent off to attachment_fu's validation. I'm sure that their are better, more elegant ways of doing so, but this worked for me. That method in detail is:


  def default_image_filehandle(name)
    faked_upload = File.open(File.join(RAILS_ROOT,'public','images/default_images',name))
    (class << faked_upload; self; end).class_eval do
    alias local_path path
      define_method(:original_filename) {name}

      # Below I just feed it values that will allow attachment_fu validation to pass.
      # Since this is being uploaded from public, no serious validation is really needed.
      define_method(:content_type) {"image/jpeg"}
      define_method(:size) { 68335 }
    end
    faked_upload
  end


And there you have it. Restful authentication and attachment_fu working with uploaded pictures, or some default site supplied photos. Plus, its kind of a neat way to use class_eval :) Aaah yes... Attachment_fu. It's nice to be intimate.

1 comment | Filed Under: Plugins | Tags:

Rake task to set up SVN

Posted by Jason, Fri May 18 07:56:00 UTC 2007

It seems like I found a way to DRY up my rails creation process. Every time I would set up a new application that was going to be imported into subversion, I would have to go through the same propset svn:ignore routine. I found this script in another project, and thought it was worthy of attention. Now, all you have to do is drop the below code in a file called svn.rake in your lib/tasks directory and run it using "rake configure_for_svn" on your initial checkout.


desc "Configure Subversion for Rails"
task :configure_for_svn do
  system "svn remove log/*"
  system "svn commit -m 'removing all log files from subversion'"
  system 'svn propset svn:ignore "*.log" log/'
  system "svn update log/"
  system "svn commit -m 'Ignoring all files in /log/ ending in .log'"
  system 'svn propset svn:ignore "*.db" db/'
  system "svn update db/"
  system "svn commit -m 'Ignoring all files in /db/ ending in .db'"
  system "svn move config/database.yml config/database.example"
  system "svn commit -m 'Moving database.yml to database.example'"
  system 'svn propset svn:ignore "database.yml" config/'
  system "svn update config/"
  system "svn commit -m 'Ignoring database.yml'"
  system "svn remove tmp/*"
  system "svn commit -m 'Removing /tmp/ folder'"
  system 'svn propset svn:ignore "*" tmp/'
end
   
desc "Add new files to subversion"
task :add_new_files do
   system "svn status | grep '^\?' | sed\
     -e 's/? *//' | sed -e 's/ /\ /g' | xargs svn add"
end

desc "shortcut for adding new files"
task :add => [ :add_new_files ]


That should get you all cleaned up for the repository.

Enjoy!

0 comments | Filed Under: Scripts | Tags:

textile and ruby on rails

Posted by Jason, Mon Apr 30 06:27:00 UTC 2007

I love textile. For those not up on this yet, textile is a lightweight markup language which converts its marked-up text input to valid, well-formed XHTML. This can be really handy if you are building a site where the user wants some formatting functionality, but doesn't with to write their own XHTML. Recently, I was developing a project that required this and I came across the plugin acts_as_textiled and the even cooler textile editor helper. The combination will allow you to easily set up textile forms in your views simply by installing the plugins and defining acts_as_textiled in your models for the appropriate fields. With the addition of the textile editor helper you can get a "what you see is what you get" UI with all the buttons and functionality to make it look like a regular desktop application.

I'm not doing to write up a howto on this, as it is really that easy. If you follow the links above and check out the sites for both, you will see how easy the setup is, I was really only wanting to point out the textile editor helper as I have found that most have (yet) to hear about it and it really puffs up acts_as_textiled to make a really nice way to incorporate textile into you application.

Enjoy!

0 comments | Filed Under: Plugins | Tags:

Ruby on rails installer script for Mac OSX

Posted by Jason, Sun Mar 25 08:32:00 UTC 2007

I finally made the purchase. After years on Linux, swearing I would never switch, the day has come where I must eat my words and announce to the world how hooked on Apple I have become. When getting my rails environment up on my new machine, thought of how nice an installation script would be kept popping into my mind, hence, I wrote one. Get it here:

mac_ruby_on_rails_installer.sh

You may want to check the versions of things being installed, and replace with whats more current, but, it will give you an idea of an easy way to get started. Enjoy!

Side note, in case your new to unix the way that you would run the script is:

  sudo sh mac_ruby_on_rails_installer.sh

From your administrator level account. Good luck!

2 comments | Filed Under: Scripts | Tags:

How to remove .svn directories from an application

Posted by Jason, Sun Feb 04 00:40:00 UTC 2007

Whenever I notice the same question coming up a few times in the course of a week, I normally consider it a flag to post something on it here. Then, when it comes up again I can just paste over a URL rather than take the time to explain it again. This is one of those flags. In the past week, there have been three people make their way onto IRC asking how to remove all of the .svn directories out of an application. Some had an interest to build this functionality into their application, others just wanted something from command line. I know it seems like a trivial thing to post, but, I figured I have the space...

If you want to build functionality to remove all of the subversion dot directories into your application, this is one of the many ways you could go about accomplishing it:


require 'find'
require 'fileutils'
Find.find('./') do |path|
  if File.basename(path) == '.svn'
    FileUtils.remove_dir(path, true)
    Find.prune
  end
end


If however, you are not a rubiest and are just looking for some Unix approach, this should get the job done:


`find . -type d -name .svn -delete`


FYI, the Unix command line version fails unless all of the .svn directories are already empty. The ruby version does not suffer from this limitation. You could always do something like:


find . -type d -name .svn > svnlist.txt && \
rm -rf `cat svnlist.txt` && rm -rf svnlist.txt
To get around it if you don't mind being a bit of a hack..

0 comments | Filed Under: Hacks | Tags:

acts_as_taggable_on_steroids database schema

Posted by Jason, Sun Jan 28 23:01:00 UTC 2007

I am currently working on a project which requires tagging and i checked out two options for accomplishing this. Acts_as_taggable and acts_as_taggable_on_steroids. From the googling i did, and the folks i talked to on irc that later seems to be the more preferred version, as it is basically a rewrite which also supports tag clouds. The problem is, that they forgot to include the database schema on the plugin website. I ended up finally finding this in the schema that was created from other folks posting their "rake test" output. Anyhow, to save time for others this is what you need:


class AddTagSupport < ActiveRecord::Migration
  def self.up
    create_table :tags, :force => true do |t|
      t.column :name, :string
    end

    create_table :taggings, :force => true do |t|
      t.column :tag_id, :integer
      t.column :taggable_id, :integer
      t.column :taggable_type, :string
      t.column :created_at, :datetime
    end

  end

  def self.down
    drop_table "tags"
    drop_table "taggings"
  end
end



If you do decide to go with the older, more depreciated plugin acts_as_taggable, you can achieve tag clouds as shown here.

2 comments | Filed Under: Plugins | Tags:

Howto add product customization to the Substruct ruby on rails shopping cart

Posted by Jason, Fri Jan 12 10:19:00 UTC 2007

I have been getting a LOT of email from the Substruct mailing list recently from software developers asking how I was able to add product customization to Substruct, the ruby on rails shopping cart. To save myself from repeatedly answering these emails, I thought it would be a great idea to post it here. This way, other fellow ruby-ists can find it via google and the ones that do contact me, I can reply with a URL rather than a solution. Its nice when laziness parallels efficiency.

Obviously, this will vary depending on the application. For the application that I built, the products were displayed with a customize button. If the user did not click the customize button the stock product is submitted for order, if he does however click customize, an ajax drop down is rendered below the product with the customization options. So, I'm sure everyone gets the picture, lets get to the code. I had originally posted the code here, but it seemed like it made the post feel littered. It seems like just linking the files for folks to download is a cleaner approach. So, here ya go:

Inside of my display_product, I have the following code:

display_product.rhtml

These are the link_to_remote statements that set your view up for the customizations functionality. You will of course need to have a div specified for you to ajax your customization options into. Now, create a customize_product.rhtml file and drop in the following code. I will put in some example options to get the point across, but you would most likely want to change this.

customize_product.rhtml

Just for the sake of being complete, I thought I would also include my _cart.rhtml partial. Most of the modifications are pretty simple, and including it is most likely not necessary, but I felt it would be best to include it for clarity sake.

_cart.rhtml

That should finish things off for what should be inside the view.. Yes yes, I know that some of this code needs a bit of polishing to get some of the small things to be done the ruby way, but I've been busy and it will give you the idea.. :) Lets get onto the controller code.

store_controller.rb

And moving right along, lets not forget the models.

cart.rb

order_line_item.rb

I had some people asking about the database, here is my schema file:

schema.rb

So, I most likely overlooked some small details when posting everything as its been a while since I have looked at this project. If anyone has any improvements or such, drop a comment below..

I hope this helps all the others that are getting into a Substruct project and need customization functionality, if I forgot anything post me a comment and I'll get it there.

2 comments | Filed Under: Substruct | Tags: