Archive for the ‘Rubyonrails’ Category

A Ruby Module that mixes in Class Methods (static) and Instance Methods

Thursday, February 4th, 2010

Ho ho, this one can catch you out more than once so it’s high time to write a blog post to cover this off. Turns out it’s a commonly used pattern to the rescue. Thanks to eoin on #ruby.ie for pointing to the solution on RailsTips.org.

Here’s quite a tasty diagram too for easy reference.

module Swingable

    def self.included(base)
        base.extend(ClassMethods)
    end

    def instance_swing
        puts 'Did an instance swing!'
    end

    module ClassMethods
        def static_swing
            puts 'Did a static swing!'
        end
    end
end

class BaseballBat
   include Swingable
end

BaseballBat.static_swing
BaseballBat.new.instance_swing

Santa’s got Gems baby! Ruby Ireland Christmas Meetup 2009

Friday, December 18th, 2009

Ho ho ho! The month’s Ruby Ireland meetup sprag right out of the traps with early adopters showing up at 6pm in the lobby area of the Trinity Capital Hotel, Wed Dec 16th. Easing into the evening with a 4 euro pint and talk of Android phones – seemingly the top item of everyone’s Christmas shopping list – the latest crop of gems in the Ruby world was in hot debate, gemcutter in particular.

A couple of folks had been playing around with RubyGame for visualising data as it changes on the fly – showing that this framework is for more than just gaming. The XML/HTML parser Nokogiri was also mentioned a few of times in passing, with the particularly eye-catching quote “XML is like violence – if it doesn’t solve your problems, you are not using enough of it” adorning the home page of its website. And the cracking little tool tig was also brought up, which has a dinky little ncurses interface into git repositories. Pretty cool; not least because it makes it easier for newbies to avoid being bitten when they start git’tin.

The downstairs lobby in the hotel worked out great for people to meet up and relax, with most people turning up at the scheduled 7 o clock for kick off. From there we took over the, what has to be said, pretty classy meeting room complete with old style couches and some Joan Miró paintings. Just in tune with the creative buzz we had going on. There wasn’t too much talk of Ruby for a while as most people were in stunned admiration of the room. Then the food platter arrived. Impressively, this is when everyone showed off their good manners by looking shyly at the platter for a few minutes, with that kind of “You first, sir” glint in their eye, before taking the plunge and sinking into the pakoras and wedges! Pretty much undoing any good work in the gym from earlier in the day!

One of the funnier moments of the night was when someone went to check the tweets against the (now settled upon) #rubyireland hashtag. Only to find lost rubyists tweeting from the hotel lobby as to where the meetup was on. After a quick runaround the lobby to herd anyone wielding a Macbook into the meeting room, the evening was back on track. We split up into a few smaller groups, with the main walk-through being on the qtonrails – a Rails plugin to simply developing applications on Linux and other platforms using Nokia’s Qt framework atop Rails.

To finish off we had a bit of improv comedy from everyone at different closing stages of the evening; in particular Paul O’Malley with his faithful rendition of an emotion beekeeper. And yes now we’re straying off topic so it’s probably time to go. We’ll leave you with Paul’s write up of last night’s shenanigans :-)

Thanks to everyone who showed. Have a great Christmas and catch ye all in Jan 2010 – surely destined to be the decade of Ruby domination!

Ciao,
Dec

Generate Rails Migrations from your PostgreSQL or MySQL database

Thursday, November 26th, 2009

1) Create a new empty Rails project called schemer

2) In your config/database.yml file, point at the database you wish to dump to a migrations file

3) Run the command ‘rake db:schema:dump’. This should create a db/schema.rb file. Amazingly this effectively is your migrations file!

4) To tidy up create a file called file db/migrate/20091125205635_create_initial_schema.rb

5) Then copy the create_table statements from the schema.rb file into the new file 20091125205635_create_initial_schema.rb. Here’s a template

class CreateInitialSchema < ActiveRecord::Migration

  def self.up
    # Put all create_table statements from schema.rb file here
    # Note: You don't need the 'ActiveRecord::Schema.define(:version' line or it's enclosing end statement
    # ...
    # ...
  end

  def self.down
    # Don't really need this
  end

end

6) Once you’ve all this done you can just run ‘rake db:migrate’ and you should have a new sqlite db up and running under db/development.sqlite3

Thanks to Justin Ball on this Nobody Listens Anyway blog at Dump an Existing Database Schema Into a Ruby On Rails Migration Ready Format for the basis of this tip. Sometimes somebody does…

Understanding how Ruby stores objects in memory – the Ruby Heap

Thursday, October 29th, 2009

Ruby has it’s own heap management which actually consists of several ‘Ruby Heaps’ to manage objects created during the execution of a Ruby program; this is separate from the System Heap for your Operating System. Each individual Ruby Heap contains Slots, with each Slot able to one reference one object.

The entire space that an object takes up in memory ***is not stored inside the Slot***. Rather each Slot is a small fixed size space which can be thought of as the Ruby interpreter’s handle a location in memory. This location exists outside of the Ruby Heap itself and contains the real ‘meat’ of the object. To be clear, if you have a 50MB string – the 50MB of data is stored outside of Ruby’s Heap. If you really want to know the story of the 50MB, the space for it is actually allocated by something like the malloc command in C (as good ol’ Ruby is written in C) and then stored on the System Heap. The Slot in the Ruby Heap simply contains a reference to that memory location on the System Heap which contains the 50MB of data.

Here’s an example. Let’s say that a Ruby program creates a single string of 50MB
* A single free Slot in a Ruby Heap becomes filled
* Memory to store the 50MB of data that makes up the string itself is allocated in memory and put on the System Heap (outside the Ruby Heap!) and a reference to this location is stored in the Filled Slot on the Ruby Heap
* There comes a point in time when this string is no longer needed. This slot is garbage collected on the next GC iteration
* The Filled Slot is turned into a free slot. The 50MB of data in memory referred to by the slot is also freed and returned to the Operating System

Ruby starts of with a minimal set of Ruby Heaps. These are managed by by a Ruby Heap list. Ruby creates Ruby Heaps when needed and frees Ruby Heaps back to the OS when no longer needed (the latter is done in a sub-optimal manner – more on this later). Each Ruby Heap created will be 1.8 times the size of the previous heap. In other words, it will contain 1.8 times the number of slots of the previous heap. Ruby’s Garbage Collector, periodically iterates through the Ruby Heaps and frees up any Slots as appropriate (and also the memory that an object really occupies which is referenced by the Slot – ie. the 50MB data of the String) back to the system. Once a GC iteration is complete, some of the Slots that were filled will now be empty – known as Free Slots. Remember that we said that Ruby’s Heap management actually consists of many Ruby Heaps. Well if one of these Ruby Heaps consists of only Free Slots then the Ruby Heap itself will be freed back to the Operating System.

There is a problem with this last statement however – if a Ruby Heap contains mostly Free Slots and one Filled Slot then it will not be freed. You could have many Ruby Heaps in this state. As long as a Ruby Heap contains even one Filled Slot it will not be returned to the Operating System. It just takes one bad apple to spoil everything! What would be nice is if some sort of Heap Compaction (kind of like disk fragmentation) took place where all Filled slots were pushed together into completed filled Ruby Heaps. This would leave you with completely filled Ruby Heaps, one semi-filled Ruby Heap and then a bunch of completely empty Ruby Heaps. The completely empty Ruby Heaps could then be freed, releasing precious memory back to the Operating System. Alas the current mainstream Ruby interpreter does not do this.

References
* How the Ruby Heap is Implemented Phusion Passenger’s Hong Lai gives a great explanation of the Ruby Heap – the banner may not be quite suitable for work. Fortunately, there’s a censor button :-)

* Fine tuning your garbage collector Chris Heald explains some of the settings around garbage collection

* Ruby’s Garbage Collections effect on Ruby on Rails Pluron Inc’s blog discusses so of the knock-on effects of Ruby GC on Rails and importantly mentions the 8 MB memory allocation tigger for the garbage collector

Bleak House – A Tool for measuring Objects in Memory for a Ruby Program

Wednesday, October 28th, 2009

Bleak House is a tool that tells us
- How many Slots there are in total at a point in time in a Ruby program
- How many Slots in total are filled
- How many Slots in total are empty (free)
- How many Filled Slots can be attributed to a particular line of code

Bleakhouse can be used to tell you if program is holding on to objects that it should be relinquishing. But it doesn’t tell you how much data is stored in memory for the ‘meat’ of the object (ie. that 50MB of data in a 50MB String). Just because you know there is a Filled Slot exists – you don’t know if the data in memory that correlates back to that Slot is 1MB, 10MB or 100MB.

However, if you repeat a series of a specific set of operations a small number of times, measuring with Bleakhouse, and then restart the server with Bleak house and repeat the operations a large number of times and see a big difference in the number of filled slots can tell that your program is holding onto objects (references) that it should not. Of course, if your program is supposed to keep hold of an increasing number of references (such as a global variable or a singleton that keeps accumulating references for the duration of your program) then this would be expected. Though you might want to double check your design. You will be able to see the cause of the problem from the detailed breakdown of which lines of code were the biggest offenders in terms of creating objects. If you see a large number of free slots (relative to the number of filled slots) then this means that at some point in your program a lot of objects existed (possibly due to a spike in application usage) but then reduced.

Does the free slots count matter? Well, yes because there is an memory overhead due to each free slot that exists – how much depends on your particular system. If your system has a slot size of 20 bytes then every one million free slots costs you an additional 20MB that is not being utilised. This becomes a problem if your application is subject to large but infrequent spikes in the number of objects that exist within your program a particular moment in time because the free slots are taking up significant amounts of memory even when your application is twiddling its thumbs between the spikes.

Simple straight up caching for pages served by Heroku

Wednesday, September 16th, 2009

So you’ve got an app that’s ticking along nicely; being served up a good steak in a 5 star restaurant – but you’d like to boost it’s performance with some caching. For those who develop their apps on the Heroku platform, a great way to do this is to cache a dynamic page using Varnish. This means that your page is served up super fast without hitting Rails/Sinatra/whatever. And best of all it requires no extra gems or anything, just a well placed one-liner in your controller.

Firstly, you can only use this technique if all users that visit this page expect to see the exact same content – in other words you have no ‘per user’ customised content on a page. To help understand how this type of caching works, imagine that the first time your page (let’s say an Events index page) is hit it is turned into a static html page for a pre-defined amount of time (let’s say 60 seconds). Anyone else who visit this page (ie. anyone else who visits this particular controller action) during the next 60 seconds gets that static html page. After the 60 seconds the static html page is removed from the cache. Thus the next hit will cause your underlying dynamic page to be invoked; then the caching process kicks off again lasting another 60 seconds. And so on and so fourth.

With the increasing amount of web applications that call APIs, such as Twitter’s API, this is a really easy way to ensure that you do not end up spamming a service provider with an unreasonable number of calls per hour. This is the technique we use on www.thelisbontweety.com to keep our API overhead down.

So how do you do this? Simply put something along the lines of

response.headers['Cache-Control'] = ‘public, max-age=60′

as the first line of your action for the page you wish to cache. The max-age setting means that this will be cached for 60 seconds. After you put this in your application and redeploy to Heroku, you can see if it’s working by using http://hurl.it

Just enter the  URL for your action and click Send. You should see something like “Cache-Control: max-age=60, public” in the output if it’s working.

And that’s it! No need to install anything. Just cache your little heart out with Varnish. Top marks to chaps at Heroku for making this so easy to use out of the box at Heroku. For more on this technique check out their HTTP caching docs at http://docs.heroku.com/http-caching

Packaging Ruby Apps for Ubuntu: Dissecting an existing Ruby Ubuntu Package

Wednesday, September 9th, 2009

One of the best ways to learn about how a Ubuntu package is put together is reverse engineer the package into it’s constituent components. We are going to take a look at how to do this for the chef application and it’s related libchef library is packaged as a Debian package.

* Visit the page http://packages.ubuntu.com/karmic/ruby/chef
* Under the Download chef section, download the package via the ‘All’ link into a directory called chef
* Visit the page http://packages.ubuntu.com/karmic/ruby/libchef-ruby1.8
* Under the Download libchef-ruby1.8 section, download the package via the ‘All’ link into a directory called libchef1.8

From the following guide (http://www.g-loaded.eu/2008/01/28/how-to-extract-rpm-or-deb-packages) you can learn how to ‘unzip’ a Debian package. This is easy as they are pure ar archives. Here’s what we need to do

* In the chef directory, run the commands

ar vx chef_0.7.8-0ubuntu2_all.deb
tar -zxvf data.tar.gz

* In the libchef1.8 directory, run the commands

ar vx libchef-ruby_0.7.8-0ubuntu2_all.deb
tar -zxvf data.tar.gz

Now you can study the layout of the of the data payload of the package (this is where to look in order to study the anatomy of the application as it was being packaged). This layout is what will be of most interest to you.

If you have an application in a particular programming language that you wish to package, pick a similar application for which a package already exists and dissect it as shown above. Then bend your app into a similar shape in terms of directory layout before attempting to package it. To find out more about how to create your own Ubuntu packages check out this great video by Horst Jens Ubuntu: Making a .deb package out of a python program. It’s worth the effort of watching it to the end!

Happy packaging!

A Breath of Fresh Air – The Well Grounded Rubyist

Tuesday, June 9th, 2009

After a tough day in the office you want to catch up on the news, so you look at the ticker on a TV channel or tune in your car radio. Other days you’ll want to sit down with a meaty broadsheet and really take in the detail of what lies behind the headlines. This is a book about Ruby which which triumphs at walking the line between these two styles. The Well Grounded Rubyist aims to appeal to a developer that has been exposed to some Ruby coding and take their knowledge to the next level. And it succeeds brilliantly.

This is not a book about Rails or any other web frameworks; purely Ruby. Though much of the material will also apply to the 1.8.x series of Ruby, this is a book about the 1.9 version of the Ruby language. It’s broken into three parts – Ruby foundations, Built-in classes and modules and finally Ruby dynamics. But don’t let the title of Ruby foundations fool you for part one – this is not some remedial rush through the basics of Ruby in six chapters. Rather, after a couple of warm-up chapters, it moves quickly to clarify the key aspects of how classes and module inter-relate, as well things such as crystallising what ’self’ really means in different contexts in a Ruby program. The author sets out his stall early – what makes Ruby different from other languages is it’s focus on objects rather than classes. Everything else stems from this and by the end of the section you feel like you have an understanding of Ruby’s design and focus.

Part two of the book is Built-in classes and modules. Now that you know what makes Ruby tick, it’s time to get seeped in all aspects of the core library that ships with it. One of the problems when learning a language is that becoming familiar with all methods of a particular core class is a tedious task. It’s much more interesting to learn about concepts such as meta programming than memorising lists of methods by rote. But if you don’t take the time to familiarise yourself with the dusty corners of a language’s API then you’re less likely to think of those handy methods when a problem they would elegantly solve presents itself. At this point the book shifts gear to a more reference style of text. However, it still gives the reader an interesting story to follow as it documents arrays, hashes and other classes – throwing in the occasional golden nugget of information that will be a valuable additional to the toolbox of even experienced Rubyists. One side effect of the change in style is that this section is probably the most accessible to beginners. Again it’s broken down into six chapters. In addition to collections, it also covers topics such as regular expressions and file handling. Each topic takes a zero-to-hero approach meaning that you can bring little regular expression knowledge to the table yet still walk away learning an immense amount about the subject.

The final part of the book, Ruby dynamics, returns to the book’s roots from part one – a focus on imparting a deep knowledge of Ruby’s design. Before you even pick up this book you have an inkling that procs and lamdas are going to make a guest appearance at this late stage. And they do not disappoint. Extending the behaviour of objects takes centre-stage and meta-programming based techniques move quickly to the fore. Any block/proc/lamda confusion you may have will be a distant memory by the time you finish this section. Threading is also covered here – though a detailed discussion of 1.9’s new native OS threads vs green threads is left to one side to focus on the usage of threads regardless of which underlying type you use. Lots of material is also provided on querying objects; which is not only useful for program design but also invaluable as a debugging aid. The book really shines in this section because a lot of other texts make the mistake of going into ’super-boffin’ mode at this point, leaving the reader lost, whereas the author here continues to provide patience and context to get you round that final lap on you way to becoming a Well Grounded Rubyist!

This book cannot be all things to all people. Because it is catering for a wide range of intermediate to advanced Rubyists, it will feel like it’s moving a little too slowly for some. By the author’s own admission, this is in order to make it accessible to a wider audience and no doubt it will make it easier for developers of all levels to digest – your humble reviewer very much included! Ruby first-timers would be best off having some straight-to-the-point tutorials or entry-level text to hand in order to get some instant gratification – as part one of the book, by it’s very nature, is a little more abstract than a complete beginner would expect. But all in all, this book is a great way to learn just how Ruby crams so much expressiveness into such a simple clean framework. Whenever I read a book like this I keep a list of new things learned along the way. For The Well Grounded Rubyist it is a very long list! Well done to David A. Black and Manning for producing a book that fills those gaps in many Rubyists’ understanding of the language while at the same time delivering an absorbing readable book that would sit proudly on any Ruby programmer’s bookshelf.

Optimising Apache serving Ruby on Rails via Passenger

Monday, May 11th, 2009

An unusual blog post this in that I have such little value to add other than pointing you at two great links. The first is a wonderfully useful post on tweaking Phusion Passenger when running some Rails apps on a 256MB Slicehost VPS and the other link is a no-nonsense explanation of tweaking Apache config options to cut down on your swap memory usage. Thanks so much to these folks!

Phusion Passenger Configuration File Location – passenger.conf

Wednesday, March 4th, 2009

This was not easy for me to find! My beloved Ruby on Rails apps just sitting there untweaked! Google and even the Passenger website didn’t specify where the config file may be found. And eventually it made a startling difference to my apps performance – possibly given that I’m trying to host a few sites at once – so it would be nice if it was easier to learn about how to setup the config file when new to Passenger,

Out of the box on a vanilla passenger install there is no passenger configuration file. However, there are a lots of configuration options and you can either dump these into

a) /etc/apache2/apache2.conf file (messy) or
b) Create a new file called passenger.conf under the /etc/apache2/conf.d directory as files in thei directory automatically get loaded by apache
c) Create a passenger.load file in mods-available and then enable the module

By the way, this setup is on Ubuntu, your mileage may vary depending on your distro. For an example, passenger.conf file see this one, just be sure to get the ‘LoadModule passenger_module’, ‘PassengerRoot’ and ‘PassengerRuby’ settings correct. You may already have these specified correctly in your apache2.conf file if you already have passenger working; if so you can reuse these values.

Apparently the recommended PassengerMaxPoolSize is 2 if you’re on a 256 MB Virtual Private Server System that’s running things like MySQL and keep the PassengerMaxInstancesPerApp smaller than this. If you’re running a couple of rails sites then maybe set PassengerMaxInstancesPerApp to 1 if you want to have 1 instance always available for each site. A RailsSpawnMethod of ’smart’ can also lead to better performance depending on your setup.

Certainly playing around with this config file helped me greatly improve my sites’ responsiveness for little effort and in particular it got the usage of the all important swap space down.

One final tip for today is to install htop (is available through aptitude). This gives a very useful and pretty display of the ‘top’ commands information and can make monitoring things like memory and swap at a glance much easier. So armed with this knowledge go forth and spawn! Well, till you run out of memory at least!