Archive for the ‘Programming’ Category

A Ruby Module that mixes in Class Methods (static) and Instance Methods

Thursday, February 4th, 2010

Ho ho, this one can catch you out more than once so it’s high time to write a blog post to cover this off. Turns out it’s a commonly used pattern to the rescue. Thanks to eoin on #ruby.ie for pointing to the solution on RailsTips.org.

Here’s quite a tasty diagram too for easy reference.

module Swingable

    def self.included(base)
        base.extend(ClassMethods)
    end

    def instance_swing
        puts 'Did an instance swing!'
    end

    module ClassMethods
        def static_swing
            puts 'Did a static swing!'
        end
    end
end

class BaseballBat
   include Swingable
end

BaseballBat.static_swing
BaseballBat.new.instance_swing

Santa’s got Gems baby! Ruby Ireland Christmas Meetup 2009

Friday, December 18th, 2009

Ho ho ho! The month’s Ruby Ireland meetup sprag right out of the traps with early adopters showing up at 6pm in the lobby area of the Trinity Capital Hotel, Wed Dec 16th. Easing into the evening with a 4 euro pint and talk of Android phones – seemingly the top item of everyone’s Christmas shopping list – the latest crop of gems in the Ruby world was in hot debate, gemcutter in particular.

A couple of folks had been playing around with RubyGame for visualising data as it changes on the fly – showing that this framework is for more than just gaming. The XML/HTML parser Nokogiri was also mentioned a few of times in passing, with the particularly eye-catching quote “XML is like violence – if it doesn’t solve your problems, you are not using enough of it” adorning the home page of its website. And the cracking little tool tig was also brought up, which has a dinky little ncurses interface into git repositories. Pretty cool; not least because it makes it easier for newbies to avoid being bitten when they start git’tin.

The downstairs lobby in the hotel worked out great for people to meet up and relax, with most people turning up at the scheduled 7 o clock for kick off. From there we took over the, what has to be said, pretty classy meeting room complete with old style couches and some Joan Miró paintings. Just in tune with the creative buzz we had going on. There wasn’t too much talk of Ruby for a while as most people were in stunned admiration of the room. Then the food platter arrived. Impressively, this is when everyone showed off their good manners by looking shyly at the platter for a few minutes, with that kind of “You first, sir” glint in their eye, before taking the plunge and sinking into the pakoras and wedges! Pretty much undoing any good work in the gym from earlier in the day!

One of the funnier moments of the night was when someone went to check the tweets against the (now settled upon) #rubyireland hashtag. Only to find lost rubyists tweeting from the hotel lobby as to where the meetup was on. After a quick runaround the lobby to herd anyone wielding a Macbook into the meeting room, the evening was back on track. We split up into a few smaller groups, with the main walk-through being on the qtonrails – a Rails plugin to simply developing applications on Linux and other platforms using Nokia’s Qt framework atop Rails.

To finish off we had a bit of improv comedy from everyone at different closing stages of the evening; in particular Paul O’Malley with his faithful rendition of an emotion beekeeper. And yes now we’re straying off topic so it’s probably time to go. We’ll leave you with Paul’s write up of last night’s shenanigans :-)

Thanks to everyone who showed. Have a great Christmas and catch ye all in Jan 2010 – surely destined to be the decade of Ruby domination!

Ciao,
Dec

Generate Rails Migrations from your PostgreSQL or MySQL database

Thursday, November 26th, 2009

1) Create a new empty Rails project called schemer

2) In your config/database.yml file, point at the database you wish to dump to a migrations file

3) Run the command ‘rake db:schema:dump’. This should create a db/schema.rb file. Amazingly this effectively is your migrations file!

4) To tidy up create a file called file db/migrate/20091125205635_create_initial_schema.rb

5) Then copy the create_table statements from the schema.rb file into the new file 20091125205635_create_initial_schema.rb. Here’s a template

class CreateInitialSchema < ActiveRecord::Migration

  def self.up
    # Put all create_table statements from schema.rb file here
    # Note: You don't need the 'ActiveRecord::Schema.define(:version' line or it's enclosing end statement
    # ...
    # ...
  end

  def self.down
    # Don't really need this
  end

end

6) Once you’ve all this done you can just run ‘rake db:migrate’ and you should have a new sqlite db up and running under db/development.sqlite3

Thanks to Justin Ball on this Nobody Listens Anyway blog at Dump an Existing Database Schema Into a Ruby On Rails Migration Ready Format for the basis of this tip. Sometimes somebody does…

Eight Useful Git Tips

Sunday, November 22nd, 2009

1.) You can check your config using git config -l

This tells you useful stuff like what remotes you’re bound to, etc.

2.) You can blow away files listed as ‘untracked’ (by the git status command) using git clean (be careful!).

If you want any of those files to hang around (but not appear as ‘untracked’) then add them to your  .gitignore file.

3.) You can re-enter your last commit using git commit --amend. What’s nifty is that you can change files (including file addition and deletion) as well as the commit message. Just do what you gotta do before running the amend command and enter a replacement commit message.

4.) You can undo your last commit completely using git reset HEAD^

*** This does not change the working tree ***

Alternatively, you can use the syntax git HEAD~2 instead of HEAD^^

5.) Refer to the last revision by HEAD, the second last as HEAD^ and the third last as HEAD^^

You can keep adding carrets forever! Sounding like Bugs Bunny there.

6.) You can see the contents of a file from a revision using

git show rev:path/to/file

7.) Ok, this tip is a bit of a longer one. Use git merge some_other_branch to merge another branch into the one your working on. (Note that git pull does a merge automatically once it’s completely – if you don’t want this to happen use git fetch). There are a few possible outcomes:

Fast-forward merge : If changes were made on only one of the branches since the last merge, then the merge should happen without any need for you to act.

Three-way merge: If changes were made on both branches, then git decides how to merge them using an algorithm. If any changes conflict with each other, then git will report them and let you resolve them. Once you’ve fixed any conflicts then you can git commit.

If there were no conflicts, git automatically does a commit with a stock commit message for the log. If you don’t like the idea of this then use git merge --no-commit some_other_branch. You need to manually do the commit afterwards. (This is more like how other distributed VCS’s such as Mercurial. This was the tip that inspired this post!).

8.) Finally, one tip that I found useful is that you don’t want to git pull into your repository when you have uncommitted changes. Especially if you treasure your sanity. Instead git stash is your friend. Here’s how
* Do a git stash to move your uncommitted changes to a magical place in git temporarily
* Then do the git pull
* And finally do a git stash apply. This will now move your ’stashed’ uncommitted changes back (from the magical temporary place in git) into your working directory and take into account what what was changed by the pull. Neat!

Tips 1-7 taken from:  Git – SVN Crash Course. Tip 8 taken from painful experience :-)

If you want a good detailed yet friendly reference on a particular command search the Git User Manual on kernel.org.

Getting Contact Form Emailing working with CForms in Wordpress

Friday, November 20th, 2009

“Oh thou horrid little hard-coded piece of Javascript.” That’s our take on the jumping through hoops required to get this little plugin emailing this week. Here’s why…

On uploading the wordpress site from a development machine (which had the cforms plugin installed locally) to a server online, we could not get a contact form created with CForms to send out an email. Instead, it would say “One moment please” but alas that moment would never end. We tried TLS settings and permission changes but nothing worked. Fortunately there was a solution which we came across at the plugin’s somewhat squashed forum.

When we had developed the site locally, the plugin had hardcoded a variable ’sajax_uri’ into the /wp-content/plugins/cforms/js/cforms.js. This is what was causing the pesky problemo and a quick edit of that file to point the variable at a correct URL for our server fixed the issue.

Hmm… I’m sure there’s a good reason is not a variable in a config somewhere. There’s a few hours of my evening I’ll never get back…

Installing Ruby On Rails on Windows

Thursday, November 19th, 2009

There were a couple of great outcomes from the first Free Ruby Lesson we ran in the Havana cafe on Dublin’s Georges St last Monday. The first was getting everyone hacking with the Rails stack and some practical examples in double quick time. The second was managing to get RoR installed on Windows Vista as the lesson rumbled on in the background. Here’s how Rails conquered Vista.

We kicked off by following the instructions on the rails wiki. It’s worth paying careful attention to anywhere it says to set path variables. If you find that you are getting an error that says that you version of rubygems is not recent enough (and you can’t get rubygems to update itself) then you can install an older version of Rails, for example 2.3.2, using the following from the command line

gem install rails -v 2.3.2

(Note: If you have already installed a different version of rails you can uninstall it first by using ‘gem uninstall rails’).

The big problem however was getting sqlite3 working. We did install the SQLite Command Line Tool and the SQLite DLL as the Rails Wiki instructions said to – but to no avail. We kept getting a popup error saying that the sqlite3 dll was not found and to please consider reinstalling. Fortunately, we gambled on one of the answers on StackOverflow. Basically the solution was from the command line

gem uninstall sqlite3-ruby
gem install sqlite3-ruby --source http://gems.rubyinstaller.org

And with that we were away! The final thing was to get a nice friendly IDE installed so we plumbed for Netbeans on the Netbeans Downloads page. Look out for the special Ruby version of Netbeans which is one of the links around the middle of the downloads page.

Anyway, that’s all for this week. Happy Ruby hacking!

Understanding how Ruby stores objects in memory – the Ruby Heap

Thursday, October 29th, 2009

Ruby has it’s own heap management which actually consists of several ‘Ruby Heaps’ to manage objects created during the execution of a Ruby program; this is separate from the System Heap for your Operating System. Each individual Ruby Heap contains Slots, with each Slot able to one reference one object.

The entire space that an object takes up in memory ***is not stored inside the Slot***. Rather each Slot is a small fixed size space which can be thought of as the Ruby interpreter’s handle a location in memory. This location exists outside of the Ruby Heap itself and contains the real ‘meat’ of the object. To be clear, if you have a 50MB string – the 50MB of data is stored outside of Ruby’s Heap. If you really want to know the story of the 50MB, the space for it is actually allocated by something like the malloc command in C (as good ol’ Ruby is written in C) and then stored on the System Heap. The Slot in the Ruby Heap simply contains a reference to that memory location on the System Heap which contains the 50MB of data.

Here’s an example. Let’s say that a Ruby program creates a single string of 50MB
* A single free Slot in a Ruby Heap becomes filled
* Memory to store the 50MB of data that makes up the string itself is allocated in memory and put on the System Heap (outside the Ruby Heap!) and a reference to this location is stored in the Filled Slot on the Ruby Heap
* There comes a point in time when this string is no longer needed. This slot is garbage collected on the next GC iteration
* The Filled Slot is turned into a free slot. The 50MB of data in memory referred to by the slot is also freed and returned to the Operating System

Ruby starts of with a minimal set of Ruby Heaps. These are managed by by a Ruby Heap list. Ruby creates Ruby Heaps when needed and frees Ruby Heaps back to the OS when no longer needed (the latter is done in a sub-optimal manner – more on this later). Each Ruby Heap created will be 1.8 times the size of the previous heap. In other words, it will contain 1.8 times the number of slots of the previous heap. Ruby’s Garbage Collector, periodically iterates through the Ruby Heaps and frees up any Slots as appropriate (and also the memory that an object really occupies which is referenced by the Slot – ie. the 50MB data of the String) back to the system. Once a GC iteration is complete, some of the Slots that were filled will now be empty – known as Free Slots. Remember that we said that Ruby’s Heap management actually consists of many Ruby Heaps. Well if one of these Ruby Heaps consists of only Free Slots then the Ruby Heap itself will be freed back to the Operating System.

There is a problem with this last statement however – if a Ruby Heap contains mostly Free Slots and one Filled Slot then it will not be freed. You could have many Ruby Heaps in this state. As long as a Ruby Heap contains even one Filled Slot it will not be returned to the Operating System. It just takes one bad apple to spoil everything! What would be nice is if some sort of Heap Compaction (kind of like disk fragmentation) took place where all Filled slots were pushed together into completed filled Ruby Heaps. This would leave you with completely filled Ruby Heaps, one semi-filled Ruby Heap and then a bunch of completely empty Ruby Heaps. The completely empty Ruby Heaps could then be freed, releasing precious memory back to the Operating System. Alas the current mainstream Ruby interpreter does not do this.

References
* How the Ruby Heap is Implemented Phusion Passenger’s Hong Lai gives a great explanation of the Ruby Heap – the banner may not be quite suitable for work. Fortunately, there’s a censor button :-)

* Fine tuning your garbage collector Chris Heald explains some of the settings around garbage collection

* Ruby’s Garbage Collections effect on Ruby on Rails Pluron Inc’s blog discusses so of the knock-on effects of Ruby GC on Rails and importantly mentions the 8 MB memory allocation tigger for the garbage collector

Bleak House – A Tool for measuring Objects in Memory for a Ruby Program

Wednesday, October 28th, 2009

Bleak House is a tool that tells us
- How many Slots there are in total at a point in time in a Ruby program
- How many Slots in total are filled
- How many Slots in total are empty (free)
- How many Filled Slots can be attributed to a particular line of code

Bleakhouse can be used to tell you if program is holding on to objects that it should be relinquishing. But it doesn’t tell you how much data is stored in memory for the ‘meat’ of the object (ie. that 50MB of data in a 50MB String). Just because you know there is a Filled Slot exists – you don’t know if the data in memory that correlates back to that Slot is 1MB, 10MB or 100MB.

However, if you repeat a series of a specific set of operations a small number of times, measuring with Bleakhouse, and then restart the server with Bleak house and repeat the operations a large number of times and see a big difference in the number of filled slots can tell that your program is holding onto objects (references) that it should not. Of course, if your program is supposed to keep hold of an increasing number of references (such as a global variable or a singleton that keeps accumulating references for the duration of your program) then this would be expected. Though you might want to double check your design. You will be able to see the cause of the problem from the detailed breakdown of which lines of code were the biggest offenders in terms of creating objects. If you see a large number of free slots (relative to the number of filled slots) then this means that at some point in your program a lot of objects existed (possibly due to a spike in application usage) but then reduced.

Does the free slots count matter? Well, yes because there is an memory overhead due to each free slot that exists – how much depends on your particular system. If your system has a slot size of 20 bytes then every one million free slots costs you an additional 20MB that is not being utilised. This becomes a problem if your application is subject to large but infrequent spikes in the number of objects that exist within your program a particular moment in time because the free slots are taking up significant amounts of memory even when your application is twiddling its thumbs between the spikes.

Open Letter to the Irish Government on Open Source Driven Innovation

Friday, September 18th, 2009

“Recent years show that openness and collaboration is essential to the generation of innovation in the software sector. Technology increasingly means software. In Ireland, we can see that the production of hardware technology in many, but not all, cases is providing ever diminishing returns. Here we outline some key policy recommendations that are crucial to the fulfilling the vision of making the Irish Smart Economy a reality for the software industry through the adoption and encouragement of Open Source technologies.”

The above extract is from a paper we are submitting to the Innovation Taskforce as requested by the Department of Taoiseach. The draft paper is available at Positioning Ireland as an International Innovation Hub

Please note, we are submitting the paper ahead of the deadline which is Friday the 18th of September. We appreciate any feedback, support or criticisms you may have. Please post them as comments below.

Simple straight up caching for pages served by Heroku

Wednesday, September 16th, 2009

So you’ve got an app that’s ticking along nicely; being served up a good steak in a 5 star restaurant – but you’d like to boost it’s performance with some caching. For those who develop their apps on the Heroku platform, a great way to do this is to cache a dynamic page using Varnish. This means that your page is served up super fast without hitting Rails/Sinatra/whatever. And best of all it requires no extra gems or anything, just a well placed one-liner in your controller.

Firstly, you can only use this technique if all users that visit this page expect to see the exact same content – in other words you have no ‘per user’ customised content on a page. To help understand how this type of caching works, imagine that the first time your page (let’s say an Events index page) is hit it is turned into a static html page for a pre-defined amount of time (let’s say 60 seconds). Anyone else who visit this page (ie. anyone else who visits this particular controller action) during the next 60 seconds gets that static html page. After the 60 seconds the static html page is removed from the cache. Thus the next hit will cause your underlying dynamic page to be invoked; then the caching process kicks off again lasting another 60 seconds. And so on and so fourth.

With the increasing amount of web applications that call APIs, such as Twitter’s API, this is a really easy way to ensure that you do not end up spamming a service provider with an unreasonable number of calls per hour. This is the technique we use on www.thelisbontweety.com to keep our API overhead down.

So how do you do this? Simply put something along the lines of

response.headers['Cache-Control'] = ‘public, max-age=60′

as the first line of your action for the page you wish to cache. The max-age setting means that this will be cached for 60 seconds. After you put this in your application and redeploy to Heroku, you can see if it’s working by using http://hurl.it

Just enter the  URL for your action and click Send. You should see something like “Cache-Control: max-age=60, public” in the output if it’s working.

And that’s it! No need to install anything. Just cache your little heart out with Varnish. Top marks to chaps at Heroku for making this so easy to use out of the box at Heroku. For more on this technique check out their HTTP caching docs at http://docs.heroku.com/http-caching