Santa’s got Gems baby! Ruby Ireland Christmas Meetup 2009
Ho ho ho! The month's Ruby Ireland meetup sprag right out of the traps with early adopters showing up at 6pm in the lobby area of the Trinity Capital Hotel, Wed Dec 16th. Easing into the evening with a 4 euro pint and talk of Android phones - seemingly the top item of everyone's Christmas shopping list - the latest crop of gems in the Ruby world was in hot debate, gemcutter in particular.
A couple of folks had been playing around with RubyGame for visualising data as it changes on the fly - showing that this framework is for more than just gaming. The XML/HTML parser Nokogiri was also mentioned a few of times in passing, with the particularly eye-catching quote "XML is like violence - if it doesn’t solve your problems, you are not using enough of it" adorning the home page of its website. And the cracking little tool tig was also brought up, which has a dinky little ncurses interface into git repositories. Pretty cool; not least because it makes it easier for newbies to avoid being bitten when they start git'tin.
The downstairs lobby in the hotel worked out great for people to meet up and relax, with most people turning up at the scheduled 7 o clock for kick off. From there we took over the, what has to be said, pretty classy meeting room complete with old style couches and some Joan Miró paintings. Just in tune with the creative buzz we had going on. There wasn't too much talk of Ruby for a while as most people were in stunned admiration of the room. Then the food platter arrived. Impressively, this is when everyone showed off their good manners by looking shyly at the platter for a few minutes, with that kind of "You first, sir" glint in their eye, before taking the plunge and sinking into the pakoras and wedges! Pretty much undoing any good work in the gym from earlier in the day!
One of the funnier moments of the night was when someone went to check the tweets against the (now settled upon) #rubyireland hashtag. Only to find lost rubyists tweeting from the hotel lobby as to where the meetup was on. After a quick runaround the lobby to herd anyone wielding a Macbook into the meeting room, the evening was back on track. We split up into a few smaller groups, with the main walk-through being on the qtonrails - a Rails plugin to simply developing applications on Linux and other platforms using Nokia's Qt framework atop Rails.
To finish off we had a bit of improv comedy from everyone at different closing stages of the evening; in particular Paul O'Malley with his faithful rendition of an emotion beekeeper. And yes now we're straying off topic so it's probably time to go. We'll leave you with Paul's write up of last night's shenanigans 
Thanks to everyone who showed. Have a great Christmas and catch ye all in Jan 2010 - surely destined to be the decade of Ruby domination!
Ciao,
Dec
Generate Rails Migrations from your PostgreSQL or MySQL database
1) Create a new empty Rails project called schemer
2) In your config/database.yml file, point at the database you wish to dump to a migrations file
3) Run the command 'rake db:schema:dump'. This should create a db/schema.rb file. Amazingly this effectively is your migrations file!
4) To tidy up create a file called file db/migrate/20091125205635_create_initial_schema.rb
5) Then copy the create_table statements from the schema.rb file into the new file 20091125205635_create_initial_schema.rb. Here's a template
class CreateInitialSchema < ActiveRecord::Migration
def self.up
# Put all create_table statements from schema.rb file here
# Note: You don't need the 'ActiveRecord::Schema.define(:version' line or it's enclosing end statement
# ...
# ...
end
def self.down
# Don't really need this
end
end
6) Once you've all this done you can just run 'rake db:migrate' and you should have a new sqlite db up and running under db/development.sqlite3
Thanks to Justin Ball on this Nobody Listens Anyway blog at Dump an Existing Database Schema Into a Ruby On Rails Migration Ready Format for the basis of this tip. Sometimes somebody does...
Eight Useful Git Tips
1.) You can check your config using git config -l
This tells you useful stuff like what remotes you're bound to, etc.
2.) You can blow away files listed as 'untracked' (by the git status command) using git clean (be careful!).
If you want any of those files to hang around (but not appear as 'untracked') then add them to your .gitignore file.
3.) You can re-enter your last commit using git commit --amend. What's nifty is that you can change files (including file addition and deletion) as well as the commit message. Just do what you gotta do before running the amend command and enter a replacement commit message.
4.) You can undo your last commit completely using git reset HEAD^
*** This does not change the working tree ***
Alternatively, you can use the syntax git HEAD~2 instead of HEAD^^
5.) Refer to the last revision by HEAD, the second last as HEAD^ and the third last as HEAD^^
You can keep adding carrets forever! Sounding like Bugs Bunny there.
6.) You can see the contents of a file from a revision using
git show rev:path/to/file
7.) Ok, this tip is a bit of a longer one. Use git merge some_other_branch to merge another branch into the one your working on. (Note that git pull does a merge automatically once it's completely - if you don't want this to happen use git fetch). There are a few possible outcomes:
Fast-forward merge : If changes were made on only one of the branches since the last merge, then the merge should happen without any need for you to act.
Three-way merge: If changes were made on both branches, then git decides how to merge them using an algorithm. If any changes conflict with each other, then git will report them and let you resolve them. Once you've fixed any conflicts then you can git commit.
If there were no conflicts, git automatically does a commit with a stock commit message for the log. If you don't like the idea of this then use git merge --no-commit some_other_branch. You need to manually do the commit afterwards. (This is more like how other distributed VCS's such as Mercurial. This was the tip that inspired this post!).
8.) Finally, one tip that I found useful is that you don't want to git pull into your repository when you have uncommitted changes. Especially if you treasure your sanity. Instead git stash is your friend. Here's how
* Do a git stash to move your uncommitted changes to a magical place in git temporarily
* Then do the git pull
* And finally do a git stash apply. This will now move your 'stashed' uncommitted changes back (from the magical temporary place in git) into your working directory and take into account what what was changed by the pull. Neat!
Tips 1-7 taken from: Git - SVN Crash Course. Tip 8 taken from painful experience
If you want a good detailed yet friendly reference on a particular command search the Git User Manual on kernel.org.
Getting Contact Form Emailing working with CForms in WordPress
"Oh thou horrid little hard-coded piece of Javascript." That's our take on the jumping through hoops required to get this little plugin emailing this week. Here's why...
On uploading the wordpress site from a development machine (which had the cforms plugin installed locally) to a server online, we could not get a contact form created with CForms to send out an email. Instead, it would say "One moment please" but alas that moment would never end. We tried TLS settings and permission changes but nothing worked. Fortunately there was a solution which we came across at the plugin's somewhat squashed forum.
When we had developed the site locally, the plugin had hardcoded a variable 'sajax_uri' into the /wp-content/plugins/cforms/js/cforms.js. This is what was causing the pesky problemo and a quick edit of that file to point the variable at a correct URL for our server fixed the issue.
Hmm... I'm sure there's a good reason is not a variable in a config somewhere. There's a few hours of my evening I'll never get back...
Installing Ruby On Rails on Windows
There were a couple of great outcomes from the first Free Ruby Lesson we ran in the Havana cafe on Dublin's Georges St last Monday. The first was getting everyone hacking with the Rails stack and some practical examples in double quick time. The second was managing to get RoR installed on Windows Vista as the lesson rumbled on in the background. Here's how Rails conquered Vista.
We kicked off by following the instructions on the rails wiki. It's worth paying careful attention to anywhere it says to set path variables. If you find that you are getting an error that says that you version of rubygems is not recent enough (and you can't get rubygems to update itself) then you can install an older version of Rails, for example 2.3.2, using the following from the command line
gem install rails -v 2.3.2
(Note: If you have already installed a different version of rails you can uninstall it first by using 'gem uninstall rails').
The big problem however was getting sqlite3 working. We did install the SQLite Command Line Tool and the SQLite DLL as the Rails Wiki instructions said to - but to no avail. We kept getting a popup error saying that the sqlite3 dll was not found and to please consider reinstalling. Fortunately, we gambled on one of the answers on StackOverflow. Basically the solution was from the command line
gem uninstall sqlite3-ruby
gem install sqlite3-ruby --source http://gems.rubyinstaller.org
And with that we were away! The final thing was to get a nice friendly IDE installed so we plumbed for Netbeans on the Netbeans Downloads page. Look out for the special Ruby version of Netbeans which is one of the links around the middle of the downloads page.
Anyway, that's all for this week. Happy Ruby hacking!
Offline Documentation for Rails (and other Ruby gems)
I used to pretty much always install gems with the --no-ri --no-rdoc options to speed up installation. Recently however, I've found myself needing to get access to documentation whilst on the move. So just in case you don't know how that works, here's the deal.
* Install your gems as per normal (ie. don't use --no-ri or --no-rdoc)
* Run the command 'gem server' from the command line
* Browse to http://localhost:8808
* And voila! You should have all the docs you need available by clicking on the rdoc link for any given gem
* But if you really want to get fancy check out the searchable Rails documentation at http://railsapi.com - there's an online version as well as the downloadable (the links are at the top of the page and can easily be mistaken for an advert!)
Understanding how Ruby stores objects in memory – the Ruby Heap
Ruby has it's own heap management which actually consists of several 'Ruby Heaps' to manage objects created during the execution of a Ruby program; this is separate from the System Heap for your Operating System. Each individual Ruby Heap contains Slots, with each Slot able to one reference one object.
The entire space that an object takes up in memory ***is not stored inside the Slot***. Rather each Slot is a small fixed size space which can be thought of as the Ruby interpreter's handle a location in memory. This location exists outside of the Ruby Heap itself and contains the real 'meat' of the object. To be clear, if you have a 50MB string - the 50MB of data is stored outside of Ruby's Heap. If you really want to know the story of the 50MB, the space for it is actually allocated by something like the malloc command in C (as good ol' Ruby is written in C) and then stored on the System Heap. The Slot in the Ruby Heap simply contains a reference to that memory location on the System Heap which contains the 50MB of data.
Here's an example. Let's say that a Ruby program creates a single string of 50MB
* A single free Slot in a Ruby Heap becomes filled
* Memory to store the 50MB of data that makes up the string itself is allocated in memory and put on the System Heap (outside the Ruby Heap!) and a reference to this location is stored in the Filled Slot on the Ruby Heap
* There comes a point in time when this string is no longer needed. This slot is garbage collected on the next GC iteration
* The Filled Slot is turned into a free slot. The 50MB of data in memory referred to by the slot is also freed and returned to the Operating System
Ruby starts of with a minimal set of Ruby Heaps. These are managed by by a Ruby Heap list. Ruby creates Ruby Heaps when needed and frees Ruby Heaps back to the OS when no longer needed (the latter is done in a sub-optimal manner - more on this later). Each Ruby Heap created will be 1.8 times the size of the previous heap. In other words, it will contain 1.8 times the number of slots of the previous heap. Ruby's Garbage Collector, periodically iterates through the Ruby Heaps and frees up any Slots as appropriate (and also the memory that an object really occupies which is referenced by the Slot - ie. the 50MB data of the String) back to the system. Once a GC iteration is complete, some of the Slots that were filled will now be empty - known as Free Slots. Remember that we said that Ruby's Heap management actually consists of many Ruby Heaps. Well if one of these Ruby Heaps consists of only Free Slots then the Ruby Heap itself will be freed back to the Operating System.
There is a problem with this last statement however - if a Ruby Heap contains mostly Free Slots and one Filled Slot then it will not be freed. You could have many Ruby Heaps in this state. As long as a Ruby Heap contains even one Filled Slot it will not be returned to the Operating System. It just takes one bad apple to spoil everything! What would be nice is if some sort of Heap Compaction (kind of like disk fragmentation) took place where all Filled slots were pushed together into completed filled Ruby Heaps. This would leave you with completely filled Ruby Heaps, one semi-filled Ruby Heap and then a bunch of completely empty Ruby Heaps. The completely empty Ruby Heaps could then be freed, releasing precious memory back to the Operating System. Alas the current mainstream Ruby interpreter does not do this.
References
* How the Ruby Heap is Implemented Phusion Passenger's Hong Lai gives a great explanation of the Ruby Heap - the banner may not be quite suitable for work. Fortunately, there's a censor button
* Fine tuning your garbage collector Chris Heald explains some of the settings around garbage collection
* Ruby's Garbage Collections effect on Ruby on Rails Pluron Inc's blog discusses so of the knock-on effects of Ruby GC on Rails and importantly mentions the 8 MB memory allocation tigger for the garbage collector
Bleak House – A Tool for measuring Objects in Memory for a Ruby Program
Bleak House is a tool that tells us
- How many Slots there are in total at a point in time in a Ruby program
- How many Slots in total are filled
- How many Slots in total are empty (free)
- How many Filled Slots can be attributed to a particular line of code
Bleakhouse can be used to tell you if program is holding on to objects that it should be relinquishing. But it doesn't tell you how much data is stored in memory for the 'meat' of the object (ie. that 50MB of data in a 50MB String). Just because you know there is a Filled Slot exists - you don't know if the data in memory that correlates back to that Slot is 1MB, 10MB or 100MB.
However, if you repeat a series of a specific set of operations a small number of times, measuring with Bleakhouse, and then restart the server with Bleak house and repeat the operations a large number of times and see a big difference in the number of filled slots can tell that your program is holding onto objects (references) that it should not. Of course, if your program is supposed to keep hold of an increasing number of references (such as a global variable or a singleton that keeps accumulating references for the duration of your program) then this would be expected. Though you might want to double check your design. You will be able to see the cause of the problem from the detailed breakdown of which lines of code were the biggest offenders in terms of creating objects. If you see a large number of free slots (relative to the number of filled slots) then this means that at some point in your program a lot of objects existed (possibly due to a spike in application usage) but then reduced.
Does the free slots count matter? Well, yes because there is an memory overhead due to each free slot that exists - how much depends on your particular system. If your system has a slot size of 20 bytes then every one million free slots costs you an additional 20MB that is not being utilised. This becomes a problem if your application is subject to large but infrequent spikes in the number of objects that exist within your program a particular moment in time because the free slots are taking up significant amounts of memory even when your application is twiddling its thumbs between the spikes.
Getting Drupal 7 (development snapshot) running on Ubuntu
Notes before starting
- At time of writing, only experimental snapshots of Drupal were available with the actual Drupal 7 release being some way off. The Drupal 5.x or Drupal 6.x series is recommended if you want to use Drupal in production
- This guide was written against Ubuntu 8.04
- This guide assumes
- You have apache, mysql and php happily installed on your machine
- That you are serving websites out of /var/www/
- That that sites under /var/www/ are accessible in your browser via http://localhost/
Instructions
- Download Drupal 7 from the development release section at and http://drupal.org/project/drupal
- Extract the folder and rename the extracted folder to drupal7
- Move the folder to an appropriate place on your webserver, such as under /var/www/
- Create a fresh settings file for your new Drupal 7 site. You should use the supplied settings file as a basis, eg.
- cp sites/default/default.settings.php sites/default/settings.php
- Make the settings file writeable, so that the installer can edit it, using
- chmod a+w sites/default/settings.php
- Create a database for your Drupal 7 site
- Log into mysql, eg.
- mysql -uroot -p
- Create a database once logged into mysql from the 'mysql>' prompt, eg.
- create database drupal7;
- Create a user for the database and grant permissions from the 'mysql>' prompt, eg.
- GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES, CREATE TEMPORARY TABLES ON drupal7.* TO 'drupal7user'@'localhost' IDENTIFIED BY 'drupal7password';
- Log into mysql, eg.
- Run the install script as follows
- Browse to your new Drupal site at http://localhost/drupal7/install.php
- Elect to proceed with a normal Drupal install (ie. not minimal) in your chosen language
- As you proceed through the wizard, the installer may encounter issues creating certain directories for Drupal on the file system. If so, ensure then do the following
- cd to the root of your drupal7 directory
- mkdir sites/default/files
- mkdir -p sites/default/private/files
- mkdir sites/default/private/temp
- Change the ownership of the created directories to belong to user account under which the web server runs under (www-data)
- chown -R www-data:www-data private
- chown -R www-data:www-data files
- The wizard now asks you to configure the database settings. In the form presented by the wizard, replace the user 'admin' with the 'drupal7user' you created earlier and point at the drupal7 database. Remember to put in the correct password for 'drupal7user'
- If you get a 500 error in the last step, then ignore it and elect to continue through to the error page. This will actually let you continue the install
- If you get the error starting with something like "Fatal error: Allowed memory size of 16777216 bytes exhausted" then allocate Drupal more memory in the settings file (see See http://drupal.org/node/90605 for more information), eg. Add the line
- ini_set('memory_limit', '30M');
Hopefully now you should be good to go!
Audacity Tip of the Day – How Not to Lose Data!
One issue when editing audio, is copy and pasting a section of a track from one open Audacity project to another. To save space audacity does not copy the underlying track completely to the new project, rather it links to it. This means that your second project (the one you are pasting into) is not completely self-contained as it depends on external files. This can be a quickfire way for the unsuspecting podcaster to lose a whole bunch of data (yes, me!). This problem is particularly nasty as you don't realise something has gone wrong until you close and reopen the project - finding that a long stretch of audio containing your beautiful voice is missing. To avoid this, adhere to the following workflow when copy and pasting from one project to another.
* In the first project, select and copy the audio you wish to duplicate
* Paste the audio into the second project
* In the File menu of the second project, click 'Check Dependencies'
* Click 'Copy All Audio into Project (Safer)'
* Just to stress the previous point, you really do want to use the Safer of the two copying options. I've found that using the other option results in some of the audio I wanted to copy being truncated
* Save the project
By following this workflow you should hopefully avoid seeing what I call the 'dreaded blue flat-line of death' where, on reopening of a project, you find that the middle of a track has been lost. If you adhere to these instructions you should also be able to select 'Delete orphaned files' when it appears from time to time while reopening an Audacity project. However, due to the frustrating and sensitive nature of audio loss issues, I accept no responsibility for anything that goes wrong! Best of luck and happy editing!