ar_mailer 1.1

Eric Hodel | Sat, 19 Aug 2006 22:41:15 GMT

ar_mailer allows you to queue ActionMailer emails in the database that will be sent with a separate process. ar_mailer gives drastic speed-ups when you need to send many emails.

To install ar_mailer: sudo gem install ar_mailer. You can also download ar_mailer from Rubyforge.

For instructions on how to convert from the regular delivery methods to using ar_mailer see the ar_mailer documentation.

ar_mailer-1.1.0 follows quick on the heels of ar_mailer 1.0. I’ve got several important features and bug fixes:

Features

• Added—chdir to set rails directory
• Added—environment to set RAILS_ENV
• Exits cleanly on TERM or INT signals
• Added FreeBSD rc.d script
• Exceptions during SMTP sending are now logged
• No longer waits if sending email took too long

Bugs fixed

• Fixed last send attempt in—mailq
• Better SMTP error handling
  • Messages are removed from the queue on 5xx errors
  • Added Net::SMTP.reset to avoid needing to recreate the connection

Posted in Rails, Software | no comments

cached_model 1.2

Eric Hodel | Fri, 18 Aug 2006 07:38:00 GMT

Geoff writes about our cached_model and memcache-client libraries:

The Robot Co-Op has made a few libraries available. cached_model makes it easy to cache single row queries from ActiveRecord tables. memcache-client is a pure Ruby client and is included with the installation of cached_model.

—memcached Basics for Rails via Nuby on Rails

He even includes a useful how-to document to get you up and running!

(Yes, that's right, a brand new version of cached_model that works with Rails 1.1! If you've already downloaded 1.2.0, please upgrade to 1.2.1. I forgot that find_by_sql will return [], thanks to James Cox for pointing out my bug.)

Posted in Rails, Software | no comments

ar_mailer

Eric Hodel | Tue, 15 Aug 2006 23:39:00 GMT

Rubyforge Project: http://rubyforge.org/projects/rctools

Documentation: http://dev.robotcoop.com/Tools/ar_mailer

About

Even deliviring email to the local machine may take too long when you have to send hundreds of messages. ar_mailer allows you to store messages into the database for later delivery by a separate process, ar_sendmail.

Installing ar_mailer

Just install the gem:

$ sudo gem install ar_mailer

Converting to ar_mailer

Go to your Rails project:

$ cd your_rails_project

Create a new migration:

$ ar_sendmail --create-migration

You'll need to redirect this into a file. If you want a different name provide the —table-name option. Create a new model:

$ ar_sendmail --create-model

You'll need to redirect this into a file. If you want a different name provide the —table-name option. Change your email classes to inherit from ActionMailer::ARMailer instead of ActionMailer::Base:

--- app/model/emailer.rb.orig   2006-08-10 13:16:33.000000000 -0700
+++ app/model/emailer.rb        2006-08-10 13:16:43.000000000 -0700
@@ -1,4 +1,4 @@
-class Emailer < ActionMailer::Base
+class Emailer < ActionMailer::ARMailer

def comment_notification(comment)
  from comment.author.email

Edit config/environments/production.rb and set the delivery agent:

$ grep delivery_method config/environments/production.rb
ActionMailer::Base.delivery_method = :activerecord

Run ar_sendmail:

$ ar_sendmail

You can also run ar_sendmail from cron with -o, or as a daemon with -d. See ar_sendmail -h for full details.

Posted in Rails, Software | 2 comments

Freezing Rails Versions

Eric Hodel | Wed, 09 Aug 2006 18:58:00 GMT

Ryan Davis has a nice Rake tasks that lets you freeze a Rails release. Its probably the easiest way to upgrade to Rails 1.1.5.

Posted in Rails | no comments

Timezones 1, Rails 0

Eric Hodel | Tue, 04 Jul 2006 00:17:00 GMT

For Trackmap I need to take the time a picture was taken and a user supplied time zone and convert that to a correctly offset time. Since Flickr won't give me a time zone (cameras don't record them) I ask the user. If they say Mountain time I need to create a time that is correct for the server's clock (in my case Pacific Time).

Rails TimeZone

When I originally wrote the code back in November and December of 2005 I noticed that Rails provides the handy TimeZone class which worked great! Rails gave me handy offsets that I could use to adjust times correctly.

Then April rolled around, my clocks switched to daylight savings time, and Rails' TimeZone class broke. TimeZone thought the offset for Pacific time was -8 hours instead of -7 hours giving inaccurate conversions. I filed a ticket, but it was closed WONTFIX (the breakage remains undocumented). I was, however, pointed at the TZInfo Timezone plugin. Since April I haven't had much time to work on Trackmap, so I left it broken.

TZInfo Timezone Plugin

On Friday I switched over to the plugin and found it provides incorrect UTC offsets as a misfeature:

>> tzt = TimeZone['Pacific Time (US & Canada)']
=> #
>> tzt.utc_offset
=> -28800
>> Time.now.gmtoff
=> -25200

While the TZinfo Timezone plugin will convert times between known time zones, it doesn't provide a way of interpreting a user-input time so you get a correctly offset time, making it only half of a fix for Rails' TimeZone.

Interpreting Times

To handle user-input times you must use TZInfo::Timezone#period_for_utc or TZInfo::Timezone#period_for_local to retrieve the correct UTC offset. You can retrieve the TZInfo::Timezone object from TzinfoTimezone#tzinfo (which is all the utility the plugin provides).

Correct conversions require some work. I chose to do it this way:

1. Create a Time using the user-supplied time
2. Get the local TZInfo::TimezonePeriod for the timestamp
   • If a TimezonePeriod doesn't exist for the given time, get the utc period
   • If the TimezonePeriod is ambiguous, choose standard time
3. Use TimezonePeriod#to_utc to adjust the Time to a UTC Time

A TimezonePeriod won't exist during the standard time to daylight savings time switch, the user probably chose the wrong time zone. The TimezonePeriod is ambiguous during the daylight savings time to standard time switch since the user's clock traverses the same time period twice. I chose to have strange behavior over failing. If the user notices and emails me I can say they probably fat-fingered something.

Rails Strikes Back

Having time interpretation finished I thought I was done. My database has timestamp with time zone columns so the database and Rails should just figure things out. Turns out Rails still can't handle timestamp with time zone despite having a patch in the bug tracking database for three months.

Instead of applying the patch (which I'd have to do twice, once on my laptop and once online, and for every Rails upgrade until it gets checked in), I adapted it into a 75 line extension to the PostgreSQL connection adapter. Maybe I'll make a plugin out of it while I'm waiting.

I'd like to think I was unlucky and the patch just got lost in the shuffle, but a look at the bug database makes me think that lost tickets happen frequently. One quarter of Rails' defect tickets are open (1098 out of 4034) so I don't see how anybody can figure out what tickets are invalid, valid, duplicate, fixed but not closed, or outright bogus. Furthermore, the amount of bugmail generated must be overwhelming.

The Rails core team should do what Mozilla did back in its early days and organize a bug day. Get a bunch of people on IRC to crawl through the open tickets to validate, cross-reference, prioritize and categorize bugs. (Mozilla even made it a weekly event with prizes!) Having a clean bug database will prevent useful patches from falling on the floor and rotting to the point that their authors must re-write them.

Posted in Hacking, Rails, Trackmap | 4 comments

Upgrade to Rails 1.1.3 Now

Eric Hodel | Wed, 28 Jun 2006 19:05:35 GMT

DHH writes:

We’ve found and fixed a security issue with routing that could cause excess CPU usage in Rails processes when triggered by certain URLs. We strongly encourage anyone running 1.1.x to upgrade to the latest version. It’s fully backwards compatible and should serve as a small drop-in fix.

While certain URLs cause excess CPU usage, other URLs cause Rails to shut down uncleanly or halt (depending upon deployment environment). You need to upgrade. (It appears that Rails 1.0 is not vulnerable to this DOS, but I haven't tested.) While you're upgrading, check your dispatch.fcgi, it should look like the current dispatch.fcgi. If it doesn't, you need to upgrade it. There are other DOSs in older versions of dispatch.fcgi.

Posted in Rails | 2 comments

Supercharged Rails Development

Eric Hodel | Thu, 20 Apr 2006 21:42:22 GMT

Geoff brings us a movie of autotest in action and has this to say:

My development process has recently been supercharged by autotest, a part of the ZenTest package.

[...]

The revolutionary part of this is that it speeds development by helping you develop without needing to open your web browser! I find myself thinking more about the functional issues that need to be solved rather than the placement of an image or the color of a link.

Posted in Rails, Testing, ZenTest | no comments

Speeding up Test Runs with fork

Eric Hodel | Sun, 09 Apr 2006 03:47:00 GMT

Loading Rails takes a significant portion of your test run time, especially when you want to run only one test file or one test method. On my Powerbook loading Rails takes between four and six seconds. If you're frequently running unit tests this constant overhead can quickly become annoying.

When using autotest I may have to wait as much as ten seconds (five seconds between scans for changes, four seconds to load rails, one second to run the test) before I know if my changes fixed a problem or not. Ten seconds is past the threshold where I can keep paying attention which makes my mind wander. (A wandering mind is no good for productive work.) Also, those extra four seconds of loading Rails per test start to add up. I may load rails hundreds of times in a day just to run a tiny test.

There's one already existing way to reduce or eliminate that constant overhead of loading Rails. In development mode Rails reloads files to keep things running without restarting Rails on every change. I prefer to have an environment that is guaranteed to be clean when the tests start and reloading files removes this option.

Since I want Rails loaded without any application code I chose to create a process that would load rails then open up a server socket and wait for connections. When a connection comes in the process will fork to make a copy of the environment that can then load the application and run the tests.

A regular test run for just one file runs like this:

$ time ruby test/controllers/route_controller_test.rb Loaded suite test/controllers/route_controller_test
Started
......................................................
Finished in 13.192465 seconds.

54 tests, 268 assertions, 0 failures, 0 errors

real    0m17.884s
user    0m8.147s
sys     0m1.424s

The difference between the real time and the Test::Unit run time accounts for Rails and app loading overhead, about five seconds.

I've tentatively named the parent process spawner 'ruby_fork' and the client 'ruby_fork_client', so you start up the parent process:

$ RAILS_ENV='test' ruby_fork -r rubygems -e 'require_gem "rails"'
/Users/drbrain/Links/ZT/bin/ruby_fork Running as PID 3570 on 9084

ruby_fork understands -r, -I and -e just like regular ruby so I can just load Rails and none of the rest of my application.

Then I run ruby_fork_client which takes its arguments and passes them across to the child process and then reads from the socket and prints to STDOUT.

$ time ruby_fork_client -r test/controllers/route_controller_test.rb
Loaded suite /Users/drbrain/Links/ZT/bin/ruby_fork
Started
......................................................
Finished in 12.442556 seconds.

54 tests, 268 assertions, 0 failures, 0 errors

real    0m13.947s
user    0m0.077s
sys     0m0.022s

Now that extra time spent loading Rails is gone and I'm left with application loading and Test::Unit overhead which is miniscule in comparison.

ruby_fork is not Rails specific. The server and client can do anything they like, so this has applications beyond testing Rails (for example, handling incoming mail) or even Rails itself.

I'd like to release ruby_fork and ruby_fork_client as part of ZenTest but I'll be holding it until 3.3.0. Currently ZenTest is almost ready for release and ruby_fork and ruby_fork_client needs to act more like a regular invocation of ruby.

Posted in Hacking, ZenTest, Rails, Testing | 5 comments

2.5 million

Eric Hodel | Tue, 07 Mar 2006 06:40:00 GMT

On Saturday March 4th the sites of The Robot Co-op handled 2,587,240 requests through Rails. That number includes redirects and error pages but excludes images, CSS and static javascript (we dynamically generate some JS for blog posting).

Posted in Rails, Robot Co-op

Memcache-client benchmarks

Eric Hodel | Mon, 06 Mar 2006 22:15:00 GMT

Stefan Kaes writes about using memcached for Ruby on Rails session storage wherein he finds that memcache-client beats out all comers in his session store benchmarks:

memcache-client-1.0.3 provides much better performance: much faster than either the old implementation, pstore or ActiveRecordStore, but also faster than my optimized SQLSessionStore using MysqlSession.

Posted in memcached, Rails