Graphing Spam

Eric Hodel | Mon, 24 Dec 2007 11:49:00 GMT

For fun I decided to chart various statistics from my mail server’s logs. I ended up with these charts using John Barnette’s GChart, some regular expressions, and cron.

The details of parsing out the data from the log files and setting up a crontab is boring, so I’ll spare you that.

Using GChart is really cool and easy. Here’s what I used to generate the Amavis Statistics chart, the others are all nearly the same, just different labels and titles.

def graph_amavis(data)
  # These are the labels and colors I'm using 
  labels = %w[Banned Spam   Spammy Bad\ Header Clean]
  colors = %w[000000 ff0000 ff7f00 ffff00      00ff00]

  max = data.map { |vals| vals.max }.max

  # Since axis labels aren't yet supported by the API in 0.2.0, I use :extras
  extras = { 'chxt' => 'r', 'chxl' => "0:|#{axis_labels max}" }

  chart = GChart.line :title => 'Amavis Statistics', :data => data,
                      :labels => labels, :colors => colors, :extras => extras

  chart_path = File.join File.dirname(@file), 'amavis_statistics.png'

  chart.size = '750x400'# can't be > 300,000 pixels
  chart.write chart_path
end

That’s it! One minor gotcha I had was that I needed to transpose the data set after reading it in because it wasn’t in the right order for GChart to consume it, but with Array#transpose, it’s just an extra method call before graphing.

Also, I wrote this simple utility function to calculate some good-enough axis labels:

def axis_labels(max)
  axis_labels = []
  0.upto 10 do |i| axis_labels << (max * 0.1 * i).to_i end
  axis_labels.join '|'
end

The values aren’t prettily chosen, but they work well enough.

Posted in  | no comments

My Favorite gem Commands

Eric Hodel | Mon, 24 Dec 2007 03:14:00 GMT

My two favorite gem commands are gem install -i ~/tmp/gems and gem which, followed closely by the gem fetch gemname; gem unpack gemname combo.

Now that the install command automatically installs all the necessary dependencies into the installation directory, it's easy to pull down a gem and play with it without having to do the work of cleaning out all it's dependencies from your main repository. A simple rm -r ~/tmp/gems is all it takes to clean up.

While I seldom have a need to use it, gem which tells you which file would get loaded when you require something. For example: 

$ gem list activerecord

*** LOCAL GEMS ***

activerecord (1.15.6, 1.15.3)

$ gem which active_record
(checking gem activerecord-1.15.6 for active_record)
/System/Library/Frameworks/[...]/gems/activerecord-1.15.6/lib/active_record.rb

Finally, if I just want to poke at some code from a gem without bothering to poke through a gem repository, you can use gem fetch gemname; gem unpack gemname and you'll have a gemname-version directory with the gem sitting right in front of you.

Posted in  | no comments

RubyGems 1.0.1

Eric Hodel | Fri, 21 Dec 2007 03:25:37 GMT

Release 1.0.1 fixes a few bugs.

Bugs Fixed:

  • Installation on Ruby 1.8.3 through 1.8.5 fixed
  • gem build on 1.8.3 fixed

Other Changes Include:

  • Since RubyGems 0.9.5, RubyGems is no longer supported on Ruby 1.8.2 or older, this is official in RubyGems 1.0.1.

How can I get RubyGems?

NOTE: If you have installed RubyGems using a package you may want to install a new RubyGems through the same packaging system.

If you have a recent version of RubyGems (0.8.5 or later), then all you need to do is:

gem update --system   (you might need to be admin/root)

(Note: You may have to run the command twice if you have any previosly installed rubygems-update gems).

If you have an older version of RubyGems installed, then you can still do it in two steps:

gem install rubygems-update  (again, might need to be admin/root)
update_rubygems              (... here too)

If you don’t have any gems install, there is still the pre-gem approach to getting software … doing it manually:

To File Bugs

The RubyGems bug tracker can be found on RubyForge at: http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, gem env output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can do that with gem --debug the_command.

Thanks

Keep those gems coming!

—Jim & Chad & Eric (for the RubyGems team)

Posted in ,  | 3 comments

RubyGems 1.0.0

Eric Hodel | Thu, 20 Dec 2007 08:32:58 GMT

Release 1.0.0 fixes several bugs.

NOTE: There is a bug installing on Ruby 1.8.5 and earlier. I will fix this tomorrow.

Major New Features Include:

  • RubyGems warns about various problems with gemspecs during gem building
  • More-consistent versioning for the RubyGems software

Other Changes Include:

  • Fixed various bugs and problems with installing gems on Windows
  • Fixed using gem server for installing gems
  • Various operations are even more verbose with—verbose
  • Built gems are now backwards compatible with 0.9.4
  • Improved detection of RUBYOPT loading rubygems
  • ruby setup.rb now has a—help option
  • Gem::Specification#bindir is now respected on installation
  • Executable stubs can now be installed to match ruby’s name, so if ruby is installed as ‘ruby18’, foo_exec will be installed as ‘foo_exec18’
  • gem unpack can now unpack into a specific directory with—target
  • OpenSSL is no longer required by default

Deprecations and Deletions:

  • Kernel#require_gem has been removed
  • Executables without a shebang will not be wrapped in a future version, this may cause such executables to fail to operate on installation
  • Gem::Platform constants other than RUBY and CURRENT have been removed
  • Gem::RemoteInstaller was removed
  • Gem::Specification#test_suite_file and #test_suite_file= are deprecated in favor of #test_file and #test_file=
  • Gem::Specification#autorequire= has been deprecated
  • Time::today will be removed in a future version

How can I get RubyGems?

NOTE: If you have installed RubyGems using a package you may want to install a new RubyGems through the same packaging system.

If you have a recent version of RubyGems (0.8.5 or later), then all you need to do is:

gem update --system   (you might need to be admin/root)

(Note: You may have to run the command twice if you have any previosly installed rubygems-update gems).

If you have an older version of RubyGems installed, then you can still do it in two steps:

gem install rubygems-update  (again, might need to be admin/root)
update_rubygems              (... here too)

If you don’t have any gems install, there is still the pre-gem approach to getting software … doing it manually:

Thanks

Keep those gems coming!

Posted in ,  | 22 comments

Rubinius

Eric Hodel | Thu, 06 Dec 2007 02:28:02 GMT

I’m down in San Francisco this week working on Rubinius with Evan, Ryan, Wilson and Brian, which has been my Top Secret job at Engine Yard for a little over a month. Also joining us has been Josh, Kevin and Nathan (among others) have stopped by to hang out and hack too.

Primarily I’ve been working on getting RubyGems working on Rubinius, along with build system and other cleanups. I’ve started by trying to get just test_gem running, and I’ve become hung up waiting for Kernel#eval and Kernel#binding, which will be finished with the compiler2 work.

Posted in  | 1 comment