OptionParser Argument Casting

Eric Hodel | Sun, 06 Jan 2008 04:56:03 GMT

OptionParser is a command-line argument parsing library for Ruby that provides several really nice features. Using OptionParser tends to be verbose, but it is also very flexible. One of it’s features that I really like is argument casting.

Argument casting allows you to validate a command-line option and convert it from the user-supplied String into whichever object you like. The ri for OptionParser has an example similar to this one for casting a floating-point argument into a Float value:

require 'optparse'

options = {}

opts = OptionParser.new do |opts|
  # Cast 'delay' argument to a Float.
  opts.on("--delay N", Float,
          "Delay N seconds before executing") do |n|
    options[:delay] = n
  end
end

opts.parse! ARGV

p options

The second argument to opts.on, Float, tells OptionParser to cast the option’s value to a Float before passing it to the handler block. When you run this example with a number as the argument, you’ll see the value in options is a Float:

<samp>$ ruby op_test.rb --delay 1
{:delay=>1.0}</samp>

If you pass a value that can’t be cast into a Float an OptionParser::InvalidArgument is raised:

<samp>$ ruby op_test.rb --delay X
[...]/optparse.rb:454:in `parse': invalid argument: --delay X (OptionParser::InvalidArgument)
    [...]
    from [...]/optparse.rb:1353:in `parse!'
    from op_test.rb:13</samp>

You can rescue this and provide an appropriate help message.

At the bottom you’ll find some tables of the various casts that are built-in to OptionParser. If none of those do what you want, writing your own is very easy.

In RubyGems, various arguments are automatically cast to the appropriate objects. For example, when you specify a version with `gem install—version ’= 1.2.3’`, the argument ’= 1.2.3’ is turned into a Gem::Requirement:

OptionParser.accept Gem::Requirement do |value|
  Gem::Requirement.new value
end

Gem::Requirement.new knows how to cast a String and raises an exception if it can’t, so we delegate to it to do the work.

If you only want to have a specially formatted string, you can provide a regular expression instead. The DecimalInteger cast is defined like this:

DecimalInteger = /\A[-+]?\d+(?:_\d+)*/io
accept(DecimalInteger) {|s,| s.to_i if s}

So the pattern referenced by the name is used to validate the argument.

You can also provide a pattern as the second argument. The Float cast is defined like this:

floatpat = %r"\A[-+]?[...]"io
accept(Float, floatpat) {|s,| s.to_f if s}

Notice that for each of these, you still need to turn the string argument into the appropriate object.

OptionParser Built-in Casts

With no extra requires, OptionParser can cast the following arguments for you:

Name	Requirements	Cast to
Object, NilClass	Any string, no conversion	String
String	Any non-empty string	String
Integer	Binary (0b), octal (0), hexadecimal (0x), or decimal number	Integer
Float	Floating point number	Float
Numeric	Generic number format	Float for floats, Integer for integers
OptionParser::DecimalInteger	Decimal integer	Integer
OptionParser::OctalInteger	Octal, binary or hexadecimal number	Integer
OptionParser::DecimalNumeric	Decimal number	Integer or Float
TrueClass	+, -, yes, no, true, false, nil	true or false, defaults to true
FalseClass	+, -, yes, no, true, false, nil	true or false, defaults to false
Array	Comma-separated list	Array of Strings
Regexp	Regular expression with options	Regexp

If you require ‘optparse/date’:

Name	Requirements	Cast to
DateTime	Anything handled by DateTime.parse	DateTime
Date	Anything handled by Date.parse	Date

If you require ‘optparse/shellwords’:

Name	Requirements	Cast to
Shellwords	Anything handled by Shellwords.shellwords	Array of Strings

If you require ‘optparse/time’:

Name	Requirements	Cast to
Time	Anything handled by Time.parse	Time

If you require ‘optparse/uri’:

Name	Requirements	Cast to
URI	Anything handled by URI.parse	URI

Posted in Ruby | 2 comments

Graphing Spam

Eric Hodel | Mon, 24 Dec 2007 11:49:00 GMT

For fun I decided to chart various statistics from my mail server’s logs. I ended up with these charts using John Barnette’s GChart, some regular expressions, and cron.

The details of parsing out the data from the log files and setting up a crontab is boring, so I’ll spare you that.

Using GChart is really cool and easy. Here’s what I used to generate the Amavis Statistics chart, the others are all nearly the same, just different labels and titles.

def graph_amavis(data)
  # These are the labels and colors I'm using 
  labels = %w[Banned Spam   Spammy Bad\ Header Clean]
  colors = %w[000000 ff0000 ff7f00 ffff00      00ff00]

  max = data.map { |vals| vals.max }.max

  # Since axis labels aren't yet supported by the API in 0.2.0, I use :extras
  extras = { 'chxt' => 'r', 'chxl' => "0:|#{axis_labels max}" }

  chart = GChart.line :title => 'Amavis Statistics', :data => data,
                      :labels => labels, :colors => colors, :extras => extras

  chart_path = File.join File.dirname(@file), 'amavis_statistics.png'

  chart.size = '750x400'# can't be > 300,000 pixels
  chart.write chart_path
end

That’s it! One minor gotcha I had was that I needed to transpose the data set after reading it in because it wasn’t in the right order for GChart to consume it, but with Array#transpose, it’s just an extra method call before graphing.

Also, I wrote this simple utility function to calculate some good-enough axis labels:

def axis_labels(max)
  axis_labels = []
  0.upto 10 do |i| axis_labels << (max * 0.1 * i).to_i end
  axis_labels.join '|'
end

The values aren’t prettily chosen, but they work well enough.

Posted in Misc | no comments

My Favorite gem Commands

Eric Hodel | Mon, 24 Dec 2007 03:14:00 GMT

My two favorite gem commands are gem install -i ~/tmp/gems and gem which, followed closely by the gem fetch gemname; gem unpack gemname combo.

Now that the install command automatically installs all the necessary dependencies into the installation directory, it's easy to pull down a gem and play with it without having to do the work of cleaning out all it's dependencies from your main repository. A simple rm -r ~/tmp/gems is all it takes to clean up.

While I seldom have a need to use it, gem which tells you which file would get loaded when you require something. For example:

$ gem list activerecord

*** LOCAL GEMS ***

activerecord (1.15.6, 1.15.3)

$ gem which active_record
(checking gem activerecord-1.15.6 for active_record)
/System/Library/Frameworks/[...]/gems/activerecord-1.15.6/lib/active_record.rb

Finally, if I just want to poke at some code from a gem without bothering to poke through a gem repository, you can use gem fetch gemname; gem unpack gemname and you'll have a gemname-version directory with the gem sitting right in front of you.

Posted in Rubygems | no comments

RubyGems 1.0.1

Eric Hodel | Fri, 21 Dec 2007 03:25:37 GMT

Release 1.0.1 fixes a few bugs.

Bugs Fixed:

• Installation on Ruby 1.8.3 through 1.8.5 fixed
• gem build on 1.8.3 fixed

Other Changes Include:

• Since RubyGems 0.9.5, RubyGems is no longer supported on Ruby 1.8.2 or older, this is official in RubyGems 1.0.1.

How can I get RubyGems?

NOTE: If you have installed RubyGems using a package you may want to install a new RubyGems through the same packaging system.

If you have a recent version of RubyGems (0.8.5 or later), then all you need to do is:

gem update --system   (you might need to be admin/root)

(Note: You may have to run the command twice if you have any previosly installed rubygems-update gems).

If you have an older version of RubyGems installed, then you can still do it in two steps:

gem install rubygems-update  (again, might need to be admin/root)
update_rubygems              (... here too)

If you don’t have any gems install, there is still the pre-gem approach to getting software … doing it manually:

• DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126
• UNPACK INTO A DIRECTORY AND CD THERE
• INSTALL WITH: ruby setup.rb (you may need admin/root privilege)

To File Bugs

The RubyGems bug tracker can be found on RubyForge at: http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, gem env output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can do that with gem --debug the_command.

Thanks

Keep those gems coming!

—Jim & Chad & Eric (for the RubyGems team)

Posted in Rubygems, Software | 3 comments

RubyGems 1.0.0

Eric Hodel | Thu, 20 Dec 2007 08:32:58 GMT

Release 1.0.0 fixes several bugs.

NOTE: There is a bug installing on Ruby 1.8.5 and earlier. I will fix this tomorrow.

Major New Features Include:

• RubyGems warns about various problems with gemspecs during gem building
• More-consistent versioning for the RubyGems software

Other Changes Include:

• Fixed various bugs and problems with installing gems on Windows
• Fixed using gem server for installing gems
• Various operations are even more verbose with—verbose
• Built gems are now backwards compatible with 0.9.4
• Improved detection of RUBYOPT loading rubygems
• ruby setup.rb now has a—help option
• Gem::Specification#bindir is now respected on installation
• Executable stubs can now be installed to match ruby’s name, so if ruby is installed as ‘ruby18’, foo_exec will be installed as ‘foo_exec18’
• gem unpack can now unpack into a specific directory with—target
• OpenSSL is no longer required by default

Deprecations and Deletions:

• Kernel#require_gem has been removed
• Executables without a shebang will not be wrapped in a future version, this may cause such executables to fail to operate on installation
• Gem::Platform constants other than RUBY and CURRENT have been removed
• Gem::RemoteInstaller was removed
• Gem::Specification#test_suite_file and #test_suite_file= are deprecated in favor of #test_file and #test_file=
• Gem::Specification#autorequire= has been deprecated
• Time::today will be removed in a future version

How can I get RubyGems?

NOTE: If you have installed RubyGems using a package you may want to install a new RubyGems through the same packaging system.

If you have a recent version of RubyGems (0.8.5 or later), then all you need to do is:

gem update --system   (you might need to be admin/root)

(Note: You may have to run the command twice if you have any previosly installed rubygems-update gems).

If you have an older version of RubyGems installed, then you can still do it in two steps:

gem install rubygems-update  (again, might need to be admin/root)
update_rubygems              (... here too)

If you don’t have any gems install, there is still the pre-gem approach to getting software … doing it manually:

• DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126
• UNPACK INTO A DIRECTORY AND CD THERE
• INSTALL WITH: ruby setup.rb (you may need admin/root privilege)

Thanks

Keep those gems coming!

Posted in Software, Rubygems | 22 comments

Rubinius

Eric Hodel | Thu, 06 Dec 2007 02:28:02 GMT

I’m down in San Francisco this week working on Rubinius with Evan, Ryan, Wilson and Brian, which has been my Top Secret job at Engine Yard for a little over a month. Also joining us has been Josh, Kevin and Nathan (among others) have stopped by to hang out and hack too.

Primarily I’ve been working on getting RubyGems working on Rubinius, along with build system and other cleanups. I’ve started by trying to get just test_gem running, and I’ve become hung up waiting for Kernel#eval and Kernel#binding, which will be finished with the compiler2 work.

Posted in Rubinius | 1 comment

How I Use Autotest

Eric Hodel | Mon, 26 Nov 2007 22:33:23 GMT

During the Q&A portion of my talk, I was asked a very important question about autotest, something like “How often do you save when using autotest?” I save all the time when using autotest. There was a followup question, something like “Don’t you get syntax errors?” and I don’t.

Before I wrote autotest I was making fine-grained saves that were syntactically correct. I wrote autotest to automate the running of tests so I wouldn’t have to choose which tests to run. My changes were so small that I spent an annoying fraction of my time editing my command line.

When I’m using autotest, rather than writing all of a method’s tests, I start by performing the setup (which should cause an error), add a flunk, and save, which is usually four lines:

  def test_blah
    result = @thingy.blah
    flunk
  end

When the tests run they’ll see the method blah doesn’t exist and fail, so I define the method, (the def blah and end lines), and save. Then I write an assertion and save the test, which will fail, so I implement what I need to make the assertion pass, and save. Now I repeatedly perform the minimum changes to go from failure to flunking until my test is complete, remove the flunk, and start over with the next method.

Each save I do is a handful of lines, so it’s easy to keep them syntactically correct. I know I’m going from working state to working state as I work towards my end goal so I can easily roll back my changes with undo.

Posted in Testing, ZenTest | 1 comment

RubyGems 0.9.5

Eric Hodel | Tue, 20 Nov 2007 00:35:31 GMT

RubyGems 0.9.5 adds several new features and fixes several bugs.

To upgrade to the latest RubyGems:

gem update --system

To upgrade to the latest RubyGems by hand:

1. Download RubyGems from http://rubyforge.org/frs/?group_id=126&release_id=16500
2. gem install rubygems-update-0.9.5.gem
3. update_rubygems

To install RubyGems from scratch:

1. Download RubyGems source .tgz or .zip file from http://rubyforge.org/frs/?group_id=126&release_id=16500
2. Unpack the source .tgz or .zip
3. ruby setup.rb

To install RubyGems on Ruby 1.9 update your ruby trunk checkout and reinstall.

To file bugs:

http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, gem env output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can do that with gem --debug the_command. For example:

<samp>$ gem --debug unknown_command
Exception `RuntimeError' at [...]/rubygems/command_manager.rb:114 - Unknown command unknown_command
ERROR:  While executing gem ... (RuntimeError)
    Unknown command unknown_command
        [...]/rubygems/command_manager.rb:114:in `find_command'
        [...]/rubygems/command_manager.rb:103:in `process_args'
        [...]/rubygems/command_manager.rb:74:in `run'
        [...]/rubygems/gem_runner.rb:39:in `run'
        /usr/local/bin/gem:22</samp>

Changes

Select new features include:

• Automatic installation of platform gems
• New bandwidth and memory friendlier index file format
• “Offline” mode (—no-update-sources)
• Bulk update threshold can be specified (-B,—bulk-threshold)
• New gem fetch command
• gem now has “really verbose” output when you specify -v
• Ruby 1.9 compatible

Other changes include:

• Time::today is deprecated and will be removed at a future date
• gem install --include-dependencies (-y) is now deprecated since it is the default, use—ignore-dependencies to turn off automatic dependency installation
• gem.bat and bin stubs on mswin platforms are improved and compatible with the One-Click Installer
• Multi-version diamond dependencies only are installed once
• Bulk index updates take less memory
• -V now enables verbose instead of -v to avoid collision with—version’s -v
• gem install -i makes sure all depenencies are installed
• gem update --system reinstalls into the prefix it was originally installed in
• gem update --system respects—no-rdoc and—no-ri flags
• HTTP basic authentication support for proxies
• Gem::Specification#platforms should no longer be a String, use Gem::Platform::CURRENT when building binary gems instead
• gem env has more diagnostic information
• require ‘rubygems’ loads less code
• sources.gem is gone, RubyGems now uses built-in defaults
• gem install --source will no longer add—source by default, use gem sources --add to make it a permanent extra source
• gem query (list) no longer prints details by default
• Exact gem names are matched in various places
• mkrf extensions are now supported
• A gem can depend on a specific RubyGems version
• gem_server is now gem server
• gemlock is now gem lock
• gem_mirror is now gem mirror
• gemwhich is now gem which
• gemri is no longer included with RubyGems
• index_gem_repository.rb is now gem generate_index
• gem performs more validation of parameters
• Removed gem* commands are now replaced with stubs that warn
• Custom rdoc styles are now supported
• Gem indexer no longer removes quick index during index creation
• Kernel#require only rescues a LoadError for the file being required now
• gem dependencies can now display some information for remote gems

Notes and issues:

• Old gem scripts (gem_mirror, gem_server, gemlock, gemri, gemwhich, index_gem_repository.rb) are not cleaned up
• There still appears to be a bug related to bulk updates of YAML indexes

Special Thanks

• Daniel Berger for win32 support and testing
• Luis Lavena for win32 support and testing
• Tom Copeland for help testing and releasing the new indexer
• Wilson Bilkovich for the new Marshal index format
• To the rest of the RubyGems bug reporters and patch contributors

The full set of changes including contributors is included in the ChangeLog.

Platforms

RubyGems now automatically handles platform gems. This means that gem install will no longer prompt for gem selection. RubyGems uses Ruby’s built-in configuration to match the running ruby’s platform to choose the correct gem to install. The automatically chosen platform may be overridden with the—platform option.

The dependency, fetch, install, outdated, specification, uninstall and update commands all respond to—platform.

For more information, see gem help platforms

Thanks

Keep those gems coming!

—Jim & Chad & Eric (for the RubyGems team)

Posted in Rubygems, Software | 13 comments

RubyConf 2007

Eric Hodel | Tue, 20 Nov 2007 00:32:50 GMT

I had a good time at RubyConf, but was too busy obsessing over my presentation. I think it turned out well, though, as many people complimented me on my talk. I did manage to play my “you owe me a beer” card a couple times though, which was good.

Oh, yes, here’s my slides for Maximizing Productivity in PDF, the Keynote (zipped) form, and the presentation courtesy of Confreaks.

This RubyConf had the feel of the pre-Rails RubyConfs, which was very welcome. The dual track format was alright, except that one talk would be far more popular than the other leaving one room standing-room-only.

I think my favorite talk of the conference was Ben Bleything’s Controlling Electronics with Ruby, because he had all sorts of fun toys to pass around and play with.

Also, Rich Kilmer, Akira Tanaka and Koichi Sasada figured out the last issues to importing RubyGems into 1.9, so we’ll be cleaning that up over the next week or two. I’ll have a separate post on the RubyGems differences between 1.8 and 1.9 shortly.

Posted in Conference, Ruby | 1 comment

RubyGems Beta 0.9.4.6

Eric Hodel | Sat, 20 Oct 2007 08:10:12 GMT

RubyGems 0.9.4.6 is a beta release for the upcoming 0.9.5 which adds several new features and fixes several bugs.

To upgrade to the beta:

gem update --system --source http://segment7.net/

To file bugs:

http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, gem env output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can do that with gem --debug the_command. For example:

<samp>$ gem --debug unknown_command
Exception `RuntimeError' at [...]/rubygems/command_manager.rb:114 - Unknown command unknown_command
ERROR:  While executing gem ... (RuntimeError)
    Unknown command unknown_command
        [...]/rubygems/command_manager.rb:114:in `find_command'
        [...]/rubygems/command_manager.rb:103:in `process_args'
        [...]/rubygems/command_manager.rb:74:in `run'
        [...]/rubygems/gem_runner.rb:39:in `run'
        /usr/local/bin/gem:22</samp>

Changes Since 0.9.4.5

• gem update won’t install gems multiple times (due to dependencies)
• gem.bat and bin stubs on mswin platforms are improved and compatible with the One-Click Installer
• gem install no longer installs dependencies for old versions of a gem
• Removed gem* commands are now replaced with stubs that warn
• RubyGems now installs correctly with RUBYOPT=-rubygems<code> </ul> For the rest of the updates since RubyGems 0.9.4, see the <a href="http://blog.segment7.net/articles/2007/10/13/rubygems-beta-0-9-4-5">RubyGems Beta 0.9.4.5 release notes</a>.

  Posted in Rubygems, Software | no comments