RubyGems Beta 0.9.4.5

Eric Hodel | Sat, 13 Oct 2007 23:43:55 GMT

RubyGems 0.9.4.5 is a beta release for the upcoming 0.9.5 which adds several new features and fixes several bugs.

To upgrade to the beta:

gem update --system --source http://segment7.net/

To file bugs:

http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, gem env output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can do that with gem --debug the_command. For example:

<samp>$ gem --debug unknown_command
Exception `RuntimeError' at [...]/rubygems/command_manager.rb:114 - Unknown command unknown_command
ERROR:  While executing gem ... (RuntimeError)
    Unknown command unknown_command
        [...]/rubygems/command_manager.rb:114:in `find_command'
        [...]/rubygems/command_manager.rb:103:in `process_args'
        [...]/rubygems/command_manager.rb:74:in `run'
        [...]/rubygems/gem_runner.rb:39:in `run'
        /usr/local/bin/gem:22</samp>

Changes

Select new features include:

• Automatic installation of platform gems
• New bandwidth and memory friendlier index file format
• “Offline” mode (—no-update-sources)
• Bulk update threshold can be specified (-B,—bulk-threshold)
• New gem fetch command
• gem now has “really verbose” output when you specify -v
• Ruby 1.9 compatible

Other changes include:

• Time::today is deprecated and will be removed at a future date
• gem install --include-dependencies (-y) is now deprecated since it is the default, use—ignore-dependencies to turn off automatic dependency installation
• Multi-version diamond dependencies only are installed once
• Processing a YAML bulk index update takes less memory
• gem install -i makes sure all depenencies are installed
• gem update --system reinstalls into the prefix it was originally installed in
• gem update --system respects—no-rdoc and—no-ri flags
• HTTP basic authentication support for proxies
• Gem::Specification#platforms should no longer be a String, use Gem::Platform::CURRENT when building binary gems instead
• gem env has more diagnostic information
• require ‘rubygems’ loads less code
• sources.gem is gone, RubyGems now uses built-in defaults
• gem install --source will no longer add—source by default, use gem sources --add to make it a permanent extra source
• gem query (list) no longer prints details by default
• Exact gem names are matched in various places
• mkrf extensions are now supported
• A gem can depend on a specific RubyGems version
• gem_server is now gem server
• gemlock is now gem lock
• gem_mirror is now gem mirror
• gemwhich is now gem which
• gemri is no longer included with RubyGems
• index_gem_repository.rb is now gem generate_index
• gem performs more validation of parameters
• Custom rdoc styles are now supported
• Gem indexer no longer removes quick index during index creation
• Kernel#require only rescues a LoadError for the file being required now
• gem dependencies can now display some information for remote gems

Notes and issues:

• Old gem scripts (gem_mirror, gem_server, gemlock, gemri, gemwhich, index_gem_repository.rb) are not cleaned up
• There still appears to be a bug related to bulk updates of YAML indexes

Special Thanks

• Daniel Berger for win32 testing of early betas
• Luis Lavena for help with win32 platforms
• Tom Copeland for help testing and releasing the new indexer
• Wilson Bilkovich for the new index format
• To the rest of the RubyGems bug reporters and patch contributors

The full set of changes including contributors is included in the ChangeLog.

Platforms

RubyGems now automatically handles platform gems. This means that gem install will no longer prompt for gem selection. RubyGems uses Ruby’s built-in configuration to match the running ruby’s platform to choose the correct gem to install. The automatically chosen platform may be overridden with the—platform option.

The dependency, fetch, install, outdated, specification, uninstall and update commands all respond to—platform.

For more information, see gem help platforms

Thanks

Keep those gems coming!

—Jim & Chad & Eric (for the RubyGems team)

Posted in Rubygems, Software | 4 comments

Time.today going away

Eric Hodel | Mon, 08 Oct 2007 22:22:32 GMT

So you’ve been using Time::today for some time, and thought it was part of time.rb or ActiveSupport? Well, you’d be wrong. I was!

It turns out that rubygems/specification.rb defines Time::today, and that’s wrong. RubyGems has no business adding methods to the core when it doesn’t need to, so I’ve marked it for removal.

If you want to use Time::today in the future you’ll need to define it in your code somewhere, here is its definition:

require 'time'

def Time.today
  t = Time.now
  t - (t.to_i % 86400)
end unless Time.respond_to? :today

Go ahead and throw it in wherever you think is appropriate. I suggest you do this now.

UPDATED: Now with a better Time::today.

Posted in Ruby, Rubygems | 6 comments

#sort_by and #sort_obj

Eric Hodel | Sun, 07 Oct 2007 08:42:32 GMT

I was speeding up RubyGems and found a bunch of places that still used #sort instead of #sort_by. #sort_by is faster than #sort because it performs fewer comparisons resulting in fewer method calls.

All these places were external to the thing I was sorting, so they’d need to know how to perform the sorting, which is just plain wrong. I did a bit of thinking, and created a #sort_obj method to return an object that can be used for sorting instead.

So Gem::Specification#<=> went from:

def <=>(other)
  platform_num = platform == Gem::Platform::RUBY ? -1 : 1
  other_platform_num = other.platform == Gem::Platform::RUBY ? -1 : 1

  [@name, @version, platform_num] <=>
    [other.name, other.version, other_platform_num]
end

To:

def sort_obj
  [@name, @version.to_ints, @platform == Gem::Platform::RUBY ? -1 : 1]
end

def <=>(other)
  sort_obj <=> other.sort_obj
end

So now instead of:

specs.sort_by do |spec|
  [@name, @version.to_ints, @platform == Gem::Platform::RUBY ? -1 : 1]
end

I can write:

specs.sort_by { |spec| spec.sort_obj }

Posted in Hacking, Rubygems | no comments

RubyGems Beta Approaching

Eric Hodel | Fri, 05 Oct 2007 18:01:36 GMT

RubyGems 0.9.5 is almost done, and has loads of good stuff in it, including platform support and an improved indexer script:

[W]e were rebuilding the gem index on RubyForge, [...] doing it “in place”, so that the current index would be overwritten and then populated over the course of the build.  These take a fair while – 10 minutes or so – and during that time the index was essentially empty.  Booooo.

Well, no longer.  Eric Hodel has twiddled the gem index builder to build it in a temporary directory and then move it in place.  So those gem index outages should be a thing of the past.  Thanks Eric!

—More reliable gem installs via Junior developer

This will also reduce the number of bulk index updates dramatically, since it’ll actually be there nearly all the time.

Also, Wilson Bilkovich added a new Marshal formatted index that will reduce both bandwidth usage and memory consumption. Instead of 120M or so it takes to do a bulk yaml index update, it takes about 30M with a Marshal index update. You’ll have to wait for the beta to test this one out, though.

Posted in Ruby, Rubygems | 7 comments

RubyGems Tests Pass on 1.9

Eric Hodel | Thu, 20 Sep 2007 10:00:25 GMT

455 tests, 1600 assertions, 0 failures, 0 errors

Wooo!

Here’s some of the changes and notes I made for you:

• Removed some privates, because life is better without privates.
• Added Object#send! stub on 1.8.
• lambda cares about number of arguments now, so add |*a|.
• ENV['FOO'] = nil becomes ENV.delete 'FOO'. This is the most annoying change, since restoring defaults is a pain.
• WEBrick’s response object’s #code alias is gone, use #status
• (i = 0; 5.times do |i| end; i == 4) is false, so use a separate variable like count, and add count = i
• Kernel#require stores the full path to the require’d file now.
• There’s something going on with class variable sharing between a module and its singleton class. See [ruby-core:12200].
• Strings aren’t Array-like anymore, so use things like Kernel#Array instead of String#to_a
• #methods and #instance_variables return arrays of symbols instead of strings now.

Also, the tests run 20-25% faster on 1.9 than 1.8. Look for a beta of RubyGems coming next week!

Posted in Ruby, Rubygems | 4 comments

Gem Platform Matching

Eric Hodel | Sat, 25 Aug 2007 20:59:31 GMT

RubyGems now figures out if your platform matches the gem you want to install, and here’s how it does it.

First, the platform is compared to the list of legacy platforms. These are the platform strings that currently exist in the gems.rubyforge.org index. If the platform matches one of these it is expanded into an Array of CPU, OS and OS version. (New platforms are already an Array.)

Next the platform is compared with your RubyGems configured platforms (“ruby” and your OS platform by default) to see if any match. A platform’s CPU will match if it matches the local platform’s CPU exactly or if either side is ‘universal’. The OS must match exactly. Either OS version can be nil, or it must match exactly.

A potential improvement to the platform matching would be to upgrade a platform to a real object from an Array and make the OS version a Gem::Version and encode in each OS’ versioning scheme. I don’t think there is much benefit to that at present since most gems with platforms are for win32 which doesn’t have versions in Config::CONFIG[‘arch’].

Posted in Rubygems | 2 comments

Dear Lazyweb: Gem Platforms

Eric Hodel | Tue, 21 Aug 2007 02:53:04 GMT

As you may or may not have heard, RubyGems will be merged into Ruby 1.9 sometime in October. Before this can happen RubyGems needs to automatically install dependencies based on platforms. Fortunately I’ve got the automatic install part written. Unfortunately I don’t know if I’ve got figuring out the platforms right. This is where you come in.

Dear Lazyweb,

Here’s my proposal for how we recognize platforms. From Config::CONFIG, take the arch, split it on ’-’ into cpu and os and run os through a case statement to figure out OS and OS version (if any). Combine the cpu, OS and OS version. This value is your platform.

(There will be a rubygems-platforms.gem much like sources.gem that can be updated as necessary.)

Using the tattle data, the following code recognizes 29 unique platforms:

def match(arch)
  cpu, os = arch.split '-', 2
  cpu, os = nil, cpu if os.nil? # java

  cpu = case cpu
        when /i\d86/ then 'x86'
        else cpu
        end

  os = case os
       when /cygwin/ then            [ 'cygwin',  nil ]
       when /darwin(\d+)?/ then      [ 'darwin',  $1  ]
       when /freebsd(\d+)/ then      [ 'freebsd', $1  ]
       when /^java$/ then            [ 'java',    nil ]
       when /^java([\d.]*)/ then     [ 'java',    $1  ]
       when /linux/ then             [ 'linux',   $1  ]
       when /mingw32/ then           [ 'mingw32', nil ]
       when /mswin32/ then           [ 'mswin32', nil ]
       when /openbsd(\d+\.\d+)/ then [ 'openbsd', $1  ]
       when /solaris(\d+\.\d+)/ then [ 'solaris', $1  ]
       else                          [ 'unknown', nil ]
       end

  [cpu, os].flatten.compact.join("-")
end

require 'rbconfig'

arch = Config::CONFIG['arch']
cpu, os = arch.split '-', 2

puts "Your cpu is:   #{cpu.inspect}" 
puts "Your os is:    #{os.inspect}" 
puts "Your platform is: #{match(arch).inspect}" 

raise "need a tattle arch dump yaml file!" if ARGV.empty?

puts "loading archs..." 

require 'yaml'
archs = YAML.load(ARGF.read)['arch'].keys

def recognize(*archs)
  unmatched = {}
  seen = {}

  archs.each do |arch|
    platform = match arch

    seen[platform] = true
    unmatched[arch] = true if platform =~ /-unknown$/
  end

  return unmatched.keys, seen.keys
end

unmatched, unique = recognize(*archs)

puts "found #{unique.length} unique platforms" 
puts
puts unique.sort.join("\n")

unless unmatched.empty? then
  puts
  puts "unmatched" 
  puts
  puts unmatched.join("\n")
end

Lazyweb, I have two major questions for you:

Did I get something wrong? Am I using autoconf’s target_os correctly? Is Solaris 2.8 really incompatible with Solaris 2.9? What is a 64-bit Windows’ target_os value?

Do I have all the platforms people run Ruby and RubyGems on? If the answer to this one is no, do this: gem install tattle; tattle. (Yes, AIX users, I’m talking to you.)

UPDATE: I switched from target_os and target_cpu to arch to support modern JRuby, thanks Nick Sieger! See ruby-talk:265596.

Posted in Rubygems | 7 comments

Two Weeks of Vacation, Vlad, RubyGems

Eric Hodel | Fri, 17 Aug 2007 21:04:05 GMT

I’m not working this month so I can do whatever I want. So far that’s featured:

• The long overdue MogileFS 1.2.1 with a minor bugfix
• A mostly-evil Kernel#caller enhancement, SuperCaller 1.0.0
• memcache-client 1.5.0 with a couple of new features
• Sphincter 1.1.0 with friendlier setup and association inclusion
• The first release of Vlad the Deployer by the Ruby Hit Squad
• Tons of hacking on RubyGems

Building Vlad was a lot of fun. Ryan and I flew up Wilson Bilkovich, watched Bourne Ultimatum, then hacked for four days straight to bring Vlad into the world.

InfoQ published an interview with us about Vlad:

Capistrano, a popular deployment tool for Rails, is challenged by Vlad the Deployer, a tool which offers similar functionality with a much simpler implementation. We talked to the Ruby Hit Squad group that released version 1.0 of Vlad.

—Capistrano gets competition: Vlad the Deployer via InfoQ

The RubyGems hacking has been mostly bug fixes and refactoring so far. I’m working towards teaching RubyGems your platform so it can automatically install the correct version.

Here’s a sample of what’s I’ve done to RubyGems so far:

• —sources is no longer remembered forever, use `gem sources` to manage the permanent list
• The sources gem is gone, RubyGems uses a built-in list now (but can be upgraded in the future)
• `gem list` respects its default of just gem names now
• Only exact gem names are matched on install, “foo_bar 2.0” won’t install instead of “foo 1.0” if you run `gem install foo`
• RubyGems requires only what it needs when you require 'rubygems'
• Fewer bulk updates when updating the gem index
• `gem dep -r` lists dependencies for remote gems
• `gem info -r` shows information for remote gems
• `gem -v` turns on “really verbose” mode (verbose mode is the default)
• `gem_mirror`, `gem_server`, `gemlock`, `gemri`, `gemwhich`, `index_gem_repository.rb` have been merged into `gem`

Posted in Hacking, Ruby, Rubygems, Software | 2 comments

tinderbox version 1.0.0 has been released!

Eric Hodel | Wed, 31 Jan 2007 09:24:00 GMT

tinderbox version 1.0.0 has been released!

http://seattlerb.rubyforge.org/tinderbox

Description

Tinderbox tests projects and tries to make them break by running them on as many different platforms as possible.

Features & Problems

• Tests gems in a sandbox
• Submits gem test results to http://firebrigade.seattlerb.org
• Understands test/unit and RSpec

Changes:

1.0.0 / 2007-01-30

• Tests gems in a sandbox
• Submits results to Firebrigade
• Birthday!

http://seattlerb.rubyforge.org/tinderbox

Posted in Rubygems, Seattle.rb, Software, Testing | 4 comments