Packaging from a gemspec is not the Best Way

drbrain | Fri, 09 Dec 2011 22:57:06 GMT

Or, why you should use Hoe (or a tool like it) to package your gems.

A while back Yehuda wrote Using .gemspecs as Intended and asserted that packaging directly from a .gemspec file was both the way you were supposed to build gems and the best way to build gems.

This is not true.

Rake::GemPackageTask (now Gem::PackageTask) was added to rake less than two weeks after the first commit on RubyGems. Which points to the recognition of a need to have a better way to describe and build a gem than just a file.

Using rake allows you to do more than just package a static list of files in a clean and reusable manner. You can write tasks to generate files that are then packaged in your gem at build time. Hoe builds atop Rake to support many types of generated files easily through task dependencies.

For example, generating a .gemtest file to work with GemTesters, integrating with rake-compiler for pre-compiled gems or generating parsers.

I can hear you thinking "you might have a point there, but I don't generate any files, so how does this apply to me?" Well, what if you wanted to include generated man pages in your gem using binman?

In the world of packaging directly from a gemspec the way to automatically generate files is to violate RubyGems' internals by overriding Gem::Specification#initialize. I can't blame Suraj Kurapati for this, he's working within the framework of the popular thing to do and this is the only way he could implement it.

(Hoe, on the other hand, handles such pre-package dependencies through a set of pre-defined rake tasks that you can add dependencies too. If you need to generate a file you make the package task dependent on it. See the examples above.)

How can we make this better?

First of all, stop using gemspecs as the One True Description of your gem. Let rake generate it. While Hoe is obviously the best way to do this, it doesn't support generating the gemspec by default. You can use one of the gemspec plugins for that. I'm sure there are other build tools have the equivalent built right in.

Second, if you've writing a tool that generates files that will be packaged into a gem, provide a rake task like the one that ships with binman that build tool authors can hook to make packaging seamless.

(Also, please, don't use system when you can use API.)

Posted in ,  | 6 comments | no trackbacks

RubyGems 1.8.9

drbrain | Tue, 23 Aug 2011 23:16:15 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or `ri Gem`)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

1.8.9 / 2011-08-23

  • Bug fixes:

    • Fixed uninstalling multiple gems using `gem uninstall`

    • Gem.use_paths splatted to take multiple paths! Issue #148

Posted in ,  | no comments | no trackbacks

RubyGems 1.8.8

drbrain | Fri, 12 Aug 2011 00:52:21 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or `ri Gem`)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

1.8.8 / 2011-08-11

  • Bug fix:

    • The encoding of a gem’s YAML spec is now UTF-8. Issue #149

Posted in ,  | no comments | no trackbacks

RubyGems 1.8.7

drbrain | Fri, 05 Aug 2011 00:55:39 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or ri Gem)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

1.8.7 / 2011-08-04

  • Bug fixes:

    • Added missing require for gem uninstall --format-executable

    • The correct name of the executable being uninstalled is now displayed with --format-executable

    • Fixed gem unpack uninstalled_gem default version picker

    • RubyGems no longer claims a nonexistent gem can be uninstalled

    • gem which no longer claims directories are requirable files

    • gem cleanup continues cleaning up gems if one can’t be uninstalled due to permissions. Issue #82

    • Gem repository directories are no longer created world-writable. Patch by Sakuro OZAWA. Ruby Bug #4930

Posted in ,  | 1 comment | no trackbacks

Ruby 1.9.3 preview 1

drbrain | Mon, 01 Aug 2011 19:28:36 GMT

Ruby 1.9.3 preview 1 has been released! Please install it and report bugs! The NEWS file contains the changes since Ruby 1.9.2 and the ChangeLog contains all the gory details.

If you're using RVM you can follow these instructions to install preview 1. If not, you can download the tarball, unpack it, then run configure; make; make install as you like.

Bug Reports

Ruby 1.9.3 preview 1 ships with RDoc, RubyGems and Rake. I can fix critical bugs in all of these before the final release. Please file bug reports for Ruby via redmine for any issue you find. You can also search for existing issues to avoid duplicate reports.

RDoc

The preview 1 release contains RDoc 3.8, but I've released RDoc 3.9.1 to fix a few additional bugs. Please install RDoc 3.9.1 atop the preview 1 release, or standalone on your main ruby. I prefer reports of RDoc issues via github.

RubyGems

The preview 1 release contains RubyGems 1.8.6.1 which is slightly newer than RubyGems 1.8.6. RubyGems 1.8.7 will be released shortly with the combined changes. I prefer reports of RubyGems issues via github.

Rake

The preview 1 release contains Rake 0.9.2.1 which is slightly newer than Rake 0.9.2. I prefer reports of Rake issues via github.

Posted in , ,  | no comments | no trackbacks

RubyGems 1.8.6

drbrain | Tue, 26 Jul 2011 00:55:49 GMT

rubygems-update version 1.8.6 has been released!

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or `ri Gem`)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

Changes:

1.8.6 / 2011-07-25

  • Major enhancements:

    • Restore behavior of Gem::Specification#loaded? Ruby Bug #5032

  • Minor enhancements:

    • Add autorequires and delay startup of RubyGems until require is called. See Ruby bug #4962

  • Bug fixes:

    • Clean up SourceIndex.add_specs to not be so damn noisy. (tadman)

    • Added missing APPLE_GEM_HOME in paths.

    • Extend YAML::Syck::DefaultKey fixing to `marshal_dump` as well.

    • Fix #29216: check correct bin_dir in check_that_user_bin_dir_is_in_path.

    • Revert Gem.latest_load_paths to working order (PathSupport revert).

    • Restore normalization of GEM_HOME.

    • Handle the Syck DefaultKey problem once and for all.

    • Fix SystemStackError occurring with “gem list -r -a” on 1.9.

Posted in ,  | no comments | 1 trackback

RubyGems 1.8.4

drbrain | Thu, 26 May 2011 04:35:37 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or `ri Gem`)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

Changes:

1.8.4 / 2011-05-25

  • 1 minor enhancement:

    • Removed default_executable deprecations from Specification.

Posted in ,  | no comments | no trackbacks

RubyGems 1.8.3

drbrain | Fri, 20 May 2011 00:44:53 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or `ri Gem`)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

1.8.3 / 2011-05-19

  • 4 bug fixes:

    • Fix independent testing of test_gem_package_tar_output. Ruby Bug #4686 by Shota Fukumori

    • Fix test failures for systems with separate ruby versions. Ruby Bug #3808 by Jeremy Evans

    • Fixed some bad calls left behind after rolling out some refactorings.

    • Syck has a parse error on (good) times output from Psych. (dazuma, et al)

Posted in ,  | no comments | no trackbacks

RubyGems Deprecations

drbrain | Thu, 19 May 2011 01:03:47 GMT

I've been maintaining RubyGems a long, long time now. I believe my first release was 0.9.5 back at the end of 2007, four and a half years! In that time I've looked at pretty much every line of RubyGems. I know the history and intent behind most of the classes that exist in RubyGems and what purposes they served. (For more on that history watch my RubyConf 2010 presentation, History of RDoc and RubyGems.)

One thing I haven't done enough of in RubyGems is cleanup of the little things that make RubyGems easy to use as a library. While I've done some work to make it easier to use as a library I've ignored too many of the little things that make a polished library. Over the past several releases Ryan Davis has been driving these small cleanups to make RubyGems easier to use and maintain.

There are many things that are clumsy to do in RubyGems 1.7 and earlier due to poor encapsulation that are now clean, simple and intuitive in RubyGems 1.8.

Given a Gem::Specification, how do you activate it? 

# RubyGems 1.8

spec.activate

# RubyGems 1.7 and older

Gem.activate spec.name, spec.version

Given a Gem::Specification, how do you determine various directories for files in the gem? 

# RubyGems 1.8

spec.ri_dir
spec.cache_dir
spec.bin_dir
# etc.

# RubyGems 1.7 and older

File.join spec.installation_path, 'doc', 'ri'
File.join spec.installation_path, 'cache'
File.join spec.installation_path, 'gems', spec.full_name, 'bin'

How do you list the name of every gem you have installed? 

# RubyGems 1.8

Gem::Specification.all_names

# RubyGems 1.7 and older

Gem.source_index.map { |_, s| s.full_name }

... and so on. While these look like trivial examples, the work underlying these changes is large. Also, note that for each of these examples the Ruby 1.7 and older code still works, but it will complain.

Deprecations

Gem::SourceIndex has been deprecated because it has no real purpose anymore. Back in the olden days before RubyGems 1.2 Gem::SourceIndex was used to determine which gems you could install (I cover this my RubyConf 2010 talk above). It got pressed into service as a local repository of gems as well, but since RubyGems 1.2 it's really become just a wrapper around a Hash.

Instead of keeping around the baggage of an ancient, mostly-useless Hash wrapper we've moved the important functionality into Gem::Specification as class methods. Now Gem::Specification behaves as an Enumerable container.

Gem::GemPathSearcher was used to figure out which files belonged to which gems so require 'rake' would activate the rake gem. Unfortunately its implementation was so complicated I could barely understand it let alone improve it.

Instead of keep around a complex, incomprehensible class we moved the functionality into Gem::Specification as well. 

# RubyGems 1.8

spec = Gem::Specification.find_by_path 'rake'

# RubyGems 1.7 and older

spec = Gem.searcher.find 'rake'

Gem::Specification#default_executable has been deprecated as it was intended to be used for a feature that never was implemented. The idea behind it was a command like gem exec rake that would invoke the proper version of rake through RubyGems. Since this was never implemented we've deprecated the attribute.

In all, RubyGems releases since 1.4 have focused on cleaning up much of the historic cruft left behind after years of releases. We've properly encapsulated much of the functionality so it's implemented in a sensible place. Most of the deprecations we have made can be fixed by a single line change or running a single command.

Deprecation Warnings

I learned from the removal of #version_requirements that people will ignore deprecation warnings if you only output them once. When RubyGems 1.5 was released with the removal #version_requirements people were confused about why their Rails applications broke.

A single warning that prints every time you start your rails app and says Gem::Dependency#version_requirements is deprecated and will be removed on or after August 2010 can be swept under the rug, so for this release we decided to print warnings every time you use deprecated functionality.

Since we knew that users would need to update their gemspecs we provided the following instructions:

After installing RubyGems 1.8.0 you will see deprecations when loading your exsting gems. Run `gem pristine --all --no-extensions` to regenerate your gem specifications safely.

Currently RubyGems does not save the build arguments used to build gems with extensions. You will need to run `gem pristine gem_with_extension -- --build-arg` to regenerate a gem with an extension where it requires special build arguments.

These instructions were printed when you ran gem update --system and running the command would immediately solve the problem. They were also duplicated in my blog posts for the RubyGems 1.8.0, 1.8.1 and 1.8.2 (which also fixed a bug in gem pristine for some rvm users).

If you upgraded from RubyGems 1.7 to RubyGems 1.8.1 you may have missed the note about running gem pristine. We've changed our install process to print release notes from the version you're upgrading from to the version you're upgrading to.

The remaining deprecations should be rarely encountered as the deprecated functionality is rarely used outside RubyGems.

PS: If you write a library that interacts with RubyGems beyond require you should be on the RubyGems-developers mailing list.

Posted in  | 6 comments | no trackbacks

RubyGems 1.8.2

drbrain | Thu, 12 May 2011 00:55:39 GMT

RubyGems is a package management framework for Ruby.

This gem is an update for the RubyGems software. You must have an installation of RubyGems before this update can be applied.

See Gem for information on RubyGems (or ri Gem)

To upgrade to the latest RubyGems, run:

$ gem update --system  # you might need to be an administrator or root

See UPGRADING.rdoc for more details and alternative instructions.

If you don’t have RubyGems installed, your can still do it manually:

  • Download from: rubygems.org/pages/download

  • Unpack into a directory and cd there

  • Install with: ruby setup.rb # you may need admin/root privilege

For more details and other options, see:

ruby setup.rb --help

Changes:

1.8.2 / 2011-05-11

After installing RubyGems 1.8 or newer you will see deprecations when loading your exsting gems. Run gem pristine --all --no-extensions to regenerate your gem specifications safely.

Currently RubyGems does not save the build arguments used to build gems with extensions. You will need to run gem pristine gem_with_extension -- --build-arg to regenerate a gem with an extension where it requires special build arguments.

  • 2 minor enhancements:

    • Moved #outdated from OutdatedCommand to Specification (for Isolate).

    • Print out a warning about missing executables.

  • 3 bug fixes:

    • Added missing requires to fix various upgrade issues.

    • gem pristine respects multiple gem repositories.

    • setup.rb now execs with --disable-gems when possible

Posted in ,  | no comments | 1 trackback