Forever-valid SSL certificates
drbrain |
If your library uses X509 cryptography, naturally your tests will need a key and valid certificate to test against. Creating a key and certificate frequently can quickly drain your entropy pool which slows down your tests.
Instead of creating the key for every test startup you can create it once and load it off the disk like this:
class TestMyGem < MyGem::TestCase
private_key = File.expand_path '../../../test/private_key.pem', __FILE__
private_key = File.read private_key
PRIVATE_KEY = OpenSSL::PKey::RSA.new private_key
# …Sure, you can rebuild the certificate every time with a validity time of an hour, but why not create a forever-valid certificate to go with it? No reasonable person would ever use a key shipped with an open project anyhow. Here's how to generate such a key and certificate:
require 'openssl'
# purposefully short key length
key = OpenSSL::PKey::RSA.new 512
# bogus subject and issuer
name = OpenSSL::X509::Name.parse 'CN=nobody/DC=example'
cert = OpenSSL::X509::Certificate.new
cert.subject = name
cert.issuer = name
cert.version = 2
cert.serial = 1
cert.not_before = Time.now
# lasts as long as X509 allows
cert.not_after = Time.gm 9999, 12, 31, 23, 59, 59
cert.public_key = key.public_key
cert.sign key, OpenSSL::Digest::SHA1.new
open 'private_key.pem', 'w' do |io| io.write key.to_pem end
open 'public_cert.pem', 'w' do |io| io.write cert.to_pem endYou can load this certificate just like the key as described above:
public_cert = File.expand_path '../../../test/public_cert.pem', __FILE__
public_cert = File.read public_cert
PUBLIC_CERT = OpenSSL::X509::Certificate.new public_certReplace your test helpers with reusable API
drbrain |
test/test_helper.rb is a great idea Rails brought to the Ruby world as a place for functionality that helps you write better tests. There's now a standard place for you to implement common setup/teardown, shortcuts and custom assertions. However, a test helper is not the best place to store this functionality for a Ruby library.
One of the benefits you get out of writing tests is knowing where your API is clumsy and inadequate. If you have a test helper file full of methods to make your library easy to use in a test why is that not part of your library's API? Wouldn't your users also want a bunch of methods that make your library easier to use in their applications?
For example, in RubyGems we have a test helper that does this: gem_file = Gem::Builder.new(spec).build which is a little silly. Every time you create a Gem::Builder you want to build a gem. You don't create a Gem::Builder object for fun! To help out RubyGems users I added a new method: gem_file = Gem::Builder.build spec which immediately creates and builds them gem which is much nicer for everyone (but really, you should use Gem::PackageTask when building gems).
Whether you're writing a library or a Rails app, this kind of functionality belongs in your library (or application) code, not in the test helper where only your tests can benefit from it.
Even after you improve your API by moving helpful functionality back into your library there's still going to be some things that only make sense for tests. For example, you probably don't want to type t = Some::Deeply::Namespaced::Thing.new 1, 2, 3 many, many times in your tests, so you write a short wrapper method you can call like this: t = thing 1, 2, 3. Your tests may need setup and teardown to maintain a clean environment between tests, custom assertions for readability or you may want to include a pre-built stub or mock.
While this having this functionality in a test helper is fine for a Rails app, it shouldn't go in a library's test helper. When you keep testing functionality hidden in the test directory a user who wants to write a third-party extension for your gem can't access them. Why force a happy user to re-implement (possibly poorly or incorrectly) the work you've done to have nice, clean tests that are easy to read and write?
Instead of having a private test helper I have a public test case like MyGem::TestCase that lives in lib/my_gem/test_case.rb. This gives anyone who wants to extend my libraries a documented, ready-to-go API for writing tests for their extension.
My gem-specific test case typically contains all the requires needed to load the library (ideally require 'my_gem'), proper setup and teardown to sandbox the tests, any utility methods that don't belong in the library itself and possibly some custom assertions. This makes a brand new test easy to start:
require 'my_gem/test_case'
class TestMyGemSomeClass < MyGem::TestCase
def setup
super
# …
end
def test_something
# …
end
endThere is the minor downside that an extension writer must use minitest (my preferred testing library) to test their extension. Perhaps this inconvenience could be solved by a module providing setup, teardown and shortcuts that is included in the proper place for the extension writer's favorite testing library.
PS: Actually, Gem::Builder.build is Gem::Package.build since Gem::Format, Gem::Builder and Gem::Package are getting merged into one convenient class that deals with reading and writing gem files for RubyGems 2.0. This means there will only be one place to look for the API of messing with packages and it reduces the implementation of Gem::Installer a bit.
Packaging from a gemspec is not the Best Way
drbrain |
Or, why you should use Hoe (or a tool like it) to package your gems.
A while back Yehuda wrote Using .gemspecs as Intended and asserted that packaging directly from a .gemspec file was both the way you were supposed to build gems and the best way to build gems.
This is not true.
Rake::GemPackageTask (now Gem::PackageTask) was added to rake less than two weeks after the first commit on RubyGems. Which points to the recognition of a need to have a better way to describe and build a gem than just a file.
Using rake allows you to do more than just package a static list of files in a clean and reusable manner. You can write tasks to generate files that are then packaged in your gem at build time. Hoe builds atop Rake to support many types of generated files easily through task dependencies.
For example, generating a .gemtest file to work with GemTesters, integrating with rake-compiler for pre-compiled gems or generating parsers.
I can hear you thinking "you might have a point there, but I don't generate any files, so how does this apply to me?" Well, what if you wanted to include generated man pages in your gem using binman?
In the world of packaging directly from a gemspec the way to automatically generate files is to violate RubyGems' internals by overriding Gem::Specification#initialize. I can't blame Suraj Kurapati for this, he's working within the framework of the popular thing to do and this is the only way he could implement it.
(Hoe, on the other hand, handles such pre-package dependencies through a set of pre-defined rake tasks that you can add dependencies too. If you need to generate a file you make the package task dependent on it. See the examples above.)
How can we make this better?
First of all, stop using gemspecs as the One True Description of your gem. Let rake generate it. While Hoe is obviously the best way to do this, it doesn't support generating the gemspec by default. You can use one of the gemspec plugins for that. I'm sure there are other build tools have the equivalent built right in.
Second, if you've writing a tool that generates files that will be packaged into a gem, provide a rake task like the one that ships with binman that build tool authors can hook to make packaging seamless.
marshal-structure 1.0
drbrain |
This gem is part of #rbxday, but instead of contributing to Rubinius, it mostly takes from Rubinius!
marshal-structure dumps a tree based on the Marshal format. It supports the Marshal 4.8 format.
INSTALL
gem install marshal-structure
SYNOPSIS
From the command line:
ruby -rpp -rmarshal-structure \ -e 'pp Marshal::Structure.load Marshal.dump "hello"'
Fancier usage:
require 'pp' require 'marshal-structure' ms = Marshal::Structure.new Marshal.dump %w[hello world] # print the Marshal stream structure pp ms.construct # print ruby objects in Marshal stream pp ms.objects
EXAMPLE
str =
"\004\b{\006:\006a[\031c\006Bm\006C\"\006d/\006e\000i\006" \
"f\0322.2999999999999998\000ff" \
"l+\n\000\000\000\000\000\000\000\000\001\0000TF}\000i\000" \
"S:\006S\006:\006fi\000o:\vObject\000@\017" \
"U:\006M\"\021marshal_dump" \
"Iu:\006U\n_dump\006" \
":\026@ivar_on_dump_str\"\036value on ivar on dump str" \
";\000e:\006Eo;\b\000" \
"I\"\025string with ivar\006:\v@value\"\017some value" \
"C:\016BenString\"\000"
structure = Marshal::Structure.load str
pp structure
Prints:
[:hash, 0, 1, [:symbol, 0, "a"], [:array, 1, 20, [:class, 2, "B"], [:module, 3, "C"], [:string, 4, "d"], [:regexp, 5, "e", 0], [:fixnum, 1], [:float, 6, "2.2999999999999998\u0000ff"], [:bignum, 7, 1, 10, 18446744073709551616], :nil, :true, :false, [:hash_default, 8, 0, [:fixnum, 0]], [:struct, 9, [:symbol, 1, "S"], 1, [:symbol, 2, "f"], [:fixnum, 0]], [:object, 10, [:symbol, 3, "Object"], [0]], [:link, 10], [:user_marshal, 11, [:symbol, 4, "M"], [:string, 12, "marshal_dump"]], [:instance_variables, [:user_defined, 13, [:symbol, 5, "U"], "_dump"], 1, [:symbol, 6, "@ivar_on_dump_str"], [:string, 14, "value on ivar on dump str"]], [:symbol_link, 0], [:extended, [:symbol, 7, "E"], [:object, 15, [:symbol_link, 3], [0]]], [:instance_variables, [:string, 16, "string with ivar"], 1, [:symbol, 8, "@value"], [:string, 17, "some value"]], [:user_class, [:symbol, 9, "BenString"], [:string, 18, ""]]]]
Ruby 1.9.3 preview 1
drbrain |
Ruby 1.9.3 preview 1 has been released! Please install it and report bugs! The NEWS file contains the changes since Ruby 1.9.2 and the ChangeLog contains all the gory details.
If you're using RVM you can follow these instructions to install preview 1. If not, you can download the tarball, unpack it, then run configure; make; make install as you like.
Bug Reports
Ruby 1.9.3 preview 1 ships with RDoc, RubyGems and Rake. I can fix critical bugs in all of these before the final release. Please file bug reports for Ruby via redmine for any issue you find. You can also search for existing issues to avoid duplicate reports.
RDoc
The preview 1 release contains RDoc 3.8, but I've released RDoc 3.9.1 to fix a few additional bugs. Please install RDoc 3.9.1 atop the preview 1 release, or standalone on your main ruby. I prefer reports of RDoc issues via github.
RubyGems
The preview 1 release contains RubyGems 1.8.6.1 which is slightly newer than RubyGems 1.8.6. RubyGems 1.8.7 will be released shortly with the combined changes. I prefer reports of RubyGems issues via github.
Rake
The preview 1 release contains Rake 0.9.2.1 which is slightly newer than Rake 0.9.2. I prefer reports of Rake issues via github.
Proposal: Net::HTTP user-agent class for Ruby stdlib
drbrain |
I have proposed adding a new class to Ruby's standard library that makes Net::HTTP easier to use. You can read my HTTP user-agent class proposal on redmine.
The first question is not "What should the API look like?" but "Should it be in the standard library?". My argument is this:
I have had discussions where people say "Net::HTTP's API is not very good" and people seem to want a better way to use Net::HTTP without trying out one of the many Net::HTTP::Agent-like extensions to the base API.
Having a recommended way to perform HTTP access in ruby will be useful to the community since they don't have to do any additional research. Also users will be able perform basic HTTP requests for multiple servers and will gain the efficiencies of persistent connections without extra work.
The biggest issue people have is that they need to pass the host, port and SSL params for every HTTP connection made. The nature of the API means that most people make a new connection for every request. Since this is confusing for new users and annoying for experienced users I think it should be addressed in stdlib.
Do you think a smoother API for performing HTTP requests would be useful in Ruby's standard library? If you do (or don't) I'd appreciate further discussion on the ticket.
Doc Challenge Update 2
drbrain |
Here's an update on the Ruby 1.9.3 Documentation Challenge. Since the last week's update Ruby's documentation coverage has grown to 61.35%. Perhaps we should move our goal to 75% since we've done so well?
Thanks to the following contributors for improving Ruby's documentation coverage:
Doc Challenge Update
drbrain |
Here's an update on the Ruby 1.9.3 Documentation Challenge. Since Monday, Ruby's documentation coverage has grown to 59.02%. Thanks to the following contributors for improving Ruby's documentation coverage:
- Vincent Batts
- #4695
- #4690
- #4677
- #4664
- #4690
- Simon Chiang
- #4694
- Justin Collins
- #4693
- Sebastian Martinez
- #4688
- #4687
- Mark Turner
- #4691
- #4671
- Darragh Curran
- #4684
- Jason Dew
- #4667
- Sergio Gil Pérez de la Manga
- #4678
- Pete Higgins
- #4665
- RDoc issue #34
- Steve Klabnik
- #4663
It seems my goal of 60% was pessimistic as coverage has grown 5% in only a week. There's still plenty of time to help and you can find all the resources you need to get started on the Documentation Challenge post.
Here's the report from ruby trunk r31570:
Files: 728 Classes: 1299 ( 700 undocumented) Modules: 296 ( 163 undocumented) Constants: 1295 ( 914 undocumented) Attributes: 1025 ( 479 undocumented) Methods: 10108 (3491 undocumented) Total: 14023 (5747 undocumented) 59.02% documented
Here's Monday's report:
Files: 511 Classes: 1036 ( 624 undocumented) Modules: 228 ( 136 undocumented) Constants: 1335 (1134 undocumented) Attributes: 738 ( 381 undocumented) Methods: 8098 (2960 undocumented) Total: 11435 (5235 undocumented) 54.22% documented
I also removed lib/.document which artificially restricted the files RDoc was allowed to parse which added 331 classes and modules.
Doc Challenge Update
drbrain |
Here's an update on the Ruby 1.9.3 Documentation Challenge. Since Monday, Ruby's documentation coverage has grown to 59.02%. Thanks to the following contributors for improving Ruby's documentation coverage:
- Vincent Batts
- #4695
- #4690
- #4677
- #4664
- #4690
- Simon Chiang
- #4694
- Justin Collins
- #4693
- Sebastian Martinez
- #4688
- #4687
- Mark Turner
- #4691
- #4671
- Darragh Curran
- #4684
- Jason Dew
- #4667
- Sergio Gil Pérez de la Manga
- #4678
- Pete Higgins
- #4665
- RDoc issue #34
- Steve Klabnik
- #4663
It seems my goal of 60% was pessimistic as coverage has grown 5% in only a week. There's still plenty of time to help and you can find all the resources you need to get started on the Documentation Challenge post.
Here's the report from ruby trunk r31570:
Files: 728 Classes: 1299 ( 700 undocumented) Modules: 296 ( 163 undocumented) Constants: 1295 ( 914 undocumented) Attributes: 1025 ( 479 undocumented) Methods: 10108 (3491 undocumented) Total: 14023 (5747 undocumented) 59.02% documented
Here's Monday's report:
Files: 511 Classes: 1036 ( 624 undocumented) Modules: 228 ( 136 undocumented) Constants: 1335 (1134 undocumented) Attributes: 738 ( 381 undocumented) Methods: 8098 (2960 undocumented) Total: 11435 (5235 undocumented) 54.22% documented
I also removed lib/.document which artificially restricted the files RDoc was allowed to parse which added 331 classes and modules.
Ruby 1.9.3 Documentation Challenge
drbrain |
Yugui announced a draft Ruby 1.9.3 release plan today with a feature freeze of end of May and a release target of end of July or early August.
I've seen many complaints in the form of "X lacks documentation" and have responded by writing it. First was an improvement of Net::HTTP along with Mathew Murphy and Yui NARUSE, next up was OpenSSL which received its first-ever toplevel documentation and today I committed toplevel documentation to WEBrick.
Both OpenSSL and WEBrick have some further documentation to flesh out the basic examples to help get you going but they need more work.
With revision 31499 of trunk RDoc reports that just under half of ruby has not even a single character of comments with over half of the classes and modules missing documentation:
Files: 511 Classes: 1036 ( 624 undocumented) Modules: 228 ( 136 undocumented) Constants: 1335 (1134 undocumented) Attributes: 738 ( 381 undocumented) Methods: 8098 (2960 undocumented) Total: 11435 (5235 undocumented) 54.22% documented
What can we do in the last month or so before the feature freeze to make this release the most-documented ever? To hit 60% documentation we only need to document another 1626 items.
To do my part, if you assign documentation patches to me on the Ruby 1.9 tracker on redmine I will commit them. I'll also keep writing documentation to fill in the gaps.
If you need help figuring out what to document you can look at the ruby documentation coverage report which updates hourly. You can run the full report yourself using make rdoc-coverage or run a partial report by installing the latest rdoc and running, for example, rdoc --encoding=UTF-8 -C lib/webrick* for coverage of all of WEBrick.
To keep us from stepping on each other's toes you can check this Pivotal Tracker set up by Cory Monty.
If you don't know how to contribute documentation to ruby Steve Klabnik has an excellent tutorial on contributing documentation to ruby. Note that if you send a pull request to github I can't quickly commit your change as I don't have the proper access.
