Working Style

Eric Hodel | Fri, 22 Aug 2008 00:42:25 GMT

In Rubinius we’ve been slowly working towards using a VM written in C++ instead of the current C VM we’ve got (called “shotgun”). Most of the mechanics of the VM are already complete, so over the past couple weeks we’ve been wiring up the primitives necessary for our ruby kernel (equivalent to Ruby’s core library) to run.

This primitive work is rather tedious, and until Friday I found it highly unenjoyable. I couldn’t perceive how much closer I was to our goal of actually running ruby code on the new VM.

On Friday I recalled that we can run a compiled ruby file against the VM, which gave me a way to shoot towards my goal. Over the past couple days during our Rubinius Team Meeting we’ve gotten up to loading all of our kernel bootstrap and platform, and now we’re just starting to do work in the core libraries.

Switching my working style drastically improved my enjoyment of a process that I found quite annoying. I wish I’d figured this out a couple weeks ago.

Posted in ,  | no comments

Return of Syck

Eric Hodel | Thu, 12 Jun 2008 18:30:20 GMT

From #rubinius, the royal struggle between syck and RbYAML.

Defiler: re rbyaml also
Defiler: Why aren't we using syck though?
Defiler: rbyaml is totally not compliant with what MRI is expecting to
         have happen
drbrain: syck period, or the syck extension from MRI?
Defiler: "Why are we using rbyaml" is what I should have said, I guess
drbrain: Back in Ye Olden Times of Late Last December
drbrain: We had a wrapper for the syck library that was not MRI's syck
         extension
drbrain: For in Ye Olden Days, Fair Subtend was not so Fair
drbrain: Thus, a Newcomer was Raised to Ye Throne of YAML
drbrain: RbYAML
drbrain: Now, with the waxing power of Subtend, the MRI syck extension
         may reclaim its Rightful Crown

Posted in  | no comments

.au Sprint

Eric Hodel | Tue, 11 Mar 2008 09:16:16 GMT

I spent Saturday and Sunday at the Sydney Rubinius sprint, and it was quite productive. Not only were local beers sampled, but we also got some good work done.

Lincoln Stoll helped me shake the last bugs out of RubyGems, so we integrated it into Rubinius. We decided to make it a subcommand rbx gem like rbx compile or rbx describe. There are still a few things broken in RubyGems, namely installing gems with extensions because mkmf.rb doesn’t work in Rubinius.

Lincoln also pointed out and gave me patches for a few backwards-compatibility problems with RDoc, so now both RubyGems and RDoc work on Rubinius.

Upon my return next week I’ll be cooking up a release of RubyGems with a handful of bug fixes and persistent connection support for RubyGems’ HTTP fetching which will make those incremental updates quite a bit faster.

Posted in ,  | 1 comment

RubyGems in Rubinius

Eric Hodel | Thu, 14 Feb 2008 23:58:00 GMT

Last night I checked an improved version of Zlib into Rubinius, and now gems can be installed: 

$ ~/Links/RUBINIUS/shotgun/rubinius \
    -Ilib bin/gem install rake --no-ri --no-rdoc -i ~/tmp/rbxgems
Successfully installed rake-0.8.1
1 gem installed

$ GEM_HOME=~/tmp/rbxgems ~/Links/RUBINIUS/shotgun/rubinius \
    -Ilib ~/tmp/rbxgems/bin/rake --help
rake [-f rakefile] {options} targets...

Options are ...
[...]

Building the ri and RDoc information doesn't work correctly yet on Rubinius, but Rubinius is now using RDoc to generate some documentation, particularly the VM opcode information (run `rake doc:vm` in a Rubinius checkout).

Next I'm going to work on the RDoc/ri end of things some more, including improving or importing more documentation into Rubinius, and make a release of my recent RDoc changes.

Posted in  | 3 comments

Rubinius' Foreign Function Interface

Eric Hodel | Tue, 15 Jan 2008 09:22:00 GMT

I really, really, really love Rubinius’ Foreign Function Interface (FFI) since it allows you to replace C code with Ruby code. Earlier today I wrote Socket::getaddrinfo in C for Rubinius, and just now I finished a rewrite using FFI and Ruby. I’ve commented the code for clarity. 

def self.getaddrinfo(host, service, family = nil, socktype = nil,
                     protocol = nil, flags = nil)
  service = service.to_s

  # MemoryPointer.new is kind-of like malloc(3), but understands what's inside
  hints_p = MemoryPointer.new Socket::Foreign::AddrInfo.size

  # Socket::Foreign::AddrInfo is a struct addrinfo wrapper with friendly accessors
  hints = Socket::Foreign::AddrInfo.new hints_p
  hints[:ai_family] = family || 0
  hints[:ai_socktype] = socktype || 0
  hints[:ai_protocol] = protocol || 0
  hints[:ai_flags] = flags || 0

  # getaddrinfo(3) asks for a struct addrinfo **.
  # This creates a pointer to a pointer
  res_p = MemoryPointer.new :pointer

  # call out to C
  err = Socket::Foreign.getaddrinfo host, service, hints_p, res_p

  # check for errors
  raise SocketError, Socket::Foreign.gai_strerror(err) unless err == 0

  # now we read out the pointer that getaddrinfo() passed us, and cast it
  # to a struct addrinfo *
  res = Socket::Foreign::AddrInfo.new res_p.read_pointer

  addrinfos = []

  loop do
    addrinfo = []

    # Extract data
    addrinfo << Socket::Constants::AF_TO_FAMILY[res[:ai_family]]

    ai_sockaddr = res[:ai_addr].read_string res[:ai_addrlen]

    sockaddr = Socket::Foreign::unpack_sa_ip ai_sockaddr, true

    addrinfo << sockaddr.pop # port
    addrinfo.concat sockaddr # hosts
    addrinfo << res[:ai_family]
    addrinfo << res[:ai_socktype]
    addrinfo << res[:ai_protocol]

    addrinfos << addrinfo

    # struct addrinfo is a linked list, so if we've hit the end, stop
    break unless res[:ai_next]

    # otherwise, down the linked-list
    res = Socket::Foreign::AddrInfo.new res[:ai_next]
  end

  return addrinfos
ensure
  # like a C code, we have to free our MemoryPointer objects
  hints_p.free if hints_p

  if res_p then
    # also, we have to do any C-side cleanup
    Socket::Foreign.freeaddrinfo res_p.read_pointer
    res_p.free
  end
end

getaddrinfo(3), freeaddrinfo(3) and gai_strerror(3) are wrapped up by FFI like this: 

attach_function "gai_strerror", :gai_strerror, [:int], :string

attach_function "getaddrinfo", :getaddrinfo,
                [:string, :string, :pointer, :pointer], :int

attach_function "freeaddrinfo", :freeaddrinfo, [:pointer], :void

The first argument is the C function name, the second is the Ruby name, the third is the input arguments, and the fourth is the return type. Currently, FFI can only wrap up C functions with six or fewer args.

The AddrInfo struct is wrapped up like this: 

class AddrInfo < FFI::Struct
  config("rbx.platform.addrinfo", :ai_flags, :ai_family, :ai_socktype,
         :ai_protocol, :ai_addrlen, :ai_addr, :ai_canonname, :ai_next)
end

The config method pulls pre-generated struct information out of a Rubinius config file and hooks up accessors to each of the struct’s fields. The accessors know which offset into the struct the data lives at and what type to convert data from and to when working with the struct. The information is collected at Rubinius build time by a small bit of C code.

I still have some confusion between passing an FFI::Struct like Socket::Foreign::AddrInfo vs. passing a MemoryPointer instance (which is what an FFI wrapped function understands) to an FFI-wrapped function, so we’re going to clean up that part of the API to make it more natural. Instead you’ll be able to initialize an FFI::Struct directly and pass it to the FFI-wrapped function. This will make the code quite a bit cleaner.

Posted in  | 15 comments

RubyGems on Rubinius

Eric Hodel | Sun, 13 Jan 2008 06:36:00 GMT

Friday I passed an important milestone for RubyGems on Rubinius, test/test_gem.rb ran to completion! Here are the dots to prove it: 

$ ~/Links/RUBINIUS/shotgun/rubinius -Ilib:test test/test_gem.rb
Loaded suite test/test_gem
Started
.................................
Finished in 18.667198 seconds.

33 tests, 56 assertions, 0 failures, 0 errors

These tests include building, installing, and activating a Gem, so it should be possible now to use RubyGems in Rubinius. Unfortunately, the `gem` command doesn't work yet. That will be among my next tasks.

The final holdup for these RubyGems tests was a Dir.glob that handled {}. Finishing this off should also allow rake to work, according Evan's recent work on rake.

A full run of all RubyGems tests reports: 

100 tests, 221 assertions, 6 failures, 13 errors

A full run with MRI is 533 tests, 1816 assertions. The 433 missing tests are due to a missing feature of Module#module_function in Rubinius that prevents the remaining test files from loading.

If you'd like to duplicate my results, here's how:

Posted in ,  | 3 comments

"What should I work on in Rubinius?"

Eric Hodel | Thu, 10 Jan 2008 07:23:35 GMT

Occasionally, people drop into the Rubinius IRC channel and ask what they should work on. The answer in the past has been, “write some specs” which can be some terribly boring work and is not very motivational. Fortunately, I have a solution for that.

I think the best way to get involved in Rubinius is to take your favorite Ruby project and try to run it on Rubinius. Whatever problem you run into first, fix that (including writing specs, of course).

My work on running RubyGems in Rubinius has involved running the RubyGems tests, seeing where they fail, then moving over to Rubinius and fixing whatever failure I’m having. This way I’m highly motivated because my end goal, making RubyGems work, is easily in sight. (Of course, when RubyGems is working, I’ll need a new goal, maybe RDoc).

Posted in  | no comments

Rubinius

Eric Hodel | Thu, 06 Dec 2007 02:28:02 GMT

I’m down in San Francisco this week working on Rubinius with Evan, Ryan, Wilson and Brian, which has been my Top Secret job at Engine Yard for a little over a month. Also joining us has been Josh, Kevin and Nathan (among others) have stopped by to hang out and hack too.

Primarily I’ve been working on getting RubyGems working on Rubinius, along with build system and other cleanups. I’ve started by trying to get just test_gem running, and I’ve become hung up waiting for Kernel#eval and Kernel#binding, which will be finished with the compiler2 work.

Posted in  | 1 comment