Inliners are Hard to Debug
Eric Hodel | Sun, 01 Jul 2007 08:40:00 GMT
So I wrote SuperCaller
require 'rubygems'
require 'super_caller'
require 'super_caller/exception'
class X
def y; z; end
def z; raise; end
end
begin
X.new.y
rescue => e
e.backtrace .each do |frame|
puts frame
puts frame.source || frame.sexp.inspect
puts "---"
end
endGives:
/usr/local/lib/ruby/gems/1.8/gems/SuperCaller-1.0.0/lib/super_caller/exception.rb:12:in `initialize' def initialize(message) old_initialize(message) @backtrace = super_caller end --- test.rb:7:in `new' [[:vcall, :raise]] --- test.rb:7:in `z' [[:vcall, :raise]] --- test.rb:6:in `y' def y z end --- test.rb:11 [[:call, [:call, [:const, :X], :new], :y]] ---
Coming Soon
I wrote an inliner
Eric Hodel | Wed, 27 Jun 2007 07:35:00 GMT
$ ruby test.rb
caller result: 12
caller:
def caller
v1 = (2 + 3)
x = callee(v1)
(x + 2)
end
callee:
def callee(v)
(v + 5)
end
inline callee into caller
caller result: 12
caller:
def caller
v1 = (2 + 3)
x = (inline_callee_v = v1
(inline_callee_v + 5))
(x + 2)
end
$ ruby -Ilib bm.rb
Rehearsal -------------------------------------------
empty 0.090000 0.000000 0.090000 ( 0.083842)
plain 1.030000 0.010000 1.040000 ( 1.037302)
inlined 0.810000 0.000000 0.810000 ( 0.821394)
---------------------------------- total: 1.940000sec
user system total real
empty 0.080000 0.000000 0.080000 ( 0.084179)
plain 1.100000 0.000000 1.100000 ( 1.105742)
inlined 0.900000 0.000000 0.900000 ( 0.900799)
Tim Lucas on Logging Rails to SysLog with SyslogLogger
Eric Hodel | Wed, 06 Jun 2007 21:09:02 GMT
Tim on switching to SyslogLogger and Production Log Analyzer:
One of the benefits of switching your Rails app to use SysLog is taking advantage of your OS’s system-wide logging, as well as having finer grained control over how to process your log messages. For example, you can split the logs up based on Rails app, mongrel process, or even by app server if you’re using remote syslog.
[...]
...and now we’re using SysLog we can use Eric’s other tool, ProductionLogAnalyzer, to identify performance bottleknecks of our application. Geoffrey Grosenbach (aka topfunky) wrote a Hodel3000CompliantLogger if you want to use this tool without SysLog).
— Logging Rails to SysLog with SyslogLogger via tech on toolmantim.com
ar_mailer version 1.2.0 has been released!
Eric Hodel | Tue, 05 Jun 2007 22:28:52 GMT
Even delivering email to the local machine may take too long when you have to send hundreds of messages. ar_mailer allows you to store messages into the database for later delivery by a separate process, ar_sendmail.
sudo gem install ar_mailer
Changes:
1.2.0
- Bugs fixed
- Handle SMTPServerBusy by backing off @delay seconds then re-queueing
- Allow email delivery class to be set in ARMailer.
- ar_sendmail—mailq works with—table-name now.
- Miscellaneous Updates
- Added documentation to require ‘action_mailer/ar_mailer’ in instructions.
- Moved to ZSS p4 repository
- Supports TLS now. Requested by Dave Thomas. smtp_tls.rb from Kyle Maxwell & etc.
production_log_analyzer version 1.5.0 has been released!
Eric Hodel | Wed, 16 May 2007 21:43:57 GMT
production_log_analyzer provides three tools to analyze log files created by SyslogLogger. pl_analyze for getting daily reports, action_grep for pulling log lines for a single action and action_errors to summarize errors with counts.
http://seattlerb.rubyforge.org/production_log_analyzer
sudo gem install production_log_analyzer
Changes:
1.5.0
- Fixed empty log bug. Patch by Tim Lucas.
- Fixed bug where sometimes lines would be logged before the Processing line. Patch by Geoff Grosenbach.
rails_analyzer_tools version 1.4.0 has been released!
Eric Hodel | Wed, 16 May 2007 21:41:45 GMT
Rails Analyzer Tools contains Bench, a simple web page benchmarker, Crawler, a tool for beating up on web sites, RailsStat, a tool for monitoring Rails web sites, and IOTail, a tail(1) method for Ruby IOs.
http://seattlerb.rubyforge.org/rails_analyzer_tools
sudo gem install rails_analyzer_tools
Changes:
1.4.0
- Moved to seattlerb project.
SyslogLogger 1.4.0 Released
Eric Hodel | Wed, 09 May 2007 03:44:01 GMT
SyslogLogger version 1.4.0 has been released!
http://seattlerb.rubyforge.org/SyslogLoggerSyslogLogger is a Logger replacement that logs to syslog. It is almost drop-in with a few caveats.
Changes:
1.4.0 / 2007-05-08
- Split from rails_analyzer_tools.
- Added eh methods for compatibility with Logger.
- Added syslog-ng instructions. Patch by Tom Lianza.
- Fixed require in documentation. Reported by Gianni Jacklone.
Process Growth with Railsbench's GC patch
Eric Hodel | Tue, 17 Apr 2007 19:57:00 GMT
My friend Kevin Watt who runs Allpoetry had me help him diagnose problems with memory consumption on his Rails application. He was seeing processes suddenly jump in size by 46MB, when the processes were already over 100MB to begin with. After instrumenting to locate the growth in one innocuous method we went through his patches to Rails.
One of the patches Kevin was using was the Railsbench GC patch. It turns out that this patch has some very useful instrumenting of its own, so I had Kevin call GC.dump in that method, and he got the following values:
HEAP[ 0]: size= 650000 HEAP[ 1]: size=1170001
This means there are slots for 1820001 objects in his process. If you end up creating that many objects Ruby will create a new chunk of object slots by multiplying the previous allocation by 1.8 and adding 1.
The next set of slots would contain slots for 2106002 objects. Each slot takes up 20 bytes of memory, so roughly 40MB of memory would be used. Kevin ended up pulling the patch and now has his processes holding on to around 70MB RSS each, rather than 100 or more.
The Railsbench page contains this note about memory consumption alongside the GC patch:
In addition, a patch for the ruby garbage collector is provided, which can be used to reduce the amount of time spent doing garbage collection, trading memory for speed, as usual (see file GCPATCH for details).
The initial size of the slots can be tuned, but we neglected to explore that option. In large-memory situations the patch's behavior is too negative. Be sure to follow the instructions to the GCPATCH file to get proper operation. Also, monitor your application's memory usage as the patch may cause your machine to use swap when too many objects are being used or if you've tuned improperly.
memcache-client version 1.3.0 has been released!
Eric Hodel | Wed, 07 Mar 2007 05:29:02 GMT
http://seattlerb.rubyforge.org/memcache-client
memcache-client is a pure-ruby client to Danga’s memcached.
Changes:
1.3.0
- Apply patch #6507, add stats command. Submitted by Tyler Kovacs.
- Apply patch #6509, parallel implementation of #get_multi. Submitted by Tyler Kovacs.
- Validate keys. Disallow spaces in keys or keys that are too long.
- Perform more validation of server responses. MemCache now reports errors if the socket was not in an expected state. (Please file bugs if you find some.)
- Add #incr and #decr.
- Add raw argument to #set and #get to retrieve #incr and #decr values.
- Also put on MemCacheError when using Cache::get with block.
- memcache.rb no longer sets $TESTING to a true value if it was previously defined. Bug #8213 by Matijs van Zuijlen.
Controlling Rails Process Size: Update
Eric Hodel | Wed, 28 Feb 2007 20:56:24 GMT
A while back Simon Lundström wrote me because he was having problems limiting process sizes using the Process::setrlimit as described in my previous post, Controlling Rails Process Size.
After some back-and-forth we found that on Debian, to limit process sizes use the Process::RLIMIT_MEMLOCK and Process::RLIMIT_AS constants.
On OS X we couldn’t find any RLIMIT_* constants that would effect a limit, nor when setting limits using the ulimit shell builtin. (Probably because OS X doesn’t use the UNIXy brk(2) syscall when you malloc(3) memory.) If somebody knows the magic to limit process size on OS X, I’d like to hear it.
Articles