Sphincter version 1.1.0 has been released!
Eric Hodel | Tue, 14 Aug 2007 03:10:51 GMT
Sphincter uses Dmytro Shteflyuk’s sphinx Ruby API and automatic configuration to make totally rad ActiveRecord searching. Well, you still have to tell Sphincter what models you want to search. It doesn’t read your mind.
When updating to 1.1.0, run: rake sphincter:setup_sphinx
Changes in 1.1.0:
- 2 major enhancements:
- Fields across relationships may be included via add_index.
- Sphincter now automatically configures Dmytro Shteflyuk’s sphinx API. Run `rake sphincter:setup_sphinx` and check in vendor/plugins/sphinx.
- 1 bug fix:
- `rake sphincter:index` task didn’t correctly run reindex. Bug submitted by Lee O’Mara.
ar_mailer version 1.3.1 has been released!
Eric Hodel | Wed, 01 Aug 2007 04:52:18 GMT
Even delivering email to the local machine may take too long when you have to send hundreds of messages. ar_mailer allows you to store messages into the database for later delivery by a separate process, ar_sendmail.
Changes:
- 1.3.1
- Fix bug #12530, gmail causes SSL errors. Submitted by Kyle Maxwell and Alex Ostleitner.
- Try ActionMailer::Base::server_settings then ::smtp_settings. Fixes bug #12516. Submitted by Alex Ostleitner.
Sphincter version 1.0.0 has been released!
Eric Hodel | Mon, 30 Jul 2007 22:12:18 GMT
http://seattlerb.org/Sphincter
Sphincter uses Dmytro Shteflyuk’s sphinx Ruby API and automatic configuration to make totally rad ActiveRecord searching. Well, you still have to tell Sphincter what models you want to search. It doesn’t read your mind.
Features:
- Automatically configures itself.
- Handy set of rake tasks for easy, automatic management.
- Automatically adds has_many metadata for searching across the association.
- Stub for testing without connecting to searchd, Sphincter::SearchStub.
- Easy pagination support.
- Filtering by index metadata and ranges, including dates.
Problems:
See README.txt for quick-start, examples, etc.
Changes:
1.0.0 / 2007-07-26
- 1 major enhancement
- Birthday!
Tim Lucas on Logging Rails to SysLog with SyslogLogger
Eric Hodel | Wed, 06 Jun 2007 21:09:02 GMT
Tim on switching to SyslogLogger and Production Log Analyzer:
One of the benefits of switching your Rails app to use SysLog is taking advantage of your OS’s system-wide logging, as well as having finer grained control over how to process your log messages. For example, you can split the logs up based on Rails app, mongrel process, or even by app server if you’re using remote syslog.
[...]
...and now we’re using SysLog we can use Eric’s other tool, ProductionLogAnalyzer, to identify performance bottleknecks of our application. Geoffrey Grosenbach (aka topfunky) wrote a Hodel3000CompliantLogger if you want to use this tool without SysLog).
— Logging Rails to SysLog with SyslogLogger via tech on toolmantim.com
ar_mailer version 1.2.0 has been released!
Eric Hodel | Tue, 05 Jun 2007 22:28:52 GMT
Even delivering email to the local machine may take too long when you have to send hundreds of messages. ar_mailer allows you to store messages into the database for later delivery by a separate process, ar_sendmail.
sudo gem install ar_mailer
Changes:
1.2.0
- Bugs fixed
- Handle SMTPServerBusy by backing off @delay seconds then re-queueing
- Allow email delivery class to be set in ARMailer.
- ar_sendmail—mailq works with—table-name now.
- Miscellaneous Updates
- Added documentation to require ‘action_mailer/ar_mailer’ in instructions.
- Moved to ZSS p4 repository
- Supports TLS now. Requested by Dave Thomas. smtp_tls.rb from Kyle Maxwell & etc.
production_log_analyzer version 1.5.0 has been released!
Eric Hodel | Wed, 16 May 2007 21:43:57 GMT
production_log_analyzer provides three tools to analyze log files created by SyslogLogger. pl_analyze for getting daily reports, action_grep for pulling log lines for a single action and action_errors to summarize errors with counts.
http://seattlerb.rubyforge.org/production_log_analyzer
sudo gem install production_log_analyzer
Changes:
1.5.0
- Fixed empty log bug. Patch by Tim Lucas.
- Fixed bug where sometimes lines would be logged before the Processing line. Patch by Geoff Grosenbach.
rails_analyzer_tools version 1.4.0 has been released!
Eric Hodel | Wed, 16 May 2007 21:41:45 GMT
Rails Analyzer Tools contains Bench, a simple web page benchmarker, Crawler, a tool for beating up on web sites, RailsStat, a tool for monitoring Rails web sites, and IOTail, a tail(1) method for Ruby IOs.
http://seattlerb.rubyforge.org/rails_analyzer_tools
sudo gem install rails_analyzer_tools
Changes:
1.4.0
- Moved to seattlerb project.
SyslogLogger 1.4.0 Released
Eric Hodel | Wed, 09 May 2007 03:44:01 GMT
SyslogLogger version 1.4.0 has been released!
http://seattlerb.rubyforge.org/SyslogLoggerSyslogLogger is a Logger replacement that logs to syslog. It is almost drop-in with a few caveats.
Changes:
1.4.0 / 2007-05-08
- Split from rails_analyzer_tools.
- Added eh methods for compatibility with Logger.
- Added syslog-ng instructions. Patch by Tom Lianza.
- Fixed require in documentation. Reported by Gianni Jacklone.
Process Growth with Railsbench's GC patch
Eric Hodel | Tue, 17 Apr 2007 19:57:00 GMT
My friend Kevin Watt who runs Allpoetry had me help him diagnose problems with memory consumption on his Rails application. He was seeing processes suddenly jump in size by 46MB, when the processes were already over 100MB to begin with. After instrumenting to locate the growth in one innocuous method we went through his patches to Rails.
One of the patches Kevin was using was the Railsbench GC patch. It turns out that this patch has some very useful instrumenting of its own, so I had Kevin call GC.dump in that method, and he got the following values:
HEAP[ 0]: size= 650000 HEAP[ 1]: size=1170001
This means there are slots for 1820001 objects in his process. If you end up creating that many objects Ruby will create a new chunk of object slots by multiplying the previous allocation by 1.8 and adding 1.
The next set of slots would contain slots for 2106002 objects. Each slot takes up 20 bytes of memory, so roughly 40MB of memory would be used. Kevin ended up pulling the patch and now has his processes holding on to around 70MB RSS each, rather than 100 or more.
The Railsbench page contains this note about memory consumption alongside the GC patch:
In addition, a patch for the ruby garbage collector is provided, which can be used to reduce the amount of time spent doing garbage collection, trading memory for speed, as usual (see file GCPATCH for details).
The initial size of the slots can be tuned, but we neglected to explore that option. In large-memory situations the patch's behavior is too negative. Be sure to follow the instructions to the GCPATCH file to get proper operation. Also, monitor your application's memory usage as the patch may cause your machine to use swap when too many objects are being used or if you've tuned improperly.
Test Profiling by Lines Logged
Eric Hodel | Mon, 22 Jan 2007 23:41:00 GMT
At work I've been cleaning up the tests and trying to make them run faster. One way of doing this is profiling the tests and fixing the slow spots. That only works so well, especially if there's lots of duplication or extra work in the tests. With Rails you can go about this a different way, since you have a second source of information on your tests' operation, the log file.
I added the following code to test/test_helper.rb:
raise 'require\'d test/test_helper twice!, you broke it!' if
ENV['RAILS_ENV'] == 'test'At the very top, even above ENV["RALIS_ENV"] = "test" to make sure that the hack to Test::Unit::TestCase below happens only once, then the magic to figure out which test outputs which log lines:
class Test::Unit::TestCase
alias unlogged_run run
def run(result, &block)
RAILS_DEFAULT_LOGGER.debug "RUNNING #{self.class} #{@method_name}"
unlogged_run result, &block
end
end if ENV['PROFILE_LOG']Each test run will be prefixed with text like RUNNING SomeTest test_blah which I can then run this script on:
#!/usr/local/bin/ruby -w
test = nil
tests = Hash.new 0
File.open 'log/test.log' do |fp|
fp.each_line do |line|
if line.strip =~ /^RUNNING (.*)/ then
test = $1
else
tests[test] += 1
end
end
end
tests.sort_by { |test, count| -count }.each do |test, count|
puts "%5d %s" % [count, test]
end
To get the profile information, I run:
rake log:clear; PROFILE_LOG=y rake && script/count_logs
When run on Firebrigade, I get the following output (top 10 only):
130 ProjectViewTest test_show 74 DummyControllerTest test_error_500 59 OwnerTest test_class_owner_count 42 ProjectControllerTest test_index 39 HomeControllerTest test_index_no_builds 38 RestControllerTest test_add_build 35 ProjectControllerTest test_show_no_versions 35 ProjectControllerTest test_search_many_matches 34 RestControllerTest test_add_project 33 RestControllerTest test_add_version
Now I can quickly discover good candidates for refactoring. #test_error_500 above dumps an email into the logs, so it is a bogus result. That leaves ProjectViewTest#test_show as a candidate for simplification or refactoring.
Articles