A use of Enumerable#chunk

drbrain | Fri, 16 Mar 2012 23:19:07 GMT

Posted in

In Ruby 1.9, Enumerable has a few new methods including Enumerable#chunk (which was added for 1.9.2). The #chunk method walks your Enumerable and divides it into chunks based on a selecting block. Unlike Enumerable#partition, the chunks are returned in-order. Here's an example from the documentation:

[3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5].chunk { |n|
  n.even?
}.each { |even, ary|
  p [even, ary]
}
#=> [false, [3, 1]]
#   [true, [4]]
#   [false, [1, 5, 9]]
#   [true, [2, 6]]
#   [false, [5, 3, 5]]

When I first saw this method I thought, "this looks like a useful method… but how?"

I'm working on bringing Markdown support to RDoc and the last remaining base Markdown feature I need to support is a hard break due to two spaces at the end of a line in a paragraph.

For background, RDoc parses various formats into a common syntax tree which is can then be transformed for any supported output (such as HTML, colored ANSI text, etc.). In this syntax tree a paragraph can contain one or more strings which are joined at output time into the paragraph you see.

To add hard line breaks, I decided to create a new HardBreak object and inject it into the paragraph where two trailing spaces are encountered in the source document. The formatters can then be updated to insert the appropriate line break character when emitting a paragraph.

Enumerable#chunk comes in because the Markdown parser doesn't join strings as it's parsing (since the grammar rules get re-used) and is instead performed as a post-processing step. (String joining as a post-processing step also makes the parser cleaner by hiding the ugliness in one spot rather than spreading it across multiple grammar rules.) Before inserting HardBreak objects this was sufficient:

parts = paragraph.parts.join.rstrip
paragraph.parts.replace [parts]

But now I need to join String chunks and include HardBreaks as-is which is a perfect use of Enumerable#chunk:

parts = paragraph.parts.chunk do |part|
  String === part
end.map do |string, chunk|
  string ? chunk.join.rstrip : chunk
end.flatten

paragraph.parts.replace parts

The 1.8-compatible implementation is much uglier since I have to track whether I'm in a String chunk or not in addition to performing the processing. I'm too embarrassed to post it, but you'll be able to find it in the rdoc source once I commit and push it. comments

Comments RSS FEED

Needs more flip/flop operator!

Ryan Davis said about 2 hours later

Cute.
BTW, might as well use `flat_map`.

Marc-André Lafortune said about 18 hours later

Have an RSS feed?

John Hinnegan said 3 days later

> I’m too embarrassed to post it

Don’t be embarrassed – it’s not a reflection on you.

Your identity ≠ Your code: http://collectiveidea.com/blog/archives/2012/03/16/your-identity-your-code/

Gareth Rees said 6 days later

Yes, chunk is nice! I’ve also used chunk recently, while iterating over a long, sorted list of per-minute log files generated from multiple nodes. I look for the points where the loop reaches a file from a different hour than the previous file, to divide out a new chunk. I then pass each chunk to another method that takes the list of all the log files from that hour, and combines them into an hour summary.

Chris said 6 days later

Marc: Unfortunately flat_map doesn’t exist in 1.8 either ☹

John: You can find the feed at the bottom of the sidebar on the left.

Eric Hodel said 10 days later

I have to agree with the last comment on this rahetd. we have to ask ourselves what kind of society we are at the moment. it seems that we are a very unforgiving one. If our young people or children are cautioned or convicted of a minor offence then it should theoretically be wiped from their records when turned 18. If we takeinto account various psychologists Freud, Bowlby, Carl Rogers et al we can see that according to them and in social care/social work practice it is often seen that teenagers particularly tend to rebel and then grow out of it. Its fair comment that certain incidences can and do go too far which would require further and more lengthy rehabilitation periods with the help and support from the authorities etc to ensure public safety and protection. You could even argue that such YP could be classed and are in many respects Vulnerable Adults..working with this group would entail tailored support in housing, training and work as well as equipping these youngster with the life skills and resilience to cope with life’s knocks. If you work within the field of providing support to YP on this capacity then how can that be achieved if there is widespread fear, condemnation and panic about employing a YP within Care? If employers such as Dr Fairbairn above are discriminating against these YP then what hope do we have for a fairer society? A lot of these YP tend to understand early on in their lives due to childhood traumas, familial breakdowns, abuse and neglect what it is like to not be cared for. most on this level want to care for others. Most I would argue would be and are in places excellent role models to younger children as they can share their experiences and teach through their mistakes. We are living in appalling society where a barrier is placed between young and old people. how can we break down this barrier is employers such as yourself are not willing to take a risk and employ even with a minor discrepancy on their record. ALL people no matter what age and with the proper training, support, supervision within the care sector know that if they are found to abuse a client they are sacked. They also are aware that this mean that they are barred from work with vulnerable groups probably for life. We all know codes of practice for ourselves as workers and the responsibilities of employment agencies or orgs in ensuring risk assessments are undertaken, and clients rights are respected on every level. If a YP is barred for a discrepancy in their childhood by you as an employer then its fair to say that we are all barred for life in our older adulthood. This is why hundreds of thousands of individuals are barred from working ANYwhere for life it seems. You do not have the role of judge and juror. That role is fundamentally not assigned to you. that is the role of the courts. That task has already been undertaken and the penalty or reprimand already given. As a result Dr Fairbairn there are hundreds of thousands if not millions of people FORCED to claim benefits as a result of the persistent discrimination of ex offenders and even more disgustingly those with mere cautions or who have had a malicious allegation made against them. We have a million YP on benefits at the moment. because cautions and convictions are given more readily these days how many more YP will we have on the dole in say 5 years? How many of these YP do you think may have a minor criminal record? At what stage do employers stop discriminating and start trusting again?

Sooturraxma said 13 days later

Comments are disabled