Twitter4R shifts RSpec onto My Front Burner

… so much so, that I thought to write a post about it.

I promise I’ll get to Twitter4R in a moment – but first, a prelude. Think of it like eating dessert before dinner, except that dessert is a testing framework.

Coming Up To Speed on Rspec

Learning how to write rspec had been on my list for a while. And this development effort allowed me to finally got around to that. It’s a nice framework. I am familiar with EasyMock, and am aware of JMock. I never had an opportunity to get into Mockito, but I would give it a glance the next time I put on my Java hat.

There is some decent rspec documentation out there. I found these links particularly helpful:

The formal documentation.
David Chelimsky’s from-the-ground-up description
rspec.info’s drill-down into Mocks and Stubs
Luke Redpath’s cross-over into usage under Rails

To ramp-up up on this codebase I’d be extending, I took the usual meta-approach of creating a Test Suite – yaaay, that’s what rspec is meant for! – which I then used to test out Twitter4R’s range of capabilities. The Modules in the rspec rdoc which I’ve found to provide the most value are:

Spec::Expectations::ObjectExpectations for conditionals (eg. should & should_not)
Spec::Matchers for expectations (eg. equal(*value*), be_a(*class*), respond_to(*method_sym*), raise_error)
Spec::Mocks::Methods for mock method definition (eg. should_receive(*method_sym*))
Spec::Mocks::MessageExpectation for mock behaviour (eg. with(*args*), once, exactly(*n*).times, any_number_of_times)
Spec::Mocks::ArgumentConstraints for mock arguments (eg. an_instance_of(*class*), anything)
Spec::Mocks::BaseExpectation for mock responses (eg. and_return(*value*), and_yield(*&block*))

I won’t got into the deep details, but here are some examples of conditionals and expectations that I paraphrased into my meta-test:

specify "knowledge of nil" do ''.should_not be_nil end specify "numeric calculations" do (355.0 / 113).should be_close(Math::PI, 0.1) end specify "changes made by a closure" do array = [] lambda { array << :a }.should change(array, :size) end specify "equality" do 5.should eql 5 5.should equal 5 [:a].should_not equal [:a] end specify "raising of errors" do # always construct fresh!!! # otherwise it won't be re-wrappable def raiser(s=nil) lambda { raise(RuntimeError, s) if s } end raiser('x').should raise_error raiser('y').should raise_error(RuntimeError, 'y') end specify "closure satisfaction" do 5.should satisfy {|n| (4..6).include?(n) } end specify "what lists do" do [].should respond_to(:each, :find, :size) end

And here’s a little bit of silly mocking:

def expect_raise(type=Spec::Mocks::MockExpectationError, &block) abort 'block must be provided' unless block_given? block.should raise_error(type) end specify "the basics" do @mock.should_receive :hello @mock.should_not_receive :goodbye @mock.hello expect_raise { @mock.stay } expect_raise { @mock.goodbye } end specify "how many times, and with what" do @mock.should_receive(:one).once.with(1) @mock.one 1 @mock.should_receive(:string).exactly(1).times.with(an_instance_of(String)) @mock.string 'a string' @mock.should_receive(:anything).exactly(3).times.with(any_args()) @mock.anything @mock.anything :again @mock.anything :third, 'time' @mock.should_receive(:array_ish).with(duck_type(:each, :find, :size)) @mock.array_ish [:item] end specify "what i return, raise or throw" do @mock.should_receive(:get_one).and_return(1) @mock.should_receive(:put_one).with(1) @mock.put_one @mock.get_one @mock.should_receive(:increment).any_number_of_times.with(instance_of(Fixnum)).and_return {|i| i + 1 } @mock.increment(1).should equal(2) @mock.should_receive(:raises_string).and_raise('something runtime') expect_raise(RuntimeError) { @mock.raises_string } @mock.should_receive(:pitch).and_throw(:ball) lambda { @mock.pitch }.should throw_symbol(:ball) end specify "yielding in a complex fashion" do @mock.should_receive(:gimmee).exactly(3).times.and_yield(:x) holder = [] @mock.gimmee {|value| holder << value } holder.should eql([:x]) 2.times { @mock.gimmee {|value| holder << value } } holder.should eql([:x, :x, :x]) end specify "validation via closure" do (@mock.should_receive(:threely) do |value| value.to_s.size.should eql(3) end).exactly(3).times @mock.threely 'x' * 3 @mock.threely :key @mock.threely 333 end

These are just ways I thought of to exercise the width and breadth of the Twitter4R’s featureset. Very nice. I hope that these are useful examples for people new to this gem.

I recommend the other rspec references from above for filling in the missing details. Once you have a context / specify or a describe / it specification set up, you’ll be good to go. There’s much more to the library – Stories, for instance – but that’s for another day.

An Informative Walk through Twitter4R

Yes, now onto our core story.

Because no, ultimately this effort wasn’t about sinking time into learning rspec – rather, it was a means to an end, since I had a personal need to expand the Twitter4R gem. Specifically, I wanted to add on some Twitter search features. And contribution-wise, the final step that Susan Potter recommends is to craft up some rspec Test Cases.

Of course, mock testing is only as good as the framework you’re built upon. Often it can be avoided. I built my Bitly4R gem (given that name, you can see that my thinking has clearly been influenced), I was able to do everything as full-on functional tests. It was easy; the bit.ly service has both shorten and expand endpoints, so I could reverse-test real values without having any fixed expectations.

However, the Twitter service is live and user-generated, so who knows what you’ll find. A good approach is to pinpoint a cross-cutting concern in your stack and write a mock around it. In in this case, the library relies upon Net::HTTP. That seems like a good place to simulate responses. Of course, not having to hit the Twitter servers shortens the testing time dramatically. Anc you mock it up, and you can even test your features offline.

Here’s one of my Test Suites, again foreshortened:

before(:each) do Twitter::Client.send :public, :create_http_get_request @client = Twitter::Client.new @client_clone = Twitter::Client.new end it "produces a querystring with all the crazy stuff Twitter imagined" do the_params = nil @client.should_receive(:create_http_get_request).and_return {|uri, params| the_params = params @client_clone.create_http_get_request(uri, params) } @client.should_receive(:http_connect).and_return Net::HTTPNotFound.mock(:code => '404') # pass through a few we can pick back out # also, side-effect ... nil on invalid response @client.search('query_value', :rpp => 3).should be_nil the_params.should_not be_nil the_params[:q].should eql('query_value') the_params[:rpp].should eql(3) end

Plus a little mocking of Net::HTTPResponse:

class Net::HTTPResponse def self.mock(ops={}) # defaults ops = { :http_version => '1.1', :code => '200', :message => 'OK', :body => nil, }.merge(ops || {}) # construct ###clazz = CODE_TO_OBJ[ops[:code]] clazz = self response = clazz.new(ops[:http_version], ops[:code], ops[:message]) # inject ops.each do |k, v| response.instance_variable_set "@#{k}".to_sym, v end # mockulate response.instance_eval %q{ def body; @body; end } response end end

Nothing like having some good inspiration to get you thinking about maintainability. I’m a not a strong adherent to any of the TDD or BDD denominations, but testability itself is close to my heart. Just ask the folks who’ll be looking at the micro-assertion Module that I wrote for my Facebook Puzzle submissions (which work locally under test conditions but fail miserably once thrown over their wall). Then again, I won’t need to write that (and test that) from scratch again.

So, back to talking glowingly about the Twitter4R architecture. Yes, I used the “A” word. Yet regardless of that term’s heavyweightedness, it’s simply the result of carefully thinking out the formalization of a complex design. And once a non-author delves into extending an existing implementation, that impl’s demeanor becomes readily clear

Some of the interesting things I saw:

An inner bless-ish strategy, to inject the Twitter::Client into instanciated result classes. I’ve seen blessing as an OO-Perl-ism, for self-ication, and that metaphor carries over very nicely to this context.
Generous use of blocks / lambdas / closures in methods, for contextual iteration (eg. iterate through each Twitter::Message in a timeline response). Unnecessary, but an excellent convenience for a language that offers that optional capability in a very disposable fashion (from the caller’s perspective).
Retroactive sub-object population after construction. Twitter4R relies upon the json gem, which deals primarily in Hashes. Post-injection, the library itereates through Arrays of Hashes and transforms them into suitable Twiltter::User containers, etc. A great place to put such logic in the construction chain, and it doesn’t take long to get really tired of Hash hierarchies.

Good stuff, and learning with in a Ruby-centric mindset was invaluable for me. We all have to start somewhere, eh.

The Acute Long-Term Pain of Staticness

There was one issue that I ran into; static configuration. During my years of using the Spring Framework, my IOC-addled brain started thinking of everything in terms of instances – Factory Beans, POJOs, Observers & Listeners, Chain-of-Responsibility wrappers. Static configuration is a common metaphor, and in this case, there’s a static Twitter::Config instance. Convenient and centralized. Makes perfect sense.

I mean, the fact that it was a configurable library at all was awesome. I was able to easily set up a Twitter::Client to reference search.twitter.com. However, of course as soon as I did that, I whacked the ability for clients to talk to twitter.com in the process. Oops!

On GPs, I refused to modify the original code. And I wanted to make sure that my superficial tweaks to the library would be thread-safe – temporarily swaping out the global Twitter::Config in mid-operation would be an issue. Using Mutex.synchronize seemed like the perfect choice. After finding that the same thread can’t lock a Mutex instance twice – grr!, that’s a great trick if you can work it – I overrode the one method that cared the most about @@config:

@@config_mutex = Mutex.new def configuration_mutex @@config_mutex end alias :raw_http_connect :http_connect def http_connect(*args, &block) ops = (Hash === args.last) ? args.pop : {} result = nil self.configuration_mutex.synchronize do ops[:call_before].call if ops[:call_before] result = raw_http_connect *args, &block ops[:call_after].call if ops[:call_after] end result end # # ... # @@SEARCH_CONNECT_HOOKS = { :call_before => lambda { self.configure_apply(:search) }, :call_after => lambda { self.configure_apply(:default) }, } # # ... # # broken out of closure for mocking req = create_http_get_request(uri, params) response = http_connect(@@SEARCH_CONNECT_HOOKS) {|conn| req } return nil unless Net::HTTPOK === response model = bless_model(Twitter::Search.unmarshal(response.body))

It works like a charm. Please, everyone just line up to tell me how I could have done it better. And I don’t mean quicker or cheaper, I mean better. Believe me, I would not have sunk the time into this end-around approach, if not for the fact that:

I don’t want to maintain a local gem code modification (even though my impl is closely coupled to the gem impl already)
I intend to follow that practice, so every opportunity to pull and end-around is a Valuable Learning Experience.

So, now my local Twitter4R has search capability gracefully latched onto it (and implemented much in the flavor of the library itself). I have a mass of rspec examples to work off in the future.

Now, I haven’t spent a great amount of time testing the thread-safeness – no one in their right mind wants to do that – but my sundry Twitter::Client instances play nicely together in between searches and normal status operations.

And I had something useful to blog about!

NOTE: If your screen reader is reading this, please contact me at admin@cantremember.com ... because it shouldn't. FIXME: build this dynamically based upon the maximum content in any sub-Element of this Element. I will call this my "Safari Reader Counterweight". In some of my Posts, I have huge code excerpts, etc. Safari Reader, at least in iOS, will identify the 'main Element', the one it features, based upon its content length. Sometimes those code excerpts get identified as the 'main Element', and the Post is borked in Safari Reader mode. This is a counter-weight; it gives the <main> Element additional content so that it gets featured, algorithmically. Yes, it increases the payload of every page (@see FIXME above). But not by that much. Then again, this is a guess as to how much content any given Element could contain. If it's not enough, BOOM, Safari Reader looks like crap. So, here's a great article on how to enable Safari Reader on your site. It's mostly guesswork, but those guesses helped me debug this obtuse goddamn problem. Oh, and look, you can enter and exit Reader programmatically. JavaScript can fix anything. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload. I promise I will never cut-and-paste lines of text simply to add Element payload.