Fast, isolated/independent, repeatable, self-validating and thorough/timely. Those are the F.I.R.S.T princliples of proper software testing. But, those criterias might not be achievable because of the nature of the tested code. What could we do to achieve those principles? Fool the tests.
Let’s say that we’re developing what I was in charge of, an API wrapper. Its purpose is to communicate with external API through HTTP requests, parse the response, and present in the programming language’s native data structure. Now, we’re presented with some obstacle in testing our code:
- Speed. Doing HTTP requests is often the opposite of fast, thus violating the first F.I.R.S.T principleshehe. And it will only get worse if the API endpoints suddenly become unreachable on testing.
- Dependency. Creating an API call means that the test depends on external factor, such as the availability of the API host, the network connection, network speed, etc.
- Consistency. What if the data returned by the API changed? That could break assertion statements on the tests.
Introducing: VCR and Webmock
Those violations boils down to one source of problem: HTTP requests. If we could sever the dependency to HTTP requests, our test would have no problem. Fortunately, there is a solution to this problem: mocking. No, we’re not insulting the code, we’re going to intercept the HTTP requests and mock (as in fake) the response with our own predetermined data. It’s just like fooling your own test, but in a good way. Kinda.
In ruby, HTTP request mocking can be achieved with VCR and webmock. Webmock is responsible for intercepting HTTP requests and redirect it somewhere else, while VCR will receive those redirected requests and returns proper HTTP responses using predetermined data. Those two modules can be used in conjunction with rspec, the testing suite we use.
To use those two modules, all we need to do is add these lines to the
require 'webmock/rspec' require 'vcr' VCR.configure do |c| c.cassette_library_dir = 'spec/fixtures/rbcaw_cassettes' c.hook_into :webmock end
Capturing Real HTTP Traffic
Additional problem comes if we mock the data: how do we determine the predetermined datahehe. Inputing the data by hand would be troublesome if the tested endpoints are numerous. VCR is helpful in this case as it allows us to record real HTTP request, save it in a file (called cassettes), and replay it later (hence the name VRC).
On the sample code earlier, the cassettes directory can be configured by changing the
c.cassette_library_dir value. The directory is relative to the rails’ root directory.
As a warning, the captured traffics will be stored as YAML file as plaintext. This might be troublesome if the network traffics are sensitive information. But, we haven’t found any solution that is more secure and compatible with our current git setup.
Playing the Cassettes
This is a sample of our test case:
context 'with proper connection to the host' do before do VCR.insert_cassette 'sprint_info', :record => :new_episodes, :match_requests_on => [:method, :uri, :body] @task_sprint_info = conduit.sprint.info(test_task.id) @project_sprint_info = conduit.sprint.get_project_start_end_date(test_project.phid) end it 'should retrieve the task\'s sprint info properly' do # This function do HTTP request, which is mocked by VCR+webmock expect(@task_sprint_info).to eq(test_task.sprint_info) end it 'should retrieve the project\'s sprint info properly' do # This function do HTTP request, which is mocked by VCR+webmock expect(@project_sprint_info).to eq(test_project.sprint_info) end after do VCR.eject_cassette end end
First, we’ll need to tell the RSpec context to utilize VCR (and thus, webmock). This can be achieved by using the
before block and using the
VCR.insert_cassette statement. The statement’s first parameter would be the cassettes’s name (without the .yml extension). The
:record => :new_episodes parameter tells VCR to record new requests that weren’t recorded on the cassette. The
:match_requests_on parameter tells VCR to match requests based on the
:method, :uri, :body of the original HTTP request. The match_on’s argument are important as POST API calls with same method and uri might be different if their parameters — which are stored on the body — are not considered for matching.
The HTTP requests done after the
before block will not use real network connection (unless the HTTP request is not recorded beforehand) but will be redirected to VCR and it will return the pre-recorded data. Because the requests don’t use real HTTP connection, it also will be much faster. Neat stuff.
Lastly we’ll need to eject the cassette, as other test context might use different cassettes. This can be done with
Mocking a Network Failure
Another nice feature of webmock is its ability to simulate network timeout. This will be useful to test our code ability to handle network-related errors. This can be done with this block:
before do stub_request(:any, /www.localhost.com/).to_timeout end
stub_request().to_timeout will make request made to its specified URL returns a
Timeout::Error. The URL is expressed in regular expression, hence the lack of quotes and the use of slashes.
Fooling Your Tests is Not That Bad After All
With this method we could easily test our API wrapper’s code while still abiding the F.I.R.S.T principle, which is nice. But, we might need to regularly update the stored cassettes (which should be added to git, if not then our whole effort will be for naught), which can be done simply by deleting the cassette files and re-run the test, which will repopulate the cassettes with latest API call responses.