Lately, I've been reading a lot of articles on unit testing. When I say 'unit testing', I'm referring to the emerging definition of unit testing as a method of testing a specific class in isolation of other dependencies. That is, a true 'unit test' should not use a database connection, a network connection to the internet or even access the file system.
As I've been developing ScoreboardFeed, I've been saving HTML pages locally to use in testing. By saving a variety of scoreboard pages from the same source, I can test all sorts of different conditions. I can ensure that upcoming games, current games and past games are all handled properly. I can test a scoreboard page that has postponed or delayed games (I saw a lot of those during Hurricane Katrina). This makes the application better because the requirements can be defined by the test data. Any time I make a change to the code, I can run some automated tests to ensure that these conditions still function correctly.
However, these test pages make a true unit test impossible because the tests must access the filesystem in order to get the HTML pages. As I get more familiar with this emerging definition, I'm also getting better at writing tests. My personal line in the sand for a unit test definition is whether I can move my code to another machine and still run my test without any configuration. That is, no database connections are required, not network access is required and no filesystem traversing is required.
All of the tests that I had previously written as unit tests have now been reclassified as integration tests. They are testing the integration between my parsing engine and my scraping engine. If I need to test my parsing engine in isolation, then I create a mock object to do that.
Now that leaves me with one question: how do I write a unit test for my scraping engine' After all, the sole reason for this class to exist is to get data from a remote source. If I can't access a remote source (because that violates the purity of the unit test rule), then how do I write a unit test for this class' Am I stuck with just integration tests here'