The “Test Drivel” …

What is the provenance of this need to torture, to test? A link between torture and experiment has been asserted ever since Francis Bacon; yet, what has allowed acts and idioms of testing to top out as an essential and widening interest, a nearly unavoidable drive?

Ronell, Avital. The Test Drive (p. 5). University of Illinois Press. Kindle Edition.

Over the last few years, evaluating candidates for development roles has migrated from a matter of interviews with the hiring manager and usually the most proficient technical person in the area, to being predicated initially on success in passing some sort of test or exercise. A well recited list of ways to create ‘top’ development teams, authored by a certain Joel, includes the necessity to have the candidate ‘write code’ during the interview itself.

That ‘Joel’ justifies his ‘joels of wisdom’ by pointing out that Microsoft teams routinely get a perfect score should be warning enough, given the vast range in quality of the products those teams develop. Were Joel’s ad-vice not simply a relatively short, easy to quote summary of popular ideas, it might be worth finding him and giving him a good kick in his own ‘crown joels’.

The idea of writing code during an interview, aside from the additional stress, generally means writing it outside one’s usual development environment(s) and thus losing the context those environments provide. This is particularly debilitating to the more creative type of developer who often works in numerous languages, with various libraries and frameworks, and as a result is more dependent on that context to ‘mode-switch’ into thinking in the appropriate terms.

As it happens, I tend to do well on tests. What does that say about me as a developer? Nothing, well, nothing other than that I tend to do well on tests. However I did better on those tests as a junior developer than I do now, though I’ve improved as a developer significantly in the 20+ years that have passed by. And this is not at all accidental.

In any case, the results of this trend have not been especially positive, and for good reasons. One of the major providers of such tests is IBM, which involves a certain irony since IBM doesn’t utilize such tests themselves. The type of questions asked though is directly related to IBM and other providers’ lack of belief in the relevance of such tests, since they generally give the work to write them to junior developers.

Naturally, developers just out of school with little industry experience tend to write questions much like those they had to answer in university, with the result that the tests favor those just out of school who have little actual industry development experience, which is usually not what the company requiring candidates take these tests is looking for.

Their existence has more to do with relieving HR and hiring managers of the responsibility to ask pertinent questions, the kind that could determine both the candidate’s ability and ‘fit’ for a given role. Of course, this results in less relevant interviews and poorer hiring outcomes. Compounding this is the notion that since the ‘test’ determines ability beyond the ability to do well on tests, the ‘technical’ interviewer is often at a lower level than the candidate in the specific technical area, and having a candidate interviewed by someone incapable of understanding appropriate answers is hardly a good means of determining a candidate’s real abilities.

None of the above, however, considers the reaction of the candidate to being subjected to such tests, and this is where bigger problems surface.

The top candidates, those the companies involved are particularly looking for, are liable to be busier when looking for work, simply because their CV attracts more responses. Of the two main types of tests and exercises demanded, the non-timed exercises are a bigger problem than the timed ones, since anyone busy with interviews and other aspects of looking for a job will put something likely to be time consuming off in favour of more urgent tasks.

In the case of the ‘timed’ tests, since they often resemble lines of code metrics, like LoC metrics (the worst metrics ever devised for measuring productivity) they tend to discriminate against top candidates, who often spend more time thinking about how to best solve a problem, and as a result require far less code to accomplish the solution.

Along with only having the choice of candidates ‘left’ after top candidates have been hired elsewhere, there is the further problem in of candidates’ own reactions, which tend to worsen as the number of tests increases.

Initially I thought it was something personal and probably not that common, similar to my unwillingness to interview twice with the same company (unless I was hired the first time, and left for reasons that had nothing to do with the company), but I’ve found in speaking with other developers that my reaction to taking such tests is not at all uncommon. And it isn’t good.

It takes a fair amount of effort to get sufficiently “up” for an interview when the extent of one’s knowledge of the role usually amounts to a generic job description, perhaps some (rather untrustworthy) information about the company from the recruiter, and what one may have heard about the company from other sources.

The negative experience that taking such tests produces often leads, at least in my case and the cases of a number of developers I regard as among the best in the field, to no longer wanting the job. The result is either a poor interview or simply refusing the interview altogether.

I have taken such tests as part of the process regarding four roles recently. In three of the four, after passing the test refused the interview. In the fourth I went to the interview without being able to contain the attitude of ‘now prove to me why I should want the job’ which taking the test generated — usually not a fruitful attitude in an interview situation (no matter how much HR types insist interviews are two-way, they become alarmed rather quickly if the candidate behaves as if they were).

It’s not a surprising result, though — it’s a very human response to being tested that one wants to turn around and test the tester; or, as the twinning of torturing/testing implies, to torture the torturer.

I’ve personally tortured a number of interviewers, particularly the poor sods who drew the short straw and had to do the technical interviews.

There is nothing as such new about the desire bound up in the test; yet the growing promiscuity of testing poses novel problems and complicates the itinerary of claims we make about the world and its contractions, the shards of immanence and transcendence that it still bears. Our contract with Yahweh, whether piously observed or abominated, involves the multiplication of test sites.

Shortly after completing his Critique of Judgment, Kant, in response to a public questionnaire, examined the problem of testing the faith of theology students. Can faith be tested or is it not the essence of faith to refuse the test — to go along, precisely on blind faith, without ground or grade? Or again, perhaps the Almighty Himself has proven time and again to be addicted to the exigencies of testing. If God can be said to have a taste for anything, then it may well be in the incontrovertible necessity of the test. No one is not tested by God, at least by the God of the Old Testament who showed a will to perpetual pursuit, perpetual rupture. Even the satanic beloved, who got away or was kicked out (depending on whether you are reading the satanic version of Goethe or God), became a subsidiary testing device for the paradisiacal admissions policy.

In German, Versuch unites test with temptation — a semantic merger of which Nietzsche makes good use. The devil is the visible mark of a permanent testing apparatus. It is one name for an operation that engages the frazzled subject in a radical way.

Ronell, Avital. The Test Drive (pp. 5–6). University of Illinois Press. Kindle Edition.

Aside from the desire to abdicate responsibility for negative results of hiring a given person, the growing distrust between corporate management and developers in fact seems to be the main driver of the trend. The test doesn’t torture as an unfortunate side effect, the torture and proving one’s willingness to undergo it is the point.

Management represents absentee capital, and technology stands in much the same relation to capital as capital did to labour in the 19th century. While information technology required significant capital early on, the more it has developed the less reliant on capital it has become. Since there’s no obvious reason for that trend to not continue, the distrust of technologists by management is perhaps even justified.

It should be obvious that this is hardly going to lead to hiring the top technologists, but it does lead to hiring people management believes it can trust, because at this point, that involves trusting them to not be too good,

Software is inherently disruptive, and disruptive to capital more than to any other single aspect of society. The irruption of better software has led to an overall decapitalization in every industry (wherever a company has, via software advances, taken the leading position in a market, their increase in capitalization is far smaller than the total decrease of their competitors).

The better the developer, the more disruptive the software they develop is likely to be. But companies afraid of disruption are always the first to succumb to it.

Within software development, the “Test Drive” as Avital Ronell termed it, is at best a drive to a lowest common denominator, while that denominator itself appears to be a rapidly lowering bar. I’ve seen a significant number of listings for the role of a senior developer that required less than five years total experience; in other words, less than it takes to graduate from being an apprentice electrician to a journeyman.

Of course, given the average developer’s career spans ~seven years, the number of software engineers with significantly more than five years experience is itself problematically low, since apprentices are learning from engineers who haven’t sufficient experience to be considered anything other than apprentices themselves.

This also goes some way to understanding the prevalence of developers who treat ideas such as asynchronous I/O, futures and being able to maintain a network connection for longer than one request, never mind actually maintaining state, as exciting new developments (or in the latter case, a potentially reachable goal), when they were intrinsic to the first language I learned in a serious way at university ~30 years ago.

Whatever the eventual results of the overall decapitalization of industry, and the ensuing flight of funds into the refeudalized real estate market, from the perspective of an individual company HR, hiring managers and others involved in the hiring process are simply being provided with sufficient rope to hang the entire works.