Karen Nicole Johnson

I am making this post to share my thoughts on testing certifications & standards and the recent activity that took place at CAST 2014.Since the activities that took place at the CAST 2014 conference, I’ve had several people ask questions or make comments via Twitter. As I do not see Twitter as an easy place to clarify deep thinking, I decided to write this blog post to state my thoughts.

This post has multiple parts:

Part 1: My thoughts on the Professional Tester’s Manifesto
Part 2: Questions Asked
Part 3: CAST 2014, The back-story to current activities
Part 4: The Professional Tester’s Manifesto

Part 1: My thoughts on the Manifesto

First, I want to review the statements made within the manifesto, statements from the manifesto are in bold – my thoughts follow.

That standards compliance is no substitute for knowledge and skills, and that possessing a certificate demonstrates neither.

That organizations who use certification as a surrogate for rigorous selection processes place the quality of their testing at risk.

For several years when I was working as a test manager, I interviewed and hired numerous people. Once during an interview, a candidate pulled out a certificate and held it up for me, telling me he should be hired instantly due to his certification. I asked that he put the certificate away and talk with me. After discussion, it was my assessment that the candidate had little working understanding of how to test a product. Perhaps he had memorized material or had someone else take the exam for him – whatever had been the case; there was no evidence that the candidate would be able to perform well. As this event took place some years ago, I will refrain from trying to recall more specifics.
Given this same scenario, if I did not have a testing background but needed to hire someone, perhaps I would have been convinced based on the certificate that the candidate was equipped for the job.

If a company is hiring testers and the person or persons interviewing do not understand testing (which is often the case with HR), I do not believe that hiring person is qualified to make a decision – or even qualified to establish a pool of candidates for others to interview. This is where certification is dangerous – at first glance, it sounds good, it seems like it should provide some assurance that a person is qualified. But as I have found, having a certificate does not provide evidence of a person’s knowledge or skill.

In my work, my own personal experience as an employee and consultant, I have not worked at a company requiring certification. I have heard from several colleagues in Europe that companies do require certification and I find this concerning. Further I have heard colleagues say they have paid the money, taken the exams and become certified just so they can find work or keep employment even though they do not believe the information gathered from certification has helped them.

Given that set of circumstances (people believing they must be certified to find or keep their jobs), I believe it is important for people within the testing community who do not believe in certification to say something. Is the Professional Tester’s Manifesto perfectly worded? No. Should a statement be added that says something such as: I am certified but I do not believe in certification. I am certified but have not found the certification to help me with my work. Yes, I wish we had included that statement but with 60 signatures collected, I cannot change the wording without an amendment process and recollect signatures. At this time, I’m not going to do that.

That organizations who make money from creating or promoting standards and certifications are biased in their thinking by the potential financial rewards of convincing organizations that only certified testers are professional testers. Those organizations may include those who sell training, consulting or other related services.

I find it disturbing that certification companies are making money, while lone individuals feel they cannot fight against the increasing number of companies requiring certification. I have concerns that the same scenario could eventually take place in the US.

As the certification companies have significant influence and their own marketing dollars to spend promoting their view, I believe that testers who do not believe in certification lack this same organized voice to state their opinion – this further enables the certification companies to their benefit. And while people in the testing community agree (against certification) but have not pooled together to make a collective voice against certification, the situation continues. It is my hope that this statement signed (hopefully) by a large number of testers, will give us this voice.

That testing benefits from diversity and not homogeneity: that testing is not a profession that can be standardized but instead needs to remain an intellectual professional activity.

The knowledge needed for a tester varies based on what type of software a tester is working with and the type of testing being executed.

a) For example, a tester working with ecommerce software and focused on performance testing may need to test spike conditions as well as simulating other load conditions. In this case, a tester would likely do best to have knowledge of math and statistics.

b) For example, a tester working with business intelligence software and focused on pulling accurate data into a data warehouse may need to test data cleansing routines as well as simulating data loading from a variety of sources. In this case, a tester would likely do best to have knowledge of SQL and stored procedures.

c) For example, a tester working with a medical device and focused on science lab simulation may need to test data transmissions and multi-user scenarios. In this case, a tester would likely do best to have knowledge of FDA regulations, GLP – good lab practices, GDP – good documentation practices.

All three of these examples come from my own personal experiences as a professional tester in the past twenty plus years testing.

While it is true that the context around these examples could not possibly be covered in a more generic testing certification, there is no evidence that the base knowledge obtained by the certifications is sufficient base knowledge. There is also no evidence that the materials within the certification are developed or continue to evolve in a timely enough manner to address the every changing software/hardware/technology world in which software testers work. So while the certifications could not possibly address the vast array of software contexts, there is no evidence that the base knowledge acquired is “the right base” or a “sufficient base” or that the “base is updated in a timely fashion.”

In each of these examples, a tester would do best to have a thorough understanding of the software solution and its intended use. As you can tell from the three examples, each context is entirely different and therefore the knowledge background varies accordingly. With certification, little information to none of this material is addressed and yet these are three real examples. In these cases, certification would not help. I base this last statement on the fact that although I do not hold a certification, I have read and reviewed the body of knowledge for one certificate program and purchased and reviewed the book of another certification – and in each of these three working experiences, did not find anything from the body of knowledge that would have assisted me in this work. In all three examples, I expanded my own knowledge from an assortment of books, regulations and other materials.

I am not saying the information from the certifications is detrimental but I am saying I have not found the information from the certifications to be instrumental in getting the job done.

That choosing not to be certified does not mean I do not take my profession seriously. It is because I take my profession seriously that I choose not to be certified.

I am consciously not certified. I believe obtaining a certification would further enable certification companies a financial benefit while providing me with nothing further than the ability to say, I am certified.

Each year, I continue to invest in my education. I continue to learn technology, skills – at times technical and at times “soft” skills. I openly share the books that have shaped my education on my account on a free website known as: www.librarything.com I write articles, I tweet and I do all online activities by my own name, making it clear what my background, knowledge and experiences are. I take my profession seriously. I do not believe in certification.

Part 2: Questions Asked

Some questions I have been asked:

What about having a standard set of definitions and terms?

Having a standard definition and understanding of terms such as: test case, test plan, and test strategy is one of the most common arguments in favor of certification. In each work environment I have been in, including environments where many people where certified there has never been a singular understanding of terms. In each environment, on each team, people have needed to come together to understand what each other means and come to some type of informal understanding of the terms based on the context. So clearly a standard set of definitions have not been achieved.

Do you speak at conferences hosted by certification organizations?

I do speak at numerous conferences each year. In some cases, I do not know where all the funds come from or where all the funds are spent. The one time I was asked by a certification company to speak at a conference, I said I could only speak if I was given an opportunity to have a panel discussion on certification. I also was clear in stating that I am opposed to certification but would welcome a healthy debate on the topic. The response was they would select someone else to speak. I did not attend.

Why are you speaking out against certifications now?

I am a long-standing member in the software testing community. Although I typically avoid heated public debates and prefer instead to more quietly and peacefully pursue my career and business, I believe the issue over certifications is a growing issue. I recognize that silence can imply compliance and I am not willing to do so any longer. I believe it is time for me to be a vocal opponent as many of my colleagues have been already.

What do you hope to achieve with the “Professional Tester’s Manifesto?”

I hope that we can find agreement in the community that certification is not effective. I hope that while certification companies and certified testers have one another to rely upon that we as a group of people – otherwise not affiliated and connected can discover that we have many like-minded people who will band together. If we can collect a large enough number of signatures, we can make a unified statement and hopefully make a change. After all, it was the signing of the Agile Manifesto that drew and continues to draw a tremendous change in our industry. Change is possible. We may not agree on many topics but may the signing of this statement, align us on this one topic.

Why don’t you believe the current certifications are beneficial?

There is an absence of peer validation of the “certification” standards and procedures by recognized experts outside the “certification” organizations. There is no evidence that these organizations’ “certifications” provide a useful or meaningful baseline of competence or effectiveness for companies hiring testers. The fact that an organization creates a concept of “certification” and then proclaims that those who complete materials and testing that have not been peer validated are “certified” is an exercise in circular logic.

 
Part 3: CAST 2014, The back-story to current activities –

At CAST 2014, James Christie (@james_christie ) gave a presentation on standards and certifications. (AST recorded his presentation; the video will be available sometime this week – meaning the week starting Monday August 18th) through AST (@AST_News). To summarize, Jim’s presentation highlighted disadvantages of both (certifications and standards). Although I cannot claim to know what people were thinking in the audience, it appeared as though many people in the audience were in agreement with the presentation. At the end of Jim’s presentation, several questions were raised – the most notable question being: what can we do? (I don’t recall who asked this question.)

Following the format of the conference I held up my “pink card” out to speak – the pink card signifies to the facilitator that I as an audience member have something urgent to say. To summarize, I suggested that since it was the start of the conference (at this point it was the end of the first session of the conference) and many of us seemed to be in agreement on a position against certifications and standards) that perhaps we could “do something” that would make an impact. It seemed to me then (and now) that many people have been in agreement on this topic for years. In my view, there was “energy” and support in the room to do something.

Iain McCowatt (@imccowatt ), Fiona Charles ( @FionaCCharles) along with Jim Christie and I talked at the end of Jim’s session. As each of us had other obligations for the remainder of the day, we agreed to think about what took place and meet again at the end of the day.

At the end of the day, I had drafted a statement – which statement is now being referred to as the “Professional Tester’s Manifesto.” The statement was drafted and circulated at CAST2014. Sixty signatures were collected at the conference. Numerous people have asked to sign this statement since the conference. A website is being built to provide that opportunity. At this time, the site is not up. The text of the statement is included at the end of this post for reference.

In addition, Iain wrote a petition that is now referred to as “Stop 29119.” See the website: ipt.io/tiul The petition is a call to action, the call being made to the ISO president.

The two statements are separate. The statements serve a different purpose.

 
Part 4: The Professional Tester’s Manifesto

The exact words from the statement follow.

A website will be up (hopefully within the week) for people to review and sign.

The Professional Tester’s Manifesto

I, as a professional software tester, believe:

That standards compliance is no substitute for knowledge and skills, and that possessing a certificate demonstrates neither.

That companies have been convinced that only certified testers should be hired.

That organizations who use certification as a surrogate for rigorous selection processes place the quality of their testing at risk.

That organizations who make money from creating or promoting standards and certifications are biased in their thinking by the potential financial rewards of convincing organizations that only certified testers are professional testers. Those organizations may include those who sell training, consulting or other related services.

That testing benefits from diversity and not homogeneity: that testing is not a profession that can be standardized but instead needs to remain an intellectual professional activity.

That choosing not to be certified does not mean I do not take my profession seriously. It is because I take my profession seriously that I choose not to be certified.