Keeping a Work Journal

In 2015, I’m making a goal to keep a work journal. This post is about the why, what, when and how of my work journal.

I wanted to make this post in January but I’m a bit late on that goal. As you read on, you’ll see why that timing seemed particularly smart. Oh well, it’s still early in the year ~

Why a Work Journal?

I want to keep a work journal for many reasons. Starting a new position and starting a new year seemed like a great time to start a work journal. I’ve kept many work notebooks and still keep notes at work, but my work journal has a different purpose and is separate from project notes.

My primary reasons for keeping a work journal are:

  • To establish my focus for the day.
  • To prevent the loss of ideas.
  • To reflect on positive and negative situations and to think about what I may want to do about these situations.
  • To track info (see the section on “what”) and to have that information over a span of time – hence starting at the start of the year seemed a great time to begin. (I did start my work journal in December but I haven’t gotten to blogging about it until now.) Note that if you want to start a work journal, it is still Q1 – still early in the year and still worth trying. Also the “annual” aspect of journalling is fairly irrelevant, starting anytime is worthwhile.

Need more “why keep a work journal” – see these articles.

Inc. magazine: “Why You Should Keep a Work Journal”

Harvard Business Review: “Four Reasons to Keep a Work Diary”

Forbes magazine: “6 Ways Keeping a Journal Can Help Your Career”

What to record in work journal


I record ideas even if I cannot execute on the idea for a while. I imagine some people thinking – what’s the point if I can’t execute but I don’t like to think that way – instead I prefer to think – you never know – sometimes I can fit in a simpler version of an idea and instead of waiting for the perfect alignment of opportunity, I try to see if I can somehow get an idea moving in a simpler form.

(In software development terms – think the MVP (minimal viable product) of an idea.)

I know from past note-taking experience that going back over my notes can be inspiring on days I might need inspiration.


I record snippets of conversations that I want to recall. Perhaps this harkens back to my journalism or technical writing background but the way someone phrases something can be pivotal and if I record a snippet of a conversation – the entirety of the conversation can come back to my memory. Recording concepts from a conversation is much more likely to end up in a project notebook than my journal – this probably sounds confusing but the split or division of what belongs where is not confusing to me – it’s probably a matter of coming up with what makes sense for you.


I take pictures of whiteboards, screenshots, pictures and such. If we talk and you write on a whiteboard, I’m likely to ask if I can take a picture. If your drawing looks like doodles, it will still help me recall the conversation and those doodles provide surprisingly good recall and value to me.

(As a side note see this Ted Talk on the benefits of doodling: )

I’m a big fan of mindmapping although I don’t find mindmaps to suit all purposes – there is a place for checklists, a place for notes and a place for mindmaps. My preferred tool for mindmapping is SimpleMind+. It’s free and easy to use, see:


Nearly everyday I have a list of people I want to contact. I might make a note of why I want to contact each person if I think I will forget. Often in the morning, I think of different people I want to contact that day so many days start with a list of names. For some reason, I’ve found if I write names in the morning, I’m much more likely to actually contact those people that day.


I record connections, which might be one of the more elusive items to explain. As I meet with one person to another, a concept or an idea will come to mind. Often I can see the connection between one person and another – connections that might not be obvious – in this case, I connect people when I can. Other times I have an idea and I can see a connection to a person – as in – this person can help with this idea or this person knows something that could be helpful. Noting connections and finding synchronicity in people and ideas is one of the most valuable bits I recognize and record.

Follow-up Items

I record items I want to follow up on which can range from an email, a phone call, a meeting that needs to happen – all sorts of possible things. Some days the list I write at the end of the day is longer than the list at the start of the day – frustrating to me as it always seems there is more to do than hours in the day, everyday.


Everyday I start my work journal with the top three goals I have for the day. This is a baked in habit, to think in terms of top three. It is tempting to have more but when I list more, I can get sidetracked or overloaded and end up achieving nothing.

A friend and colleague recommended the book The Progressive Principle (back in 2012.) The Progress Principle has a short PDF overview, see: “Tips from The Progress Principle” Notice the book tips encourage keeping a work journal. When this book was first referred to me, I used a half an index each day (for about a year), in fact, I often tucked that index card into pocket and would refer to it later in the day to see if I’d made my top three intentions or goals for the day, I liked the simplicity and the physical card as well as the personal push it encouraged to make my goals.

And although I have not been recording these items – it is possible I will begin to:

Appointments. I want to track and tag where I spend my time. Currently I use my calendar but that might be too fine of a grain of information – although color coding meetings and appointments helps me get a quick visual on where my time is going.

Long View Goals. I want my longer view goals in front of me continually and while I do track my goals of the day – those mini-goals feed into bigger picture goals – or at least I hope they do!

When to write in a work journal

1. At the moment because the next moment happens so darn quickly and then the thought is gone.

2. Alternately a time to write is when I find a quiet moment n between activities or meetings.

3. The start of the day – early morning hours hold such promise for the day. Writing in the morning drives my focus for the day.

4. The end of the day – end of day seems to create another flurry of thinking as the reality of time sets in and I can better see what will and will not happen – yet – I don’t want to lose ideas and follow-ups – so typically late afternoon, I find I’m jotting lots of items down.

5. At the end of the week, I’m trying to build a habit to go back through my daily journal to catch anything that was missed (I’m still baking this habit in.) It is also clearly a time to capture items from the week that might have been lost or missed.

How – the format of a work journal

Format matters. I want a paper journal but it is just not feasible or maintainable. I tried to keep a paper journal, I tried for the month of December (2014) but I encountered too many issues. And my back refuses to carry more than needed – besides with all the devices in my bag, why carry a notebook too?

In January, I downloaded the Day One app and have been experimenting this past month. I’ve encountered some issues but reached out to their support team and have resolved what I needed. (Yahoo to the support group for being responsive and friendly.) I’ve also been experimenting with Evernote and that appears to be the winner.

The convenience of a mobile app that works on both my iPhone and Android device means I’m not waiting to get back to a paper journal in a desk drawer, it means I can record super short notes anywhere, anytime – and I do. Convenience means not losing ideas or the moment. Although I have two more computers to install and sync with – there is no location in my life that I don’t have at least one of the following: an iPhone, Samsung/Note4, iPad or MacBook.

I think it’s clear to see the benefits. I think a work journal like any journal is a private matter so feeling secure no one else will read your journal means you could use your journal to vent and blow off steam – I’ve done that and it’s sure better than blowing off steam to a person.

Think about keeping a work journal, try it – it’s a good time of year to start a new habit – especially a habit that can boost productivity and a place to record all those little bits of things you don’t want to lose track of. The how (format) and the when (time of day) matter less than why. And based on the why – what you’re trying to achieve will certainly influence what you choose to record.

Posted in productivity, time management | Comments Off on Keeping a Work Journal

My Thoughts on Testing Certifications

I am making this post to share my thoughts on testing certifications & standards and the recent activity that took place at CAST 2014.Since the activities that took place at the CAST 2014 conference, I’ve had several people ask questions or make comments via Twitter. As I do not see Twitter as an easy place to clarify deep thinking, I decided to write this blog post to state my thoughts.

This post has multiple parts:

Part 1: My thoughts on the Professional Tester’s Manifesto
Part 2: Questions Asked
Part 3: CAST 2014, The back-story to current activities
Part 4: The Professional Tester’s Manifesto

Part 1: My thoughts on the Manifesto

First, I want to review the statements made within the manifesto, statements from the manifesto are in bold – my thoughts follow.

That standards compliance is no substitute for knowledge and skills, and that possessing a certificate demonstrates neither.

That companies have been convinced that only certified testers should be hired.

That organizations who use certification as a surrogate for rigorous selection processes place the quality of their testing at risk.

For several years when I was working as a test manager, I interviewed and hired numerous people. Once during an interview, a candidate pulled out a certificate and held it up for me, telling me he should be hired instantly due to his certification. I asked that he put the certificate away and talk with me. After discussion, it was my assessment that the candidate had little working understanding of how to test a product. Perhaps he had memorized material or had someone else take the exam for him – whatever had been the case; there was no evidence that the candidate would be able to perform well. As this event took place some years ago, I will refrain from trying to recall more specifics.
Given this same scenario, if I did not have a testing background but needed to hire someone, perhaps I would have been convinced based on the certificate that the candidate was equipped for the job.

If a company is hiring testers and the person or persons interviewing do not understand testing (which is often the case with HR), I do not believe that hiring person is qualified to make a decision – or even qualified to establish a pool of candidates for others to interview. This is where certification is dangerous – at first glance, it sounds good, it seems like it should provide some assurance that a person is qualified. But as I have found, having a certificate does not provide evidence of a person’s knowledge or skill.

In my work, my own personal experience as an employee and consultant, I have not worked at a company requiring certification. I have heard from several colleagues in Europe that companies do require certification and I find this concerning. Further I have heard colleagues say they have paid the money, taken the exams and become certified just so they can find work or keep employment even though they do not believe the information gathered from certification has helped them.

Given that set of circumstances (people believing they must be certified to find or keep their jobs), I believe it is important for people within the testing community who do not believe in certification to say something. Is the Professional Tester’s Manifesto perfectly worded? No. Should a statement be added that says something such as: I am certified but I do not believe in certification. I am certified but have not found the certification to help me with my work. Yes, I wish we had included that statement but with 60 signatures collected, I cannot change the wording without an amendment process and recollect signatures. At this time, I’m not going to do that.

That organizations who make money from creating or promoting standards and certifications are biased in their thinking by the potential financial rewards of convincing organizations that only certified testers are professional testers. Those organizations may include those who sell training, consulting or other related services.

I find it disturbing that certification companies are making money, while lone individuals feel they cannot fight against the increasing number of companies requiring certification. I have concerns that the same scenario could eventually take place in the US.

As the certification companies have significant influence and their own marketing dollars to spend promoting their view, I believe that testers who do not believe in certification lack this same organized voice to state their opinion – this further enables the certification companies to their benefit. And while people in the testing community agree (against certification) but have not pooled together to make a collective voice against certification, the situation continues. It is my hope that this statement signed (hopefully) by a large number of testers, will give us this voice.

That testing benefits from diversity and not homogeneity: that testing is not a profession that can be standardized but instead needs to remain an intellectual professional activity.

The knowledge needed for a tester varies based on what type of software a tester is working with and the type of testing being executed.

a) For example, a tester working with ecommerce software and focused on performance testing may need to test spike conditions as well as simulating other load conditions. In this case, a tester would likely do best to have knowledge of math and statistics.

b) For example, a tester working with business intelligence software and focused on pulling accurate data into a data warehouse may need to test data cleansing routines as well as simulating data loading from a variety of sources. In this case, a tester would likely do best to have knowledge of SQL and stored procedures.

c) For example, a tester working with a medical device and focused on science lab simulation may need to test data transmissions and multi-user scenarios. In this case, a tester would likely do best to have knowledge of FDA regulations, GLP – good lab practices, GDP – good documentation practices.

All three of these examples come from my own personal experiences as a professional tester in the past twenty plus years testing.

While it is true that the context around these examples could not possibly be covered in a more generic testing certification, there is no evidence that the base knowledge obtained by the certifications is sufficient base knowledge. There is also no evidence that the materials within the certification are developed or continue to evolve in a timely enough manner to address the every changing software/hardware/technology world in which software testers work. So while the certifications could not possibly address the vast array of software contexts, there is no evidence that the base knowledge acquired is “the right base” or a “sufficient base” or that the “base is updated in a timely fashion.”

In each of these examples, a tester would do best to have a thorough understanding of the software solution and its intended use. As you can tell from the three examples, each context is entirely different and therefore the knowledge background varies accordingly. With certification, little information to none of this material is addressed and yet these are three real examples. In these cases, certification would not help. I base this last statement on the fact that although I do not hold a certification, I have read and reviewed the body of knowledge for one certificate program and purchased and reviewed the book of another certification – and in each of these three working experiences, did not find anything from the body of knowledge that would have assisted me in this work. In all three examples, I expanded my own knowledge from an assortment of books, regulations and other materials.

I am not saying the information from the certifications is detrimental but I am saying I have not found the information from the certifications to be instrumental in getting the job done.

That choosing not to be certified does not mean I do not take my profession seriously. It is because I take my profession seriously that I choose not to be certified.

I am consciously not certified. I believe obtaining a certification would further enable certification companies a financial benefit while providing me with nothing further than the ability to say, I am certified.

Each year, I continue to invest in my education. I continue to learn technology, skills – at times technical and at times “soft” skills. I openly share the books that have shaped my education on my account on a free website known as: I write articles, I tweet and I do all online activities by my own name, making it clear what my background, knowledge and experiences are. I take my profession seriously. I do not believe in certification.

Part 2: Questions Asked

Some questions I have been asked:

What about having a standard set of definitions and terms?

Having a standard definition and understanding of terms such as: test case, test plan, and test strategy is one of the most common arguments in favor of certification. In each work environment I have been in, including environments where many people where certified there has never been a singular understanding of terms. In each environment, on each team, people have needed to come together to understand what each other means and come to some type of informal understanding of the terms based on the context. So clearly a standard set of definitions have not been achieved.

Do you speak at conferences hosted by certification organizations?

I do speak at numerous conferences each year. In some cases, I do not know where all the funds come from or where all the funds are spent. The one time I was asked by a certification company to speak at a conference, I said I could only speak if I was given an opportunity to have a panel discussion on certification. I also was clear in stating that I am opposed to certification but would welcome a healthy debate on the topic. The response was they would select someone else to speak. I did not attend.

Why are you speaking out against certifications now?

I am a long-standing member in the software testing community. Although I typically avoid heated public debates and prefer instead to more quietly and peacefully pursue my career and business, I believe the issue over certifications is a growing issue. I recognize that silence can imply compliance and I am not willing to do so any longer. I believe it is time for me to be a vocal opponent as many of my colleagues have been already.

What do you hope to achieve with the “Professional Tester’s Manifesto?”

I hope that we can find agreement in the community that certification is not effective. I hope that while certification companies and certified testers have one another to rely upon that we as a group of people – otherwise not affiliated and connected can discover that we have many like-minded people who will band together. If we can collect a large enough number of signatures, we can make a unified statement and hopefully make a change. After all, it was the signing of the Agile Manifesto that drew and continues to draw a tremendous change in our industry. Change is possible. We may not agree on many topics but may the signing of this statement, align us on this one topic.

Why don’t you believe the current certifications are beneficial?

There is an absence of peer validation of the “certification” standards and procedures by recognized experts outside the “certification” organizations. There is no evidence that these organizations’ “certifications” provide a useful or meaningful baseline of competence or effectiveness for companies hiring testers. The fact that an organization creates a concept of “certification” and then proclaims that those who complete materials and testing that have not been peer validated are “certified” is an exercise in circular logic.

Part 3: CAST 2014, The back-story to current activities –

At CAST 2014, James Christie (@james_christie ) gave a presentation on standards and certifications. (AST recorded his presentation; the video will be available sometime this week – meaning the week starting Monday August 18th) through AST (@AST_News). To summarize, Jim’s presentation highlighted disadvantages of both (certifications and standards). Although I cannot claim to know what people were thinking in the audience, it appeared as though many people in the audience were in agreement with the presentation. At the end of Jim’s presentation, several questions were raised – the most notable question being: what can we do? (I don’t recall who asked this question.)

Following the format of the conference I held up my “pink card” out to speak – the pink card signifies to the facilitator that I as an audience member have something urgent to say. To summarize, I suggested that since it was the start of the conference (at this point it was the end of the first session of the conference) and many of us seemed to be in agreement on a position against certifications and standards) that perhaps we could “do something” that would make an impact. It seemed to me then (and now) that many people have been in agreement on this topic for years. In my view, there was “energy” and support in the room to do something.

Iain McCowatt (@imccowatt ), Fiona Charles ( @FionaCCharles) along with Jim Christie and I talked at the end of Jim’s session. As each of us had other obligations for the remainder of the day, we agreed to think about what took place and meet again at the end of the day.

At the end of the day, I had drafted a statement – which statement is now being referred to as the “Professional Tester’s Manifesto.” The statement was drafted and circulated at CAST2014. Sixty signatures were collected at the conference. Numerous people have asked to sign this statement since the conference. A website is being built to provide that opportunity. At this time, the site is not up. The text of the statement is included at the end of this post for reference.

In addition, Iain wrote a petition that is now referred to as “Stop 29119.” See the website: The petition is a call to action, the call being made to the ISO president.

The two statements are separate. The statements serve a different purpose.

Part 4: The Professional Tester’s Manifesto

The exact words from the statement follow.

A website will be up (hopefully within the week) for people to review and sign.

The Professional Tester’s Manifesto

I, as a professional software tester, believe:

That standards compliance is no substitute for knowledge and skills, and that possessing a certificate demonstrates neither.

That companies have been convinced that only certified testers should be hired.

That organizations who use certification as a surrogate for rigorous selection processes place the quality of their testing at risk.

That organizations who make money from creating or promoting standards and certifications are biased in their thinking by the potential financial rewards of convincing organizations that only certified testers are professional testers. Those organizations may include those who sell training, consulting or other related services.

That testing benefits from diversity and not homogeneity: that testing is not a profession that can be standardized but instead needs to remain an intellectual professional activity.

That choosing not to be certified does not mean I do not take my profession seriously. It is because I take my profession seriously that I choose not to be certified.

Posted in certification | 10 Comments

Applying the SFDPOT heuristic to mobile testing, adding “I”

After years of SFDPOT being created by James Bach and used by many testers, James has amended SFDPOT to include “I” for interface testing. A couple of weeks back someone contacted me via Twitter and asked me about “I” for mobile testing. Here’s my response:

Interface. Test the ways we interact with it.
Does the app work well on an array of devices such as phones and tablets?
Does the app work well when used with “one hand?”
Does the app work well when used with “one eye?”
Is the mobile version consist or complementary to the desktop version?
Is the app consistent with UI current constructs in the marketplace?
Does the mobile version address what makes sense when a user is on the go?

On another note, I’ve long thought that “E” for error testing should be added to the SFDPOT mnemonic. Here’s how I would address error testing.

Errors. Test the coded-for known error states and around those error conditions.
Does the app continue gracefully after an error?
Is the message helpful, instructive?
If a user provokes multiple errors in succession, does the app continue?
Can I get to an error state that forces the app to shut down?
What happens when I restart the crashed app – is previous data or state saved?
Is there a method to send an error report?

I think this question about adding “I” was a good question because it raises the awareness of not testing just to the suggestions of a heuristics but continuing on, adding our own thoughts, considering the context and while using a heuristics – thinking beyond a heuristic as needed.

Posted in heuristics, mobile, usability | Comments Off on Applying the SFDPOT heuristic to mobile testing, adding “I”

Discipline & Software Testing

At the STP Fall 2012 conference in Miami, I gave a keynote on “The Discipline Aspect of Software Testing.”

I created tweets for my presentation in advance (and had those tweets posting while I was presenting) so I included those tweets as part of my slides. Discipline & Software Testing slide deck

My presentation kicked off with a video showing the marshmallow test, a well-known test – the video speaks for itself. You can find the video on YouTube here.


Posted in conferences, time management | Comments Off on Discipline & Software Testing

BI + Agile + ET in a mind map of charter possibilities

Data warehouse/business intelligence (BI) projects are different. The primary reason BI testing is so different (from say web application testing) is that most of the testing within a BI project is data reconciliation testing for which there is no visual application for a tester to work with. I say “most” testing because I suppose you could point to a dashboard or analytic report that represents output from the warehouse and claim that those tangibles are visuals but for the most part, the testing that takes place on a BI project is non-visual, there is no application to interact with, instead testing is generally focused on data accuracy and completeness. This type of testing requires a sense of investigation and being able to test and construct ideas about what to test and where to find issues without the aid of a of a tangible interactive application.

I construct mental images in my mind about the process the data flows through. I consider the originating sources of data and how that data has to traverse to get to the warehouse. On one BI project I worked on, I summarized a chunk of the testing efforts as “the story of a cube” and relayed the process of how the warehouse was built and tested to the business owners by way of the story. My storytelling in this case worked fairly well. I used a bit of humor, I drew some visuals and talked through the process. But that was then and this is now …

A client recently asked me about testing on a BI project where they are using Agile. They are also intrigued about the prospect of using Exploratory Testing but are unsure what Charters could be. So in the background of juggling several tasks and conference work, I’ve been puzzling over this question.

I’ve been building out a mind map to address this question and to help me think it through. Not long ago I referenced I was having a solo brainstorming session on Twitter about this topic. Minus one branch of the map that is client-sensitive, here’s my BI + Agile + ET Mind Map.

In a presentation that I’ve given a few times this past year, I’ve identified ways in which data changes or can be changed through “silent” non-visual processes, the point of the list is to remind myself and other testers of ways things/data can changes that we might not see but still need to consider, detect and investigate. Here’s the list:

1. stored procedures
2. triggers
3. views (in SQL in the sense that a user might not be able to see, query or access data and this “missing” data can “seem” like a defect)
4. indexes
5. caching
(again these last two, items 4 and 5 can make an impact on what a user can access making data seem to be missing, or not available)
6. batch runs (say EOM end of month, EOY end of year processes) that can disrupt the flow of data into a warehouse
7. ETLs (but you expected this one)

I’d be happy to grow this list or mind map. I post items like this to help the testing community – so if you have an ideas to expand this … please share.

Posted in business intelligence, exploratory testing, heuristics, storytelling | Comments Off on BI + Agile + ET in a mind map of charter possibilities

Testing Mnemonics as a Card Deck v2

I’ve updated the index card deck to include “AIM” by Robert Sabourin.

An article referencing this mnemonic was published by Better Software in an article called “X Marks the Test Case: Using Mind Maps for Software Design” back in 2006.

Here’s the update card deck .

Posted in Uncategorized | Comments Off on Testing Mnemonics as a Card Deck v2

Reporting a mobile defect

This past week I taught a class on mobile testing at CAST 2012. We brainstormed a list of factors to consider when reporting a mobile defect. Not all of these items will be a factor in the defect but it is a list to think about.

• Hardware Specific: Try another device, try a device that is close in “class”
• Note the OS, manufacturer, version of device, and the version of software
• Browser
• Device orientation
• Battery status
• Data
• Applications running in background
• Connectivity. Network – wifi – if there was any carrier switching taking place
• Location
• Device Settings
• Drivers, LED notifications, Bluetooth
• SIM Cards
• SD Cards
• Integration with other software
• Language Settings
• Varying Bandwidth
• Input speed and mechanism
• Navigation Path

Thanks to attendees: Kelli Cruz, Nicole Thompson, Cathy Clary, Cherri Shang, Rajiv Bhati, Saraswathy Ramadas, Chris Anders, Sakina Crocker, Jon Hagar, Gary Norris, Anand Ramdeo, and Jean Ann Harrison. And to Eric Proegler who facilitated.

Posted in conferences, mobile | Comments Off on Reporting a mobile defect

Testing Mnemonics as a Card Deck

I have an interest in helping testers new to exploratory testing or experienced testers who may feel be at a loss for testing ideas from time to time. That interest is the primary purpose behind a new class I’m working on currently titled Brainstorming for Testers. I taught the class yesterday at CAST 2012 for the first time.

One of the exercises in the class is to give each group a deck of index cards, the cards have about 20 testing mnemonics listed – each card has a mnemonic on the front and back of the card. The mnemonic is meant to inspire testing ideas.

The basic premise is for a tester to flip through the card deck until a testing idea comes to mind. A tester may find one mnemonic in particular helps generate ideas or they may find ideas come from one word of multiple mnemonics. There is no right or wrong way to use the deck. The deck is meant as an idea generator.

For each mnenomic I’ve listed the author as well as a URL to find more information about the mnemonic. If you’re an experienced tester, chances are a single testing-related word generates ideas and if you’ve been around the testing community awhile, you’re probably already familiar with the mnemonics and seeing the mnemonic will be enough of a jumpstart.

As a practical matter, I’ve formatted the mnemonics to an Avery template so that printing card deck is fairly easy to do. Here’s the deck.

Posted in exploratory testing, heuristics | Comments Off on Testing Mnemonics as a Card Deck

Applying the SFDPOT heuristic to mobile testing

One of the comments I’ve made about mobile testing is that we need to “find our mobile sanity” in part what I mean by this is – we need to recognize that everything we’ve learned about software testing doesn’t change because we have a new computing platform – in fact, we can apply lessons we’ve learned and skills we’ve already acquired to mobile testing. Perhaps this is obvious but I’ve met a few testers so anxious about “figuring out their mobile testing strategy” that I thought it was worth stating.

I thought I’d take the well-known testing heuristic from James Bach known as SFDPOT (sometimes referred to as San Francisco Depot) and apply that heuristic to mobile testing. If you’re not familiar with this heuristic see:
James Bach:

Let’s see how it holds up:

Structure. Test what it’s made of.
Can I download the app?
Can I download an update?
Can I update the app when I have multiple apps to update?
What happens when the operating system of the device is updated?
Does the app write to an SD card? What if the card is filled, removed or changed?

Function. Test what it does.
Does the app or site perform the tasks or features it was designed to?
Does the app or site prevent, block or not include features not intended on a mobile device?
Does the app or mobile website ever direct me to the “full site or desktop version”?
Does the app direct me to turn on necessary related services such as location services if those services are shut off?

Data. Test what it does it to.
Does the app find time related data (if relevant, such as movie show times) based on the device time?
Does the app find locations based on my location (such as movie theaters or hotels)?
Does the app integrate with any hardware such as bar code scanners for payment or other purposes?
If the app tracks rewards or points for frequent purchases, does this data get updated?
If the app integrates with a desktop profile or account information (such as My Profile or My Account), does the app and the desktop version keep information in sync with each other?

Platform. Test what it depends upon.
Does the app use location services?
Does the app depend on any device settings?
Does the app detect the platform it is running on such as a phone or tablet?
Can I find the app in the app store?

Operations. Test how it’s used.
Does the app function when I am moving and traveling around?
What happens when I switch to wifi vs 3G ?
What happens if my wifi connection is down or intermittent?
What happens when a mobile interruption occurs such as an incoming call or text message?
What happens when the device is set to airplane mode?

Time. Test how it’s affected by time.
What happens if the time zone is switched?
What happens when my location is switched?
Does the device auto detect a change in time zones and reset the time (and possibly) the day and date?
Does the app find time-related data based on my device time (such as reservation times available or movies show times)?

Can you think of more specific testing ideas based on your mobile app or mobile website – good.

Can you think of more ideas “generically” ? Good, please email or add a comment to this blog. I’d be happy to build on this for all of us in the software testing community.

Comments are welcome.

Posted in heuristics, mobile | Comments Off on Applying the SFDPOT heuristic to mobile testing

mobile: a few comments on testing, emulators & tools

Recently a colleague within the testing community emailed me three questions re: mobile testing. After answering the email, I realized that our exchange of questions and answers might be helpful to share – so following is the Q &A from our exchange (minus personal identification).

Hello –

I’ve shared my thoughts on each of your questions. I have to say – this is all based on my experiences and not necessarily reflective of the broader and global mobile market. Testing like so much else about mobile, will continue to change significantly in the next 1 and 2 years time.

1. What is mix of simulator based testing vs actual device based testing ? What factors decide/influence this mix?

In general, emulators have not been used with clients I have worked with. Occasionally a developer will use an emulator during unit testing but otherwise, emulators do not seem to be used or trusted.

The factor used seems to be a general distrust that an emulator will be accurate and this is based on opinions vs experiences encountered. It is also well known that through Device Anywhere and Perfecto Mobile, you can get your hands on nearly any device and so why tolerate an emulator?

Unreliable to depend upon but a typical experience I’m seeing is that within most teams, there is a natural mix of devices and people generally use their own devices at least part of the time. Bigger companies have funding and I know of a couple of companies that are building out mobile test labs – accumulating what they can.

There are a few “quick and free tools” on the market to execute mobile readiness tests. See the W3C, and Gomez, and dotMobil, See a comment on the next question for my “reaction” to these tools.

2. What are the class of defects or problems simulator based testing cannot discover?

I cannot fairly comment on this as I have more experience with devices than emulators.

I have seen issues reported from the quick tools that I mentioned above but found in most cases, the issues detected were either obvious or more obscure GUI-issues that we (the team) were not likely to address – the net result, not so helpful.

3. With so many tools claiming to do automation for mobile apps – how does one pick? In what ways automating (driving GUI from a tool) mobile app is diff from other software apps/web apps?

HP seems to be pushing their QTP app to handle mobile automation. I do not know anyone using it. Perhaps it works? Also a shocking but true fact – at this point in time, which I do expect will change – I do not know of any company that is successfully using automation in mobile. I know a couple of companies trying different tools, wanting to find a tool but at this time – and again, I think this will change before 1 or 2 years time – most companies are trying to figure out what their app and/or mobile site is going to do, what they need to test, what devices they need to cover, how to gather decent statistics and then will move onto mobile automation. The old expression – you have to walk before you can run seems to be the situation.

And although you didn’t directly ask me, I know that you are keen to continue to learn, as am I – so what do I read and where do I continue to learn?

I learn by other people sharing with me (why I feel it is important to get back to you and share as well.) I look to my colleagues Julian Harty (UK) and Jonathan Kohl (CA) and share with them as all of our schedules permit. Google their names, track down their work and articles.

I learn by experience and am currently (which has been true for the past couple of years) most eager for any mobile work I can get my hands on. When I do not have client work that gets me that experience, I continue with experimenting with a variety of apps and my own devices.

I follow Twitter closely because mobile is so current nearly anything in print is already dated. (You might look to see who/companies I follow as much of my Twitter following is mobile-centric.) That said, I think I own at least the “core” books on mobile offered from Amazon and O’Reilly as of today. I scan every testing newsletter for articles and attend webinars as feasible due to my sometimes crazy schedule. In this last 24 hours I devoured one book from Amazon in e-format only as the book is only days old, titled: Mobile Analytics by Jesus Mena. Straightforward title and rock-solid information.

Like most things I suppose, we find what we are truly interested in – no matter the day of week or time of day. At this time, mobile is a top interest of mine.

Now on an unexpected final note, given that I have taken the time to reply to you – I realize that your questions are not so very different than questions other people have asked me – I have decided to take my replies and post them into my blog. I won’t identify your name as it would be inappropriate of me to do that – but the replies I have shared with you are not different than I would share in my blog – and perhaps this exchange will help another tester? And so the circle of sharing continues.

Happy Testing~

Posted in mobile | Comments Off on mobile: a few comments on testing, emulators & tools