Data warehouse/business intelligence (BI) projects are different. The primary reason BI testing is so different (from say web application testing) is that most of the testing within a BI project is data reconciliation testing for which there is no visual application for a tester to work with. I say “most” testing because I suppose you could point to a dashboard or analytic report that represents output from the warehouse and claim that those tangibles are visuals but for the most part, the testing that takes place on a BI project is non-visual, there is no application to interact with, instead testing is generally focused on data accuracy and completeness. This type of testing requires a sense of investigation and being able to test and construct ideas about what to test and where to find issues without the aid of a of a tangible interactive application.
I construct mental images in my mind about the process the data flows through. I consider the originating sources of data and how that data has to traverse to get to the warehouse. On one BI project I worked on, I summarized a chunk of the testing efforts as “the story of a cube” and relayed the process of how the warehouse was built and tested to the business owners by way of the story. My storytelling in this case worked fairly well. I used a bit of humor, I drew some visuals and talked through the process. But that was then and this is now …
A client recently asked me about testing on a BI project where they are using Agile. They are also intrigued about the prospect of using Exploratory Testing but are unsure what Charters could be. So in the background of juggling several tasks and conference work, I’ve been puzzling over this question.
I’ve been building out a mind map to address this question and to help me think it through. Not long ago I referenced I was having a solo brainstorming session on Twitter about this topic. Minus one branch of the map that is client-sensitive, here’s my BI + Agile + ET Mind Map.
In a presentation that I’ve given a few times this past year, I’ve identified ways in which data changes or can be changed through “silent” non-visual processes, the point of the list is to remind myself and other testers of ways things/data can changes that we might not see but still need to consider, detect and investigate. Here’s the list:
1. stored procedures
2. triggers
3. views (in SQL in the sense that a user might not be able to see, query or access data and this “missing” data can “seem” like a defect)
4. indexes
5. caching
(again these last two, items 4 and 5 can make an impact on what a user can access making data seem to be missing, or not available)
6. batch runs (say EOM end of month, EOY end of year processes) that can disrupt the flow of data into a warehouse
7. ETLs (but you expected this one)
I’d be happy to grow this list or mind map. I post items like this to help the testing community – so if you have an ideas to expand this … please share.