Test-Driven Development (TDD) is a widely used concept used in most areas of software development. Software developers using TDD start with writing test cases (based on what the software or piece of code is expected to do), then write the code, so that the test passes, and keep repeating these steps on a regular basis. This concept enables developers to find issues at the earliest possible phase when developing new features so they will be in the know of any issues before merging the changes into the existing codebase. A large number of studies prove that writing tests is crucial when developing software and will result in a decrease in production issues and of course less headache with fixing bugs.
TDD IN THE BI WORLD
In an ideal world, this should be no different in the BI space. BI developers not only visualize data but also make a lot of data transformations, implementing various business logic into the visualization layer, blending data from multiple sources, etc. This simply results in a very high risk of potentially showing data incorrectly.
Test Driven Development steps in general purpose programming languages.
In BI, however, due to various reasons like the heavy reliance on third-party software solutions such as Tableau, Power BI, etc., the different skillset of BI developers, and the nature of BI dashboards, using Test-Driven Development would be very complicated. Even the most simple test cases are very difficult to set up with traditional web frameworks.
Let’s say you want to check if a particular filter exists on a Tableau dashboard. Checking it will be difficult once you realize that Tableau renders the page content dynamically full of HTML <canvas> elements where CSS/XPath selectors are not working, hence you can’t interact with these objects on the page. Testing will quickly become even more complicated once you start implementing business logic into calculated fields/dimensions and want to validate if these figures are calculated correctly. However, leaving these objects untested sooner or later will cause pain and increases the risk of production issues on a business-critical dashboard.
Prevailing test frameworks, like Selenium or xUnit, are designed for web applications or general-purpose programming languages and not compatible with these BI solutions, so BI Developers don’t have many testing tools to rely upon. So, what can developers do to overcome this challenge? I’ll show you a detailed example of how to develop a Tableau dashboard using the TDD concept with Kinesis CI. Every Tableau professional who’s developing dashboards for decision-makers should heavily rely on automated testing for their dashboards, to make sure they work as expected, and existing functionality is maintained after rolling out changes.
TDD FOR TABLEAU WITH KINESIS CI
With a variety of flexible test cases, Kinesis CI allows you to implement Test-Driven Development for Tableau. Kinesis CI uses the Tableau JS/REST APIs and further internal communication methods to interact with the visualizations where web testing frameworks are not efficient. Moreover, it comes with built-in data comparison tasks that are mandatory to validate the figures shown on the dashboard.
Using Kinesis CI, the Test-Driven Development cycle with Tableau looks as follows:
Test Driven Development steps in Tableau with Kinesis CI
These steps are similar to the ones in the original TDD lifecycle but there are a few small differences:
You will need to use two tools at different stages; Tableau Desktop/Server as usual and Kinesis CI as an additional tool to design and run test cases for Tableau.
Make the Viz is equivalent to the standard Write Code step.
Pimp to Viz is equivalent to the standard Refactor step.
REAL-LIFE TDD EXAMPLE FOR A TABLEAU DASHBOARD
If you work in an Agile environment, you’ll most likely get a requirement such as the example below:
“As the Chief Financial Officer, I want to have a Tableau dashboard that shows the daily sales in a chart with year-on-year figures in a selected period and country, so that I will have a quick insight into how the business is progressing on a daily basis.”
Let’s say you or a business analyst discussed this in more detail and they made a draft of how they can imagine the dashboard in action:
First draft of the dashboard.
You were thinking of testing and also defined four basic acceptance criteria:
There should be one sheet with the name ‘Daily Sales’.
There should be a country filter with three values: ‘The United States of America’, ‘United Kingdom’, ‘Australia’.
There should be two date parameters for ‘Start Date’ and ‘End Date’.
Two figures always should be on the chart: ‘Total Sales’ and ‘Total Sales Previous Year’. Both of them should always be greater than zero as a day with negative sales is not feasible.
There are quite a few options to make mistakes when the Tableau dashboard is developed, for instance, the filter is missing, no YoY figures, wrong color coding, or malfunctioning calculated fields. To prevent these mistakes from occurring, you should write test cases that verify if the acceptance criteria are met:
STEP 1: WRITE FAILING TEST
Traditionally once you get the requirements, you start developing the dashboard, however, with Test-Driven Development you need to define the tests first. In Kinesis CI, in functional testing, every test case is made up of a series of tasks that are run one after another by the automated test runner to simulate the actions of a user as a test case. In this example, we will create functional tests with a few steps to cover every acceptance criteria: Login to Tableau, Open Viz, Assert Worksheets Exist, Assert Parameter Equals, Assert Filter Equals and Assert Data Rules.
Login to Tableau and Open Viz as preparatory tasks to simulate a user navigating to the relevant Tableau dashboard. The Assert Parameter Equals and Assert Filter Equals tasks will verify the existence of Start/End date parameters and the country filter with the required values: United States of America, United Kingdom, and Australia. The Assert Data Rules task verifies if ‘Total Sales’ and ‘Total Sales Previous Year’ are both larger than zero.
Series of selected tasks will test the functional acceptance criteria.
At this stage, you will save the test and push the Kinesis CI project into your git repository.
STEP 2: RUN AND FAIL TEST
Running the test first time validates that the test itself is working correctly. It should fail because the dashboard doesn’t exist as we haven’t started to build the dashboard.
First failing test run.
STEP 3: WRITE CODE AND PASS TEST (MAKE THE VIZ)
OK, now it’s time to open your Tableau Desktop and develop the dashboard. Based on the requirements you will end up with something like this:
First version of the viz focusing only to the functional requirements.
In this step, we should focus on the functional requirements as we need to make sure it meets the acceptance criteria. We should not spend too much time at this stage on cosmetics and fancy layouts (we can do that a little bit later). Ideally, once the viz is developed, all of the above test cases should pass, verifying that the required user criteria are met.
STEP 4: ENSURE ALL OLD TEST CASES PASS
In this step we need to run every test case previously done, making sure the new features are not breaking the existing ones. At the moment we only have one test case, hence it is not adding too much extra, but this step will be a crucial one going forward.
Passing test cases on a functionally working dashboard.
Note that the Assert Data Rules task allows you to create advanced data validation rules by excel-like-formulas that reference Tableau dimensions and measures. To validate the sales figures from this year and last year we added two formulas. Both values should be greater than zero but null values are allowed. Every formula returns TRUE or FALSE values and if it is false then the test will fail too.
OR([SUM(In Scope – During – Amount)] > 0, ISBLANK([SUM(In Scope – During – Amount)])) — Formula to validate sales amount this year
OR([SUM(In Scope – LY – Amount)] > 0, ISBLANK([SUM(In Scope – LY – Amount)])) — Formula to validate sales amount in the previous year
Tests are green so we can proceed to the last step.
STEP 5: REFACTOR (PIMP THE VIZ)
At this stage, you can ‘pimp your viz’ and improve the layout for a visually pleasing Tableau dashboard that is in-line with any formatting requirements of your organization. Every test should be green at this stage too and if needed you can add an extra Assert Image Equals task in the Kinesis CI test case to validate the layout of the dashboard.
Functionally tested and nice looking daily sales Tableau dashboard (Viz inspired by Eva Murray’s Tri My Data blog).
Next time when you need to modify something on the dashboard you will REPEAT the entire flow so you will go back to point 1 and make new test cases, first focusing on the new feature requested by the business. i.e. adding new filters, worksheets, filter actions, calculated measures, etc.
Of course, with the above TDD functional test example, we’ve not covered all the testing concepts that non-BI developers have been using for a long time and for a very good reason. The good news is that most of these are now possible with Tableau too so you will not only be convinced that your dashboard is looking great but you can feel confident that the underlying business logic is always working correctly.
If you’re looking for help on your Test-Driven Development for Tableau, get in touch and one of our experts will help you out.
Privacy & Cookies Policy
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.