Quality assurance and end user testing
ThoughtFarmer Development and QA approach
ThoughtFarmer's Development and QA team follows an Agile approach to software development. Agile development is a widely accepted and effective approach to project management within the software development and testing community and is characterized by a highly focused and rapidly iterative software process. The Agile process practiced by our engineering team is based on the principles laid out in the Agile Manifesto.
We use Jira to manage the development tasks and issues raised by the QA team. Each ticket is developed and tested in an isolated branch, allowing the QA team to be confident that the code they are testing is the only change. Our build process uses pull requests to ensure that releases contain only the changes that meet our high standards for code quality. Pull requests also allow us to track the history of code changes and provide release notes.
Secure development approach
An application is only as secure as the code it depends on. Our development team follows the work of the Open Web Application Security Project (OWASP) with a focus on its list of top 10 security vulnerabilities. During development, we use code review software to ensure not only the correct implementation, but also that the code is not susceptible to security issues. Wherever possible we use secure frameworks, such as the Microsoft AntiXSS framework, to eliminate entire classes of vulnerabilities (e.g. XSS and CSRF) from our application.
ThoughtFarmer QA methodology
Some engineers on our team have been working on ThoughtFarmer since its conception in 2006. With the full ThoughtFarmer history and an exceptionally strong knowledge of the product, ThoughtFarmer QA engineers support our team of developers by diligently testing each ThoughtFarmer release that is built.
In addition to all of the testing performed by the development team, we also use new builds of ThoughtFarmer internally. Our intranet runs on ThoughtFarmer, and we are able to test and provide feedback on new features early in the development process. Before a release ever makes it to a customer, we have used it heavily internally.
Our team combines automated tests with manual testing, as outlined below.
ThoughtFarmer testing methods
|Type of test||Description|
|Manual tests||ThoughtFarmer QA engineers perform manual testing by assuming the role of an end user using the application. Engineers test the application manually, according to detailed task workflows that are described for each feature of the application. Manual tests test for bugs, interaction workflow, accessibility and visual design specs. Issues discovered via manual testing typically result in the creation of a regression or unit test, to find this issue (should it appear again) immediately via automation.|
Regression tests consist of thousands of individual automated tests that, if executed manually, would take days to complete. They give us a great safety harness and are monitored daily. Every failure is investigated. Regression tests rigorously examine all of the main use cases and paths through the application, and are written to allow us to easily switch and test against multiple environments. Regression testing allows us to easily monitor overall product quality.
Our regression tests are set up in Visual Studio with Selenium WebDriver in C#, an automated testing tool that uses an actual web browser to perform tests.
We have additional regression tests for the FormFlow application using Cypress.
Regression tests are run every night. Tests are written in a way that Selenium will recognize elements on the interface through CSS or XPath. To ensure accuracy, test cases are updated every time the design of the user interface is changed.
ThoughtFarmer currently has over 3900 regression tests that cover over 4100 assertions (things to validate) within the ThoughtFarmer codebase.
|Continuous tests||Continuous tests are similar to regression tests, but they run every time a developer commits a change to any branch. They validate the core of the system, and are run many times throughout the day. Continuous tests allow us to capture issues immediately.|
|API integration tests||API integration tests are regression tests focussed on the public APIs of ThoughtFarmer. They run along with the regression tests and enable us to have confidence that APIs available to customers stay consistent and reliable. These tests also allow us to verify that the documentation published for these APIs is up-to-date and correct. Our API tests are written with the Frisby framework.|
|Mobile app regression tests||We have regression tests for the mobile application using the BrowserStack App Automate tool. These tests run every night using a variety of iOS and Android devices.|
|Load and performance tests||We perform regular benchmark testing, which performs load tests on different components of the ThoughtFarmer application, as well as the entire application. For these types of QA tests, our team uses k6.io.|
We use Qualys, the leading provider of cloud-based information security and compliance solutions. As part of our development process, a test instance of ThoughtFarmer is tested against the latest known security threats. Issues raised from these scans are fed back into ThoughtFarmer and incorporated into our development process.
We also run tests on every ThoughtFarmer build using SonarCloud which reports on bugs, vulnerabilities and code smells. The SonarCloud tests are built into our continuous integration.
What can you do to test your ThoughtFarmer site and its content?
Before launch and after your content migration is complete, you may want to perform some additional testing with your team. While our team is very thorough in ensuring the ThoughtFarmer product passes all our Quality Assurance tests, we aren't able to test the site with your unique content in place. Below are some areas of ThoughtFarmer we recommend you verify before going live.
If you run into certain areas that are not behaving as expected and require assistance, please contact us through the ThoughtFarmer Helpdesk and one of our friendly support team members will help you out.
- Browser and environment
- When testing your intranet, ensure you are using the specific operating systems, devices, and browsers that your end users and intranet administrators will be using.
- You may want to run some bandwidth testing on your site, especially if you are planning to stream videos on the intranet. For hosting videos, we recommend using an external service such as YouTube, Vimeo, or Wistia.
- Ensure all content is formatted correctly and adheres to your organization's internal brand standards and guidelines. Working with your content authors, ensure all content adheres to your Intranet Style Guidelines.
- Ensure that all content is searchable through the ThoughtFarmer search (regular search and predictive search from the main search bar).
- Ensure that all users show up in the employee directory.
- If you are setting up Groups and you are using News from Subscribed Groups on your homepage, verify that all content feeds are going to group pages and the homepage.
- If you are feeding in external RSS feeds into any intranet page, verify the RSS content is coming in and the RSS post thumbnail images are displaying as expected.
- Images & videos
- Do a spot check of videos (if posted) to ensure they play correctly. Test on different browsers if your users use a variety of browsers.
- Do a spot check of a few news articles to ensure the thumbnail images display correctly.
- Security permissions
- Do a spot check of major sections on the intranet to ensure the security permissions have been set up correctly. If needed, set up a test user to ensure they see the correct content or turn off admin mode if you're an admin to verify how your content displays for an end user. Don't forget to delete or deactivate your test user if you no longer need it afterward, to avoid using an extra user license.
- Check to ensure all required security groups have been created via the Admin panel: Users & security section > Security groups page.
- Do a spot check of all the custom cards on your intranet (if applicable).
- ThoughtFarmer Analytics
- Check that ThoughtFarmer Analytics is running via the Admin panel: Logs & statistics section > ThoughtFarmer Analytics page (if requested).
- Ensure all users that require access have been added. If they don't appear, and you are syncing with an external location such as Active Directory, you can check the Admin panel: Users & security section > Employee directory connector > [your external user store] > Synchronization Settings tab to check which group you are syncing with. Ensure users that require access are members of that group.
- If you are syncing with an external location, ensure all field mappings that you would like to include have been added. Field mappings are located in the Admin panel: Users & security section > Employee directory connector > [your external user store] > Field Mappings tab.
- If you use regular users, you can double check the Admin panel: Users & security section > User management page to see which users have already been added.
- If you are setting up SSO on ThoughtFarmer, include this in your user testing. If Windows SSO is not working, you may need to make changes to users' browser settings.
Please sign in to leave a comment.