To begin, let’s talk about the types of testing needed when rolling out a new dealer systems integration . We group testing into 3 types or phases.
- DSP Developer Testing
This is independent testing by the DSP where the performance of its integration code is evaluated. Developer Testing is generally unstructured, meaning there are no formal test cases provided (or required) by the publisher of the specifications (i.e. the automaker). Each DSP determines its own test plan in accordance with its internal, standard QA procedures. This should be completed in advance of any formal certification testing with an automaker. For the most part, this is not supported in any way by automakers – either testing tools for DSPs or SME support. Motive takes a different approach. We’ve invested heavily in a suite of testing tools, some deployed for DSPs during this phase of an integration project with the goal of identifying and resolving problems in their code as early as possible. We’ll talk more about this in Part 3.
- Certification Testing
Certification testing is highly structured testing intended to comprehensively test that the dealer system meets the automaker’s integration certification criteria for each interface. Each DSP is required to perform a prescribed set of test cases. These are evaluated for satisfaction of test case criteria and failed test cases are repeated until passed. Once Certification Testing is complete, a system would be approved to move to the final phase of testing - Dealer Pilot. Motive provides tools to support certification testing – the Motive Integrator Test Case Evaluator – enabling DSPs to perform an initial set of test cases entirely within this platform in advance of testing with live QA systems at the automakers. More on this in Part 4.
- Dealer Pilot
The last phase of testing is a live pilot where dealer systems are installed with a limited number of dealers and run in a production environment. This provides the final confirmation that the systems perform as expected and provides an opportunity for dealer feedback, identification of training or support issues and more. Common pilot exit criteria include: uninterrupted operations for a period of time, performance of a certain number of events for each type of interface, and execution of a pre-determined set of dealer business conditions. Once the pilot is completed, the dealer system is considered certified and able to roll out their integrated software to their dealers.
The testing process used for each of these phases can vary considerably. It’s important to keep in mind, testing with 3rd parties in a B2B integration program requires a different perspective. What might work for internal testing for a new software application doesn’t necessarily apply when testing and supporting 3rd party systems. And certainly internal testing and dealer system integration testing should be kept separate. Keep in mind that the better you support those DSP parties to enable them to quickly and efficiently develop and test their implementation of your integration requirements, the greater success your program will have overall.