Building the VoiceXML Forum Certification Program

By Gerald M. Karam and Kenneth G. Rehor

(Continued from Part 1)

Getting Certified

We envision a certification program that has the steps below that give the Vendor opportunities to explore their product’s conformance, and then commit to certification, and ultimately use that to market advantage.

Informal Testing.  With this first step, the Vendor obtains test suite or access to test suite to prepare for certification.  The CA will make the test suite available to vendors and others as appropriate; there may be a fee involved. By getting access to the test suite in advance, the Vendor will be able to run the tests, and understand the outcomes before entering into the next phase of certification.

Implementation Conformance Statement.  This is the first formal phase of the certification process.  The Vendor must complete and submit the implementation conformance statement that will include: definition (configuration) of what platform will be tested, including optional features to be claimed, and agreement to policies and legal documents for certification.  This is an important reference for the conformance testing because the configuration being certified will constrain the extent of the resulting certification label once the conformance testing is successfully passed.  For example, if a Vendor’s VoiceXML Platform were deployed in a Linux operating environment and certified, then if the Vendor wanted to also claim certification for a platform in a Windows 2000 operating environment, then there may be a requirement to go through conformance test again since the operating environment is different.  The fee to enter the certification process would be paid to the CA at this time, and a time clock for when the certification process must be completed begins.  This document would be confidential between Vendor and CA; the VoiceXML Forum would not know that a Vendor was going to undergo certification or even interested in certification.  Only at later stages, and at the discretion of the Vendor, would the VoiceXML Forum be told of any certification information. 

The implementation conformance statement is an important reference for the conformance testing because the configuration being certified will constrain the extent of the resulting certification label once the conformance testing is successfully passed.  For example, if a Vendor’s VoiceXML Platform were deployed in a Linux operating environment and certified, then if the Vendor wanted to also claim certification for a platform in a Windows 2000 operating environment, then there may be a requirement to go through conformance test again since the operating environment is different; i.e., the certification would not automatically extend.  The fee to enter the certification process would be paid to the CA at this time. 

Formal Testing.  The Vendor next tests their product against reference web server for the range for tests that are required based on their implementation conformance statement.  Each test would have one of three outcomes: PASS, FAIL or UNTESTED.   If a test outcome is FAIL/UNTESTED does not automatically imply that a Vendor has failed the conformance testing because such an outcome could become relabeled as PASS if there is a known/discovered Test Suite Deficiency or Specification Deficiency.  A Test Suite Deficiency is an error in a test case; such errors would be raised during the testing, and then resolved through the Problem Reporting process.  Since a test suite, like any other part of software development operates under version control, a repair to a test case would not appear until it was included in a new version of the test suite.  A Specification Deficiency is an ambiguity or error in the specification on which the test case is based.  Such issues would be resolved through the body responsible for the specification, and would remain as deficiencies until they were resolved (again usually through a new version of the specification).

Completion of Certification.  During the formal testing, the Vendor may fail some tests and need to make corrections.  We expect that a Vendor will be able to do a limited amount of correction retesting included in certification fee. Finally the results of certification (pass/fail) are maintained in confidence between the Vendor and the CA.

Using the VoiceXML Forum Certification Mark.  Once certification is completed successfully, it is the Vendor's discretion on how to proceed further.  There are at least the following three choices being considered. (1) Do nothing. Having shown that the platform passed the certification process may be sufficient for some internal reason. (2) Private Disclosure. The Vendor may wish to disclose the result to a third party, such as a buyer as part of a purchase qualification.  In this case the CA would discloses to the other party the satisfactory completion of certification process. (3)

Public Disclosure. In this case the CA would disclose to the VoiceXML Forum that the Vendor’s processor has satisfactorily completed the certification process.  Consequently the Vendor could use their certification mark on their marketing materials, and the VoiceXML Forum could publicize the Vendor’s certification.  For private or public disclosure the vendor will be required to sign the licensing agreement for use of marks (this does not imply certification, but how to behave if you are certified).  The certifications will have a valid lifetime (such as one year) with possible renewals handled through the CA.

For the Future

There are a lot more details to this task than revealed in this short article. For example, detailed processes for how to handle changes to a certified platform, processes for handling the problem reporting, and so on.  A branding program needs to be developed so that certified systems can be clearly represented and identified. Next phases of the Certification Program will include test suites for development tools and application testing. This information will be presented in future articles and on the VoiceXML Forum's website as we expand these programs. 

A strong conformance program will help the VoiceXML market grow through interoperable platforms, tools and applications.

back to the top

 

Copyright © 2001 VoiceXML Forum. All rights reserved.
The VoiceXML Forum is a program of the
IEEE Industry Standards and Technology Organization (IEEE-ISTO).