This tutorial explains how to create a custom test suite and run tests against the device you want to test in the console. After the tests are complete, you can view the test results and detailed logs.
PrerequisitesBefore you begin this tutorial this tutorial, complete the steps outlined in Setting up.
Create a test suite definitionFirst, install an AWS SDK.
rootGroup
syntax
A root group is a JSON string that specifies which test cases to include in your test suite. It also specifies any necessary configurations for those test cases. Use the root group to structure and order your test suite based on your needs. The hierarchy of a test suite is:
test suite â test group(s) â test case(s)
A test suite must have at least one test group, and each test group must have at least one test case. Device Advisor runs tests in the order in which you define the test groups and test cases.
Each root group follows this basic structure:
{
"configuration": { // for all tests in the test suite
"": ""
}
"tests": [{
"name": ""
"configuration": { // for all sub-groups in this test group
"": ""
},
"tests": [{
"name": ""
"configuration": { // for all test cases in this test group
"": ""
},
"test": {
"id": ""
"version": ""
}
}]
}]
}
In the root group, you define the test suite with a name
, configuration
, and the tests
that the group contains. The tests
group contains the definitions of individual tests. You define each test with a name
, configuration
, and a test
block that defines the test cases for that test. Finally, each test case is defined with an id
and version
.
For information on how to use the "id"
and "version"
fields for each test case (test
block), see Device Advisor test cases. That section also contains information on the available configuration
settings.
The following block is an example of a root group configuration. This configurations specifies the MQTT Connect Happy Case and MQTT Connect Exponential Backoff Retries test cases, along with descriptions of the configuration fields.
{
"configuration": {}, // Suite-level configuration
"tests": [ // Group definitions should be provided here
{
"name": "My_MQTT_Connect_Group", // Group definition name
"configuration": {} // Group definition-level configuration,
"tests": [ // Test case definitions should be provided here
{
"name": "My_MQTT_Connect_Happy_Case", // Test case definition name
"configuration": {
"EXECUTION_TIMEOUT": 300 // Test case definition-level configuration, in seconds
},
"test": {
"id": "MQTT_Connect", // test case id
"version": "0.0.0" // test case version
}
},
{
"name": "My_MQTT_Connect_Jitter_Backoff_Retries", // Test case definition name
"configuration": {
"EXECUTION_TIMEOUT": 600 // Test case definition-level configuration, in seconds
},
"test": {
"id": "MQTT_Connect_Jitter_Backoff_Retries", // test case id
"version": "0.0.0" // test case version
}
}]
}]
}
You must supply the root group configuration when you create your test suite definition. Save the suiteDefinitionId
that is returned in the response object. You can use this ID to retrieve your test suite definition information and run your test suite.
Here is a Java SDK example:
response = iotDeviceAdvisorClient.createSuiteDefinition(
CreateSuiteDefinitionRequest.builder()
.suiteDefinitionConfiguration(SuiteDefinitionConfiguration.builder()
.suiteDefinitionName("your-suite-definition-name
")
.devices(
DeviceUnderTest.builder()
.thingArn("your-test-device-thing-arn
")
.certificateArn("your-test-device-certificate-arn
")
.deviceRoleArn("your-device-role-arn
") //if using SigV4 for MQTT over WebSocket
.build()
)
.rootGroup("your-root-group-configuration
")
.devicePermissionRoleArn("your-device-permission-role-arn
")
.protocol("MqttV3_1_1 || MqttV5 || MqttV3_1_1_OverWebSocket || MqttV5_OverWebSocket
")
.build()
)
.build()
)
Get a test suite definition
After you create your test suite definition, you receive the suiteDefinitionId
in the response object of the CreateSuiteDefinition
API operation.
When the operation returns the suiteDefinitionId
, you may see new id
fields within each group and test case definition within the root group. You can use these IDs to run a subset of your test suite definition.
Java SDK example:
response = iotDeviceAdvisorClient.GetSuiteDefinition(
GetSuiteDefinitionRequest.builder()
.suiteDefinitionId("your-suite-definition-id
")
.build()
)
Get a test endpoint
Use the GetEndpoint
API operation to get the test endpoint used by your device. Select the endpoint that best fits your test. To simultaneously run multiple test suites, use the Device-level endpoint by providing a thing ARN
, certificate ARN
, or device role ARN
. To run a single test suite, provide no arguments to the GetEndpoint operation to choose the Account-level endpoint.
SDK example:
response = iotDeviceAdvisorClient.getEndpoint(GetEndpointRequest.builder()
.certificateArn("your-test-device-certificate-arn
")
.thingArn("your-test-device-thing-arn
")
.deviceRoleArn("your-device-role-arn
") //if using SigV4 for MQTT over WebSocket
.build())
Start a test suite run
After you create a test suite definition and configureyour test device to connect to your Device Advisor test endpoint, run your test suite with the StartSuiteRun
API.
For MQTT customers, use either certificateArn
or thingArn
to run the test suite. If both are configured, the certificate is used if it belongs to the thing.
For MQTT over WebSocket customer, use deviceRoleArn
to run the test suite. If the specified role is different from the role specified in the test suite definition, the specified role overrides the defined role.
For .parallelRun()
, use true
if you use a Device-level endpoint to run multiple test suites in parallel using one AWS account.
SDK example:
response = iotDeviceAdvisorClient.startSuiteRun(StartSuiteRunRequest.builder()
.suiteDefinitionId("your-suite-definition-id
")
.suiteRunConfiguration(SuiteRunConfiguration.builder()
.primaryDevice(DeviceUnderTest.builder()
.certificateArn("your-test-device-certificate-arn
")
.thingArn("your-test-device-thing-arn
")
.deviceRoleArn("your-device-role-arn
") //if using SigV4 for MQTT over WebSocket
.build())
.parallelRun(true | false)
.build())
.build())
Save the suiteRunId
from the response. You will use this to retrieve the results of this test suite run.
After you start a test suite run, you can check its progress and its results with the GetSuiteRun
API.
SDK example:
// Using the SDK, call the GetSuiteRun API.
response = iotDeviceAdvisorClient.GetSuiteRun(
GetSuiteRunRequest.builder()
.suiteDefinitionId("your-suite-definition-id
")
.suiteRunId("your-suite-run-id
")
.build())
Stop a test suite run
To stop a test suite run that is still in progress, you can call the StopSuiteRun
API operation. After you call the StopSuiteRun
operation, the service starts the cleanup process. While the service runs the cleanup process, the test suite run status updates to Stopping
. The cleanup process can take several minutes. Once the process is complete, the test suite run status updates to Stopped
. After a test run has completely stopped, you n start another test suite run. You can periodically check the suite run status using the GetSuiteRun
API operation, as shown in the previous section.
SDK example:
// Using the SDK, call the StopSuiteRun API.
response = iotDeviceAdvisorClient.StopSuiteRun(
StopSuiteRun.builder()
.suiteDefinitionId("your-suite-definition-id
")
.suiteRunId("your-suite-run-id
")
.build())
Get a qualification report for a successful qualification test suite run
If you run a qualification test suite that completes successfully, you can retrieve a qualification report with the GetSuiteRunReport
API operation. You use this qualification report to qualify your device with the AWS IoT Core qualification program. To determine whether your test suite is a qualification test suite, check whether the intendedForQualification
parameter is set to true
. After you call the GetSuiteRunReport
API operation, you can download the report from the returned URL for up to 90 seconds. If more than 90 seconds elapse from the previous time you called the GetSuiteRunReport
operation, call the operation again to retrieve a new, valid URL.
SDK example:
// Using the SDK, call the getSuiteRunReport API.
response = iotDeviceAdvisorClient.getSuiteRunReport(
GetSuiteRunReportRequest.builder()
.suiteDefinitionId("your-suite-definition-id
")
.suiteRunId("your-suite-run-id
")
.build()
)
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4