Assertions

Assertions are hooks that validate QaaS.Runner session data after the Runner act phase completes. Each assertion receives the stored session data (inputs, outputs, metadata) and returns a Pass or Fail status that is written to the Allure Report.

The QaaS.Common.Assertions NuGet package ships the built-in assertion library. You can also author custom assertions by implementing BaseAssertion<TConfig> from QaaS.Framework.SDK.

Package	QaaS.Common.Assertions
Source	QaaS.Common.Assertions
Depends on	QaaS.Framework.SDK

Installation

<PackageReference Include="QaaS.Common.Assertions" Version="x.x.x" />

Writing a Custom Assertion

See the Write Hooks guide for a step-by-step walkthrough.

Available Hooks

The built-in hooks below are grouped by usage area so it is easier to shortlist the right hook before drilling into configuration details.

Latency

• DelayByAverage: Checks for delay between an input source to an output source by subtracting the average timestamp of all inputs from the average timestamp of all the outputs.
• DelayByChunks: Checks for delay between an input source to an output source by subtracting the timestamp of input chunks of a configured size from a timestamp of output chunks of a configured size, takes the chunks in ascending order of the input/output lists.

Hermeticity

• HermeticByInputOutputPercentage: Performs a hermetic test by comparing the the count of a specified input with multiplied by given percentage modifier to the count of a specified output
• HermeticByExpectedOutputCount: Performs a hermetic test by comparing the count of a given output in a session to a given expected count
• HermeticByInputOutputPercentageInRange: Checks whether the percentage between configured inputs and outputs stays within the expected minimum and maximum range.
• ValidateHermeticMetricsByInputOutputPercentage: Compares the hermetic percentage calculated from configured inputs and outputs with the hermetic percentage reported by the metrics output.
• HermeticByExpectedOutputCountInRange: Performs a hermetic test by comparing the count of a given output in a session to a given expected minimum and maximum

Content validation

• OutputContentByExpectedCsvResults: Checks that the configured output content matches the expected values loaded from a CSV results file.
• HttpStatus: Performs a logic test on the http status of all selected outputs in a session by checking they all have the desired http status code

Contract validation

• OutputDeserializableTo: Performs a logic test that checks if the items of a configured output can all be deserialized using a configured deserializer
• ObjectOutputJsonSchema: Validates that each configured output item matches at least one JSON schema provided by the configured data sources.