Assertions¶

TL;DR — Assertions validate Runner session data after the act phase and turn observed inputs and outputs into reportable results.

Assertions are hooks that validate QaaS.Runner session data after the Runner act phase completes. Each assertion receives the stored session data (inputs, outputs, metadata) and returns a Pass or Fail status that is written to the Allure Report.

The QaaS.Common.Assertions NuGet package ships the built-in assertion library. You can also author custom assertions by implementing BaseAssertion<TConfig> from QaaS.Framework.SDK.

Installation¶

<PackageReference Include="QaaS.Common.Assertions" Version="x.x.x" />

Writing a Custom Assertion¶

See the Write Hooks guide for a step-by-step walkthrough.

Available Hooks¶

The built-in hooks below are grouped by usage area so it is easier to shortlist the right hook before drilling into configuration details.

Latency¶

• DelayByAverage: Checks for delay between an input source to an output source by subtracting the average timestamp of all inputs from the average timestamp of all the outputs.
• DelayByChunks: Checks for delay between an input source to an output source by subtracting the timestamp of input chunks of a configured size from a timestamp of output chunks of a configured size, takes the chunks in ascending order of the input/output lists.

Hermeticity¶

• HermeticByInputOutputPercentage: Performs a hermetic test by comparing the the count of a specified input with multiplied by given percentage modifier to the count of a specified output
• HermeticByExpectedOutputCount: Performs a hermetic test by comparing the count of a given output in a session to a given expected count
• HermeticByInputOutputPercentageInRange: Checks whether the percentage between configured inputs and outputs stays within the expected minimum and maximum range.
• ValidateHermeticMetricsByInputOutputPercentage: Compares the hermetic percentage calculated from configured inputs and outputs with the hermetic percentage reported by the metrics output.
• HermeticByExpectedOutputCountInRange: Performs a hermetic test by comparing the count of a given output in a session to a given expected minimum and maximum

Content validation¶

• OutputContentByExpectedCsvResults: Checks that the configured output content matches the expected values loaded from a CSV results file.
• HttpStatus: Performs a logic test on the http status of all selected outputs in a session by checking they all have the desired http status code

Contract validation¶

• OutputDeserializableTo: Performs a logic test that checks if the items of a configured output can all be deserialized using a configured deserializer
• ObjectOutputJsonSchema: Validates that each configured output item matches at least one JSON schema provided by the configured data sources.