OpenAPI Specification Contracts
Providers may specify an OpenAPI Specification as a Provider Contract, enabling teams to get reuse out of existing tools and processes.
Supported versionsβ
Version | Supported? |
---|---|
Swagger (1.x) | β |
Swagger (2.0) | β |
OAS (up to 3.0.3) | β |
Compatibility with Consumer Contractsβ
Pact Consumer Contracts are the only compatible contracts at this time.
Publishing the Provider Contract + Results to Pactflowβ
There are several ways to currently publish contracts to Pactflow.
The links in the list will take you to the respective documentation in each repo.
Otherwise see the Installation section and Usage instructions on this page.
Installationβ
Dockerβ
- The Pact Broker CLI is packaged with the other Ruby command line tools in the pactfoundation/pact-cli Docker image.
docker pull pactfoundation/pact-cli
Pact Standalone CLIβ
- Download the latest pact-ruby-standalone package.
- Available from version
v1.89.00
and upwards
- Available from version
- Installation instructions are provided in the above link, for
windows
/macos
/linux
- You do not need Ruby to run the CLI, as the Ruby runtime is packaged with the executable using Travelling Ruby.
windows (bash shell)β
curl -LO https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v1.89.00/pact-1.89.00-win32.zip && \
unzip pact-1.89.00-win32.zip && \
./pact/bin/pactflow.bat help
MacOSβ
curl -LO https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v1.89.00/pact-1.89.00-osx.tar.gz && \
tar xzf pact-1.89.00-osx.tar.gz && \
./pact/bin/pactflow help
Linuxβ
curl -LO https://github.com/pact-foundation/pact-ruby-standalone/releases/download/v1.89.00/pact-1.89.00-linux-x86_64.tar.gz && \
tar xzf pact-1.89.00-linux-x86_64.tar.gz && \
./pact/bin/pactflow help
Pact Broker Client (Ruby)β
- Either
- Add
gem 'pact_broker-client'
to your Gemfile and runbundle install
- Install the gem directly by running
gem install pact_broker-client
- Add
GitHub Actionsβ
Copy and paste the following snippet into your .yml file.
- name: publish-provider-contract action
uses: pactflow/actions/publish-provider-contract@v0.0.2
Publishingβ
Usageβ
Usage:
pactflow publish-provider-contract CONTRACT_FILE \
--broker-token=BROKER_TOKEN
--broker-base-url=BROKER_BASE_URL
--provider PROVIDER \
--provider-app-version PROVIDER_APP_VERSION \
--branch BRANCH \
--content-type CONTENT_TYPE \
--verification-exit-code=EXIT_CODE \
--verification-results REPORT_PATH \
--verification-results-content-type REPORT_CONTENT_TYPE \
--verifier VERIFIER
Options:
--provider=PROVIDER
# The provider name
-a, --provider-app-version=PROVIDER_APP_VERSION
# The provider application version
-h, [--branch=BRANCH]
# Repository branch of the provider version
-t, [--tag=TAG]
# Tag name for provider version. Can be specified multiple
times.
[--specification=SPECIFICATION]
# The contract specification
# Default: oas
[--content-type=CONTENT_TYPE]
# The content type. eg. application/yml
[--verification-success], [--no-verification-success]
# Whether or not the self verification passed successfully.
[--verification-exit-code=N]
# The exit code of the verification process. Can be used instead
of --verification-success|--no-verification-success for a
simpler build script.
[--verification-results=VERIFICATION_RESULTS]
# The path to the file containing the output from the
verification process
[--verification-results-content-type=VERIFICATION_RESULTS_CONTENT_TYPE]
# The content type of the verification output eg. text/plain,
application/yaml
[--verification-results-format=VERIFICATION_RESULTS_FORMAT]
# The format of the verification output eg. junit, text
[--verifier=VERIFIER]
# The tool used to verify the provider contract
[--verifier-version=VERIFIER_VERSION]
# The version of the tool used to verify the provider contract
-o, [--output=OUTPUT]
# json or text
# Default: text
-b, --broker-base-url=BROKER_BASE_URL
# The base URL of your Pactflow account e.g. https://myaccount.pactflow.io
-k, [--broker-token=BROKER_TOKEN]
# The RW Token from your Pactflow account - https://myaccount.pactflow.io/settings/api-tokens
-v, [--verbose], [--no-verbose]
# Verbose output. Default: false
Dockerβ
docker run --rm -v /${PWD}:/${PWD} -w ${PWD} \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:0.50.0.28 \
pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 3a0994c \
--branch test-pactflow-command \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-06-09-14-18-33-406-0.json \
--verification-results-content-type text/plain\
--verifier postman
Pact Standalone CLIβ
./pact/bin/pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 3a0994c \
--branch test-pactflow-command \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-06-09-14-03-30-715-0.json \
--verification-results-content-type text/plain\
--verifier postman
Pact Broker Client (Ruby)β
pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 3a0994c \
--branch test-pactflow-command \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-06-09-14-03-30-715-0.json \
--verification-results-content-type text/plain\
--verifier postman
GitHub Actionsβ
# (This just saves defining these multiple times for different pact jobs)
env:
version: "1.2.3"
application_name: "my-api-provider"
pact_broker: ${{ secrets.PACT_BROKER_BASE_URL }}
pact_broker_token: ${{ secrets.PACT_BROKER_TOKEN }}
jobs:
pact-publish-oas-action:
steps:
- uses: actions/checkout@v3 # MANDATORY: Must use 'checkout' first
- uses: pactflow/actions/publish-provider-contract@v0.0.2
env:
oas_file: oas/swagger.yml
results_file: ${{ env.results_file }}
Interpreting verification result failuresβ
Verification results are generated dynamically when can-i-deploy
is invoked for a given set of application versions and target environments.
When can-i-deploy
is called, you will get a table of results that shows which applications are compatible and the verification URL which explains the results.
Output from can-i-deploy
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-------------------------------|--------------------|--------------------------------|--------------------|----------|--------
pactflow-example-consumer-w... | 5785fb8+1622544123 | pactflow-example-provider-r... | 7f3d83f+1622544125 | true | 1
VERIFICATION RESULTS
--------------------
1. https://testdemo.pactflow.io/hal-browser/browser.html#https://testdemo.pactflow.io/contracts/provider/pactflow-example-bi-directional-provider-restassured/version/7f3d83f%2B1622544125/consumer/pactflow-example-bi-directional-consumer-wiremock/pact-version/b421f8d1c0691e8304492c716e546427c4267c7f/verification-results (success)
Clicking on the verification results will take you to the resource in Pactflow and show you the detailed analysis.
Response Objectβ
summary
Whether or not the verification was successful
crossContractVerificationResults
This element contains the results of comparing the mock (pact contract) to the OpenAPI specification
providerContractVerificationResults
This contains the results of the provider verification, including the tool used to verify it, whether the test passed or failed and the base64 encoded OAS contract.
Successful resultβ
{
"summary": {
"success": true
},
"crossContractVerificationResults": {
"success": true,
"results": {
"errors": [],
"warnings": []
},
"verificationDate": "2021-06-01T10:42:30.980+00:00",
"verifier": "atlassian-swagger-mock-validator",
"verifierVersion": "10.0.0"
},
"providerContractVerificationResults": {
"success": true,
"content": "dGVzdGVkIHZpYSBSZXN0QXNzdXJlZAo=",
"contentType": "text/plain",
"verifier": "verifier"
},
"_links": {
"self": {
"title": "Cross contract and Provider Contract verification results",
"href": "https://testdemo.pactflow.io/contracts/provider/pactflow-example-bi-directional-provider-restassured/version/7f3d83f%2B1622544125/consumer/pactflow-example-bi-directional-consumer-wiremock/pact-version/b421f8d1c0691e8304492c716e546427c4267c7f/verification-results"
}
}
}
Failure resultβ
{
"summary": {
"success": false
},
"crossContractVerificationResults": {
"success": false,
"results": {
"errors": [{
"code": "request.body.incompatible",
"message": "Request body is incompatible with the request body schema in the spec file: should NOT have additional properties",
"mockDetails": {
"interactionDescription": "POST_/products_7436b06b-b387-4535-9d3a-da149d9826ba",
"interactionState": "[none]",
"location": "[root].interactions[0].request.body",
"mockFile": "pact",
"value": {
"id": "27",
"name": "pizza",
"type": "food",
"price": 27
}
},
"source": "spec-mock-validation",
"specDetails": {
"location": "[root].paths./products.post.requestBody.content.application/json.schema.additionalProperties",
"pathMethod": "post",
"pathName": "/products",
"specFile": "oas",
"value": false
},
"type": "error"
}, {
"code": "response.body.incompatible",
"message": "Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - price",
"mockDetails": {
"interactionDescription": "GET_/products_646e1d83-da87-4155-9a43-3b24a2014cf3",
"interactionState": "[none]",
"location": "[root].interactions[1].response.body[0]",
"mockFile": "pact",
"value": {
"name": "pizza",
"id": "10",
"type": "food",
"price": 100
}
},
"source": "spec-mock-validation",
"specDetails": {
"location": "[root].paths./products.get.responses.200.content.application/json; charset=utf-8.schema.items.additionalProperties",
"pathMethod": "get",
"pathName": "/products",
"specFile": "oas",
"value": false
},
"type": "error"
}],
"failureReason": "Mock file \"pact\" is not compatible with spec file \"oas\"",
"warnings": []
},
"verificationDate": "2021-06-02T01:27:54.815+00:00",
"verifier": "atlassian-swagger-mock-validator",
"verifierVersion": "10.0.0"
},
"providerContractVerificationResults": {
"success": true,
"content": "dGVzdGVkIHZpYSBSZXN0QXNzdXJlZAo=",
"contentType": "text/plain",
"verifier": "verifier"
}
}
In the case of a failure, the following elements of the error
are most helpful in diagnosing the problem:
message
Summary of the problem. In most cases, you can understand the problem immediately. As above, you can see there are two errors - one for the request body, and another for the response body. In both cases (the request and response body), there is an additional unexpected property
price
expected by the consumer.mockDetails
Contains details of the Consumer Contract (mock) that are problematic, including the path to the interaction in the contract and the request/response details.
specDetails
Contains details of the Provider Contract (spec) that are problematic, including the path to the component of the resource in the OpenAPI specification the mock is incompatible with.
Considerationsβ
When using OpenAPI Specifications as a Provider Contract, you should be aware of the following limitations.
Document limitationsβ
- The OAS must be a valid YAML or JSON file. Pactflow will pre-validate the document is parseable and error if they aren't valid.
- OAS documents must not be split across multiple files. You should combine any documents together, using tools like OpenAPI Merge or speccy.
- YAML formatted OAS documents must not use anchors, due to the potential security issues (see YAML bomb for more). If your auto-generated specs have anchors, you can pre-process them via tools like spruce, that will expand them for you.
Testingβ
- Note, Pactflow automatically sets
additionalProperties
in your OAS is set tofalse
on any response body, to ensure a consumer won't get false positives if they add a new field that isn't actually part of the spec (see https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects for an interesting read on why this is necessary. TL;DR - it's JSON Schemas fault) - It is recommended to allow
additionalProperties
on request items to align with Postel's Law - Implementing a spec is not the same as being compatible with a spec (read more). Most tools only tell you that what youβre doing is not incompatible with the spec. NOTE: We plan to address this problem in the future via our OAS Testing Tool
- You are responsible for ensuring sufficient OAS coverage. To highlight this point, in our Dredd example, we do not test the 404 case on the provider, but the consumer has a pact for it and it's tests still pass! NOTE: We plan to address this problem in the future via our OAS Testing Tool
allOf
support and other logical keywordsβ
Because Pactflow sets additionalProperties
to false
to prevent false positives during validation, it means that the use of allOf
is not supported.
See this write up on this specific issue.