Set up and use Jaylert Test

Quick start

1. Open a Jira project and click Tests in the project navigation.
2. Create a folder (＋ next to Folders) and a test case (New test case), or bring your existing tests in with Import.
3. Click Run these to create a test cycle from the test cases shown, then run each test step by step.
4. Link test cases to the Jira issues they verify, so coverage shows up on the issues and in Reports.

Test cases

A test case has a title, a folder, a priority (Highest to Lowest), labels, a precondition and steps. Each step has an action, optional test data and an expected result. Reorder steps with the arrows; empty steps are dropped when you save.

Test cases get a key per project, for example TST-T12. Folders can be nested; only empty folders can be deleted. Use the search box to filter by title.

Import

Click Import at the top of the folder list. Choose a CSV file, or copy the cells in Excel or Google Sheets, including the header row, and paste them.

• Columns are recognised automatically from common headers used by Zephyr, TestRail and Xray exports (for example Issue Key, Test Summary, Step, Test Data, Expected Result). You can change every mapping.
• Steps on separate rows are grouped by the test ID column, or follow the row that has a title. Numbered steps in one cell ("1. … 2. …") become separate steps.
• Folders: a folder column such as Checkout/Payments creates nested folders. Test cases without a folder go to the folder you pick.
• Priorities such as Critical, Major or Minor are mapped to Jira-style priorities.

You see a preview before anything is saved. Large files are imported in batches with a progress bar.

Test cycles

A test cycle is a set of test cases you run for a release. Create one on the Test cycles tab from a folder, or with Run these from the test cases currently listed. Add more later with Add test cases.

A cycle keeps a snapshot of each test case, so later edits to a test case do not change past results. For each step choose Passed, Failed or Blocked and optionally note the actual result. The test's result follows its steps: any failed step makes it failed, any blocked step makes it blocked, and all passed steps make it passed. A test without steps gets its result directly.

Bugs from failed steps

On a failed step, click Create bug. Jaylert Test creates an issue in the project (type Bug, or the project's first standard type if there is no Bug) with the steps up to the failure, the expected result and the actual result you entered. The bug key appears on the test run. The issue is created with your permissions.

Coverage

In a test case, add issue keys under Covers Jira issues. On the Jira issue, open the Test coverage panel to see the linked test cases and each one's latest result. You only see tests from projects you have access to.

AI drafts

Click Draft with AI and enter an issue key. Jaylert Test reads the issue's summary and description with your permissions and proposes up to 6 test cases with steps, starting with the main success path and then edge cases. You pick which ones to save; saved drafts get the label ai-draft and are linked to the issue.

Drafts use Atlassian-hosted models (Forge LLMs), so issue content stays on Atlassian. Each Jira site can create 200 drafts per month; the counter resets on the 1st. A draft usually takes 10 to 20 seconds.

CI results

Send the JUnit XML report of your automated tests and it becomes a test cycle named CI · … with the results filled in. Playwright, Cypress, Jest, pytest, JUnit and most runners can write this format.

1. On the Automation tab, create a token. It is shown once; store it as a CI secret. Creating a new token replaces the old one; Revoke stops uploads.
2. Name automated tests with the test case key, for example TST-T12 checkout accepts EU cards, or give them exactly the same title as the test case.
3. Post the report after your tests run:

curl -X POST "<URL from the Automation tab>?name=main-42" \
  -H "Authorization: Bearer $JAYLERT_TEST_TOKEN" \
  -H "Content-Type: application/xml" \
  --data-binary @test-results/junit.xml

Skipped tests are recorded as blocked. If one test case gets several results, a failure wins over blocked, and blocked over passed. Results that match no test case are ignored.

Reports

• Requirement coverage: for the issues matched by a JQL query (by default the project's issues that are not bugs), how many have tests, how many have all tests passing, and which have a failing test. Issues without tests are highlighted. Up to 100 issues per query.
• Test cycles: tests, run percentage, passed, failed and blocked counts, pass rate and bugs for each cycle.

Who can do what

Limits

Data and privacy

Jaylert Test runs on Atlassian Forge. Test cases, cycles and results are stored in Atlassian's Forge database for your site; the CI token is stored only as a hash. AI drafts use Atlassian-hosted models. The app sends no data to Jaylert or other outside services. Details: Privacy policy.

Troubleshooting

Still stuck? Contact support.