Set up and use Jaylert Test

Install

Before you start

• Jira Cloud, any plan. Jira Data Center and Server are not supported.
• A Jira administrator installs the app for the whole site.
• People who view tests need the Browse projects permission; people who write and run tests need Edit issues in the project.

Install from the Atlassian Marketplace

1. In Jira, go to Settings (⚙) → Apps → Explore more apps, and search for Jaylert Test. You can also start from the app's Atlassian Marketplace listing.
2. Click Try it free. The first 30 days are free, and sites with up to 10 users never pay.
3. Review the permissions and confirm. Jaylert Test asks to:
   • read issues and projects, to link tests, show coverage and run the coverage report's JQL;
   • create issues, only when someone clicks Create bug on a failed step;
   • store its own data in Atlassian's Forge storage;
   • use Atlassian-hosted AI models for Draft with AI. Jira shows a notice about this during installation.
   Everything people do in the app runs with their own Jira permissions; CI uploads are accepted only with the project's token. The app sends no data outside Atlassian.

Where to find it

• In every project: Tests in the project navigation. If the navigation is full, look under More.
• On an issue: click the apps button under the issue title (View app actions, a hexagon icon) and choose Test coverage. The panel then stays on that issue.

First visit: Allow access

The first time each person opens Tests, Jira shows Allow access. Jaylert Test reads Jira with that person's own permissions (for example to check they may edit tests, or to create a bug in their name), and Jira asks every user once to allow this. Click Allow access, then Accept. Nothing is shared outside Atlassian.

Updates

Updates install automatically. If an update ever needs a new permission, Jira asks an administrator to approve it under Apps → Manage apps; until then the previous version keeps working.

Quick start

1. Open a Jira project and click Tests in the project navigation.
2. Create a folder (＋ next to Folders) and a test case (New test case), or bring your existing tests in with Import.
3. Click Run these to create a test cycle from the test cases shown, then run each test step by step.
4. Link test cases to the Jira issues they verify, so coverage shows up on the issues and in Reports.

Test cases

A test case has a title, a folder, a priority (Highest to Lowest), labels, a precondition and steps. Each step has an action, optional test data and an expected result. Reorder steps with the arrows; empty steps are dropped when you save.

Test cases get a key per project, for example TST-T12. Folders can be nested; selecting a folder lists its test cases together with those in its sub-folders, and a test cycle created from a folder includes them too. Only empty folders can be deleted. Use the search box to filter by title.

Import

Click Import at the top of the folder list. Choose a CSV file, or copy the cells in Excel or Google Sheets, including the header row, and paste them.

• Columns are recognised automatically from common headers used by Zephyr, TestRail and Xray exports (for example Issue Key, Test Summary, Step, Test Data, Expected Result). You can change every mapping.
• Steps on separate rows are grouped by the test ID column, or follow the row that has a title. Numbered steps in one cell ("1. … 2. …") become separate steps.
• Folders: a folder column such as Checkout/Payments creates nested folders. Test cases without a folder go to the folder you pick.
• Priorities such as Critical, Major or Minor are mapped to Jira-style priorities.

You see a preview before anything is saved. Large files are imported in batches with a progress bar.

Test cycles

A test cycle is a set of test cases you run for a release. Create one on the Test cycles tab from a folder, or with Run these from the test cases currently listed. Add more later with Add test cases.

A cycle keeps a snapshot of each test case, so later edits to a test case do not change past results. For each step choose Passed, Failed or Blocked and optionally note the actual result. The test's result follows its steps: any failed step makes it failed, any blocked step makes it blocked, and all passed steps make it passed. A test without steps gets its result directly.

Bugs from failed steps

On a failed step, click Create bug. Jaylert Test creates an issue in the project (type Bug, or the project's first standard type if there is no Bug) with the steps up to the failure, the expected result and the actual result you entered. The bug key appears on the test run. The issue is created with your permissions.

Coverage

In a test case, add issue keys under Covers Jira issues. On the Jira issue, open the Test coverage panel (the apps button under the issue title) to see the linked test cases and each one's latest result. You only see tests from projects you have access to.

AI drafts

Click Draft with AI and enter an issue key. Jaylert Test reads the issue's summary and description with your permissions and proposes up to 6 test cases with steps, starting with the main success path and then edge cases. You pick which ones to save; saved drafts get the label ai-draft and are linked to the issue.

Drafts use Atlassian-hosted models (Forge LLMs), so issue content stays on Atlassian. Each Jira site can create 200 drafts per month; the counter resets on the 1st. A draft usually takes 10 to 30 seconds and runs in the background, so it also finishes on slow connections. Proposed drafts are kept for one day at most, for the person who asked for them, until they are saved or discarded.

CI results

Send the JUnit XML report of your automated tests and it becomes a test cycle named CI · … with the results filled in. Playwright, Cypress, Jest, pytest, JUnit and most runners can write this format.

1. On the Automation tab, create a token. It is shown once; store it as a CI secret. Creating a new token replaces the old one; Revoke stops uploads.
2. Name automated tests with the test case key, for example TST-T12 checkout accepts EU cards, or give them exactly the same title as the test case.
3. Post the report after your tests run:

curl -X POST "<URL from the Automation tab>?name=main-42" \
  -H "Authorization: Bearer $JAYLERT_TEST_TOKEN" \
  -H "Content-Type: application/xml" \
  --data-binary @test-results/junit.xml

Skipped tests are recorded as blocked. If one test case gets several results, a failure wins over blocked, and blocked over passed. Results that match no test case are ignored. Only test cases that got a result are added to the CI cycle.

Reports

• Requirement coverage: for the issues matched by a JQL query (by default the project's issues that are not bugs), how many have tests, how many have all tests passing, and which have a failing test. Issues without tests are highlighted. Up to 100 issues per query. A query with an error (for example an unknown status) shows Jira's error message instead of an empty report.
• Test cycles: tests, run percentage, passed, failed and blocked counts, pass rate and bugs for each cycle.

Who can do what

Limits

Uninstalling

Remove Jaylert Test under Settings (⚙) → Apps → Manage apps. Test cases, cycles, results and the CI token are stored in Atlassian's Forge storage for your site and are deleted by Atlassian after the app is uninstalled. Bugs created from failed steps are normal Jira issues and stay in Jira.

Data and privacy

Jaylert Test runs on Atlassian Forge. Test cases, cycles and results are stored in Atlassian's Forge database for your site; the CI token is stored only as a hash. AI drafts use Atlassian-hosted models. The app sends no data to Jaylert or other outside services. Details: Privacy policy.

Troubleshooting

Still stuck? Contact support.