Archives/iceraven-browser
2
0
Fork 0
mirror of https://github.com/fork-maintainers/iceraven-browser synced 2024-11-05 21:20:45 +00:00
Code Issues Releases Wiki Activity
f66c7fa872
iceraven-browser/taskcluster/docker/visual-metrics/performance-artifact-schema.json

231 lines
8.2 KiB
JSON
Raw Normal View History

[fenix] Bug 1613483 - Add all Browsertime tests with visual metrics to Fenix repo. (https://github.com/mozilla-mobile/fenix/pull/9087) * Add visual-metrics docker type. * Add required browsertime toolchain fetches. * Add browsertime tests for technical and visual metrics. * Run browsertime tests in a cron task. * Run visual metrics on all browsertime tests. * Use spaces instead of tabs, and resolve visual-metric nits. * Enable browsertime on pull request for testing. * Restrict PR tests to amazon on browsertime. * First attempt using multi_dep. * Add a primary dependency to browsertime. * Try by not popping. * Debug prints. * Make one grouping per browsertime task. * Try without the multi_dep transform. * Delete dependent-tasks in visual-metrics transformer. * Update setuptools installed and copy run-on-tasks-for. * Use get when getting run-on-tasks-for. * Add new pinned requirements. * Try it. * Set run-on-tasks-for properly. * Remove print statement. * Remove single_dep loader, and print statements. * Remove run-on-tasks-for testing setting. * Restart testing, and set user to root in visual-metrics Docker. * Remove testing settings. * Remove fetch-content from Docker. * Change attributes grouping method. * Run all tests as a check. * Undo testing changes, and fix a bad test name.
2020-03-17 17:21:41 +00:00
{
"definitions": {
"application_schema": {
"properties": {
"name": {
"title": "Application under performance test",
"enum": [
"firefox",
"chrome",
[fenix] Update perfherder data schema for visual metrics. (https://github.com/mozilla-mobile/fenix/pull/16837)
2020-12-16 17:56:48 +00:00
"chrome-m",
[fenix] Bug 1613483 - Add all Browsertime tests with visual metrics to Fenix repo. (https://github.com/mozilla-mobile/fenix/pull/9087) * Add visual-metrics docker type. * Add required browsertime toolchain fetches. * Add browsertime tests for technical and visual metrics. * Run browsertime tests in a cron task. * Run visual metrics on all browsertime tests. * Use spaces instead of tabs, and resolve visual-metric nits. * Enable browsertime on pull request for testing. * Restrict PR tests to amazon on browsertime. * First attempt using multi_dep. * Add a primary dependency to browsertime. * Try by not popping. * Debug prints. * Make one grouping per browsertime task. * Try without the multi_dep transform. * Delete dependent-tasks in visual-metrics transformer. * Update setuptools installed and copy run-on-tasks-for. * Use get when getting run-on-tasks-for. * Add new pinned requirements. * Try it. * Set run-on-tasks-for properly. * Remove print statement. * Remove single_dep loader, and print statements. * Remove run-on-tasks-for testing setting. * Restart testing, and set user to root in visual-metrics Docker. * Remove testing settings. * Remove fetch-content from Docker. * Change attributes grouping method. * Run all tests as a check. * Undo testing changes, and fix a bad test name.
2020-03-17 17:21:41 +00:00
"chromium",
"fennec",
"geckoview",
"refbrow",
"fenix"
],
"maxLength": 10,
"type": "string"
},
"version": {
"title": "Application's version",
"maxLength": 40,
"type": "string"
}
},
"required": ["name"],
"type": "object"
},
"framework_schema": {
"properties": {
"name": {
"title": "Framework name",
"type": "string"
}
},
"type": "object"
},
"subtest_schema": {
"properties": {
"name": {
"title": "Subtest name",
"type": "string"
},
"publicName": {
"title": "Public subtest name",
"description": "Allows renaming test's name, without breaking existing performance data series",
"maxLength": 30,
"type": "string"
},
"value": {
"description": "Summary value for subtest",
"title": "Subtest value",
"type": "number",
"minimum": -1000000000000.0,
"maximum": 1000000000000.0
},
"unit": {
"title": "Measurement unit",
"type": "string",
"minLength": 1,
"maxLength": 20
},
"lowerIsBetter": {
"description": "Whether lower values are better for subtest",
"title": "Lower is better",
"type": "boolean"
},
"shouldAlert": {
"description": "Whether we should alert",
"title": "Should alert",
"type": "boolean"
},
"alertThreshold": {
"description": "% change threshold before alerting",
"title": "Alert threshold",
"type": "number",
"minimum": 0.0,
"maximum": 1000.0
},
"minBackWindow": {
"description": "Minimum back window to use for alerting",
"title": "Minimum back window",
"type": "number",
"minimum": 1,
"maximum": 255
},
"maxBackWindow": {
"description": "Maximum back window to use for alerting",
"title": "Maximum back window",
"type": "number",
"minimum": 1,
"maximum": 255
},
"foreWindow": {
"description": "Fore window to use for alerting",
"title": "Fore window",
"type": "number",
"minimum": 1,
"maximum": 255
}
},
"required": [
"name",
"value"
],
"type": "object"
},
"suite_schema": {
"properties": {
"name": {
"title": "Suite name",
"type": "string"
},
"publicName": {
"title": "Public suite name",
"description": "Allows renaming suite's name, without breaking existing performance data series",
"maxLength": 30,
"type": "string"
},
"tags": {
"type": "array",
"title": "Free form tags, which ease the grouping & searching of performance tests",
"description": "Similar to extraOptions, except it does not break existing performance data series",
"items": {
"type": "string",
[fenix] Update perfherder data schema for visual metrics. (https://github.com/mozilla-mobile/fenix/pull/16837)
2020-12-16 17:56:48 +00:00
"pattern": "^[a-zA-Z0-9-]{1,24}$"
[fenix] Bug 1613483 - Add all Browsertime tests with visual metrics to Fenix repo. (https://github.com/mozilla-mobile/fenix/pull/9087) * Add visual-metrics docker type. * Add required browsertime toolchain fetches. * Add browsertime tests for technical and visual metrics. * Run browsertime tests in a cron task. * Run visual metrics on all browsertime tests. * Use spaces instead of tabs, and resolve visual-metric nits. * Enable browsertime on pull request for testing. * Restrict PR tests to amazon on browsertime. * First attempt using multi_dep. * Add a primary dependency to browsertime. * Try by not popping. * Debug prints. * Make one grouping per browsertime task. * Try without the multi_dep transform. * Delete dependent-tasks in visual-metrics transformer. * Update setuptools installed and copy run-on-tasks-for. * Use get when getting run-on-tasks-for. * Add new pinned requirements. * Try it. * Set run-on-tasks-for properly. * Remove print statement. * Remove single_dep loader, and print statements. * Remove run-on-tasks-for testing setting. * Restart testing, and set user to root in visual-metrics Docker. * Remove testing settings. * Remove fetch-content from Docker. * Change attributes grouping method. * Run all tests as a check. * Undo testing changes, and fix a bad test name.
2020-03-17 17:21:41 +00:00
},
"uniqueItems": true,
"maxItems": 14
},
"extraOptions": {
"type": "array",
"title": "Extra options used in running suite",
"items": {
"type": "string",
"maxLength": 100
},
"uniqueItems": true,
"maxItems": 8
},
"subtests": {
"items": {
"$ref": "#/definitions/subtest_schema"
},
"title": "Subtests",
"type": "array"
},
"value": {
"title": "Suite value",
"type": "number",
"minimum": -1000000000000.0,
"maximum": 1000000000000.0
},
"unit": {
"title": "Measurement unit",
"type": "string",
"minLength": 1,
"maxLength": 20
},
"lowerIsBetter": {
"description": "Whether lower values are better for suite",
"title": "Lower is better",
"type": "boolean"
},
"shouldAlert": {
"description": "Whether we should alert on this suite (overrides default behaviour)",
"title": "Should alert",
"type": "boolean"
},
"alertThreshold": {
"description": "% change threshold before alerting",
"title": "Alert threshold",
"type": "number",
"minimum": 0.0,
"maximum": 1000.0
},
"minBackWindow": {
"description": "Minimum back window to use for alerting",
"title": "Minimum back window",
"type": "integer",
"minimum": 1,
"maximum": 255
},
"maxBackWindow": {
"description": "Maximum back window to use for alerting",
"title": "Maximum back window",
"type": "integer",
"minimum": 1,
"maximum": 255
},
"foreWindow": {
"description": "Fore window to use for alerting",
"title": "Fore window",
"type": "integer",
"minimum": 1,
"maximum": 255
}
},
"required": [
"name",
"subtests"
],
"type": "object"
}
},
"description": "Structure for submitting performance data as part of a job",
"id": "https://treeherder.mozilla.org/schemas/v1/performance-artifact.json#",
"properties": {
"application":{
"$ref": "#/definitions/application_schema"
},
"framework": {
"$ref": "#/definitions/framework_schema"
},
"suites": {
"description": "List of suite-level data submitted as part of this structure",
"items": {
"$ref": "#/definitions/suite_schema"
},
"title": "Performance suites",
"type": "array"
}
},
"required": [
"framework",
"suites"
],
"title": "Perfherder Schema",
"type": "object"
}
Reference in New Issue Copy Permalink