Important dates
Coming soon
Artifact Evaluation
SysTEX this year also, includes an optional artifact evaluation phase for accepted papers, inspired by similar efforts in the security and systems communities.
The artifact evaluation is carried out by a dedicated volunteer artifact evaluation committee (AEC). The explicit focus is on providing constructive feedback and cooperatively improving the quality and reproducibility of any code or data research artifacts. This initiative aims to add an optional "artifacts evaluated" badge on the website, linking to the corresponding open-source repository. The presence of this badge will not only acknowledge the commitment to open science by the authors but also enhance the visibility of these efforts.
Submission Instructions
Coming soon
Badges
We will aim to have badges of increasing levels, similar to ACM and USENIX:
- Artifacts Available: When the AEC could verify that
the artifacts were made available permanently and
publicly via a stable URL.
Any public hosting website is allowed, but not
a personal webpage. When using a git repository (e.g.,
GitHub), a commit hash or tag is required for a stable
URL. To earn this badge, no further requirements on
documentation, completeness, or functionality are
needed.
- Artifacts Functional: When the AEC deems the artifact to be (i) complete (i.e., all key components described in the paper are included); (ii) reasonably well-documented; and (iii) exercisable (e.g., building of the software could be verified).
- Artifacts Reusable: When the quality of the artifact significantly exceeds minimal functionality, e.g., the code can be ran and at least some results could be reproduced.
