Submit

We invite you to submit your model-generated results via email to causalai@pjlab.org.cn. Submissions will be acknowledged within three business days. Contributions to our benchmark, including models, datasets, prompts, metrics, and other resources, are highly encouraged. For additional details, please refer to our GitHub repository. If you wish to include your results in our leaderboard, please inform us by email.


Submission Guidelines

Submission Limit

Participants may submit results for up to three records per week. At the moment, submissions should be sent via email. A dedicated submission platform is under development. For submissions exceeding the weekly limit, please contact us directly to discuss further arrangements.

Submission Format

Before submitting, please compress your files into zip, rar or tar format. Submissions should consist solely of a 'responses.json' files. Ensure that all extraneous files are removed before submission. The structure of your submission folder should appear as follows:

  • output
    • causal_discovery
      • abstract_reasoning
        • AR-B_(CaLM-AR)_CN
          • vicuna_33b
            • zero-shot-IcL
              • responses.json
            • basic
              • responses.json
          • ...
        • ...
      • ...
    • ...