Evaluate a substantial collection of unpublished terminal and repository-level software-engineering tasks for model post-training, reinforcement learning and private evaluation programs.
Private task inventory built to the exact formats used by the industry's best-known coding-agent benchmarks.
Deaimer's private task collection built to the Terminal-Bench task and environment format — difficult, multi-step terminal workflows with objectively verifiable outcomes, covering system administration, DevOps, debugging, security and data processing.
A private task collection built to the Terminal-Bench Science format — terminal-driven scientific-computing and research workflows, from data analysis pipelines to computational-methods tasks with objectively verifiable outcomes.
A private task collection built to the SWE-bench repository format — repository-level software-engineering problems requiring codebase understanding, multi-file changes, and test-driven resolution.
Deaimer is an independent contributor building task collections compatible with these benchmark formats. Deaimer does not own, operate, sponsor or hold an official partnership with Terminal-Bench or SWE-bench, and does not display their marks without permission.
Need a task set that doesn't exist yet? See Custom RL Environments. Need to validate model performance against a benchmark? See Benchmarks & Research.
Available package contents vary by collection and can include:
Task prompts and setup instructions provided to the agent.
Structured labels for category, difficulty, and capability area.
Environment definitions used to run each task reproducibly.
Underlying repository state for repository-level tasks.
Supporting data and files required to run a task.
Scoring logic used to verify task completion.
Test coverage associated with repository-level tasks.
A working solution used to validate solvability.
Recorded results from a known-correct solution run.
Recorded results from a known-failing baseline run.
Documentation from the independent review process.
Trial run data and agent trajectories captured during validation.
Classification of each task by difficulty and capability area.
Not every item is included in every collection — availability is confirmed during the sample review.
Fine-tune coding agents against high-difficulty terminal and repository tasks.
Use verified task and solution pairs as supervised training data.
Verifiable-outcome tasks suited to RLVR and RL training pipelines.
Run confidential evaluations that avoid public benchmark contamination.
Identify where a model's coding-agent capabilities fall short.
Develop and validate graders and reward models against varied tasks.
Probe agent behavior against difficult, adversarial-adjacent tasks.
Extend an existing benchmark's coverage with additional private tasks.
Study how and where agents fail across a broad task distribution.
Use the dataset while Deaimer retains the ability to license it to additional customers.
Negotiate exclusive rights for a defined collection, capability area, language, format, market, or period.
Acquire an agreed dataset package and associated rights under a negotiated asset-purchase or data-licensing agreement.
Final scope, permitted uses, exclusivity, warranties, delivery formats and intellectual-property rights are defined contractually.
Tell us about your intended use and the collection you're interested in.
A mutual NDA is put in place before any detailed material is shared.
Review a catalog and metadata summary describing available tasks.
Review a small set of controlled sample tasks from the collection.
Agree on technical diligence and dataset-overlap checks with your team.
Choose a licensing or acquisition structure that fits your use case.
Receive the versioned delivery package under the agreed terms.
Request a structured catalog and a small set of controlled samples under your preferred confidentiality process.