add swe_verified, swebench_pro, mcp_atlas agent evaluations.

examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseekv4/README.md

@@ -110,3 +110,83 @@ bash run_quant.sh \
   - `config.model_free=True`
   - `model` passed to `prepare/convert` is a `str` (model path or model name)
 - The example uses `reloading=False` by default and saves quantized artifacts to `--output_model`.
+
+
+---
+
+## Agent Evaluation (run_agent.sh)
+
+Run SWE-bench Pro, SWE-bench Verified, or MCP-Atlas against the quantized model
+via a local vLLM server.
+
+### One-time setup
+
+```bash
+# From within an activated environment (conda/venv):
+BENCHMARK_DIR=$PWD bash setup_agent.sh [swebp|swe-verified|mcp-atlas]
+```
+
+Place the SWE-bench Pro Docker image patch at `patches/swebench_pro_image.patch`
+before running `setup_agent.sh swebp`.
+
+### Run
+
+```bash
+# SWE-bench Pro — smoke test (10 instances, TP=2)
+CUDA_VISIBLE_DEVICES=0,1 TENSOR_PARALLEL_SIZE=2 \
+bash run_agent.sh \
+  --task      swebp \
+  --model     /path/to/DeepSeek-V4-Flash \
+  --port      8888 \
+  --num-tasks 10 \  # only run 10 cases.
+  --workers   2 \
+  --tag       smoke_swebp
+
+# SWE-bench Verified — full run (vLLM already running)
+SWEBENCH_API_KEY=<your_key> \
+bash run_agent.sh \
+  --task      swe-verified \
+  --port      8888 \
+  --skip-serve \
+  --workers   4 \
+  --tag       run_verified_01
+
+# MCP-Atlas — smoke test
+CUDA_VISIBLE_DEVICES=0,1 TENSOR_PARALLEL_SIZE=2 \
+bash run_agent.sh \
+  --task      mcp-atlas \
+  --model     /path/to/DeepSeek-V4-Flash \
+  --port      8889 \
+  --workers   5 \
+  --tag       smoke_mcp
+```
+If the model basename is exactly `DeepSeek-V4-Flash` or `DeepSeek-V4-Pro`,
+`--enable-expert-parallel --moe-backend deep_gemm_mega_moe` are added automatically.
+
+### Arguments
+
+`run_agent.sh` common arguments:
+
+- `--task`: `swebp` | `swe-verified` | `mcp-atlas`
+- `--model PATH`: model path; launches vLLM automatically when set
+- `--port N`: vLLM API port (default: `8888`)
+- `--served-name NAME`: vLLM served-model-name (default: `gpt-3.5-turbo`)
+- `--max-model-len N`: vLLM max_model_len (default: `262144`)
+- `--tag NAME`: label for output directory and logs
+- `--skip-serve`: skip vLLM launch (use already-running server)
+
+SWE-bench Pro / Verified:
+
+- `--workers N`: parallel agent workers (default: `2`)
+- `--step-limit N`: max steps per instance (default: `250`)
+- `--num-tasks N`: limit to first N instances
+- `--slice S:E`: exact slice, e.g. `0:20`
+- `--redo`: re-run already-completed instances (swebp only)
+
+MCP-Atlas:
+
+- `--workers N`: parallel eval concurrency (default: `5`)
+- `--num-tasks N`: limit to first N tasks
+- `--sandbox-port P`: MCP sandbox Docker port (default: `1984`)
+- `--harness-port P`: agent harness port (default: `3001`)
+- `--skip-sandbox`: skip Docker sandbox startup

examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseekv4/patches/swebench_pro_image.patch

@@ -0,0 +1,23 @@
+diff --git a/src/minisweagent/run/extra/swebench.py b/src/minisweagent/run/extra/swebench.py
+index b252fed..981a6c9 100644
+--- a/src/minisweagent/run/extra/swebench.py
++++ b/src/minisweagent/run/extra/swebench.py
+@@ -70,10 +70,14 @@ def get_swebench_docker_image_name(instance: dict) -> str:
+     """Get the image name for a SWEBench instance."""
+     image_name = instance.get("image_name", None)
+     if image_name is None:
+-        # Docker doesn't allow double underscore, so we replace them with a magic token
+-        iid = instance["instance_id"]
+-        id_docker_compatible = iid.replace("__", "_1776_")
+-        image_name = f"docker.io/swebench/sweb.eval.x86_64.{id_docker_compatible}:latest".lower()
++        # SWE-bench Pro: use jefzda/sweap-images via dockerhub_tag
++        if "dockerhub_tag" in instance:
++            image_name = f"jefzda/sweap-images:{instance['dockerhub_tag']}"
++        else:
++            # Docker doesn't allow double underscore, so we replace them with a magic token
++            iid = instance["instance_id"]
++            id_docker_compatible = iid.replace("__", "_1776_")
++            image_name = f"docker.io/swebench/sweb.eval.x86_64.{id_docker_compatible}:latest".lower()
+     return image_name
+
+