462 9 347

Zhimin Zhao PRO

zhiminy

https://zhimin-z.github.io

AI & ML interests

SE4AI, AI4SE, LLMOps, LLM4Code

Recent Activity

updated a dataset about 11 hours ago

SWE-Arena/leaderboard_data

liked a Space about 14 hours ago

microsoft/MageBench-Leaderboard

updated a Space 5 days ago

SWE-Arena/SWE-Model-Arena

View all activity

Organizations

Posts 5

Post

2006

Hey everyone,

We're thrilled to introduce our latest project: a hand-curated list of the best production-level generative-ai open-source toolkits! 🚀

Check it out here: https://huggingface.co/spaces/zhiminy/awesome-production-genai-search

Post

1957

# 🚀 SWE-Arena: Evaluating Foundation Models for Software Engineering

SWE-Arena is the first open-source platform for evaluating foundation models in real-world software engineering workflows.

## What makes it unique?

- RepoChat: Automatically injects repository context (issues, commits, PRs) into conversations for more realistic evaluations
- Multi-round interactions: Tests models through iterative workflows, not just single prompts
- Novel metrics: Includes a "model consistency score" that measures model determinism through self-play matches and "conversation efficiency index" that evaluates model performance while accounting for the number of interaction rounds required to reach conclusions.

Try it now: https://huggingface.co/spaces/SWE-Arena/Software-Engineering-Arena

## Why it matters

Traditional evaluation frameworks don't capture how developers actually use models in their daily work. SWE-Arena creates a testing environment that mirrors real engineering workflows, helping you choose the right model for your specific software development needs.

From debugging to requirement refinement, see which models truly excel at software engineering tasks!

View all Posts