The same tasks for different models — quality and speed in one table.
AI evaluation & model selection
Repository on GitHub (GitHub)An automated bench for evaluating language models on real working scenarios. It helps choose a local LLM not by the impression of a few answers but by test results with common criteria.
You add tasks and answer requirements in Google Sheets. The system sends requests to the model under test, saves the answers and measures performance. Then GPT-4.1 acts as an AI evaluator: it checks the result against the given criteria, gives a score and explains the grade.
By repeating the set of tests for different models, the team gets a basis for comparison: who follows instructions better, produces correct JSON or solves coding tasks — and how fast. Answers, metrics and grades stay in a shared table for further analysis.
Tests for your own processes. From customer support and text preparation to QA, programming and logic tasks.
Evaluation by common criteria. Following instructions, completeness of the answer, data structure and signs of invented facts.
A clear check result. A five-point score with an explanation.
Performance measurement. Response time, generation speed and token count.
Run history. Model, task, original answer, metrics and grade are saved together.
Repeat checks. One set of scenarios can be reused when the model or settings change, to track changes in quality.
Test automation & QA · AI evaluation & model selection
Check answer quality and resistance to provocative requests — without running conversations by hand.
Automatic checks of AI chatbots in Telegram: functional tests, AI evaluation of answers and security checks.
Process automation & integrations · Knowledge bases & RAG
Work with the OpenAI API in your browser — with full control over the request and a clear view of the response.
Work with OpenAI models without a command line or code: see exactly what was sent to the API and what came back.
Process automation & integrations · Test automation & QA
The team works in Jira — managers get a clear summary without manual data transfer.
An automatic bug summary for QA, managers and other teams instead of shortening tickets by hand.
Result Manual data transfer when preparing the summary has been eliminated
Describe what is currently done manually, what data is involved and what result you would like to achieve. You do not need a technical specification or a chosen model to begin.
Please do not share trade secrets, source code, or sensitive personal data before an NDA (non-disclosure agreement) is signed.
