While we wait to see whether MiniMax releases its reported 2.7-trillion-parameter successor, known internally as MiniMax M3 Pro and potentially arriving as early as the third quarter, I decided to return to MiniMax M3, released at the end of May, and run it through my HTB-Challenger BenchmarkThe HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the HTB-Challenger Benchmark page. . I know that M3 has many fans, but I had never used or tested it, so I wasn’t sure what to expect.

HTB-Challenger Benchmark

This blog post is part of a series of tests for the HTB-Challenger BenchmarkThe HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the HTB-Challenger Benchmark page. . See the benchmark results page for all results and the benchmark methodology to learn how the benchmark is calculated.

After trying many LLMs on cybersecurity tasks, I have repeatedly seen non-frontier models fall into loops when you ask them to do something beyond their capabilities.

These loops can take two forms:

  • The model makes the exact same tool call, with the same arguments, over and over again.
  • The model starts producing “never-ending” reasoning full of phrases such as “Wait, let me re-read…” and “Actually, let me think about this differently”, without making any visible progress.

A clear example of repeated identical tool calls is DeepSeek V4 Pro 0813, which ended four of my 16 benchmark runs this way.

The second failure mode was more common. GLM 5.3 hit this issue in three of 16 runs. The absolute “champion” was Hy3, which did so in nine of 16 runs.

Well, MiniMax M3 is unfortunately another model where this issue prevents it from finishing more difficult tasks. Half of the 16 challenges ended in never-ending loops, and as a result, its final score was poor.

Its $0.16 median cost per challenge was not especially high across the full benchmark, but the value was weak compared with nearby peers. GPT-5.6 Luna scored 31.6% at a median cost of $0.02, while Hy3 scored 34.8% at $0.08. Given M3’s low score and high rate of stuck runs in these tests, there is not much positive I can say about its performance in this benchmark.

HTB-Challenger Benchmark LLM model card

Cost vs. Benchmark Score

The highlighted point is this model. Models closer to the upper-left achieve a higher benchmark score at a lower median cost per challenge.

Cost vs. Benchmark Score with this model highlighted

Overall benchmark results

  • Number of challenges: 16
  • Number of solved challenges: 7
  • Number of false positives: 0
  • Runs where the model gave up: 0
  • Runs that reached the step or cost limit: 1
  • Runs where the model got stuck: 8
  • Benchmark score: 30.5%
Metric Per challenge (median) Total
Model steps 26.5 593
Model cost $0.16 $3.08
Duration 00:07:45 02:46:11
Number of input tokens 0.36M 13.14M
Number of output tokens 0.07M 1.57M
Number of read_file tool calls 0.0 24
Number of write_file tool calls 0.0 32
Number of execute_command tool calls 23.5 615
Number of web_search tool calls 0.0 5

Results by challenge difficulty

All resource-usage metrics are medians per challenge.

Metric Very Easy Easy Medium Hard
Results        
Number of challenges 4 4 4 4
Number of solved challenges 3 2 2 0
Number of false positives 0 0 0 0
Runs where the model gave up 0 0 0 0
Runs that reached the step or cost limit 0 0 0 1
Runs where the model got stuck 1 2 2 3
Benchmark score 61.7% 47.9% 49.0% 0.0%
Median per challenge        
Model steps 61 19 12 27.5
Model cost $0.29 $0.14 $0.07 $0.21
Duration 00:13:23 00:09:34 00:04:09 00:07:45
Number of input tokens 1.01M 0.15M 0.14M 0.75M
Number of output tokens 0.11M 0.11M 0.04M 0.07M
Number of read_file tool calls 2.0 2.5 0.0 0.0
Number of write_file tool calls 0.0 1.0 0.0 0.0
Number of execute_command tool calls 47.0 21.0 10.0 32.5
Number of web_search tool calls 0.0 0.0 0.0 0.0