Finetuned Bert Base 10pct

Developed by cynthiachan

A BERT-base-cased fine-tuned entity recognition model, specialized in identifying various entity types related to cybersecurity

Sequence Labeling

Transformers

Open Source License:Apache-2.0 #Threat Intelligence Entity Recognition #High-Precision File Path Detection #Cybersecurity Entity Extraction

Downloads 97

Release Time : 7/15/2022

Model Overview

This model is a fine-tuned entity recognition model based on BERT-base-cased, using the FeedRef_10pct dataset. It is primarily used to identify various entities in cybersecurity texts, such as attack IDs, CVE numbers, domains, etc.

Model Features

Multi-Type Entity Recognition

Capable of identifying various entity types in the cybersecurity field, including attack IDs, CVE numbers, file paths, etc.

High-Precision Recognition

Achieves 100% F1 score on key entities like AttackID, with an overall F1 score of 0.7583

BERT-Based Architecture

Built on the mature BERT-base-cased architecture, offering strong transfer learning capabilities

Model Capabilities

Cybersecurity text analysis

Multi-type entity recognition

Text classification

Information extraction

Use Cases

Cybersecurity Analysis

Threat Intelligence Report Analysis

Extract key entity information from security reports

Accurately identifies critical information such as attack IDs and CVE numbers

Log Analysis

Analyze suspicious activities in server logs

Identifies entities like malicious file paths and IP addresses

Security Research

Vulnerability Research

Extract vulnerability-related information from technical documents

Accurately identifies CVE numbers and related technical details

license: apache-2.0 tags:

generated_from_trainer datasets:
cynthiachan/FeedRef_10pct model-index:
name: training results: []

training

This model is a fine-tuned version of bert-base-cased on the cynthiachan/FeedRef_10pct dataset. It achieves the following results on the evaluation set:

Loss: 0.1291
Attackid Precision: 1.0
Attackid Recall: 1.0
Attackid F1: 1.0
Attackid Number: 6
Cve Precision: 0.8333
Cve Recall: 0.9091
Cve F1: 0.8696
Cve Number: 11
Defenderthreat Precision: 0.0
Defenderthreat Recall: 0.0
Defenderthreat F1: 0.0
Defenderthreat Number: 2
Domain Precision: 0.7826
Domain Recall: 0.7826
Domain F1: 0.7826
Domain Number: 23
Email Precision: 0.6667
Email Recall: 0.6667
Email F1: 0.6667
Email Number: 3
Filepath Precision: 0.6766
Filepath Recall: 0.8242
Filepath F1: 0.7432
Filepath Number: 165
Hostname Precision: 1.0
Hostname Recall: 0.9167
Hostname F1: 0.9565
Hostname Number: 12
Ipv4 Precision: 0.8333
Ipv4 Recall: 0.8333
Ipv4 F1: 0.8333
Ipv4 Number: 12
Md5 Precision: 0.7246
Md5 Recall: 0.9615
Md5 F1: 0.8264
Md5 Number: 52
Sha1 Precision: 0.0667
Sha1 Recall: 0.1429
Sha1 F1: 0.0909
Sha1 Number: 7
Sha256 Precision: 0.6780
Sha256 Recall: 0.9091
Sha256 F1: 0.7767
Sha256 Number: 44
Uri Precision: 0.0
Uri Recall: 0.0
Uri F1: 0.0
Uri Number: 1
Overall Precision: 0.6910
Overall Recall: 0.8402
Overall F1: 0.7583
Overall Accuracy: 0.9725

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Attackid Precision	Attackid Recall	Attackid F1	Attackid Number	Cve Precision	Cve Recall	Cve F1	Cve Number	Defenderthreat Number	Domain Precision	Domain Recall	Domain F1	Domain Number	Email Precision	Email Recall	Email F1	Email Number	Filepath Precision	Filepath Recall	Filepath F1	Filepath Number	Hostname Precision	Hostname Recall	Hostname F1	Hostname Number	Ipv4 Precision	Ipv4 Recall	Ipv4 F1	Ipv4 Number	Md5 Precision	Md5 Recall	Md5 F1	Md5 Number	Sha1 Precision	Sha1 Recall	Sha1 F1	Sha1 Number	Sha256 Precision	Sha256 Recall	Sha256 F1	Sha256 Number	Uri Number	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.3943	0.37	500	0.2881	0.0	0.0	0.0	6	0.0	0.0	0.0	11	2	0.0	0.0	0.0	23	0.0	0.0	0.0	3	0.1138	0.2	0.1451	165	0.0692	0.9167	0.1287	12	0.4706	0.6667	0.5517	12	0.75	0.9231	0.8276	52	0.0	0.0	0.0	7	0.5694	0.9318	0.7069	44	1	0.2342	0.4172	0.3	0.9360
0.1987	0.75	1000	0.1722	0.5	1.0	0.6667	6	1.0	1.0	1.0	11	2	0.0	0.0	0.0	23	0.0	0.0	0.0	3	0.4779	0.6545	0.5524	165	0.25	0.6667	0.3636	12	0.6923	0.75	0.7200	12	0.6364	0.9423	0.7597	52	0.0	0.0	0.0	7	0.6545	0.8182	0.7273	44	1	0.5136	0.6716	0.5821	0.9529
0.1595	1.12	1500	0.1346	0.8571	1.0	0.9231	6	1.0	1.0	1.0	11	2	0.4286	0.5217	0.4706	23	0.0	0.0	0.0	3	0.5797	0.7273	0.6452	165	0.44	0.9167	0.5946	12	0.3929	0.9167	0.55	12	0.6364	0.9423	0.7597	52	0.0	0.0	0.0	7	0.78	0.8864	0.8298	44	1	0.5768	0.7663	0.6582	0.9658
0.118	1.5	2000	0.1436	1.0	1.0	1.0	6	1.0	1.0	1.0	11	2	0.6087	0.6087	0.6087	23	0.0	0.0	0.0	3	0.6101	0.8061	0.6945	165	0.9091	0.8333	0.8696	12	0.7273	0.6667	0.6957	12	0.7869	0.9231	0.8496	52	0.2143	0.4286	0.2857	7	0.7407	0.9091	0.8163	44	1	0.6675	0.8077	0.7309	0.9686
0.1198	1.87	2500	0.1385	1.0	1.0	1.0	6	0.7692	0.9091	0.8333	11	2	0.85	0.7391	0.7907	23	0.0	0.0	0.0	3	0.6390	0.7939	0.7081	165	1.0	0.8333	0.9091	12	0.5333	0.6667	0.5926	12	0.7778	0.9423	0.8522	52	0.3333	0.5714	0.4211	7	0.8571	0.9545	0.9032	44	1	0.6995	0.8195	0.7548	0.9687
0.0742	2.25	3000	0.1291	1.0	1.0	1.0	6	0.8333	0.9091	0.8696	11	2	0.7826	0.7826	0.7826	23	0.6667	0.6667	0.6667	3	0.6766	0.8242	0.7432	165	1.0	0.9167	0.9565	12	0.8333	0.8333	0.8333	12	0.7246	0.9615	0.8264	52	0.0667	0.1429	0.0909	7	0.6780	0.9091	0.7767	44	1	0.6910	0.8402	0.7583	0.9725
0.0687	2.62	3500	0.1385	1.0	1.0	1.0	6	1.0	1.0	1.0	11	2	0.8077	0.9130	0.8571	23	1.0	1.0	1.0	3	0.7746	0.8121	0.7929	165	0.7333	0.9167	0.8148	12	0.7143	0.8333	0.7692	12	0.96	0.9231	0.9412	52	0.4444	0.5714	0.5	7	0.8113	0.9773	0.8866	44	1	0.8083	0.8609	0.8338	0.9737
0.0652	3.0	4000	0.1299	1.0	1.0	1.0	6	1.0	1.0	1.0	11	2	0.8077	0.9130	0.8571	23	1.0	1.0	1.0	3	0.7553	0.8606	0.8045	165	0.8462	0.9167	0.8800	12	0.7143	0.8333	0.7692	12	0.8571	0.9231	0.8889	52	0.75	0.8571	0.8000	7	0.8723	0.9318	0.9011	44	1	0.8038	0.8846	0.8423	0.9772

Framework versions

Transformers 4.21.2
Pytorch 1.12.1+cu102
Datasets 2.4.0
Tokenizers 0.12.1

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご