Open-source Segformer-b5 Human Body Part Segmentation Model - Accurately Divide Clothing and Body Parts

Segformer B5 Finetuned Human Parsing

Developed by krnl

A human body part segmentation model fine-tuned based on MIT-B5 architecture, excelling in detailed partitioning of clothing and body parts

Image Segmentation

Transformers

Open Source License:Other #Human body part segmentation #High-precision apparel recognition #Multi-category semantic segmentation

Downloads 55

Release Time : 1/17/2024

Model Overview

This model is a variant of the SegFormer architecture, specifically designed for semantic segmentation tasks of human body parts and clothing. It can accurately identify 20 categories of human-related elements including hats, tops, pants, etc., suitable for virtual fitting, behavior analysis, and other scenarios.

Model Features

Detailed part segmentation

Supports precise segmentation of 20 categories of human body parts and clothing, including small items like glasses and scarves

High-precision performance

Achieves over 90% accuracy in key areas such as face and hair

Lightweight fine-tuning

Efficient fine-tuning based on pre-trained models to adapt to specific human parsing tasks

Model Capabilities

Human body part recognition

Clothing classification

Pixel-level segmentation

Multi-category labeling

Use Cases

Fashion technology

Virtual fitting system

Accurately segments clothing areas to achieve virtual try-on effects

Top/pants segmentation accuracy exceeds 85%

Apparel e-commerce search

Improves search precision through clothing feature analysis

Human-computer interaction

Motion capture preprocessing

Provides precise body part localization for action recognition

🚀 segformer-b5-finetuned-human-parsing

This model is a fine - tuned version of nvidia/mit-b5 on the None dataset. It specializes in image segmentation for human parsing, delivering high - quality results in identifying different parts of the human body within an image.

🚀 Quick Start

This model is ready to be used for human parsing tasks. You can load it and start making predictions right away.

📚 Documentation

Model Evaluation Results

It achieves the following results on the evaluation set:

Loss: 0.2292
Mean Iou: 0.6258
Mean Accuracy: 0.7547
Overall Accuracy: 0.8256
Accuracy Background: nan
Accuracy Hat: 0.8561
Accuracy Hair: 0.8974
Accuracy Sunglasses: 0.7540
Accuracy Upper - clothes: 0.8553
Accuracy Skirt: 0.7026
Accuracy Pants: 0.8913
Accuracy Dress: 0.7525
Accuracy Belt: 0.4251
Accuracy Left - shoe: 0.6014
Accuracy Right - shoe: 0.6374
Accuracy Face: 0.9094
Accuracy Left - leg: 0.8452
Accuracy Right - leg: 0.8343
Accuracy Left - arm: 0.8506
Accuracy Right - arm: 0.8287
Accuracy Bag: 0.8232
Accuracy Scarf: 0.3662
Iou Background: 0.0
Iou Hat: 0.7625
Iou Hair: 0.8171
Iou Sunglasses: 0.6400
Iou Upper - clothes: 0.7700
Iou Skirt: 0.6211
Iou Pants: 0.7788
Iou Dress: 0.5512
Iou Belt: 0.3564
Iou Left - shoe: 0.5032
Iou Right - shoe: 0.5381
Iou Face: 0.8294
Iou Left - leg: 0.7412
Iou Right - leg: 0.7591
Iou Left - arm: 0.7579
Iou Right - arm: 0.7705
Iou Bag: 0.7729
Iou Scarf: 0.2956

Training Procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 6e - 05
train_batch_size: 2
eval_batch_size: 2
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
num_epochs: 15

Training results

Property	Details
Training Loss	The loss value during the training process, showing the model's error level at different steps.
Epoch	The number of complete passes through the entire training dataset.
Step	The number of training steps completed.
Validation Loss	The loss value on the validation set, used to evaluate the model's generalization ability.
Mean Iou	Mean Intersection over Union, a common metric for evaluating segmentation models.
Mean Accuracy	The average accuracy of the model's predictions.
Overall Accuracy	The overall accuracy of the model on all classes.
Accuracy Background	The accuracy of predicting the background class.
Accuracy Hat	The accuracy of predicting the hat class.
Accuracy Hair	The accuracy of predicting the hair class.
Accuracy Sunglasses	The accuracy of predicting the sunglasses class.
Accuracy Upper - clothes	The accuracy of predicting the upper - clothes class.
Accuracy Skirt	The accuracy of predicting the skirt class.
Accuracy Pants	The accuracy of predicting the pants class.
Accuracy Dress	The accuracy of predicting the dress class.
Accuracy Belt	The accuracy of predicting the belt class.
Accuracy Left - shoe	The accuracy of predicting the left - shoe class.
Accuracy Right - shoe	The accuracy of predicting the right - shoe class.
Accuracy Face	The accuracy of predicting the face class.
Accuracy Left - leg	The accuracy of predicting the left - leg class.
Accuracy Right - leg	The accuracy of predicting the right - leg class.
Accuracy Left - arm	The accuracy of predicting the left - arm class.
Accuracy Right - arm	The accuracy of predicting the right - arm class.
Accuracy Bag	The accuracy of predicting the bag class.
Accuracy Scarf	The accuracy of predicting the scarf class.
Iou Background	Intersection over Union for the background class.
Iou Hat	Intersection over Union for the hat class.
Iou Hair	Intersection over Union for the hair class.
Iou Sunglasses	Intersection over Union for the sunglasses class.
Iou Upper - clothes	Intersection over Union for the upper - clothes class.
Iou Skirt	Intersection over Union for the skirt class.
Iou Pants	Intersection over Union for the pants class.
Iou Dress	Intersection over Union for the dress class.
Iou Belt	Intersection over Union for the belt class.
Iou Left - shoe	Intersection over Union for the left - shoe class.
Iou Right - shoe	Intersection over Union for the right - shoe class.
Iou Face	Intersection over Union for the face class.
Iou Left - leg	Intersection over Union for the left - leg class.
Iou Right - leg	Intersection over Union for the right - leg class.
Iou Left - arm	Intersection over Union for the left - arm class.
Iou Right - arm	Intersection over Union for the right - arm class.
Iou Bag	Intersection over Union for the bag class.
Iou Scarf	Intersection over Union for the scarf class.

Training Loss	Epoch	Step	Validation Loss	Mean Iou	Mean Accuracy	Overall Accuracy	Accuracy Background	Accuracy Hat	Accuracy Hair	Accuracy Sunglasses	Accuracy Upper - clothes	Accuracy Skirt	Accuracy Pants	Accuracy Dress	Accuracy Belt	Accuracy Left - shoe	Accuracy Right - shoe	Accuracy Face	Accuracy Left - leg	Accuracy Right - leg	Accuracy Left - arm	Accuracy Right - arm	Accuracy Bag	Accuracy Scarf	Iou Background	Iou Hat	Iou Hair	Iou Sunglasses	Iou Upper - clothes	Iou Skirt	Iou Pants	Iou Dress	Iou Belt	Iou Left - shoe	Iou Right - shoe	Iou Face	Iou Left - leg	Iou Right - leg	Iou Left - arm	Iou Right - arm	Iou Bag	Iou Scarf
1.1597	0.04	20	1.5815	0.1179	0.1991	0.4296	nan	0.0060	0.6905	0.0	0.7657	0.0108	0.6431	0.2946	0.0	0.0288	0.0366	0.1480	0.0025	0.5692	0.0096	0.0259	0.1537	0.0	0.0	0.0051	0.4253	0.0	0.5199	0.0103	0.3388	0.1700	0.0	0.0258	0.0338	0.0895	0.0025	0.3162	0.0094	0.0253	0.1495	0.0
0.6963	0.08	40	0.8073	0.1759	0.2719	0.4628	nan	0.0015	0.8699	0.0	0.4736	0.4932	0.5141	0.2946	0.0	0.0062	0.1038	0.5301	0.0916	0.5692	0.0096	0.0259	0.2889	0.0	0.0	0.0015	0.6169	0.0	0.4242	0.2202	0.3522	0.2251	0.0	0.0062	0.0904	0.4914	0.0852	0.3160	0.0092	0.0541	0.2731	0.0
0.5786	0.12	60	0.6136	0.2538	0.3642	0.4679	nan	0.0180	0.8122	0.0	0.1998	0.0000	0.6621	0.8592	0.0	0.1440	0.2772	0.8381	0.4032	0.6068	0.4182	0.3097	0.6434	0.0	0.0	0.0179	0.6760	0.0	0.1951	0.0000	0.5471	0.2218	0.0	0.1147	0.2032	0.6403	0.3189	0.4204	0.3505	0.2947	0.5676	0.0
0.324	0.16	80	0.4282	0.2893	0.4044	0.6041	nan	0.0147	0.7890	0.0	0.8222	0.7984	0.6646	0.1038	0.0	0.0896	0.3308	0.8277	0.4099	0.6839	0.2401	0.5474	0.5521	0.0	0.0	0.0147	0.6800	0.0	0.6159	0.3049	0.5913	0.0938	0.0	0.0802	0.2394	0.6598	0.3178	0.4504	0.2288	0.4189	0.5113	0.0
0.297	0.2	100	0.4020	0.3034	0.4230	0.6332	nan	0.0048	0.8076	0.0080	0.9042	0.6567	0.8036	0.0317	0.0	0.0481	0.5298	0.7728	0.2589	0.7232	0.5941	0.3839	0.6643	0.0	0.0	0.0048	0.6708	0.0080	0.6300	0.3836	0.5929	0.0314	0.0	0.0441	0.3152	0.6726	0.2420	0.4745	0.4532	0.3631	0.5759	0.0
0.2608	0.24	120	0.3538	0.3444	0.4554	0.6504	nan	0.2922	0.8078	0.0753	0.8472	0.0425	0.6961	0.6197	0.0	0.2550	0.3074	0.8020	0.5636	0.6895	0.3779	0.6930	0.6734	0.0	0.0	0.2757	0.6940	0.0747	0.6457	0.0419	0.6098	0.3611	0.0	0.1849	0.2412	0.7038	0.4513	0.5038	0.3439	0.4760	0.5915	0.0
0.3306	0.28	140	0.3281	0.3562	0.4736	0.6560	nan	0.4111	0.8576	0.1953	0.8081	0.6916	0.7888	0.3489	0.0	0.0809	0.3612	0.8132	0.0622	0.7078	0.6328	0.5437	0.7482	0.0	0.0	0.3895	0.7227	0.1857	0.6777	0.3750	0.6015	0.2749	0.0	0.0740	0.2602	0.7070	0.0612	0.4348	0.5114	0.4966	0.6385	0.0
0.364	0.32	160	0.3368	0.3689	0.4836	0.6531	nan	0.3898	0.8453	0.1743	0.9269	0.2493	0.7922	0.0842	0.0	0.4874	0.2384	0.8116	0.6226	0.5731	0.6049	0.6620	0.7597	0.0	0.0	0.3746	0.7246	0.1690	0.6015	0.1998	0.5942	0.0786	0.0	0.2682	0.1904	0.7015	0.4781	0.4781	0.5452	0.5804	0.6562	0.0
0.635	0.36	180	0.3092	0.3699	0.4903	0.6319	nan	0.4996	0.8387	0.2136	0.6184	0.0129	0.7920	0.8199	0.0	0.1895	0.3028	0.8307	0.7258	0.3386	0.7480	0.6543	0.7511	0.0	0.0	0.4613	0.7126	0.2042	0.5589	0.0128	0.6658	0.3529	0.0	0.1622	0.2426	0.7363	0.4646	0.3144	0.5794	0.5575	0.6321	0.0
0.1464	0.4	200	0.3306	0.3809	0.5041	0.6544	nan	0.6110	0.8337	0.2420	0.8913	0.8862	0.6492	0.0004	0.0	0.2888	0.2949	0.8514	0.4630	0.7751	0.7020	0.5429	0.5386	0.0	0.0	0.5329	0.7348	0.2331	0.6567	0.3661	0.5769	0.0004	0.0	0.2221	0.2333	0.7431	0.4133	0.5478	0.5718	0.5125	0.5107	0.0
0.2257	0.44	220	0.2751	0.4089	0.5400	0.6752	nan	0.6851	0.8458	0.4204	0.7241	0.1085	0.7997	0.7657	0.0	0.2458	0.4039	0.8858	0.7863	0.3199	0.7405	0.6974	0.7508	0.0	0.0	0.5815	0.7437	0.3776	0.6458	0.1033	0.6526	0.3966	0.0	0.2027	0.3078	0.7438	0.4680	0.2966	0.6204	0.5942	0.6260	0.0
0.3069	0.48	240	0.2614	0.4163	0.5499	0.6868	nan	0.6246	0.8571	0.3130	0.7765	0.8266	0.7786	0.3212	0.0	0.3560	0.3736	0.8579	0.1780	0.8761	0.7423	0.7693	0.6970	0.0	0.0	0.5597	0.7370	0.2931	0.6733	0.4032	0.6889	0.2487	0.0	0.2662	0.2901	0.7425	0.1724	0.4957	0.6373	0.6376	0.6470	0.0
0.1454	0.52	260	0.2563	0.4316	0.5610	0.6965	nan	0.6707	0.8388	0.5572	0.7616	0.3854	0.7280	0.7114	0.0	0.1934	0.3621	0.8718	0.7860	0.6140	0.7403	0.5340	0.7820	0.0	0.0	0.5710	0.7446	0.4497	0.6637	0.3125	0.6624	0.4219	0.0	0.1731	0.2862	0.7295	0.5339	0.5054	0.5742	0.4967	0.6449	0.0
0.1522	0.56	280	0.2521	0.4327	0.5567	0.7138	nan	0.5098	0.9135	0.3399	0.8898	0.5537	0.7508	0.2922	0.0	0.3367	0.2484	0.8388	0.7460	0.7191	0.7496	0.7996	0.7753	0.0	0.0	0.4902	0.7541	0.3196	0.6924	0.3853	0.6261	0.2512	0.0	0.2575	0.2171	0.7393	0.5563	0.5633	0.6403	0.6335	0.6621	0.0
0.1872

📄 License

The license of this model is other.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご