Allmini Ai Embedding Similarity

🚀 Sentence Similarity Model for Job Skills

This project is a sentence similarity model based on sentence-transformers. It can effectively measure the similarity between sentences, which is useful for tasks like job skill matching, feature extraction, etc.

✨ Features

• Sentence Similarity: Accurately calculate the similarity between different sentences, helping to match job requirements with candidate skills.
• Feature Extraction: Extract useful features from sentences for further analysis.
• Generated from Trainer: Trained with specific datasets to ensure high performance.

📦 Installation

The base model used is sentence-transformers/all-MiniLM-L6-v2. You can install relevant dependencies according to the requirements of sentence-transformers.

💻 Usage Examples

Basic Usage

The following is an example of using the widget to calculate sentence similarity:

# The widget provides source sentences and candidate sentences for similarity calculation
# For example:
source_sentence = "Data engineering, AWS services, Big Data manipulation"
sentences = [
    "Skills: SQL, PySpark, Databricks, Azure Synapse, Azure Data Factory. Need hands - on coding...",
    # Other candidate sentences...
]
# Here should be the actual code for calculating similarity using sentence-transformers
# For example:
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
source_embedding = model.encode(source_sentence)
for sentence in sentences:
    sentence_embedding = model.encode(sentence)
    similarity = util.pytorch_cos_sim(source_embedding, sentence_embedding)
    print(f"Similarity between '{source_sentence}' and '{sentence[:50]}...': {similarity[0][0]}")

Advanced Usage

# In more complex scenarios, you can batch process multiple source sentences and candidate sentences
source_sentences = [
    "Data engineering, AWS services, Big Data manipulation",
    "Digital advertising, MLOps, audience segmentation"
]
candidate_sentences = [
    # Multiple candidate sentences...
]
# Encode all source sentences and candidate sentences
source_embeddings = model.encode(source_sentences)
candidate_embeddings = model.encode(candidate_sentences)
# Calculate similarity matrix
similarity_matrix = util.pytorch_cos_sim(source_embeddings, candidate_embeddings)
for i, source in enumerate(source_sentences):
    for j, candidate in enumerate(candidate_sentences):
        print(f"Similarity between '{source}' and '{candidate[:50]}...': {similarity_matrix[i][j]}")

📚 Documentation

Model Information

📄 License

No license information is provided in the original document.