Llama 2 7b Privacy

🚀 Fine-tuning Llama2-7B-Chat for Privacy Policy Q&A and Summarization

This project aims to fine-tune Meta's Llama 2 Transformer using PEFT and QloRA for privacy policy analysis, helping users understand personal data usage.

🚀 Quick Start

The basis of this problem is to fine-tune Meta's Llama 2 Transformer using PEFT and QloRA in order to be able to analyze privacy Policies. Privacy Policies are written to be confusing and extremely technical, so a tool that helps users answer questions and summarize privacy policies can be very useful in knowing how personal data is being used. The HOWTO.md file contains simple instructions on running the tool.

✨ Features

• Fine-tune Meta's Llama 2 Transformer for privacy policy analysis.
• Help users answer questions and summarize privacy policies.

📦 Installation

This tool is designed to be used on Colab with the Hugging Face Transformers library. For more info, check out the model on GitHub. The model was trained using a training code for Llama from Maxime Labonne, available here.

There is no requirements file in this repo because it uses dependencies pre-installed on Colab.

💻 Usage Examples

Basic Usage

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ChrisPuzzo/llama-2-7b-privacy")
model = AutoModelForCausalLM.from_pretrained("ChrisPuzzo/llama-2-7b-privacy")

📚 Documentation

Results

We did some small amounts of testing on the model; however, we got pretty inconclusive data. As you can see in the ROUGE Excel sheet, the scores weren't great. However, we believe that the ROUGE testing metric might not be the best way to judge this.

Next Steps

The next steps will be to train this data on another dataset I found for summarizing text in privacy policies.

📄 License

This project is licensed under the Apache-2.0 license.