Berturk Legal

🚀 BERTurk-Legal

BERTurk-Legal is a transformer-based language model designed for retrieving prior legal cases, pre-trained on Turkish legal domain data.

🚀 Quick Start

We introduce BERTurk-Legal, a transformer-based language model for retrieving prior legal cases. BERTurk-Legal is pre-trained on a dataset from the Turkish legal domain, which doesn't contain any labels related to the prior court case retrieval task. It is trained in a self-supervised manner using masked language modeling. With zero-shot classification, BERTurk-Legal achieves state-of-the-art results on the dataset of legal cases from the Court of Cassation of Turkey. The experimental results highlight the need for developing language models specific to the Turkish law domain. You can find more details about BERTurk-Legal in the paper cited below.

The test dataset can be accessed via this link: https://github.com/koc-lab/yargitay_retrieval_dataset

You can load the model and create document embeddings as follows. These embeddings can then be used for retrieval.

💻 Usage Examples

Basic Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

bert_model = "KocLab-Bilkent/BERTurk-Legal"

model = AutoModelForSequenceClassification.from_pretrained(bert_model, output_hidden_states=True)
tokenizer = AutoTokenizer.from_pretrained(bert_model)

tokens = tokenizer("Örnek metin") # a dummy text is provided as input

output = model(tokens) 
docEmbeddings = output.hidden_states[-1]

📚 Documentation

📄 License

This project is licensed under the MIT license.

📚 Citation

If you use the model, please cite the following conference paper:

  @inproceedings{ozturk23berturkLegal,
    author={\"{O}zt\"{u}rk, Ceyhun E. and \"{O}z\c{c}elik, {\c{S}}. Bar{\i}\c{s} and Aykut Ko\c{c}},
    booktitle={2023 31st Signal Processing and Communications Applications Conference (SIU)}, 
    title={{A Transformer-Based Prior Legal Case Retrieval Method}}, 
    year={2023},
    volume={},
    number={},
    pages={1-4}
  }
  @mastersthesis{ozturk23legalNlp,
  author  = "\"{O}zt\"{u}rk, Ceyhun E.",
  title   = "Retrieving Turkish Prior Legal Cases with Deep Learning",
  school  = "Bilkent University",
  year    = "2023"
  }