Deepseek Coder 1.3b Typescript

🚀 deepseek-coder-1.3b-typescript

CodeGPTPlus/deepseek-coder-1.3b-typescript is a fine - tuned model based on deepseek-ai/deepseek-coder-1.3b-base, crafted by the CodeGPT team. It specializes in generating precise and efficient TypeScript code. With a 16K window size and a fill - in - the - middle task, it offers project - level code completion.

🚀 Quick Start

This model is designed for completion tasks. Here are some usage examples:

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("CodeGPTPlus/deepseek-coder-1.3b-typescript",
                                          trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("CodeGPTPlus/deepseek-coder-1.3b-typescript",
                                              trust_remote_code=True).cuda()

input_text = """<｜fim▁begin｜>function quickSort(arr: number[]): number[] {
  if (arr.length <= 1) {
    return arr;
  }
  const pivot = arr[0];
  const left = [];
  const right = [];
<｜fim▁hole｜>
  return [...quickSort(left), pivot, ...quickSort(right)];
}<｜fim▁end｜>"""

inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_length=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Advanced Usage

Running with Ollama

Model: https://ollama.ai/codegpt/deepseek-coder-1.3b-typescript

ollama run codegpt/deepseek-coder-1.3b-typescript

Running with Ollama and CodeGPT Autocomplete in VSCode

Documentation: https://docs.codegpt.co/docs/tutorial-features/code_autocompletion

Select "Ollama - codegpt/deepseek-coder-1.3b-typescript" in the autocomplete model selector. Then, write any code or comment in the vscode text editor, and the model will provide you with code suggestions through the CodeGPT code autocomplete.

✨ Features

• Specialized for TypeScript: Fine - tuned on a TypeScript dataset, it can generate precise and efficient solutions in this programming language.
• Project - level Completion: Utilizes a 16K window size and a fill - in - the - middle task to offer project - level code completion.
• Good Performance: Achieves a loss of 0.7681 on the evaluation set.

📚 Documentation

Model Details

Evaluation Results

It achieves the following results on the evaluation set:

• Loss: 0.7681

Training Details

Training hyperparameters

• learning_rate: 2e - 05
• train_batch_size: 20
• eval_batch_size: 20
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 40
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 06
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 261
• num_epochs: 1

Training results

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0

📄 License

The model is released under the other license.