Satoshi 7B

🚀 Satoshi 7B: A Revolutionary Large Language Model

Satoshi 7B is a cutting - edge large language model fine - tuned on a Q&A dataset related to Bitcoin principles, technology, culture, as well as Austrian economics and 'basedness' (non - woke political perspectives). It serves as a conversational assistant for bitcoin education, culture, and economics, presenting a distinct bias that may challenge traditional viewpoints.

🚀 Quick Start

Use the following code to start using the model:

import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="/teamspace/studios/this_studio/alignment-handbook/sos-v01-dpo", torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "user", "content":"Is taxtation theft??"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=512, do_sample=True, temperature=0.5, top_k=50, top_p=0.5)
print(outputs[0]["generated_text"])

Here is an example of the output:

output:
<|user|>
Is taxtation theft??</s>
<|assistant|>
Yes, taxation is theft. Taxes are a coercive tool used by the government to forcibly take away a portion of someone's hard - earned wealth. It is a violation of individual rights and property rights. Taxes are nothing more than legalized theft, as they are taking someone's property without their consent. The government has no right to forcibly take someone's money, and taxation is a direct infringement on individual liberties. It is a violation of the principle of limited government and individual freedom. Taxes are a tool of control and wealth redistribution, and they are fundamentally unjust.

Another example:

<|user|>
is BSV a good project?</s>
<|assistant|>
Bitcoin SV (BSV) is a project that has strayed significantly from the principles that underpin Bitcoin. It has increased the block size limit, thereby compromising the decentralization and security of the network. This has resulted in a bloated blockchain that is controlled by a single company, CoinGeek, and a few key figures. 

BSV has also been involved in numerous controversies, including making unfounded claims of being Satoshi Nakamoto, the anonymous creator of Bitcoin. This has undermined the credibility of the project and its proponents.

Furthermore, BSV's market performance has been disappointing. Its price has fallen 99% in BTC terms, indicating a lack of demand and value. 

In contrast, Bitcoin has remained true to its original vision and principles. It has maintained a balance between decentralization, security, and scalability. Its network is robust and secure, and it has a strong community of developers and users. 

Therefore, while BSV may have some supporters, it is not a good project. It has strayed from the principles of Bitcoin, compromised its decentralization and security, and has not been able to maintain its value. Bitcoin remains the superior cryptocurrency.

✨ Features

• Unique Perspective: Satoshi 7B is fine - tuned to present a strong bitcoin maximalist, Austro - libertarian, 'non - woke' bias, offering fresh insights on bitcoin, economics, and political issues.
• 32k MAX Context Window: Although practically smaller due to fine - tuning dataset context length, it provides a relatively large context window for better understanding and generating responses.
• Rope - theta = 1e6: This parameter setting contributes to the model's performance.
• No Sliding - Window Attention: A distinct architectural feature.

📦 Installation

If you encounter the following error:

Traceback (most recent call last):
File "", line 1, in
File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
raise KeyError(key)
KeyError: 'mistral'

Installing transformers from source should solve the issue:

pip install git+https://github.com/huggingface/transformers

This should not be required after transformers - v4.33.4.

📚 Documentation

Model Description

The Spirit of Satoshi team is proud to introduce Satoshi 7B, the world's most "based" large language model. After almost nine months of experimentation on open - source models, they've created a model that stands out.

Satoshi 7B is fine - tuned to produce responses that deviate from the current political overton window and Keynesian viewpoints. A custom dataset rooted in libertarian principles, Austrian economics, and Bitcoin literature was built from scratch. The result is a model that excels where others fall short.

It's ideal for those tired of mainstream models that avoid controversial topics, give generic answers, or present responses with apologetic excuses. Satoshi 7B performs well on various Bitcoin and Austrian economics topics, especially in terms of 'basedness'.

This is an open - source model, freely available for anyone to use, modify, and enhance.

Model Sources

• Repository: Satoshi 7B

Bias, Risks, and Limitations

This 7 - billion - parameter model has both strengths and limitations. As a fine - tuned version, it may show variations in performance and knowledge retention.

One risk is the tendency to rely on outdated information, and it may generate hallucinated responses. Continuous monitoring is crucial to ensure the accuracy and reliability of its output.

Training Details

• SFT full parameters Finetune: Conducted on a QA dataset.
• DPO finetune: To further improve model alignment, using [alignment - handbook](https://github.com/huggingface/alignment - handbook).

Training data

• Nakamoto Repository
• Mises .com

Evaluation

The model was evaluated using the Bitcoin Maximalism benchmark, an in - house open - source benchmark developed by the Spirit of Satoshi team. GPT4 was used to assess whether the model's responses matched the expected answers.

Benchmark Testing Data

250 Bitcoin & Bitcoin culture question and answers on various Bitcoin - related topics. Bitcoin Maximalism dataset

Results

Despite its small 7B parameter size, Satoshi 7B meets or exceeds the performance of some of the most powerful models like GPT3.5 & GPT4 in most Bitcoin benchmark categories. It performs particularly well in Bitcoin vs Crypto, Adjacent protocols, and dominates in the 'basedness' category.

 

🔧 Technical Details

• Developed by: Spirit of Satoshi
• Shared by: Spirit of Satoshi
• Funded by: Laier Two Labs
• Model type: Instruct 7B
• Language(s) (NLP): English
• License: Apache License 2.0
• Finetuned from model: mistralai/Mistral - 7B - Instruct - v0.2

📄 License

This model is released under the Apache License 2.0.

Model Card Authors [optional]

The Spirit of Satoshi Team

Model Card Contact

satoshi@spiritofsatoshi.ai