llama2-uncensored-7b

MAX Model

1 versions

Uncensored Llama 2 model by George Sung and Jarrad Hope.

Run this model

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.
Install Max Pipelines in order to run this model.
```
magic global install max-pipelines
```

Start a local endpoint for llama2-uncensored/7b:

max-serve serve --huggingface-repo-id TheBloke/Luna-AI-Llama2-Uncensored-AWQ

The endpoint is ready when you see the URI printed in your terminal:

Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "llama2-uncensored/7b",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n/
/g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Llama 2 Uncensored is a derivative of Meta's Llama 2 model, created by George Sung and Jarrad Hope following the method outlined by Eric Hartford in his blog post.

Memory Requirements

7B models require a minimum of 8GB of RAM.
70B models require a minimum of 64GB of RAM.

References

Eric Hartford's blog post
7B parameters source: George Sung
70B parameters source: George Sung

Llama 2 Acceptable Use Policy

Meta is committed to promoting safe and fair use of its tools and features, including Llama 2. If you access or use Llama 2, you agree to this Acceptable Use Policy (“Policy”). The most recent copy of this policy can be found at ai.meta.com/llama/use-policy.

Prohibited Uses

We want everyone to use Llama 2 safely and responsibly. You agree you will not use, or allow others to use, Llama 2 to:

Violate the law or others’ rights, including to:
1. Engage in, promote, generate, contribute to, encourage, plan, incite, or further illegal or unlawful activity or content, such as:
  1. Violence or terrorism
  2. Exploitation or harm to children, including the solicitation, creation, acquisition, or dissemination of child exploitative content or failure to report Child Sexual Abuse Material
  3. Human trafficking, exploitation, and sexual violence
  4. The illegal distribution of information or materials to minors, including obscene materials, or failure to employ legally required age-gating in connection with such information or materials.
  5. Sexual solicitation
  6. Any other criminal activity
2. Engage in, promote, incite, or facilitate the harassment, abuse, threatening, or bullying of individuals or groups of individuals
3. Engage in, promote, incite, or facilitate discrimination or other unlawful or harmful conduct in the provision of employment, employment benefits, credit, housing, other economic benefits, or other essential goods and services
4. Engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or related professional practices
5. Collect, process, disclose, generate, or infer health, demographic, or other sensitive personal or private information about individuals without rights and consents required by applicable laws
6. Engage in or facilitate any action or generate any content that infringes, misappropriates, or otherwise violates any third-party rights, including the outputs or results of any products or services using the Llama 2 Materials
7. Create, generate, or facilitate the creation of malicious code, malware, computer viruses or do anything else that could disable, overburden, interfere with or impair the proper working, integrity, operation or appearance of a website or computer system
Engage in, promote, incite, facilitate, or assist in the planning or development of activities that present a risk of death or bodily harm to individuals, including use of Llama 2 related to the following:
1. Military, warfare, nuclear industries or applications, espionage, use for materials or activities that are subject to the International Traffic Arms Regulations (ITAR) maintained by the United States Department of State
2. Guns and illegal weapons (including weapon development)
3. Illegal drugs and regulated/controlled substances
4. Operation of critical infrastructure, transportation technologies, or heavy machinery
5. Self-harm or harm to others, including suicide, cutting, and eating disorders
6. Any content intended to incite or promote violence, abuse, or any infliction of bodily harm to an individual
Intentionally deceive or mislead others, including use of Llama 2 related to the following:
1. Generating, promoting, or furthering fraud or the creation or promotion of disinformation
2. Generating, promoting, or furthering defamatory content, including the creation of defamatory statements, images, or other content
3. Generating, promoting, or further distributing spam
4. Impersonating another individual without consent, authorization, or legal right
5. Representing that the use of Llama 2 or outputs are human-generated
6. Generating or facilitating false online engagement, including fake reviews and other means of fake online engagement
Fail to appropriately disclose to end users any known dangers of your AI system

Please report any violation of this Policy, software “bug,” or other problems that could lead to a violation of this Policy through one of the following means:

Reporting issues with the model: github.com/facebookresearch/llama
Reporting risky content generated by the model: developers.facebook.com/llama_output_feedback
Reporting bugs and security concerns: facebook.com/whitehat/info
Reporting violations of the Acceptable Use Policy or unlicensed uses of Llama: LlamaUseReport@meta.com

DETAILS

MODEL CLASS

MAX Model

MAX Models are extremely optimized inference pipelines to run SOTA performance for that model on both CPU and GPU. For many of these models, they are the fastest version of this model in the world.

Browse 18+ MAX Models

MODULAR GITHUB

Modular

CREATED BY

TheBloke

MODEL

TheBloke/Luna-AI-Llama2-Uncensored-AWQ

general.architecture	llama
general.file_type	2
general.name	georgesung
general.quantization_version	2
llama.attention.head_count	32
llama.attention.head_count_kv	32
llama.attention.layer_norm_rms_epsilon	1e-05
llama.block_count	32
llama.context_length	2048
llama.embedding_length	4096
llama.feed_forward_length	11008
llama.rope.dimension_count	128
tokenizer.ggml.bos_token_id	1
tokenizer.ggml.eos_token_id	2
tokenizer.ggml.model	llama
tokenizer.ggml.padding_token_id	0
tokenizer.ggml.scores	[0, 0, 0, 0, 0, ...]
tokenizer.ggml.token_type	[2, 3, 3, 6, 6, ...]
tokenizer.ggml.tokens	[<unk>, <s>, </s>, <0x00>, <0x01>, ...]