|
1 |
| - |
| 1 | + |
4 | 2 | # Ampere® optimized llama.cpp
|
5 |
| -Ampere® optimized build of [llama.cpp]( |
6 |
| -https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#llamacpp) |
7 |
| -with full support for rich collection of GGUF models available at HuggingFace: [GGUF models]( |
8 |
| -https://huggingface.co/models?search=gguf) |
| 3 | +Ampere® optimized build of [llama.cpp](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#llamacpp) with full support for rich collection of GGUF models available at HuggingFace: [GGUF models](https://huggingface.co/models?search=gguf) |
9 | 4 |
|
10 | 5 | This Docker image can be run on bare metal Ampere® CPUs and Ampere® based VMs available in the cloud.
|
11 | 6 |
|
12 | 7 | Release notes and binary executables are available on our [GitHub](https://github.com/AmpereComputingAI/llama.cpp/releases)
|
13 | 8 |
|
14 | 9 | ## Starting container
|
15 |
| - |
| 10 | +Default entrypoint runs the server binary of llama.cpp, mimicking behavior of original llama.cpp server image: [docker image](https://github.com/ggerganov/llama.cpp/blob/master/.devops/server.Dockerfile) |
| 11 | + |
| 12 | +To launch shell instead, do this: |
| 13 | + |
16 | 14 | ```bash
|
17 |
| -docker run --name llama -it --rm ghcr.io/amperecomputingai/llama.cpp:latest |
| 15 | +sudo docker run --privileged=true --name llama --entrypoint /bin/bash -it amperecomputingai/llama.cpp:latest |
18 | 16 | ```
|
19 | 17 | Quick start example will be presented at docker container launch:
|
20 |
| - |
21 |
| - |
24 |
| - |
25 |
| -Make sure to visit us at [Ampere Solutions Portal]( |
26 |
| -https://solutions.amperecomputing.com/solutions/ampere-ai)! |
27 |
| - |
| 18 | + |
| 19 | + |
| 20 | + |
| 21 | +Make sure to visit us at [Ampere Solutions Portal](https://solutions.amperecomputing.com/solutions/ampere-ai)! |
| 22 | + |
28 | 23 | ## Support
|
29 |
| - |
| 24 | + |
30 | 25 | Please contact us at <ai-support@amperecomputing.com>
|
31 |
| - |
| 26 | + |
32 | 27 | ## LEGAL NOTICE
|
33 |
| -By accessing, downloading or using this software and any required dependent software (the “Ampere AI Software”), you agree to the terms and conditions of the software license agreements for the Ampere AI Software, which may also include notices, disclaimers, or license terms for third party software included with the Ampere AI Software. Please refer to the [Ampere AI Software EULA v1.6]( |
34 |
| -https://ampereaidevelop.s3.eu-central-1.amazonaws.com/Ampere+AI+Software+EULA+-+v1.6.pdf) |
35 |
| -or other similarly-named text file for additional details. |
| 28 | +By accessing, downloading or using this software and any required dependent software (the “Ampere AI Software”), you agree to the terms and conditions of the software license agreements for the Ampere AI Software, which may also include notices, disclaimers, or license terms for third party software included with the Ampere AI Software. Please refer to the [Ampere AI Software EULA v1.6](https://ampereaidevelop.s3.eu-central-1.amazonaws.com/Ampere+AI+Software+EULA+-+v1.6.pdf) or other similarly-named text file for additional details. |
0 commit comments