Howto ramalama: Difference between revisions
Jump to navigation
Jump to search
Mandulete1 (talk | contribs) |
Mandulete1 (talk | contribs) |
||
| Line 19: | Line 19: | ||
run model as service: | run model as service: | ||
ramalama serve gpt-oss | ramalama serve gpt-oss | ||
run model as service with llama-stack and other options: | |||
ramalama serve --port 8085 --api llama-stack --name deepseek-service -d deepseek | |||
stop model service: | |||
ramalama stop deepseek-service | |||
= references = | = references = | ||
* https://crfm.stanford.edu/2023/03/13/alpaca.html | * https://crfm.stanford.edu/2023/03/13/alpaca.html | ||
* https://github.com/containers/ramalama/tree/main/docs | * https://github.com/containers/ramalama/tree/main/docs | ||
Revision as of 06:19, 12 August 2025
install
install on fedora:
sudo dnf install python3-ramalama
install via pypi:
pip install ramalama
install script linux/mac:
curl -fsSL https://ramalama.ai/install.sh | bash
usage
set variables:
RAMALAMA_CONTAINER_ENGINE=docker CUDA_VISIBLE_DEVICES="0"
run model ibm granite:
ramalama run granite
run model openai gpt-oss:
ramalama pull gpt-oss:latest
run model deekseek-r1:
ramalama pull deepseek
run model as service:
ramalama serve gpt-oss
run model as service with llama-stack and other options:
ramalama serve --port 8085 --api llama-stack --name deepseek-service -d deepseek
stop model service:
ramalama stop deepseek-service