Howto ollama

install packages

• archlinux

install nvidia drivers:

https://wiki.vidalinux.org/index.php?title=Howto_install_nvidia_drivers#archlinux

install ollama dependencies:

pacman -S ollama ollama-cuda nvidia-container-toolkit docker docker-compose

ollama

copy systemd ollama daemon to /etc/systemd/system/:

cp /usr/lib/systemd/system/ollama.service /etc/systemd/system/

edit /etc/systemd/system/ollama.service and add the following:

Environment="OLLAMA_HOST=0.0.0.0"

reload systemd daemon:

systemctl daemon-reload

restart ollama.service

systemctl restart ollama.service

verify ollama is listening on all interfaces:

netstat -tnulp|grep 11434

to search for supported models on ollama:

https://ollama.com/search

create your own ssl cert

create rsa key:

openssl genrsa -out server.key 3072

create certificate csr:

openssl req -new -key server.key -out server.csr

fill the following blanks:

Country Name (2 letter code) []: US
State or Province Name (full name) []: Puerto Rico
Locality Name (eg, city) []: San Juan
Organization Name (eg, company) []: Vidalinux.com Corp.
Organizational Unit Name (eg, section) []: Linux Consulting
Common Name (eg, your name or your server's hostname) []: chat.ovox.io
Email Address []: chat@ovox.io
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []: just press enter
An optional company name []: just press enter

create the certificate:

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

openwebui

clone git repo:

git clone https://github.com/vidalinux/ollama.git

copy server.key and server.crt to nginx/ssl/

cd ollama/openwebui
mkdir nginx/ssl
cp server.key server.crt nginx/ssl/

deploy services using docker-compose:

docker-compose up -d

to enter openwebui interface:

https://chat.ovoxcloud.com