Following-up on my last post, I'm sharing the fairly simple commands to install Ollama and Open-WebUI for running LLMs locally. The following was performed on an Apple iMac M1 (Sonoma 14.6.1) with 16 GB RAM and the latest patches. The Linux installation is similar. Note, you may also be able to run this on a Windows machine. Check the respective websites for instructions and pre-requisites.
Step 1: Install OLLAMA
This is the “engine” that serves the LLM to the web interface (Open-WebUI).
From a Terminal window (Spotlight Search -> Terminal):
After a while you should receive:
Step 2: Download an LLM
The LLM is the trained model, required for you to interface with. When you download it, it will pull the model file from ollama.com to your machine. The file will be large (several to tens of gigabytes) so be sure you have enough free space (30-50 GB).
From a Terminal window:
You can pull any model available on the ollama.com site. llama3.1 is created by Meta (Facebook). There is also "gemma2" by Google, "phi3" from Microsoft, and others. Visit the site and go to the “Models” section to see them all. I would recommend only installing the single-digit (e.g., 2B, 7B, 8B) parameter models unless you have enough space.
Step 3: Install Open WebUI (formerly Ollama WebUI)
This is the web-based user interface, like OpenAI’s ChatGPT, that allows you to interface with the LLM.
From a Terminal window:
Step 4: Start Ollama
From the Spotlight Search box, you can run Ollama directly, or from a Terminal window you can also run it using the command:
Replace "llama3.1" with another model name, if you pulled another. This will start Ollama and you should see an icon at the top in your menu bar. To close it, you can click it and select Quit Ollama. Once you see the icon, you can enter “/bye” in the terminal window to get back to a command prompt.
Step 5: Start Open-WebUI
From a Terminal window, start Open-WebUI:
This will start the Open-WebUI web user interface. You should see something like this, indicating that it is running:
Step 6: Browse To Use
You can browse to the interface from Safari or Chrome by entering http://localhost:8080. When you run it for the first time you will need to setup an account before using. No information that you enter is passed to the Internet, according to Open-WebUI’s documentation. After logging in for the first time, you should get a ChatGPT-style interface:
To end a session: close the browser, press CTRL-C in the terminal window, and shut down Ollama.
Enjoy!
References
Ollama – www.ollama.com
Open WebUI – www.openwebui.com