Using pushState and replaceState
better linking
- tags
- javascript
Ollama works great on my machine, but now that fly has gpu machines that scale to zero this seemed like a fun thing to test out.
First we'll just deploy a straight ollama instance and interact with it using our laptop, then we'll put a public facing instance of openwebui on the machine and have both running.
Lets start with a fly.toml
file. You should update the app name here:
|
|
We are creating a mounted volumn at /root/.ollama
which will persist
accross loading and suspending.
Update the url for all of these for the name of your app. You can
also fly ssh console
in to poke around, or fly logs
to see what's
happening.
|
|
{ "models": [] }
|
|
|
|
null
Lets run open-webui
in a transient container to interact with our
remote ollama instance. Again be sure to update the OLLAMA_BASE_URL
to be what you need.
|
|
Then open up localhost:3000 and connect to the server. You'll need to signup for an account, and then you can start pulling models.
Go to Modelfiles
on the left, and create a new model with the
following prompt.
(This is foto-forge)
|
|
Save it and then upload a file with that model and see how it works!
Check you model list:
|
|
"hub/dwrou/foto-forge:latest" "llava:13b-v1.5-q8_0"
Then send it a base64
encoded image with a prompt:
|
|
The image depicts a young boy standing on a chair at a counter in a coffee shop, preparing to make coffee by placing his cup under the espresso machine. He is surrounded by various cups and bottles, possibly indicating that this is a busy location or he's ordering more drinks for himself or others. The boy appears focused on his task, while the numerous cups scattered around the counter suggest the shop serves a variety of beverages. In addition to the espresso machine, there are several books visible in the background, which could be part of the decor or related to the coffee shop's ambiance. The overall atmosphere seems lively and bustling, with the young boy taking an active role in his coffee-making experience.
Dockerfile
:
|
|
Create both_start.sh
:
|
|
Update the fly.toml
file to use the openweb-ui port and remove the
build
section:
|
|
Then:
|
|
This will take a while, but it will build the image, push it over, create a volume and spin up the machine.
You can look at what it's doing with fly logs
.
Previously
Next