howto

Programming with ollama

automated interaction

tags
ollama
ruby
ai

Contents

Test out the ollama install

1

  ollama list
NAMEIDSIZEMODIFIED
gemma:7b430ed35350495.2 GB4 weeks ago
llava:latest8dd30f6b0cb14.7 GB4 weeks ago
mistral:7b61e88e8845074.1 GB4 weeks ago
mistral:latest61e88e8845074.1 GB3 weeks ago
zephyr:latestbbe38b81adec4.1 GB4 weeks ago
1

  ollama serve

And then we can test with curl to make sure that it's running.

1
2
3
4
5

  curl http://localhost:11434/api/generate -d '{
    "model": "gemma:7b",
    "prompt":"Why is the sky blue?",
    "stream": false
  }' | jq -r .response | fold -s
Sure, here's why the sky appears blue:

The sky appears blue because of a phenomenon called **Rayleigh Scattering**. 

Here's the breakdown:

1. **Sunlight:** Sunlight consists of all the colors of the rainbow, with each 
color having a different wavelength.
2. **Scattering:** When sunlight enters the Earth's atmosphere, particles like 
dust and air molecules scatter the different colors of the spectrum in all 
directions.
3. **Scattering Direction:** However, the particles scatter the different 
colors differently based on their size and wavelength.
4. **Blue Scatter:** The scattered light, particularly the shorter wavelengths 
of blue and violet, scatter more efficiently in the direction of the observer's 
eyes.

**Therefore, the scattered light, which primarily consists of blue and violet 
colors, is scattered in all directions, and we perceive the sky as blue.**

Here are some additional factors that influence the color of the sky:

* **Time of Day:** The intensity of the blue color is strongest at midday and 
decreases as the sun gets closer to the horizon.
* **Clouds and Pollution:** Clouds and pollution can reduce the scattering of 
blue light, making the sky appear white or gray.
* **Sunsets and Sunrises:** At sunrise and sunset, the sun's rays have to 
travel farther through the atmosphere to reach our eyes, which can cause the 
scattered blue light to be scattered more effectively, making the sky appear 
orange or red.

It's important to note that this is a simplified explanation and there are some 
complex scientific principles involved. However, it gives you a good 
understanding of why the sky appears blue.

Write some test ruby code

1
2

  bundle init
  bundle add ollama-ai
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

  require 'ollama-ai'

  client = Ollama.new(
    credentials: { address: 'http://localhost:11434' },
    options: { server_sent_events: true }
  )

  result = client.generate(
    { model: 'gemma:7b',
      prompt: 'Hi!' }
  ) do |event, raw|
    print event['response']
    $stdout.flush
  end
  puts
Hi! 👋

It's nice to hear from you. What would you like to talk about today?

Chat test

chat.rb:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

  require 'ollama-ai'

  def get_response( message, context = nil )
    client = Ollama.new(
      credentials: { address: 'http://localhost:11434' },
      options: { server_sent_events: true }
    )
    
    result = client.generate(
      { model: 'zephyr', #gemma:7b',
        context: context,
        prompt: message }
    ) do |event, raw|
      print event['response']
      $stdout.flush
      context = event['context']
    end
    puts

    context
  end

  context = nil
  message = ""
  while message != "bye"
    print ">>> "
    $stdout.flush
    message = gets.chomp

    context = get_response( message, context )
  end

Describe an image

image.rb:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

  require 'ollama-ai'
  require 'base64'

  client = Ollama.new(
    credentials: { address: 'http://localhost:11434' },
    options: {
      server_sent_events: true,
      connection: { request: { timeout: 120, read_timeout: 120 } } }
  )

  client.generate(
    { model: 'llava',
      prompt: 'Please describe this image.',
      images: [Base64.strict_encode64(File.read('newspapers.jpg'))] }
  ) do |event, raw|
    print event['response']
  end

  puts
The image shows a counter with several newspapers spread
out on it. There are three stacks of newspapers, with the largest
stack in the middle, containing multiple copies of what appears to
be the same publication. Each newspaper is open, displaying its
contents. In the background, there's a coffee cup and what seems to
be a small table or shelf. The counter is likely located inside a
café or newsstand given the presence of the newspapers and coffee
setup. There's no visible text in the image that provides additional
context or information about the location or event.

Testing out the responses from multiple LLMs

I just discovered Chain of Thought Prompting, so lets see how it works on each of the models that we have installed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

  require 'ollama-ai'

  def get_response( model,  message )
    client = Ollama.new(
      credentials: { address: 'http://localhost:11434' },
      options: { server_sent_events: true }
    )

    response = ""
    
    result = client.generate(
      { model: model,
        prompt: message }
    ) do |event, raw|
      response << event['response']
      print event['response']
      $stdout.flush
      context = event['context']
    end
    puts

    response
  end

  client = Ollama.new(
    credentials: { address: 'http://localhost:11434' },
    options: {
      server_sent_events: true,
      connection: { request: { timeout: 120, read_timeout: 120 } } }
  )

  models = client.tags.first['models'].collect { |x| x['name'] }

  models.each do |model|
    puts "Running #{model}"
    get_response( model, "peter has 5 apples, and he gives 2 apples to susan.  \
                          now susan has 2 apples and peter has 3.  If john had \
                          7 apples and gives 3 to mary, how many apples does \
                            john have left?" )
    puts
  end

There are a lot of responses, but yep it works.

References

Previously

labnotes

Deploying ollama on fly.io

scale to zero

tags
ollama
flyio

Next

howto

Geocoding with ollama

using json schema

tags
javascript
ollama
geocoding
ai