When we look at a photograph, we immediately recognise people, objects, colours and shapes.

But what does an AI or machine-learning model see?

It does not see the photograph in the same way that we do. An AI model ultimately receives numbers.

Image → Pixels → RGB Values → Tensor → Normalized Tensor → AI Model

Watch: How AI Sees an Image

This short Learn With Champak lesson shows what happens when an image is converted into a tensor using TensorFlow.js.

Try It Yourself: Image → Tensor

Use the interactive program below. Choose an image, select how many pixels you want to inspect, and convert the image into a TensorFlow.js tensor.

Open the experiment in a new tab

What Is an Image to a Computer?

A digital colour image is made from pixels. Each pixel normally contains three colour values:

  • Red
  • Green
  • Blue

These are usually called the RGB channels.

Each colour value normally ranges from:

0 to 255

So a pixel might contain values such as: 

Red   = 120
Green = 200
Blue  = 80

A computer can therefore represent an entire image as a large collection of numbers.

Converting an Image to a Tensor

TensorFlow.js provides a very convenient function for converting image pixels into a tensor: 

const tensor = tf.browser.fromPixels(canvas);

If our canvas is 240 × 240 pixels, TensorFlow.js creates a tensor with the shape: 

[240, 240, 3]
Value Meaning
240 Image height
240 Image width
3 Red, Green and Blue channels

What Is a Tensor?

A tensor is essentially a structured collection of numbers.

For an RGB image we can think of it as numbers arranged according to: 

[height, width, colour channels]

This numerical representation is what allows TensorFlow and other machine-learning libraries to perform mathematical operations on images.

Why Normalize Pixel Values?

Raw RGB pixel values normally range between 0 and 255. Machine-learning models commonly work more conveniently with smaller numerical ranges.

We can convert the tensor to floating-point numbers and divide every pixel value by 255: 

const normalized = tensor
  .toFloat()
  .div(255);

Now the values are approximately between:

0 and 1

For example:

255 / 255 = 1

128 / 255 ≈ 0.502

0 / 255 = 0

See the Actual Pixels

The accompanying Learn With Champak program goes one step further. Instead of merely saying that an image contains numbers, it lets you inspect those numbers yourself.

You can:

  • Choose an image from your computer.
  • Display it on an HTML canvas.
  • Choose how many pixels you want to inspect.
  • Convert the canvas into a TensorFlow.js tensor.
  • See the tensor shape.
  • See the tensor data type.
  • See normalized RGB values.
  • See the position of individual pixels.

The Important Idea

Computer vision can appear mysterious when we begin studying AI and machine learning.

But the fundamental idea is surprisingly simple:

A picture becomes numbers.
Those numbers become a tensor.
The AI model performs mathematics on that tensor.

Once you understand this transformation, concepts such as image classification, convolutional neural networks, object detection and facial recognition become much easier to understand.

From Image to AI

The complete pipeline can be thought of as: 

Image
  ↓
Canvas
  ↓
Pixels
  ↓
RGB values
  ↓
Tensor
  ↓
Normalization
  ↓
Machine-learning model
  ↓
Prediction

Try It Yourself

Do not stop at watching the video. Try the program with different images.

Pay particular attention to how the RGB values change when you use an image containing bright colours, dark areas, white areas or black areas.

Experimenting with the actual data is one of the best ways to understand what happens inside a computer-vision program.

Learn AI and Machine Learning by actually programming it.

Watch the video, run the program, change the code and observe what happens.
Watch on YouTube

What Should We Learn Next?

Now that we can convert an image into a tensor, the natural next step is to send that tensor into an AI model.

That takes us from simply reading pixels to actually performing computer vision.

If you found this useful, subscribe to Learn With Champak, share the lesson, and leave a comment telling me which AI/ML concept you would like to explore next.

Learn With Champak
Programming • Python • AI • Machine Learning • DSA