What is computer vision?
Computer vision is the part of AI that turns pixels into information. A photo or a video frame goes in, and a label, a box, an outline or a line of text comes out.

Wikipedia describes it as an interdisciplinary field about how computers gain high level understanding from digital images or video, and says it aims to automate tasks the human visual system can do. IBM calls it a subfield of AI that helps machines extract meaning from images and video. Four jobs cover most of it:
- Classification says what is in the image.
- Detection says what is where, with a box around each object.
- Segmentation says which pixels belong to each object.
- OCR says what the text in the image reads.
That is the answer to "what is computer vision in AI". The same idea sits under phone face unlock, a checkout camera, a factory inspection line and the image button in a chatbot. What is AI covers the wider field it belongs to.
What are the main computer vision tasks?
Classification, detection, segmentation and OCR are the four tasks that most products use. Each row below names the question the task answers, a published system built for it, and a product feature you can call today. Product features come from Google's Cloud Vision documentation as read on 22 November 2026.
| Task | Question it answers | Published or open example | Product example |
|---|---|---|---|
| Classification | What is in this image? | AlexNet sorted images into 1,000 ImageNet classes and won the 2012 competition | Cloud Vision label detection, which "Provides generalized labels for an image" with a confidence score |
| Detection | What is where? | YOLO, posted in June 2015, ran its base model at 45 frames per second | Cloud Vision object localization, which returns a label and a bounding box for each object |
| Segmentation | Which pixels belong to each object? | Meta's Segment Anything Model, trained on 1 billion masks | Gemini, which since version 2.5 returns contour masks as well as boxes, per Google's docs |
| OCR | What does the text say? | Tesseract, an Apache licensed engine that reads more than 100 languages | Cloud Vision text detection for sparse text and document text detection for dense pages and handwriting |
Other tasks build on these four. IBM lists object tracking, pose estimation, facial recognition, scene understanding and visual inspection. Wikipedia adds motion analysis, 3D scene reconstruction and image restoration, such as removing noise or filling a gap.
IBM also names real deployments: Amazon's Just Walk Out checkout, which lets shoppers take items and leave without a queue, and NASA software that estimates the pose of grappling targets for robotic arm operators on the International Space Station. IBM sells AI products, so treat its framing as a vendor's, but both examples cite AWS and NASA pages.
How does computer vision work?
A computer vision system learns from examples, then applies what it learned to a new image. IBM splits the build into four steps, and a fifth is the one you use.
- Gather and label data. For a chest X-ray model, that means scans tagged normal or pneumonia. Public sets such as ImageNet, COCO and Open Images save you that work.
- Preprocess. Resize, adjust brightness and contrast, and flip or rotate copies so the model sees more variety.
- Pick a model. Convolutional neural networks have long been the default. Vision transformers, which split an image into patches, are the other main family.
- Train. The model guesses, a loss function scores the guess, and the weights shift to cut the error.
- Run it on new images. This step is called inference, and it is the only one a user ever sees.
The model choice has moved fast. The 2020 Vision Transformer paper showed that a pure transformer applied to sequences of image patches can perform very well on image classification, with no convolutions at all. Neural networks explains the building blocks.
One claim in the current Google results is wrong. A page ranking second says developers must use deep learning or a CNN to build a computer vision system. They don't. Wikipedia dates edge extraction, line labeling and optical flow to the 1970s, and eigenface based face recognition to the 1990s. None of it used deep learning.
How is computer vision different from image processing and machine vision?
Image processing takes an image in and gives an image back, for example a sharper or brighter one. Computer vision takes an image or video in and returns an analysis of what it shows, or an action. Wikipedia notes that the fields overlap a great deal, and that some people see only one field with different names.
Machine vision is the factory version. Wikipedia says it tends to focus on applications, mainly in manufacturing, with real time processing emphasized and lighting that is often more controlled than in general computer vision.
On the hardware side, most systems use ordinary visible light cameras. Wikipedia also names thermographic cameras, hyperspectral imagers, radar, lidar and sonar for work where plain light is not enough.
Developers rarely start from zero. IBM names OpenCV, an open source library with over 2,500 algorithms, plus Keras, TensorFlow, Torchvision and scikit-image. Wikipedia lists applications such as automatic inspection, process control, and navigation for an autonomous vehicle or mobile robot.
What is the history of computer vision?
The marks for ImageNet and AlexNet carry only a year in their sources, so they sit at the start of 2009 and 2012. The dated facts behind the chart follow.
- 1 July 1966: Seymour Papert's memo "The Summer Vision Project" sets out to use summer workers on a significant part of a visual system. Its plan runs from separating an image into likely objects and background, to describing regions, to naming objects. That is a lot to ask of one summer.
- 1970s to 1990s: Wikipedia lists edge extraction, optical flow, scale space and shape from shading, then camera calibration, stereo and eigenface recognition.
- 2009: IBM dates the ImageNet dataset, millions of labeled images, to a CVPR paper this year.
- 2012: AlexNet takes the ImageNet competition.
- 2015, 2020 and 2023: YOLO, the Vision Transformer and Segment Anything arrive.
The 2012 result is the one to remember. The AlexNet authors wrote that a variant of their model achieved "a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry." Top 5 error means the right answer wasn't among the model's five best guesses.
The paper's network had 60 million parameters and five convolutional layers, and it trained on a GPU. That is why the field dates deep learning vision from this year. When was AI invented puts it on the wider timeline.
What does computer vision look like in three real cases?
Three cases from IBM's overview show the same four tasks at work. Each cites a named primary source, and none of them is a chatbot.
- A checkout with no queue. Amazon's Just Walk Out lets a shopper take items and leave. IBM files it under retail, where a camera system has to recognize each product, which is detection and classification working together.
- A chest X-ray. IBM's running example is a model trained on scans tagged normal or pneumonia. It cites a 2024 Scientific Reports paper saying that reading these scans is error prone and slow, because pneumonia looks like other lung conditions. That's a classification problem with a human expert still in charge.
- A robotic arm in orbit. NASA software gives operators on the International Space Station a real time estimate of the pose of a grappling target. That's detection plus geometry, and it has to run fast.
The pattern holds beyond these three. A camera feeds a model, the model returns a label, a box, an outline or text, and a person or another program acts on it. You'll find the weak spots in the same place every time: the data the model never saw.
How do ChatGPT, Claude and Gemini do computer vision now?
They take the image and your question into the same model and answer in text. That makes them multimodal, and it is a different design from a classifier that returns one label. Multimodal AI explains the idea in full.
The big three all handle several tasks from one upload, as of November 2026:
- Gemini's docs list captioning, classification and visual question answering, plus object detection with bounding boxes. They say that from Gemini 2.5, models also return contour masks. A request can carry up to 3,600 images.
- OpenAI's vision guide accepts PNG, JPEG, WEBP and non animated GIF files, sent as a link, a base64 string or a file ID. A request can carry up to 1,500 images and 512 MB.
- Claude takes uploads in claude.ai and images through its API. Anthropic's guide lists its limits on counting, location and diagnosis in the next section.
Use a chatbot when you want an explanation, a table built from a chart, or an answer to a specific question about one picture. Use a dedicated vision API, such as Cloud Vision, when you need labels with confidence scores for thousands of images, or an open model such as YOLO or Segment Anything when you need to run it yourself. A chatbot gives you a sentence. A vision API gives you data.
For the free caps on photo uploads, see AI that can analyze images. For reading text from pictures, see AI OCR.
Where does computer vision go wrong?
It goes wrong on counting, location and anything high stakes. Anthropic's vision guide says Claude can give approximate counts but might not be precise with many small objects, that its coordinate outputs are approximate and should be verified, and that it refuses to name people in images. It adds that Claude is not designed to interpret complex diagnostic scans such as CTs or MRIs, and its output is no substitute for professional medical advice.
Dedicated models have trade offs too. The YOLO paper says the model makes more localization errors than the best detectors of its day, and is less likely to predict a detection where no object exists.
One page in the current top results says machines never make mistakes. They do. A model trained on bright studio photos can fail on a dark, tilted phone shot, and a label with a confidence score is still a guess. Check anything you plan to act on.
What can you ask an AI about computer vision?
These six prompts work in Claude, ChatGPT, Gemini or Whizi. Fill the brackets.
- Test the four tasks: "I'm attaching a photo of [scene]. First classify what is in it, then list each object with where it sits, then transcribe any text exactly. Mark anything unclear as unclear."
- Explain a model: "Explain how [AlexNet, YOLO or a vision transformer] sees an image, at the level of a [grade or job]. Use one worked example and name its main weakness."
- Pick a tool: "I need to [read receipts, count stock on a shelf, outline products]. Name whether a chatbot, a cloud vision API or an open model fits best, with the one deciding reason."
- Audit an answer: "Here is what a model said about my photo: [paste]. Which claims could I check by eye, and which could it have invented?"
- Check a dataset: "I want to train a classifier for [categories]. List the data problems that would make it fail on new photos, such as lighting, angle and class balance."
- Compare approaches: "Compare a classical method and a deep learning method for [task]. Give one real strength of each and one case where the older one still wins."
Where does Whizi fit?
Whizi is one app instead of a pile of vision tools. You attach an image to a chat message and the model you picked looks at it. Pro carries Claude, GPT and Gemini, and its first week is $1.99.
Compare mode, which is on Pro and Powerhouse, runs two models on the same prompt side by side, so you can send one photo to two labs and read only the lines where they disagree. Web search is on by default in a new chat, which helps when a question about a model or dataset needs a current source. Scanned PDFs get a vision transcription pass.
The free plan gives 10 messages a day on Whizi AI, Auto and Claude Haiku, with no card. Whizi loses to a vision API when you need labels and boxes in bulk, and to a free local tool such as Tesseract when you only need text out of a file. See Whizi pricing for the plans.
- Name the task you need: classification, detection, segmentation or OCR
- Ask the model for the visible evidence behind each answer
- Recount objects and reread small text yourself
- Use a vision API or open model when you need data in bulk
- Keep diagnostic scans and ID of people out of chatbots
Frequently asked questions
Is computer vision the same as AI?
No. Computer vision is a branch of AI, the part that works on images and video. IBM calls it a subfield of AI, and it overlaps with machine learning, which supplies the models most systems now use.
What is computer vision used for?
It is used wherever a camera feeds a decision. IBM lists agriculture, autonomous vehicles, healthcare, manufacturing, retail, robotics and space exploration, with Amazon's Just Walk Out checkout and NASA's pose estimation on the International Space Station as named examples.
Do you need deep learning for computer vision?
No. Wikipedia traces edge extraction, optical flow and eigenface recognition to the 1970s through the 1990s, long before deep learning took over. Deep networks now lead on most benchmarks, but classical methods still exist.
Is computer vision the same as computer vision syndrome?
No. Computer vision syndrome is a different topic, about eyes and screen use, and it has nothing to do with software that reads images. Take that question to an eye doctor.