Vision and video

Perceptron, a company founded in November 2024 by two former Meta scientists, released Isaac 0.5 this week — a vision model its creators say gives machines the ability to "perceive, reason and act" in industrial settings. The software is aimed at robots with computer vision moving through warehouses and shop floors, and at companies that want to extract visual data from the video those robots…

The Toronto Blue Jays' account posted a short cartoon made as an advertising partnership with Nestea, the Nestlé drink, in the visual register of adult animated series like BoJack Horseman and Family Guy. It carries the familiar tells of hurried generative AI: hands drawn wrong and tangled together, lettering that resolves into nothing. A community note attached to the post listed them —…

Multimodal LLMs have learned to recognize objects, but how do you give them visual imagination? A look at the Cognitive Supersensing idea. Over the past few years, multimodal LLMs (MLLMs) have learned to recognize objects, read captions, answer questions about an image and even give a decent account of what is happening in a frame. But they have…

Agents that operate a computer keep failing at a step that looks trivial: finding the element on screen that a human instruction describes. That grounding is hardest on interfaces crowded with tiny controls, near-identical panels, high resolution, visual noise and rendering artifacts. The GroundCUA team shows how to solve this narrow but load-bearing problem — making the link between language…

When we ask a model to reason, it reasons in words if the medium is text, or over a static scene if the medium is an image. The world, though, is not static: objects move, and the rules often only become visible in how those objects behave over time. The authors propose video generation as a general-purpose channel for reasoning. Text can be written directly into the frames, visual hypotheses…

Making a game is more than getting code to run. It takes mechanics a player can grasp, art that looks decent, smooth animation and a steady 60 FPS. Large language models handle algorithmic problems confidently, but evaluations of their code rarely account for playability or aesthetics. The authors of V-GameGym set out to fill that gap: they assembled a realistic benchmark for visual game…