The First Art Newspaper on the Net   Established in 1996 United States Thursday, June 21, 2018

Carnegie Mellon creates computerized method for matching images in photos, paintings, sketches
Photo of a car that was matched to the sketch of the car (see front page)..

PITTSBURGH, PA.- Computers can mimic the human ability to find visually similar images, such as photographs of a fountain in summer and in winter, or a photograph and a painting of the same cathedral, by using a technique that analyzes the uniqueness of images, say researchers at Carnegie Mellon University's School of Computer Science.

The research team, led by Alexei Efros, associate professor of computer science and robotics, and Abhinav Gupta, assistant research professor of robotics, found that their surprisingly simple technique performed well on a number of visual tasks that normally stump computers, including matching sketches of automobiles with photographs of cars.

The team from the Robotics Institute and Computer Science Department will present its findings on "data-driven uniqueness" on Dec. 14 at SIGGRAPH Asia, a computer graphics and interactive techniques conference in Hong Kong. Their research paper is available online.

Most computerized methods for matching images — in contrast to image searches based on keywords — focus on similarities in shapes, colors and composition. That approach has proven effective for finding exact or very close image matches and enabled successful applications such as Google Goggles.

But those methods can fail miserably when applied across different domains — photographs taken in different seasons or under different lighting conditions, or in different media, such as photographs, color paintings or black-and-white sketches. "The language of a painting is different than the language of a photograph," Efros explained. "Most computer methods latch onto the language, not on what's being said."

One problem, Gupta said, is that many images have strong elements, such as a cloud-filled sky, that may have superficial similarities to other images, but really only distract from what makes the image interesting to people. He and his collaborators hypothesized that it is instead the unique aspects of an image, in relation to other images being analyzed, that sets it apart and it is those elements that should be used to match it with similar images.

On the pixel level, a photo of a garden statue in the summer or fall will look very different than the same statue photographed in winter, said Abhinav Shrivastava, a master's degree student in robotics and first author of the research paper. But the unique aspects of the statue will carry over from a summer image to a winter image, or from a color photo to a sketch.

Estimating uniqueness is no simple task. The team computes uniqueness based on a very large data set of randomly selected images. Features that are unique are those that best discriminate one image from the rest of the random images. In a photo of a person in front of the Arc de Triomphe in Paris, for instance, the person likely is similar to people in other photos and thus would be given little weight in calculating uniqueness. The Arc itself, however, would be given greater weight because few photos include anything like it.

"We didn't expect this approach to work as well as it did," Efros acknowledged. "We don't know if this is anything like how humans compare images, but it's the best approximation we've been able to achieve."

In addition to automated image searches, this technique has applications to computational rephotography — the combination of historic photographs with modern-day photos taken from the same perspective. By using the new technique, it may be possible in many cases to eliminate the need for rephotography by simply matching the historic photo with an existing online photo that matches its perspective. Likewise, the technique can be combined with large GPS-tagged photo collections to determine the location where a particular painting of a landmark was painted.

The technique also can be used to assemble a "visual memex" — a data set that explores the visual similarities and contexts of a set of photos. For instance, the researchers downloaded 200 images of the Medici Fountain in Paris — paintings, historic photographs and recent snapshots from various seasons and taken from various distances and angles — and assembled them into a graph, as well as a YouTube video that shows a particular path through the data.

Future work includes using the technique to enhance object detection for computer vision and investigating ways to speed up the computationally intensive matching process.

Today's News

December 7, 2011

Städel Museum purchases portrait of Pope Julius II, one of Raphael's most famous works

Rediscovered Velazquez among old masters for sale at Christie's, Sotheby's and Bonhams

Tutankhamun Exhibition international tour finishes with 8 million visitors; sets Australian visitor record

Getty Museum acquires rare sculpture by the accomplished Master of the Harburger Altar

World Chess Hall of Fame presents highlights from the Dr. George and Vivian Dean Collection

Work by Eva Rothschild is first contemporary art acquisition for the Hepworth Wakefield

Charles Dickens' 200th birthday celebrated with coin made up of portrait with author's titles

9th annual New Art Dealers Alliance Miami Beach reports most successful year to date

18th annual SOFA Chicago 2011 and Intuit show of Folk & Outsider art wraps with successful edition

The Three Stooges in Three Little Beers movie poster brings $59,750 to lead movie poster auction

Art that tells the story of 900 years of Monarchy: A new eight-part series from BBC Radio 4

Collection of thirty seven Doctor Who costumes for sale at Bonhams' Entertainment Memorabilia auction

Subject in Kertész photograph discovers herself in image at Academy Art Museum exhibition

Carnegie Mellon creates computerized method for matching images in photos, paintings, sketches

Leslie Hindman Auctioneers' Fine Jewelry and Timepieces auction realizes $4.4 million

Ruben Ochoa's exhibition, Cores and Cutouts, at Locust Projects in Miami

Michener Art Museum looking ahead to expansion of education wing

David Zimmerman's "Last Refuge" opens at Sous Les Etoiles Gallery

3 charged for murder plot against Swedish artist

Exhibition of recent paintings by Serge Strosberg at Ludwig Trossaert Gallery in Antwerp

Most Popular Last Seven Days

1.- Porsche Super Speedster offered for first time in 50 years at RM Sotheby's Porsche 70th Anniversary Auction

2.- Cummer Museum of Art & Gardens opens 'Storytelling: French Art from the Horvitz Collection'

3.- Gauguin: Voyage to Tahiti stars Vincent Cassel as the famed French artist

4.- Stunning colored diamonds expected to dazzle at Heritage Auctions' Summer Fine Jewelry Auction

5.- US designer Kate Spade found dead at 55

6.- Vincent Van Gogh painting sells for over 7 million euros: Artcurial auction house

7.- Sir Stanley Spencer painting discovered hidden under a bed during a drugs raid

8.- Oxford's Bodleian Libraries unveil UK's first major Tolkien exhibition in decades

9.- Major exhibition at the Guggenheim explores decades of work by Alberto Giacometti

10.- World's largest freshwater pearl goes for 320,000 euros

Museums, Exhibits, Artists, Milestones, Digital Art, Architecture, Photography,
Photographers, Special Photos, Special Reports, Featured Stories, Auctions, Art Fairs,
Anecdotes, Art Quiz, Education, Mythology, 3D Images, Last Week, .


Ignacio Villarreal
Editor & Publisher:Jose Villarreal - Consultant: Ignacio Villarreal Jr.
Art Director: Juan José Sepúlveda Ramírez

Royalville Communications, Inc
Founder's Site. The most varied versions
of this beautiful prayer.
to a Mexican poet.

The First Art Newspaper on the Net. The Best Versions Of Ave Maria Song Junco de la Vega Site Ignacio Villarreal Site
Tell a Friend
Dear User, please complete the form below in order to recommend the Artdaily newsletter to someone you know.
Please complete all fields marked *.
Sending Mail
Sending Successful