From the understated opulence of a Bentley to the stalwart family minivan to the utilitarian pickup truck, many Americans view the car one drives to be an outward statement of one’s personality. Now, researchers at Stanford University have taken that notion to a new level.
The research involved the analysis of millions of publicly available images on Google Street View by complex computer algorithms with the ability to see and learn. The researchers say they can use that knowledge to determine the political leanings of a given neighborhood just by observing the cars on the streets.
“Using easily obtainable visual data, we can learn so much about our communities, on par with some information that takes billions of dollars to obtain via census surveys. More importantly, this research opens up more possibilities of virtually continuous study of our society using sometimes cheaply available visual data,” said Fei-Fei Li, an associate professor of computer science at Stanford and director of the Stanford Artificial Intelligence Lab and the Stanford Vision Lab, where the work was completed. Li is an expert in computer vision and deep learning, a type of artificial intelligence in which computers teach themselves to recognize three-dimensional objects in two-dimensional images – computers that see, as she describes them.
The algorithms were programmed to train themselves to recognize the make, model, and year of every car produced since 1990 in each of more than 50 million Google Street View images from 200 U.S. cities. The data on car types and locations were then compared against the most comprehensive demographic database in use today, the American Community Survey, and against presidential election voting data to estimate demographic factors such as race, education, income, and voter preferences.
Li and her team found a simple linear relationship exists between cars, demographics, and political persuasion. The societal associations were described as “simple and powerful” in their paper published on Nov. 28 in the Proceedings of the National Academy of Sciences. For instance, if the number of sedans in a neighborhood is greater than the number of pickup trucks, there is an 88 percent chance that the precinct will vote Democratic. Invert those numbers to have more pickups than sedans and there is an 82 percent chance a precinct will vote Republican.
Beyond the obvious political repercussions, the researchers believe their algorithms could help provide more timely and continuous supplements to current demographic surveys. The American Community Survey is now conducted through expensive and labor-intensive door-to-door canvassing, which costs the U.S. over $250 million annually. Even worse than the high cost is the lag time between data collection and publication, which can stretch past two years or more, especially for small cities and rural areas.
By contrast, Li’s work piggybacks on a publicly available, regularly updated image database that is already built, paid for, and maintained by Google Street View, enabling her to generate analyses in near real time.
“I don’t see something like this replacing the American Community Survey, but as a supplement to keep the data up to date,” said Timnit Gebru, first author of the paper and formerly a member of Li’s lab. Gebru is now a postdoctoral researcher in the Fairness Accountability Transparency and Ethics (FATE) in Artificial Intelligence group at Microsoft Research.
Getting to this point was not easy, recalls Gebru. The team first had to build by hand an image database of all cars since 1990 – year, make, model, trim packages – and then teach a computer to recognize the subtle differences between cars in partially obscured and odd-angle images.
They initially started with a database of 15,000 cars from car-sale website Edmunds.com, but that was only the beginning. Human experts next had to categorize the cars one-by-one to even the most minute detail. The difference between the 2007 and 2008 Honda Accord, for example, is an almost-imperceptible change to the rear taillights. The algorithm worked fast, taking just two weeks to sort the cars in all 50 million images into 2,657 categories by make, model and year. A human working at a relatively high rate of six images a minute would need 15 years to complete the same task.
Some outside the research group were skeptical, Gebru said. They pointed to inconsistencies in the times of day when images were taken that might affect traffic patterns and vehicle makeup. In fact, Gebru adds, many Street View images are taken in the early morning hours specifically to avoid traffic, providing some consistency to the time of day when the images were taken. Regardless of the level of traffic, Gebru said, the images provided valuable data.
“If you walk around a neighborhood looking at cars, the density of traffic sometimes tells you things as valuable as the types of cars you see on the streets,” Gebru said. “We can use all this information in our algorithms.”
Gebru has high hopes for her new application. She is excited to move beyond demographics and use visual imagery analysis to improve surveys of hard-to-reach areas or for other beneficial uses like monitoring carbon dioxide levels and easing traffic congestion.
“It can help us understand how our society works, the things people need and how we can improve lives,” Li concurred. “There is great potential to use computer vision technology in a constructive and benevolent way.”
Timnit Gebru et al. “Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States,” Proceedings of the National Academy of Sciences (2017). DOI: 10.1073/pnas.1700035114
Provided by: Stanford University
* Expanded from original source by Andrew Myers published on November 29, 2017 at TechExplore.com. © Copyright 2017. All rights reserved.