Street View and reCAPTCHA technology just got smarter

Posted by Vinay Shet, Product Manager, reCAPTCHA 

Have you ever wondered how Google Maps knows the exact location of your neighborhood coffee shop? Or of the hotel you’re staying at next month? Translating a street address to an exact location on a map is harder than it seems. To take on this challenge and make Google Maps even more useful, we’ve been working on a new system to help locate addresses even more accurately, using some of the technology from the Street View and reCAPTCHA teams.

This technology finds and reads street numbers in Street View, and correlates those numbers with existing addresses to pinpoint their exact location on Google Maps. We’ve described these findings in a scientific paper at the International Conference on Learning Representations (ICLR). In this paper, we show that this system is able to accurately detect and read difficult numbers in Street View with 90% accuracy. 

Street View numbers correctly identified by the algorithm

These findings have surprising implications for spam and abuse protection on the Internet as well. For more than a decade, CAPTCHAs have used visual puzzles in the form of distorted text to help webmasters prevent automated software from engaging in abusive activities on their sites. Turns out that this new algorithm can also be used to read CAPTCHA puzzles—we found that it can decipher the hardest distorted text puzzles from reCAPTCHA with over 99% accuracy. This shows that the act of typing in the answer to a distorted image should not be the only factor when it comes to determining a human versus a machine.

Fortunately, Google’s reCAPTCHA has taken this into consideration, and reCAPTCHA is more secure today than ever before. Last year, we announced that we’ve significantly reduced our dependence on text distortions as the main differentiator between human and machine, and instead perform advanced risk analysis. This has also allowed us to simplify both our text CAPTCHAs as well as our audio CAPTCHAs, so that getting through this security measure is easy for humans, but still keeps websites protected.

CAPTCHA images correctly solved by the algorithm

Thanks to this research, we know that relying on distorted text alone isn’t enough. However, it’s important to note that simply identifying the text in CAPTCHA puzzles correctly doesn’t mean that reCAPTCHA itself is broken or ineffective. On the contrary, these findings have helped us build additional safeguards against bad actors in reCAPTCHA.

As the Street View and reCAPTCHA teams continue to work closely together, both will continue to improve, making Maps more precise and useful and reCAPTCHA safer and more effective. For more information, check out the reCAPTCHA site and the scientific paper from ICLR 2014.