Web Crawling Quotes

Before Google, I spent the summer building a program that would look at what websites you would go to and what websites other people would go to - and built a collaborative filtering program that helped you find related sites to look at.

I think the search engines are the new equivalent of publishing: an enabler of information.

With Internet technology you can capture a photo, a quote, or an article, store it locally and upload it into the Net more than once, if you wish, to multiple sites. Can you imagine then forcing the search engines to somehow not index that information?

That is really not much different from the search engines that are being constructed today for users throughout the entire world to allow them to search through databases to access the information that they require.

All the information in the world has been pretty dispersed, but Google's mission has been to organize it and make it universally accessible.

It turns out a human being in two, three or four hours can build a search result that's much better than Google, Yahoo or Ask.

I'm interested in sites that help people find information and filter what's available. The Internet is so big that no one can stay on top of everything.

Google is all about information. So the notion of using and presenting information in the right point at the right time to users is what, in essence, describes Google.

After graduate school, I joined Johnson & Wales University in Rhode Island as an assistant professor, but I continued to program in addition to teaching and working on research. I built a program that crawled job boards to determine which skills employers value, which helped Johnson & Wales explore ways to improve its curriculum.

To me, the biggest surprise is that Google still functions despite the explosion in the number of sites.

The Web as we've known it for a long time has been pages linking and pointing to other pages.

Basically, we convert the entire Web into a big equation, with several hundred million variables, which are the page ranks of all the Web pages, and billions of terms, which are the links. And we're able to solve that equation.

People should understand that Google changes every day.

Google search was important - one of the most important applications ever on the Web. People accessed everything through a browser, and for us it was important for making sure we had an option there.

I'm on Twitter, I'm on HuffPo, just any website - you go down a rabbit hole. You start out on HuffPo, then you get a link to this, you get a link to that. Or an article, or a blog. One thing leads to another.

In spite of my own reservations about Bing's ability to convert Google users, I have to admit that the search engine does offer a genuine alternative to Google-style browsing, a more coherently organized selection of links, and a more advertiser-friendly environment through which to sell space and links.

The Google algorithm was a significant development. I've had thank-you emails from people whose lives have been saved by information on a medical website or who have found the love of their life on a dating website.

It has been aptly noted that web browsers are less Internet navigation tools than they are ebooks with highly diverse content.

There's nothing that cannot be found through some search engine or on the Internet somewhere.

If you don't find a new website when you Stumble, we've failed.

In comparison, Google is brilliant because it uses an algorithm that ranks Web pages by the number of links to them, with those links themselves valued by the number of links to their page of origin.

Google indexes the world's information.

Yandex originated from a company called Arkadia, which created two search programs under the DOS operating system in 1990.

The largest issue with search is that we learned about it when the web was young, when the universe was 'complete' - the entire web was searchable! Now our digital lives are utterly fractured - in apps, in walled gardens like Facebook, across clunky interfaces like those in automobiles or Comcast cable boxes.

We believe PulseRank will replace PageRank over time for the real-time Web.