The content previously published here has been withdrawn. We apologise for any inconvenience.

Google has changed its search engine indexing system.  And this, as you’ll see below, could have huge implications for blogger of all persuasions and shapes and sizes.

Called “Caffeine”, Google tells us that the new system provides 50 percent fresher results for web searches than its last index, and it’s the largest collection of web content they’ve ever offered.

The key here, and this is crucial for bloggers, is that any type of media (blog, other site, news story) will be indexed and available much much quicker following publication than ever before.  Yes, you as a blogger can get it out there, faster.

Rather than trying to paraphrase search engine technology in one paragraph here’s a good explanation from Google of how it all works.  But it’s interesting to really consider WHY Google have built a new search indexing system.  Let’s face it, this is like Coke altering its formula.

Well, Google says:

“….Content on the web is blossoming. It’s growing not just in size and numbers but with the advent of video, images, news and real-time updates, the average webpage is richer and more complex. In addition, people’s expectations for search are higher than they used to be. Searchers want to find the latest relevant content and publishers expect to be found the instant they publish.

To keep up with the evolution of the web and to meet rising user expectations, we’ve built Caffeine. The image below illustrates how our old indexing system worked compared to Caffeine:

Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.

With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before—no matter when or where it was published.

Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.

We’ve built Caffeine with the future in mind. Not only is it fresher, it’s a robust foundation that makes it possible for us to build an even faster and comprehensive search engine that scales with the growth of information online, and delivers even more relevant search results to you. So stay tuned, and look for more improvements in the months to come.”

Wow, that’s a lot to take in. But getting under the skin of the basic reality of it all, here, this is our take:

Twitter has been hammering home the benefits of ‘nearly live’ search for some time.  People tweet – it’s right there.  And we’ve been waiting for Google to catch-up in terms of “live search”, despite them doing a deal with Twitter previously to use Twitter material in the Google search results.

Well, it looks like they’re going a long way down the road of doing just that.  And for bloggers, this is potentially huge.  Imagine being able to have your content indexed much quicker, available to your potential audience much quicker.  And if you’re a business blogger, you don’t need to have it spelled out to you how much impact this could have for you.

We’ll be keeping a really close eye on this – look out for several more posts in the near future.

Bye for now, CB

As you’ll probably know, the huge success of Twitter has been built on a number of key foundations, including near-instant (short form) communication.

Quite recently, Bing tied-up a deal with Twitter. Guess what? Google wasn’t going to be far behind. Surprise, surprise: not.

Twitter’s tweets, an amazing feed of ‘live’, topical, content-rich data is going to be crucial to Google’s search results. Whether it’s the latest news, celebrity gossip or details of inter-planetary collision

From the Google blog:

“Given this new type of information and its value to search, we are very excited to announce that we have reached an agreement with Twitter to include their updates in our search results. We believe that our search results and user experience will greatly benefit from the inclusion of this up-to-the-minute data, and we look forward to having a product that showcases how tweets can make search better in the coming months. That way, the next time you search for something that can be aided by a real-time observation, say, snow conditions at your favorite ski resort, you’ll find tweets from other users who are there and sharing the latest and greatest information.”

See more here for how relevance and real-time have hooked up – big time.

This kind of stole the thunder of Bing…

For Twitter, though, a site with massive usage but a, let’s say, ‘stretched’ business plan, this is amazing news and a tribute to the founder Jack Dorsey and those who run the thing. It’s also great news for those CIOs of major businesses and others in many other areas of life who took a step off the precipice and decided to integrate Twitter functionality/interactivity into their own ventures, seeing it as much, much more than a passing fad.

Put the world’s best search engine together with a dynamic, fresh source of data from across the planet: sound like a potential success to anyone?

Google advertising represents a massive slice of the Internet’s global revenue base; the AdSense Software contributes a large percentage to Google’s annual multi-billion dollar turnover.

There was real confusion in the early days of the internet as to how a search engine, or other business, would sensibly make money from (“monetize”) traffic and, particularly, allow third party advertisers of a range of sizes to get their ads onto the web and for others to make money from displaying them.  The real genius of AdSense was to link the advertisers and the advertising ‘hosts’ (websites displaying AdSense adverts), using its amazing Google search engine as the platform to pull both sides together.  Google advertising was born.

In one relatively simple (in hindsight!) step, Google completely revolutionized the ad market on the internet and gave substance to the “click-through” advertising market. It was bold and innovative. It was the start of a multi-billion dollar cash cow. A revolution

How Does it Work?

Users sign up their websites for affiliation with Google.  When approved, every site which displays AdSense ads is given special code to place targeted adverts on the website.  The adverts are specifically targeted to the CONTENT of the website.  Crucial to Google’s model of delivering relevance.

As well as delivering real relevance, the holy grail for every internet user, this approach saved Google from the hassle, web page space and administration associated with managing banner adverts.  Sweet, indeed.

In its absolute essence, AdSense is a software program by virtue of which Google will pay you for targeting/directing the users of your website to another website.  Advertising in its simplest, neatest, sense.

• A “click through” occurs when a visitor clicks on a hyperlink which links to an advert.
• The AdSense code used by Google now offers those who display AdSense code an assortment of methods of displaying ads: from text links to graphic links, banner ads to video ads (each having a hyperlink in-built automatically by the AdSense code).
• Those who display AdSense put these ‘boxes’ in appropriate places on their websites. They get clicked. Click = cash for the site owner displaying the relevant, useful ads.  Win, win.

But How Does This Link to AdWords?

What determines the amount paid by the AdWords advertiser is how much the AdWords advertiser is prepared to pay for each click.  The more you pay, the better placed the ads, the more often they’ll be displayed, the more clicks…you get it!  There’s real competition for ‘hot’ keywords.

There is a correlation between how much an AdWords advertiser has paid to display their ad and how much an AdSense click delivers.  AdWords advertisers can pay a few cents per click right up to $10 or more. You can see, therefore, that the displayer of AdSense can take a ‘slice’ of an expensive ad or a relatively cheap one and the more traffic you receive will increase the possibility of a click and a click through fee into the AdSense account.

We’ll come back to this soon.  Enjoy experimenting.