147: Like Darth Vader, Google is unilaterally altering the deal with publishers

There’s a scene in Star Wars: The Empire Strikes Back where Darth Vader tells Lando Calrissian to take Chewbacca and Princess Leia to his shuttle. Lando complains that wasn’t the deal, and Darth Vader says, “I am altering the deal. Pray I do not alter it further.”

It’s funny that Darth Vader is, I think, the only character in Star Wars who encourages us to pray.

Anyway, Darth Vader is able to change the deal because he has all the power. The same is true with Google.

When Google says jump, content creators say “how high?” They change the deal all the time — by changing their algorithm — and we just go along with it. But recent changes aren’t just monkeying with the details. It’s a wholesale, radical change.

The original deal was this. Google was allowed to crawl our websites and create these incredible indexes (which we weren’t allowed to see) connected to complicated algorithms (which we weren’t allowed to see), and we permitted this because it was the primary means of content discovery. People used Google search to find our sites.

Publishers should have asserted their rights. They should have said “you can’t put our copyrighted information in your database and not tell us what you’re doing with it.” But they were, as usual, very short-sighted.

It was a bad deal, but it wasn’t that bad. And now, even that bad deal has changed.

Can you guess to whose advantage?

Remember when Wikipedia dominated the top spots in Google search results? If you were curious about The Battle of Trafalgar, for instance, you could be certain Wikipedia would be the first result.

It still is for that search (I just checked), but now it’s competing with a lot of answers that are provided right there on the search engine results page (which the cool kids call the SERP, by the way).

This has resulted in a significant decline in traffic to Wikipedia. Why go to Wikipedia when you get the answer right there on the results page?

From the user’s perspective, this is a great deal. Maybe all I wanted to know was a couple details, and if Google provides those details right there, that’s one fewer clicks for me, and I don’t have to scan through the Wikipedia page to find what I want.

But from Wikipedia’s perspective, it’s a disaster. Google is using Wikipedia’s content in a way Wikipedia never agreed to.

The only reason Google can answer the question is that it’s stealing the content from Wikipedia (and others) and using it in a way the authors never intended.

This is a fundamental shift in the terms of the deal. Google is no longer crawling content so that users can find that content, they’re crawling content so they can replace the publisher.

Did publishers agree to this? Were they consulted?

No. Google did it because it’s Darth Vader and it can do such things. It doesn’t need to ask our permission.

The problem is even worse with large language models. Pretty soon very few people will “search” in the way we used to think about that — where search was a discovery tool for publisher content. They’ll just get answers — derived from copyrighted material that was stolen from publishers.

I’ve singled out Google because Google has been the 800 pound gorilla in this game. They established the presumption that content on the internet was there for the taking, and publishers have foolishly gone along with that presumption.

Because of that presumption — that if content is out there on the internet, anybody can take it and do what they please with it — we now have ChatGPT and Bing and a host of other services that are stealing copyrighted information for the purpose of making the publisher irrelevant.

It has to stop.

Lando Calrissian is, in some ways, a good stand in for publishers. He’s a weak-kneed, go along kind of guy who loves the high life in his city in the clouds. But even Lando can be pushed too far.

Let’s see if publishers have even that spark of courage.