235 points | by bwb13 小时前
2) Google has many incentives to make the search more difficult for you
3) Google has proven that it prefers money over quality of results with allowing "malvertising"
4) It is true that the landscape is more difficult. There are more walled gardens, to which even Google might not have access. There are more scams, casinos. More AI slop. The game was always hard on the other hand, so these are just 'excuses'
5) Why so often I see in Google results leading to major news sites instead of normal links?
6) If I write "Warhammer" I would expect thousands and thousands of pages in results. I think that Google prefers "content" over "quality". "newer" is "better". I would expect thousands of fan pages, which do exists, but are not crawled, or forgotten. Why can't I browse older pages? Why is there a limit to 10 pages?
7) For "Emulation" first page leads to "wikipedia", "cambridge dictionary", "vocabulary", it is so f boring
[1] https://www.404media.co/google-search-really-has-gotten-wors...
The example list given just looks a lot like spam when you squint. It's a list of affiliate links to buy products, and there are many HN threads talking about the abundance of affiliate link aggregators being a blight on the web. The commentary does look useful, but distinguishing between good commentary and bad commentary is hard, whereas distinguishing between a site designed to extract affiliate commission vs one more about the content is easier.
The comparison given to the other results here is frustrating, I know, but probably not a valid experiment. All the major search engines change results based on the user using them, or the IP address, or the region, or whatever, so it's impossible to know what others see. The developer of a book-focused shopping site is likely to get very skewed results for a book related query. My results were noticeably better.
The author says that a Bookshop.org list they created that links back to Shepherd is ranking #2, and this kinda makes sense to me. Bookshop.org sells the books, it makes sense that would rank above a site that only links to (and makes money from) sites that sell books.
SEO, and people getting annoyed at not ranking, has been a thing for 25+ years, I don't think this instance is any different.
"People have always complained about X" is always a bad argument: it's entirely possible that X has gotten much worse, and yes, there were complaints before and after.
In this case, there is a very loud chorus of people saying Google has gotten much worse in the last few years, and it certainly matches my experience.
I had to do some deep breathing before responding (creator of Shepherd here) :)
When I squint while reading a Pulitzer Prize-winning book, it looks like every other book with words everywhere.
When my wife really squints, she says I "look" like Brad Pitt.
I say these things with humor... as the core of what Google is supposed to do is determine what is good content and what is spam.
Is that hard?
Yes, but you have a lot of smart people and are swimming in money.
The point here, and with all the other posts by indie publishers, is that Google has destroyed them while elevating much worse results (they used to rank just fine along with other accurate results). I am just one example of many, and sadly better than most because books are a passion and not that commercial. For indie websites scientifically reviewing air purification systems, they are destroyed, and now you just get Forbes spam where nobody reviewed it.
My suggestion for Google is to stop squinting :)
In other words google is giving preference to a site that probably pays them for ads to a site that is effectively a competitor (since the site would make revenue from affiliate links that google might have gotten from ads, especially if the user liked the site and went directly there for future book recommendations).
Also, if I am searching for the "best" of something, I want something with commentary, not a list from a vendor without relevant information for choosing the right product.
One idea I've had is that it would be interesting if Google created a webmaster partner program to help build the web they want to see. Webmasters could join and embed Google's code into their website, and Google could get access to engagement stats and anything else to help them determine positive reactions by users (instead of pretending they are not pulling it from Chrome - see DOJ trial).
At YouTube, Google does a great job of helping creators, telling them specifically what they want, and promoting high-quality content that people love.
I'd love to see Google get the data they need so they can do that.
I personally hardly ever use any version of streetview because I don’t feel the need to so forgive me if I am missing what’s so great about Google’s version.
I stopped using Google for web search, but I still use it for Google search.
I do see the Google info box style results, but I find these to be one of the most useful parts and it's one of the reasons I like using Google for things that are basic facts, media fact finding (like "who was that person in that show" style queries), etc.
- Google made me ruin a perfectly good website (2023) / https://theluddite.org/#!post/google-ads
- https://news.ycombinator.com/item?id=40184673 (HN comments)
I'm specifically thinking about this line:
> It's a list of affiliate links to buy products, and there are many HN threads talking about the abundance of affiliate link aggregators being a blight on the web.
This is a pattern that was made hugely popular specifically by google, by the way
Is this really the sort of content Google should be returning?
Edit: it seems I missed the link to the actual book reviews because the link text is uninformative: "Chosen by 1 person - see why." (Sometimes it goes to reviews, sometimes not.)
And the word "review" never appears on the pages that have reviews. Seems like bad SEO?
If you're looking for book reviews, here's a website with some pretty great reviews: https://www.thepsmiths.com/ (Content warning: the authors are conservative.)
Given the nightmare that is Goodread's unmoderated review sludge, I wanted to focus on the positivity and why someone loved a book, so we are uniquely focused on highlighting only books that readers love.
In 2025 I will be adding "reviews" of any books using our Book DNA review format (which tries to narrow in on why someone liked/disliked a book so we can match you with people who share your Book DNA). A big part of my mission is to focus only on books that people love and positivity.
What is Google doing?
A lot of us believe that Google trained their AI on spam websites, but in the process, the AI now identifies a lot of things that independent websites have on them versus big brands like Forbes, which spam and license out huge sections of their websites to spammers. So we got pulled into this and Google doesn't care about the false positives.
At this point, a lot of websites are not coming back. So get used to Forbes and other large brands being the only results along with AI bots on Quora.
That's the thing. It DID work. Really well for a while. But it was always atomic and context-less. We now have the opportunity to make it even better by refining results through dialogue. I hope someone does.. soon.
The problem is simply that there's too much money to be made by capturing these. For years, you had content farms and fake "review" sites stepping up their game. Now, LLMs essentially make it a losing proposition to try and surface the small web. The least-bad option for Google would be to send you to moderated communities, such as Reddit, Quora, Stack Exchange, Wikipedia, and so on. But not all queries can be handled that way.
If you look at the article that these guys are complaining about... how do you distinguish it from content-farmed spam? You can't.
Now, I think Google is throwing in the towel and just want an LLM to answer info queries instead. That has a ton of problems, but to the average user, probably feels more helpful. At least until the spammers start gaming that.
You can, Google just hasn't bothered to even try. As an example it's pretty easy to detect affiliate marketing links.
Plus, even without affiliate links, purchase attribution is already scary good. Did you know that credit card issuers and ad tech companies collaborate to attribute brick-and-mortar purchases to online ad views? You have untold billions of dollars at stake. The industry is not banging rocks together.
You could make a search engine for non-commercial content only, and I would actually love to have that, but (a) it would be a woefully tiny sliver of the internet; and (b) Google is obviously not the right company to do it.
So, there is a huge problem with hiring experts and telling them to make subjective decisions. Instead, Google publishes guidelines for webmasters and then enforces them rigidly. This usually ends up penalizing some good sites, while spammers swiftly discover workarounds.
This is now a bygone era - after discarding their original motto of "don't be evil," search and "organizing the world's information" are no longer Google's business, it's hawking advertisements [0]:
When Gomes pushed back on the multiple requests for growth, Fox added that all three
of them were responsible for search, that search was “the revenue engine of the
company,” and that bartering with the ads and finance teams was potentially “the new
reality of their jobs.”
On February 6th 2019, Gomes said that he believed that search was “getting too close
to the money,” and ended his email by saying that he was “concerned that growth is all
that Google was thinking about.”
If you want to be treated as a customer instead of cattle, you ought then to pay for your services, including search, to align the financial incentives with your own. Advertising needs to die, for it is a root cause of most of the ills of the modern internet [1]. If you can pay for streaming services or music, you can certainly pay for access to high quality organic information that actually aligns with your interests - not that of the advertisers. I've been using Kagi for a few years now and it really does hearken back to Google SERP quality maybe not at its peak, but rounding near to it.
At the risk of sounding elitist (and so what), this is just another consequence of the recurring Eternal September phenomenon - highly focused communities with a strong concentration of geeks, hobbyists, and experts were the norm back then, when computers were still new and arcane devices that were difficult to operate. The bar to entry was much higher, and one had to do a little bit of "reading the fucking manual" simply to get online and understand how to navigate the net effectively. Now that all the balls have been poured into the Galton board we have regressed to the mediocrity of content that exists on the contemporary Web, absent those pressures that once selected for high quality content online.
People pay for their Windows license, yet Windows now has ads baked into the start menu. People pay for Youtube Premium, but most videos now have "sponsor segments" -- yet more ads (though admittedly not controlled by or directly profiting YT). People pay for streaming services, but last I heard, Netflix was adding ads. Ages ago, people paid for cable television, and it wasn't long before it had ads too.
These companies are going to treat you like cattle whether you're paying them directly or not.
after MBA's start to get diminishing returns on new subscriptions per month, the focus shifts to advertisements.
There's a wide gulf between "path of least resistance" and "least-bad". I don't think it's productive to conflate the first path with the second.
No, often it absolutely doesn't, and I posted two epic fails here previously. Worse still, Google organic results no longer understand(/distinguish) the difference between information-seeking queries vs product searches (or else, SEO people have been gaming it for years, and Google search rankings have made this worse):
____________________________________________________________
1) Google search relevance fail: result for “Africa longitude” https://news.ycombinator.com/item?id=30337563
Googling for "Africa longitude" should return a range of longitudes, like: "17.5°W - 51.5°E" or "["17°31′13″W - 51°27′52″E"]" [1], or for the A+ answer: *"Africa lies between 17°33'22" W, (Cape Verde, westernmost point) and 51°27'52" E, (Ras Hafun, Somalia, easternmost projection).
But it doesn't. Moreover these coordinates haven't substantially changed in 10,000 years (other than political/territorial disputes about islands, but the coords for the mainland certainyl haven't).
Googling returns the grossly misleading "Africa/Coordinates 8.7832° S, 34.5085° E". When you dig into why this so, it seems to be "optimized" for the SEO activities of a digital map storefront, MapsofWorld.com, acquired by MapSherpa Inc., based in Ottawa. And those mystery nonsense coordinates ("8.7832° S, 34.5085° E") bizarrely point not even to the geographical centre of Africa but to a random rural location 2400km ESE away, in southern Tanzania, which appears to have been deceptively mislabeled, in Cyrillic, as a Russian store (by Russian SEO?). For a pin dropped in rural Tanzania. No QA!
https://news.ycombinator.com/item?id=30337563
____________________________________________________________
Similarly:
2) Google search for "How many landings on dark side of moon" is grossly incorrect https://news.ycombinator.com/item?id=40862146
(Update: the AI overview factbox has at least since been corrected to give Two: "Chang'e 4 (January 3, 2019) and Chang'e 6 (2024), instead of repeating what jagranjosh.com says).
However the #1 organic search hit is still the woefully inaccurate unauthoritative page: https://www.jagranjosh.com/general-knowledge/list-of-all-suc... which cheerfully claims "Aug 23, 2023 — There are over 21[!!] moon missions that have been launched successfully on the dark side of the moon by 4 countries."
This is hopelessly wrong, even if we utterly misunderstood the key word "landing" and also count any mission which merely photographed the dark side (Luna 3, 1959) or human overflight over it (Apollo 8, 1968). But not "landing".
And why on earth did Google decide jagranjosh.com was more authoritative than any reference website or wiki?
____________________________________________________________
[1] https://en.wikipedia.org/wiki/Africa#Geology,_geography,_eco...
And searching google.com for "longitudes continent africa" returns "Africa's geographical coordinates span from Latitude 37˚21'N to 34˚51'15"S and Longitude 51˚27'52"E to 17˚33'22"W"
It seems that GIGO still holds true.
Maybe I'm naive but... has Google even considered doing that? Re the first one, I know top management had a project to reenter China back in the teens.
From the earliest papers on Google PageRank, Brin and Page warned, "we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers." <http://infolab.stanford.edu/~backrub/google.html>
TBH, I don't want dialog (too long and slow, leave dialogs for human to human comms), I just need my query words to exist in the result pages. Nothing more. Google messed this up long time ago.
Google didn't mess that up, SEO did.
You don't "just need" that. You need the page to be authored by a human with good intentions and end up containing those words because those are meaningfully part of what the human was trying to convey to you without ulterior motives.
But those kinds of pages are dwarfed by the millions of pages of content farm nonsense jammed full of every possible keyword and contaning absolutely nothing of value.
A search engine like you describe would be like walking in Chernobyl without a radiation suit on. A pleasant stroll in 1985, but not today.
Substack is a great platform and I love them. I also use Beehiiv for our mailing lists with authors/readers.
I was on Twitter's free email thing but they shut down and imported to Substack. I just didn't want to pay for anything as the only person who reads it is me and my mom. It has 100 readers on it, as its just for me really.
https://kagi.com/search?q=best+books+on+Battle+of+Midway&r=u...
https://search.brave.com/search?q=best+books+on+the+battle+o...
Get your friends and family and other internet users to do the same.
The only thing likely to get their attention is if enough people follow suit.
Just do the thing and the problem is gone.
Very similar to “Draw the rest of the owl.”
The problem is people don’t “just”. We’re complicated critters with our own needs, capabilities, and goals. At an individual level, a person may not be capable of the “just” due to factors outside of their control. With a society, the resistance to change grows exponentially. You can’t scale one person at a time.
This doesn’t mean you shouldn’t try. Sometimes you can make a big difference to the people around you. But you’re gonna have a difficult time changing in the world.
Quicker and easier than "Draw the rest of the owl" and not outside anyone's control.
Yes and no. There's certainly the potential for a more flexible and adapt-minded culture for the same reason there's not... Human behavior.
Yes, once norms are established they're slow to change but that's how the culture is nudged and "managed". It's not necessarily how humans arw hardwired.
Eons ago, slow to adapt would mean certain death. Currently, fast to adapt labels you a rebel, a freak, an odd-ball, etc.
That is all I meant when I said society.
"Simple answer, just stop electing corrupt politicans"
"Simple answer, just stop war"
k
Not saying calling for a boycott is enough but it’s easy to avoid Google nowadays.
If it says "Google", you really don't want it.
Some may be a little slow to come around to this realization but it's growing.
For example, more than half of internet users are now taking active steps to block "personalized advertising" which is Google's bread and butter.
Edit to add: and since we're talking about indie web: https://search.marginalia.nu/
When someone complains to you directly about Google search results being shit, maybe point them to Kagi and mention it has a free trial account (100 searches still?) that's not time limited.
That way they'll have a solution they can turn to the next time Google pisses them off significantly. :)
https://nautilus.org/napsnet/napsnet-special-reports/the-rol...
(duckduckgo.com returned it as the 5th returned result (not bad)),
Thanks!
Using a different search engine, is just about the easiest kind of switch you could do. It's easier than, say, eating a different type of snack than you usually eat, or switching out your usual mug/plate/utensil.
Ok another note, your karma is very close to 2381. Make sure you get a screenshot :)
This morning Google couldn't find it, neither could Bing.
[1] https://mikewarot.blogspot.com/2007/10/mikes-law-of-intermit...
https://kagi.com/search?q=%22Mike%27s+law+of+intermittent+ha...
(granted, I had to perform search with quotes on, but both Google and Bing still fail)
Hopefully demonstrates the difference between legacy and a modern search engine whose entire purpose of existence is to surface what you want to find, not what other people want you to see. (wave from Kagi founder here)
AyyEye's observation that their site is loaded with trackers and what look a lot like affiliate marketing links is one reason why Google Search might not like it any more.
Google are unreliable and untrustworthy. Their focus is ad revenue for themselves and nothing else. Build sites for humans and let Google do what it wants to.
I know a bunch flocked to Google early on. But the landscape has changed a lot since then, and many people weren't even born until after the Internet and society were very different.
I worry about that company.
It felt like I was fighting some AI that was sure it knew what I wanted despite my "exact phrase" search.
Then checked my mailchimp subscriptions
and my grow.me subscriptions
And my substack subscriptions
Made sure my Cloudinary was properly configured
And my newrelic analytics
And my sentry analytics
And my rlcdn analytics
And my growplow analytics
And my 33across analytics
And my Scorecard research analytics
And my openxcdn analytics
And my trustarc analytics
And my creativecdn analytics
And my Google Tag Manager
And my Google Analytics
Then I checked my Mediavine ads
And my adsrvr ads
And my adform ads
And my adnxs ads
And my yieldmo ads
And my criteo ads
And my mediavine ads
And my pubmatic ads
And my id5-sync ads
And my rubicon-project ads
And my triplelift ads
And my pghub ads
And my zemanta ads
And my cognitivlabs ads
And my doubleverify ads
And my media.net ads
And my kargo ads
And my Amazon ads
And my Google Adsense
When I finally got to WeWork
I booted up my Macbook
And checked my Google Workspace email
Our Amazon Affiliate account was approved
And I checked our private Github issues for tasks
Then I let Microsoft CoPilot write Stripe integration
I write a new post for my Substack
Finally using my Chrome browser and ycombinator's platform I posted:
'Google is killing the independent web'
Should he have clarified "properties not backed by tens, hundreds, thousands, or millions of millions of dollars?"
That's less than half of the malware it tried to run on my computer but I got bored checking all the domains it tried to run code from. It's repetitive. I get it, you put on every tracker and ad network you could find, and put your stuff on a bunch of CDNs.
Everything that can be outsourced to big tech was outsourced to big tech. How very independent.
Independent means that we are a small team and trying to create something cool for people. It means we are not part of Forbes or some giant publication that is focused purely on money and spamming the shit out of you :). You want a diverse and indie web so that the only search results and voices you here are Forbes, Disney, and other giant media conglomerates.
Independent does not mean we don't eat.
Independent does not mean we don't have server bills.
Independent does not mean our part-time developer works for free.
Independent does not mean our designer works for free.
There is no malware on the website crazy; it's just an ad network that pays for our servers. I will remove all those ugly display ads as soon as we have enough income from readers and authors directly.
I got in my friends Telsa
Waiting at the traffic lights started looking at Twitter
I got home and asked Siri
For food from UberEats
Only to get bored and open a tab of Reddit
Followed by television series from Netflix
With food from KFC
I then decided to go for a pee
Looked in the mirror
And questioned who is me.
I wonder if Prabhakar Raghavan was moved out of Google Search because of Search's decline in quality?
I used to love crafting websites and cared about SEO. What's the point now, no one is going to find your content. It won't even be on the third page. Google will answer questions by regurgitating whatever it swallowed on your websites and presume no one will click through, it won't even bother marking the authors.
Instead it appears to be prioritising whichever website is going to give it revenue first, e.g. the click farms.
The regular folks don't care, they google for stuff like "am I dying if I have a pimple?" (to which the answer is always yes, apparently). No one does actual meaningful research using Google anymore, if you do, good luck, get your gloves out <picture of dinosaur poop in Jurassic Park>.
The global internet as it stands is close to dead. Discoverability of "cool" things is down to social media, tricked by "influencers", who are tricked by marketing themselves.
We need a hard reset button, it needs to start from the ground up with site rings, and good content. Ah... that last part, "good content", is now stuffed with AI Samey McSamey sounding text. I really don't see a way out of here.
The funny part is we used to think that the internet was going to change the world. We thought all idiots needed was information. Access to information would fix the world! Instead, it only has given the village idiots a global voice: if you can think of some dumb crazy thing, you'll find dumb crazy people agreeing with you, so you must be right!
I've been on the internet since 1997 and I think it's the worst it's ever been.
As far as query trackers on google urls, they are necessary
One good case of necessity of google tracking parameters is --> you search for movie or related, but muktiple movies by same name have been made , you dont remember year, you do remember some actor or story description Searxg and google will in show with media carda foe movies you ckick iy and yoi get exact same film with year yoi wantes
Wait ->I am telling above because when that mefia image of film i inspecred itsurl it was just Googke ?search=filmname&teackers The search ket wasnt modified with year same as i searched first yet it gave me the desires year film
may be because of trackers &ved amd all paramers link tags to search teaukts its loke ctoss maching in databases
It is too big to evem worry about that 4k a day clicks for one site. It is like us optimizing the expense of 0.01c. It makes a difference when that 0.01c is an API call that you call a million times. But it only surfaces if you do aggregate it.
Therefore this problem can only even be seem by Google if it can be surfaced in aggregate overy say a billion queries.
I wonder how that can be done.
Probably only can be done using data. Which means spying on people in various ways. And making assumptions about length of time on site equals quality.
They probably use machine learning too. There may be no reason for the lost rankings other than a wind change caused by some updated parameters in an OKR chasing model.
I mean when you say "Google can't please all of the sites all of the time, or all the visitors.", I wholeheartedly agree, but this blog post was excellently sourced with data that shows exactly how Google raising sites that any reasonable human would say are considerably shittier than this site that is getting down ranked. It also seems pretty clear that the things that have changed are Google's ranking algorithm at specific points.
> They probably use machine learning too. There may be no reason for the lost rankings other than a wind change caused by some updated parameters in an OKR chasing model.
That is literally what TFA says in the very first section: "Some people believe they have lost control of their AI ranking systems, ..."
Whenever this point comes up, I see people claim they ONLY see the results they want in Google. How would you know if you don't actually use anything else? Kagi is excellent search. Neeva was pretty great when it was active. DuckDuckGo is passable. Idk how Qwant gets money but it's been around a bit.
Complaining about the same thing forever and expecting a change doesn't make any sense. Y'all are in abusive relationships with Google and refuse to leave. Sure, your job may use Google Suite and you need to make money. What about the rest of your life? Stop hitting yourself.
Been working on it for 2 years now and even then I'm more or less locked into 3 services
1. Gmail, 20 year old account I use for pretty much everything business. Even if I move I'll need to forward Gmail stuff to a new email for years.
2. YouTube. Pretty self explanatory (and the go to for why monopolies are always bad). Trying to avoid it entirely is like trying to avoid dang Twitter. Too many other companies use it as a go to for any video, no matter how inefficient (nothing better for graphical showcases than nitrate compression ruining all the details)
3. Play store. Used android all my life and while I can mostly move out I will be missing some critical apps from that result (financial apps are a big example)
It's a network effect like any other for 2 of those, and a lock in part of my online identity for another.
I use Kagi, but I am a nerd.
I will boost this. I am very happy with Kagi. I'm on ultimate and am glad to have my money taken for reliable and high-quality search.
> Complaining about the same thing forever and expecting a change doesn't make any sense. Y'all are in abusive relationships with Google and refuse to leave.
This would be a more solid stance if they were complaining about this as a consumer and not as a business operator. They can't control what search engines other people use. The best they can do is optimise for other engines like Bing and DuckDuckGo.
The other alternative is finding other avenues to advertise away from search engines, which may be what you were alluding to.
I was actually speaking from the consumer perspective, thanks for making the distinction. The OP has valid reasons for being upset with the most popular search engine ruining visibility.
To be fair, I have no idea what business could/should do. Bigger businesses can afford to hire social media folks to keep them relevant on Facebook or whatever. SMBs/mom & pops are screwed.
I didn't think so either, coming from Neeva. Of course, YMMV.
> I have yet to se anything that is consistently qualitatively better.
And you won't, until you try.
It’ll give you time to get used to Kagi and set up some personal up- and downranks and blocks. Especially together with the Summarizer and Lenses your search result quality will dramatically improve.
Do you know that this forum isn't one person and thousands of people access it in different days and time, right? The notion that the same people are complaining while still using Google is a projection of your mind.
I put a fair bit of time into trying to improve the sites with JSON-LD and breadcrumbs and what not. It seems to have helped just a little bit.
I don't make any money off any of it, but it's still kind of irritating that no one can find it.
But its just one data point and nobody knows.
It seems more likely at this stage that Google has lost control of their AI system. they trained it on x websites, and indie websites got swept into that. And they don't care.
How’s shepherd.com (for example) doing on bing or duckduckgo?
Maybe it’s time for google to die.
If small publishers don't change their behavior in radical ways, nothing will ever change.
That's probably what MySpace people thought. And Friendster. And Altavista. And ...
If you want content made by humans to continue to exist and want to see some of them be able to make a buck off of it, support smaller search engines and creative people and their publications.
I could understand your complaint if it was about image search, but I don't see how anyone can look at today's Google results and think they're still superior.
Its only serious competitor is Bing, which isn't even a search engine anymore but a billboard for Microsoft to advertise ChatGPT.
Happiness.
Contentment.
A workforce that is motivated and happy.
People don't want just money.
Company: No.
Big companies have pivoted before on the heels of brave executives. This applies as much to search as it does to privacy, AI, their transparency, their support of open source, and their weaponization of their browser.
It's a crying shame to see how enshittified a company that could be changing the world has become.
Line must always be going up, and if it doesn't go up as fast as someone thought it would then we make it go down.
If line goes sideways we sell and make it go down.
It must go up. Always up. Or it will go down.