There’s a new digital divide, a fissure opening wider and wider as the social web makes encroachment into most forms of information. You may have heard of the ‘darknets’ — unseen networks of computers for filesharing — networks you’re only allowed onto if you’re trusted not to give the game away. What I think I’m seeing the emergence of is almost the exact opposite, but increasingly disconnected.

There are hundreds of websites, lovingly researched and maintained by enthusiastic and knowledgeable people, that it’s becoming almost impossible to find. The sites are built on old technology, and that contributes to their decreasing visibility but it’s not the only reason. The lack of RSS feeds, pinging servers, dynamically generated sitemaps and up-to-date robots.txt files makes it more difficult for other sites to keep in touch with them. That they are often built in HTML by hand makes them more difficult to update, and fresh content is prized by search engines.

The lack of RSS and knowing when updates occur also decreases people’s awareness of the sites, you either have to remember they’re there and how you found them — or bookmark — and check for updates on a regular schedule. Less reminders, less nudges, so less incoming links.

They are often in very niche areas, like local history, local news, and so generate a limited number of hyperlinks from other sites. They often only get links from each other, which is great in a community sense but means the incoming links are low in pagerank (which would push them higher up Google searches).

As more and more sites get better and better at search engine optimisation, as blogs and other social websites link and link again and expand into more areas, and as Google relies on the same sources more and more the sites are getting less and less visible.

And that’s bad because they have a wealth of important content that we need to be able to find.

I’m calling it the ‘hinternet‘.

(From hinterland, in German the part of a country where only few people live and where the infrastructure is underdeveloped.)

Solving the problem is a tricky one: Google’s mission to index the entire World’s information doesn’t always mean that we can find what we’re searching for, the semantic web will only work if the correct metadata is stored with the hinternet sites (and they’re already often “behind” technology-wise).

Search needs to get better, but us on the social-web also need to help. We need not only to link to these sites, but — where we can — help nudge the guardians of the hinternet towards greater visability by becoming “social”.

13 Responses

  1. And bloggers can play a huge part in raising the profile by writing and posting about these sites as they find them. The posts should provide a well ranked, permanent sign post to the “less optimised” sites.

  2. Is what happened when we went digging for Phyllis Nicklin’s photos a good example of the social web in action? Here were amazing photos that were rediscovered, put on the web and then kind of lost again. The old web put them behind a barrier and prevented access, something the social web prizes most of all.

    The link on the CIB story needs updating. This is the one that works (for me at least):


  3. I would wonder if, despite the growing active population of the net, such relatively-static sites actually have far bigger audiences than they did three years ago? But that the wash and swell of the social media tidal-wave makes them _seem_ as if they’re getting covered up and ignored?

    But, if you’re correct, then there are opportunities in such a situation for building new solutions for people who want to find only such information-rich and lovingly-tended websites. Several methods suggest themselves: 1) people learn to actually use keyphrases and search-engines in a deep and disciplined way (instead of seeing Google as a “magic oracle” they just type two words into, and the answer comes back in the top-three results); 2) niche hand-built search-engines ( springs to mind immediately) which are properly funded as a public service; 3) new forms of aggregation of links to intelligent content (perhaps via the likes of Yahoo Pipes, etc), including from guilds of same-subject bloggers; 4) Google has shown itself able to create popular and useful subset searches such as Google Scholar, so why not a ‘Google Deep’ or a ‘Google Sparkle’ for searching only the best-of-the-web?

  4. @D’log isn’t it the case if something ” _seem_[s] as if they’re getting covered up and ignored?” then they are ? I don’t have web statistics (an inexact science anyway) for any “hinterweb” sites, but I would be surprised if the increase in thier traffic matched the increase in general web usage.

    The problems arise as you don’t know this stuff is out there — take the local history example ‘ A History of BIRMINGHAM Places & Placenames . . . from A to Y’. I’ve been looking at and searching for information on Birmingham for ten years at least, but the first time I saw this site was when the author emailed me to tell me he was using one of my CC licensed Flickr photos. No matter how good I am at searching, if I’m not looking exactly for it I won’t find it.

    Those of us who use the web a lot, and may possibly be attracted to use different search methods have a bit of a responsibility here to make sure this content isn’t lost. I’m not sure linking is enough, although it’s a start.

    Dave’s example is a good one, but it also illustrates the problem — the link that worked now already needs updating. And if it wasn’t for him going back to check we wouldn’t know.

    This is an issue that’s been bothering me for a while. Gary O’Brien runs a website for Goosemore Lane in Erdington, it’s old fashioned looking but info dense. But the only place I can find it is searching the archive. Similarly I found (King’s Heath) not by a link, but from seeing a sign in a shop window.

    I’m not necessarily looking for a technical solution (tho’ one is always welcome), but if we could somehow translate the effort shown by people like Gary into something social we could be doing some good.

  5. I agree that linking helps, it’s something we are doing over at . The tricky part is that many of these sites have been lovingly made, with enthusiast lavishing loads of time on platforms that don’t communicate well. I myself am wondering how to pluck up the courage amongst Birmingham’s heritage types to encourage them to shift over to the social web.

    Perhaps I need a meet first?

Leave a Reply

Your email address will not be published.