{"id":3186,"date":"2020-10-08T09:00:07","date_gmt":"2020-10-08T14:00:07","guid":{"rendered":"https:\/\/www.kasdorf.name\/wordpress\/?p=3186"},"modified":"2022-09-01T12:49:43","modified_gmt":"2022-09-01T17:49:43","slug":"playing-around","status":"publish","type":"post","link":"https:\/\/www.kasdorf.name\/wordpress\/2020\/10\/08\/playing-around\/","title":{"rendered":"Playing around with Spiders"},"content":{"rendered":"

Quite some time back I turned off all access to web crawlers. Why? I’m not really sure anymore. I had removed the genealogy sections long ago because I was tired of people griping about it. While everything that was on it was in the public domain, a few did not like to see it aggregated.\u00a0 One especially threatened me, even though the information came from their local newspaper detailing their wedding, and their upcoming trip to Jamaica…not that is plain stupid. Thieves now know a) there is a house full of shiny new wedding presents, and b) exactly when the owners will be gone and for how long!
I digress.
If you had a prior link to the web page it still worked, but most search engines would never find it. Funny, most web people spend lots of time and energy working on “SEO” to get them higher in the search results! Me, I wanted it hidden. As an experiment I think I am going to restore it, and see how long it takes to get back on, and when in the standing.
So… we’ll see.<\/p>\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n\n\n\n\n

After editing the “robots.txt” file, I went to Google and found how to request indexing. I filled out the forms and got the following responses.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

I’ll give it a couple of days and see what happens.<\/p>\n","protected":false},"excerpt":{"rendered":"

Quite some time back I turned off all access to web crawlers. Why? I’m not really sure anymore. I had removed the genealogy sections long ago because I was tired of people griping about it. While everything that was on it was in the public domain, a few did not like to see it aggregated.\u00a0 […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[199,231,232],"series":[],"class_list":["post-3186","post","type-post","status-publish","format-standard","hentry","category-general-news","tag-genealogy-sites","tag-seo","tag-web-crawler"],"_links":{"self":[{"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/posts\/3186","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/comments?post=3186"}],"version-history":[{"count":0,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/posts\/3186\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/media?parent=3186"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/categories?post=3186"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/tags?post=3186"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/www.kasdorf.name\/wordpress\/wp-json\/wp\/v2\/series?post=3186"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}