parentid
titleдrobotsļ_robotsļд_robot.txt
webtitleдrobotsļ_robotsļд_robot.txt
webkeywordsдrobotsļ_robotsļд_robot.txt
webdescriptionrobots.txtһıļзվʱҪ鿴ĵһļrobots.txtļ֩ڷʲôļǿԱ鿴ġÿվýһrobots.txtļSEOѺ QQ313801120
sortrank2
authorsharembweb
adddatetime2015-08-28 06:02:08
filename/SEO/write-robots.html
customaurl
flags|c|
relatedtagsSEO
bodycontent[&ȫ&]div class="article_lable"><strong>1robots.txtļʲô</strong>/div>
robots.txtһıļзվʱҪ鿴ĵһļ
robots.txtļ֩ڷʲôļǿԱ鿴ġ
ÿվýһrobots.txtļSEOѺá
ÿ֩ѰҲڵrobots.txtļʱ־м¼һ404Ӧվһrobots.txtʹrobots.txtļֻһļ


div class="article_lable">2robots.txtд﷨/div>
1User-agent: ֵֵ֩֡Ϊ*Эκλ˾Ч
2Disallow: ֵϣʵһURLһĿ¼վDisallow ͷURL ᱻ֩ʵ
κһDisallow ¼Ϊգ˵վвֶʡ
3robots.txtļﻹֱӼsitemapļӡSitemap: http://sharembweb.com/sitemap.xml

User-agent: * *е࣬*һͨ
Disallow: /admin/ ﶨǽֹѰadminĿ¼Ŀ¼
Disallow: /sharembweb/ ﶨǽֹѰsharembwebĿ¼Ŀ¼
Disallow: /ABC/ ﶨǽֹѰABCĿ¼Ŀ¼
Disallow: /cgi-bin/*.html ֹ/cgi-bin/Ŀ¼µ".html"Ϊ׺URL(Ŀ¼)
Disallow: /*?* ֹվеĶ̬ҳ
Disallow: /jpg$ ֹץȡҳе.jpgʽͼƬ
Disallow:/ab/adc.html ֹȥabļadc.htmlļ
Allow: /cgi-bin/ ﶨѰcgi-binĿ¼Ŀ¼
Allow: /tmp ﶨѰtmpĿ¼
Allow: .htm$ ".htm"Ϊ׺URL
Allow: .gif$ ץȡҳgifʽͼƬ

div class="article_lable">3robots.txt÷/div>
վĿ¼ļܱ֩
User-agent: *
Disallow:

ֹ֩վκβ
User-agent: *
Disallow: /

ֹе֩վļĿ¼
User-agent: *
Disallow: /inc
Disallow: /admin

ֹ֩Ŀ¼aĿ¼b
Disallow: /a/
Disallow: /b/

ֻĳ֩
User-agent: Googlebot
Disallow:

div class="article_lable">3robots.txtļķԼ/div>
robots.txt ļʹҲһķգָվĿ¼ṹ˽ڵλá
취÷Ȩޣ˽ʵʩ뱣߱޴ӽ롣
ΪȫǣҪļвҪдrobots.txtļС

div class="article_lable">4֩Robots֡/div>
Baiduspider http://www.baidu.com
Scooter http://www.altavista.com
ia_archiver http://www.alexa.com
Googlebot http://www.google.com
Inktomi Slurp http://www.yahoo.com
FAST-WebCrawler http://www.alltheweb.com
Slurp http://www.inktomi.com
MSNBOT http://search.msn.com

div class="article_lable">5robots.txtдʱע/div>
1robots.txtվĸĿ¼£ļȫСд
2Ҫrobots.txtļеļԱ֩ץȡ
3ΪȫǣҪļвҪдrobots.txtļ


div class="article_lable">6robots.txtʹ/div>
bһ/b
վеļҪ֩ץȡǾûҪrobots.txtļˡļڣе֩뽫Ĭܹվûбҳ档

b/b
robots.txtļеļԱ֩ץȡվ¼ʡ
վеĽűʽļʹ֩¼Ҳվ¼ʣֻռ÷洢ռ䡣
˱robots.txtļòҪ֩űʽļ

b/b
֩ץȡҳ̫˷ѷԴrobots.txtļе֩붼ץȡȫҳ
Ļᵼվܱ¼

div class="article_lable">7ٶȸƷʹòͬuser-agent/div>
 Baiduspider-mobile
ͼƬ Baiduspider-image
Ƶ Baiduspider-video
 Baiduspider-news
ٶѲ Baiduspider-favo
ٶ Baiduspider-cpro
 Baiduspider-ads
ҳԼ Baiduspider


[&ȫend&] 
{$TitleInSearchEngineList divclass='article_lable'  title='дrobotsļ'$}

<br>
/bodycontent


 

