robots.txt鏂囦欢鏄竴绉嶆帶鍒剁埇铏▼搴忥紙涔熺О涓虹綉缁滆湗铔涖€佺綉缁滄満鍣ㄤ汉鎴栫綉缁滆湝铚傦級鍦ㄧ綉绔欎笂鐖彇鍜岀储寮曞唴瀹圭殑鏂瑰紡銆傚畠鏄竴涓枃鏈枃浠讹紝閫氬父浣嶄簬缃戠珯鐨勬牴鐩綍涓嬶紝鐢ㄤ簬鍚戞悳绱㈠紩鎿庣埇铏寚绀哄摢浜涢〉闈㈠彲浠ヨ鐖彇锛屽摢浜涢〉闈笉搴旇琚埇鍙栥€?/p>

鍏蜂綋鏉ヨ锛宺obots.txt鏂囦欢鐨勪綔鐢ㄦ湁浠ヤ笅鍑犵偣锛?/p>

1. 鎸囧畾鐖櫕璁块棶鏉冮檺锛氱綉绔欑鐞嗗憳鍙互閫氳繃robots.txt鏂囦欢鍛婅瘔鎼滅储寮曟搸鐖櫕鍝簺閮ㄥ垎鍙互琚姄鍙栵紝鍝簺閮ㄥ垎涓嶅簲璇ヨ鎶撳彇銆傝繖鏍峰彲浠ユ帶鍒舵悳绱㈠紩鎿庣储寮曠綉绔欑殑鍐呭锛岄伩鍏嶆姄鍙栨晱鎰熶俊鎭垨涓嶅笇鏈涜鎼滅储寮曟搸绱㈠紩鐨勯〉闈€?/p>

2. 鎻愬崌缃戠珯鐨勭埇鍙栨晥鐜囷細閫氳繃robots.txt鏂囦欢锛岀綉绔欑鐞嗗憳鍙互鎸囧畾鐖櫕璁块棶棰戠巼闄愬埗锛屼緥濡傝缃姄鍙栭棿闅旀椂闂达紝閬垮厤鍥犱负棰戠箒鎶撳彇鑰屽缃戠珯鎬ц兘浜х敓璐熼潰褰卞搷銆?/p>

3. 闅愯棌鏁忔劅淇℃伅锛氱綉绔欑鐞嗗憳鍙互閫氳繃robots.txt鏂囦欢灞忚斀鏌愪簺鏁忔劅鐩綍鎴栨枃浠讹紝浠ヤ繚鎶ょ綉绔欑殑瀹夊叏鎬у拰闅愮銆?/p>

闇€瑕佹敞鎰忕殑鏄紝robots.txt鏂囦欢鍙槸涓€绉嶅缓璁紝鑰屼笉鏄己鍒舵墽琛岃鍒欍€傚ぇ澶氭暟鎼滅储寮曟搸鐖櫕浼氶伒瀹坮obots.txt鏂囦欢鐨勬寚绀猴紝浣嗕篃鏈変竴浜涚埇铏彲鑳戒細蹇界暐瀹冦€傛澶栵紝robots.txt鏂囦欢骞朵笉鏄敤鏉ヤ繚鎶ょ綉绔欏唴瀹圭殑瀹夊叏鎺柦锛屽畠鍙槸鍚戠埇铏彁渚涗簡涓€浜涙寚瀵硷紝鑰岀湡姝ｇ殑璁块棶鏉冮檺鎺у埗搴旇閫氳繃鍏朵粬鏂瑰紡鏉ュ疄鐜般€?/p>