You should create robot.txt file following search engines guidelines. Here is GOOGLE DOCUMENTS on robot.txt file. It is advisable that you should go through it.
A Sitemap makes it easy for search engines to index the pages of your website . Make sure you have submitted the sitemap of your website to search engines.
