You need a robots.txt file only if your site includes content that you don't want search engines to index. If you want search engines to index everything in your site, you don't need a robots.txt file (not even an empty one).

In order to use a robots.txt file, you'll need to have access to the root of your domain (if you're not sure, check with your web hoster). If you don't have access to the root of a domain, you can restrict access using the robots meta tag.

Important links : robotstxt.org

__________________________________________________

Know more about NOINDEX NOFOLLOW META tags

Know more about robots.txt file

__________________________________________________

Know what Google says on ROBOTS.TXT file :

Visit the following URL or click here

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449&from=35237&rd=1

How to Prevent Your Site or Certain Subdirectories From Being Crawled

Just click here to know what Yahoo says on the above topic

Know more about Yahoo web crawlers or Yahoo slurp or Yahoo bots