Windows Vista has popularised a notion that feeds are stored centrally and called upon by a range of programs. Unfortunately, Microsoft decided to force upon its users a couple of default feeds that mean everytime you setup or configure a feed client (be that Outlook, Live or otherwise) those feeds are automatically imported.
Not so great.
To delete them, the easiest method is to open Internet Explorer, press ALT+C and then press the Feeds button. From there, you can delete them as needed.