Deleting orders is a dangerous thing to do, in the accountancy world. In fact, the functionality of deleting orders is not included in Magento. If you want to delete orders nonetheless, there is a third party module Delete Orders created by Boutique Circus.
The free module can be installed by using the modules extension key within your personal MagentoConnect manager. From the module page on the official Magento website, you can obtain the right extension key.
There's one caveat: The Boutiq Circus Delete Orders module only allows to delete orders when they have the status Pending. The reason for this is that as soon as an order is completed, a whole lot of entities are stored in the database: Invoices, shipping information, etcetera. Deleting an order becomes a lot more complicated and a lot more dangerous. There for this functionality is not included in the free Boutiq Circus Delete Orders module.
However, Yireo has developed a commercial module which does the same job, but for all other order-statuses as well: Delete-Any-Order.