There are two ways:
1. When you delete the folder, please check the "Destroy permanently" option in the Delete dialog box.
2. Right-click its parent folder, and click Properties. In the resulting dialog box, go to Deleted Items tab, select the folder you want to purge and click Purge button.