Decorations increase your Empire Happiness.
They also are just fun to place around your island.
Unlike other items in the Build menu, Decorations do not have to be deleted to remove them from your island. They may be placed in Storage instead. The number you have in storage will then show in the Build menu as a tiny crate in the upper right hand side, with a number for how many you have stored.
Placing Decorations in Storage removes their Happiness from your Empire Happiness total and may lower your Tax rate. But it's a nice way to store seasonal Decorations like Christmas trees or Easter Eggs out of sight when their holiday is past.
The goose is considered the best decoration in money for happiness returned.