Preheat the oven to 350°F. Line a baking sheet with parchment paper.

In a large bowl, beat together the butter and granulated sugar until smooth. Mix in the pumpkin purée and vanilla until well combined.

In a separate bowl, whisk together the flour, baking powder, baking soda, salt, cinnamon, nutmeg, ginger, allspice, and cloves.

Gradually add the dry ingredients to the wet ingredients, stirring with a spatula just until combined. Do not overmix.

Drop rounded tablespoons of dough onto the prepared baking sheet, spacing them about 2 inches apart.

Bake for 12 minutes, or until the edges are lightly golden. The centers will still look slightly soft but will set as the cookies cool.

Let the cookies cool on the baking sheet for 5 minutes, then transfer them to a wire rack to cool completely.

For the glaze, whisk the powdered sugar with the milk until smooth and pourable, adding more or less milk to reach your desired consistency.

Once the cookies are completely cool, drizzle the glaze over the tops with a spoon or piping bag, or dip the tops directly into the glaze.