Preheat oven to 350°F. Grease a 9x13 pan and set aside.
Place 1 cup milk in a small saucepan over medium heat. Heat milk until it just begins to simmer, then remove from the heat and immediately add 4 tea bags. Set aside for 10 minutes. Meanwhile, continue with the cake.
Using a hand mixer or stand mixer, beat butter and sugar for several minutes, until smooth and creamy. Add eggs, one at a time, then vanilla extract. Add sour cream and beat until combined.
In a separate bowl, combine flour, baking powder, salt and chai spice.
Once the milk and tea have steeped for 10 minutes, remove the tea bags, gently squeezing them into the milk to extract additional flavor. Discard the tea bags.
To the butter mixture, alternate adding the dry ingredients and the chai milk, mixing well after each addition. I beat in ⅓ of the dry ingredients, ½ the milk, ⅓ dry ingredients, remaining milk, then the remaining dry ingredients. Once the batter is fully combined pour it into the prepared pan. Crumble the topping over the top of the batter, pressing some of it between your fingers to form clumps. Bake for 45-55 minutes, until a toothpick inserted into the center comes out clean.
In a small bowl, stir together all the icing ingredients. Add more milk or powdered sugar, if necessary, to get your desired consistency. Allow the cake to cool slightly then drizzle the icing over the cake. Serve warm.