Preheat oven to 350℉. Spray a 9×5” loaf pan with cooking spray and line with parchment.
In a large bowl whisk together flour, Stevia In The Raw®, baking powder, ginger, baking soda, and salt. In a second medium bowl whisk together buttermilk, lemon juice, butter, vanilla, lemon zest, and fresh ginger. Add wet ingredients to flour mixture and stir just until moistened. Stir in candied ginger.
Scrape batter into loaf pan, smooth the top, and bake until a toothpick inserted in the center comes out clean, about 45 minutes. Let cool in the pan for 10 minutes, then use parchment to lift cake out of pan and let cool completely on a cooling rack.
In a small bowl whisk together powdered sugar, lemon juice, and vanilla until smooth. Drizzle over cooled cake.