In a small bowl, combine peach slices and 1/4 cup sugar; cover and refrigerate. Line a greased 9-in. square baking pan with waxed paper; set aside. For topping, combine flour and brown sugar in a small bowl; cut in butter until mixture is crumbly. Stir in ginger if desired; set aside.
For shortcake, combine the flour, sugar, baking powder and salt in a large bowl; cut in butter until crumbly. In a small bowl, whisk together the egg, milk and vanilla; stir into crumb mixture just until moistened. Spread in prepared pan. Sprinkle with reserved topping.
Bake at 400° for 20-25 minutes or until a toothpick inserted near the center comes out clean. Cool for 10 minutes before removing from pan. Gently peel off waxed paper. Cool crumb side up on a wire rack. Serve warm or at room temperature.
Just before serving, beat cream in a small mixing bowl until it begins to thicken. Add confectioners’ sugar; beat until soft peaks form. Cut cake into nine squares. Split each square horizontally. Place bottom half on serving plate; top with 1/4 cup peach slices. Top with top of shortcake, peaches and whipped cream.