After a bit of research, I could identify it - Polygala paucifolia or the common name, Gay-wings.
It is one of the more curious-looking local wildflowers. It emerges from the forest floor in the month of May. This low growing plant of dry, rich woods, has unusual orchid-like flowers, and is known by several common names including Flowering Wintergreen and Fringed Polygala. The blossoms are pink to rose-purple in color, about 1-1.7 cm long, and usually found growing singly in the axils of the upper leaves. When in full bloom, two prominent sepals flare out from the corolla, framing a whimsically fringed petal at the center.
Gay-wings is not an orchid but is actually one of over 60 members of the genus Polygala that occur in the US. Polygalas make up a large part of the Milkwort family (Polygalaceae), the name stemming from the Greek for ‘much’ (poly) and ‘milk’ (gala). This is because eating milkworts was believed to increase lactation in mammals such as cows and even humans.
Gay-wings is an endangered species in Indiana, Kentucky, and Ohio, but it is secure throughout the rest of its range from central and eastern Canada down the east coast to Georgia and is thriving here on Mackinac.