Zesty Lime and Ginger Winter Fruit Salad

1/3 cup passion fruit juice
2 tbsp. packed light brown sugar
2 tsp. lime zest
1 tsp. peeled and finely grated fresh ginger 
2 lbs. navel, Cara Cara or blood oranges
2 lbs kiwis (about 10) peeled and cut into large dice
3 lbs. pineapple (1 med pineapple), cu it to large dice