Add apples, oranges, and sugar to a large pitcher and muddle with a muddler or wooden spoon for 45 seconds. Add orange juice and brandy and muddle again to combine for 30 seconds. Add red wine and stir to incorporate, then taste and adjust flavor as needed. Stir to combine.
Add ice and stir once more to chill. Serve as is, or with a little more ice. Garnish with orange slices (optional).