Heat 2 tablespoons olive oil in a pot set over medium-high heat. Add the onion and ginger. Cook until the onion is translucent and ginger is softened and fragrant.
Add the carrots, sweet potato, and caraway seeds. Cook until carrots and sweet potatoes begin to soften.
Pour in the vegetable broth, stirring and scraping any drippings up from the bottom of the pot. Cover and cook until carrots and sweet potato are fork tender. About 15 minutes.
Remove the pot from the heat and transfer contents to a blender. Add the orange juice. Work in batches if necessary. Blend until completely smooth.
Return the soup to a pot set over medium heat. Drizzle in heavy cream and stir until fully incorporated. Season with salt and pepper.
Combine remaining olive oil with cayenne. Stir to combine. Drizzle each bowl of soup with cayenne oil.